Agentic Support Copilot

Problem

Support teams answer the same questions repeatedly, and a plain chatbot bolted onto an LLM hallucinates because it has no grounding in the product's actual knowledge base. A useful support copilot needs reliable retrieval, background ingestion of new docs, tool access, and safety guardrails — not just a single model call.

Approach

An agentic support system split into three cooperating services:

• API (FastAPI): serves chat and tool endpoints; business logic lives in services (chat, RAG, tools) with thin routes, plus safety checks, conversation memory, and an abstracted LLM client for provider swapping.
• Worker (Celery + Redis): background ingestion and long-running jobs, keeping the knowledge base fresh without blocking requests.
• Telegram bot: a lightweight channel that reuses the same API/worker so users get the copilot where they already are.

The RAG layer is split into ingestion and retrieval (chunking, embeddings, search) so the knowledge base can be updated incrementally. State lives in PostgreSQL with Alembic migrations; the stack runs locally via docker-compose and deploys to Azure via Bicep IaC.

Results

• A modular API + worker + bot architecture where adding a channel reuses existing logic.
• An incremental RAG pipeline that ingests new documents in the background.
• Provider-agnostic LLM access and safety/moderation at the API boundary.

Learnings

• Separating ingestion from retrieval makes the knowledge base maintainable as it grows.
• Keeping routes thin and logic in services keeps an agentic system testable as tools multiply.
• Abstracting the LLM client early avoids lock-in and eases model upgrades.