Multi Agent

AgentLens coordinates two specialized agents through the TwoAgentOrchestrator. This separation of concerns lets each agent focus on what it does best – retrieval or evaluation – rather than combining both into a single monolithic prompt.

Agent Roles

• Retrieval Agent – finds documents via a ReAct reasoning loop with 1-5 tool calls per round
• Grader – scores each chunk on a 1-5 relevance scale and filters irrelevant ones (CRAG pattern)
• Quality Judge – evaluates filtered chunks in a single LLM call, generates an answer, and issues ACCEPT or RETRY

Feedback Loop

When the Judge issues RETRY, its feedback is forwarded to the Retrieval Agent for the next round. The feedback is specific – for example, "the retrieved documents cover BM25 but not vector search, retrieve more on embedding-based retrieval." This targeted guidance produces better results than blind re-retrieval.

Rejection Set

Chunks rejected by the Grader are tracked via MD5 text hash. On subsequent rounds, previously rejected chunks are excluded before re-grading, preventing the agent from re-surfacing discarded material.

Query Classification

Before entering the multi-agent pipeline, a heuristic classifier identifies greetings and short non-questions. These skip retrieval entirely and go to a direct LLM call, avoiding unnecessary overhead.

Round Budget

The orchestrator allows a maximum of 2 rounds. This caps the total LLM calls at 7 in the worst case (2 retrieval rounds + 2 grader calls + 2 judge calls + fallback), with the common case being 2-3 calls. On the final round, the Judge is forced to ACCEPT via a special instruction appended to its prompt.