AgentLens
Try AgentLens
Docs / Features / SSE Streaming

SSE Streaming

AgentLens uses Server-Sent Events (SSE) to stream pipeline execution to the browser in real time. The /query/stream endpoint yields NDJSON events as each pipeline stage completes, and the frontend renders them immediately via an EventSource connection.

Event Format

Each SSE event is a JSON object with a type field and adata payload. Seven event types flow through the stream:

  • status – pipeline stage transitions (classified, grading, judging, retry, fallback)
  • step – each ReAct agent iteration with thought, action, tool result, and timing
  • grader – chunk relevance scores, filtered/removed counts, Grader timing
  • judge – verdict (ACCEPT/RETRY), confidence, answer or feedback, Judge timing
  • fallback – Fallback LLM completion with model, tokens, thinking, and raw response
  • done – the complete QueryResponse with sources, metrics, and totals
  • error – error message if the pipeline fails

Callback Architecture

The TwoAgentOrchestrator accepts 5 async callback functions that bridge the pipeline to the SSE stream:

  • on_step(AgentStep) – fired after each ReAct reasoning iteration
  • on_status(stage, message) – fired on pipeline stage changes
  • on_grader(dict) – fired after Grader completes with scores and filtering results
  • on_judge(dict) – fired after Judge completes with verdict and confidence
  • on_fallback(dict) – fired after Fallback LLM completes (if triggered)

Each callback serializes its payload to JSON and yields it through the SSE response. The callbacks are async to avoid blocking the pipeline while the event is being sent.

Frontend Connection

The pipeline debugger UI connects via the browser's native EventSource API. On each event, the UI:

  1. Parses the NDJSON payload
  2. Routes by event type to the appropriate renderer
  3. Updates the Live tab output with role-colored headers and content
  4. Captures an HTML snapshot into stageSnapshots for the Trace tab
  5. Stops the active spinner and starts the next one (if applicable)

Why SSE Over WebSockets

SSE is a natural fit for pipeline streaming because the data flows in one direction (server to client). The connection is simpler than WebSockets – no handshake upgrade, automatic reconnection built into the browser API, and standard HTTP infrastructure (proxies, load balancers) handles it without special configuration.