Skip to main content

Router

An LLM or rule-based classifier examines incoming input and routes it to the appropriate specialized handler — a different prompt, model, agent, or pipeline. Unlike the orchestrator, the router makes a one-shot dispatch decision with no iteration, no synthesis, and no dynamic task creation.

Structure​

The classifier determines intent and dispatches once. Each handler is optimized for its domain — different system prompts, tools, models, or even different pipelines. The router itself does not process the task.

How It Works​

  1. Classify — analyze input to determine intent/category
  2. Dispatch — route to the matching handler
  3. Handle — the selected handler processes the task end-to-end
  4. Fallback — if no handler matches, route to a default handler

Classification can be:

  • LLM-based — model picks from a list of categories
  • Embedding-based — semantic similarity to category descriptions
  • Rule-based — keyword matching, regex, or heuristics

Key Characteristics​

  • Low latency — one classification step, then direct to handler
  • Specialized handlers — each path is optimized for its domain
  • No iteration — single dispatch, not a loop
  • Misrouting is costly — wrong classification sends the task to the wrong handler
  • Scales by adding handlers — new categories just need new handlers

When to Use​

  • Input falls into distinct, well-defined categories
  • Each category benefits from a specialized handler (different prompts, tools, or models)
  • You need fast routing without orchestrator overhead
  • The task doesn't require combining outputs from multiple handlers
  • Customer service triage, multi-tenant routing, intent-based dispatch