Skip to main content

Tool Dispatch

When the model asks to call a tool, the harness has to turn that request into a real action: parse it, validate the arguments, route it to a handler, run it, and feed the result back into the loop. Tool dispatch is that plumbing — the bridge between the model's intent and the system's effects.

The model doesn't run tools; it emits a request to run one. Everything between "the model wants to call search" and "the result is back in context" is harness code. Get it wrong and a typo'd argument becomes an unhandled exception three turns deep.

Structure​

This is the runtime realization of the Tool Router pattern — one entry point that validates and dispatches every call.

The contract this pipeline validates against has an industry standard: the Model Context Protocol (Anthropic, 2024) defines tools as name + description + JSON-Schema parameters, giving dispatch a wire-level shape to parse and validate regardless of which server the tool lives on.

How It Works​

  1. Parse — extract the tool name and arguments from the model's structured output.
  2. Validate — check the call against the tool's schema and the registry: does the tool exist, are the arguments well-typed and complete?
  3. Authorize — confirm the call is permitted in the current context before anything executes.
  4. Route & execute — dispatch to the handler, running inside a sandbox when the tool touches untrusted input or state.
  5. Normalize the result — turn success or failure into a consistent, model-readable observation, then append it to context for the next turn.

Key Characteristics​

  • Validate before you execute — schema validation at the boundary turns a malformed call into a correctable error, not a crash.
  • Errors are observations, not exceptions — a failed or rejected call should come back as text the model can react to and retry, keeping the loop alive.
  • Uniform result shape — every tool returns the same envelope (status, content, error) so the loop treats a calculator and a deploy identically.
  • Parallel calls need coordination — when a model emits several tool calls at once, decide whether they run concurrently, and merge results deterministically.
  • Dispatch is the choke point for control — permissions, sandboxing, rate limits, and tracing all hook in here, because every effect flows through it.

Pitfalls​

  • Trusting model arguments — passing unvalidated args straight to a handler is how you get injection and crashes. Validate everything.
  • Leaking raw stack traces — dumping an exception into context confuses the model; normalize errors into a clean, actionable message.
  • A giant switch with no registry — hand-routing every tool inline doesn't scale and becomes a Tool Junk Drawer. Route through a registry.