Skip to main content

Structured Output

Force the LLM to produce output conforming to a defined schema — JSON, XML, or typed objects. The runtime validates the output against the schema and rejects or retries malformed responses. This turns free-text generation into a typed contract between the agent and downstream systems.

Structure​

The schema is provided to the model (in the prompt or via API-level enforcement). The output is validated before being passed downstream. Invalid output triggers a retry with the validation error as feedback.

How It Works​

  1. Define schema — specify the expected output format (JSON Schema, Pydantic model, TypeScript type)
  2. Constrain generation — provide the schema in the prompt or use API-level enforcement
  3. Generate — model produces output matching the schema
  4. Validate — runtime checks the output against the schema
  5. Retry if needed — validation failure triggers regeneration with error feedback

Enforcement levels:

  • Prompt-level — "respond in this JSON format" (weakest, model may deviate)
  • API-level — response_format parameter forces schema compliance (OpenAI, Anthropic)
  • Library-level — Instructor, DSPy, LangChain output parsers with automatic retries

Key Characteristics​

  • Reliable parsing — output can be deserialized directly into typed objects
  • Clear contract — schema is the interface between agent and consumer
  • Validation catches errors — malformed output is caught before downstream use
  • Less flexible — constrains the model's expressiveness to the schema
  • Schema design matters — overly complex schemas increase prompt size and reduce quality

When to Use​

  • Output feeds into other systems (APIs, databases, UIs)
  • You need to extract specific fields reliably (entities, classifications, scores)
  • Downstream code expects typed data, not free text
  • Consistency matters across many invocations
  • Building data pipelines where agents produce machine-readable output