Public Preview

Structured Output JSON Schema documentation

Structured Output with JSON Schema

Abstract

Constrain model outputs to machine-validated structures (JSON/JSON Schema) to improve reliability, enforce contracts, and simplify downstream parsing.

Motivation

  • Reduce parsing failures and hallucinated keys
  • Guarantee types/enums/ranges for API and data workflows
  • Enable safe tool orchestration and analytics

Reference Architectures

  • Decoder with grammar/constrained decoding against a JSON Schema
  • Vendor-native structured outputs (response_format / tool schema) with server-side validation
  • Post-validate and repair strategy when constraints are violated

Design Choices

  • Schema expressiveness (JSON Schema draft, regex patterns, enums)
  • Strict vs. lenient: reject vs. auto-repair on invalid JSON
  • Streaming: chunked JSON objects vs. token-buffered emit
  • Schema evolution and versioning strategy

Pros/Cons

  • Pros: Deterministic structure, easier integration, safer automation
  • Cons: Potential creativity loss, schema drift risk, strictness may increase retries

Evaluation Metrics

  • Structured validity rate (% valid JSON, % schema-conformant)
  • Field-level accuracy and coverage
  • Retry rate and latency overhead

Vendor/Tooling

  • OpenAI: response_format json_schema; function calling with explicit schemas
  • Azure OpenAI: Structured Outputs (JSON Schema) for compatible models
  • Anthropic: Tool use with JSON tool schemas; Markdown JSON mode guidance
  • Google: Gemini function calling and schema-based tool definitions

Design Checklist

  • Define minimal, stable schemas with enums/ranges
  • Add server-side validation and repair heuristics
  • Track validity/latency metrics; implement backoff on repeated invalids
  • Version schemas and document deprecation strategy

References

  • Title: Structured Outputs URL: https://platform.openai.com/docs/guides/structured-outputs Publisher/Vendor: OpenAI Accessed: 2025-08-14 Version_or_release: provider_reported
  • Title: Function calling URL: https://platform.openai.com/docs/guides/function-calling Publisher/Vendor: OpenAI Accessed: 2025-08-14 Version_or_release: provider_reported
  • Title: Structured outputs (Azure OpenAI) URL: https://learn.microsoft.com/azure/ai-services/openai/how-to/structured-outputs Publisher/Vendor: Microsoft Accessed: 2025-08-14 Version_or_release: provider_reported
  • Title: Tool use (Anthropic Docs) URL: https://docs.anthropic.com/en/docs/build-with-claude/tool-use Publisher/Vendor: Anthropic Accessed: 2025-08-14 Version_or_release: provider_reported
  • Title: Function Calling (Gemini API) URL: https://ai.google.dev/gemini-api/docs/function-calling Publisher/Vendor: Google Accessed: 2025-08-14 Version_or_release: provider_reported