Public Preview

Tool Augmented Generation documentation

Tool-Augmented Generation

Abstract

Enable models to call external tools/functions with structured arguments and integrate results into reasoning for accuracy, up-to-dateness, and action-taking.

Motivation

  • Overcome knowledge cutoff and hallucinations
  • Execute computations, retrieval, and API actions
  • Enforce structured IO via schemas

Architectures

  • Tool planner (model) + Tool registry (schema) + Executor + Merger
  • Multi-turn tool calling with schema validation
  • Retrieval tools (search/vector DB) + domain APIs + calculators/code

Design Choices

  • Schema convention (OpenAI function calling, JSON Schema, provider-specific)
  • Tool selection strategy (implicit vs. explicit prompting)
  • Safety: tool allowlists, argument validation, output sanitization
  • Caching and idempotency for external calls

Pros/Cons

  • Pros: Higher factuality, capability extension, interoperability
  • Cons: Added latency, complexity, and security surface

Evaluation Metrics

  • Tool success rate and argument validity
  • Final-task accuracy vs. no-tool baseline
  • End-to-end latency, tool time share

Vendor/Tooling

  • OpenAI: Tool calling & structured outputs
  • Anthropic: Tool use APIs
  • Google: Gemini function calling
  • Azure OpenAI: same patterns with Azure governance

Design Checklist

  • Define clear schemas, types, and constraints for each tool
  • Validate/sanitize inputs/outputs; log audit trails
  • Implement retries/timeouts and circuit breakers
  • Measure contribution of each tool via ablations

References

  • Title: Function calling URL: https://platform.openai.com/docs/guides/function-calling Publisher/Vendor: OpenAI Accessed: 2025-08-14 Version_or_release: provider_reported
  • Title: Structured Outputs URL: https://platform.openai.com/docs/guides/structured-outputs Publisher/Vendor: OpenAI Accessed: 2025-08-14 Version_or_release: provider_reported
  • Title: Tool use (Anthropic Docs) URL: https://docs.anthropic.com/en/docs/build-with-claude/tool-use Publisher/Vendor: Anthropic Accessed: 2025-08-14 Version_or_release: provider_reported
  • Title: Function Calling (Gemini API) URL: https://ai.google.dev/gemini-api/docs/function-calling Publisher/Vendor: Google Accessed: 2025-08-14 Version_or_release: provider_reported