Function Calling vs Tool Use in LLMs Explained

If you've read API docs from OpenAI, Anthropic, and Google in the same week, you've probably wondered whether function calling and tool use are different features that require different mental models. They aren't. They are the same idea wearing two name tags, and a third if you count Google's "function declarations." Once you understand the underlying loop, every provider's documentation collapses into the same five steps with slightly different field names.

OpenAI introduced "function calling" in June 2023 as an extension to its Chat Completions API. Anthropic shipped its equivalent in the Claude 3 era and named it tool use. Google's Gemini API calls it function declarations. Under the hood, all three do the exact same thing: let a language model ask your program to run a function, wait for the result, and continue the conversation with that result sitting in context. This article walks through the mechanism in detail, compares the three major implementations field by field, works through real code, and covers the failure modes that actually show up in production — plus how the whole pattern is being absorbed into the Model Context Protocol (MCP), the wire-format standard that's turned "define a tool per app" into "connect to a shared catalog of tools."

Why LLMs Need Tools At All

A large language model is, at its core, a next-token predictor trained on text. It has no built-in ability to check today's stock price, query a production database, send an email, or run a calculation with guaranteed precision. Ask GPT-4o "what's 847293 times 12938?" without tools and it will confidently produce an answer that is often wrong, because multiplying large numbers isn't something transformer architectures do reliably through pattern-matching on tokens alone.

Function calling exists to patch that gap. Instead of asking the model to simulate being a calculator, a search engine, or a database client, you give it access to the real thing. The model's job shrinks to something it's actually good at: understanding what the user wants and deciding which real-world action would satisfy it.

Think of it like a translator working with a specialist. The translator (the LLM) doesn't perform surgery, but they can listen to a patient describe their symptoms and tell the surgeon exactly what operation is needed, in the surgeon's own technical vocabulary. The LLM translates a fuzzy human request into a precise, machine-readable instruction that a deterministic system executes.

The most surprising part for developers new to this pattern: the model never executes anything. It only ever emits text that looks like a function call. Your application code is the only thing with permission to actually touch a database, hit an API, or run a shell command.

What "Function Calling" Actually Does

LLMs are text predictors. They don't natively run code, query databases, or call APIs. Function calling closes that gap with a simple loop:

1. You declare tools. A JSON schema describes each tool: its name, what it does, and what arguments it accepts.
2. The LLM decides. Given a user message, the model decides whether to answer directly or call a tool. If it calls a tool, it emits a structured JSON object with the tool name and arguments — not free text.
3. Your runtime executes. The model itself never runs anything. Your host code parses the JSON, runs the corresponding function, and captures the return value.
4. The result feeds back. You send the tool's output back to the model as a new message, tagged so the model knows it's a tool result rather than something the user typed.
5. The model continues. It either answers the user directly or calls another tool. The loop repeats until the model produces a final text answer.

This loop is the foundation of every modern LLM agent. See AI agents explained for how this scales into multi-step autonomous workflows that chain dozens of tool calls together without a human in the loop at every step. The OpenAI Agents SDK is one framework that packages agents, handoffs, guardrails, and tracing around this pattern.

The JSON Schema Contract

Every provider requires roughly the same shape for a tool definition. Here's the OpenAI-style version, which is close enough to the others to serve as a reference:

{
 "name": "get_weather",
 "description": "Get current weather for a city. Use when the user asks about weather.",
 "parameters": {
 "type": "object",
 "properties": {
 "city": { "type": "string", "description": "City name, e.g. Istanbul" },
 "unit": { "type": "string", "enum": ["celsius", "fahrenheit"] }
 },
 "required": ["city"]
 }
}

Three fields matter most:

• name — must be a valid identifier the model can emit exactly, with no spaces or special characters that would break parsing on the way back.
• description — the only signal telling the model when to use the tool. Treat it like prompt engineering, because that's exactly what it is.
• parameters — JSON Schema. Use enum, required, and description aggressively. Vague schemas cause hallucinated arguments, where the model invents a plausible-sounding value instead of asking for clarification.

A tool definition is, functionally, a contract written in two languages at once: JSON Schema for your parser, and natural language (in the description fields) for the model. Weak descriptions are the single most common cause of a model picking the wrong tool or filling in the wrong argument, more common than any bug in the actual code that executes the call.

Parallel Tool Calls

Modern models — GPT-4o and later, Claude 3.5 and later, Gemini 1.5 and later — can call multiple tools in a single response. If the user asks "what's the weather in Paris and Tokyo?", the model emits two tool calls at once instead of two sequential turns. Your runtime executes them in parallel and returns an array of results. This roughly halves latency for fan-out queries, and the effect compounds further when a request needs five or six independent lookups: a booking assistant checking flight availability, hotel availability, and currency exchange rates simultaneously rather than one after another.

Parallel calling changes how you should design your host loop. A naive implementation that assumes exactly one tool call per turn will silently drop the second and third calls, or throw an error when it tries to index into a list of one. Production code needs to iterate over the full array of tool calls in a turn, execute them concurrently where the tools are independent, and preserve ordering only where the model's own tool_call IDs require matching a specific result to a specific request.

Tool Choice Modes

All three providers expose ways to force or block tool calls:

• auto — the model decides. This is the default in every provider's SDK.
• required / any — the model must call some tool, even if it would otherwise have answered directly.
• none — the model must answer in plain text, ignoring any declared tools.
• specific tool — force a particular tool by name. Useful for deterministic data extraction, where you already know the shape of data you want and you're using function calling purely as a structured-output mechanism rather than for actual tool execution.

The "specific tool" mode deserves a second look because it's widely underused. Developers often reach for regex or a second LLM call to pull structured fields — a name, a date, a price — out of unstructured text, when forcing a single tool call with a tight schema does the same job with a guaranteed JSON shape. It's effectively free structured extraction riding on infrastructure you already have wired up for tool use.

Provider Comparison

Feature	OpenAI	Anthropic	Google Gemini
Marketing name	Function calling / Tools	Tool use	Function calling
API parameter	tools	tools	tools.function_declarations
Schema format	JSON Schema	JSON Schema	OpenAPI-subset JSON Schema
Parallel calls	Yes	Yes	Yes
Forced tool	tool_choice: {type: "function", function: {name}}	tool_choice: {type: "tool", name}	tool_config.function_calling_config.mode: "ANY"
Streaming partials	delta.tool_calls chunks	input_json_delta events	Streamed via candidates
Strict schema	Yes (strict: true)	No (validate via your own layer)	Partial
Result message role	tool	tool_result content block	function

Table: Field-by-field comparison of function calling implementations across the three largest LLM API providers, current as of mid-2026.

The differences are small but trip up cross-provider code constantly. The schema dialects diverge most: OpenAI's strict: true mode bans oneOf and requires additionalProperties: false on every object in the schema, which means a schema written loosely for Anthropic will often fail OpenAI's strict validator outright. Gemini rejects some JSON Schema keywords entirely, including certain combinations of anyOf with format, so a schema that validates cleanly against the general JSON Schema spec can still bounce off Gemini's narrower OpenAPI-subset parser.

The Berkeley Function-Calling Leaderboard (BFCL), maintained by UC Berkeley's Gorilla research group, is the most widely cited independent benchmark for tool-calling accuracy. It evaluates models using an Abstract Syntax Tree method across thousands of function definitions in Python, Java, and JavaScript, scoring both whether the model calls the right function and whether it correctly declines to call any function when none applies. Source: [Gorilla BFCL Leaderboard, UC Berkeley](https://gorilla.cs.berkeley.edu/leaderboard.html).

Python Example: OpenAI

from openai import OpenAI
client = OpenAI()

tools = [{
 "type": "function",
 "function": {
 "name": "get_weather",
 "description": "Get current weather for a city.",
 "parameters": {
 "type": "object",
 "properties": {"city": {"type": "string"}},
 "required": ["city"]
 }
 }
}]

resp = client.chat.completions.create(
 model="gpt-4o",
 messages=[{"role": "user", "content": "Weather in Tokyo?"}],
 tools=tools,
)

call = resp.choices[0].message.tool_calls[0]
# call.function.name == "get_weather"
# call.function.arguments == '{"city":"Tokyo"}'

After you run get_weather("Tokyo") in your own code, you send the result back as a new message with role="tool" and the matching tool_call_id, and the model produces its final natural-language answer on the next turn.

Python Example: Anthropic

import anthropic
client = anthropic.Anthropic()

tools = [{
 "name": "get_weather",
 "description": "Get current weather for a city.",
 "input_schema": {
 "type": "object",
 "properties": {"city": {"type": "string"}},
 "required": ["city"]
 }
}]

resp = client.messages.create(
 model="claude-opus-4-7",
 max_tokens=1024,
 tools=tools,
 messages=[{"role": "user", "content": "Weather in Tokyo?"}],
)

for block in resp.content:
 if block.type == "tool_use":
 print(block.name, block.input)

Note the field name difference: OpenAI uses parameters, Anthropic uses input_schema. Everything else is interchangeable at the conceptual level, which is why cross-provider abstraction libraries like LangChain and the OpenAI-compatible adapters that many gateway services expose can translate one schema into the other with a handful of key renames rather than a full rewrite.

How do I send a tool result back to the model?

You append a new message to the conversation with the tool's output, tagged with the same identifier the model used when it requested the call (tool_call_id for OpenAI, the tool_use_id inside a tool_result content block for Anthropic). The model then reads that result as part of its next turn and either answers the user or issues another tool call.

Streaming Considerations

When you stream a response, tool calls arrive as partial JSON fragments spread across many small chunks rather than one complete object. You can't JSON.parse until the stream ends, because the string is genuinely incomplete mid-stream — a chunk might cut off in the middle of a number or a quoted string. Most SDKs hand you an accumulator that stitches fragments together automatically, but if you're rolling your own client you'll need a streaming JSON parser or, more simply, buffer the full string before parsing.

Don't try to start executing the tool before the arguments are fully assembled. Partial arguments are a leading source of bugs in hand-rolled streaming integrations, because a half-received JSON string will sometimes parse "successfully" into a truncated but syntactically valid object, silently passing the wrong arguments to your function.

Common Pitfalls

• Hallucinated tool names. The model invents a tool you didn't declare, usually because a similarly-named tool exists elsewhere in a long system prompt or the model is pattern-matching against training data. Validate every incoming tool call against your actual registry and return an explicit error tool_result rather than silently ignoring it.
• Malformed JSON. Despite "structured" output, models occasionally emit invalid JSON when temperature is high, especially with long or deeply nested argument schemas. Lower the temperature for tool-heavy workflows or use strict mode where available. See JSON mode vs structured outputs for the deeper mechanics of how providers guarantee schema conformance.
• Tool result truncation. Returning a 50 KB SQL dump as a tool_result blows your context window and pushes earlier conversation turns out of scope. Summarize or paginate before sending results back, and consider returning a reference (a row count and a link to fetch details) rather than the full payload.
• Infinite loops. The model keeps calling the same tool with the same or slightly varied arguments, often because a tool result didn't actually answer the question it thought it was asking. Cap iterations in your host loop; ten is a reasonable default for most consumer-facing agents, though research and coding agents sometimes need higher ceilings.
• Stale schemas. Adding a new optional field to a tool without updating the description means the model never discovers it exists. The model only ever sees the schema and description you send on that specific API call; it has no persistent memory of your tool's capabilities beyond what's in the request.
• Overloaded tool sets. Declaring thirty or forty tools "just in case" degrades accuracy across the board, even for the tools the model would have picked correctly with a smaller set. The model has to weigh every declared tool against the user's message on every turn, and more options mean more chances for confusion between similarly-named or similarly-described tools.
• Silent execution failures. If your function throws an exception, don't let that crash the whole conversation. Catch the error and return it to the model as a tool result describing what went wrong, so the model can adjust its next call — retry with different arguments, pick a different tool, or tell the user the action isn't possible right now.

Real-World Patterns Where This Shows Up

Function calling isn't a demo trick; it's the connective tissue behind most production LLM applications shipped since 2024. A few patterns recur across industries:

Customer support triage. A support bot declares tools like look_up_order, issue_refund, and escalate_to_human. The model reads a customer's message, decides which tool (if any) applies, and the host application enforces business rules — for instance, refusing to let the model itself approve a refund over a dollar threshold without a human tool call in the loop.

Coding assistants. Tools like read_file, write_file, run_tests, and search_codebase turn a chat model into something closer to a junior engineer that can inspect a real repository, make edits, and verify its own work by running the test suite, rather than guessing at file contents from memory. Aider is one open-source, terminal-based example of an AI coding tool.

Data analysis copilots. A tool like run_sql_query or execute_python lets a model answer "what were our top five products by revenue last quarter?" by generating and running actual code against real data, instead of hallucinating plausible-sounding numbers.

Enterprise workflow automation. Tools that wrap internal APIs — creating a Jira ticket, updating a CRM record, scheduling a calendar event — let a model act as the front end for systems that previously required a human to navigate a specific internal tool's UI.

In every one of these, the underlying mechanism is identical to the get_weather toy example above. What changes is the number of tools, the sensitivity of what they do, and how much validation sits between the model's request and the actual execution.

How This Connects to MCP

The Model Context Protocol (MCP), which Anthropic introduced in late 2024, is essentially tool use standardized as a wire protocol rather than something you redefine per API provider. Instead of writing a fresh set of JSON Schema tool definitions for every application you connect an LLM to, an MCP server exposes a tool catalog that any MCP-compatible client can discover and call. Under the hood it's still the exact same JSON-schema-call-execute-return loop described throughout this article; MCP just standardizes the transport and discovery layer sitting on top of it.

Adoption moved fast. By March 2026, the MCP SDK was seeing roughly 97 million monthly downloads and more than 9,400 public MCP servers were registered, up from around 100,000 downloads at launch less than two years earlier, according to industry tracking cited by WorkOS's 2026 MCP overview. OpenAI adopted MCP support across its products in 2025, including the ChatGPT desktop app, and began migrating customers off its older, proprietary Assistants API toward MCP-based tool integration. Google DeepMind confirmed MCP support for Gemini in June 2025. In December 2025, Anthropic donated the protocol's governance to the Agentic AI Foundation, a Linux Foundation-hosted directed fund co-founded with Block and OpenAI, with AWS, Google, Microsoft, Cloudflare, GitHub, and Bloomberg joining as supporting members.

Practically, this means a developer building an agent today doesn't need to know whether they're talking to Claude, GPT, or Gemini to decide how to write a tool. Write one MCP server, and any MCP-compatible client — regardless of which model sits behind it — can use the tools inside it. For the full architecture and a deeper walkthrough of how MCP servers and clients communicate, see what MCP is and how it works.

Is MCP a replacement for function calling?

No. MCP sits on top of function calling rather than replacing it. The model still emits the same structured tool-call JSON it always has; MCP standardizes how that tool's schema was discovered and how the call gets routed to a server, rather than changing the model-level mechanism at all.

Structured Extraction vs. Real Tool Execution

It's worth separating two use cases that both ride on the same API surface but serve different goals. Real tool execution means the model is triggering an actual side effect: sending an email, querying a live database, booking a flight. Structured extraction means you're using the tool-calling machinery purely to force the model's output into a predictable JSON shape, with no real "tool" behind it at all — the schema is the whole point, and there's no function to actually run afterward beyond parsing the JSON.

This distinction matters for how carefully you validate. A structured-extraction call that returns a malformed field just means your parser needs to handle an edge case. A real tool-execution call that receives hallucinated or malformed arguments might mean sending an email to the wrong address or running a destructive database query, so production systems generally need a stricter validation and confirmation layer in front of any tool with real-world side effects — a pattern often called a "human-in-the-loop" gate, where higher-risk tool calls pause for explicit approval before your host code executes them.

FAQ

Q: Are function calling and tool use literally the same feature? A: Yes. Same loop, same JSON schema contract, same execution model. The only differences are field names in the API and small schema-dialect quirks between providers.

Q: Does the LLM actually run the function? A: No. The model only emits a structured request describing which tool it wants and what arguments to pass. Your application code executes the function and sends the result back. This is why function calling is safe even with arbitrary tools — the model has no execution capability of its own.

Q: Can I use the same tool definitions across OpenAI and Anthropic? A: Mostly. You need to rename parameters to input_schema for Anthropic and wrap each tool in {"type": "function", "function": {...}} for OpenAI. The JSON Schema body itself usually ports cleanly, though strict-mode requirements on OpenAI can force small adjustments.

Q: What happens if the model picks the wrong tool? A: Sharpen the tool's description. The description is the only prompt the model sees when deciding whether and when to use a tool. Add concrete trigger phrases, such as "Use when the user asks about current weather conditions," rather than a generic one-line summary.

Q: How many tools can I declare at once? A: Technically dozens, but accuracy on benchmarks like BFCL drops measurably past roughly 20 declared tools. For larger toolsets, use a retrieval step that filters the full catalog down to the top handful of relevant tools before calling the model, rather than declaring everything on every turn.

Q: Is tool use deterministic? A: No. The same input can produce different tool calls across separate runs, especially at higher temperatures. Set temperature: 0 and use tool_choice: required for the most repeatable behavior, but expect some variance regardless, since the underlying model is still a probabilistic system.

Q: Do I need a separate library like LangChain to do function calling? A: No, function calling works directly against each provider's raw API using nothing but standard JSON. Frameworks like LangChain or LlamaIndex add convenience around multi-step orchestration, cross-provider abstraction, and memory, but the core mechanism described in this article works with plain HTTP requests and no framework at all.

Q: What's the difference between function calling and the Model Context Protocol? A: Function calling is the model-level mechanism: the loop of declaring a schema, letting the model emit a structured call, executing it, and returning a result. MCP is a transport and discovery standard built on top of that mechanism, letting one tool server serve many different clients and models without redefining the schema for each one.

Q: Can multiple tools be called in a single turn? A: Yes, on current-generation models from all three major providers. Parallel tool calls let the model request several independent actions at once, and your host loop should execute them concurrently and return an array of matched results rather than assuming one call per turn.

Q: How do I stop a model from calling a tool when it isn't actually needed? A: Set tool_choice: none if you want to guarantee a plain-text answer for that specific turn, or tighten each tool's description so it clearly signals when the tool applies and, just as importantly, when it doesn't. Vague descriptions cause both missed calls and unnecessary ones.

Q: Why does my streamed tool call arrive broken when I try to parse it early? A: Streamed tool-call arguments come across as incremental JSON fragments, not a complete object, until the model finishes generating them. Buffer the full string using your SDK's built-in accumulator, or a custom one if you're building a raw client, and only parse once the stream signals the tool call is complete.

The takeaway

Function calling, tool use, and function declarations are the same architecture described three times by three companies with three different style guides. The model reads a JSON Schema description of what's available, decides whether to answer directly or request a specific action with specific arguments, and your own code is the only thing that ever actually touches a real system. Get the schema descriptions right, cap your tool count, validate every call before executing it, and the rest of the API surface across OpenAI, Anthropic, and Gemini differs by little more than field names. The Model Context Protocol is now standardizing the layer above that loop, but the loop itself, five steps deep, hasn't changed since OpenAI first shipped it in 2023.

---

Read next:

• AI agents explained
• JSON mode vs structured outputs
• What is MCP? Model Context Protocol explained

Sources

• OpenAI Function Calling Guide — platform.openai.com/docs/guides/function-calling
• Anthropic Tool Use Documentation — docs.anthropic.com/en/docs/build-with-claude/tool-use
• Google Gemini Function Calling — ai.google.dev/gemini-api/docs/function-calling
• Model Context Protocol Specification (Anthropic, 2024) — modelcontextprotocol.io
• Gorilla BFCL Leaderboard, UC Berkeley
• The Berkeley Function Calling Leaderboard: From Tool Use to Agentic Evaluation of LLMs, ICML 2025 / OpenReview
• Everything your team needs to know about MCP in 2026 — WorkOS