Function calling and tool use unlock powerful agentic behavior, letting a model query a database, call an external API, or run a calculation as part of generating its response. That power comes with cost implications that are easy to underestimate, since a single user request can now trigger several model calls chained together rather than one.
Tool Definitions Are Billed on Every Call
The schema describing each available tool, including its name, description, and parameter definitions, is included as input tokens on every single API call where tools are enabled, whether or not the model actually decides to invoke any tool on that particular request. A system with many available tools pays this overhead repeatedly across high call volume.
Review your tool definitions periodically for verbosity: an overly detailed description or an unnecessarily complex parameter schema adds tokens on every call without necessarily improving the model's ability to use the tool correctly. Keep tool descriptions as concise as possible while remaining unambiguous.
Multi-Step Loops Multiply Request Count
A tool-use interaction typically involves multiple sequential model calls: one call where the model decides to invoke a tool, then a follow-up call including the tool's output so the model can incorporate it into a final response, and potentially several more rounds if the task requires multiple tool invocations in sequence.
Each of these calls carries its own input token cost, including the accumulating conversation history and tool outputs from prior steps, so a single user request handled through a five-step tool-use loop can cost meaningfully more than a naive single-call estimate would suggest, and that multiplier compounds directly with call volume.
Tool Output Size Matters
The data returned by a tool call, such as a database query result, a search API response, or a file's contents, is injected into the next model call as input tokens. A tool that returns a large, unfiltered result set, when only a small subset is actually relevant, inflates cost on every subsequent step of the loop.
Design tools to return concise, relevant results rather than raw dumps: paginate large result sets, summarize where appropriate, or let the tool itself filter to the most relevant subset before returning, rather than pushing that filtering responsibility onto the model at token cost.
Bounding Agentic Loops
Any system that allows the model to chain multiple tool calls autonomously needs an explicit maximum step count and, ideally, a maximum cumulative token budget per task, since a model can in principle loop on tool calls far longer than a human would find useful or a task actually requires, especially if it is stuck on a task it cannot resolve.
Log the distribution of steps-per-task across your production traffic and periodically review whether the maximum step count is being hit frequently, since that is a signal either that the cap is too low for legitimate tasks or that a meaningful share of tasks are getting stuck in unproductive loops worth investigating directly.
Key takeaways
- Keep tool descriptions and parameter schemas concise; they are billed as input tokens on every call.
- Account for the full multi-call chain of a tool-use loop, not a single-call cost estimate.
- Design tools to return concise, filtered results instead of large raw data dumps.
- Set an explicit maximum step count and token budget for any autonomous multi-step tool-use loop.
- Monitor how often the step cap is hit to catch both overly restrictive limits and unproductive loops.
Bottom line
Tool use turns a single request into a chain of model calls, and each link in that chain carries its own token cost that compounds quickly if left unbounded. Concise tool definitions, filtered tool outputs, and explicit loop limits together keep agentic, tool-using systems cost-predictable rather than open-ended.