Where an AI Agent's API Bill Actually Goes, and the Three Levers That Cut It
Synergised Consulting Ltd
Last Updated: 10 October 2026
An AI agent's API bill is driven by what rides along with every turn, not by what you asked it. Measure the turn multiplier, which is average model calls per task multiplied by average prompt size, then apply three provider pricing levers: prompt caching bills repeated context at a tenth of its normal rate, batch processing halves the price of patient work, and model routing sends routine calls to cheaper models. Cap the loops before the invoice caps you.
Where the money actually goes
The bulk of an agent's bill is context carried across turns, not the words you typed. On every turn of a multi-step task, the agent re-sends its system prompt, the schemas for every tool it can call, the conversation history so far, and the results of earlier tool calls. None of that is optional: the model needs it to stay coherent. A simple illustration, using round numbers only to show the arithmetic: a task that takes twenty-five model calls with a twenty-thousand-token prompt attached to each call has consumed half a million tokens before the final answer is written, and the per-token price of output typically exceeds that of input. The user's prompt, the part most cost-cutting advice talks about, is usually the smallest component on the invoice. Until you have looked at a per-request token log, the natural assumption is that cost tracks conversation length. It tracks re-billed context, which grows with every step the agent takes.
Measure the turn multiplier first
Before touching any pricing lever, work out one number per agent: the average number of model calls per task multiplied by the average prompt tokens per call. That turn multiplier is the honest size of the bill, and it converts every later decision into simple arithmetic, because halving the multiplier halves the cost at any per-token price. In discovery workshops, owners rarely arrive with this number. The workflows that justify an agent get identified in conversation, walking through a normal week task by task, and the same walk is what reveals which candidate tasks involve many small steps and would therefore carry a high multiplier. The two questions belong together: a task is a good automation candidate partly because it repeats, and repetition is exactly what makes the turn multiplier worth measuring before the first invoice rather than after. A gateway or proxy that logs tokens and cost per call makes this a standing record instead of a one-off estimate, and that log is the baseline every saving is later measured against.
The three pricing levers that cut the rate
Three provider features cut what you pay per token without changing what the agent does: prompt caching, batch processing and model routing. They attack different parts of the bill. Caching cuts the cost of repeated context, batching cuts the price of work that can wait, and routing cuts the price of calls that do not need the strongest model. They also differ in effort: caching and batching are configuration decisions measured in hours, while routing needs someone to decide which tasks deserve which model and to keep that decision current. According to the published documentation behind each lever, none of the three requires changing your prompts or your agent's logic, which is why they are the right first moves before any deeper surgery on the agent itself.
Prompt caching
Prompt caching lets the provider store the unchanging front of your prompt, the system prompt and tool schemas, and bill re-reads of it at a steep discount. According to Anthropic's documentation [1], cache reads are billed at a 0.1x multiplier, meaning 90 percent below the normal input rate, while the initial cache write carries a 1.25x multiplier and cached entries default to a 5-minute lifetime before they need refreshing. For an agent making many calls with the same large prefix, that is the difference between paying full price twenty-five times and paying it once. The practical requirement is that the front of the prompt stays stable: if the system prompt or tool list changes between calls, the cache misses and the discount is lost. A small illustration: an agent whose system prompt and tool schemas total fifteen thousand tokens, running ten calls per task, re-bills that prefix ten times at full price. With caching, nine of those reads bill at a tenth of the rate, and the write premium on the first call is a fraction of the saving.
Batch processing
Batch processing trades speed for a flat discount on every token. According to OpenAI's Batch API documentation [2], asynchronous batches cost 50 percent less than the synchronous API on both input and output tokens, with a completion window of 24 hours. Not every agent workload qualifies: anything a customer or a staff member is waiting on cannot sit in a queue for hours. But plenty of agent work is not time-critical, overnight report generation, end-of-day data tidy-ups, scheduled reconciliation, summarising documents nobody is actively reading. Routing that work through a batch endpoint halves its cost with no change to the prompts and no change to the model. The decision is purely organisational: someone has to sort the agent's tasks into "answer needed now" and "answer needed by tomorrow", and point the second group at the batch endpoint. For a stack where most of the volume falls into the second group, this is the cheapest halving available.
Model routing
Model routing means sending routine calls to a cheaper model and reserving the strong model for work that genuinely needs it. The evidence that this preserves quality comes from LMSYS's RouteLLM research: according to the RouteLLM study [3], routers trained on preference data cut costs by over 85 percent on MT Bench while maintaining 95 percent of GPT-4's performance, and the underlying paper reports cost reductions of more than 2x without sacrificing response quality. The finding that matters for an owner is the shape of the result: a large share of everyday requests are simple enough that a smaller model handles them indistinguishably, but nobody can tell which share without routing data. A managed router, or even a crude rule that sends defined task types to a cheaper model, captures most of the available saving. What routing costs you is a decision to maintain: somebody owns the list of which tasks go where, and reviews it when models or prices change.
Stop the runaway run
Cost control is not only about rates; it is also about containment when something goes wrong. An agent stuck in a retry loop, or given a task that never terminates, can multiply a normal bill many times over before anyone notices, because the meter keeps running as long as the loop does. The containment measures are plain and worth writing down: a cap on iterations per task, a ceiling on spend per run, a timeout on each call, and API keys scoped so that one misbehaving agent cannot spend against another's budget. In our own agent stacks, running on a self-hosted server with scoped, purpose-built API access, these limits are what turn an API bill from an open-ended exposure into a bounded one: the worst a single run can cost is the ceiling, not whatever the loop would have consumed. An owner does not need to understand token pricing to set a ceiling; they only need to decide what one task is allowed to cost and have someone enforce it mechanically.
What an owner can usefully take from it
The order of operations is the takeaway. Measure the turn multiplier per agent, because that number tells you whether the problem is too many calls, too much context or too expensive a model, and each has a different fix. Then take the free money: caching for repeated context, batching for patient work, routing for routine calls, all of which change the rate without changing the agent. Finally, put containment in place, ceilings and iteration caps and scoped keys, so that a bad run is an incident rather than a surprise invoice. This is work that fits inside the first hundred days: it needs no new system, no new vendor and no rebuild, only a spend log, a sorting decision and a ceiling, and it leaves the business with a documented handle on the cost of every agent it runs.
Sources
- [1] Anthropic, Prompt caching documentation: OpenAI, Batch API documentation: LMSYS, RouteLLM: An Open-Source Framework for Cost-Effective LLM Routing:
