Why cached tokens decide what your coding agent costs

When people compare LLM providers they look at two numbers: input price and output price per million tokens. For chat workloads that is roughly right. For agent workloads it is badly wrong, because agents do not send a prompt once.

An agent re-sends its whole history on every tool call

A coding agent works in a loop: send the conversation, get a tool call back, run the tool, append the result, send the whole conversation again. After 50 tool calls the original system prompt and every earlier message have been sent, and billed, roughly 50 times. On real coding-agent traffic we see 80 to 95% of all input tokens served from the provider's prompt cache. The cache-hit rate is the number that decides what a session costs.

The discounts vary far more than the list prices

Providers advertise their input price loudly and their cache-read price quietly, and the spread is enormous. DeepSeek's official API discounts cache hits by about 97%. Fireworks serves GLM 5.2 with a 10x cache discount ($0.14 per million cached, against $1.40 input), while the official Z.ai rate for the same model is $0.26 cached, only a 5.4x discount. Across popular aggregators, cache reads for comparable models range from about $0.07 to $0.33 per million tokens. That is a 4x spread hiding behind near-identical headline prices.

Worked example

Take a realistic agent session: 1M input tokens at a 90% cache-hit rate, plus 100k output tokens, on GLM 5.2.

• At the official rate ($1.40 input, $0.26 cached, $4.40 output): 100k uncached input costs $0.14, 900k cached input costs $0.234, output costs $0.44. Total about $0.81.
• Through above.dev ($1.54 input, $0.154 cached, $4.84 output, which is the Fireworks rate plus a flat 10%): $0.154 + $0.139 + $0.484. Total about $0.78.

Note what happened: the gateway charges 10% more on every token class, and the session still comes out cheaper than a zero-markup reseller of the official API, because the cache discount underneath is twice as deep. A flat 10% on the right upstream beats 0% on the wrong one, and the gap widens the more agentic your traffic is.

How to check this yourself

Do not take any provider's "typical savings" claim at face value, including ours. Run one real session and read the numbers. Every above.dev response carries x-cost-usd (the exact cost of that request) and x-input-cache-hit-rate (cache hits as a fraction of input). Multiply your own cache-hit rate against any provider's cached price and the comparison takes about a minute.

Current cache-hit prices on above.dev start at $0.0077 per million tokens (DeepSeek V4 Flash, off-peak). The live catalog with all three token classes per model is on the homepage and in the docs.