Claude prompt caching — the rates, the usage object, and why hit rates drop behind a gateway

Prompt caching is the largest single lever on an agent bill, and the one most likely to be quietly not working. Marked correctly, the repeated part of your request is billed at a tenth of the input rate. Marked incorrectly — or routed through something that rewrites your prefix — it is billed at more than the plain rate, because you are paying to write cache entries nothing ever reads.

The mechanism is well documented elsewhere. What is not documented anywhere is what changes when the request goes through a gateway, which is the situation of most people reading this. Both are below.

The mechanism, briefly

You mark a stable prefix — a system prompt, a tool schema, a document — and the provider stores its processed form. The next request whose prefix matches exactly reads it back instead of reprocessing.

{
  "model": "claude-sonnet-4-6",
  "max_tokens": 1024,
  "system": [
    {
      "type": "text",
      "text": "<your long, stable instructions>",
      "cache_control": { "type": "ephemeral" }
    }
  ],
  "messages": [{ "role": "user", "content": "..." }]
}

Two properties do all the work and cause all the trouble. The match is on a prefix, so anything that changes early invalidates everything after it. And an entry has a lifetime, so a conversation with long gaps between turns rebuilds rather than reads.

What it costs

The third column is the standard ratio: a read costs a tenth of input. The fourth is not standard. Anthropic prices a cache write above the input rate; Kunavo does not pass that surcharge through, so a write bills at the plain input rate here. That matters specifically for workloads that rebuild often — long-running agents, slow conversations, anything whose prefix churns — because those are the workloads where writes, not reads, dominate the cache line of the bill.

Both figures are read from the catalog at render time, the same source the pricing table and the Claude calculator read — there are no hand-typed rates on this page.

Reading the usage object

Do not infer cache behaviour from the bill; every response tells you directly, and the three token counts are the whole diagnosis.

// The four numbers that tell you whether caching is working.
// Read them from every response before you tune anything.
"usage": {
  "input_tokens": 312,                     // billed at the input rate
  "cache_creation_input_tokens": 0,        // billed at the write rate
  "cache_read_input_tokens": 24106,        // billed at the read rate
  "output_tokens": 890
}

// A healthy cached agent turn looks like the above: a small
// input_tokens, a large cache_read, and a zero creation.
// If cache_creation is large on EVERY turn, the cache is being
// rebuilt each time and you are paying more, not less.

Why hit rates drop behind a gateway

A cache lives at the upstream that created it, and the prefix match is exact. Those two facts together produce three failure modes that never occur when you call one provider directly:

Routing moved. If consecutive requests land on different upstreams, the second one finds no cache — the entry the first one wrote is not visible from where the second one ran. This is invisible in your own logs and shows up only as a cache-creation count that should have been a read. It is the single most common cause, and it is why a gateway that reroutes freely between upstreams can be cheaper per token and more expensive per session.

Something was injected ahead of your prefix. A layer that rewrites the system message, normalises tool definitions or reorders fields produces a different prefix, and a different prefix is a miss. Nothing errors; the bill just goes up.

The gap outlived the entry. An agent that thinks for several minutes between turns can rebuild every time. This one is a property of your workload rather than of the gateway, but it interacts badly with the first: a rebuild is also a re-route opportunity.

The diagnostic for all three is the same — watch cache_creation_input_tokens across a session. Flat and large means you are rebuilding. Large once and zero after means it is working. On Kunavo, request-level cost and cached-token counts are in the usage view, and the caching reference has the endpoint-level detail.

What to do about it

Keep the cached prefix genuinely stable — put anything that changes per request after the marked block, never before it. Batch an agent's turns closer together where you can, since idle time is the enemy of a short-lived entry. And measure a session end-to-end rather than a request: caching is a property of a sequence, and a per-request estimate cannot see it. The Claude token cost calculator has a cached-tokens field for exactly this reason — set it to the fraction of your prompt that really repeats and the monthly figure stops being fiction.

FAQ