Case note
Repeated context and caching: paying for the same tokens
A practical buyer question: why does resending the same context on every call show up so heavily in usage?
Many AI calls carry a large prefix that barely changes: a system prompt, a style guide, a schema, tool definitions, or a document the model is supposed to consult. The new work is often a short user message. If your integration resends the whole prefix every time, you are paying for the same tokens again. If your provider offers caching for a stable prefix and you are not checking cache fields in the usage export, those repeated tokens may be billed at the full input rate.
For a high-volume endpoint or project, see how much of each request is repeated prefix and how much is new, and whether the provider reported a cache hit.
Synthetic example. An internal helper sends the same long policy document with every short question. The questions change. The document does not. Usage shows large input token counts on every call. After the team pins the document as a cacheable prefix and confirms cache reads in the export, later calls show the document portion as cache reads. How that is billed depends on the provider's caching terms, so check the invoice rather than assuming a change. This illustration names no totals and is not a customer result.
What to inspect
- Usage exports that split input tokens from output tokens, and that report cache creation, cache reads, or uncached input if your provider includes those fields.
- Request logs that show the stable prefix versus the new user content.
- Conversation length, and whether prior turns are replayed in full on every follow-up.
- Spend exports for the same project or endpoint.
If the export does not distinguish cached from uncached input, write that down as a gap. You can still measure prefix size versus new content from your own request logs.
Repeated tokens inside a call are a different issue from unused plans or limit-driven routing: Unused AI plans, overlap, and limit-driven routing. After you change caching or prompt assembly, measure it with Before and after: measuring AI quality, cost, and time. See also How it works.
One next action
For your highest-volume endpoint or project, measure how much of each request is repeated prefix versus new content, and check whether your provider's cache is enabled and reported in the usage export. Record both findings next to the rows you used.
If you want help reading spend and usage exports together, Overdrive helps you maximize your AI spend from inside your AI assistant. It is a remote MCP server with OAuth sign-in, and it works today in Claude, ChatGPT, and Grok Bot (setup steps). Start with a labeled sample pack and expand the lineage on each figure. Using Overdrive with your own company data is still in security review. The plan is for you to connect the tools you already use instead of uploading files, and those connectors are not available yet.
Claims and caveats
- This post claims that resending a stable prefix on every call means paying for the same tokens again, and that provider cache fields (when present) are part of the evidence you should inspect.
- It does not claim that caching is available for every provider or every request shape. It reports no token totals, hit rates, or spend reductions, and it does not project what any team would have been billed with caching turned on.
- The short illustration is a synthetic example. It is not customer data and it is not a result.