Prompt caching: how OpenAI, Anthropic and Google differ

Monitor in production
HoneyHive Engineering

Prompt caching reuses a repeated prompt prefix at a lower price. OpenAI and Google cache automatically; Anthropic needs a marker, and each has its own limits.

Prompt caching lets a model provider reuse the processed beginning of a prompt, so a repeated prefix costs less and can start faster. All three big providers support it, but they differ in how you turn it on, how long an entry lives and what a write costs. The rule they share is that a hit needs the whole prefix to match exactly, so put stable content first and anything that changes last.

The mechanism underneath is the KV cache: the provider keeps the attention state it computed for your prefix instead of recomputing it. Caching mostly helps the time to first token and the input bill; OpenAI says it reduces the time spent processing input before the response starts.

The three providers side by side

Figures are the providers’ docs as of November 2026. Prices and per-model minimums change often, so check the linked pages before you budget.

Anthropic OpenAI Google
How to enable Add cache_control, either one top-level field (automatic) or on individual blocks (explicit) On by default for supported models Implicit caching is on by default for Gemini 2.5 and newer
Minimum prefix Depends on the model: 512 to 4,096 tokens 1,024 visible tokens on the newest models, varies on older ones 2,048 or 4,096 tokens, depending on the model
Lifetime 5 minutes, refreshed on each use, or 1 hour with "ttl": "1h" Older models: 5 to 10 minutes of inactivity, or 24h extended; newest models: 30 minutes by default or 12 hours Not stated in the caching guide for implicit caching
Cache read price 0.1 times base input by default, lower on a few newest models 0.1 times on most newest models, up to a 95% discount Savings passed on automatically; the caching guide gives no percentage
Cache write price 1.25 times for 5 minutes, 2 times for 1 hour No extra charge on earlier models; 1.25 or 2.5 times on the newest No write charge stated for implicit caching
Where hits show up cache_read_input_tokens usage.input_tokens_details.cached_tokens usage.total_cached_tokens

Anthropic

Anthropic’s cache covers the request in the order tools, then system, then messages. A change at one level invalidates that level and every later one, so editing a tool definition resets everything.

  • Automatic or explicit. A single top-level cache_control field makes the system place the breakpoint on the last cacheable block and move it forward as the conversation grows. For finer control you mark blocks yourself, with up to 4 explicit breakpoints. Automatic caching is not available on Amazon Bedrock.
  • Lookback. Each breakpoint checks back 20 blocks for an earlier entry, so a long conversation needs a fresh breakpoint to keep hitting.
  • What breaks a hit. Changing tool_choice, or adding or removing images, invalidates only the messages level, while toggling web search or citations changes the system prompt and so invalidates the system and messages levels. Changing thinking settings always invalidates the messages level, and on some models the tools and system levels too.
  • Isolation. Caches are isolated by workspace on the Claude API.
  • Parallel calls. An entry is available only after the first response starts, so send one request, wait, then fan out.
  • Token accounting. Total input is the sum of cache_read_input_tokens, cache_creation_input_tokens and input_tokens, where the last counts only tokens after the final breakpoint.

Because a write costs more than a normal read, a prefix you write and never reuse loses money. The 1-hour option suits traffic that arrives less often than every 5 minutes, at the higher write price.

OpenAI

OpenAI needs no code change: caching is on by default for supported models. The knobs are about routing and lifetime.

  • Routing. Caches live on individual machines. OpenAI says traffic above about 15 requests per minute can overflow to other machines, which lowers the hit rate, and suggests splitting heavy traffic across several prompt_cache_key values.
  • prompt_cache_key. On earlier models it influences routing but does not guarantee a hit. On the newest models it is optional and separates cache accounting per customer or user.
  • Retention. Older models offer in_memory or 24h through prompt_cache_retention. GPT-5.6 and later use prompt_cache_options.ttl, with 30 minutes by default and 12 hours on the Responses API.
  • Explicit markers. On the newest models you can switch to explicit-only mode and place prompt_cache_breakpoint markers yourself.
  • What counts as the prefix. The whole rendered context, including tool definitions and images. Output schema, reasoning effort and verbosity also affect it, so changing them mid-session can miss.
  • Data handling. All prompt caching features are compatible with Zero Data Retention, and caches are not shared across organizations.

Google

Google’s implicit caching is on for Gemini 2.5 and newer models, with savings passed on when a request hits. Google advises putting large common content at the start of the prompt and sending requests with a similar prefix close together in time.

Two details are easy to miss. For Gemini 4 multi-turn and agent loops, the full previous request is cached only if the new request adds at least 2,050 tokens, otherwise the last 2,050 tokens are processed again at the standard rate. And explicit caching, where you create and manage cache objects yourself, is not supported in the Interactions API or for Gemini 4 models; it is available through generateContent on Gemini 3 and 2.5 models.

Rules that work on all three

  1. Order from stable to volatile. System prompt and tool definitions first, then reference material, then history, then the new user turn. Dates, user names and retrieved passages go after the stable part. See system prompt for what usually sits at the top.
  2. Do not edit the prefix between calls. Reordering tools, changing the output schema or swapping images all change the prefix.
  3. Keep the prefix above the minimum. Below it, caching silently does nothing.
  4. Space out parallel starts. Let the first request finish writing before fanning out.
  5. Watch the usage fields. A hit rate you never measure is a guess. Log cached and uncached input tokens on every call and alert when the cached share drops after a prompt change. LLM cost optimization covers where caching sits among the other levers, and context engineering covers how to structure what goes into the window.

When it does not pay

  • Prompts shorter than the provider’s minimum.
  • Prefixes that change on every call, such as a timestamp at the top.
  • Traffic too sparse to reach the cache again before it expires. On Anthropic this also means paying the write premium for nothing.
  • Needs that are really about repeated questions rather than repeated prefixes. That is semantic caching, which skips the model call but can return a wrong match.

FAQ

What is prompt caching?

A provider feature that reuses the already processed beginning of a prompt when a later request starts with the same tokens. Anthropic's docs say caching does not change the output, only the cost and speed of reading the prefix.

How do I know a request hit the cache?

Read the usage fields. Anthropic reports cache_read_input_tokens and cache_creation_input_tokens, OpenAI reports usage.input_tokens_details.cached_tokens, and Google reports usage.total_cached_tokens.

Why is my prompt not being cached?

The usual causes are a prompt below the model's minimum length, a prefix that changes between calls, or parallel requests that arrive before the first one has written the entry. Anthropic's docs note that short prompts are processed without caching and no error is returned.

Is prompt caching the same as semantic caching?

No. Prompt caching still calls the model and discounts the repeated prefix. Semantic caching returns a stored answer without calling the model when a new query means the same as an old one.

Does prompt caching change the model's answers?

No. Anthropic's docs state that caching does not change the output content, and OpenAI's guide describes it as reuse of the computed prefix.

Share