I'll synthesize the four perspectives into a single wiki article. The Skeptic's warning that caching is not a portable abstraction shapes the framing; the Architect's mechanics-first structure provides the skeleton; the Risk Analyst's failure modes populate the pitfalls section; the Empiricist's citations and break-even math provide the evidence backbone.

Prompt Caching Across Providers: Mechanics, Pricing, and Pitfalls

Prompt caching reuses precomputed prefill state for repeated input prefixes, discounting the input-token cost of stable context across all three major frontier providers. The mechanics, pricing geometry, observability fields, and failure modes differ enough between Anthropic, OpenAI, and Google that "prompt caching" is best understood as three distinct operational features sharing a common name, not as a portable abstraction. This article maps the three provider models onto a common substrate, derives break-even formulas from published pricing, and gives orchestrators a working mental model for treating cache as a measured efficiency layer rather than a guaranteed discount.

Coverage note: verified against provider documentation through May 6, 2026. Pricing tables in this article are dated; formulas and multipliers are stated explicitly so the reasoning survives subsequent pricing churn.

The invariant: stable prefixes buy cheaper prefill

A transformer processing a prompt does two distinct kinds of work. Prefill is the parallel computation over input tokens that produces the per-layer key-value (KV) tensors representing each token's attention state. Decode is the autoregressive generation of output tokens, each conditioned on the full KV state of everything before it. Prefill scales roughly linearly with input length and dominates time-to-first-token (TTFT) for long contexts. Decode scales with output length and dominates total wall time for short contexts with long generations.

Prompt caching is a serving-side optimization that targets prefill, not decode. When the same input prefix has been processed recently, the provider can reuse the cached KV tensors instead of recomputing them, then only run prefill over the non-cached suffix and proceed to decode. This produces two distinct effects:

Cost effect. Cached input tokens are billed at a steep discount relative to fresh input tokens — typically around 0.1× the base input price across providers, though with provider-specific premiums on the cache write that paid for them.
Latency effect. TTFT drops sharply for cache-heavy prompts. OpenAI's published cookbook measurements show roughly 7% TTFT improvement at 1,024 tokens of cached prefix and roughly 67% improvement at 150k+ tokens, though these are vendor-operated benchmarks rather than independent measurements (OpenAI Prompt Caching 201).

Two things that prompt caching does not do are worth naming up front, because they shape every downstream design decision:

Caching does not improve model quality on long context. A cached 100k-token codebase is exactly as susceptible to position-dependent recall failures as an uncached one. The "Lost in the Middle" effect — degradation of recall for facts placed in the middle of long contexts — is a property of the model's attention behavior, not of how its prefill was computed (Liu et al., 2023). Cheaper long context is not automatically more reliable long context.
Caching does not affect output token pricing. Output tokens are recomputed every generation regardless of cache state. For workloads where decode dominates total cost — long-form generation from short prompts — caching is nearly irrelevant.

The invariant that makes caching work is prefix stability: an exact, bit-level match between the cached prefix and the new request's prefix, up to the cache boundary. Every provider implements this invariant slightly differently, but every provider depends on it. The practical orchestration consequence is that the prompt assembly layer becomes a cache-hit-rate-determining surface. Anything that introduces nondeterminism into the prefix — timestamps, randomly-ordered tool schemas, model SDK upgrades that change serialization, varying image detail flags — silently destroys hit rates that the orchestrator believed it was getting.

Provider mechanics

Anthropic: explicit breakpoints, two TTLs, multiplier-priced writes

Anthropic exposes the most explicit cache control surface of the three providers. Caching is opt-in at the block level: callers tag specific content blocks with cache_control: {"type": "ephemeral"}, which marks the block as a cache breakpoint. The provider then attempts to cache everything in the request up to and including that breakpoint, and to read from cache anything that matches a previously-written prefix (Anthropic prompt caching).

Several mechanics matter for orchestrator design:

Two TTLs. The default cache TTL is 5 minutes. A 1-hour TTL is available via cache_control.ttl: "1h". The TTL refreshes on each cache hit. Mixing 5-minute and 1-hour breakpoints in the same request is supported but constrained: 1-hour entries must precede 5-minute entries in the prompt order, because the cache is hierarchical and a longer-TTL block cannot be a prefix of a shorter-TTL block.
Up to four breakpoints. A single request can declare at most four cache_control markers. This forces orchestrators to choose carefully which boundaries are worth caching — typically (system prompt + tools), (large document context), (conversation history checkpoint), and (latest user turn assembly). Each breakpoint creates a distinct cache segment that can be read or written independently.
20-block lookback window. When evaluating whether a request can hit cache, Anthropic checks the most recent 20 blocks before the breakpoint for an exact match. Requests that drift outside this window will miss even if their content is otherwise identical.
Model-specific minimums. Each model defines a minimum cacheable prefix length below which cache_control markers are silently ignored. For Claude Opus and Sonnet models, the minimum is 1,024 tokens; for Haiku it is 2,048. Marking a 500-token system prompt with cache_control produces no caching and no error; the orchestrator must verify via usage metadata that caching actually occurred.
Pricing. Cache reads are billed at 0.1× the base input price. Cache writes carry a premium: 1.25× base input for the 5-minute TTL, 2× for the 1-hour TTL (Anthropic pricing). The first request that establishes a cache pays the write premium; subsequent requests inside the TTL pay the read discount.
Observability. The usage field of every response reports cache_creation_input_tokens (tokens written to cache on this request) and cache_read_input_tokens (tokens served from cache). These are the canonical fields for measuring hit rate; the absence of either across many requests indicates a configuration problem.

The control surface is powerful but unforgiving. A breakpoint placed after a content block that includes a per-request timestamp will write to cache on every request and read from cache never. The 1-hour TTL doubles the write cost; choosing it without measured reuse beyond the 5-minute window can be more expensive than not caching at all.

OpenAI: automatic prefix caching with routing knobs

OpenAI's prompt caching is largely automatic. There is no explicit cache_control equivalent. For prompts at or above 1,024 tokens, the provider transparently caches the longest matching prefix and serves matching prefixes from cache on subsequent requests (OpenAI prompt caching). The mental model is "any sufficiently long prompt is cacheable; whether it actually hits depends on routing and recency."

Key mechanics:

1,024-token activation threshold. Prompts shorter than 1,024 tokens are not cached. Prompts longer than the threshold are cached in increments — typically 128-token blocks — meaning that the cached portion is the largest prefix-aligned multiple of the block size that matches a previously-seen prompt.
Exact prefix matching. Cache hits require exact byte-level match of the prefix, including system messages, tool definitions, and message ordering. Any drift — a re-serialized JSON tool schema with reordered keys, a changed model parameter, a different image detail flag — invalidates the cache from the point of drift onward.
prompt_cache_key. Callers can supply a prompt_cache_key string to influence routing. Requests that share a prefix and a prompt_cache_key are routed to the same cache-holding server when possible. OpenAI's documentation notes that beyond roughly 15 requests per minute for the same key/prefix combination, additional requests may overflow to other servers, reducing hit rate. This is a soft sharding mechanism, not a hard guarantee.
Retention. Cached prefixes typically persist 5–10 minutes of inactivity in the in-memory tier, with extended retention available on some models that can hold prefixes for up to 24 hours under low-traffic conditions. Retention is described as best-effort; cache eviction can happen earlier under load.
No explicit write charge. OpenAI does not bill a separate cache-write premium. The first request that creates a cache pays the standard input price for the full prompt; subsequent hits pay the cached-input rate (~0.1× input) for the cached portion plus standard input for the suffix.
Pricing. Current flagship pricing shows GPT-5.5 at $5.00 per million input tokens and $0.50 per million cached input tokens, a 10× discount on the cached portion (OpenAI pricing). GPT-5.4 and Mini variants follow similar ratios.
Observability. Responses report usage.prompt_tokens_details.cached_tokens, the count of tokens served from cache on this request. There is no separate "creation" counter; the first miss looks identical to a request with no cacheable prefix from the billing-field perspective.

The asymmetry between control surfaces is significant. Anthropic gives the orchestrator the responsibility (and the ability) to decide what to cache; OpenAI takes that decision off the table but offers less visibility into when caching is actually working. The OpenAI model is friendlier for naive workloads — anything over 1,024 tokens reused often will benefit automatically — but harder to operate at scale, because the orchestrator cannot directly express "always cache the system prompt" and must instead infer from cached_tokens whether automatic caching is doing what it expects.

Google Gemini: implicit caching plus explicit cache objects

Google's Gemini API distinguishes two distinct caching modes, and conflating them is a common error.

Implicit caching is automatic and behaves similarly to OpenAI's mechanism: shared prefixes across requests within a short window may be served from cache, with cached-token counts reported in the response's usage_metadata. Google's documentation is explicit that implicit caching offers no guaranteed savings and provides no API-level controls (Gemini context caching). Treat it as opportunistic.

Explicit caching uses the cachedContents API: callers create a named CachedContent object containing the system instructions, tools, and large document context they want to reuse. Subsequent generation requests reference the cache by name and append only the dynamic suffix. Explicit caches have:

A user-controlled TTL. Default is 1 hour; configurable from minutes to many hours. Storage is billed per token-hour for the duration the cache exists, separately from per-request input/output costs.
A guaranteed-savings contract — at sufficient reuse volume, the cached-token rate plus storage cost is lower than uncached input. Below break-even reuse, explicit caching is more expensive than not caching.
Lifecycle management. Caches are explicit objects that can be listed, updated (TTL only), and deleted. Failure to delete unused caches accrues storage costs indefinitely until TTL expires.

Pricing for Gemini 2.5 Pro shows $1.25 per million input tokens (≤200k context), $0.125 per million cached input tokens (the same 10× ratio), and $4.50 per million token-hours of cache storage (Gemini API pricing). Vertex AI offers the same primitives under cachedContentTokenCount and related fields (Vertex context caching).

A specific observability point worth calling out: Google's countTokens API (CountTokens API) sizes inputs before generation. It is a preflight tool for staying within context windows. It is not a cache-hit observability tool. Cache hits are reported in the response's usage_metadata and, for explicit caches, in the cache object's metadata. Articles that conflate "countTokens visibility" with "cache observability" are wrong on this point.

Comparison table

Dimension	Anthropic	OpenAI	Google Gemini
Activation	Explicit per-block `cache_control`	Automatic for prompts ≥1,024 tokens	Implicit (automatic) + explicit (`cachedContents`)
Control surface	Up to 4 breakpoints per request; 20-block lookback	`prompt_cache_key` for routing; no explicit cache markers	Explicit cache object lifecycle for explicit mode; none for implicit
TTL / retention	5 min (default) or 1 hour (`ttl: "1h"`); refreshes on hit	~5–10 min in-memory; up to 24 hours extended on some models	Implicit: short, unspecified. Explicit: caller-set TTL (default 1 hour)
Cache unit	Block (with token minimum: 1,024 / 2,048 by model)	128-token aligned prefix; min 1,024 tokens	Implicit: prefix. Explicit: named `CachedContent` object
Observable fields	`cache_read_input_tokens`, `cache_creation_input_tokens` in `usage`	`prompt_tokens_details.cached_tokens` in `usage`	`cached_content_token_count` in `usage_metadata`; cache object metadata
Pricing shape (writes)	1.25× base input (5m); 2× base input (1h)	No explicit write premium; first miss billed at base input	Implicit: no premium. Explicit: standard input + per-token-hour storage
Pricing shape (reads)	0.1× base input	~0.1× base input (varies by model)	~0.1× base input
Main failure mode	Breakpoint placed after a non-stable block; silent ignore below min length	Prefix drift from SDK serialization or routing overflow above ~15 RPM/key	Implicit: no savings guarantee. Explicit: storage cost dominates if reuse is sparse
Best-fit workload	Large stable system prompts + tools, multi-turn agent loops where the orchestrator controls prompt assembly	Bursts of repeated long prompts where automatic caching is sufficient	Long static documents reused across many requests within a known window

Cost geometry

Provider pricing for prompt caching follows a common pattern with provider-specific deviations. The general form for a single cached prefix of length L reused N times within its TTL is:

total_cost = L × (W × P_in + (N - 1) × R × P_in)
           + Σ (suffix_i × P_in + output_i × P_out)
           + storage_cost

Where P_in is base input price per token, P_out is output price per token, W is the write multiplier, R is the read multiplier, and storage_cost covers any per-token-hour charges. For the prefix portion alone, caching is favorable when:

W + R × (N - 1) < N

Solving for the break-even reuse count N per provider gives concrete thresholds:

Anthropic, 5-minute TTL (W = 1.25, R = 0.1):

1.25 + 0.1 × (N - 1) < N
1.15 < 0.9 × N
N > 1.28

Two reuses inside the 5-minute window break even. A single use is always more expensive cached than uncached.

Anthropic, 1-hour TTL (W = 2, R = 0.1):

2 + 0.1 × (N - 1) < N
1.9 < 0.9 × N
N > 2.11

Three reuses inside the 1-hour window break even. This makes the 1-hour TTL economically narrower than it might seem: it only pays off when reuse is moderate-to-high, and 5-minute caching is strictly better when reuse is dense.

OpenAI (W = 1, R = 0.1, no explicit write premium):

1 + 0.1 × (N - 1) < N
0.9 < 0.9 × N
N > 1

Two reuses are favorable; even a single repeat after the initial miss saves money on the cached portion. This is the cleanest cost geometry of the three providers, but it is also the least controllable — the orchestrator cannot force a write or guarantee retention.

Google Gemini, explicit caching is structurally different because storage is billed separately. For Gemini 2.5 Pro at $1.25 / $0.125 / $4.50 per million (input / cached / token-hour storage), a 1-million-token cache held for one hour costs $4.50 in storage alone. That storage cost has to be amortized across the reuses:

break_even_N satisfies:
  N × P_in > N_uses_at_cached_rate × P_cached + storage_per_hour × hours_held

For a 1M-token cache held one hour at Gemini 2.5 Pro pricing, naive uncached cost per use is $1.25; cached cost per use is $0.125; storage adds $4.50 fixed. Break-even is roughly 5 uses per hour. With a 5-minute TTL — storage held for 1/12 of an hour, so $0.375 fixed — break-even drops to roughly 2 uses. TTL choice is not a side parameter for Google explicit caching; it is the central economic lever.

Two consequences follow from this geometry:

The first call is always more expensive than baseline. Anthropic charges a write premium; OpenAI charges full input on the miss; Google explicit caching charges full input plus initiates storage billing. Caching only pays off across the second-and-subsequent calls within the TTL. Workflows that issue a single long-context call per session — one-shot document analysis, ad-hoc research queries — gain nothing from caching and may pay more.
Output tokens are outside this geometry. Workloads where output dominates total cost — long-form generation, code synthesis, multi-paragraph summaries from short prompts — see proportionally smaller savings from caching, even with high hit rates. The savings calculation should always be done on input-token cost as a fraction of total request cost.

For orchestrators, the correct cost-of-call estimator is provider-specific:

expected_cost = (1 - p_hit) × full_input_cost
              + p_hit × (cached_prefix × R × P_in + dynamic_suffix × P_in)
              + write_premium_amortized
              + storage_cost_amortized
              + output_tokens × P_out
              + retry_overhead

p_hit is the orchestrator's measured token-weighted cache hit rate, not a provider-promised value. Hard cost ceilings should be computed against p_hit = 0 (the worst case) and reported with an actual-vs-uncached delta against measured hit rates.

Cache hit rate as an observable

Cache hit rate is the metric that tells orchestrators whether their prompt assembly is actually preserving the prefix stability that caching requires. It is also the metric most often misdefined.

The token-weighted hit rate is the right primary measure:

hit_rate_tokens = Σ cached_tokens_per_request / Σ eligible_input_tokens_per_request

Where "eligible" excludes prompts below the provider's minimum cacheable threshold. This is the metric that tracks dollars: a 90% token-weighted hit rate on a 100k-token prefix is a fundamentally different financial outcome than a 90% request-weighted hit rate where 90 short requests hit cache and 10 long ones miss.

Request-weighted hit rate is a useful secondary measure — it surfaces tail risk (a small number of cache-missing request patterns can dominate cost even if the majority of requests hit) — but should never be the only number a dashboard reports.

Each provider exposes the inputs slightly differently:

Anthropic: cache_read_input_tokens / (cache_read_input_tokens + cache_creation_input_tokens + uncached_input_tokens).
OpenAI: prompt_tokens_details.cached_tokens / prompt_tokens. There is no separate "write" counter, so the orchestrator cannot distinguish "cache miss + new write" from "cache miss + below threshold."
Google: usage_metadata.cached_content_token_count / usage_metadata.prompt_token_count. For implicit caching this counts opportunistic hits; for explicit caching it counts the named cache contribution.

Several detection signals indicate a cache configuration problem:

Anthropic cache_creation_input_tokens is non-zero on every request. The breakpoint is in or after a non-stable block.
OpenAI cached_tokens drops sharply after an SDK upgrade or a model parameter change. Serialization or routing has shifted; the prefix no longer matches.
Google cached_content_token_count is zero for explicit-cache-using requests. The cache object reference is wrong, or the cache has expired.
TTFT does not improve on repeated requests within the TTL. Caching is silently failing.
Cost per request rises after a deploy. A new field has been added to the prefix, or the prefix has shifted past the cache boundary.

These signals should be wired into the orchestrator's observability stack alongside token counts and latency. Cache hit rate is not a "nice to have" metric for cost-conscious operations; it is the only direct evidence that the prompt assembly logic is doing what it is supposed to do.

How orchestrators should reason about cache as a budget axis

The phrase "cache as a budget axis" deserves careful unpacking, because it points at a real disagreement in how to operationalize caching, and getting the framing wrong leads to expensive mistakes.

The maximalist position is: cache is a first-class budget axis on par with model selection, max output tokens, and task routing — orchestrators should plan capacity, set hard cost ceilings, and design routing rules around expected cache savings. The minimalist position is: cache is a stochastic discount whose magnitude depends on routing, retention, and prefix stability that the orchestrator cannot fully guarantee, and treating it as a budget axis encourages overconfident cost projections that production traffic will violate.

Both positions are partially right. The synthesis is:

Cache is a budget axis only after it has been measured. Treating expected cache savings as a planning input before the orchestrator has production data on its own hit rates is a mistake. Treating measured cache savings — p_hit_observed over a representative window — as a planning input is correct and often important.

This implies a three-stage maturity model for any orchestrator:

Stage 1 — opportunistic. Cache hits are bonuses, not assumptions. Cost ceilings are computed as if caching produces zero discount. Hit rates are logged but not budgeted. Suitable for new workloads, exploratory pipelines, and anything where production traffic patterns are not yet stable.

Stage 2 — instrumented. Cache hit rate is a tracked SLO, defined token-weighted, with alerts on regressions. Prompt assembly logic is canonicalized: deterministic JSON serialization for tool definitions, fixed ordering of system messages and tools, no per-request timestamps before the cache boundary, image detail flags fixed per call site. Pricing models include (p_hit_observed, p_hit_p10, p_hit_p90) rather than a single point estimate.

Stage 3 — design-driven. The prompt schema is an explicit cache-aware artifact. Static blocks (system prompts, tool definitions, large document contexts) are placed at the front and protected by breakpoints (Anthropic) or stable serialization (OpenAI/Google). Dynamic blocks are pushed to the suffix. Multi-turn conversation history is cached at periodic checkpoints rather than rebuilt fresh each turn. Cache-warming requests are issued before high-traffic windows. Routing decisions account for prompt_cache_key collision risk and the ~15-RPM-per-key OpenAI overflow threshold. Capacity planning includes cache-write premium amortization, especially for Anthropic 1-hour caches and Google explicit caches with non-trivial storage cost.

A useful framing here: cache is not a discount coupon applied at checkout. It is a constraint on how the prompt is laid out, which then has economic consequences. The orchestrator that designs around the constraint gets the savings; the orchestrator that hopes for them does not.

There is a related risk worth naming explicitly. Optimizing for cache can encourage prompt bloat — adding context "since it's cheap to keep around" — and can encourage prompt fossilization, where instructions become hard to update because changes invalidate cache. The first risk degrades model quality (more irrelevant context to attend over, including the Lost-in-the-Middle problem) while improving cost optics. The second risk slows iteration. Both are second-order effects that good observability does not catch automatically. A reasonable hygiene rule: every change to a cached prefix should be accompanied by an evaluation run, not just a hit-rate check.

Pitfalls and false intuitions

The mechanics above admit a number of failure modes that production traffic exposes regularly. These are the ones worth designing against from the start.

Timestamps before the cache boundary. A system prompt that includes "Current date: 2026-05-06" or "Session ID: abc123" at the top of the message will write to cache on every request and read from cache never. The cache_creation_input_tokens field will be non-zero on every Anthropic request; OpenAI's cached_tokens will sit near zero. Move time-varying content to the suffix, after all static content and before the user's actual turn.

Tool schema reordering. SDK upgrades, JSON serializers with non-deterministic key ordering, or migrations that add a new tool can shift the byte-level representation of the tools block. The shift invalidates the cache from the point of drift. Canonicalize tool serialization — sorted keys, fixed property ordering, explicit array ordering — and treat the tool block as a versioned artifact, not as something the orchestrator regenerates per request.

Image detail flag changes. OpenAI and Anthropic both treat image content as part of the prefix. Switching detail: "auto" to detail: "high" or vice versa changes the prefix bytes and invalidates the cache. This often surfaces as a hit-rate cliff after a frontend change that is unrelated to caching.

Per-request data hidden by SDK abstractions. Some SDK helpers inject fields the application does not directly control — request IDs, client timestamps, telemetry headers that some implementations forward into the message body. Verify by capturing the actual on-the-wire request and diffing two requests that should hit the cache. Hash the prefix and log the hash; drift in the hash field is a leading indicator of cache loss.

prompt_cache_key granularity. Setting prompt_cache_key per user maximizes cache locality but partitions the cache aggressively, reducing total hit potential when many users share the same system prompt. Setting it per workflow class — e.g., one key for "code review", one for "doc summarization" — is usually closer to optimal. Too coarse a key, combined with traffic above ~15 RPM for that key/prefix, triggers OpenAI's documented overflow behavior and silently degrades hit rate.

Anthropic minimum-length silent failure. Marking a 500-token system prompt with cache_control produces no error, no warning, and no caching on Opus and Sonnet (1,024-token minimum) or Haiku (2,048-token minimum). The only signal is that cache_creation_input_tokens and cache_read_input_tokens both stay zero. Confirm caching is active by asserting on these fields in tests, not by trusting the absence of errors.

TTL ordering for mixed Anthropic caches. A 5-minute breakpoint that precedes a 1-hour breakpoint will silently fail to enable the 1-hour TTL on the inner block. The 1-hour entries must come first in prompt order. This rule is documented but easy to miss when the prompt is assembled programmatically from blocks whose TTLs are decided by different parts of the application.

Google explicit cache creation cost. Creating a 1M-token explicit cache on Gemini 2.5 Pro costs roughly $1.25 in input tokens at creation time, plus storage at $4.50 per million token-hours for the cache lifetime. A cache that is created and then unused — because the workflow it was built for did not actually fire — represents pure cost. Treat explicit cache creation as a write operation that should be initiated only when the orchestrator has high confidence reuse will follow within the TTL.

Implicit cache treated as guaranteed. Google's documentation is explicit that implicit caching offers no savings guarantee. Code that assumes hits — for example, by committing to an SLA that depends on cached response times — is brittle. If the workload requires deterministic caching behavior on Gemini, use explicit caching even though it costs more for low-reuse workloads.

Cache stickiness reducing model adaptability. A large, stable system prompt is friendly to caching but hostile to per-request adaptation. Workloads that benefit from per-tenant or per-task customization may have their adaptation budget squeezed by cache-driven prompt rigidity. The orchestrator can mitigate this by structuring the prompt as [stable cached prefix] + [dynamic per-request adaptation] and accepting that the adaptation portion misses cache. The boundary placement is a design decision, not a default.

KV cache memory pressure, not just billing. Even when cached content is "free" to the customer, it occupies KV memory on the serving infrastructure. Provider-side eviction under load can manifest as inconsistent hit rates that correlate with traffic spikes elsewhere on the same model. The PagedAttention work behind vLLM is a reminder that KV cache is a finite physical resource on the serving side, not an unbounded abstraction (Kwon et al., 2023). Orchestrators that depend on cache hits during traffic spikes — exactly when prefill is most expensive — should test this case explicitly.

Privacy and retention assumptions. Cache retention behavior varies between providers, between modes (zero-data-retention enrollment, regional processing, workspace isolation), and sometimes between models. A blanket statement like "cached prompts are not retained" is wrong; the correct statement is provider-specific and contract-specific. Security and compliance review of cache retention should happen explicitly, not be assumed from "the cache only lasts 5 minutes."

Workload archetypes: where caching pays and where it doesn't

The break-even math above produces concrete reuse thresholds, but the realistic question for an orchestrator is "which of my workloads have those reuse patterns?" Five archetypes cover most production traffic.

Multi-turn agent loops with stable tools and system prompt. The agent receives a long system prompt, a stable tool definition block, and a growing conversation history. Every turn is a request that shares the prefix with the previous turn. Reuse is dense (turns happen seconds apart) and inside the TTL. High caching value across all three providers. Anthropic with explicit breakpoints at (system + tools) and (history checkpoint) is typically the cleanest implementation; OpenAI's automatic caching captures the same savings if the prefix is byte-stable.

Codebase-aware code review or refactoring loops. A long codebase context is loaded once and reused across many small queries. Reuse count is high; reuse interval depends on user activity. High value if interactive, marginal value for batch jobs that fire once per file. Google explicit caching is a natural fit when the codebase exceeds 100k tokens and reuse will exceed the storage break-even within the TTL.

Document Q&A with high session reuse. A user uploads a 200-page PDF and asks 20 questions over five minutes. Identical document prefix, varying question suffix. High caching value, particularly on Anthropic 5-minute or Google explicit caching. OpenAI's automatic caching also captures this, with hit rate determined by routing to the same server.

One-shot document analysis. A pipeline issues a single long-context request per document and never returns to it. No caching value; possible negative value on Anthropic (write premium with no read amortization) and Google explicit caching (storage cost with no reuse). Stage 1 opportunistic mode is correct; do not configure caching for these workloads.

Bursty multi-tenant workloads with shared system prompt. Many users sharing a system prompt template, low per-user reuse, high aggregate reuse. Value depends on routing. OpenAI's prompt_cache_key set per workflow class (not per user) maximizes shared cache locality. Anthropic caching benefits from pre-warming requests that establish the cache before user traffic. Google explicit caching is workable but requires a single global cache object with careful TTL management.

The general principle: caching value scales with reuse density inside the TTL, modulated by prefix stability and suffix-to-prefix ratio. Workloads with low reuse density, drifty prefixes, or long suffixes relative to the prefix should be treated as Stage 1 opportunistic and not optimized for cache.

A minimal verification protocol

Provider documentation is necessary but not sufficient. Each orchestrator's actual hit rate depends on prompt assembly logic that the documentation cannot see. Before committing to cache-aware budgeting, run the following microbenchmark per provider/model combination:

Build a stable prefix above the model's minimum cacheable length. Record its byte hash.
Issue a warm-up request containing the prefix plus a trivial suffix.
Issue 10 repeat requests within the 5-minute window, each with a different suffix but identical prefix. Log per-request: input tokens, cached tokens, write tokens (Anthropic), TTFT, total latency, billed cost.
Issue a "drift" variant that inserts a timestamp into the prefix at the top, then 10 more requests. Confirm cached counts drop to zero.
Issue a "suffix only" variant that varies content after the cache boundary, then 10 more requests. Confirm cached counts remain near full prefix length.
Wait 10 minutes, repeat the warm-up and 10 repeats. Confirm the 5-minute TTL has expired.
For Anthropic: repeat steps 2–6 with the 1-hour TTL configured. Confirm reuse persists past 10 minutes.
For Google: run separately with implicit (no cachedContent) and explicit (named CachedContent object).
Compute observed token-weighted hit rate and compare against the break-even formula above for the actual reuse pattern in production.

This protocol takes a few hours and a small budget per provider. It is the smallest experiment that resolves the highest-impact uncertainty: whether the orchestrator's prompt assembly preserves prefixes well enough to make cache-aware budgeting reliable. Without this evidence, "cache as a budget axis" is a claim about provider behavior, not about the orchestrator's actual cost structure.

Summary

Prompt caching is a real and material efficiency layer with provider-specific mechanics that resist clean cross-provider abstraction. The shared substrate is prefix-stability-dependent prefill reuse, billed at roughly 10× discount on the cached portion across all three major providers. The differences — explicit breakpoints and write premiums on Anthropic, automatic prefix matching with routing knobs on OpenAI, separated implicit and explicit modes with storage billing on Google — change which workloads benefit, how the orchestrator instruments the system, and what the cost-of-call function looks like.

The single most useful framing is that caching is a prompt-layout constraint with economic consequences, not a discount coupon applied at checkout. Orchestrators that design prompt schemas around the constraint — stable prefix at the front, dynamic suffix at the back, canonical serialization, breakpoints placed at deliberate boundaries, hit rate tracked as a token-weighted SLO — capture the savings. Orchestrators that hope for them get inconsistent results that correlate with deploy events and traffic patterns in ways that look mysterious until the prefix hash is logged.

Cache should be treated as a budget axis only after the orchestrator's hit rate has been measured under representative production traffic. Hard cost ceilings should be set against the worst case (zero hit rate) and reported with the actual delta against measured rates. The break-even math is generous on the second-and-subsequent calls within the TTL but punishing on first calls, on workloads where output dominates total cost, and on Google explicit caches whose storage cost is not amortized across enough reuses. The pitfalls section above is the operational checklist: timestamps, tool schema reordering, SDK serialization drift, image detail flags, minimum-length silent failures, TTL ordering, key granularity, and KV memory pressure are the sharp edges that production traffic exposes.

The intellectually honest position is that prompt caching changes the marginal economics of repeated stable context; it does not make long context free, portable across providers, or operationally simple. Used carefully, it is one of the highest-leverage cost levers available to orchestrators today. Used carelessly, it produces optimistic cost projections that production traffic violates and prompt fossilization that quietly degrades quality.

Companion entries

Core theory:

Context Window Economics
KV Cache Mechanics and PagedAttention
Prefill vs Decode: Where the Tokens Cost
Token-Weighted vs Request-Weighted Metrics

Practice:

Multi-Model Agent Orchestration Patterns
Prompt Template Stability and Canonical Serialization
Tool Schema Versioning for Cacheable Prompts
Agent Loop Cost Modeling
Observability for LLM Systems
Cache Warming and Prewarm Strategies

Counterarguments:

Lost in the Middle: Position-Dependent Recall
Premature Prompt Fossilization
KV Cache Memory Pressure on Serving Infrastructure
The Cheap Tokens Trap: When Caching Encourages Bloat

Adjacent:

Context Engineering for Agent Pipelines
Provider Lock-in via Cache-Aware Prompt Design
Privacy and Retention in LLM Caching