Caching components¶

Two components touch the provider's prompt cache: cachesplit (in every caching preset) and cacheinject (in none). Both are Reformat components whose real work happens body-level in apply, not in the pipeline's per-message rewrite.

OpenAI's implicit cache¶

OpenAI requests are cache-aware in cache_mode: auto even without cache_control: the provider caches eligible prefixes implicitly. Context-guru protects the previous turn's prefix while rewriting a new tail. It does not add prompt_cache_options.ttl to requests. For GPT-5.6 and later, OpenAI's documented default is 30m after the latest write or reuse; that is a minimum, not proof that the entry expires at 30 minutes. The entry is classified warm throughout that guarantee: its actual expiry cannot be inferred, so pre_expiry is not claimed. Past 30 minutes the cache phase is unknown, not cold. Earlier OpenAI models have different retention rules, so their phase remains unknown unless a reliable lifetime can be established. GPT-5.6+ Responses sessions can opt into a separate 28-minute keep-alive schedule; earlier OpenAI models are not pinged.

See the OpenAI prompt-caching guide for the model-specific retention rules.

Split is measured, placement isn't¶

Claude Code appends a live environment snapshot to the end of its main system block:

Current branch: main
...
Recent commits:
0898367954 SWE-bench

Measured across 50 SWE-bench tasks that block is ~7,017 tokens, of which the first 6,921 (98.4%) are byte-identical across sessions — but a cache entry hashes everything before its breakpoint, and no breakpoint position can exclude part of a single content block. So the churning tail is one cacheable unit with its breakpoint at the end, and the shared 98.4% is re-written every session regardless of where a breakpoint is placed inside it.

The tail is real content: it cannot be moved or dropped without lying to the model about the repo state. It can be split — [stable][volatile] as two text blocks with the same concatenated text, breakpoint on the first. Adjacent text blocks concatenate, so the model sees a byte-identical prompt while the provider gains a hash boundary that excludes the churn. This is what cachesplit enables, and it is measured: −34.1% mean cost on one Terminal-Bench task measured three times, and 0% → 96.7% cache hit in an isolated live A/B.

Breakpoint placement (cacheinject) is a different question — given a cache boundary, where should the (up to 4) breakpoints go? — and it has never been shown to help: the one live measurement is n=1 and mildly negative, with no mechanism established. cachesplit and cacheinject are separate config entries because their evidence is not comparable, and keeping them apart is what stops disabling placement from silently disabling the split too. Every preset that touches caching carries cachesplit, never cacheinject.

Explicit-breakpoint providers only (Anthropic, Bedrock, Vertex). Under an implicit longest-prefix cache (OpenAI, Gemini) the match already ends at the divergence, so neither component buys anything there.

cachesplit¶

cachesplit is in the default presets. It carries no logic of its own: the component's Reformat always reports Skipped, and the actual rewrite is body-level — it edits the top-level system array, which components never see — and lives in apply/prefixsplit.go, gated on this name being present in the pipeline. apply still honours cacheinject as a gate for the split, so an existing pipeline naming only cacheinject keeps working.

What the split does¶

[stable][volatile] as two text blocks with the same concatenated text, breakpoint on the first. Adjacent text blocks concatenate, so the model sees a byte-identical prompt while the provider gains a hash boundary that excludes the churn. Asserted by TestSplitIsConcatenationIdentical.

What it is worth¶

The full four-way measurement (structural target, mechanical verification, isolated live A/B, end-to-end agent run) is under What the split is worth below, because that is where the work was done. The headline: −34.1% mean cost on one Terminal-Bench task measured three times, and $0.0205 saved per warm session on Sonnet 5 in the isolated A/B. A second Terminal-Bench task was within noise and its numbers are not quoted.

Configuration¶

None. It is a name in the pipeline:

pipeline: [format, dedup, failed_run, cmdfilter, cachesplit]

A components.cachesplit: block is an error, not a no-op. The constructor used to take its raw block and discard it, so a document saying cachesplit: {ttl: 1h} looked configured and was not; it now decodes strictly like every other component and refuses the block.

When it shines¶

Anthropic-family agents whose system prompt carries a volatile tail (an environment snapshot, a timestamp, a git status) in front of a cache breakpoint.

When it's inert¶

Explicit-breakpoint providers only (Anthropic, Bedrock, Vertex). Under an implicit longest-prefix cache (OpenAI, Gemini) the match already ends at the divergence, so a block boundary buys nothing. It is also inert when the system block has no separable volatile tail.

Because the component itself always skips, /stats lists it under top_passthrough — that is expected, not dead weight. The split's saving is a provider-side cache effect, invisible to content-token counts.

cacheinject¶

How it works¶

Placement is the solution to a cost minimisation, not a heuristic. With R = 0.1 (cache read), W = 1.25 (5m cache write) and 1.0 (plain input) as multipliers on the base input price, four facts fix the policy:

1. Write anything resent even once. A token resent k more times costs W + kR written and 1 + k unwritten, so writing wins for k > (W−1)/(1−R) = 0.28. Hence a breakpoint on the last block, every turn.
2. Writes bill as a span, (highest breakpoint − read point), not per breakpoint. An extra breakpoint below the top costs exactly zero and adds a position a later turn's backward read-walk can land on, so none of the four slots is left idle.
3. Divergence is the large lever. If a turn differs from the previous one at block d, every block above d is unmatchable and the whole prefix bills at 1.0x instead of 0.1x. A breakpoint at d−1 recovers it — worth 12.5x more than any positional tuning, because it restores read→read rather than read→write.
4. Anchors must be turn-stable. A position is readable on turn t+1 only if it was written on turn t, so anchors are counted up from the start, never back from the end.

before:  [ tools ][ system ]…[ msg d−1 ][ mutated msg d ]…[ newest turn ]

after:   [ tools ][ system ]…[ msg d−1 {cache_control} ]…[ newest turn {cache_control} ]
                              ^ rescues the stable head    ^ writes this turn's growth

The component tracks a per-message digest per session to find d, and keeps every breakpoint the caller already set — an agent that places its own is usually already at the optimum, and the provider's 4-breakpoint cap means a fifth is a 400.

5m TTL by default. A 1h write costs 2.0x instead of 1.25x and only pays when the chance of an entry lapsing before its next reuse exceeds 65.2%, i.e. (2.0 − 1.25) / (1.25 − 0.1).

That denominator used to read (1 − 0.1), giving 83.3%, and it was wrong by 18 points: it assumed a lapsed entry re-bills at the 1.0x FRESH rate. Measured, it re-bills as a 1.25x cache-CREATION — those tokens still carry cache_control, so the provider writes a new entry rather than charging fresh input. Production ttl_expiry rows say so directly: cache_write averages 178,793 against fresh_input 1,712. The conclusion does not move — agent turns are seconds apart (measured median 7.6 s over 1,905 real turns) and every read refreshes the TTL for free — so set ttl: 1h only when reuse is genuinely sparse: a low-concurrency sweep with task starts more than 5 minutes apart, or a deployed agent handling a few sessions per hour.

On the IBM gateway that decision is currently moot for most models: measured live, the 1h tier is granted on aws/claude-haiku-4-5 (36,251 of 36,574 written tokens) and silently downgraded to 5 minutes on aws/claude-sonnet-5 (0 of 48,212). See apply/headttl.go.

What placement is actually worth¶

Effectively unmeasured, and the one live reading is mildly negative — which is why it ships off. One cacheonly vs off pair on SWE-bench Verified (aws/claude-sonnet-5, n=1 per arm, one task completed in both arms):

The −15.2% is not a saving: the agent took 3 fewer steps, and on this traffic cost tracks steps at corr 0.95. Per step, cost is +7.9% and cache-write +61.9%. Read that as one task, once, with a degenerate control — and as a reason to leave placement off, not as a measurement of the policy.

Against an agent that already caches well, expect exactly 0%. Claude Code marks its own final message on 466 of 472 measured requests (98.7%), so rule 1 is already satisfied and every position this component would choose collides with an existing breakpoint. The component earns its keep on agents that do not mark their own tail, and when a prefix mutates mid-conversation.

Simulated comparison¶

A simulation over a captured 91-request SWE-bench stream, calibrated to within 2.10 pp of a live 50-task run's cache-hit rate. It models the billing rule credibly; it is not a measurement of the policy.

Placing a single breakpoint before the newest turn shortens the cached prefix on every turn — a real cost increase, and the reason the policy marks the last block instead.

The 4-breakpoint budget is computed by the host¶

The provider caps cache_control at 4 across system + tools + messages together, and a component sees none of the first two. So apply counts them structurally from the raw body and passes the total as Ctx.ExistingBreakpoints for the component to budget against; the count covers the Bedrock cachePoint spelling too. apply also counts the output and logs an error if we pushed the request over 4 — a request that arrived over the cap is forwarded as-is and is not reported as ours.

On real Claude Code traffic the agent spends 3 of the 4 slots itself (1,771 of 1,794 requests carry exactly system=2, tools=0, messages=1), leaving this component one to place.

cache_control is metadata, not content, so apply writes it as a targeted sjson key on the original raw bytes after verifying the component's only change to that message was adding cache_control keys. Any wider change is discarded and counted in discarded_changes.

Lossiness¶

None. It attaches cache directives only; model-visible content is unchanged, asserted by comparing model-visible text byte-for-byte across all 91 captured requests.

Messages whose content is a bare string cannot carry a block-level directive; they are skipped rather than restructured, and the breakpoint falls back to the nearest markable block below so the prefix is still written.

Configuration¶

components:
  cacheinject:
    ttl: 5m        # or 1h — see the TTL rule above. Anything else is rejected at load.

K=4 breakpoints and L=20 lookback blocks are provider facts, not tunables.

cacheinject is in no preset: placement has never been shown to help, and shipping it on by default would enable an unmeasured policy on every request. The presets carry cachesplit instead, which enables the volatile-tail split — the measured part — without the placement. Add cacheinject by hand to run the placement study, or when your agent does not set its own cache_control.

When it's inert¶

On OpenAI- and Gemini-shaped wires, where cache_control does not exist at all. Also when four breakpoints are already present anywhere in the request (including system and tools), and on string-content messages.

What the split is worth¶

Is there a target? Across 50 real Claude Code sessions the stable half of the volatile-tail block takes 1 distinct value while the volatile tail takes 50 — so without the split, the hash covers a value unique to each session and the 6,877-token stable half can never be read from another session's cache.

Does the provider actually cache it? Two sessions differing only in the git snapshot, same breakpoints, same order, judged by cache_read_input_tokens from the API. Three runs, byte-identical results:

At Sonnet 5 rates that first request of a warm session costs $0.022205 → $0.001719 — $0.0205 saved per session (−92.3%). Per model: Sonnet 5 $0.0205 ($0.0307 from 2026-09-01), Opus 5 $0.0512, Haiku 4.5 $0.0102.

The saving lands once per request carrying the system prompt, not once per turn: within a session, turns 2…n already read the prefix turn 1 wrote in both arms. A cold first session pays the same either way.

End to end in a real agent run — Terminal-Bench, fix-code-vulnerability, 3 trials:

The end-to-end saving is 3.2× the isolated per-session figure because Claude Code issues sub-agent and side requests that each re-send the same system prompt, and each was independently re-writing it.

Treat 34.1% as one task measured three times, not a fleet average: a second Terminal-Bench task was within noise, its numbers are not quoted, and all figures are Sonnet 5 rates on one benchmark. Placement contributes $0 of it — every dollar above is the split.

See also: Components overview · Choose a preset