coref¶

How it works¶

Every other offloader here decides from content (is this a duplicate, is it noise, is it superseded) or from age. coref decides from back-references: for each tool output, which identifiers it introduced, and whether any later model turn used them.

1. Index (Tier 1, exact, no LLM). For each tool output, collect the identifier-ish tokens it introduced — paths, symbols, ids, hashes, error codes. "Introduced" is the load-bearing word: a token already present at or before the producing turn is an echo, not something taken from the output. If the agent calls Read(src/auth.py), the path is in the tool-call argument, echoed by the result, and used again by a later Edit(src/auth.py) — an exact matcher scores that as a reference, but nothing was ever lifted out. Tokens spread across many outputs are dropped as session furniture.
2. Classify, into one of four: opaque (introduced nothing trackable — no evidence, never cut) · unreferenced (introduced identifiers, no later turn used any) · closed (used once or twice, and not for a long time) · open (used recently, repeatedly, or too new to have had the chance).
3. Cut, once, in a batch. Only the classes in the cut set, and only when the batch is large enough to be worth the cache-write it costs (below).
4. Latch. The decision is stored per session and replayed byte-for-byte thereafter. It is never re-derived.

Recency is measured from the head of the transcript, not from the output's own position: "referenced recently" is a statement about now. A span referenced three times forty turns ago is a hot span that happens to be old, which is why repetition (open_reps) overrides age.

Why it is batched, budgeted, and rare¶

coref is the only component that mutates the already-cached prefix on purpose. Every age-based offloader refuses to (Ctx.TailOnly) because breaking the prefix hash at index i forces the provider to cache-write the suffix, at 11.5× the price of a cache-read. Cutting S tokens with W tokens of transcript after the cut and T turns left in the session:

cost    = 11.5 x W    (cache-read-equivalents)
benefit =  S x T
worth it when   S x T  >  11.5 x W

A single early cut cannot clear that: 5k cut from 20% depth of a 150k transcript needs T > 276 turns. The same transcript cut by 60k in one pass needs T > 23, which a long session reaches. Hence:

• min_batch_frac — one rewrite has to serve the whole pass.
• rewrite_budget — the spend is capped per session and answerable, not incidental.
• break_even — the inequality itself, with T estimated from how fast the transcript has been growing. Its consequence is counter-intuitive and deliberate: firing at 90% of the window means T ≈ 0, so the profitable moment to compact is earlier than the moment of maximum pressure.

Before → After¶

after:  [tool output compacted: no later turn referred back to it; starts: 0 tree_line_0 = compute_tree_0(arg_0) …] <<cg:…>> [full output: call context_guru_expand]
after:  [tool output compacted: the value taken from it survives in a later turn; starts: …] <<cg:…>> [full output: call context_guru_expand]

The second form is the closed cut (cut_closed: true). Its claim about a surviving witness is free rather than asserted: coref only ever cuts tool outputs, and references live in model turns, which it never cuts — so "a later turn referred back to this" and "the value taken from it is still in the request" are the same fact.

Lossiness¶

Lossy but reversible — cut outputs are stashed and recovered via context_guru_expand / GET /expand, and an expanded output is marked kept-verbatim so it is never re-cut.

What the marker leaves behind¶

Two rules, both aimed at the gap between "notices and expands the right thing" and "notices but cannot tell which marker":

• It describes the shape of structured content, not just its first line — 200 records, fields: address, id, name. That is addressable: an agent hunting for an address can tell this is the output to expand. A head peek of one arbitrary row cannot do that, though it works well for a file read or a traceback, which is what the peek is still used for.
• It never claims the cut was safe. An earlier version wrote "no later turn referred back to it" — precisely the claim that is false whenever the reference was transformed or semantic — which read as reassurance and discouraged the expand call that would have repaired the mistake. A marker that talks the model out of recovering is worse than an opaque one. A test enforces this.

Configuration¶

What it deliberately does not do¶

• It does not consult repairLostFreeze. mask and failed_run may re-derive a lost decision at depth, because their replacement is a pure function of (content, config) and so reproduces the bytes the provider already cached. A co-reference decision is history-dependent by construction — re-deriving it against a longer transcript can yield a different class and different bytes, which is exactly the prefix flip the repair exists to prevent. A lost coref freeze declines.
• It never resurrects a span. New evidence cannot un-cut, because un-cutting is a second rewrite. Monotonicity here is a cache-cost requirement, not tidiness.
• It does not cut what it cannot see. An output that introduced no trackable identifier is opaque, not unreferenced, and is never cut at any setting. Measured, that is 8% of tool-output mass on interactive traffic and 40% on LOCA-bench — bulk record dumps of plain values like [{"name":"david","id":123,"address":"foobarbaz"}], where short lowercase words and 3-digit numbers are exactly what the identifier rules exclude. Absence of evidence is not evidence of deadness.
• It cannot tell an anchor reference from a payload reference. If the model says "remember david 123 address", the reference is real but the value needed (foobarbaz) was never copied into a model turn. So a low used_frac is ambiguous, and this is the substantive reason cut_closed ships off.
• It does not see Tier 2. A value that was summed, unit-converted or reworded before being restated leaves no exact match. unreferenced means "no later exact use" and must never be read as "unused"; the LLM escalation for that case is an open question in the proposal, not shipped.

When it shines¶

Long sessions that cross a context threshold with a lot of survey-and-discard traffic — directory listings, wide searches, exploratory reads that the agent never returns to. It is complementary to mask: mask drops the old, coref drops the never-used, and an old-but-hot span is exactly the case mask gets wrong and coref protects.

When it's inert¶

Below the trigger; no output above min_tokens; every large output referenced (open); a batch below min_batch_frac; the rewrite budget spent; or the break-even inequality unmet — which, near the window edge, is the common and correct outcome.

See also: the held-out selection experiment · the proposal and its derivation (docs/proposals/coref-compaction.md) · Components overview · mask · dedup

Glossary¶

Every term the co-reference work uses, in one page, in the order you meet them. Full argument in the proposal (docs/proposals/coref-compaction.md); measured numbers in the density pass and the selection experiment; config in Configuration above.

The one-sentence version¶

coref cuts tool outputs that nothing later in the conversation ever used — and because cutting deep in a transcript forces the provider to re-write its cache, it does all its cutting at once, rarely, and never changes its mind.

1. The core idea¶

2. The two cases from the original idea¶

The idea started with: if a later turn references an earlier output, that means one of two things.

The original framing separated A from B by distance (B = recent, A = early). The measurement says distance barely works — see closed_dist below.

3. The four verdicts (what the classifier outputs)¶

For each tool output, exactly one of:

Why "closed" looks cheap to establish: coref only ever cuts tool outputs, and references live in model turns, which it never cuts. So a reference is always a surviving copy of something — no separate search for the witness is needed.

4. The two thresholds that decide closed vs open¶

5. The three measurements per output (and the one that's easy to get wrong)¶

6. The three tiers of reference (what's detectable)¶

Tier 2 is the objection raised on the original thread (values drift through paraphrase and unit conversion). Measured at ~2% of model turns on interactive traffic — but only via a numeric proxy, and Tier 3 is not measured at all, so treat that 2% as a floor on invisible references rather than a total. Real, small on this traffic, and the reason unreferenced is an upper bound.

7. The cache economics (why the component has this shape)¶

Why this reshapes everything: a single early cut can never clear it — 5k cut from 20% depth of a 150k transcript needs T > 276 turns. Three consequences, and they are the design:

The counter-intuitive consequence: firing at 90% of the context window means T ≈ 0 — paying a rewrite for a saving collected once. The profitable moment to compact is earlier than the moment of maximum pressure.

What all this means at 200k vs 1M¶

Raised in review, and the answer is less obvious than "bigger window, more to cut".

The break-even inequality is scale-invariant. Rearranged, S × T > 11.5 × W says T > 11.5 × (W/S) — the turns you need depend only on the ratio of rewritten suffix to cut mass, never on absolute size. On the measured interactive corpus that ratio is ~15 (a 10.5k cut against a 157k suffix), hence T > 138. A 1M-token transcript with the same density of cuttable mass has the same ratio and the same required T. So a larger window neither rescues nor damns the token economics; it only moves when the trigger fires. What actually improves the ratio is cutting a larger share of what lies after the shallowest cut — which is an argument for cutting deep and rarely, not for cutting more.

Three things genuinely do change:

And the prize is a step function, not a slope. You either drop below the agent's compaction threshold or you don't; cutting 90% of what was needed is worth nothing. Worse, clearing it exactly buys one turn — the next turn grows past it again, and then you pay a second cache-write at the point where W is largest. So the real requirement is (usage − threshold) + growthPerTurn × headroomTurns, which on measured traffic works out at 20–25% of the request for 40–60 turns of headroom — and Tier-1 matching finds only 4–10%.

8. Mechanism terms (how it stays cache-safe)¶

9. Where things live¶

The Go index and the Python script must agree on what a reference is. If they drift, the thresholds the measurement produces are calibrated for a different algorithm than the one that ships — silently. That's why the same fixture and the same false-positive regression cases exist on both sides.

10. Status in one line¶

Mechanism implemented and tested; cut_unreferenced on by default, justified on yield (21% of mass on interactive traffic, ~70% on benchmark traffic) and now bounded on accuracy (11% false-drop); cut_closed off until measured on the eval-box corpus; component opt-in, in no preset; no reward measurement exists, which remains the gate.

11. Experiment terms (the held-out measurement)¶

Introduced by the selection experiment, and worth having here because they are how any future claim about this component should be scored.