SWE-bench Verified, live, four arms: 13% cheaper than no compaction, and the highest reward. See the numbers

Context engineering for LLM agents

Cut the tokens every request carries.
Leave the agent alone.

A Go core that rewrites the bulky parts of a chat request — tool outputs, repeated file reads, dead command logs — and returns a smaller request of the exact same shape. Losslessly, or lossily with the original stashed behind a <<cg:HASH>> marker the model can expand on demand.

Install the plugin How it saves money Source
Apache-2.0 One Go binary OpenAI + Anthropic dialects, one port Fail open Never worse Reversible

Run it in 60 seconds no code changes

## once per machine
/plugin marketplace add rossoctl/context-guru
/plugin install context-guru@context-guru
/reload-plugins

## once per repo — decides which sessions get routed
/context-guru:install

Then /context-guru:status for what it's saving so far. Running your own org-wide proxy instead? Self-hosting quickstart →

What it measured

SWE-bench Verified, run live end to end with the claude-code agent on aws/claude-sonnet-5: 50 tasks that scored under all four arms (no compaction, context-guru, headroom, rtk).

−13.2%
billed cost
$27.77 vs $31.98 baseline
−17.8%
cache-read tokens
84.5M vs 102.8M
44/50
tasks solved
baseline 43 · headroom 40 · rtk 43
117 ms
added latency / request
what the compaction costs you
The cheapest arm and the highest reward

Cache-write stays within 1% of baseline in every arm — none of them busts the prefix cache. rtk adds 0 ms of latency but only filters Bash output. Method, decomposition and caveats →

What happens to a request

The pipeline sees the parsed messages array on bifrost's provider-agnostic schema, not a text blob, so it edits structure rather than guessing at string boundaries.

1 · normalize

One shape for every provider

An Anthropic tool_result block and an OpenAI role:"tool" message look identical to every component from here on.

2 · pipeline

Components run in order

Reformat ones re-encode losslessly; Offload ones drop bytes, stash the original, and leave a recoverable marker.

3 · splice

Changed spans only

Untouched fields survive verbatim, so upstream sees a request of the same shape it would have got anyway.

Three properties hold for every component, on every request

Fail open — an error or a panic reverts that one component and the original request is still forwarded. Never worse — a component that grows the request is reverted, and cost is measured on message content, so you never pay to compact. Reversible — every lossy drop is recoverable via the context_guru_expand tool or GET /expand. Architecture →

Fourteen components, stacked by config

Four lossless reformatters and ten reversible offloaders. Presets bundle the combinations that were measured: general for everyday agent traffic, agent for long sessions, codesafe for deterministic-only pipelines, mcp for tool-heavy JSON. In the two agent presets mask is the biggest single lever — ~27% of content tokens, with no measured reward loss.

Component reference

What each one drops, when it fires, its defaults, and whether it is recoverable.
Choose a preset

Which pipeline for coding agents, MCP traffic, or long-running sessions.
Before → after transcripts

Real captures: what the model saw before, and what it saw after.

See what it did

The proxy serves its own dashboard at /dashboard/: savings against four denominators, per-component economics, per-session and per-request diffs of exactly what was rewritten, and the cost of the safety mechanisms themselves.

Dashboard

Every panel, the honest-metrics rules behind it, and its API.
Observe mode

Measure what compaction would have saved without changing a single request.
Measure savings

What /stats counts, and the denominator each number uses.

Where to go next

How it saves money start here

The four mechanisms, each with a before/after example.
Quickstart: proxy (self-hosted)

Running it as an org-wide proxy instead of the Claude Code plugin.
Where the money went

Why a whole-request proxy beat a stream compressor and a shell filter.
Host adapters

Proxy, in-process plugin, or a bifrost LLMPlugin — one core.
Hosted service

One instance, many users: tokens, per-tenant config, cold storage.
Reproduce the results

Install and re-run all four benchmark arms yourself.