Context engineering for LLM agents
Cut the tokens every request carries.
Leave the agent alone.
A Go core that rewrites the bulky parts of a chat request — tool outputs, repeated
file reads, dead command logs — and returns a smaller request of the exact
same shape. Losslessly, or lossily with the original stashed behind a
<<cg:HASH>> marker the model can expand on demand.
Run it in 60 seconds no code changes
## build it — needs Go 1.26 and a C toolchain (the tokenizer uses cgo)
make build
## start it — one port serves both dialects
./bin/context-guru-proxy --preset general
## listening on :4000
## point the agent at it; nothing else about the agent changes
export ANTHROPIC_BASE_URL=http://localhost:4000/anthropic
export OPENAI_BASE_URL=http://localhost:4000/openai/v1
Then curl -s localhost:4000/stats | jq for the token-weighted savings
rollup, or open /dashboard/.
Claude Code setup →
What it measured
SWE-bench Verified, run live end to end with the claude-code agent on
aws/claude-sonnet-5: 50 tasks that scored under all four arms
(no compaction, context-guru, headroom, rtk).
$27.77 vs $31.98 baseline
84.5M vs 102.8M
baseline 43 · headroom 40 · rtk 43
what the compaction costs you
Cache-write stays within 1% of baseline in every arm — none of them busts the prefix cache. rtk adds 0 ms of latency but only filters Bash output. Method, decomposition and caveats →
What happens to a request
The pipeline sees the parsed messages array on bifrost's
provider-agnostic schema, not a text blob, so it edits structure rather than
guessing at string boundaries.
One shape for every provider
An Anthropic tool_result block and an OpenAI role:"tool" message look identical to every component from here on.
Components run in order
Reformat ones re-encode losslessly; Offload ones drop bytes, stash the original, and leave a recoverable marker.
Changed spans only
Untouched fields survive verbatim, so upstream sees a request of the same shape it would have got anyway.
Fail open — an error or a panic reverts that one component and the
original request is still forwarded. Never worse — a component that
grows the request is reverted, and cost is measured on message content, so you never
pay to compact. Reversible — every lossy drop is recoverable via the
context_guru_expand tool or GET /expand.
Architecture →
Fourteen components, stacked by config
Four lossless reformatters and ten reversible offloaders. Presets bundle the
combinations that were measured: general for everyday agent traffic,
agent for long sessions, codesafe for deterministic-only
pipelines, mcp for tool-heavy JSON. In the two agent presets
mask is the biggest single lever — ~27% of content tokens, with no
measured reward loss.
See what it did
The proxy serves its own dashboard at /dashboard/: savings against four
denominators, per-component economics, per-session and per-request diffs of exactly
what was rewritten, and the cost of the safety mechanisms themselves.
/dashboard/.Where to go next
Build, run, point an agent at it, read the savings.
Run it as a compaction endpoint instead of a request-path proxy.
Why a whole-request proxy beat a stream compressor and a shell filter.
Proxy, in-process plugin, or a bifrost LLMPlugin — one core.
One instance, many users: tokens, per-tenant config, cold storage.
Install and re-run all four benchmark arms yourself.