Full results — rtk (Rust Token Killer) (SWE-bench Verified, 50 tasks)¶

Live through the harness, claude-code agent on aws/claude-sonnet-5. Cache-aware billed input cost (fresh $2/M · cache-read $0.20/M · cache-write $2.50/M) + output $10/M, recomputed from each trial's token tiers. See REPRODUCE.md.

Totals¶

vs the no-compaction baseline ($31.98 / 43 solved / 36.1 steps): rtk is −9.0% billed cost, reward-neutral (43 = 43), −8% steps, at zero request-path latency and $0 tool cost. It is the 2nd-cheapest of the four arms — cheaper than headroom (−5.3%), behind context-guru (−13.2%). Full four-way table: comparison.md.

What rtk did (its own ledger)¶

rtk is a Claude Code PreToolUse hook that rewrites Bash commands in-container (pytest→rtk pytest, cat→rtk read, git status→rtk git status, …) and compresses the command output at the shell, before it enters the transcript. Because the compressed form is what gets cached from the first turn, rtk never mutates already-cached content — it sidesteps cache invalidation by construction (lowest cache-write of the four arms, 1.83M).

Across the 50 clean trials it rewrote 637 bash commands and removed 338k bash-output tokens (65.8% of bash output, its own bytes/4 estimate). Note the denominator: that 65.8% is of bash output only — a small slice of a ~98%-cached agent's total context — which is why the end-to-end saving is −9% billed cost, not −66%. (This is rtk's own documented caveat: it cuts bash output, not the bill, one-to-one.)

Savings concentrate in file reads via cat (62%) and grep (24%, the most-invoked command). Claude Code's built-in Read/Grep/Glob tools bypass the Bash hook, so rtk only sees output the agent routed through Bash — this is the structural ceiling on its reach.

Real before → after (rtk is deterministic; these reproduce its in-container behavior)¶

cat a source file → rtk read -l aggressive (332 B → 164 B): keeps imports + every signature, elides bodies.

import os
import sys
def alpha(a, b):
    // ... implementation
def beta(x):
    // ... implementation
class Widget:
    def render(self):
    // ... implementation

a failing pytest run → rtk test (1,055 B → 195 B, ~81%): failures + summary kept, 213 passing lines dropped.

[FAIL] FAILURES:
  FAILED tests/test_utils.py::test_parse_edge_case - AssertionError: assert None...
SUMMARY:
  ======================== 1 failed, 213 passed in 4.21s =========================

Per-task¶