3/6 · Four Layers, Not a Ranking: Mapping the Token-Reduction Toolbox

The frame	Token reduction is FinOps for AI. Measure first, optimize second
The map	Four layers, each filtering a different point in the token flow
The leverage	Layer 1 (CLI output) is the cheapest win: 2-8% of total volume depending on workflow, but pure noise and trivial to filter
The MCP tax	One GitHub MCP server can cost 44K tokens for a one-line question
The blind spot	API gateways see nothing on Max/Pro. No API key, nothing to intercept

Disclosure up front: I’m a core contributor to RTK, one of the tools mapped below. I’ve kept the numbers sourced and the comparisons honest, but you should know where I sit.

In 2014, cloud teams were burning money they couldn’t see. EC2 instances running overnight for nothing, forgotten S3 buckets, credits vanishing into a bill nobody could break down. Then FinOps showed up. Not a tool, a discipline: measure first, optimize second, attribute everything. In 2026 the same thing is happening with tokens. The Claude Max license itself is fine, predictable, a fixed cost. What actually hurts is the noise you inject into every session without seeing it.

This article maps the tools that cut that noise. There are a lot of them, and from the outside they look like a crowded field of competitors all claiming the same thing. They aren’t competitors. They sit at different layers of the token flow, and each layer answers a different context-rot vector. It’s a map, not a ranking.

---

The four-layer map

Think of where a token can be intercepted between your command and the model’s reply. There are four places, and the tools cluster into them.

Layer 1, CLI proxy. Filters command output before it ever enters the context. This is where the easiest wins live, per-command reductions are large and setup is trivial.

Tool	Mechanism	Reduction	Signal
RTK	Rust proxy, filters stdout before injection, native Claude Code hooks, SQLite analytics	60-90% per command	~69K stars
Snip	Go binary, declarative YAML rules per command	60-90%	360 stars
Distill	Generic pipe, any CLI output to LLM-optimized answer	varies	638 stars

Layer 2, MCP sandbox. Intercepts the tool calls that dump into context, and trims the schema cost of MCP servers.

Tool	Mechanism	Reduction	Signal
context-mode	SQLite + FTS5 session tracking, MCP output interception, “Think in Code,” routes across 17 agent platforms	up to 98% MCP output	~18.6K stars
Cloudflare Code Mode MCP	Exposes two meta-tools (search(), execute()) instead of one schema per endpoint; the model writes and runs JS in a sandboxed V8 worker	99.9% on schema loading (2,500+ endpoints, ~1.17M to ~1,000 tokens)	Production at Cloudflare
mcp2cli	Converts MCP servers to CLI commands, avoids full schema injection	~13K tokens/server	~2.2K stars
LeanCTX	MCP server, 10 context read modes (signatures, diff, task-filtered), file read cache	60-99%	~3.1K stars
OpenWolf	Hook-based middleware, pre-read file summaries (model decides before pulling full content), repeated-read blocking, .wolf/ project intelligence (anatomy.md + cerebrum.md + token ledger)	65.8% average	~2K stars
Token Savior	AST symbol index, 34 Bash output compactors, cross-session SQLite memory	-80% active tokens (author benchmark, synthetic codebase)	1K stars
Serena	Semantic code indexing, symbol navigation	40-70%	~26K stars
Context7	Documentation retrieval, relevant chunks only	doc-specific	~58K stars

Layer 3, output style. Reduces the verbosity of the model’s own responses.

Tool	Mechanism	Reduction	Signal
Caveman	Telegraphic responses, plus a PreCompact hook compressing input, now ports to 30+ agents (Codex, Gemini, Cursor, Windsurf, Cline, Copilot)	40-75% output, 46% input	~86K stars (highest in the tracked set, above LiteLLM and crewAI; treat with some skepticism until a star-history graph confirms it)
—uc mode	Symbol-based communication protocol (SuperClaude Framework)	30-65% output	SuperClaude framework
LLMLingua	ML token pruning before the call, under 2% accuracy loss	60-80%	6.4K stars, no commit since April 2026, treat as unmaintained
claude-token-efficient	Single CLAUDE.md file, no code, instructs concise output	~63% output tokens (self-reported)	5.7K stars

Layer 4, API gateway. Compresses history at the API boundary. Critical caveat below.

Tool	Mechanism	Reduction	Signal
LiteLLM	Unified proxy across 100+ providers, token counting, routing	varies	~52.8K stars
Bifrost	OSS gateway, per-dev budgets, semantic caching	up to 70% via semantic caching (self-reported)	5.8K stars
Headroom	CacheAligner stabilizes cache-hit prefixes, then routes to three specialized compressors (SmartCrusher for JSON, an AST-based compressor for code, a fine-tuned HF model for prose/logs), reversible via headroom_retrieve	47% full-session, 92% code search (author benchmark)	~57K stars
pxpipe	Optical compression: renders large tool results and stale conversation turns as PNG columns instead of text, at a computed cost threshold	59-70% end-to-end billed tokens (author-measured, cross-checked against actual invoices)	~4.3K stars
Compresr	Background compaction at 75% context via secondary model	claims 200x	619 stars · YC-backed

Across all four sits a transversal band: observability tools (the 16.5K-star ccusage, claude-spend (404 stars), cc-statistics (109 stars, growing slowly), the 8.3K-star Claude Code Usage Monitor, the 8.2K-star CodeBurn, ccboard (75 stars)) that don’t reduce tokens but tell you where they go. The Claude Code Usage Monitor gives real-time burn-rate predictions in the terminal. CodeBurn classifies spending by turn type across 13 categories and correlates sessions with git commits via codeburn yield, answering not “how much did I spend?” but “on what, and did it ship?” ccboard extends the same layer into a TUI and web dashboard: sessions, cost analytics, config, hooks, and MCP servers in a single 5.8MB binary, with a 30-day forecast and budget alerts on top of the raw token breakdown.

One caveat on the numbers before the deep-dives. The reduction figures above describe what each tool touches, not what it saves on your total session. RTK’s 60-90% applies to CLI output, which typically accounts for 2 to 8% of a session’s total token volume depending on how many scripts, tests, and CI commands the workflow runs. Caveman cuts model response verbosity, not file reads or conversation input. Context7 reduces documentation fetch cost, not the full context. Each layer is a genuine lever on a real slice of the problem; none of them is a session-wide multiplier. Combining tools from several layers plus cleaning up MCP overhead can realistically reach 40-60% on a full session in the right setup, but no single tool in this map achieves that today, and any figure that implies a global session reduction is measuring the layer, not the bill.

Headroom is worth singling out before the deep-dives. Now at around 57,000 stars (July 2026, transferred from the founder’s personal repo to the headroomlabs-ai org, a standard GitHub ownership move, not a fork or a rewrite), it is the most-starred context engineering tool outside RTK and Caveman, and it doesn’t fit neatly into one layer. In HTTP proxy mode it intercepts at Layer 4, compressing the full message history before the API call. In MCP mode it acts at Layer 2, triaging tool outputs. The pipeline runs CacheAligner → ContentRouter → CCR: the CacheAligner stabilizes prompt prefixes to preserve provider-side KV-cache hits, then the ContentRouter dispatches to one of three specialized compressors, a JSON-specific crusher, an AST-based compressor for code, and a fine-tuned Hugging Face model for prose and logs. Compress-Cache-Retrieve (CCR) stores the compressed content in a local SQLite index, sends {{HEADROOM_TAG_N}} placeholders to the model, and retrieves the original when the model calls headroom_retrieve. Their published benchmarks (reproducible via python -m headroom.evals suite --tier 1, validated against GSM8K, TruthfulQA, SQuAD v2 and BFCL) show 92% for both code search and SRE incident debugging, 73% for GitHub issue triage, and 47% for codebase exploration; the broader range is 60-95% on JSON-heavy workloads down to 15-20% on generalist coding sessions. Two open production caveats: issue #1158 documents that headroom wrap claude silently caps Claude Max users at 200K context instead of 1M; issue #1227, currently unresolved, exposes CCR endpoints without loopback protection, meaning any local web page can read cached tool outputs without authentication. The CCR retrieval TTL issue from earlier versions was fixed in v0.25.0 via the HEADROOM_CCR_TTL_SECONDS environment variable. Core compression is stable. CCR in proxy mode on a shared workstation needs caution until #1227 is patched. One maintainer claim to treat as marketing rather than fact until an outside audit confirms it: roughly $700K in aggregate savings across 200 billion tokens processed, self-reported, no third-party verification found.

pxpipe answers a different question: what if the tokens never became text in the first place. It’s a local proxy that intercepts /v1/messages calls and, past a computed cost threshold, rewrites large tool results, already-collapsed older turns, and the static system-prompt block into PNG images instead of sending them as text, on the logic that a model can read roughly ten times more information from a page of rendered text than from the equivalent token count of raw text, a pattern that traces back to DeepSeek-OCR’s original demonstration of the same compression ratio in a different context. The threshold is concrete: image rendering pays off past roughly 19 characters per token, and pxpipe’s own measurement of real Claude Code traffic finds an average of 1.91 characters per token, meaning most large payloads clear the bar. It never touches the model’s own output, recent turns, or sparse prose, exactly the content where text stays cheaper. The measured result is 59 to 70% off the end-to-end bill, logged per request and cross-checked against the actual invoice rather than estimated. The honest caveat is verbatim recall: exact strings like passwords, hashes, or IDs are explicitly excluded from image rendering because the model misreads them from a page rather than a token stream, 13 of 15 correct on Fable 5, 0 of 15 on Opus in the maintainer’s own pilot. It defaults to a narrow model allowlist (Fable 5 and GPT-5.6) for that reason, with Opus 4.7/4.8 and GPT-5.5 opt-in only. Specific to Claude Code and the Anthropic API, not a general multi-agent tool the way Headroom is, but the two complement rather than compete: same layer, API-gateway, different modality, text compression versus image rasterization.

Three of these deserve a closer look, because they carry most of the practical leverage. The guide keeps a fuller, regularly updated inventory of the token compression tools in its context engineering reference.

---

Deep-dive 1: RTK and Layer 1

Layer 1 is where most people start, because tool output is pure noise wrapped around a little signal and it’s the easiest traffic to intercept. When Claude runs git diff, pnpm list, or a test suite, the raw output is thousands of lines wrapped around a few lines that matter, and all of it enters the window and stays there, degrading every subsequent turn.

RTK is a synchronous Rust proxy that sits on Claude Code’s PreToolUse hook. Claude triggers a command, RTK intercepts it, filters the noise, and returns only what matters. To the agent, nothing looks unusual. To the context, 60 to 90% of that command’s tokens never arrive, and on the noisiest commands more than that. A git status that costs 2,000 tokens raw drops to around 200. A vitest run drops about 92%. A git log drops 92.3%. These are per-command reductions on CLI output, not session-wide totals: CLI traffic runs 2 to 8% of a session’s total token volume depending on how script-heavy the workflow is, so the gains are real but scoped.

The number I trust most is the longitudinal one. rtk gain --all reads three months of real command history and reports 437.8 million tokens saved across 70,310 commands, a 79.2% average reduction. That’s not an estimate from a benchmark, it’s the bill from actual usage. The whole thing is one 4.8MB Rust binary, no dependencies, sub-10ms startup. It crossed 69,000 GitHub stars, up from 24,000 in April, which is less a startup finding a market than a market finding its tool, though a near-tripling in three months is itself worth checking against the star-history graph before repeating in a room where it needs to hold up.

$ rtk gain --all   (monthly breakdown)

Month       Cmds      Input     Output      Saved   Save%
──────────────────────────────────────────────────────────
2026-03     6957     100.6M       8.8M      91.8M   91.3%
2026-04    20789     230.7M      41.6M     189.2M   82.0%
2026-05    32099     178.9M      34.8M     144.1M   80.6%
2026-06    10465      42.8M      30.3M      12.6M   29.5%
──────────────────────────────────────────────────────────
TOTAL      70310     553.0M     115.5M     437.8M   79.2%

Three months of real command history: 70,310 commands, 437.8M tokens that never reached the context. The recent dip is honest too. As more of a session is the model’s own output, which a CLI proxy can’t touch, the headline rate falls, exactly what the cost breakdown later in this article predicts.

---

Deep-dive 2: the MCP tax

The second hidden cost is one most people never see, because it’s paid before the first question. Every active MCP server injects its full tool schema into the system prompt, every turn, whether you use the tools or not.

The benchmark that makes it concrete comes from Scalekit. Asking “what language is this repo?” through the GitHub MCP server, which exposes 43 tools, cost 44,026 tokens. The same question answered with the gh CLI cost 1,365. A factor of 32, for an identical answer. Each active MCP server runs around 13K tokens of system-prompt overhead before you’ve typed anything. I feel this one first-hand: a Google Search Console MCP I maintain, gsc-mcp, also exposes 43 tools, and that schema rides in the system prompt every turn whether the session touches search data or not.

Cloudflare shipped the same insight as a production feature rather than a benchmark: Code Mode MCP exposes only two meta-tools, search() and execute(), backed by a typed SDK, and the model writes and runs JavaScript against it inside a sandboxed V8 worker instead of receiving a schema for every endpoint. Against Cloudflare’s full API surface, more than 2,500 endpoints, that turns roughly 1.17 million tokens of schema-loading cost into about 1,000, a 99.9% cut, at production scale, not a research demo. It’s the structural version of the fix in the paragraph below: instead of trimming which servers you keep loaded, it removes per-endpoint schema injection as a concept entirely.

The token count is only part of the problem. Zineb Bendhiba argued that each additional tool exposed to a model also raises the hallucination rate, because the model must reason across a wider option space on every turn (ep 326). A related design principle from Frédéric Barthelet, co-founder of ALPIC and creator of Skybridge (a full-stack TypeScript framework for MCP apps, type-safe, React-powered): build MCP tools as complete user journeys rather than atomic endpoints, because five sequential tool calls each multiply the model’s error probability, while a single well-scoped tool compresses that risk into one decision (ep 329). Coming from someone who has shipped production MCP infrastructure rather than benchmarked it, the advice is concrete.

At ManoMano, where 250 developers run Claude Code in production, Jocelyn N’takpe described an explicit rule that follows the same logic: anything that can be implemented as a native Claude Code skill should be, because skills carry no schema overhead and run without MCP round-trips (ep 346). MCP servers stay reserved for integrations that genuinely cannot be expressed any other way.

The fix has two halves. The mcp2cli pattern converts MCP servers into plain CLI commands, moving the schema cost client-side and cutting it 96 to 99%. Claude Code also addresses part of this natively with deferred tool loading, where schemas load on demand past a context threshold rather than all at once. One community report combined MCP cleanup with mcp2cli and compression scripts and took a session from 92K tokens to 5,500, a 94% cut, mostly by not paying for schemas nobody was using.

The practical move is duller than the number suggests: run claude mcp list, look at what’s actually loaded, and turn off the servers you’re not using this week. Most setups carry two or three servers they forgot they installed.

---

Deep-dive 3: output, and the Max/Pro blind spot

Layer 3 is smaller but real. Caveman strips the hedging and filler from responses for a 40 to 75% output reduction, and its PreCompact hook compresses input by around 46%. SuperClaude’s --uc mode does something similar with a symbol protocol, though it requires the full SuperClaude framework installed rather than being a native Claude Code flag. The practical split: Caveman for interactive sessions where you still read the output, cutting it around 40% in practice, and --uc (if you run SuperClaude) for batch or agentic runs where density beats readability, closer to 65% on those runs specifically. At the opposite end of that complexity spectrum, claude-token-efficient is a single CLAUDE.md that instructs Claude to be concise: zero install, zero code, and 5.7K GitHub stars that measure exactly how much demand exists for that level of simplicity. The ceiling is real (output tokens only, no effect on file reads or input), but so is the signal. These gains are modest next to Layer 1, but they compound: a more compressed session leaves more room for the rules from part two to stay in the window.

Layer 4 is where the interesting trap lives. Every API gateway, LangFuse, Helicone, Bifrost, and the rest, works by intercepting API calls. That’s the whole mechanism. On Claude Code Max and Pro, there is no API key to intercept. Claude connects directly to Anthropic’s servers with subscription credentials, and the gateway sees nothing. These tools aren’t immature about it, they’re structurally blind to it. For the large population of developers on Max and Pro, the entire enterprise observability layer is dark, and the only thing that can give you per-session analytics is a tool operating at the CLI layer, below the billing boundary.

Cost-aware routing at the prompt level is the approach that survives the Max/Pro blind spot. Antonio Goncalves described deploying a routing model that assesses prompt complexity and dispatches to the appropriate model tier, reporting that clients who adopted this pattern halved their inference bill (ep 357). A lightweight classifier doing that dispatch works at the application layer, with or without an API key to intercept.

Cost control on a subscription comes from different levers. Running a dual-instance setup, an Opus planner directing Sonnet executors, costs roughly $100 to $200 a month against $500 to $1K for Opus doing everything. Fast Mode trades quota for speed, it runs Opus with faster output and can chew through your Max allowance noticeably faster, so toggle it off with /fast when you’d rather conserve it. Per-task budget caps keep a runaway session from eating the week, with rough tiers of $2 for tests and docs, $5 for a medium refactor, $10 for architecture work. When a session nears its cap it summarizes what is left rather than stopping mid-task, so you lose tokens, not the thread. None of these show up in a gateway dashboard, because the gateway can’t see the traffic.

---

Prompt caching: before the tool layer

Every tool in the map above intercepts tokens on their way in or trims the model’s output on the way out. Prompt caching operates upstream of all of them: it’s a provider-side mechanism that reuses computation already done, charging 10% of the original input price on a cache hit. The tokens still enter the context window, but the provider skips reprocessing the prefix it already computed on a prior call.

The mechanics differ by provider. Anthropic requires explicit cache_control breakpoints: you mark which blocks to cache (system prompt, tools, early conversation turns), and everything before that marker is stored server-side. Write cost is 25% above normal input pricing; read cost drops to 10%, so 90% off on hits. Automatic caching has also been available since early 2026 at the request level, where Anthropic places the breakpoint on the last cacheable block without per-block annotation. The TTL was quietly reduced from 1h to 5min in March 2026. The 1h window remains available on Opus 4.5, Haiku 4.5, and Sonnet 4.5.

OpenAI handles this automatically with no client config. Cache activates at 1024 tokens, by 128-token increments, at no extra cost. The constraint is strict: the prefix must be byte-for-byte identical. One changed character in the first 1024 tokens is a complete miss. Gemini uses a third model: explicit “cached content” objects with a configurable TTL (1h by default) that you create once and reference across requests.

What kills the cache on every provider is dynamic content sitting before the breakpoint. The usual culprits are a timestamp in the system prompt, a per-request session ID, tool definitions that reorder between calls, or non-deterministic JSON serialization where key order shifts. The rule is the same everywhere: stable content first, dynamic content last, breakpoint between them.

For local inference the picture splits. Ollama reuses KV cache between requests if the model stays loaded in memory, but it unloads after 5 minutes of inactivity by default, destroying the cache on the next call. keep_alive: -1 keeps it resident, and the measured result is 962ms dropping to 54ms with a warm cache (17.7x). SGLang’s RadixAttention is more systematic: it maintains a radix tree of all prior KV caches, matches the longest available prefix automatically, and requires no configuration. Hit rates run 50 to 99% depending on workload. vLLM offers block-level automatic prefix caching, less flexible but widely deployed in production.

The data to validate all of this lives in the API responses. Anthropic surfaces cache_creation_input_tokens and cache_read_input_tokens in the usage object; OpenAI exposes cached_tokens in prompt_tokens_details. Most setups never log these fields, which means the cache degrades silently and nobody notices. Helicone (open-source) traces both fields per request with dashboards and recommends alerting if the hit rate falls below 70%. LiteLLM’s cache_control_injection_points automates breakpoint placement at the proxy level, which is roughly what Anthropic now does natively with automatic caching, but useful for OpenAI or cross-provider setups. Without hit-rate tracking, you can spend weeks tuning the tool layer while the provider cache silently misses on every call.

---

What doesn’t work, and measuring first

Before any of that, the deepest measure-first principle is whether you need a model for the specific task. Charles Cohen described a content-moderation system where rule-based symbolic AI outperformed LLMs on precision, latency, cost, and explainability in production, because the task was deterministic enough for rules to win (ep 333). Every token you never send is free and adds no noise to the context. A complementary finding from production: N’takpe at ManoMano cited a Berkeley study showing that 94% of agent errors would have been caught by a compilation step, which drove their team to migrate parts of their stack to strict TypeScript specifically to improve Claude Code output reliability (ep 346). A typed codebase is infrastructure for the agent layer, not just a style preference.

A lot of clever-sounding token tricks fail in practice, and it’s worth knowing which, so you don’t waste a weekend on them. Base64 and zlib compression fail because the model doesn’t decode binary. Binary diffs aren’t interpretable. Emoji encoding costs more than it saves, each emoji running 2 to 3 tokens. Regex redaction of secrets carries a 60%-plus miss rate, worse than useless because it feels safe. Path-prefix elision backfires when the model copies the shortened paths verbatim and the commands fail.

One honest caveat about the headline numbers, including RTK’s: most CLI-proxy savings are measured in characters or bytes, not BPE tokens. The two correlate well on noisy CLI output, but they aren’t identical, and an enterprise-grade benchmark should validate against the actual tokenizer rather than trust the byte count. I’d rather say that than quote a number I can’t fully stand behind.

And the breakdown that reframes the whole exercise: when you actually measure a session, CLI output typically runs 2 to 8% of the total token volume depending on how script-heavy the session is. That is the honest counterweight to Layer 1. It is the cheapest, noisiest traffic to cut, which is why the per-command reductions run so high, but it was never the bulk of the bill. Native file reads (Read, Grep, Glob) are 40 to 60%, the single largest cost, and most tools don’t touch them. OpenWolf is one of the rare tools explicitly targeting this layer: it intercepts file reads before they happen, gives the model a token-estimated summary first so it can decide whether the full content is needed, and blocks repeated reads within the same session, which their own telemetry puts at 71% of all file read calls. Static instructions are 5 to 15%. The model’s own output is 30 to 40%. This is the FinOps lesson over folk wisdom: you optimize where the spending is, not where the easy win feels good. Run ccusage, or rtk gain, or codeburn, and look at your own breakdown before you install anything. For tracking this across a team rather than one machine, the guide’s team metrics page covers what to measure and how.

---

What to do Monday

Four actions, escalating effort, all visible by Monday morning.

• Run rtk gain (about 2 minutes). It reads your command history and shows the real waste, per command and per session, pulled from actual usage, not a benchmark estimate.
• Run claude mcp list and disable the servers you aren’t using (about 5 minutes). That alone reclaims the MCP tax on every future session. Which servers actually earn their schema cost is its own question, and the MCP token cost guide breaks down what each one adds to context and when a plain CLI call would do the job for free.
• Add two terse rules to CLAUDE.md (about 10 minutes): confirm success in one line, explain only on errors. That trims the output layer for free.
• Add cache_read_input_tokens logging to whatever calls the API (about 15 minutes). If the count is consistently near zero on a workflow with a stable system prompt, you’re paying full input price every turn for content the provider could cache at 10%. Helicone adds this automatically if you route through it.

Together, depending on how much MCP overhead and output bloat you were carrying, these often land a 40 to 60% reduction on your next session, with no model change and no workflow change. The science in part one explains why that also makes the output better, not just cheaper. The discipline in part two keeps it that way over months. This is the layer where you prove it on the bill. And if you want the wider view, part four covers the roles this whole practice created, from context engineer to the jobs that didn’t exist three years ago.

From the field, via IFTTD episode transcripts: Zineb Bendhiba, ep 326 on tool count and hallucination; Frédéric Barthelet, ep 329 on MCP tool design as complete journeys; Antonio Goncalves, ep 357 on model routing; Charles Cohen, ep 333 on symbolic AI beating LLMs in production; Jocelyn N’takpe, ep 346 on skills-first configuration at 250-developer scale.

If you’ve measured your own token breakdown and it looks nothing like the 2 / 40-60 / 5-15 / 30-40 split above, I genuinely want to see it. The shape of real usage is more varied than any single study captures.