The Four Families of Context Relief for LLM Coding Agents

Run a coding agent on anything bigger than a toy repo and you hit the same wall. The context window fills up. Not with the answer — with the search for the answer. Twelve file reads, four grep results, a stack trace, the output of a test run that failed for an unrelated reason. By the time the agent is ready to write the fix, half its working memory is archaeology it will never look at again.

I've spent the last few months building a semantic-search-plus-working-memory MCP server (I'll call it vectr throughout — it's the running example, not the point of the post), and the single most clarifying thing I did early on was stop treating "the context is full" as one problem. It's four problems wearing a trench coat. Each has its own mechanism, its own cost, its own failure mode, and — this is the part people miss — they only work when you compose them correctly. Get the composition wrong and you don't get relief; you get a subtle new class of bug where the agent confidently reasons over information it no longer has.

So here's the map I wish someone had handed me. Four families of context relief: what each one actually buys you, where each one bites, and how they fit together.

A few definitions first, because the jargon is dense. A token is the unit an LLM reads and bills by — roughly three-quarters of a word. The context window is the fixed number of tokens the model can attend to at once (a million, on the current Claude models). Prompt caching lets you pay a reduced rate to re-send an identical prefix instead of reprocessing it. And MCP — Model Context Protocol — is the standard interface an agent uses to call external tools. Keep those four in your head and the rest follows.

Eviction is the most literal answer to a full context: delete the stale stuff. The tool result from twenty turns ago, the file you read and already edited, the thinking block from a reasoning step that's now resolved — drop it out of the live window so the model stops paying to carry it.

The harness can do this for you, and increasingly it does. Claude Code calls this compaction: as a session approaches its context limit, it clears the oldest tool outputs first and only summarizes the rest of the conversation if that isn't enough on its own. Recent tool results stay inline so you can keep reasoning over them; older ones get cleared first.

At the API level there's a more configurable version. Anthropic's context editing feature exposes a strategy with the delightfully machine-generated name clear_tool_uses_20250919. You turn it on with a beta header (anthropic-beta: context-management-2025-06-27) and it watches your accumulating tool results. Once input tokens cross a threshold — the default trigger is 100,000 input tokens — it clears the oldest tool results in chronological order, keeping the most recent few (keep defaults to 3 tool uses).

Here's the detail that matters more than any of the parameters: each cleared result is replaced with placeholder text so the model knows it was removed. The agent doesn't silently lose a tool result and then hallucinate what was in it. It sees a tombstone — "this tool result was cleared" — which is a very different thing from a gap.

The config below overrides the defaults to make the behaviour easy to see — I've set trigger to 30,000 tokens rather than the stock 100,000 so it fires early, and pinned clear_at_least so each pass removes a real chunk. In production you'd leave trigger higher.

context editing config · Messages API
context_management={
    "edits": [
        {
            "type": "clear_tool_uses_20250919",
            "trigger": {"type": "input_tokens", "value": 30000},
            "keep": {"type": "tool_uses", "value": 3},
            "clear_at_least": {"type": "input_tokens", "value": 5000},
            "exclude_tools": ["web_search"],
        }
    ]
}

What it saves

Straightforwardly, input tokens — though the headline figure Anthropic has published for this feature is a task-performance number, not a token count: on an internal agentic-search evaluation, context editing alone lifted performance 29% over baseline, rising to 39% when paired with a memory tool. Hold onto that second number. It's the whole thesis of this post hiding in a benchmark.

What it costs

Two things, and the second is non-obvious. The first is the risk that you evict something the model actually needed — mitigated by the placeholder tombstone and by exclude_tools, which lets you mark, say, your search tool's results as never-clearable. The second cost is about prompt caching, and it's where a lot of naive eviction setups quietly lose money.

Read those two facts together and the tension jumps out. Eviction edits the middle of your conversation. Every time it fires, it changes the prefix, which invalidates the cache from that point forward, which means your next request pays the 1.25× write cost to re-cache the new prefix. Evict a little bit, often, and you can spend more on cache churn than you saved on the evicted tokens. This is exactly why the clear_at_least parameter exists: it forces each clearing pass to remove a worthwhile chunk of tokens so the cache invalidation is amortized against a real saving, not a rounding error. If you take one operational lesson from this whole family, make it that one — evict in big, infrequent passes, never in a trickle.

The demo below lets you feel that trade-off directly. Push the number of eviction passes up and watch the cache-churn bar climb while the token savings stay flat.

When it fails

Eviction fails the moment the model needs something you threw away and can't cheaply get it back. A tombstone that says "tool result cleared" is honest, but honesty doesn't reconstruct the file. If the only copy of that information lived in the evicted tool result, the agent is now stuck: it has to re-run the tool, re-read the file, re-derive the thing. You've converted a token cost into a latency-and-tool-call cost — and if the underlying state has changed in the meantime, possibly into a correctness bug.

If eviction is "throw it away and hope you don't need it," offload-and-recall is "write it down somewhere durable before you throw it away, and fetch it back on demand." The agent, mid-session, notices it has learned something worth keeping — a function signature, a gotcha, a decision, a partial result — and commits it to an external store. Later, instead of carrying that finding in the context window the whole time, it recalls it in a single cheap call exactly when it's relevant.

This is the working-memory pattern, and it's the half of vectr I care about most. When the agent discovers that, say, a workspace lock is acquired at resolver.rs:214 and released on scope exit, it doesn't keep that fact parked in context for forty turns. It stores a note. The note sits in a local store keyed to the workspace, and a recall call pulls it back — in my case in under 50 milliseconds — whenever the agent's current task touches locking.

The reason this is a distinct family, and not just "eviction with extra steps," is what it survives. A finding in the live context window dies three deaths. It costs tokens the entire time it sits there. It gets mangled or dropped when the conversation is compacted into a summary — compaction preserves the gist and loses the exact line number. And it vanishes completely when the session ends. A note in an external store survives all three. It's there after /compact. It's there in tomorrow's session. It costs nothing until you ask for it.

Anthropic's own memory tool works on this principle, and it's the reason for that 39%-versus-29% gap I told you to hold onto. Context editing alone improves performance 29% over baseline on Anthropic's internal agentic-search evaluation. Context editing plus a memory tool gets you 39% — because the agent writes the important bits to memory before the eviction pass clears them, so clearing becomes safe instead of lossy. That extra ten points isn't a second independent optimization stacked on the first. It's the same optimization made safe to run harder, because family 2 backs it up.

What it saves, what it costs

It saves the standing token cost of carrying a finding you only need occasionally. And, less measurably but more importantly, it saves re-derivation — the agent doesn't have to re-read the file and re-reason to the same conclusion next time.

Against that, three real costs. The agent has to decide what's worth remembering, which is a judgment call it will sometimes get wrong — store noise and your recall gets diluted. The recall has to actually be relevant when it fires, which is a retrieval-quality problem in miniature. And there's a token cost to the recall itself, a fact I had to make peace with: store terse one-line notes and recall is cheap but thin; store full code blocks and recall is rich but heavier. There's no free lunch. There's a dial.

When it fails

It fails when recall isn't automatic. If your architecture depends on the model choosing, of its own accord, to call the recall tool at the right moment, it will frequently just… not. The model has no reliable sense of what it stored three sessions ago. The fix is to stop relying on the model's initiative and inject the relevant notes into context deterministically — which, in Claude Code, means hooks, and which is a whole post of its own. The short version: an offload store the agent forgets to read is a filing cabinet in a locked room.

The third family attacks a different waste. The first two are about getting rid of information you already loaded. This one is about never over-loading it in the first place.

The default way an agent explores an unfamiliar codebase is grep-and-read. Grep for a likely keyword, get forty hits, read the six files that look plausible, discard five of them. Every one of those reads lands the entire file in context — a 400-line module of which the agent needed one function. The signal-to-noise ratio is brutal, and unlike a human skimming, the model pays full token price for every line whether it was useful or not.

Retrieval-over-stuffing replaces the blunt read with a targeted fetch. Instead of loading whole files and letting the model sift, you run a ranked retrieval — semantic search over the codebase, ideally chunked at function and class boundaries so each result is a self-contained unit of meaning — and hand back the forty lines that actually match the query. "JWT validation logic" returns the verify_token function directly, even though neither word appears in it, and it returns that function, not the 400-line file it lives in.

This is the search half of vectr, and the payoff on unfamiliar code is large: on a big Java codebase in my own benchmarks, ranked retrieval cut the read-and-grep calls before the first edit by roughly three-quarters compared to the grep-and-read baseline. The mechanism is boring — embeddings plus a keyword index, merged — but the discipline is the point: the unit you put in context should be the unit of meaning, not the unit of storage. A file is a storage unit. A function is a meaning unit.

What it saves, what it costs

It saves the bulk of exploratory token spend, and the turns that go with it. A search that returns the right function in one call replaces a grep-plus-four-reads sequence. In exchange, you need an index — which means an indexing step and the machinery to keep it fresh as files change — and retrieval quality becomes a first-class concern.

When it fails

It fails on questions retrieval is the wrong tool for. "Who calls this function?" is not a similarity question — the callers don't contain the callee's body, they contain a reference to it by name. That's a graph traversal, not a search. Reach for semantic retrieval there and you'll get plausible-looking garbage. Part of doing this family well is knowing which questions are retrieval questions (concepts, patterns, "how does X work") and which are structural ones (definitions, call graphs) that want an exact lookup instead.

The fourth family is the cleverest and the easiest to get subtly wrong. The idea: when a subtask is going to generate a pile of context you'll never reference again — a research spike, a log-diving expedition, a broad search — you don't do it in your main conversation. You spawn a subagent, let it do the messy work in its own context window, and take back only the distilled answer.

Claude Code's subagents work exactly this way. Each one runs in its own context window with a custom system prompt, does its work independently, and returns only the result — the docs frame it as keeping exploration and implementation out of your main conversation. The parent agent spends, say, 800 tokens receiving a clean summary of an investigation that cost the subagent 40,000 tokens of reading and reasoning. Those 40,000 tokens are burned in a window that gets discarded. The parent's context stays clean.

There's a nice secondary benefit. Because a subagent has its own tool permissions and its own system prompt, you can also use it to constrain work — a read-only research agent that literally cannot write files — and to route cheap work to a cheaper, faster model. Context isolation and cost control fall out of the same mechanism.

What it saves, what it costs

It saves the largest single chunk of exploratory context there is. A well-scoped subagent is the difference between your main window holding a conclusion and holding the entire messy derivation of that conclusion. The cost is a framing-and-parsing tax at the boundary: you have to specify the subtask well enough that the subagent can run without hand-holding, and you have to trust the summary it returns without seeing its work. If the summary is lossy in exactly the way that matters, the parent proceeds on a bad abstraction — and it can't tell, because the detail that would have flagged the problem got left behind in the discarded window.

When it fails

Here's the failure mode nobody warns you about. Subagent isolation with no shared memory means every subagent starts cold. It re-derives context the parent already had and the last subagent already found. Spawn three subagents to investigate three corners of the same system and, without a shared store between them, each one re-reads the same core files, re-learns the same architecture, and re-discovers the same gotcha — three times, in three separate windows, at full price each. You've isolated the context so well that you've also isolated the learning. The isolation that saves the parent's window quietly taxes every child.

I've been dropping the composition hints deliberately, so let me make them explicit, because treating these four as a menu you pick one item from is the mistake I most want to talk you out of.

Eviction (1) is only safe on top of offload (2) or retrieval (3). This is the load-bearing relationship. You can only throw information away cheaply if you can get it back cheaply — and "getting it back cheaply" is precisely what families 2 and 3 provide. Evict a tool result whose contents you already wrote to a memory note: safe, because recall restores it. Evict a file you can re-fetch with one targeted search: safe, because retrieval restores it. Evict something that exists nowhere else and you've just planted a bug that will surface three turns later as confident nonsense. The 29% → 39% jump from adding a memory tool to context editing is this relationship, quantified — the memory tool is what makes the eviction safe to run harder.

Retrieval (3) keeps the working set small enough that eviction (1) rarely has to fire. If you never stuffed the whole file in, there's less to evict later. The two attack the same waste from opposite ends — one at load time, one at cleanup time — and a system with good retrieval needs less aggressive eviction.

Subagents (4) need a shared store (2) or they re-derive context. Covered above, but it's the composition people skip most often, because subagents feel self-contained. They're self-contained in their context, not in their knowledge. Wire them to the same working-memory store and the isolation stops being a re-derivation tax.

The demo below lets you toggle the families on and off over a simulated forty-turn session and watch what breaks. Turn on eviction alone and note the unsafe-eviction count. Add offload and watch it drop to zero. Add a second and third subagent with offload off, and watch the re-derivation tax appear.

The unifying idea is almost embarrassingly simple once you see it: cheap restoration is the license to be aggressive about relief. Every family is either a way to remove context (1, 4) or a way to make removal safe by guaranteeing you can get the important parts back (2, 3). Build only the removal half and you get an agent that forgets things it needed. Build only the restoration half and you get an agent that never frees anything and grinds to a halt at the context limit. You need the pair.

This is also why I stopped thinking of vectr as "a search tool" or "a memory tool." It's families 2 and 3 in one MCP server, deliberately, because on their own each is half a solution. Search without memory re-explores every session. Memory without search has nothing good to store. And both of them exist, in the end, to make the harness's eviction — family 1, which I don't even own — safe to run.

If you operate a coding agent and want to actually apply this, here's the compressed version I'd give a colleague over coffee.

Start with retrieval (3), because it's the one that prevents the mess instead of cleaning it up, and it pays off immediately on any codebase you don't have memorized. Add offload-and-recall (2) next, and make the recall automatic rather than something the model has to remember to do — a store the agent forgets to read is worthless. Let the harness handle eviction (1), but check that it's evicting in big infrequent passes (mind the cache-invalidation math) and that everything it evicts is backed by 2 or 3. Reach for subagent isolation (4) on genuinely large exploratory subtasks, and if you use more than one subagent on related work, give them a shared memory bus or accept that each is paying full freight to learn what the last one already knew.

None of these is exotic. The compaction and context-editing pieces ship in the tools already. The retrieval and memory pieces are a weekend to prototype — I wrote up the honest numbers on how far mine actually got if you want the unvarnished version. What's rare is treating them as one system with a single governing rule — restore-ability licenses removal — instead of four disconnected tricks. Get the rule right and the context window stops being the thing you fight and starts being the thing you manage.