A Guess Becomes a Fact on the Next Read
A user instruction, a tool result, a file fact, and the model's own guess end up as the same type once they leave the context window. Forgetting costs a re-derivation. Laundering corrupts every decision that trusts the result.
Everything an agent knows arrives as text in a context window. A user instruction is text. A tool result is text. A file the agent read is text. The agent's own speculation, written out mid-reasoning, is text. By the time any of it reaches the next turn, it is all the same type.
That flattening is invisible while the conversation is live, because position and recency stand in for provenance. The user's instruction is the thing at the top. The guess is the thing the model just wrote. Nobody needs a label because the structure carries it.
Memory removes the structure. A note is retrieved into a fresh context with no conversation around it, no position, no ordering, no author. What was a hedged inference at write time arrives at read time as a free-standing assertion, indistinguishable from something the user said outright.
The claim of this post: the dominant failure mode of agent memory is not forgetting, it is laundering. A lost note costs a re-derivation, which is bounded and usually cheap. A note that silently changed epistemic status costs every decision that trusts it, and it leaves no trace of having changed. Systems built to maximize recall are optimizing the wrong quantity, because the expensive error is not the note that failed to come back. It is the note that came back wearing a confidence it never had.
Four Things That Are the Same Type and Should Not Be
Consider four statements that could all end up in an agent's memory store, written during the same session.
- The user said the deploy script must never run on a Friday.
- The test suite emitted
FAILED tests/test_auth.py::test_expiry. - The agent read
config.yamland it containedtimeout: 30. - The agent reasoned that the timeout is probably too low for the CI runner, since the failing test waits on a network call.
The first is a normative instruction from a human. It cannot be false; it can only be obeyed or violated. The second is an observation, true of one run at one commit. The third is a fact about a file at a moment. The fourth is a hypothesis, and a decent one, but it is the agent's own guess and it was never checked.
Write all four as notes and they become the same kind of object: a string with a timestamp. Retrieve them together later and the fourth reads exactly like the first three. Worse, the fourth is often the most useful-sounding, because it is the only one phrased as an explanation. Explanations are what a retrieval query tends to be reaching for.
The problem is not that the fourth note is wrong. It might be right. The problem is that nothing in the system records that it is the kind of thing that could be wrong, while the other three are not, and they are not wrong in three different ways.
The four statements have different truth conditions, different ways of being wrong, and different correct responses on being contradicted. A human assertion that conflicts with an observation usually means the instruction was normative, not descriptive, and the right move is to ask. A tool observation that conflicts with a later observation means the world changed, and the right move is to re-run. A model conclusion that conflicts with anything is wrong, and the right move is to drop it. Collapse the three into one type and there is exactly one repair strategy, which is whatever the model guesses. That guess is itself unlabeled, and now the corruption is recursive.
Provenance is not a new idea in computing. The W3C PROV-DM standard from 2013 has spent more than a decade defining how to record who or what produced a piece of data, why, and from which inputs. Scientific workflow systems like Taverna and Kepler ship lineage-aware stores that track derivation chains across tool calls. The contribution of this post is not the discovery that provenance matters. It is the narrower claim: in agent memory, the failure that dominates is not the one provenance was originally designed to address, which is trust assessment. It is type destruction, which happens earlier and further upstream, at the boundary between context and store.
Containers Carry Provenance, Content Does Not
Inside a live context the four statements are separable, and the model separates them well. It knows the user spoke because the user's turn is a user turn. It knows the test output is tool output because it sits in a tool result block. It knows its own guess is a guess because it wrote the word "probably" three lines ago and can still see it.
None of that survives extraction. Every one of those signals is carried by the container, not by the content, and memory stores content.
When a memory tool writes a string to a file, the only thing in the file is the string. The fact that the string was the user's instruction rather than the model's guess is in the role it played in the conversation, and the role is gone. The fact that the string was a tool observation rather than a hypothesis is in the block it sat inside, and the block is gone. The fact that the string was a hedged guess is in a word that any downstream system is free to keep, drop, paraphrase, or replace.
Imagine a courtroom record reduced to a list of every sentence anyone said during the trial, with no speaker labels, no timestamps, and no markers for which sentences were testimony, which were objections, which were the judge's rulings. A future reader, given only that list, has to guess whether any particular sentence was a fact a witness swore to or a lawyer's speculation. The guess is the only move available, and the guess is wrong more often than not, because the speakers were not equivalent. The transcript is fine. The record has lost the structure that made it usable.
This is why the failure is so easy to miss in testing. Evaluate memory inside a single session and provenance appears to work, because the context still holds the structure that encodes it. The corruption only shows up across the boundary that memory exists to cross, which is exactly where it is least likely to be measured. The boundary is also the only place the system gets to demonstrate it is doing something useful, so it is the place evaluation spends the least time.
There is a second reason it hides. When a laundered note is later acted on, the action usually succeeds or fails for ordinary reasons, and the failure gets attributed to the action. Nobody traces back to ask whether the belief that motivated it was ever load-bearing. The causal chain from a bad epistemic label to a bad outcome is long, and every link in it looks like a normal engineering problem. The end failure is a missed deadline, a wrong commit, a botched deploy, and the postmortem names the tool, not the memory.
Forgetting and Laundering Are Not Symmetric
Treat memory as a channel with two error modes.
The first is loss. A note that should have come back did not. The cost is a re-derivation: the agent reads the file again, runs the test again, asks the user again. Annoying, measurable, and bounded by the cost of the original derivation. Crucially, the agent knows it does not know. It is in a state where it will go and find out.
The second is laundering. A note comes back, and it carries more authority than it earned. The cost is not bounded by anything, because the agent is no longer in a state where it will go and find out. It has an answer. Every downstream decision inherits the error, and the error is silent by construction: there is no exception, no miss, no empty result, nothing that looks like a failure at the moment it occurs.
Loss is a missing note. The agent has no information, knows it, and pays a bounded price to obtain the information. Laundering is a mislabelled note. The agent has a confident answer to something it should have hedged, builds on it, and never trips a signal that says the answer was wrong. Loss is loud and finite. Laundering is silent and unbounded, because the only cost is whatever the downstream decisions multiply it by.
These two errors are not symmetric and should not be traded off against each other as if they were. Yet almost every design decision in a memory system trades exactly that way. Store more, and you launder more. Retrieve more aggressively, and you launder more. Summarize to save tokens, and you launder more, because summarization strips hedges first, and the next section is about why.
An agent that remembers nothing is merely slow. An agent that remembers its own guesses as facts is confidently wrong, and it will defend the position, because from the inside a laundered note is indistinguishable from a thing it was told.
The Compression Tax on Uncertainty
There is a specific, mechanical reason provenance decays even when someone tries to preserve it in the text.
Epistemic status in natural language is carried by small, low-content words. "Probably." "I think." "It looks like." "Assuming the runner is network-bound." These words have almost no semantic weight. They contribute little to an embedding. They are the first casualties of any rewrite that optimizes for brevity or clarity.
So the moment a note passes through a summarizer, a compaction step, or an agent rewriting its own notes to be tidier, the hedges go and the claim stays. The sentence gets shorter and more confident at the same time, and the two are the same edit. Nobody decided to promote the guess. The promotion is a side effect of compression.
Run a Hedged Note Through a Summarizer
Pick a note with epistemic markers and watch the markers survive or vanish as a compressor rewrites it for brevity. The same note can pass through a hedge-preserving compressor and a hedge-stripping one, and the output is a different epistemic object even though the words are nearly the same. This is the mechanism that makes laundering free, and it does not require an adversary.
The published work on this drift is more recent than the intuition, and more measured than the intuition would predict. Belem, Wu, Yao, Steyvers, Singh, and Smyth, From "May" to "Is": Certainty Distortion in Language Model Rewriting (arXiv:2606.07951, June 2026), report a systematic tendency for LMs to overstate confidence when paraphrasing scientific and medical text. Their evaluation covers single rewrites, repeated paraphrases, and prompt-level mitigations, and the effect compounds over iterations: in their medical subset, one pass raises certainty in 20% of examples, five passes in 40%. The implication for memory stores is that the longer a note lives, and the more rewrite passes it sees, the more it tends to wear a confidence the original never had.
This is why "just write good notes" does not solve it. The instruction assumes the note is stable once written. It is not. It is a string that will be passed through several lossy transformations by systems that have no reason to know which of its words are the important ones.
If provenance is in the prose, provenance is negotiable. It has to be in the schema.
A Closed Enum, Not a Source String
The fix is not a source string. A free-text source field is prose again, and it will be paraphrased away like everything else.
What is needed is a small closed set of classes, assigned at write time, that the retrieval path is required to render and the model is trained by the interface to read. Small, because a taxonomy nobody can remember is a taxonomy nobody applies correctly. Closed, because an open set collapses back into prose.
The minimum viable distinction is between what a human asserted, what a tool emitted, and what the model concluded. Those three have genuinely different truth conditions, different ways of being wrong, and different correct responses on being contradicted.
| Provenance class | Truth condition | How it is wrong | Correct response on conflict |
|---|---|---|---|
| human | Reflects what the user said or endorsed | Misquoted, transcribed, or out of date | Ask. A descriptive conflict may be normative intent. |
| tool | Reflects an observation at a moment | World changed, tool was nondeterministic, or anchor drifted | Re-run. The world is the source of truth. |
| model | Reflects the model's own inference | The inference was wrong | Drop. The conclusion is what is wrong. |
There is a further refinement worth naming, because it is where this gets genuinely hard. The middle case is a claim a human made that an agent transcribed, perhaps compressed, perhaps subtly shifted in the paraphrase. It is not a model conclusion, because a human is behind it. It is not a verbatim human assertion either, because the exact words are the agent's. Systems that offer only two classes force this case into the wrong one, and it is a common case, arguably the most common one in practice, because almost every "the user said X" note in any real store was typed by the agent that heard it.
A source="user said in turn 14" column is the same as no source column, because the value is prose, and the retrieval path that does not know what to do with prose is exactly the retrieval path that needs the source. A closed enum, by contrast, is rendered on every recall. The reading model does not have to interpret it. It reads the prefix the way it reads the kind field on a tool call, and that is the entire mechanism. Open taxonomies collapse back to prose, and prose collapses back to nothing.
A note on the relationship to a prior post. An earlier piece, Agent Memory Needs a Trust Ladder, is about how much weight to give a stored note. It presupposes the system already knows what kind of note it is holding: human, agent-recorded, or auto-captured. That prior work is about the trust axis. This post is about the kind axis. The two are independent, and conflating them is a common error. A trust ladder on a typeless memory is the downstream problem: the model is being asked how much to believe a claim whose type was destroyed before the question was asked. A separate but adjacent post, Agents Over-Trust Instructions and Under-Trust Corrections, makes a related point about how instructions and corrections are weighted asymmetrically by the reading model. That post is also downstream of the type question. Both are right about their own axis, and both assume the type is known. This post is the layer underneath both.
The published literature on this kind of labelling is mostly recent. Wang et al., From Agent Traces to Trust: A Survey of Evidence Tracing and Execution Provenance in LLM Agents (arXiv:2606.04990, June 2026), explicitly groups memory writes by source type, recommending that stores carry source, authoring agent, transformation operation, confidence, and update history on every memory. The taxonomy there is richer than the three-class minimum above, and a system implementing the richer version is in better shape. What the survey does not separate cleanly is the failure mode this post is naming, which is the case where the source field exists in the schema but the retrieval path does not actually surface it to the reading model. That is a delivery problem, and it has the same shape as every other delivery problem in agent memory: the artifact is in the store and never reaches the prompt.
Information Destroyed at Write Time
A tempting shortcut: skip the write-time labeling, and have the model assess provenance when it reads a note. It is good at that sort of judgment, in some situations.
It cannot work, and the reason is worth being precise about.
Provenance is not a property of the note's text. It is a property of the note's history. Two notes with byte-identical content can have different provenance, because one was dictated by a user and the other was inferred by a model that happened to phrase it the same way. No amount of reading the string recovers which happened. The information is not in the string; it was destroyed at write time, and read-time analysis has nothing to recover it from.
Two Notes, Identical Strings, Different Histories
Two notes in the store, byte-identical in their content. One was the user speaking in a real conversation. The other was the model inferring a guess and writing it down with a phrasing it happened to share. A read-time judge sees the same text in both cases. Toggle the framing, then toggle a read-time inference on, and watch how the inference produces a plausible-sounding answer that is the original failure, one level up.
What read-time assessment actually produces is a plausible-sounding guess about provenance, stored or acted on as if it were provenance. Which is the original failure, one level up.
When information is destroyed at write time, no read-time cleverness restores it. It can only manufacture a confident substitute. The fix has to be at the boundary where the information still exists. This is not a special case. It is the standard shape of a class of memory bugs, and naming it once means the rest of the family becomes recognizable: write-time compression that loses hedging, write-time rewriting that flattens a model conclusion into a "fact", write-time summarization that drops the actor on a retraction. Each of them is the same bug in a different surface.
That last sentence is the one to keep. The mechanism in this post is not the only place agent memory destroys information it cannot recover. Every summarization pass, every rewrite-for-clarity, every compaction step is the same shape: a write-time operation that throws away the part of the string the system cannot yet use, on the bet that the part it kept will be enough. The bet is wrong, but it is wrong silently, which is what makes the bug shape worth recognizing.
What This Forbids, and the Test That Would Break It
An argument that explains everything and forbids nothing is not worth much. Here is what this one forbids.
Prediction 1. In a memory system without provenance classes, the proportion of stored notes that are model conclusions rather than observations should rise over time, and rise faster in longer sessions. Conclusions are cheap to generate and feel insightful, so an agent asked to record what it learned will preferentially record what it thought. This is measurable by classifying an existing store retrospectively. The slope should be positive and significantly larger than the slope for human-asserted or tool-observed notes.
Prediction 2. Notes that have been through a summarization or compaction step should be measurably more confident in phrasing than their originals, with no corresponding increase in support. Diff the hedge tokens before and after. The published work already gives a baseline: Belem et al. find certainty distortion in up to 75% of LM rewrites, with a 1.5 to 2 times asymmetry toward inflation. If a memory system shows flat hedge rates through rewriting, either the system is suppressing the drift by accident or the published result does not generalise; either outcome would weaken the argument.
Prediction 3. Given a memory store containing a known-wrong model conclusion, an agent should be more likely to defend and build on it than to re-derive it, compared to a control where the same claim is labeled as a guess. If labeling makes no behavioral difference, the schema fix is not worth building, and this whole post is an aesthetic preference rather than an engineering claim.
Everything in this post assumes the reading model actually changes behavior in response to an epistemic label. That assumption deserves a direct test rather than an appeal to plausibility, and it is exactly the kind of thing that could come back negative. The published literature on verbalized calibration, in particular Tian et al. 2023, has been pretty clear that LMs are poorly calibrated about their own reasoning: they overclaim correctness, underclaim uncertainty, and the calibration error is large enough that a self-reported confidence stamp on a recalled note may add noise rather than signal. If that is true, the labelling scheme has to be enforced at the rendering layer, not the reading layer, and the schema is the right place to enforce it. The test that would settle it is the third prediction above, run with a real store and a real agent.
Two things the argument does not claim. It does not claim that labelling alone makes an agent correct. A labelled-and-laundered note is still wrong, just labelled as wrong with the wrong label. It does not claim that provenance classes are a security mechanism. A malicious writer can attach a human stamp to anything it can write, and a closed enum is no defense against a hostile actor with direct store access. The argument is about accidental corruption, not adversarial input, and treating the two as one would be a category mistake.
An agent that forgets is slow. An agent that cannot tell what it was told from what it guessed is confidently wrong, and it will defend the position. Forgetting is safe. Laundering is not.
The fix is small. A closed enum, stamped at write, rendered on every recall, with the summarization pipeline forbidden from rewriting it. Three classes are the minimum. The argument does not need anything else. The argument does need that one thing, because without it every note the system stores is a free-floating claim with no information about how it came to exist, and the most plausible-sounding notes are the ones that came from the worst path.
Sources
- W3C Working Group. PROV-DM: The PROV Data Model.
- Wang, Y., Zhang, J., Cai, T., Liu, Z., Sun, Q., Sun, Z., Wu, Z., Dong, M., Zheng, M., Yin, X., Zhu, Y. From Agent Traces to Trust: A Survey of Evidence Tracing and Execution Provenance in LLM Agents.
- Belem, C. G., Wu, S., Yao, H., Steyvers, M., Singh, S., Smyth, P. From "May" to "Is": Certainty Distortion in Language Model Rewriting.
- Chhikara, P., Khant, D., Aryan, S., Singh, T., Yadav, D. Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memory.
- Packer, C., Wooders, S., Lin, K., Fang, V., Patil, S. G., Stoica, I., Gonzalez, J. E. MemGPT: Towards LLMs as Operating Systems.
- Tian, K., Mitchell, E., Zhou, A., Sharma, A., Rafailov, R., Yao, H., Finn, C., Manning, C. D. Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedback.