The Memory Legibility Problem: Why a Text File Still Beats My Own Product

I want to take the flat file seriously rather than build a strawman of it, because the strawman version of this post is easy to write and wrong.

A MEMORY.md at twenty or thirty notes is, honestly, close to ideal. Every property a memory system wants, it gets for free from being a text file:

• It never misses. There's no ranking step, so there's no way for a relevant fact to fail to surface. If it's in the file, the agent sees it, every single time, because the whole file goes into context.
• It's diffable. Version control already understands it. You can see exactly what a session added in a normal git diff, with no bespoke tooling.
• It's reviewable. A human can read the whole thing in a few minutes and form an opinion about which lines are still true.
• It's editable in place. Wrong line, delete it. Stale line, delete it. No indirection between "I know this is wrong" and "it's fixed."
• It has zero retrieval latency and zero retrieval failure modes, because it has no retrieval step at all.

None of that is a coincidence, and none of it is something a semantic store gets automatically. It's what falls out of choosing the simplest possible representation: one file, fully injected, always. A lot of memory-system pitches, mine included at various points, implicitly promise to be better than this without being honest that they're trading away properties to get there. So before I talk about what I built, I want it on record that the thing it's competing with isn't a strawman. At small scale, it's close to correct, as a real engineering claim, not a rhetorical concession.

The flat file's failure mode isn't subtle, and it isn't really about the file at all. It's about the size of the context window.

At twenty notes, MEMORY.md costs you a few hundred tokens per turn and buys you a guarantee: nothing gets missed. That trade is obviously worth it. At 749 notes, which is where my own store actually sits, the same design would mean injecting the entire history of every finding, gotcha, decision, and task checkpoint the agent has ever recorded into every single turn, forever. Most of those 749 notes have nothing to do with whatever the agent is doing right now. You'd be spending a large, fixed slice of every context window on notes that are overwhelmingly irrelevant to the current task, which is precisely the problem the memory system exists to solve in the first place.

Injecting the full note history into every turn isn't a retrieval strategy. It's the absence of one. It only looks free while the corpus is small enough that "everything" and "the relevant subset" are roughly the same set. The moment those two sets diverge, which happens well before 749 notes, you're not choosing between a retrieval system and no retrieval system. You're choosing between a retrieval system that ranks and a retrieval system that injects garbage, because injecting all 749 notes at once is not meaningfully different from injecting none of them: the agent can't usefully attend to either.

So the scale argument isn't "semantic search is smarter." It's narrower and less flattering than that: past a certain corpus size, a flat file's core guarantee, that nothing is ever missed, becomes unaffordable to keep, and you're forced to accept some form of ranking whether you like its failure modes or not. The question that actually matters is what you do about the failure modes you just accepted. That's the rest of this post.

I want to be precise about what I mean by "silent miss," because it's easy to conflate with "bad ranking," and the two aren't the same failure and don't deserve the same response.

A loud miss is one the agent can react to. It calls search, gets zero results, or gets results it can visibly tell are off-topic. That's an unpleasant outcome, but it's a legible one: the agent knows something didn't work and can fall back, ask, or search again with different terms.

A silent miss is different in kind, not degree. The right note exists. It's correct. It's exactly what the current task needs. And it simply doesn't appear in the results, because it ranked below the cutoff, or because the query's embedding happened not to land near it, or because the top-k the caller requested was too shallow to reach it. Nothing about the response signals that this happened. The agent receives a plausible, complete-looking answer and has no way to know a better one existed one rank lower. It proceeds with confidence, missing a fact it already spent effort recording.

This is, I'd argue, the worst failure mode a retrieval system can have, worse than returning nothing and worse than returning something obviously wrong, precisely because it's invisible in every metric people usually track. A dashboard of "average relevance score" or "top-1 accuracy on a benchmark set" won't surface it, because those metrics are computed over queries where you already know the right answer. A silent miss in production is a case nobody is watching, by definition, because if you were watching it you'd have caught it and it wouldn't be silent anymore.

And cutoff and embedding distance aren't the only way a note goes missing. A note can also be excluded before ranking ever runs: filtered out by kind when the query implicitly wants a different one, scoped to the wrong workspace, or dropped because it's stale_flagged and the caller's default view skips flagged notes. That failure looks identical from the outside, a plausible answer with a gap in it, but the fix is different: a ranking tweak does nothing for a note the ranker never saw in the first place.

The same failure shape shows up on the code-search side of this project, not just the memory side. A correct answer can sit in the candidate pool and still never reach the top of the list, for two genuinely different reasons: sometimes the right chunk never enters the candidate set at all, because its embedding sits too far from the query's; sometimes it enters the pool fine but the ranking pass places something more superficially similar above it. Those are different bugs needing different fixes, and conflating them is how you patch the wrong stage. Both are silent unless someone deliberately goes and checks, and almost nobody deliberately goes and checks.

Here's the part of this post that argues against my own product hardest, and it isn't about ranking at all. It's about whether the agent calls the tool in the first place.

In a controlled benchmark arm, I ran an AI coding agent on a real, non-trivial task: a cross-cutting fix in a large existing codebase, the kind of task that plausibly benefits from recalling earlier findings. The memory server was connected over MCP. The workspace's instructions file mentioned the memory tools 32 times: when to store a finding, when to recall before starting work, what each memory kind is for. The harness verified per run that the guidance file was actually in place at launch and that all ten memory tools were in the model's direct tool list, so this isn't the boring story where the configuration silently failed to load.

Across the whole session, the agent made zero calls to any memory tool.

It solved the task, and solved it well, by grep and by reading files directly, the way an agent would work with no memory system installed at all. The instructions were there. The tools were connected. The model simply didn't reach for them, and nothing in a 63-turn, multi-compaction session ever prompted it to.

Now the part I have to include, because leaving it out would make this section exactly the kind of cherry-picked evidence the post is arguing against. That zero did not replicate. I reran the same arm under a hardened harness that proves the configuration per run, and I have a third run from a sibling arm on the same daemon. The three came out like this:

So "the agent never calls the memory server" isn't the finding, and I'd be misrepresenting my own data if I let the dramatic single run stand as one. Lookup-style calls are stochastic, swinging from 0 to 10 across runs under an identical configuration.

What is stable across all three runs is narrower, and I think considerably more damning. Count the writes: of the 14 memory calls made across three sessions, exactly one stored anything. Notes stored were 0, then 1, then 0. The agent will occasionally reach for the store to look something up, on a coin flip. It essentially never puts anything in. And a store nothing gets written to has nothing to retrieve next session, which is the entire value proposition failing quietly, in a way no error message anywhere will ever tell you about.

The honest headline is therefore not "agents ignore the memory server." It is "agents do not voluntarily adopt the memory workflow." That version survives replication. The eye-catching one doesn't.

This is the finding that reframes the rest of the post. The fix for a silent miss inside the store and the fix for a silent miss because the store was never queried turn out to be the same fix: stop asking the model to opt in.

You cannot instruct your way to reliability. I'd already learned that the hard way for a different piece of this system, getting an agent to call recall mid-session reliably needed lifecycle hooks, not a stronger instruction, and the adoption finding above is the same lesson from a different angle. The fix has to live in the harness, not in the model's willingness to comply. Lifecycle hooks let the store push notes into context on a schedule the harness controls, rather than waiting to be asked.

The part that actually matters for the legibility argument is that not every note gets the same treatment. Different memory kinds get different default trigger bundles, and the defaults aren't cosmetic: they're a deliberate ladder from "must never miss" down to "cheap to miss occasionally."

Ranking, with its attendant risk of a silent miss, is reserved for the part of memory where a miss is a real but bounded cost: a finding that would have saved a re-read, not a rule whose violation breaks something. The mistake I actively avoided here is treating all memory as one homogeneous pool to rank uniformly. It isn't homogeneous. The 43 active directives in my corpus can afford the flat file's cost, so they get the flat file's guarantee. The other 706-odd notes can't afford that cost, but the cost of missing one of them is a re-read, not a broken build, so they get the ranked index's risk instead.

None of this touches the adoption half of the finding above, though, and I want to be honest about that split. Hooks fix "the agent forgot to call recall." They don't, by themselves, fix "the note that would have helped ranked below the cutoff." That's the next problem, and it's the one I have made the least progress on.

The reason silent misses persist as a class of bug is structural: nobody counts them, because counting them requires knowing, independently of the retrieval system, what the right answer was for a given query. That's expensive to build and easy to skip, which is exactly why it gets skipped.

I don't have a finished answer here, and I'd rather say that plainly than paper over it. What I do have is the conviction, earned from watching the code-search side of this project for a while now, that you cannot improve a number you have never computed. A recall-miss floor, a ground-truth set of query and note-that-should-have-surfaced pairs, checked against what the ranker actually returns, needs to exist as a measurement harness before any tuning decision on top of it means anything. Tuning a threshold, a top-k, an embedding model, against vibes instead of a measured miss rate is how you end up with a system that feels better in a demo and isn't actually better, because the demo's queries were never the ones that were failing.

This work is in flight and genuinely unfinished. I'm flagging it here rather than describing a solution I don't have, because the alternative, implying the miss-rate problem is solved when it isn't, is the exact kind of quiet dishonesty this post is trying not to commit.

This is the direct answer to the legibility complaint from the first section, and it's the one fix in this post I no longer have to describe as a plan. I want to walk through it in two stages, in the order they actually happened: one that shipped, and one I still haven't solved.

Stage one: a read-only export mirror, shipped

The store now continuously projects its current state to a human-readable file on disk. cat works again. grep works again. Code review and version control both work on your agent's memory again, the same way they work on a MEMORY.md, because the projection is a MEMORY.md-shaped file. Writes still only ever go through the store; the file is a view, not a second source of truth.

The command is:

CLI
vectr memory export [--path FILE] [--workspace DIR]

It renders a workspace's working-memory notes to a read-only markdown file, MEMORY.md by default. It merged to vectr's main branch as commit bdfafd2 on 2026-08-18. vectr is a public repo, so the source is checkable: the render lives in agent/working_context_store/_export.py, reading from a notes_for_export() method on the store.

The rendered file groups notes by kind into a fixed set of sections in a fixed order, and within a section, notes are ordered by note_id ascending, roughly like this:

illustrative shape, not literal output
## Directives
- [#4] 2026-06-02 · Run tests inside the venv, not the global interpreter.

## Findings
- [#398] 2026-07-27 · Index rebuild reuses the embed cache on unchanged files.
- [#412] 2026-08-01 · REVOKED 2026-08-09 (wrong lock order) · workspace lock
  must be released before daemon restart.

That revoked line staying in the file is the point, not an oversight. Export renders every note, including superseded and revoked ones, deliberately unfiltered, unlike the ranked recall() path. The file is an audit log of what memory learned, not a ranked view of what's relevant right now.

That determinism is what makes the file useful rather than decorative. Ordering by a stable id and using absolute dates instead of "3 days ago" means re-exporting an unchanged corpus is byte-identical, which is what makes git diff of the file meaningful instead of noise. A diff between two commits of MEMORY.md tells you exactly which notes changed between them, the same way a diff of a hand-edited file would, because nothing about the rendering itself introduces churn. All of these numbers are from the one corpus I actually have, 749 notes. I haven't measured render time or file size at ten times that, and I'd expect both to grow roughly linearly, not something worse, but that's an expectation, not a measurement.

I checked what reads the file back, because the whole point of calling this a mirror rather than a second copy of the data depends on the answer being nothing. I grepped every call site of the new export functions: exactly two, both write-side, both callers of the render function after a note write. Zero parse-back paths. The database stays the sole source of truth, and the export is lossy on purpose: embedding vectors, the full event history, anchor content hashes, trigger conditions, and provenance class don't round-trip through markdown text. They were never meant to. If the projection is ever wrong, you delete it and regenerate it from the database, which is the whole reason keeping it read-only is safe.

Once you opt a workspace into it, a debounced post-write hook keeps the file current without anyone calling export by hand. Every note-write path in the store already funnels through one choke point, _bump_notes_epoch() in app/service.py, so that single function doubles as the hook: it schedules a re-render after memory_export.debounce_seconds of quiet, three seconds by default, and a burst of writes in that window collapses into one re-render instead of one per note. That also means the file can trail the database by up to the debounce window: if the process exits before a scheduled re-render fires, the mirror is stale until the next write retriggers it. It always catches back up on the next write; it just isn't a live, transactional view.

The guarantee that shipped almost unguarded

Here's the part of building this I think is more interesting than the feature itself, because it's a lesson about testing a determinism claim specifically, not about memory systems. When the export mirror landed, its full test suite passed green: 26 tests, all green. And the guarantee the entire feature exists to buy, that re-exporting an unchanged corpus produces byte-identical output, was completely unguarded. I know because I broke it on purpose: I changed the source query's ORDER BY to something else and reran the suite. All 26 tests stayed green.

The real failure this left open: a query without a total-order tiebreak leaves ties in whatever order the database implementation happens to return them, which looks stable today and can silently change after a schema migration or a maintenance operation touches the table. Nothing in the code would notice. The next export would come out with rows in a different order, every line in a git diff would show up as changed even though not one note actually changed, and the "diffable" property this whole fix exists to buy would be dead. Quietly. Exactly the failure mode this entire post is about, just moved one level down into the projection itself. I fixed it by making note_id an explicit second sort key in the export query, and wrote two tests that pin ordering within a section instead of across sections. If you take one thing from this subsection: diffable is a guarantee with a real engineering cost behind it, not a free consequence of writing to a file.

Stage two: transactional check-out and check-in, not built

This is the harder, more interesting half, and I don't have it solved. Something like vectr memory edit would check the projection out, let you edit it in your own editor exactly the way you'd edit MEMORY.md today, deleting a stale line, fixing a wrong one, and check it back in.

The constraint that makes this hard, and the one I refuse to compromise on, is that the check-in has to be an append of events, not an overwrite. The store's note lifecycle is append-only by design: a note's current state is never stored directly, it's reconstructed by replaying a log of events in order, created, superseded, revoked, stale_flagged, reinstated, promoted, expired, so any transition can be undone by appending one more event rather than rewriting history. vectr_forget is the one deliberate hard-delete escape hatch; everything else is reversible by construction. A file check-in that overwrote a note in place, erasing its history along with its line, would break that invariant the moment a person, rather than the store's own tools, made the edit. So a deleted line has to become a revocation event carrying a reason, not a silent disappearance, and a changed line has to become a supersession with the old content retained, not erased. The file's ergonomics only stay compatible with the state machine's audit trail if the check-in path translates "this line looks different now" into "here is the event that explains why," rather than diffing bytes.

I want to be honest about the part I have not solved, because a projection is a cache, and caches have coherence problems whether or not you want them to. What happens if the daemon writes a new note from a live session while you have the file checked out for editing? What's the conflict resolution when the file and the store disagree at check-in time, because both changed since the checkout? Does a check-out lock the store against concurrent writes, which would be simple and also wrong for a system whose whole premise is that multiple agents and sessions share one memory bus? I don't have clean answers to any of these yet. I'd rather name them as open problems than describe a design that quietly assumes they don't exist.

None of this closes the ranking half of the problem from sections three and four above, and I want to be clear about that boundary. The export mirror fixes legibility for a human reading a file, or for git diff, not the silent miss inside recall() itself. That's still the next fix's job, and it's still only a partial one.

The last piece is the one that changes what "recall" returns by default, and it's the closest thing I have to a middle path between "inject everything" and "inject only what a similarity score happened to pick."

Recall returns an index tier by default: one line per note, its id, its kind and priority, its title, its age. Something like:

recall() index tier
[#412] gotcha/high · workspace lock released
       before daemon restart · 3d
[#398] finding/medium · index rebuild reuses
       the embed cache on unchanged files · 9d
[#276] decision/medium · note ranking is now
       trust-then-relevance, not relevance alone · 41d

Full bodies only come back when you ask for them: by passing a specific note id to expand, or by asking for full detail across a query's matches. The tradeoff this buys is real and, I think, underrated: the agent gets to see that a note exists for close to nothing, a line of text instead of a paragraph, and then decides whether it's worth reading in full. A miss at the index tier is a miss the agent could plausibly have caught by scanning fifteen titles, which is a much closer experience to skimming a MEMORY.md than a single opaque similarity-ranked result ever is.

I want to frame this precisely, because it would be easy to overclaim it: this doesn't eliminate the silent-miss problem from section three. A note that never makes the index tier's cutoff is exactly as invisible as one that never made a single-tier ranked result. What it does is partially recover legibility for the agent, not just for the human at a keyboard. It changes the failure from "a good answer disappeared with no trace" to "a good answer's title was on a screen the agent chose not to read closely," which is a meaningfully less silent failure, even though it isn't a solved one.

Trust plays into this tier too, and here I get to close this section by catching myself with exactly the tool the post is about. Notes carry a provenance class, four of them: auto for something captured with no reviewing judgment at all, agent for something a session recorded and is willing to stand behind, user-stated for a note that transcribes the user's own words verbatim, and human, reached only by a person explicitly promoting a note, never by the agent promoting itself there. The class doesn't change the ranking; it changes the frame the note is rendered in when it comes back, which is the honest place for it, because a trust label is a thing the reader should weigh, not a thing that should quietly reorder results underneath them.

So I queried the live store to check the distribution across those four classes, which is the kind of thing you can only do when you can read your own memory. Every one of the 749 notes is provenance='agent'. Zero human, zero user-stated, zero auto. Zero notes carry a bound user quote. That includes all 43 directives, which means the rules the user actually stated out loud, in their own words, come back to every future session rendered as "memory to verify, recorded by an AI session, not human-endorsed." The strongest content in the corpus is wearing the second-weakest label, and the frame reserved for a standing rule from the user has never once been used.

The cause isn't mysterious, and it's the same cause as everything else in this post: binding a quote is an optional parameter the writing agent has to remember to pass, and across 749 writes no agent ever remembered. An affordance the model must choose to use is an affordance that doesn't get used. That's the adoption finding again, wearing different clothes, and the fix has the same shape: bind it at the capture layer when the turn's user message actually contains the text, rather than asking the model to be more diligent about a keyword argument.

I don't think this post earns a tidy ending, and I'd rather not manufacture one.

A flat file is a memory system with retrieval disabled. At small scale, that's not a compromise, it's the correct design: the guarantee of never missing is worth far more than any ranking scheme, because there's nothing to rank and nothing to miss. Every fix in this post is an attempt to keep that guarantee, or something close to it, for the slice of memory where it matters most, as the corpus grows past the point where a context window can hold all of it at once.

Some of that is done. The unconditional tier for directives genuinely recovers "never misses" for standing rules, the same design as the flat file, deliberately scoped down to where it's affordable. The export mirror genuinely recovers cat, grep, and a meaningful git diff, and it does that on 749 of 749 real notes today, not a fixture. Some of it is a real but partial improvement: the index tier makes a miss at least visible as a title on a screen, which is not nothing and not the same as solved. And some of it, the miss-rate measurement, the file's write-back path, is genuinely unfinished, and I'd rather leave it marked that way than round it up.

The interesting engineering problem was never proving that a semantic store beats a text file. At 749 notes it clearly does, on the axis that matters at that scale, which is that the alternative is unusable. The interesting problem is holding onto what made the text file trustworthy in the first place, that it never chooses and therefore never silently fails, for exactly the class of memory where silently failing is not an acceptable outcome, while still being the kind of system that can hold 749 notes instead of thirty. That's the whole project, honestly, and it isn't finished.

• vectr source: agent/working_context_store/_export.py (notes_for_export() and the render path), app/service.py (_bump_notes_epoch(), the write-path choke point that also drives the debounced re-render hook), agent/config.py/agent/config.yaml (memory_export.debounce_seconds, default 3.0).
• vectr commit bdfafd2, merged to main 2026-08-18: read-only vectr memory export mirror.
• The controlled adoption benchmark referenced in section four: three runs of the same harness-verified arm, memory tool call counts and note-write counts as reported above.