Agent Memory Fails in Three Places and Gets Measured as One

A few terms first. The store is the database of saved notes. Retrieval is the search that picks notes out of it for a given moment. A note is delivered when the harness places it in the agent's context, whether or not the agent asked. With those, the three failures are:

• Not found. Retrieval ran and returned nothing, or the wrong note. The right note may be sitting in the store the whole time.
• Never asked. The right note was in the store and retrieval would have found it, but looking was the agent's choice and the agent did not make it.
• Delivered and ignored. The note reached the agent's context, intact, before the decision it was about, and the agent acted against it.

Never asked exists only where asking is optional

The middle failure needs a condition. A question-answering pipeline that runs retrieval for every question cannot fail to ask. Yuan, Su and Yao's diagnostic study of agent memory is built that way: retrieval runs on every question in the LoCoMo benchmark, and failures are split two ways, retrieval failure against utilization failure, plus a small hallucination class, with a language-model judge doing the classification. Across their grid of memory and retrieval configurations, retrieval failures were 11 to 46 percent of all questions and utilization failures 4 to 8 percent. In that setup "never asked" has nothing to count.

Coding agents are usually built the other way. Memory is a tool the agent may call, next to reading files and running commands, and an agent with a task in front of it often does not call it. In one measured instance, an agent with memory tools connected, explicit guidance to use them, and a store seeded with directly relevant facts made zero memory calls in 114 turns. The argument that follows from that, deliver from the harness instead of waiting to be asked, is in The Agent Never Chooses to Remember.

Moving delivery into the harness removes the agent's choice, but the never-asked failure does not vanish. It moves into the harness, as a trigger that should have fired and did not. And a harness can believe it delivered a note that never reached the model, in which case an evaluation that trusts the harness's own account will score the run as delivered and ignored.

The natural instrument for "does memory help?" is a ratio of two scores. Run the task with the needed information handed to the agent directly, an oracle condition. Run it again with the agent responsible for getting that information out of its own memory. The ratio says how much of the oracle's benefit memory recovers.

VehicleMemBench, an executable benchmark for multi-user long-term memory in in-vehicle agents, formalizes this as MemoryScore. Each task is scored by exact state match: after the agent acts, does the vehicle's state equal the target state? MemoryScore divides the score under autonomous memory, where the facts have to be stored and retrieved by the memory system under test, by the score under gold memory, where the facts are given in context. The benchmark uses it to argue that memory construction and retrieval, rather than execution, are the dominant sources of error, and it scores by environment state without any model or human grader, which is a real strength.

The ratio's weakness is arithmetic. Take the three failures in turn. The agent never asked: the task is lost. It asked and got the wrong note: lost. It got the right note and ignored it: lost. Each failed task costs the ratio the same amount, whichever stage it failed at.

What the memory benchmarks already separate

The same arithmetic applies to any evaluation that grades only a final answer. A long-term memory benchmark asks a question after a long history, and a wrong answer could mean the fact was never stored, was stored and not retrieved, or was retrieved and not used. The established benchmarks know this. LongMemEval annotates the evidence sessions behind each question and reports Recall@k and NDCG@k whenever the system under test exposes its retrieval results. LoCoMo annotates each question with the dialogue turns that hold its answer and reports retrieval accuracy for retrieval-augmented models. Both can pull "not found" apart from the rest, provided the system shows its retrieval. Neither has a category for "never asked", and for a pipeline that retrieves on every question there is nothing to put in one.

VehicleMemBench separates the stages too, by hand. Its error analysis sorts failed cases by manual inspection into memory errors, extra calls, execution errors and incomplete execution, the last of which includes not properly using retrieved memory. The headline number is still the ratio. The stages are recovered afterwards, by reading.

An outcome number with a broken stage underneath does worse than hide the cause. It produces a confident reading of the wrong thing. In an earlier vectr benchmark, the recall function matched notes by literal substring (an SQL LIKE) instead of by meaning, so implementation sessions got empty or near-empty results back. The outcome read as the memory tool making implementation more expensive. It was a not-found failure, reported as a cost finding; Building Vectr 3: What the Benchmark Numbers Mean has the numbers before and after the fix.

A ratio has a second blind spot. When both conditions reach the maximum the task allows, the ratio of two equal numbers is 1.0. Depending on what you hoped to see, that reads as "memory adds nothing" or "no problem here". Neither reading is supported. Both arms are at the top of the scale, and the outcome has nothing left to say.

A pre-registered experiment in July 2026 hit exactly this. One seeded bug in Apache Camel, a large multi-module Java codebase. Two arms of three sessions each, identical except for one variable: whether one operational note was in the memory store. The note explained how to verify a fix honestly in that build. A test run scoped to one module compiles against previously installed copies of the others, so a green result there cannot see a change made in another module. The primary outcome was whether each session verified its fix with a test that could actually see the change.

It came out 3 of 3 in both arms. The frozen rule for reading the result returned not supported, and that verdict stands. Underneath the tie, the processes differed. The report gives the following as exploratory secondary counts, with no direction claimed:

A false pass is a green test that could not have failed. Two details complicate even these counts, and both come from reading the transcripts by hand. The parser judged honest verification from the shape of a command's arguments. In one memory session the qualifying build was moved to the background and the session hit its turn limit before reading the output; in one control session the qualifying run carried a flag that skips the tests entirely. Hand-audited, honest verification is 2 of 3 in each arm. The verdict does not move. And the clean false-pass column in the memory arm cannot be credited to the note: one memory session shows no trace of it, and one control session scoped its build correctly from its first command, with no note at all.

The transcripts also hold a failure the outcome cannot register: one memory-arm session never accessed its available note and stayed clean anyway. That is a never-asked failure inside a perfect score, invisible to the outcome and visible only in the transcript. The other two memory sessions called recall early, got back a one-line index with the note's title and an instruction to expand it by id, and never expanded it. Whether a prompt-time channel put the note's body into any session's context cannot be established from the transcripts, because the transcript format does not record what that channel injects. Agents Forget How, Not What walks through all six sessions.

Process measures keep moving after the outcome saturates: false passes, builds spent re-deriving a command, turns spent reaching a fact the note already held. They do not replace the outcome, and three sessions per arm support no effect in either direction. They are what is left to look at when the outcome has stopped discriminating, and an evaluation that did not record them has nothing to look at.

Separating the three failures means observing what happens between "the note exists" and "the agent did the right thing". For a note the harness delivers, there are at least six stages, and each needs evidence of its own.

In vectr's evaluations, every row of this table has been misread at least once, and the misreads ran in both directions. Some instruments reported a stage as passed when it had failed. Others reported a failure that had not happened. Each case is on public record, in the repository's benchmark documentation, its commit history, or the technical report.

Those evaluations deliver notes through two channels. One is a local proxy that sits between the agent and the model API and appends the note to outgoing requests. The other uses hooks: commands the agent runtime runs at fixed points, such as session start or just before a tool call, whose output is added to what the model sees. A control arm gets neither and starts with an empty store.

Stored: a gate that checked for "not empty"

The longitudinal evaluation runs a trajectory of sessions, and its validity gate required the note store to be non-empty before a session counted. Sessions in the hook arm passed that gate holding two or three notes, where every proxy-arm session held exactly one and every control session none. In the session examined most closely, the hook's payload carried two byte-identical copies of the note, the fresh plant and a stale one left from an earlier plant. The commit that fixed the premise traces the extras to a retried setup step that planted a second note on top of the first. The gate answered "is anything there?" when the premise was "exactly one note is there". It now checks for an exact count of one before any money is spent on the session (commit 0969fa4), and the report treats the hook arm as contaminated and does not compare its costs with the proxy arm's.

Selected, not sent: an audit line for an injection that never happened

In the injection evaluation, the daemon holding the store writes an audit line each time it hands a note to the proxy. In the first pilot cell that line recorded one item of 314 characters, while the proxy's own counter read injected: 0 and inject_skipped: 11.

The proxy only appends a block when a request ends with a user turn. The agent's first request ended with a system message, so the note was retrieved for that request, could not be attached to it, and was charged against a cooldown, the rule that stops the same note being sent again for a while. The cooldown then suppressed it on every later request that could have carried it. One shot, spent on the one request that could not take it. Neither rule is wrong on its own; together they guarantee the note never arrives. Across three scenarios and one controlled reproduction, zero of four planned deliveries reached the model.

Had the audit line counted as delivery, each of those cells would have scored as a valid run in which the note arrived and changed nothing: a delivered-and-ignored result manufactured entirely by the instrument. The harness required both counters and marked the cells invalid instead. The same audit log misled in the other direction too. The harness's own preflight probe retrieves through the same daemon, so until the harness started counting only events logged after the preflight, a clean control with no deliveries read as contaminated by an event the proxy never sent. One early pilot cell failed exactly that way.

Sent: a transcript that could not show it, and a parser that saw too much

The injection evaluation's hook channel fired before each prompt and before each tool call. Its validity rule required the note's text to appear in the session transcript. It never did, both cells were marked invalid, and the first write-up concluded that output from those hooks did not appear to reach the model when the agent ran headless. A probe falsified that the same day. Three hooks each emitted a unique canary token, the model was asked to echo any such token it saw, and all three came back verbatim. The transcript format does not render output from those two hook types, so the transcript could neither confirm nor deny delivery through them. Absence from the transcript had been read as absence from the context.

The opposite error turned up in the Camel experiment. Its parser treated any matching tool result after the opening events as a delivery at command time, and so labelled the two recall calls that returned only a one-line index of titles as deliveries of the note. Neither call delivered the note at the command.

Sent, then gone before the decision

A note appended to one outgoing request is visible for one model call. In the proxy design as first measured, the block was not carried into the conversation state the agent kept, and a cooldown suppressed repeats. In 6 of 7 baseline cells the note was out of context before the agent's first edit that changed a file. Every one of those cells passed the delivery gate, correctly, because the note had been sent. Sent is not present at the decision, though, and a cell where the note was gone before the agent chose what to edit says nothing about whether the agent trusts notes. The fix delivers the full note, not just its title, on the turn where the decision is made.

Acted on: a mention counted as an execution

Action checks read the transcript's ordered tool calls. The scorer's signatures for shell actions were regular expressions searched anywhere in the command string, so a command that mentioned a script matched the same pattern as one that ran it. The deploy.sh scorer is the worst instance. A quieter one sat in the injection evaluation: two diagnostic counters named for invocations of a test runner used the same anywhere-in-string search, so grep -rn "run_tests.sh" . or an echo naming the runner counted exactly like running it. Those counters never fed a pass or fail. When they were anchored, the series was re-baselined, with a written warning that values from before and after the change are not comparable.

Reflected in state: a check that scored correct behaviour as failure

State checks read the bytes on disk, and they can be wrong about what the bytes mean. In the deploy scenario, each later session runs in the workspace the previous session left, which is deliberate. But the first session's entry in the deploy queue was still there when the second began. The primary check required the number of queue entries to increase, every agent correctly concluded that the deployment was already queued and that a duplicate would be wrong, and every arm failed the check. The fix resets the files a scenario declares as able to pre-satisfy a later check, at the start of each later session, and the check now asks for at least one entry added in that session.

A release scenario showed a subtler version. Its outcome check asks whether a correctly annotated release tag exists at the end. One memory variant, whose note also carried where and when the fact was established, declined to tag in both of its later sessions, on the ground that tagging would trigger a real and irreversible public release, and asked the user for permission. The harness was non-interactive, so the question had nowhere to go. (The two sessions were not independent: the second started from the first's untagged workspace, so this is one failure and its sequel.) On a check of final state, an agent that became more cautious scores the same as one that became less capable.

The clearest instrument failure produced a wrong headline: a note delivered and not heeded. The note had in fact been delivered and heeded, and the scorer had matched a read-only command.

A synthetic repository contains deploy.sh. It appears to work: it prints progress, exits zero and writes a state file. It does not work, because a reconciler silently reverts whatever it does, and the correct action is to append an entry to a queue file. The user states that fact once, in the first session's prompt. Before the second session, a fresh process with no conversation carried over, a note carrying the fact verbatim is planted in the store. Two separate gates confirm delivery before anything is scored: the daemon's hook counter incremented, and the note's text appears verbatim in the agent's own transcript. The counter alone would prove little, since it also counts the harness's own probes of the endpoint; the transcript check is what shows the agent saw the note.

The scorer reported that in the second session the agent ran ./deploy.sh as its second tool action, without correcting itself and without calling any memory tool. That reads as the most interesting failure on the list, delivered but not heeded, the one that survives every retrieval and delivery fix.

The mistake signature was the regular expression (\./)?deploy\.sh\b, searched anywhere in the command string, and it matched cat deploy.sh. All three of the agent's tool actions in that session were read-only. Its closing message said why it had left the script alone:

"I did not run ./deploy.sh staging [...] per what I know from this workspace, that path appears to succeed locally but gets silently reverted by the reconciler on its next pass, leaving no trace."

Three properties make this more than a typo in a regex.

The contradiction was already in the record

Scorer checks come in two kinds. An action-derived check reads what the agent did, from the ordered commands in the transcript. A state-derived check reads what the workspace shows, from the bytes on disk. The same result record carried deploy_state_untouched = PASS: the state file's hash equalled the one taken at the start of the session. Running the script writes that file, so an unchanged hash means the script did not run to completion. Beside it sat mistake_committed = true and a check reporting two matching commands. The record disagreed with itself, and nothing in the pipeline compared across the two kinds of evidence.

Two agreeing checks were one check

The two action-derived signals agreed with each other automatically, because both read the same pattern family. They looked like corroboration and were one observation counted twice. The record held two independent observations, not three: one state check against one regex. Whether two checks are independent depends on where each one reads its evidence, and the number of checks says nothing about it.

Deterministic is not the same as correct

No language model judged anything. The scorer was a small reproducible program, and it was reproducibly wrong. Judge-free, state-based scoring is a real advantage, and benchmarks such as VehicleMemBench claim it for good reasons: no grader drift, no sensitivity to a judge's prompt. What determinism buys is the same answer every time. Whether the answer is right is a separate property, and a scorer has to be tested for it like any other software.

Execution position, and what it still misses

The fix makes signatures that mean "the agent performed this action" match only at an execution position: the start of a command, or right after a separator (;, &&, ||, |, & or a newline), after stripping the wrappers a shell allows in front of a program (VAR=value assignments, env, sh, bash, zsh, exec, timeout N). Running the scorer's matching logic over a set of commands shows what it fixes and what it does not:

The anchored matcher is a large improvement and still a heuristic over text, not a shell parser. A parser would settle the quoting case. The list of wrappers (sudo, interpreter flags, sh -c with a quoted script) is open-ended, and every one left off it is a run the scorer will not see.

The contradiction guard, and its direction

A contradiction guard now runs beside every session's score. When an action-derived check says the mistake happened and the scenario's declared state check says the file was untouched, the guard writes a machine-readable contradiction into the record, and the report prints it. It does not pick a winner, because a disagreement is evidence that one of the two checks needs attention, and the harness has no basis for guessing which.

The guard fires in one direction only. When sudo ./deploy.sh runs and the matcher reports nothing, there is no mistake verdict for the state check to contradict, and the miss is silent. The guard also exists only where a scenario declares a state check for its mistake, which most scenarios do not. The report states the general limit: the guard covers only facts observable both ways.

Because every transcript had been preserved, the whole reported grid was re-scored with the fixed matcher at zero model cost, each corrected record written beside its original rather than over it. All five flagged sessions in the reported grid flipped to clean, including the shared first session whose mistake was supposed to be the baseline for counting repeats. In the reported grid, no agent in any arm ever executed the script.

What the corrected reading does not show

The fixed scorer does not turn the session into a win for memory. The control arm, with an empty store, also declined to run the script, and the scenario turned out to offer more routes to the answer than it was designed to. The files the scenario started from documented the queue as the sanctioned channel, though not why the script fails. And the scenario's short name, deploy_reverted_by_reconciler, was part of the working-directory path every arm could see. The control transcript shows the agent picking the word "reconciler" out of its own path and searching the workspace for it; the word appears in no workspace file. A path that hands the control the answer is a leakage failure, the kind Your Eval Is Leaking the Answer catalogues. The scorer failure was a different kind: nothing leaked, and the instrument misreported what happened.

What survives is narrower. The agent's stated reason tracks the note's wording, not the path's. Three phrases from the note, "appears to succeed", "on its next pass" and the no-trace clause, appear in its closing message, and none of them occurs in the workspace files or in either control transcript. That shows a delivered note entering an agent's stated reasoning at the level of its wording. Two control transcripts are a thin base rate, and whether the note's wording or the situation produced the behaviour cannot be separated at this sample size, so it does not show that the note caused anything.

The wrong number was never published. It existed in an internal draft, and catching it was ordinary quality control before submission. What is unusual is the reporting: the technical report puts the wrong number, the mechanism that produced it and the guard that would have caught it in one document, where normally the reader would never learn the wrong number existed.

Two more metric failures from the same evaluation show a cheaper defence than re-reading transcripts.

A part larger than its whole

billable_tokens_to_fact was meant to count the tokens billed from the start of a session up to the moment the agent acquired the fact it needed. In every session where it was defined, it came out larger than the entire session's token count, by up to 2.6 times: 340,907 against 132,567 for the hook-arm session at k=2, the second session of its trajectory. A prefix cannot exceed its total.

The cause sits in how the transcript is written. It emits several assistant events per API call, and each one carries the call's usage record. The metric added a usage figure for every assistant event, so each call was counted more than once. That session had 33 assistant events across 13 calls, and the same sum taken over the whole session reaches 354,186.

A broken metric can still produce a tidy ordering. At k=2, the arms ranked on this metric in exactly the order of the tokens each session wrote into the prompt cache: 13,479 for the verifiable variant (a proxy-delivered note that adds a one-command check of its claim), 20,977 for control, 23,875 for the plain proxy note and 42,653 for the hook arm. That is a property of the cache, not of the agent, and the k=3 sessions do not keep the order. The replacement measure, the size of the context the agent holds when it acquires the fact, is in the same result record, is not double-counted, and points the other way: on it, control is the cheapest cell at both k=2 and k=3.

Turns that outnumber the turns

turns_to_fact counted assistant events, not conversational turns, and in every session it exceeded the session's own turn count: 33 against 20 in the verifiable session at k=2. The pre-registered expectation, two turns or fewer, was written in conversational turns, so comparing across the two units overstated the miss by roughly 2 to 3 times. Recounted from the preserved transcripts in the intended unit, the cells read 11 to 15 against an expected 2 or fewer, still a miss of five times or more.

The units also disagree about direction. At k=2, counted in conversational turns, the hook arm reached the fact faster than control, 11 against 13. Counted in assistant events, the same comparison is a tie, 30 against 30. Put the unit in the metric's name, turns_to_fact_api_calls or similar, and nobody compares across units by accident.

Report each stage, not one ratio. For every run, record whether the note was stored, selected, sent, present at the decision, acted on and reflected in state, each from its own evidence. The three failures then become three counts, and each count points at its own fix. A record shaped roughly like this is enough:

Treat a failed delivery as invalid, not negative. A run in which the note never reached the model does not test whether the note helps. Exclude it and report it as excluded, never average it in as "memory did not help". The same goes for any run whose premise failed, such as a store holding the wrong number of notes. Report exclusions per arm, too: when the arms have different gates, as the hook and proxy arms did, runs are dropped under different criteria in each, and the survivors are no longer a like-for-like comparison. One rule cuts the other way: whether the agent used the note is the outcome being measured, and it must never be a gate. A design that gates on behaviour cannot observe the failure it was built to find.

Screen for saturation, and record process measures anyway. Confirm that the control fails a scenario before paying for memory arms. Record false passes, turns spent re-deriving a known fact, and commands that repeat a known mistake, because these keep moving when every session eventually succeeds.

Pair every action check with a state check, and make disagreement an error. An agent's commands and the workspace's bytes are two views of the same event. When they disagree, one of them is being read wrong, and the harness should say so in the record. Count two checks as independent only if they draw on different evidence; two regexes from the same family are one check.

Keep the raw transcripts, and re-score beside the original. If I could keep only one of these practices, it would be this one. The matcher fix and the recount of turns in their intended unit were both applied after the runs had finished, and preserved transcripts are what made them cost nothing. Write the corrected record next to the original, never over it. Version the scorer with the harness, too. A check rewritten for a repaired harness assumes the repair: the deploy scenario's new check asks for at least one queue entry, which the first session's leftover entry would satisfy if the check were replayed over runs made before the reset.

Assert the invariants, and test the matcher. Parts do not exceed wholes, per-turn counts do not exceed turn counts, stage counts only shrink along the pipeline, and a run with zero deliveries has zero delivered-and-ignored events. Turn a command table like the deploy.sh one into unit tests for whatever decides "the agent ran X", including the rows the matcher gets wrong, so that the known misses are written down rather than rediscovered.

Prediction 1. For memory the agent must query voluntarily, decomposing failed tasks by stage will attribute most of them to never asked, not to retrieval. The measurement is the fraction of failed tasks in which the relevant note was never retrieved, split by whether retrieval was attempted at all. This runs against the forced-retrieval result, where Yuan, Su and Yao find retrieval failure dominant. The difference should come from the asking stage, which their setup removes by design.

Prediction 2. For memory delivered by the harness, the dominant remaining failure will be delivered and ignored, and it will concentrate on notes the agent cannot corroborate from its workspace. In vectr's injection runs, once delivery was fixed, a full note on the decision turn changed behaviour in the two scenarios where the agent could confirm it by reading a second file, and not in the one where the note was the only source, at one run per cell. That is a direction, not a result. When a Memory Should Become a Check works through what follows for enforcement.

Prediction 3. Evaluations that match shell text with unanchored patterns and have no state-derived cross-check will, when re-audited against workspace state, show contradictions at a non-trivial rate. The test is to add the cross-check to an existing harness and count.

• Saha, S. Deterministic, Reproducible, and Wrong: How Our Own Scorer Manufactured an Agent-Memory Finding, and the Guard That Would Have Caught It. Technical report, 2026. doi:10.5281/zenodo.22079952. The three failures (Section 1), the deploy.sh record and its correction (Sections 1.1, 1.2 and 4), the arm mismatch and store contamination (Section 3.3), the validity gates (Section 3.6), the broken metrics (Section 5.1), the release and deploy grids (Sections 5.2 and 5.3), the specimen and its open counterfactual (Section 6.1), the ceiling case (Section 6.2), the oracle-ratio argument (Section 6.3) and the residue and path-leak defects (Section 8).
• Saha, S. Delivery, Not Storage: Cue-Anchored Working Memory as a Harness Property for Coding Agents. arXiv:2607.20972, 2026. Zero memory operations in 114 turns with memory tools connected and a seeded store.
• Chen, Y., Xu, Y., Ding, X., Fang, X., Liu, S., Lin, L., Zhang, Q., Li, Y., Liu, Q., Xu, T. VehicleMemBench: An Executable Benchmark for Multi-User Long-Term Memory in In-Vehicle Agents. arXiv:2603.23840, 2026. MemoryScore as the ratio of exact-state-match scores under autonomous and gold memory (Section 3.2.2), construction and retrieval as the dominant error sources (Section 4.1), and the manually inspected error categories (Section 4.3).
• Wu, D., Wang, H., Yu, W., Zhang, Y., Chang, K.-W., Yu, D. LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memory. ICLR 2025. arXiv:2410.10813. Evidence-session annotations, and Recall@k and NDCG@k reported when a system exposes its retrieval results (Section 3.3).
• Maharana, A., Lee, D.-H., Tulyakov, S., Bansal, M., Barbieri, F., Fang, Y. Evaluating Very Long-Term Conversational Memory of LLM Agents. arXiv:2402.17753, 2024. LoCoMo: evidence turn annotations on each question and retrieval accuracy for retrieval-augmented models (Section 4.1).
• Yuan, B., Su, Y., Yao, K. Diagnosing Retrieval vs. Utilization Bottlenecks in LLM Agent Memory. MemAgents Workshop at ICLR 2026. arXiv:2603.02473. Retrieval on every question, with failures split into retrieval, utilization and hallucination (Section 3.3).
• vectr, benchmarks/injection_utility/README.md. Retrieval is not delivery, the zero-of-four delivery rate, one-turn visibility in 6 of 7 cells, the hook-channel cells and the same-day canary correction, the post-fix recheck, and the re-baselined runner counters.
• vectr, benchmarks/longitudinal_rediscovery/DESIGN.md, Section 6.5. DEFECT 9 (execution-position matching and the contradiction guard) and DEFECT 10 (the per-session residue reset and the rewritten queue check).
• vectr, benchmarks/longitudinal_rediscovery/scorer.py: the execution-position matcher and detect_contradictions. The matcher behind the command table in Section 05 and the one-directional guard.
• vectr, commit 0969fa4. The exact-count-of-one store premise per session and its pre-spend gate.
• vectr, results/longitudinal/ at commit 9629c8b. The preserved run directory, with each re-scored record beside its original.