Filtering After Top-k Is a Silent Bug in RAG

That is a post-filter: a yes-or-no condition applied after the ranking has already been cut to k. Put numbers on it. Call s the pass rate near this query, meaning the chance that a candidate from the top of its ranking passes the filter. Fetch k candidates and you expect k × s survivors, so returning n results takes, on average, k ≥ n / s.

Read the third line again, because it holds the most useful fact about the bug. A post-filter over an exact nearest-neighbour ranking never returns a wrong item. Everything it returns belongs in the answer, at the right position. Its only error is omission, the tail that fell outside the fetch, and omission leaves nothing behind to inspect. (An approximate index misses a few true neighbours of its own; the post-filter's omissions come on top.) This is also why the pattern gets through code review. Read as an access-control check, a post-filter is safe: it cannot leak an item the user is not allowed to see. The damage is all on the recall side, where a reviewer checking for leaks is not looking.

Why s is not the share of the store

The trap is in what s means. It is not σ, the share of the whole store that passes. It is the share of this query's nearest neighbours that pass, and once the filter is correlated with similarity the two can be far apart.

The damaging direction is a filter that removes the closest items. In a store shared across projects, the items most similar to a query about project A are likely to be items on the same topic from project B, because topic drives similarity and projects share topics. A workspace filter removes exactly those. Project A can own half the store and still own none of a query's ten nearest neighbours. The global share does not predict this; it depends on how the filter lines up with the embedding space, and it changes from query to query.

Rarity hurts even with no correlation at all. In one live store of 565 notes, 2 were in the revoked state: σ = 0.35%. Fetch 3 candidates and the expected number of revoked survivors is 3 × 0.0035, about 0.01. Under the independence model, 100 candidates contain a revoked note about 30% of the time, and a 95% chance takes about 845.

Correlation can also run the helpful way, when the wanted items are the ones nearest the query. That case comes up with the revoked notes below, and it is the reason a fixed depth sometimes works far better than σ predicts.

The usual mitigation is to over-fetch by a constant: ask for limit * 3 and filter down to limit. The constant encodes an assumed s. Three times the limit assumes that a third of the neighbours pass. For some queries and filters that is right, for others it is silently wrong, and the code has no way to tell which case it is in.

Underneath that sits a quieter problem. Even when the assumed s is exactly right, n / s is the average depth needed, and the actual depth scatters around it. With s = 1/3, n = 5 and k = 15, the chance that at least five survive is 60%. Getting to 95% takes k = 25: five times the limit, not three. Rarer classes need more headroom still. At s = 1/7, a 95% chance of five survivors takes 62 candidates, a multiplier above twelve.

The store keeps an agent's notes in a relational table and their embeddings in a separate vector collection. Each note belongs to a workspace, has a kind (a standing rule, a task, a caveat, and so on), may have expired, and has a lifecycle state.

When a note is written, the store looks up the most similar existing notes and shows them to the writer, so it can see what the new note might duplicate or contradict. The vector collection was global across every workspace the daemon served, and the embeddings were written with their ids and nothing else, so the similarity query had nothing to filter on. The workspace check, the expiry check and the lifecycle check all ran in SQL and Python after the vector search.

The query asked for limit + 1 neighbours, budgeting for exactly one exclusion: the note itself. When the nearest neighbours belonged to another workspace or had already been revoked, they used up the whole budget and the filters removed every one of them. The function returned an empty list even when close, active, same-workspace notes existed.

agent/working_context_store/_related.py, commit 77252a6
# before: room to exclude the note itself, and nothing else
n_query = min(limit + 1, col_count)

# after
n_query = min(max(limit * 3, limit + 1), col_count)

That repair is a fixed multiplier too, and so a bet on s. It holds while the querying workspace owns most of the neighbourhood. Take a daemon serving ten projects of similar size with no topic overlap, so s is about a tenth, and the shipped limit of 3. The fetch is 9 candidates, and the first of them is the new note itself, which the lookup skips. The remaining 8 yield 0.8 survivors on average and a full list of three about 4% of the time. Topic overlap between the projects makes it worse, since overlap is exactly the correlation that puts another project's notes at the front.

The same write-time lookup has a second path: find revoked notes similar to the new one, so an agent re-deriving a belief that was already proven wrong is warned before it writes the belief down again. That path copied the first path's pool formula, which tied search depth to how many results would be rendered. With one revoked result rendered, the pool was 3 candidates.

Measured on a live 565-note store, revoked notes were 2 of 565, 0.35%, against roughly 99% active. A pool of 3 has no room for a crowd. Three closer active notes on the same topic always push the one relevant revoked note out before the lifecycle filter runs, and in the normal write path two are enough, because the new note's own vector is already in the collection and takes the first slot. The two paths had target classes whose prevalence differed by more than two orders of magnitude, and they shared one formula.

Commit 8f51803 decoupled depth from render count with a flat floor of 100 candidates on the revoked path. The measured cost, on a synthetic 50,000-vector collection: about 0.59 ms mean for 3 results against about 0.94 ms for 100. Under a millisecond either way. The bug had been saving 0.35 ms per write.

Why the floor is flat

The floor is 100 whatever the store size, and prevalence did not set it. Under the independence model, 100 candidates find a randomly placed revoked note less than a third of the time, so if prevalence were the whole story, 100 would be far too few. Here the correlation runs the helpful way. The revoked note worth surfacing is a near-duplicate of the note being written, so it ranks high. What pushes it out of the pool is the handful of active notes on the same subject that rank higher still, and that handful is bounded by how much anyone writes about one topic, not by how large the store grows. Anything further down would not be shown anyway: the lookup drops candidates below a cosine similarity of 0.75 by default.

Prevalence, then, explains why the active path's formula was wrong for this path. The depth that is right comes from a different number, the count of non-passing notes that can crowd in above the answer.

What such a warning should say once it is found is a separate question, taken up in Should a Correction Repeat the Mistake?. The wording matters only if retrieval reaches the revoked record at all.

Path-based recall answers "which notes concern the file about to be edited?" A note can match in three ways: its text mentions the path, it is explicitly anchored to the file, or it declares a trigger glob that matches it. All three were combined with OR into one SQL query with one LIMIT, and Python did the exact matching afterwards.

SQL could scope the first two arms to the file with LIKE patterns on the file name. It could not scope the third. A glob such as **/*.py has no LIKE equivalent, so the SQL step matched every note that declares any path trigger at all, anywhere in the workspace, and left the glob to Python. A workspace with enough such notes filled the shared LIMIT and pushed genuine file-scoped matches off the end before the narrowing ran. In a stress run of the test harness with a broad glob, 20 of 20 test windows lost their genuine anchored match. After the fix, 0 of 20 did.

recall_for_path, commit 24fba5f (simplified)
-- before: three arms, one pool, one LIMIT
SELECT * FROM notes WHERE workspace = ? AND valid_until IS NULL
  AND (content LIKE ? OR anchors LIKE ? OR triggers LIKE '%"path"%')
  ORDER BY ... LIMIT ?;

-- after: the arm SQL cannot scope gets its own LIMIT
SELECT * FROM notes WHERE workspace = ? AND valid_until IS NULL
  AND (content LIKE ? OR anchors LIKE ?) ORDER BY ... LIMIT ?;
SELECT * FROM notes WHERE workspace = ? AND valid_until IS NULL
  AND triggers LIKE '%"path"%' ORDER BY ... LIMIT ?;
-- union, dedupe by note_id, then narrow in Python

Commit 24fba5f split the query in two: one bounded query for the arms SQL can scope to the file, one bounded query for the trigger arm, unioned and deduplicated by note id, and only then narrowed. However many notes declare triggers, they can no longer take a file-scoped match's place in the pool.

A second defect sat behind the first. Once the pool was right, genuine matches could still exceed the limit, and the final cut to limit was ordered by recency alone, so a newer weak match could displace an older strong one. Matches are now ranked by relation strength first (a declared anchor, then a matching trigger, then a mention in the text) and by recency within each tier. Fixing the pool moved the truncation one stage downstream, and that second cut needed its own ordering.

Recall can be filtered by note kind, one of seven. The kind filter ran after a fixed limit * 3 semantic fetch, and a selective kind returned fewer than limit notes with no signal that the shortfall came from the pool rather than from an absence of matching notes.

Three days before the kind filter was fixed, the codebase had pulled the over-fetch rule into a single helper whose docstring states the invariant for every query against the collection: size the fetch through the helper, never with a raw render limit. The kind filter went through the helper and still under-filled. The helper's rule is a multiplier of 3, which assumes a third of the neighbours pass, and a filter that keeps one kind in seven sits well outside that assumption. A shared helper makes every call site pay the same bet; it cannot know that one caller's filter is seven times narrower than another's.

Now the pool widens to limit * 10 when a kind is set, a figure the code comment calls "a guess, not a measurement". Under independence, with a limit of 5, that returns a full five notes of a one-in-seven kind about 86% of the time. The rest is covered by a backstop: a plain query on the notes table, with no vector search involved, that pulls the newest notes of the requested kind and appends any the pool missed after the ranked results. It guarantees the count, not the ranking. Those appended notes carry no similarity score and sit after every note the semantic pool did rank, however relevant they are.

Sorting a pool cannot reach what the pool missed

A close relative sits in the same function. Recall can also be sorted by recency, priority or chronology instead of relevance, and those sorts were applied to the semantic candidate pool. The most recent note, or the highest-priority one, may not be semantically close to the query at all, so it never entered the pool, and no re-sorting of a pool can surface an item that is not in it. Sorting a similarity pool by recency answers "the most recent of the similar items", which is rarely what "sort by recency" was meant to mean.

Explicit sorts now re-query the store under the requested ordering, so membership follows the requested key rather than query similarity. Rows fetched that way are not dropped for low similarity either: they were never scored, and a similarity floor means nothing for a row with no score. The regression test states the case exactly: seven on-topic decisions and one decision about an unrelated subsystem, dated oldest of all, with an embedding orthogonal to the query so it cannot enter the pool. A chronological recall must return it first.

Relevance Is the Wrong Sort Order for Agent Memory takes up which ordering is right for memory in the first place. The mechanical constraint holds whichever ordering wins: it has to be applied to a candidate set chosen by that ordering.

Each instance survived because its output was plausible. An empty related-notes list reads as "nothing related exists". Two notes where five were asked for reads as "only two match". When the revoked-note warning never appears, the natural reading is that the belief was never revoked. In every case the wrong answer is a valid answer for a slightly different store, and neither the caller nor a test that checks well-formedness can tell it apart from the right one.

That is the general danger of truncating and then filtering: it converts a search error into an apparent fact about the data. A retrieval layer that returns fewer items than requested is making an implicit claim, "there are no more", and a post-filter makes that claim without having checked it.

For agents the cost compounds. A person scanning search results may notice that something they expected is missing and search again. An agent that recalls memory and gets a short list has no signal that would prompt a second, deeper query, so it proceeds as if the list were complete. The missing note is absent from every decision downstream of that answer, and nothing in the transcript marks where it would have mattered.

The list runs from weakest to strongest. The weaker fixes are worth knowing because the strongest one depends on the engine, and on the data being in the engine at all.

Report the underfill

When fewer than n results survive and the fetch stopped before the store did, say so: "returned 2 of 5 requested; candidate pool of 15 exhausted". The answer is no better, but a silent claim has become a visible uncertainty that a caller, or an agent, can act on. Telling the two cases apart is cheap if you are careful about one detail. The store ran out if the fetch covered every item, or if the last candidate fetched already falls below the similarity floor, since nothing further down could have been shown. A fetch that came back with fewer rows than it asked for is not proof on its own: some indexes cap what one scan can return. pgvector's HNSW scan, for one, returns at most hnsw.ef_search rows (40 by default) however large the LIMIT, unless iterative scans are switched on.

Size the pool for the filter, not for the render

Depth and display count are different quantities. Measure s instead of guessing it: take a sample of real queries, fetch each one deep and unfiltered (a few hundred results), and count what fraction of the top 50 pass the filter. Look at the low end of that distribution, the tenth percentile say, not the mean, because the queries that underfill are the ones with the lowest s. Then size for the tail, since n / s is the depth that works only a little over half the time. When the wanted class is rare but its members sit close to the query, as with the revoked notes, the depth is set by how many non-passing items crowd in above them. The revoked-note floor of 100 cost 0.35 ms.

Filter first when the passing set is small

If the filter keeps only a few hundred items, invert the order: select them by the filter, then score each one exactly against the query. A few hundred dot products cost less than embedding the query did, and the answer is exact. It is an easy fix to overlook, because a brute-force loop sitting next to a vector index looks like a step backwards. At that size it is both the faster option and the exact one. On the revoked path above, filtering first would have meant scoring two notes. Engines with a query planner make this switch themselves when a filter is selective enough. The kind-filter backstop is a partial version: it selects by the filter first, but orders by recency instead of scoring.

Give each arm its own limit

When a query is a union of conditions with different selectivity, run each as its own bounded query and merge afterwards, so one broad arm can never starve a narrow one. Then look at the cut that comes after the merge. Once the pool is right, the final truncation needs an ordering that reflects what the caller values, or it reproduces the starvation one stage later.

Re-query for orderings other than relevance

A sort that is not by similarity needs a candidate set chosen by that sort. Fetch the top rows by the requested key under the same metadata filters, and let membership follow the key, rather than fetching by topic and sorting the result by date.

Iterate until satisfied

Fetch, filter, and if the survivors are short and the store has more, fetch deeper and repeat. This is the general solution when selectivity is unknown in advance.

Filtered search with an honest stopping rule
def filtered_search(index, q, n, passes, min_sim=None, k_max=None):
    """Nearest n items that pass, or an explicit account of why fewer."""
    size = index.count()
    k_max = min(k_max or size, size)
    k = min(4 * n, k_max)
    while True:
        hits = index.search(q, k)                      # best first
        keep = [h for h in hits
                if passes(h) and (min_sim is None or h.score >= min_sim)]
        if len(keep) >= n:
            return keep[:n], None
        saw_all = len(hits) >= size
        below_floor = min_sim is not None and hits and hits[-1].score < min_sim
        if saw_all or below_floor:
            return keep, None                         # short, and true
        if k >= k_max or len(hits) < k:               # our cap, or the index's
            return keep, f"returned {len(keep)} of {n}; searched {len(hits)} of {size}"
        k = min(2 * k, k_max)

Four things end the loop: enough survivors, every item seen, the similarity floor crossed, or a cap reached, either the caller's k_max or one inside the index that returned fewer rows than requested. Only the last produces a short answer that might be wrong, and it is the one that carries a report. Doubling k keeps the total work within about twice the final fetch. Each round keeps its own hits rather than appending pages, because an approximate index searched deeper is not guaranteed to return a superset of what it returned shallower.

pgvector added exactly this loop inside the database in version 0.8.0, as iterative index scans.

The code search already did this

A search pipeline elsewhere in the same system did the right thing without naming it. Before trimming candidates for the reranker, it fetches 200 raw candidates and drops trivial chunks first (one-line templates, fixture stubs, a lone import), so the pool is filled with real code even when a repository carries a hundred or more trivial chunks. It then keeps the first 40 non-trivial candidates from the vector ranking and, separately, the first 40 from the keyword ranking, and reranks the union. That is over-fetch sized for the filter and a separate limit per arm, with the reason written down in the configuration comment. When a language filter is set, that condition travels with the vector query instead, because every chunk's language is stored alongside its vector.

The strongest fix is a filtered approximate nearest-neighbour search that respects the condition while it walks the index, so the answer is genuinely "the nearest k among items that pass". For graph indexes that is harder than it sounds. An HNSW graph is built so that a greedy walk from an entry point reaches a query's neighbourhood in a few hops, and its edges are chosen by geometry alone. Skip the nodes that fail a selective filter and the walk has fewer usable paths toward the answer, or ends up visiting most of the graph to find the few nodes that pass. (For HNSW from first principles, see the section on approximate search in Text Embeddings, Vector Databases and LLMs: A Full Guide.)

Two research lines address this directly. Filtered-DiskANN (Gollapudi et al., WWW 2023) builds the graph from the vectors and their labels together, so edges connect points that share a label as well as points that are close. It defines a filter's specificity as the fraction of points carrying the label, and notes that the search-then-filter approach may need a very large number of candidates before it finds a single match when specificity is low. ACORN (Patel et al., SIGMOD 2024) takes the predicate-agnostic route. It builds an HNSW-style graph with each node's candidate neighbour list expanded by a factor γ, then searches only the subgraph induced by the nodes that pass, and it reports 2 to 1,000 times higher throughput than prior methods at fixed recall. The paper sets γ from the inverse of the smallest selectivity to be supported: the same n / s arithmetic, moved from query time into index construction.

Several engines now document a filter mode that applies during the search. The vocabulary does not line up across them. Elasticsearch and Weaviate call a filter that the graph walk honours as it goes a pre-filter; much of the research literature keeps that word for "apply the filter, then scan the survivors exhaustively" and calls the in-walk version filtered search. The label matters less than two facts about each engine: where the condition meets the ranking, and whether the result can come back with fewer than k rows when enough matches exist. As documented in September 2026:

Engine	Filtered query, as documented	What to watch
pgvector	With an approximate index (HNSW, IVFFlat), the WHERE condition is applied after the index is scanned. The README's example: a condition matching 10% of rows, with HNSW and the default hnsw.ef_search of 40, returns about 4 rows on average.	Iterative index scans (0.8.0 and later) are opt-in: hnsw.iterative_scan set to strict_order or relaxed_order, capped by hnsw.max_scan_tuples (20,000 by default); ivfflat.iterative_scan with ivfflat.max_probes. relaxed_order can return rows slightly out of distance order; the README re-sorts them with a materialized CTE.
Elasticsearch	The filter parameter of the knn query is applied during the approximate search, so k matching documents are returned.	Every other filter in the query is a post-filter, including a term filter in a bool query's filter clause wrapped around the knn query. That returns fewer than k results even when enough documents match.
Qdrant	Filterable HNSW: extra graph edges built from payload indexes let the search apply filters as it walks the graph.	Create payload indexes before ingesting data: the filter-aware edges exist only if the graph is built after the indexes, and adding one later means rebuilding. When a condition matches few enough points, the query planner uses a full scan instead of the graph.
Weaviate	Pre-filtering: an allow-list from the inverted index, honoured during the HNSW search. ACORN is the default filter strategy since v1.34; sweeping is the alternative.	Switches to a flat, brute-force search over the allowed set when the filter is very restrictive.
LanceDB	Pre-filtering is the default: the where condition is applied before the vector search.	Post-filtering is opt-in and can return fewer than limit rows, or zero, if the nearest neighbours fail the filter.

Each row is taken from the project's own documentation; see Sources. Features in this area change quickly, so check the version you run.

Pushing the filter down has a precondition: the index must hold the facts the filter tests. In the memory store behind the four instances, the vectors were written with ids only, and the filters tested columns in a relational table and a lifecycle state computed from an event log. No engine can apply a condition to data it was never given. Two-store designs, vectors in one system and the facts about them in another, are where this bug breeds, and moving the filterable facts next to the vectors is often the real fix.

Native support also has a ceiling. Iterative scans stop at a configured cap, and a filtered graph search is still approximate. The underfill report from the weakest fix belongs on the strongest one too.

Every test that passes against buggy post-filter code has one property in common: the fetch already reached the answer, so k ≥ r_n held. The store was small enough, or the filter permissive enough, that depth never mattered. The test that catches the bug is the adversarial one:

1. Build a store where the nearest neighbours of the test query all fail the filter: other workspace, revoked, wrong kind.
2. Put the true answer further away, but passing the filter.
3. Assert that the true answer is returned.

An adversarial regression test
def test_answer_survives_when_nearest_neighbours_all_fail(make_store):
    # Vectors placed at chosen cosines to the query, so the ranking
    # is fixed by construction rather than by a model's opinion.
    store = make_store(embed=cosine_stub({
        "decoy a": 0.99, "decoy b": 0.98, "decoy c": 0.97,
        "the answer": 0.90,
    }))
    for text in ("decoy a", "decoy b", "decoy c"):
        store.add(text, workspace="other")
    answer = store.add("the answer", workspace="mine")

    got = store.search("query", n=1, workspace="mine")

    assert [g.id for g in got] == [answer]

Use a stub embedder. With a real model, whether the decoys outrank the answer is the model's opinion, and a model upgrade can make the test pass for the wrong reason. The vectr regression tests do this in two ways: one helper returns unit vectors at specified cosines to a base vector, another maps marked text to a vector orthogonal to everything else so a note is guaranteed to rank last.

The tests added with the related-notes, revoked-notes and path-recall fixes, and with the sort fix, all have this shape, and each fails against the code before its fix. The first two get an empty list back. The path-recall test gets a list without the anchored note. The sort test gets a chronological list that starts at the wrong note. The kind-filter fix is covered differently, by tests that pin the pool width and check that the prefetch runs; an outcome test of the adversarial kind would be the stronger guard there.

That is the regression shape worth keeping: not "does filtering work" but "does filtering still work when the filter removes the closest things".

• vectr, commit 77252a6, "Over-fetch candidates in related_active_notes before post-query filtering". The related-notes fetch depth and the two failing cases (foreign-workspace and revoked nearest neighbours). Code: agent/working_context_store/_related.py.
• vectr, commit 8f51803, "decouple revoked-path query depth from revoked_limit". The 2-of-565 revoked-note measurement, the 0.59 ms and 0.94 ms timings on a 50,000-vector collection, and the revoked_query_floor setting with its rationale in agent/config.yaml.
• vectr, commit 24fba5f, "recall_for_path declared-trigger arm can no longer starve content/anchor matches". The split into two bounded queries, the relation-strength ordering of the final cut, and the 20 of 20 versus 0 of 20 stress result. Code: agent/working_context_store/_store.py.
• vectr, commit 24726e4 and the 1.12.0 entry in the CHANGELOG, "Kind-filtered recall no longer under-fills silently". The limit * 10 kind floor and the metadata-only prefetch backstop.
• vectr, commit 841a149 and the 1.11.0 CHANGELOG entry, "An explicit non-relevance recall sort returns the true top-limit". Re-querying under the requested ordering, and the orthogonal-embedding regression test.
• vectr, commit 77247d6. The single over-fetch helper, working_memory_fetch_width, and its stated invariant.
• vectr, ranking.rerank.pre_filter_fetch_k and top_k_unfiltered in agent/config.yaml, and the trivial-chunk pool filter in agent/searcher.py. The code-search over-fetch described in Section 08.
• pgvector, README, sections "Filtering" and "Iterative Index Scans"; and the pgvector 0.8.0 release announcement. Filtering after the approximate index scan, the 10% and ef_search 40 example, and the iterative scan settings.
• Elasticsearch, Knn query reference, "Pre-filters and post-filters in knn query". Which filters are applied during the approximate search and which after it.
• Qdrant, Indexing, "Filterable Index". Payload-index edges in the HNSW graph and the full-scan threshold.
• Weaviate, Filtering concepts. Allow-list pre-filtering, the ACORN and sweeping strategies, and the flat-search switch for restrictive filters.
• LanceDB, Metadata filtering. Pre-filtering as the default and the documented short results of post-filtering.
• Gollapudi, S., Karia, N., Sivashankar, V., Krishnaswamy, R., Begwani, N., Raz, S., Lin, Y., Zhang, Y., Mahapatro, N., Srinivasan, P., Singh, A., Simhadri, H. V. Filtered-DiskANN: Graph Algorithms for Approximate Nearest Neighbor Search with Filters. Proceedings of the ACM Web Conference 2023 (WWW '23), pp. 3406 to 3416. Label-aware graph construction (FilteredVamana, StitchedVamana) and the definition of filter specificity.
• Patel, L., Kraft, P., Guestrin, C., Zaharia, M. ACORN: Performant and Predicate-Agnostic Search Over Vector Embeddings and Structured Data. SIGMOD 2024, Proceedings of the ACM on Management of Data 2(3); arXiv:2403.04871. Predicate subgraph traversal and the γ neighbour-list expansion.
• Saha, S. Relevance Is the Wrong Sort Order for Agent Memory. Which ordering memory retrieval should use, and why similarity ranking drops corrections.
• Saha, S. Text Embeddings, Vector Databases and LLMs: A Full Guide. Embeddings, cosine similarity and HNSW search from first principles.