Embedding Dilution in Semantic Code Search

Before the failure makes sense, you need the shape of the pipeline it happened in, because the shape is where the whole story turns. At measurement time, vectr retrieved in the way most production semantic-search systems do — a two-stage funnel.

Stage one is hybrid retrieval. It runs two searches in parallel and merges them. One leg is dense retrieval: encode the query into a vector, encode every chunk into a vector, and rank chunks by cosine similarity. The other leg is BM25, a keyword-scoring function that rewards exact term overlap. Together they produce a candidate pool of the top 200 chunks.

Stage two reranks that pool. A cross-encoder reranker — bge-reranker-base — reads the query alongside each of the 200 pool members and re-scores them properly, followed by a quality pass. That reranker is the smart part of the system. It is also the expensive part, which is exactly why it only ever sees 200 candidates instead of all 40,538.

Now the pieces that matter for the failure. The dense embedder was snowflake-arctic-embed-m-v1.5 — a general text embedder, not a code-specialized one. Hold onto that; it is not the villain, but it shapes the numbers. The corpus was a Django checkout from June 2026: 4,129 files, 40,538 indexed chunks. And the chunk for QuerySet.get was, by any reasonable standard, ideal. It carried a class marker (QuerySet), the full method signature, and that near-perfect docstring — followed by roughly thirty lines of mechanical implementation.

That last clause is the whole problem in embryo. But to see why, we first have to pin down where the miss happened, because the fix depends entirely on that.

My first instinct was the wrong one, and it is probably yours too: the reranker must have mis-scored it. Bump the reranker, add an importance prior for well-known symbols, tune the quality pass. Every one of those fixes operates on the pool. So I checked the pool directly, leg by leg, and the reranker turned out to be innocent — it never got the chance to be guilty.

The dense leg

For every natural-language phrasing I tried — "get a single object from the database," "fetch one row matching criteria," "retrieve a single record by lookup" — QuerySet.get was absent from the top 200 dense results. Not low-ranked. Absent. The only phrasing that got it into the dense pool at all was a deliberately ORM-flavored control, written in Django's own vocabulary, and even that reached only #123 of 200 — barely inside a pool it should have topped.

The keyword leg

BM25 was all over the place, which is its nature: it lives and dies on exact term overlap. Phrase the query as "return exactly one matching object or raise…" — words that literally appear near the method — and BM25 ranked it #1. Phrase it as "fetch one row matching criteria" and the same method fell to #127. Other phrasings missed entirely. BM25 wasn't a safety net; it was a coin whose bias depended on whether I happened to echo the source text.

This is also the moment most retrieval dashboards lie to you by omission. They display the reranked top-k — the final, polished output — which looks fine because the reranker did a competent job on the pool it received. The miss is one layer up, invisible on that screen. You have to instrument pool entry itself to see it. The demo below traces one chunk's fate through the funnel for three real scenarios from this run, so you can watch exactly which stage it dies at.

So the dense leg failed to rank a chunk whose docstring paraphrases the query. Why? The lazy answer is "the embedder isn't good enough, throw a bigger model at it." That answer is wrong, and I can show it is wrong with one micro-experiment.

I took the exact same embedder and embedded two things. First, the full chunk: class marker, signature, perfect docstring, plus the ~30 lines of body — the combinator handling, the _chain() call, the select_for_update checks, the NotSupportedError raises. Second, just the signature and docstring alone — call it the "purpose-only" version, the part that says what this is for with none of the machinery that says how it does it. Then I measured cosine similarity from each version to four query phrasings.

Purpose-only is closer to the query on every single phrasing, by +0.06 to +0.10 cosine. Same embedder, same docstring, same query. The only thing I removed was the implementation body — and the chunk got measurably more relevant to the thing it does. The signal that answers the query was in the chunk the whole time. The body was burying it.

Now the calibration that turns this from a curiosity into a recall failure. In this embedding space, for the "get a single object" query, the weakest chunk that made it into the 200-deep pool sat at a cosine of about 0.697. Purpose-only scored 0.706 — over the line, into the pool, in front of the reranker. The full chunk scored 0.601 — under the line, out of the pool, invisible. The entire difference between "the reranker gets a shot at the right answer" and "the right answer is never considered" is that +0.10 of cosine the body ate.

There is an obvious objection here: maybe I just phrased the queries badly. Maybe the right words would have pulled the chunk in. So I ran a 60-query sweep — 10 topics, 6 phrasings each — to give phrasing every chance to matter.

It didn't. Rephrasing shuffled which wrong answers came back; it did not surface the right ones. Ask for a "signal dispatcher implementation" and the top results were 1-to-6-line re-export stubs — the little shim modules that just re-expose a name — while the real Signal class, the thing that actually implements dispatch, was absent from the top three. Across the sweep, conceptual queries kept returning wrong or incomplete top-5 sets no matter how I said them. Phrasing is a knob on the query side. Dilution is a problem on the document side. Turning the query knob cannot un-average a document vector.

Then came the control that settled it. vectr also keeps a deterministic symbol graph: a plain lookup from a name to its definition site, no embeddings involved. For every canonical symbol that semantic search had just missed, the deterministic lookup resolved it exactly and instantly — Signal, BaseCache, Query, SQLCompiler, QuerySet.get, each to the correct file and line.

One more result, and it is the one that made me stop trusting cold semantic search on its own. I ran a round of "famous symbols" — targets every Django developer knows by heart. Cold semantic search was roughly a coin flip even there. get_object_or_404, QuerySet.get, and reverse were all absent from the top-5; ForeignKey came in at #2 and Paginator at #4, each sitting behind look-alikes with more generic wording. If a system cannot reliably surface get_object_or_404, the failure is not exotic. It is the common case wearing a docstring.

This one is a short aside, but it burned me while I was debugging the above, so it earns its place. While hunting the dilution bug I ran control queries for concepts that do not exist in Django core — CORS handling, for instance, which Django leaves to middleware and third-party packages. A search for something absent should come back empty, or at least visibly unsure.

It came back with five hits scored between 0.77 and 1.0, looking every bit as confident as a real match. Nothing in the corpus answered the query, and the system reported near-certainty anyway.

The reason is a modeling choice that is easy to make and easy to forget. The displayed score was the reranker's output after per-query normalization — rescaled so the best result of this query becomes ≈1.0. That rescaling throws away the only thing you needed: how good the top match is in absolute terms.

The practical rule that fell out of this: never show a per-query-normalized score as if it were confidence. Keep a non-normalized signal alongside it — a raw cosine, or a BM25 floor — so the system retains an honest way to say "nothing here is actually close." Recall failures like the dilution one are already invisible enough; a score that reads 0.99 over an empty result set makes them worse, because it converts a silent miss into a confident wrong answer.

The measurement points at its own fix. If purpose-only embeddings score +0.06 to +0.10 higher — enough, in the calibrated case, to clear the pool floor — then the answer is not to throw away the body vector. It is to also keep a purpose vector, and let a query match whichever one fits it.

That is dual-vector indexing. At index time, store two vectors per symbol: a purpose vector built from the qualified signature and docstring with the body stripped out, alongside the existing full-body vector. At query time, retrieve over both, and blend or take the max of the two similarities for pool entry. Nothing else in the pipeline changes.

What I like about this shape is that it does not privilege one kind of query. Intent-shaped queries — "get a single object from the database" — land on the purpose vector, where the docstring is undiluted. Implementation-detail queries — "where is select_for_update checked" — still land on the body vector, because that string only exists in the body. You are not trading one failure mode for its mirror image; you are giving each query the surface it needs.

Two properties make it safe to apply blindly across a whole corpus:

• Undocumented symbols degrade gracefully. No docstring? The purpose vector is just the qualified signature. It never gets worse than the name itself, and the name is often enough.
• It is a uniform structural transform. The same body-stripping rule applies to every symbol, at index time, with no query-side special-casing — no keyword lists, no "if the query looks conceptual, reroute it." That matters to me specifically, because query-side heuristics are the thing I have spent a long time deleting from this system; they are brittle, they compound, and they never generalize. A transform on the index side has none of those failure modes.

It is not free, though, and I would rather name the costs than let you find them. Storing a second vector per symbol roughly doubles the number of vectors in the index — reason enough to confirm dilution is actually your problem before you spend on it. And it does not stand alone: dual-vector composes with structural ranking signals like symbol importance rather than replacing them. A diluted docstring and an under-weighted call graph are different failure classes; fixing one leaves the other exactly where it was.

The honest sequence, stated plainly: I measured the cause first, then shipped the fix. The spike proved the mechanism — purpose-only embeddings clear the pool floor where full-chunk embeddings do not — and that measurement, not a hunch, is why dual-vector indexing shipped in vectr v1.0.0 on 8 July 2026. If someone tells you a retrieval change "should work," ask them for the cosine table. This one had one before a line of the fix was written.

The fusion bug hiding underneath

While validating the direction I tripped over a second, separate problem worth its own paragraph, because it will bite anyone running hybrid retrieval. Remember that BM25 ranked the target #1 for one phrasing. You would assume a #1 in either leg guarantees pool entry. It did not: the fused final top-60 did not contain the target even though BM25 had ranked it first. The fusion was dense-dominated, and the dense leg's absence outvoted the keyword leg's #1.

I found this in code search, but nothing about the mechanism is about code. Dilution appears in any corpus where a document mixes "what this is for" with "how it works" or with plain boilerplate. The purpose is a small fraction of the tokens; the pooled vector drifts toward the bulk; a query written in terms of purpose lands short.

You have almost certainly hit it without naming it:

• API reference pages where a one-line summary sits on top of exhaustive parameter tables and examples. Search for what the endpoint does and the parameter soup dominates the vector.
• Legal clauses buried inside pages of recitals and boilerplate. The operative sentence is three lines; the surrounding scaffolding is three hundred.
• Product descriptions embedded in spec sheets, where the one line a buyer would search for is outweighed by dimensions, SKUs, and compliance notices.

The two mitigations generalize as cleanly as the problem does. First, embed a purpose or summary field separately from the full text, and retrieve over both — the dual-vector idea, minus the word "symbol." A short, curated summary vector per document is often the single highest-leverage change you can make to recall, precisely because it is immune to dilution by construction. Second, and this is the cheaper habit to build: audit recall at the pool level, not just the final ranking. Most RAG dashboards show you the reranked top-k and nothing else, which means a pool-entry miss is completely invisible on the screen you are staring at. The failure that started this whole post would never have shown up on a top-k view. I only found it because I went looking one layer up.