Vector search is the memory behind semantic search and most RAG systems. Swap in a newer, better embedding model, and that memory quietly breaks, with no error and no warning. Here is why it happens, and how to upgrade without it.

In 2025 Google shipped gemini-embedding-001. It topped the multilingual benchmarks, beat Google's own previous models, and swapping it in looked like a one-line change. Teams that changed that line and re-embedded nothing watched their retrieval quality drop. The new model was fine. The catch was that the old and new models produce vectors in different coordinate systems, and until every document is re-embedded, the index holds vectors from both models at once.

This catches good teams because nothing about it looks like failure. There is no error message. The search runs as fast as it did before. It just ranks by a number that has stopped meaning anything. What you're really running is a data migration with a consistency window, a period where the old and new models both live in one index. In that window recall drops, and nothing in your logs says why. Most teams treat the one-line swap as a quick, low-risk change. It is a slow and expensive one, because the damage stays hidden for weeks and you pay it back in debugging time nobody ever traces to the upgrade.

An embedding model is a coordinate system

"The vector generated from a new user query wouldn't map cleanly to a vector space created by an older embedding model."

— Deepti Naidu, Weaviate

Every embedding model has its own coordinate system for placing text in space. Similar things land near each other in that space, but only if embedded by the same model. Measure the distance between an old-model vector and a new-model one and the number means nothing. So the two can't share an index, and a real upgrade means running every stored document through the new model again, a step a lot of teams miss.

Google's own models show both cases. The old text-embedding-004 turns each document into a vector of 768 numbers, the new gemini-embedding-001 into a vector of 3072. When the sizes differ, the mismatch throws an error and you catch it right away. But gemini-embedding-001 can be shrunk back to 768 to fit the old index, so the sizes match, no error shows, and the mismatch runs silently. That same-size swap is the dangerous one.

The half-migrated index does not fail loudly. It fails silently. The search still returns ten results, ranked confidently, by a quantity that means nothing.

It looks like everything except the real cause

It takes days to debug because the failure looks like something else, bad data, a weak model, a broken reranker, all checked before you suspect the migration.

Every row is the same bug in disguise, a query and a document compared across two coordinate systems. A dashboard won't catch it, because it shows an average, and the average is mostly fine.

A worked example

To watch this up close, I ran a version of the migration. I embedded about 5,000 documents from SciFact, a standard retrieval benchmark, twice, with two different open models of the same size, one standing in for the old model and one for the new, and searched with the new one. It is far smaller than the Google case, but the moving parts are the same, and two things stood out.

First, how far recall drops depends on how different the two models are. These are closely related (their nearest-neighbor lists agree 43% of the time), so swapping the query model costs about 9% of recall, not a collapse. The opening's text-embedding-004 and gemini-embedding-001, trained separately, would fall much further. The point isn't the 9%, it's that you can't know your own number without checking your own two models.

Second, the surprising part. You'd expect a half-migrated index to be worst on day one and improve as documents convert. It isn't. Recall drops below where it began the moment migration starts, bottoms out around 10 to 20% done, then climbs back, the same across every random order I tried.

The mix is the problem. Fully old and fully new both work. It is the in-between, with only a fraction migrated, where recall falls to its worst.

So a half-migrated index is worse than either pure one. The migrated documents rank higher against a new-model query, so even a few of them take the top spots early, pushing the relevant old documents down before most of them have moved.

The migrated documents win by default. Against a new-model query they score higher across the board, filling the top results and pushing un-migrated ones, some the query needs, below the cutoff.

"Scores are not directly comparable across embedding models due to differing geometric properties, complicating model migration and limiting threshold reuse."

— Rozmus & van der Putten, on comparing embedding models

The fix is one search engineers already use. Put the two groups' scores on the same scale before ranking them together. Normalize each separately and the dip almost disappears, the worst point rising from 0.32 back to 0.59 (the second column above). It's the same move hybrid search uses to combine keyword and vector scores.

How to detect the silent recall drop

The number everyone watches, average quality across all traffic, is the one that buries it. So stop averaging. Tag every vector with the model that made it, and track quality separately for queries whose answers are already re-embedded and those that aren't. When the second group drops while the first holds, you're watching the problem happen live.

No labeled data? Use canary queries. Record what a handful of fixed searches return before you start, re-run them on a schedule, and count how many survive. Top results that churn for no reason are being scored across two spaces. Either check is a cron job, not a research project.

Re-embedding is the cost you can plan for

"Reprocessing for any model change might be easy for small-scale applications with a few million data points. Still, it quickly gets out of hand with larger-scale evolving datasets in production."

— Jo Kristian Bergum, Vespa

At $0.02 to $0.13 per million tokens, the API cost stays somewhat low. Re-embedding a dataset the size of MS MARCO v2 (about 138 million short passages) works out to somewhere between a few hundred and a couple thousand dollars. Time is the harder limit. Encoding it takes many GPU-hours, and rebuilding the index on top pushes a large migration into days, all with the index mixed. You can budget the money. The quality you lose in that window never shows up on an invoice.

The vectors are not the only thing that moved

This is the step that turns a re-embed into a full reindex, and most upgrade plans miss it. An index isn't just vectors. Engines build helper structures on top, the compressed lookups that save memory and the graph that finds nearest neighbors fast. Both are built from the old vectors, so both go stale the moment you swap models. Rebuilding them means a full reindex, not a quick field update, the real reason "just update the vector field" isn't migrating.

Three ways through the migration

There are three ways to do this, and they're not equally good. For a search that your users rely on in real time, the answer is dual-write with shadow-read; the other two are for internal or offline search where a little downtime is fine.

Full reindex with an atomic swap. Build a complete second index in the new space (some engines automate this as a background reindex), then switch everything to it at once. It never serves a mixed index, the whole point. You pay in either downtime or two full indexes at once. When your data fits in a second index, this is the safe default, and anything cleverer solves a problem you don't have yet.

Dual-write and shadow-read. Write both embeddings for every document, keep serving the old space, and run the new one in shadow mode, scoring the same live queries but never showing the results. When they're good enough, switch reads over and drop the old field. Zero downtime, instant rollback, temporary extra storage. For any search real users rely on, this is the right call. Most teams skip it not by choice, but because running two embeddings at once is hard on their storage layer.

Drift adapters. Instead of re-embedding, fit a formula that maps your old vectors into the new space, from a sample run through both models, and apply it to the rest. It is cheap, but a single formula is only an average. It holds where the new model shifted the whole space the same way, and breaks down where it rearranged which concepts sit near which. Fine for rarely-searched documents, risky as your whole answer. The Drift-Adapter paper reports recovering 95 to 99 percent of recall. My own test recovered nearly all of it, but only because the two models were close.

Your storage engine decides how hard the migration is

How hard this is comes down to your storage engine. In many vector databases an embedding is tied to its document, so changing it means rewriting the whole record, which turns dual-write into a custom pipeline and pushes teams toward the risky shortcut of swapping the model in place and leaving the documents alone. Engines that treat the vector as one field among many make the safe option cheap. A partial update rewrites a single field, and more than one embedding per record lets the old and new spaces sit side by side until you switch ranking to the new one. Vespa, Elasticsearch, OpenSearch, and pgvector sit at different points on this range, and the easier your engine makes a field-level change, the easier every future upgrade.

What to remember

The important point isn't which embedding model or storage engine you pick. It's that swapping an embedding model is a data migration, not a config change, and it now shapes the accuracy, cost, and reliability of everything built on top. As better models keep shipping and indexes keep growing, the teams that stay reliable are the ones that version every vector, watch the window where two models overlap, and run the next upgrade on purpose, instead of finding the quality drop weeks too late.