A gradual embedding migration should dip and recover. In my test recall fell below both the index I started with and the one I ended with, and bottomed out with less than a third of the documents moved. So I'd keep every live query inside one embedding space for the whole migration. That rules out the one-line model change, where you swap the query model in config and re-embed as you go, and leaves the choice between a full reindex and dual-write to rebuild time, write catch-up, and your engine.

What the migration window actually looks like

I simulated the migration on a benchmark. I embedded 5,000 documents from SciFact with two 384-dimensional models, all-MiniLM-L6-v2 standing in for the old and paraphrase-MiniLM-L6-v2 for the new, then searched with the new one using exact nearest-neighbor search. On this benchmark the one playing the new model is the weaker retriever, so the test is about the migration window, not the value of an upgrade. Then I re-embedded documents in random order, a slice at a time.

I expected a dip, worst on day one and climbing as documents moved over. By the time a tenth of the documents had moved recall sat under the 0% row in the table, bottomed out between 10% and 30% done, and only then climbed back. All five random orders dipped below both the start and the finish.

Before anything moved, old queries against old documents scored 0.78. Switching the query model alone took that to 0.59, the 0% row below. The migration then took it to 0.33.

Figure 1. SciFact recall@10 by share of documents re-embedded, mean of five random orders, new-model queries throughout.

How far recall falls depends on how far apart your two models are, so I ran a distant pair from a different family, all-MiniLM-L6-v2 to bge-small-en-v1.5. Swapping only the query model cost the first pair about 9% against its finished index, 0.65 to 0.59. The second pair lost 74% on the same swap, from 0.84 to 0.22, and it bottomed out earlier and lower, at 10% migrated in all five orders, down to 0.10 on average. Nearest-neighbor overlap, how often the two models' top-10 lists agree, did not predict this. The distant pair's lists agree more often, 53% against 43%. I'd measure the query swap on my own documents before trusting either number.

To check this at scale I repeated the whole run on FiQA, 58,000 finance documents, eleven times the size of SciFact. With the close pair, the two MiniLMs, it bottomed out at 10% in all five orders, recall falling from 0.18 to 0.07, and migrated documents took 97% of the top-10 slots. The distant pair's query swap alone dropped recall to 0.05, so there was little left to lose, and the migration was a climb from near zero. In these runs the low point showed up only where the query swap left some recall to lose, so measure that swap first.

Why the worst point comes early

Against a new-model query, migrated documents score higher on average than ones still in the old space. Their mean cosine similarity to the query was 0.15, against 0.08. At 10% migrated they held 84% of the top-10 slots, and at 20% they held 92%. Early on almost every relevant document is still old-space, so almost all get pushed out.

Figure 2. Mean cosine to a new-model query is 0.15 for migrated documents against 0.08 for old-space ones, so migrated documents fill the top slots.

"[S]cores are not directly comparable across embedding models due to differing geometric properties, complicating model migration and limiting threshold reuse."

— Rozmus & van der Putten, on comparing embedding models

You can fix the scoring by putting each group's scores on its own scale before ranking them together. Normalize each space separately and the drop nearly disappears, the worst point rising from 0.33 back to 0.60 in the normalized column, about where you started.

Everything above is exact search, where every document is scored. A production index walks an approximate graph and only scores the candidates it reaches. So I reran the close pair on an HNSW graph (M 16, ef_search 50, which matched exact recall on the finished index within 2%), both updating a graph built on the old vectors in place and rebuilding it over the mix at each step. The graph barely moved the raw low point, 0.31 to 0.32 against 0.33.

The graph left the low point where it was, but it broke the normalization fix. On the in-place graph, normalizing per space over the candidates it returned recovered only to 0.38 at 20% migrated, against 0.60 in exact search, because by then the graph reached just 40% of each query's relevant old-space documents, down from 79% before the migration began. Its links were built from distances across two spaces, and they steer a new-space query away from the old-space documents it needs. Normalization can only reorder what the graph returned. Four times the candidates, ef_search 200, lifted the fix to 0.48 and reach to 75%, still short of 0.60. On FiQA, at 10% migrated the graph reached 9% of each query's relevant old-space documents, down from 37%, and normalization recovered 14% of the loss.

I also ran the migration the other way round, documents updated in place under the old query model and queries switched last. The low point moved to the other end, worst at 80% for the close pair and 90% for the distant one, and the close pair's 0.42 sat below both its 0.78 start and the 0.54 it reached once every document had moved but queries were still old. The index is worst when a small but real share of documents sit in the query's space, at 10% to 30% if queries switch first and 80% to 90% if they switch last.

How I'd run the migration

The constraint I'd put in the migration plan is that no live query ever scores documents from both spaces. With models close enough to have a low point, a tenth of your documents migrated puts you below the 0% row, and any strategy that lets both spaces answer the same query runs your users through that low point. Migrating one shard or tenant at a time is fine, as long as each query stays inside one space.

Under an hour, switch reads in one move

Rebuild time decides the strategy. If your whole index rebuilds, or a second field backfills, in under an hour, I wouldn't build a dual-write pipeline. I'd validate and switch reads in one move, whatever the engine. In that window, pause writes or replay what lands during the rebuild. I use an hour because that is about where a write pause stops fitting a maintenance window. If your users can't take the pause, I'd treat the job as past the hour.

Past an hour, check catch-up first, then the engine

Past the hour, a rebuild has to replay everything that landed while it ran. If it can't catch up, I'd dual-write into a separate table or index and shadow-read it, whatever the pipeline costs. A Postgres rebuild with a replay backlog that keeps growing is that case. If it can catch up, the engine decides. On an engine whose documents hold two named vectors and can update one without resending the rest, I'd dual-write with shadow-read. On one that rewrites the record to change a field, I'd do a full reindex and an atomic swap.

Vespa lets a document hold more than one embedding field and update one without resending the rest. There, adding the new space is a schema change, a backfill of the second field, and a rank-profile switch. A Vespa backfill that finishes in twenty minutes, with writes paused or replayed inside it, falls under the hour, so the backfill and the rank-profile switch are the one move and I wouldn't run a shadow. Past an hour on an engine like that, my default is dual-write with shadow-read.

"Reprocessing for any model change might be easy for small-scale applications with a few million data points. Still, it quickly gets out of hand with larger-scale evolving datasets in production."

— Jo Kristian Bergum, Vespa

Dual-write saves you the second full index and the write pause, and the switch is one field change. Shadow-read tells you when the new space is good enough to switch. Its bill is double embedding during the backfill, double scoring while the shadow runs, and two vectors per document until you drop the old field. Stamp each embedding with the document revision it came from, so a backfill result that lands after a live update can't overwrite it.

Run the shadow over a full traffic cycle and switch reads once the new space comes within 0.02 of the old one on recall@10 against the evaluation set. The new model in my SciFact run would fail that check even fully migrated.

If a rebuild catches up and a second embedding field means rewriting every record, I'd build a fresh index and switch reads atomically instead. Check it against your evaluation set, then switch every read in one move. Its bill is a second full index during the build and a write pause or replay at the switch. It has to be a full reindex, because the quantization tables and the HNSW graph came from the old vectors and go stale. Elasticsearch and OpenSearch sit on Lucene, which rewrites the whole document to change a field, and Postgres rewrites the row. For those I prefer the temporary cost of a fresh index to a second embedding in every record, so an Elasticsearch rebuild that takes two hours and keeps up with writes gets a full reindex and an atomic swap.

Having no evaluation set doesn't change which default you land on. I'd label a few hundred logged queries first and validate against them before following the engine's default.

Two other fixes

A drift adapter fits a small transform between the two spaces from a sample of documents run through both models. The Drift-Adapter paper maps new-model queries back into the old space, so the existing index stays untouched, and reports recovering 95 to 99 percent of recall. My own test, mapping old documents the other way, was scored on documents it was fitted on, so I have no clean number. Normalization, for when you are already mixed and can't stop, recovered about a quarter of the SciFact loss and a seventh of FiQA's at the default search width. It needs a model tag on every vector and a rescoring pass.

The hierarchy

Row one applies to every route, the first migration row that fits wins, and the last row adds to that choice.

What I'd tell a team before the next model change

I'd version every vector with the model that made it and track quality separately for migrated and unmigrated documents, so the next change is planned.