Does your RAG app need a separate vector database? Databricks measured search in Postgres at 4M passages
A Databricks engineer sized hybrid vector and keyword search in Lakebase Postgres with a real app and public corpora. The numbers give teams a way to decide where retrieval should live.
Listen to this article · 5 min
AI-generated narration of the full article.
Many enterprise RAG projects start with an architecture decision nobody examined: add a vector database. The application data stays in its transactional database, embeddings go to a separate store, and a synchronization pipeline joins them. That is three systems, two copies of the data and a freshness problem before the first question is answered.
A Databricks engineering post from October 15 tests the alternative with numbers. Philipp Tiefenbacher sized hybrid search, vector plus BM25 keyword search fused with Reciprocal Rank Fusion, inside Lakebase Postgres, with the app data, the vector index and the keyword index in one instance. He used an internal app with about 23,000 slide rows and public MIRACL English passages at 1, 2 and 4 million rows, with vector indexes of 5.8, 11.5 and 23 GB. Lakebase Search itself became generally available in September; this is a measurement study, not a launch.
What the measurements say
Four findings are worth carrying into any design review, with the caveat that they come from one corpus, one embedding model and synthetic query mixes:
- Size the cache from the vector index, not the row count. The ratio of cache to index size predicted hit rate across corpus sizes: 45% at a ratio of 0.07, 86% at 0.57 and 97% at 1.14. Warm median vector latency fell from 187 ms to 4 ms as the ratio rose.
- Fewer candidates cost little. Cutting the candidate pool from 1,000 to 200 kept 90.5% overlap in the top 10 results, touched about a third of the pages and cut cold latency from 740 ms to 387 ms. Overlap with the larger pool is not the same as retrieval quality, but it is a useful bound.
- Recall is a setting. The default index gave recall@10 of about 0.74 to 0.75; a quality build mode raised it to 0.80, and to 0.97 with more probes, at a latency cost.
- The bottleneck moves. Up to about 4 compute units the database was the limit; above 8, the app workers were. Connection pooling at 50 users nearly doubled throughput and cut p95 latency from 2.4 to 0.7 seconds.
On cost, a 14-minute load test served 109,602 requests for $0.13, of which $0.02 was Lakebase compute. That covers one short test, not a month of production; the author recommends keeping a small always-on minimum, about $31 to $40 a month at list price, to avoid cold starts of around a second.
A framework for where retrieval lives
The study does not compare Lakebase with dedicated vector databases. It does make the decision concrete enough to frame:
| Points to search in the app database | Points to a separate service | |
|---|---|---|
| Freshness | Results must reflect each write immediately | Minutes of lag are acceptable |
| Where the corpus lives | In the app's own tables | Across many sources, shared by many apps |
| Scale | Index fits in memory you are willing to pay for | Index far beyond one instance's cache |
| Permissions | Row-level rules already in the database | A separate permission model per source |
| Team | No appetite to operate another system | A platform team already runs retrieval |
The permissions row is the one teams underweight. When search runs in the database that already enforces who can see which rows, retrieval inherits those rules. A separate index needs its own copy of the permission model, kept in sync, and every sync is a chance to leak. That is the custody question applied to embeddings: each copy is a new place data lives.
Measure before you choose
The author’s own advice is the most transferable part: measure recall on your own data, because it depends on the corpus and the embedding model. That echoes our argument that retrieval should be measured as its own system, with its own metrics, before anyone debates which product hosts it.
A practical sequence for a new RAG application: start with search where the data already lives, build a small evaluation set from real questions, measure recall and latency at your expected corpus size, and move retrieval to a separate service only when a specific requirement on the right side of the table forces it. The extra system should be a conclusion, not a starting assumption.