Hyperscaler vs. Open Source: Can LanceDB Compete with VectorSearch?

For a recent client project, we built a RAG pipeline to help technicians search and navigate large volumes of technical documentation more efficiently. After deciding on Docling for parsing and chunking the source PDFs, we faced the next decision: how should we store and search the resulting embeddings?

Our initial candidate was Google Cloud’s Vertex AI Vector Search. We were already running on GCP, the integration was straightforward, and using a managed service is attractive in production: deploy an index, expose an endpoint, and let Google take care of the retrieval infrastructure.

However, for the projected corpus of roughly nine million vectors, our planned configuration came out at around CHF 1,900 per month. That was high enough to make us reconsider.

We started evaluating LanceDB as an open-source alternative. LanceDB’s tables and indices can be stored directly in Google Cloud Storage, potentially reducing our fixed database costs dramatically. But there was an obvious trade-off: LanceDB is an embedded library, not a managed search endpoint. Choosing it would mean becoming responsible for providing the infrastructure serving those searches ourselves.

Before switching to LanceDB, we therefore wanted to answer three questions:

• How much more effort is LanceDB to set up and operate?
• What infrastructure does it actually require for retrieval?
• Is the cost saving large enough to justify taking on that responsibility?

Here is what we learned.

Comparing architecture: Managed service vs. embedded database

At first glance, Vertex AI Vector Search and LanceDB solve essentially the same problem: given an embedding, find the most relevant vectors. Architecturally, however, they are very different products.

With Vertex AI Vector Search, the retrieval infrastructure is part of the service. You create an index, deploy it to an endpoint, configure machine types and replicas, and query that endpoint from your application. Google operates the actual serving infrastructure.

LanceDB takes another approach. Its tables, vectors, metadata, and indexes are stored as files and can live directly in object storage such as Google Cloud Storage. The LanceDB query engine then runs inside your own application. In our setup, the Lance table and indexes live in GCS, while our existing MCP service opens the table and performs the searches.

From the application side, this was surprisingly straightforward. LanceDB provides vector search, full-text search, filtering, and hybrid retrieval. Query-time parameters allow additional search work to be traded for potentially better recall: nprobes determines how many index partitions are searched, while refine_factor controls how many vector candidates are fetched and reranked using distances computed from the original vectors. Writes, index creation, and table optimization are also handled through the SDK.

That makes the data layer pleasantly simple. There is no separate vector database cluster or managed endpoint. But GCS is storage, not a search service, so the application still needs enough compute and memory to perform retrieval.

That distinction became the central point of our evaluation. We were not really comparing one vector database with another. We were comparing managed retrieval infrastructure with self-operated retrieval. To estimate whether our MCP server could handle incoming requests, we performed a dedicated load test.

Comparing performance under load

Our expected production traffic is moderate. More importantly, this is a RAG application where a retrieval step taking a few seconds is acceptable; we do not need a single-digit-millisecond vector search.

That matters because Vertex AI Vector Search is designed for low-latency ANN search at scale. For our workload, however, that level of serving infrastructure looked closer to overprovisioning than a hard requirement. The more relevant question was whether the compute already running our MCP service can also perform retrieval at acceptable latency and concurrency. We therefore focused the benchmark on end-to-end latency, CPU and memory consumption, stability under parallel requests, and how quickly performance deteriorated as load increased.

We estimated that the final document corpus would contain roughly nine million vectors, so that became the target scale for our test dataset. We used data based on the German deu_Latn subset of FineWeb 2 rather than client documents. The goal was to reproduce the scale of our expected vector workload, not the semantic characteristics of technical PDFs, as this benchmark was not intended to systematically measure retrieval quality or recall.

Each request followed the same application path we expect to use in production: the query was embedded through Vertex AI, passed to LanceDB for hybrid search and reranking, and reduced to the top five results. We used an IVF_RQ index, which partitions the vector space and stores compressed vector representations to reduce the search workload and index size. We configured each search to probe 40 partitions (nprobes=40) and retrieve twice the requested number of vector candidates for refinement using the original vectors (refine_factor=2). We chose these settings based on our experience and estimated production needs. Higher nprobes and refine_factor values increase CPU work and can also increase memory usage.

For the benchmark itself, we generated 700 different queries, each based on a randomly selected chunk from the corpus. This was important for the memory test - repeating the same handful of queries would increasingly exercise already-cached index data and create a much smaller working set. In production, we actually expect some degree of repetition, but we intentionally chose the most pessimistic scenario. By continuously querying different areas of the corpus, we tried to expand the active working set as much as possible and force LanceDB to keep fetching and caching new data. The goal was to stress our MCP setup under a deliberately demanding scenario. If the service could handle a broad, constantly changing working set, we could be more confident that a production workload with more repeated access would not require more resources than our test suggested.

We then increased concurrency from sequential requests to 2, 5, 10, 15, 20, and 30 simultaneous searches. Including the first cold request, the benchmark consisted of 731 requests. We measured end-to-end latency, errors, CPU utilization, and memory consumption.

What happened under load?

With our setup, the first cold request took around 22 seconds. Once the process and relevant index data had warmed up, sequential requests were much faster, with a median end-to-end latency of around 2.5 seconds.

As concurrency increased, latency rose steadily:

All 731 requests completed successfully. The service therefore remained stable even with 30 simultaneous searches; what degraded under load was response time.

CPU utilization increased with parallelism, but the container was not simply pinned at 100% CPU throughout the run. Even during the highest-concurrency phase, median CPU usage remained around 64%.

This suggests that the scaling behaviour is not explained by CPU alone. Searching a GCS-backed LanceDB table also involves remote reads, cache lookups, and shared retrieval resources. As more searches run at the same time, requests can increasingly compete for those resources.

For our expected workload, that is a useful result. We do not normally expect 30 technicians to issue searches at exactly the same moment, and at lower concurrency the latency is much closer to what we consider acceptable. Even when pushed beyond that expected load, the service continued processing requests without failures.

How much RAM do nine million vectors require?

Memory was probably our biggest concern before running the benchmark.

Fortunately, when querying LanceDB, the table and its indexes remain in GCS, and opening the table does not materialize all nine million vectors in application memory. Instead, LanceDB loads the index structures and candidate data required for each search and keeps frequently accessed information in an in-process cache.

The IVF index helps make this possible. Rather than searching every vector, LanceDB selects the most relevant partitions according to nprobes and searches those. As queries touch different parts of the index, more data enters the cache. If a later search needs the same data, it can be reused directly from memory. Otherwise, LanceDB retrieves the missing data from GCS. Crucially, accessing these partitions does not require downloading entire index files from GCS. The Lance storage layer can use byte-range requests to fetch the portions of remote files needed for a search. Together, selective index access and partial file reads allow retrieval without loading the full dataset into memory.

Our 700 different queries deliberately stressed this mechanism by continuously touching different areas of the vector space. This should be read as a relatively pessimistic memory scenario rather than a prediction of normal production behaviour. To confirm our assumption that memory usage plateaus once queries can benefit from caching, we conducted a second run using only 100 distinct queries, which were executed repeatedly in a loop.

The resulting session memory consumption of both runs is shown below.

With the cache-unfriendly setup, cache growth was gradual, as expected. By the end of the run, the session contained roughly 5,362 cached items. This was one of the most important findings for us: even under a workload specifically designed to continuously expand the working set, total memory consumption was around 1.1 GB. Our assumption about repeated queries was also confirmed: Queries that were executed repeatedly benefited from caching and did not lead to any further increase in memory usage. Response time improved slightly, as cached data could be reused without requiring additional read operations from GCS.

By default, LanceDB can technically consume up to 6GB and 1 GB for its index and metadata cache, respectively. However, LanceDB allows the size of its in-process caches to be constrained through index_cache_size_bytes and metadata_cache_size_bytes:

session = lancedb.Session(
    index_cache_size_bytes=...,
    metadata_cache_size_bytes=...,
)

This gives us another useful deployment control. A larger cache keeps more index structures warm and reduces repeated reads from GCS. A smaller cache makes memory usage more predictable, but entries are evicted sooner and may need to be fetched again from object storage later.

The practical sizing question is therefore not “Do nine million vectors fit into RAM?” They do not have to. The more relevant questions are how large the active working set is, how much of it we want to keep warm, how much parallel traffic we expect, and how much additional GCS latency we are willing to accept after cache eviction.

What about Vertex AI Vector Search performance?

We did not run the exact same end-to-end workload against Vertex AI Vector Search, so comparing our seconds-level LanceDB measurements directly with Google’s published millisecond figures would be misleading.

Google’s public benchmarks demonstrate what a dedicated distributed search service can achieve. For example, Google reports p95 latency of 7–9 ms at roughly 3,000 QPS for 768-dimensional datasets with 10–100 million vectors using two replicas; it also supports managed scaling by adding replicas for higher throughput. [1]

But those tests measure the vector-search layer under controlled conditions. Our benchmark included the query embedding, hybrid vector and full-text retrieval, reranking and remote GCS access. Additionally, our measurements depend heavily on the machine used to run the load test. The meaningful takeaway is therefore simpler:

Vertex AI Vector Search is designed to provide predictable, low-latency retrieval at significant concurrency. Our LanceDB architecture was optimized much more strongly for low fixed cost and simplicity of storage.

Comparing cost

For our expected workload, the planned Vertex AI Vector Search deployment resulted in an estimated ~CHF 1,900 per month. This introduces a recurring serving cost even when query traffic is relatively modest.

LanceDB is free. Its persistent database files can simply live in GCS. For our projected footprint, we estimated GCS costs at roughly CHF 20 per month—rather than a four-figure managed-search bill. In our case, much of the required application infrastructure already existed, so adding LanceDB did not introduce meaningful additional compute and engineering cost. That made economics particularly attractive.

The workload also matched the trade-off well. We expected moderate demand rather than thousands of searches per second, and a response taking a few seconds was perfectly acceptable for the application. Paying for infrastructure optimized for extremely low and predictable vector-search latency therefore provided relatively little additional value to us.

This does not mean the additional performance of Vertex AI Vector Search is unnecessary in general. For any latency-sensitive workload with substantial parallel traffic, predictable millisecond-level retrieval can be an architectural requirement rather than a luxury.

Therefore, the calculation could look different in other cases. If you need several permanently warm replicas and strict latency guarantees, Vector Search is worth considering.

Conclusion

The experiment changed how we framed the decision. It is tempting to describe the choice as:

CHF 1,900 managed database vs. CHF 20 open-source database.

But that misses the actual trade-off. A better formulation is:

CHF 1,900 for managed retrieval vs. inexpensive storage plus infrastructure that we have to operate ourselves.

Vertex AI Vector Search buys us dedicated serving infrastructure, predictable performance, managed scaling and a much smaller operational surface. LanceDB gives us an open format, very inexpensive object storage and the freedom to size retrieval infrastructure around the workload we actually have.

For some use cases, the latency we observed even under moderate parallelism could quickly become unacceptable. If adding more retrieval resources is not feasible, Vertex AI Vector Search becomes an attractive low-maintenance alternative.

For our use case, however, our load test showed that LanceDB was absolutely feasible. Memory requirements were modest, the service stayed stable even at 30 concurrent requests, and the potential cost reduction was substantial.

The final question is not which vector database wins a feature comparison. It is: For your workload, what costs more: the managed endpoint, or owning the infrastructure that replaces it?

For our setup, the choice was easy: we decided to go with LanceDB.

References

1. Lewis, E. (2024). How Vertex AI’s vector search helps unlock high-performance gen AI apps. Google Cloud Blog, December 10, 2024. https://cloud.google.com/blog/products/ai-machine-learning/build-fast-and-scalable-ai-applications-with-vertex-ai