Achieving Fast Vector Search in MariaDB

MariaDB Server 12.3 makes vector search faster where it matters most: the high-recall levels that production AI workloads actually require. A new Matryoshka-aware optimization uses a cheap check on the first slice of each embedding to discard far-away candidates, then confirms the close ones with the full vector — so search stays just as accurate while doing far less work. 

The result is higher throughput, lower compute cost per query, and no added developer complexity.

• Faster production vector search: on the dbpedia-openai-1000k benchmark, MariaDB 12.3 delivered up to 30% higher queries per second (QPS) than MariaDB 11.8 at the same recall.
• Built for real recall targets: the gains show up at the high recall levels production applications need — typically 0.94 and above — not just at the lower recall where benchmarks look flattering.
• No schema or query changes: the optimization turns on automatically when the data supports it. Developers do not choose a prefix length, maintain extra indexes, or build a two-stage retrieval pipeline.
• Lower infrastructure cost per query: skipping unnecessary full-vector comparisons reduces the work each search performs.
• Less operational sprawl: for many workloads, teams can run transactional data, JSON, analytics, and vector search in one database instead of adding a separate vector engine.
• Forward compatibility: as higher-dimensional and Matryoshka-style embeddings become more common, MariaDB exploits their prefix structure without locking teams into model-specific schema decisions.

For developers, that means faster vector search on upgrade. For businesses, it means faster AI applications with less infrastructure, fewer moving parts, and lower operational cost.

What’s the problem with vector search at production recall?

Vector search powers the retrieval layer in almost every modern AI application: RAG over enterprise knowledge bases, semantic product search, recommendations, and fraud detection. As datasets grow, the distance calculations behind that search become the expensive part — and in each of these systems, two numbers ultimately drive cost:

• QPS — how many vector lookups per second a database can serve.
• Recall — how often the search returns the correct nearest neighbours.

In benchmarks it is easy to chase QPS at recall of 0.80 or 0.90. In production, that is rarely where applications run. A RAG pipeline that misses the right chunk one time in five is a RAG pipeline that hallucinates. Real workloads target recall of 0.94 and above — and that is exactly where every vector engine slows down sharply, because each additional point of recall costs more distance calculations against high-dimensional embeddings. So the real engineering question is not “How fast is your vector search?” but “How fast is it at the recall your application requires?”

How the Matryoshka optimization works

A growing class of embedding models — including OpenAI and Gemini embedding models, open-source Sentence Transformers, and Matryoshka-style models — produce embeddings whose lower-dimensional prefixes preserve useful semantic information. The name comes from Russian nesting dolls: the first slice of the vector already carries the coarse meaning, and each further slice adds detail. This means a low-dimensional prefix is often enough to tell whether two vectors are even in the same neighborhood.

MariaDB Server 12.3 applies exactly this at search time, in a single automatic pass. For every candidate the HNSW graph touches, the engine first computes a partial distance over just the first 192 dimensions. That partial distance is a reliable predictor of the full 1,536-dimensional distance: if it already places a candidate well outside the range that could matter, the engine discards it without ever computing the rest. Only the candidates that survive this cheap filter get the full, precise distance calculation. The final ranking is based on full distances, so recall is preserved and the engine has avoided full computations for many candidates that were never going to make the cut.

Crucially, both stages happen inside the engine, on the same index, in the same query. It is the two-stage filtering pattern that teams normally build by hand — but MariaDB does it internally and automatically.

What the Matryoshka benchmark shows

The dbpedia-openai-1000k benchmark comprises one million 1,536-dimensional OpenAI embeddings. It is a general-purpose vector-search benchmark used to evaluate how engines trade QPS against recall, which makes it a good proxy for the real-world efficiency and infrastructure cost of a vector database at production recall.

In a recent 10-database benchmark on the dbpedia-openai-1000k dataset — one million 1536-dimensional OpenAI embeddings — the field separates into three clear tiers at 94% recall:

Tier	Databases	QPS at 94% recall
Top	MariaDB, RediSearch	850–1000
Middle	Weaviate, pgvectorscale, pgvecto.rs	470–570
Trailing	pgvector	250
Bottom	OpenSearch, Qdrant, Milvus	90–250

MariaDB 12.3 sits in the top tier at 94% recall, delivering 850–1000 QPS alongside RediSearch, and pulls into a clear lead above 95%.

Compared with the previous MariaDB Server 11.8: 12.3 is faster across the high-recall range, delivering up to 30% more QPS at the same recall. The new Matryoshka-aware optimization does the work.

Developer Advantages of MariaDB for Matryoshka Embeddings

The optimization turns itself on. Developers do not need to know whether their embedding model is Matryoshka-trained, choose a prefix length, maintain multiple vector indexes, or build the two-stage retrieval pipeline themselves.

MariaDB observes the relationship between prefix distance and full-vector distance on the actual dataset. If the prefix is a reliable predictor, the optimization is enabled. If it is not, MariaDB continues using the standard full-distance computation. The same SQL, the same vector column, and the same index work in both cases.

To our knowledge, MariaDB Server 12.3 is the first general-purpose database to apply this kind of Matryoshka-aware distance pruning automatically at query time. Other engines that support Matryoshka embeddings push the work onto developers — choosing a prefix length, maintaining parallel indexes, and wiring up two-stage retrieval in the application. 

That difference is operational:

• No data-modelling overhead. Teams do not need to understand Matryoshka embeddings, or which model produces them, to benefit.
• No application changes. Existing queries against MariaDB Vector get faster on upgrade — the optimization needs no schema, index, or query changes.
• No fragile manual tuning. Switching embedding models — Matryoshka to non-Matryoshka or back — does not force schema migrations, index rebuilds, or retrieval-layer rewrites. The database detects the change and adapts.
• It can help even on non-Matryoshka data. In practice, many embedding spaces show usable prefix structure even when the model was not explicitly trained for it. MariaDB detects this and exploits it when present.

Try it

MariaDB Server 12.3 with the Matryoshka-aware optimization is available now. Existing MariaDB vector search users can upgrade in place — no schema changes are required for the optimization to take effect.

• MariaDB’s vector documentation
• Talk to us about your AI workloads

Benchmark details: dbpedia-openai-1000k dataset (1,000,000 vectors, 1,536 dimensions, angular distance), run on Intel Xeon E5-2660 v4 (AVX2), using a fork of ann-benchmarks with MariaDB support. QPS figures are approximate tier ranges from the published results; verify the latest numbers against the source benchmark.

Frequently Asked Questions