What Is a Vector Database? Beyond SQL

A vector database is a database built to store and search high-dimensional embeddings — lists of numbers that represent the meaning of a piece of text, an image, or a sound. Instead of matching rows where email = 'x', you ask a very different question: "which of these ten million vectors sit closest to this one?" That single change in the question — from exact match to nearest neighbor — is why a normal relational database chokes on this workload and a specialized one does not. Vector databases now sit underneath most of the AI features people use daily: semantic search, product recommendations, fraud-pattern matching, and retrieval-augmented generation (RAG), the technique that lets a chatbot answer questions from documents it was never trained on.

Why a relational database can't just do this

Postgres, MySQL, and MongoDB were built around a promise: give me an exact key or a range, and I'll find matching rows fast using a B-tree index. That promise breaks down completely once you're asking "what's similar to this," because similarity in a 768- or 1,536-dimensional space isn't something a B-tree can index the way it indexes a customer ID.

Three specific gaps show up. First, there is no cheap way to compute "closeness" between two vectors and use an index to skip the ones that are obviously far away — a brute-force scan of even 10 million vectors, each compared with a dot product, is too slow for a search bar that needs to respond in under 100 milliseconds. Second, standard databases have no built-in operator for cosine distance, dot product, or Euclidean distance that also benefits from indexing; you'd have to write that math yourself, in application code, on every row. Third, vectors are heavy: a single 1,536-dimension embedding stored as 32-bit floats takes roughly 6 kilobytes, and at 10 million rows that's around 60 gigabytes before you've stored a byte of the actual content.

None of this is unsolvable — Postgres itself can be extended to handle it, which is the whole premise of pgvector, covered below — but the underlying math (approximate nearest neighbor search) needed purpose-built index structures that took years to mature into something production-grade.

pgvector 0.8, released in 2024 and now the default choice for most Postgres-based RAG stacks, added iterative index scans specifically to fix a hidden failure mode: with the default HNSW search width, a filtered query where only 10% of rows match a `WHERE` clause was returning as few as four results out of a requested ten, because the index stopped searching before it found enough candidates that also passed the filter. Iterative scanning keeps searching until the filter is satisfied. Source: [Crunchy Data, pgvector 0.8 release notes](https://www.crunchydata.com/blog/hybrid-vector-search) and [pgvector 0.8.0 announcement](https://www.postgresql.org/about/news/pgvector-080-released-2952/).

How similarity search actually works

The mechanism has two layers: turning content into vectors, and then searching those vectors fast. Understanding both is the difference between picking a vector database that works and one that quietly returns garbage.

From text to numbers: what an embedding actually is

An embedding model — something like OpenAI's text-embedding-3-small, Cohere's embed-v4, or the open-source bge-m3 — takes a chunk of text and outputs a fixed-length list of numbers, typically 384 to 3,072 of them. The model was trained so that pieces of text with similar meaning end up as vectors that point in similar directions in that high-dimensional space. "The cat sat on the mat" and "a feline rested on the rug" produce vectors that are close together, even though they share almost no words. This is the mechanism that makes semantic search possible: you're not matching keywords anymore, you're matching meaning as the model encoded it.

Think of it like a library where every book gets assigned a precise coordinate in a vast, invisible map based on its subject matter, tone, and themes — not its title or author. Two mystery novels about grief written in different decades would land near each other on that map even if they never use a single overlapping sentence. A vector database is the system that stores that map and can instantly tell you which books sit closest to any point you give it.

Why exact search doesn't scale, and what ANN does instead

Once you have millions of these coordinate points, finding the true nearest neighbors means comparing your query vector against every single stored vector — an operation whose cost grows linearly with the size of the collection. That's fine at 10,000 vectors. At 100 million, it isn't. Approximate Nearest Neighbor (ANN) search accepts a small, tunable amount of inaccuracy — finding 98 of the true top 100 rather than all 100 — in exchange for search times that barely grow as the collection grows.

The dominant ANN algorithm in production today is HNSW (Hierarchical Navigable Small World), introduced in a 2016 paper by Yu. A. Malkov and D. A. Yashunin. HNSW builds a multi-layer graph where each vector is a node connected to its approximate neighbors, with sparser layers on top acting like an express lane. A search starts at the top, sparse layer, jumps toward the right neighborhood quickly, then descends into denser layers to refine the answer — similar to how you'd navigate a city by first picking the right highway exit, then the right street, then the right house number, rather than checking every house in the city one by one. The original paper reported that HNSW outperformed prior ANN methods on standard benchmarks by a wide margin in both recall and query speed. Source: Malkov & Yashunin, 2016/2018, arXiv:1603.09320.

The other major indexing families

HNSW isn't the only approach, and the alternatives exist because HNSW has a real weakness: it holds the entire graph in memory, which gets expensive at billion-vector scale.

• IVF (Inverted File Index) clusters vectors into buckets ahead of time using a technique like k-means, then at query time only searches the handful of buckets closest to the query. It uses far less memory than HNSW but needs a training step and can lose recall if the clusters are unbalanced.
• Product Quantization (PQ) compresses each vector by splitting it into sub-vectors and replacing each with the ID of its nearest "codeword" from a small learned dictionary — shrinking storage 4 to 32 times at the cost of some precision. It's almost always combined with IVF (as "IVF-PQ") rather than used alone.
• DiskANN, developed by Microsoft Research, and ScaNN, from Google, are built specifically to keep most of the index on disk or in compressed form rather than fully in RAM, trading a bit of latency for the ability to index billions of vectors on modest hardware.

What is a vector database used for?

Vector databases power any feature that depends on matching meaning rather than exact text: semantic search engines, recommendation systems, fraud and anomaly detection, reverse image search, deduplication, and — most visibly today — the retrieval step in RAG pipelines that ground chatbot answers in real documents.

Inside a real vector database: the features that matter in production

A vector index alone is not a database. What separates FAISS (a library) from Pinecone or Qdrant (databases) is everything built around the index to make it usable at scale.

Metadata filtering lets you combine a vector search with structured conditions, such as "find similar articles, but only ones published after January 2025 and tagged ai." This sounds trivial until you realize a naive implementation either filters after the ANN search runs (which can starve you of results, the exact bug pgvector 0.8 fixed) or filters before it (which can be slow if the filter matches millions of rows). Every mature engine has spent real engineering effort on this problem specifically.

Hybrid search fuses vector similarity with traditional keyword search, usually BM25, the ranking algorithm behind most full-text search engines. This matters because pure embedding search is weak at exact matches: a query for a specific product SKU, a legal citation, or a person's name can get buried under semantically related but factually wrong results. Blending the two signals — often with a reranking step — reliably improves retrieval quality on queries containing proper nouns or identifiers.

Multi-tenancy and namespaces let a single cluster serve many customers or projects without spinning up a separate deployment for each one, which matters enormously for SaaS companies building AI features on top of a vector store.

Replication, snapshots, and access control are the unglamorous production requirements every database eventually needs: point-in-time recovery, role-based access, and the ability to survive a node failure without losing an index that might have taken hours to build.

The 2026 vector database landscape

Database	Deployment model	Default index	Hybrid search	Best fit	Notable
Pinecone	Managed SaaS	Proprietary, HNSW-derived	Yes	Serverless RAG at scale	Raised $138M+ across funding rounds; pioneered the serverless vector DB category
Weaviate	Open-source + managed cloud	HNSW	Native BM25 + vector	Knowledge graphs, multimodal search	Built-in vectorizer modules; raised a $50M Series C in 2023
Qdrant	Open-source + managed cloud	HNSW	Yes	High-throughput, filter-heavy workloads	Written in Rust; raised a $50M Series B in early 2026, bringing total funding to roughly $87.8M
Milvus / Zilliz	Open-source + managed cloud	HNSW, IVF, DiskANN	Yes	Billions of vectors, distributed workloads	Distributed architecture by design; Zilliz has raised an estimated $100M+ total
Chroma	Open-source, embedded	HNSW	Limited	Prototyping, notebooks, small projects	Fastest setup of any engine on this list
pgvector	Postgres extension	HNSW, IVFFlat	Via full-text search	Teams already running Postgres	Most-deployed vector store by raw adoption; version 0.8 added iterative scans and parallel index builds
Vespa	Open-source	HNSW + tensor operations	First-class hybrid	Large-scale production search engines	Powers search at Yahoo and Spotify
LanceDB	Open-source, embedded	IVF-PQ	Yes	Local development, S3-backed analytics	Columnar storage built on the Apache Arrow/Parquet format
Turbopuffer	Managed	Custom, object-storage-first	Yes	Very large, cost-sensitive, read-heavy workloads	Stores vectors on object storage (like S3) rather than local disk, cutting cost per gigabyte sharply

Table: the major vector database options as of mid-2026, compared by deployment model, indexing approach, and ideal use case.

Sources for the table: Pinecone learning center, Qdrant documentation, pgvector GitHub, Weaviate developer docs, and funding figures via CB Insights, Qdrant financials.

By raw deployment count, pgvector wins simply because most teams already run Postgres somewhere — on Neon, Supabase, Amazon RDS, or self-hosted — and adding vector search to an existing database removes an entire moving part from the stack. Pinecone still leads in pure managed, vector-only deployments, while Qdrant and Turbopuffer are the fastest-growing newer entrants, both built around the idea that most teams overpay for vector storage they don't need to keep fully in memory.

Choosing the right one for your project

The honest answer is that the "best" vector database is almost always the one that fits your existing infrastructure and scale, not the one topping a benchmark chart.

If you already run Postgres and have fewer than roughly 50 million vectors, pgvector removes an entire service from your architecture — one fewer thing to deploy, monitor, and back up. Tune m and ef_construction for an HNSW index, or lists for IVFFlat, and you get production-grade ANN search inside a database you already operate.

If you want zero infrastructure to manage and pay-per-use pricing, Pinecone or Turbopuffer fit — you trade some cost efficiency for not thinking about servers at all. If you need full control, self-hosting, advanced filtering, or you're operating at billions of vectors, Qdrant, Weaviate, or Milvus are built for exactly that. Teams that already run Elasticsearch or OpenSearch for logs and full-text search can bolt on their built-in kNN vector support, which is "good enough" up to roughly 10 million vectors before dedicated engines start pulling ahead on latency. For local prototyping, notebooks, or edge analytics where you don't want a server at all, Chroma and LanceDB run embedded, in-process.

How do I pick an embedding model?

Start with a well-benchmarked general-purpose model — OpenAI's text-embedding-3-small or the open-source bge-m3 are common defaults — and evaluate it against your own retrieval task rather than a generic leaderboard. In practice, the embedding model choice usually matters more for final answer quality than which vector database you pick, because a mediocre embedding puts irrelevant chunks in the top results no matter how well the database searches them.

How a RAG pipeline actually uses a vector database

Retrieval-augmented generation is the single biggest driver of vector database adoption right now, and understanding the full pipeline explains why the database matters so much for the quality of the final answer. For a deeper look at the memory and retrieval systems inside modern chatbots, see our explainer on how AI systems remember context across conversations.

Step 1 — Ingest. Documents are split into chunks, typically a few hundred tokens each, and every chunk is run through an embedding model. The resulting vector, along with the original text and any metadata (source, date, author), is written into the vector database.

Step 2 — Query. When a user asks a question, the same embedding model turns that question into a vector, and the database returns the top-K most similar chunks — often 5 to 20 of them — sometimes re-ranked afterward by a more expensive but more accurate cross-encoder model that directly compares the query and each candidate chunk.

Step 3 — Generate. Those retrieved chunks are inserted into the prompt sent to a large language model, which the model reads before writing its answer. This is what lets a chatbot answer questions about a company's internal documentation or last week's news, neither of which was in its training data.

The vector database is not a peripheral component here — it is the retrieval engine, and the quality of what it returns puts a hard ceiling on how good the final answer can be. No amount of prompt engineering fixes a RAG pipeline that retrieved the wrong five paragraphs. Hybrid search (combining BM25 keyword matching with vector similarity) is one of the most reliable levers for improving recall, particularly on queries that include exact identifiers, model names, or proper nouns that pure embedding similarity tends to blur together. Teams that also need hosted models for the generation step can compare Together AI and Fireworks AI as inference providers.

A working example: Postgres with pgvector

Because pgvector is the most widely deployed option, it's worth seeing exactly what the workflow looks like in practice.

CREATE EXTENSION vector;

CREATE TABLE docs (
  id bigserial PRIMARY KEY,
  content text,
  embedding vector(1536)
);

CREATE INDEX ON docs USING hnsw (embedding vector_cosine_ops);

-- Query: find the 5 most similar rows to a given vector
SELECT id, content
FROM docs
ORDER BY embedding <=> '[0.12, ...]'::vector
LIMIT 5;

The <=> operator computes cosine distance, and because there's an HNSW index on that column, Postgres uses it instead of comparing against every row — the query stays fast even as the docs table grows into the millions. As of pgvector 0.8, you can also enable hnsw.iterative_scan = on so that queries combining this similarity search with a WHERE clause keep searching until enough matching rows are found, rather than silently returning fewer results than requested. Source: pgvector 0.8.0 release, PostgreSQL.org.

Embedding dimensions and the storage trade-off

The size of your embedding vectors is a direct lever on both storage cost and search speed, and it's set by whichever embedding model you choose:

Dimensions	Typical model	Notes
384	all-MiniLM-L6-v2	Lightweight and fast; lower semantic resolution
768	BERT-base, bge-base	Common mid-tier default
1,024	bge-large, mxbai-embed-large	Strong open-source option
1,536	OpenAI text-embedding-3-small, legacy ada-002	Most widely used commercial default
3,072	OpenAI text-embedding-3-large	Higher accuracy, roughly double the storage cost
4,096	Cohere and some domain-specific models	Used where maximum retrieval precision matters most

Table: common embedding dimensions and the models that produce them.

OpenAI's text-embedding-3 family supports a technique called Matryoshka Representation Learning, which lets you truncate a 1,536- or 3,072-dimension vector down to 256 or 512 dimensions with only a modest accuracy loss — a useful lever for cutting storage costs on large collections. Quantization stacks on top of that: converting 32-bit float vectors to 8-bit integers typically costs less than 1% in recall while shrinking storage fourfold, and binary quantization can shrink it 32-fold at a steeper — but often still acceptable — accuracy cost, especially when paired with a re-ranking pass over the full-precision vectors for the final shortlist.

What this actually costs at scale

For 10 million documents embedded at 1,536 dimensions in 32-bit floats, raw vector storage runs to roughly 60 gigabytes — before accounting for the original text, metadata, or index overhead, which for HNSW typically adds another 50 to 100% on top of raw vector size because the graph structure itself has to be stored. Apply binary quantization and that same collection can shrink to somewhere in the low single-digit gigabytes.

On a managed platform like Pinecone, a collection in that range typically runs somewhere in the low hundreds of dollars per month depending on query volume and replication settings. Self-hosting an engine like Qdrant on a modest cloud VM is viable for read-heavy workloads once quantization is applied, though you're then responsible for uptime, backups, and scaling it yourself. In practice, the managed-versus-self-hosted decision usually comes down to whether your team has spare operational capacity, not which option is cheaper per gigabyte on paper.

What people get wrong about vector databases

Assuming you need one from day one. Below roughly 100,000 chunks, an in-memory library like FAISS, or even a plain NumPy array with a brute-force loop, is fast enough and adds zero operational overhead. The moment to reach for a real vector database is when latency or scale actually becomes a problem, not before.

Treating chunking as an afterthought. How you split documents into chunks affects retrieval quality more than which database or index type you pick. A chunk that cuts a sentence in half, or that bundles three unrelated topics together, produces a mediocre embedding no matter how good the underlying search algorithm is.

Assuming cosine similarity equals semantic truth. Embedding models encode a mix of genuine meaning and surface-level statistical co-occurrence from their training data. Two passages can score as "similar" because they share stylistic or topical patterns rather than because they actually answer the same question, which is exactly why hybrid search and reranking exist as correctives.

Underestimating embedding drift. Swapping to a new or upgraded embedding model invalidates your existing index, because vectors from two different models don't share a coordinate space — you cannot mix them. Any embedding model upgrade means a full re-embed and reindex of the entire collection, which needs to be budgeted for as an operational cost, not treated as a free upgrade.

Ignoring cold-start latency in serverless setups. HNSW indexes perform best fully warm in memory. Serverless vector platforms that spin resources down between requests can add 100 to 500 milliseconds of cold-read latency on the first query after idle time — worth testing before committing to a serverless tier for a latency-sensitive product.

The takeaway

A vector database is not a replacement for your relational database — it is a specialized tool for one job: finding what's semantically close, fast, at scale. That single capability is what turns a large language model from a closed system that only knows its training data into one that can be grounded in your documents, your product catalog, or last week's news through RAG. The choice of engine matters less than most benchmarks suggest; chunking strategy, embedding model quality, and hybrid search tuning usually move the needle on answer quality more than which database is running underneath. Start with the simplest option that fits your existing stack — often pgvector if you're already on Postgres — and only reach for a specialized, horizontally scaled engine once you have the vector count or latency requirements that actually demand one.

FAQs

Q: Do I need a vector database for RAG? A: Not always. Below roughly 100,000 chunks, an in-memory library like FAISS or even a NumPy array works fine. Past that scale, a dedicated vector database (or pgvector) becomes necessary for latency and operational features like filtering and replication.

Q: pgvector vs. a dedicated vector database — which should I pick? A: If you already run Postgres and have fewer than roughly 10 to 50 million vectors, pgvector with an HNSW index is an excellent, low-complexity choice. For billions of vectors, heavy metadata filtering, or advanced hybrid search, a specialized engine like Qdrant, Weaviate, or Milvus will scale further.

Q: What's the difference between HNSW and IVF? A: HNSW is a graph-based index that gives near-instant queries but uses more memory because the whole graph typically stays in RAM. IVF partitions vectors into clusters and only searches the closest few, which uses less memory but usually costs a little recall at the same latency.

Q: How do I pick an embedding model? A: Start with a strong general default like OpenAI's text-embedding-3-small or the open-source bge-m3, then benchmark it against your own retrieval task. Model choice typically affects final answer quality more than which vector database you use.

Q: Can I do exact, non-approximate search instead of ANN? A: Yes. pgvector and most other engines support an exact/flat index alongside their ANN indexes. Exact search is practical up to roughly a million vectors; beyond that, the latency cost usually makes ANN necessary.

Q: Is quantization safe to use in production? A: Scalar (8-bit integer) quantization typically costs less than 1% in recall for a fourfold storage reduction, and is safe as a default. Binary quantization saves far more space (roughly 32-fold) but loses more accuracy, so it's best paired with a reranking step over full-precision vectors for the final results.

Q: What is HNSW and why do most vector databases use it as the default index? A: HNSW (Hierarchical Navigable Small World) is a graph-based approximate nearest neighbor algorithm, introduced by Malkov and Yashunin in 2016, that organizes vectors into layered graphs so a search can jump to the right neighborhood quickly instead of scanning everything. It's the default in most engines because it offers a strong balance of query speed and accuracy for in-memory collections.

Q: How is a vector database different from a traditional database like Postgres or MongoDB? A: Traditional databases index for exact matches and ranges using structures like B-trees, which don't help when the question is "what's most similar to this." Vector databases add specialized ANN indexes (HNSW, IVF) and distance operators (cosine, dot product, Euclidean) built specifically for similarity search across high-dimensional embeddings.

Q: What happens if I change my embedding model after building a vector database? A: You have to re-embed and reindex your entire collection. Vectors produced by different models don't share a coordinate space, so mixing old and new embeddings in the same index produces meaningless similarity scores.

Q: Does adding metadata filters slow down vector search? A: It can, depending on how the filter interacts with the ANN index. Older HNSW implementations without iterative scanning could return too few results when a filter was restrictive; pgvector 0.8 and comparable updates in other engines addressed this by continuing to search until enough filtered matches are found.

Sources

• Pinecone learning center — https://www.pinecone.io/learn/
• Qdrant documentation — https://qdrant.tech/documentation/
• pgvector GitHub — https://github.com/pgvector/pgvector
• pgvector 0.8.0 release announcement, PostgreSQL.org — https://www.postgresql.org/about/news/pgvector-080-released-2952/
• Crunchy Data, "Hybrid Search Patterns with Postgres and pgvector" — https://www.crunchydata.com/blog/hybrid-vector-search
• Weaviate developer documentation — https://weaviate.io/developers/weaviate
• Malkov, Y. A. and Yashunin, D. A., "Efficient and Robust Approximate Nearest Neighbor Search Using Hierarchical Navigable Small World Graphs" (2016/2018) — https://arxiv.org/abs/1603.09320
• CB Insights, Qdrant financials — https://www.cbinsights.com/company/qdrant/financials

---

Read next:

• How AI Memory Systems Work
• How Transformers Pay Attention to Words
• Vector Database — Glossary