Vector databases quietly became the backbone of AI systems, and most teams rushed to adopt one for the wrong reasons — or avoided them for equally wrong ones. A vector store is not a magic retrieval engine; it is a specialized index tuned for one job: finding the nearest neighbors to an embedding vector fast enough to serve live queries. When to use one, how to size it, whether to go Qdrant, Pinecone, Milvus, or just pgvector — that is the comparison worth understanding before you commit.
Why Your SQL Database Can't Answer Semantic Questions
Your application can tell you which user pressed the subscribe button, how many orders shipped Tuesday, and every row where status = 'refunded'. It cannot tell you which customer support message is most similar in meaning to "my billing is wrong but I am not sure why," because a classic relational store matches exact values, not meaning. Vector databases exist to answer that second kind of question. They store embeddings, the dense numerical representations that machine learning models produce, and search them by similarity rather than equality. This is the piece that powers recommendation engines, semantic search, anomaly detection, and retrieval-augmented generation, and it explains why vector search has gone from a research curiosity to a mainstream infrastructure concern in just a few years.
From Numbers to Meaning: How Embeddings Work
An embedding is a list of floating-point numbers, usually a few hundred to a few thousand of them, that a model assigns to a piece of text, an image, or a product. The model learns, during training, to place similar items closer together in this mathematical space, so "dog" and "puppy" end up near each other while "canine dentist" lands somewhere farther away. Vector search is then the job of finding, for a given query vector, the stored vectors whose distance is smallest. Euclidean distance and cosine similarity are the two metrics you see most often; cosine similarity is the safer default for text because it ignores magnitude and focuses purely on direction, which matches how humans judge topical similarity.
What Embeddings Cost You
Embeddings are powerful but not free, and the costs matter at design time. Generating them requires an embedding model, which means either an API call (with per-token pricing) or running a model yourself (with GPU and memory costs). Storing them consumes memory; a 1536-dimensional float32 vector is 6 KB per row before you count the original text. If you embed one million documents, that is roughly 6 GB of pure vector storage on top of everything else you keep. The lesson is to think about total cost, not just the appeal of semantic understanding, and that trade-off exercise is exactly what a good database design foundation teaches. Choosing your storage type is itself a design problem, and it informs whether you isolate vectors in a dedicated store with a proper indexing strategy to keep queries within the latency budget you promised users.
Approximate Search: The Trade Everyone Quietly Accepts
Finding the exact nearest neighbor across millions of vectors is mathematically expensive in high-dimensional space, so real systems use approximate nearest neighbor (ANN) techniques. ANN indexes like HNSW (Hierarchical Navigable Small World) and IVF (Inverted File) trade a tiny amount of accuracy for a massive speedup. A well-tuned HNSW index can answer a top-k query in milliseconds over tens of millions of vectors while returning results that are 99% as good as an exact scan. The parameters you control, like the number of neighbors checked at each level or the number of candidate results, become knobs that let you trade recall for latency. Knowing how recall and latency interact is the difference between a fast-but-wrong search and a system that meets its quality bar, and it is the same reason that database indexing fundamentals carry over so directly to vector workloads.
Getting It Into Production: Ingestion and the ML Lifecycle
A vector database is only as good as the pipeline that fills it. The typical production loop looks like this: documents arrive, a chunker splits them into digestible pieces, an embedding model converts each chunk to a vector, the vector is upserted with its metadata and its source text, and then queries come in to retrieve neighbors. For a real product, the ingestion pipeline must handle re-embedding when you change your model, versioning embeddings so old and new vectors do not live side by side with incompatible meanings, and reconciliation when source documents are updated or deleted.
If you want to operate this loop reliably, the surrounding machine-learning operations matter as much as the database itself. Model versioning, embedding drift, and reproducible pipelines are core concerns of MLOps fundamentals, and treating vector search as part of a broader data and ML platform, rather than a standalone silo, is what separates teams that ship and debug quickly from teams that rebuild their index every month. On the data side, that means thinking about where the raw source lives, how it is transformed, and how the vector index syncs with it, all of which is central to data engineering basics.
The Retrieval-Augmented Generation Use Case
The hottest application of vector search in 2026 is retrieval-augmented generation, or RAG. Instead of sending an entire corpus to a model on every question, you retrieve the handful of most relevant chunks from your vector store and feed only those to the model along with the question. That approach dramatically lowers cost and hallucination risk because the model has the actual source material in front of it. The quality of a RAG system hinges almost entirely on retrieval quality, which depends on chunking strategy, embedding choice, and index tuning. Chunks that are too small lose context; chunks that are too large dilute the signal; and a mismatch between how you chunk at ingestion and how you query at runtime produces bafflingly wrong answers.
Choosing a Store: What the Market Actually Looks Like
The vendor landscape is crowded, and the right choice depends on scale, hosted versus self-managed, and how much specialized search you already run. The important realization is that you rarely need to pick "vector database" at all; many options are either purpose-built vector engines or existing databases that added a vector index. A dedicated engine like Qdrant or Weaviate gives you first-class vector features from the start, while PostgreSQL, with the pgvector extension, and Elasticsearch give you vector search inside an infrastructure you may already operate.
| Platform / Tool | Key Features | Pricing |
|---|---|---|
| pgvector (PostgreSQL) | Vector index inside Postgres, easy joins with metadata | Free, open source |
| Qdrant | Purpose-built vector engine, filters, HNSW index | Open source; cloud from ~$25/mo |
| Weaviate | Managed vector search, hybrid search, modules | Open source; cloud free tier then paid |
| Pinecone | Fully managed, serverless, high availability | Free tier 1M vectors; paid from ~$0.10/hr |
| Milvus | Distributed vector DB, GPU acceleration options | Open source, free; Zilliz cloud paid |
If you already run Postgres and your volume is under a few million vectors, start with pgvector, because you avoid an entire operational layer and you get to join vector results with your existing relational data. If you need a dedicated, scale-out store with advanced filtering and managed operations, Qdrant, Milvus, or Pinecone become more compelling. If your search needs to coexist with keyword search across full documents, Weaviate and Elasticsearch offer hybrid modes that combine dense vectors with BM25-style lexical retrieval.
Where Vector Databases Fall Down
It is worth being honest about the limits, because overselling vector search leads to failed projects. Vector search does not understand semantics the way a human does; it finds neighbors in the embedding space the model learned, and a badly trained or task-mismatched model produces garbage neighbors even with a perfect index. Exact filtering on metadata is often slower than people expect because filters interact poorly with ANN indexes. And cold-start is real: a small dataset gives you no similarity signal worth building a product on. These are operational caveats, not reasons to avoid the technology, but they are the kind of nuance an evaluation-focused engineer will weigh before committing architecture.
Measuring Whether Your System Is Actually Good
Do not ship a vector search system on vibes. Define recall as the fraction of ground-truth relevant results that appear in your top-k, track query latency at high percentiles, and look at end-to-end answer quality for RAG using both automated metrics and human review. Build a golden set of queries with known correct answers before you tune; without that, every knob adjustment is guesswork. Establish a dashboard early so that when you change a model, a chunker, or an index parameter, you can see the effect on quality instead of relying on anecdotes from a single demo query.
For more, check out: .
For more, check out: .
Frequently Asked Questions
Do I need a separate database, or can I use pgvector in Postgres?
For many teams, pgvector is the right answer because it avoids adding an infrastructure layer and lets you combine vector similarity with ordinary relational queries. As a rule of thumb, start with pgvector when your vector count is in the low millions and you already operate Postgres. Reach for a dedicated engine when you need scale beyond a few million vectors, advanced filtering at high load, or a managed service that offloads operations entirely.
What embedding model should I use?
It depends on your domain and your budget. Popular general-purpose models offer a strong balance of quality and latency, but the fastest path to a good choice is empirical: embed a representative test set with two or three candidate models and measure retrieval recall on your own queries. Do not assume the biggest model is best; a smaller, cheaper model that fits your data well usually wins on both cost and latency.
Should I use cosine similarity or Euclidean distance?
Start with cosine similarity for text embeddings, because it compares direction rather than magnitude and is robust to the scale variations that appear across models. If your vectors are already normalized to unit length, cosine and Euclidean distances become monotonically related, so the choice matters less. Switch to Euclidean only when you have a reason to care about magnitude, such as embeddings where length encodes confidence or intensity.
How do I handle vector updates when my embedding model changes?
Re-embed everything, but do it in a versioned, staged way. Keep the old index serving queries while you build the new one against the new model, then switch over and monitor recall and latency on the golden set. Version your embeddings by storing the model ID in each vector's metadata, so you can audit which model produced a given result and detect drift.
Can I use a vector database for something besides semantic search?
Yes. Anomaly detection uses embeddings to flag points that are far from their neighbors. Recommendation systems use vectors for item-to-item similarity. Deduplication uses embeddings to find near-duplicate documents or images. In each case the mechanics are the same; only your choice of what to embed, and your similarity threshold, differ.