If you have ever asked a chatbot a question and received an answer that felt genuinely relevant — not just keyword-matched — there is a good chance a vector database was involved. Vector databases are the infrastructure layer behind semantic search, recommendation engines, and retrieval-augmented generation (RAG). This post explains what they are, how they work under the hood, and when you should reach for one.
Relational databases are optimized for exact lookups: find every row where user_id = 42, or every order placed after a certain date. That works well when you know exactly what you are looking for.
But meaning is fuzzy. If a user searches for "running shoes for flat feet," a SQL LIKE clause will miss products described as "orthopaedic trainers for overpronation." The data is semantically related, but lexically distant.
This is the gap vector databases fill. Instead of storing and querying structured rows, they store and query high-dimensional numerical representations of meaning.
An embedding is a fixed-length array of floating-point numbers — a vector — that encodes the semantic content of a piece of data. Text, images, audio, and code can all be embedded.
A sentence embedding model (such as OpenAI's text-embedding-3-small or the open-source all-MiniLM-L6-v2) reads input text and outputs a vector with hundreds or thousands of dimensions. Words and phrases that carry similar meaning end up close together in this high-dimensional space.
For example:
- "golden retriever" and "labrador" will have vectors that are close together.
- "golden retriever" and "income tax" will be far apart.
The distance between two vectors (measured with cosine similarity, dot product, or Euclidean distance) becomes a proxy for semantic relatedness. This is the foundational insight that makes vector search possible.
Storing millions of embeddings is straightforward. The challenge is querying them at speed.
A naive approach — scanning every stored vector and computing its distance to the query vector — is called an exact nearest neighbor (kNN) search. It is accurate, but its cost scales linearly with the size of the dataset. At tens of millions of vectors, it becomes too slow for interactive workloads.
Vector databases solve this with Approximate Nearest Neighbor (ANN) algorithms. These trade a small amount of accuracy for a large speedup by building an index that allows the search to skip most of the dataset.
Three widely deployed ANN approaches are:
The key parameter when building an ANN index is the recall-latency tradeoff: a more thorough search finds more accurate neighbors but takes longer. Most production systems tune this to achieve 95–99% recall with sub-millisecond query times.
With embeddings stored and indexed, a vector database query works like this:
k nearest neighbors — the stored vectors closest to the query.Suppose you are building a documentation search feature. You have 50,000 support articles. At index time, you embed each article and store the vector alongside the article ID.
When a user types "my account is locked and I cannot reset my password," you:
text-embedding-3-small.A keyword search would have matched only documents containing "locked" or "reset." The vector search matched by meaning.
Pure vector search is powerful, but real applications usually need to combine it with structured filters. A user searching for "budget laptops" on an e-commerce site also wants results filtered to items that are in stock and priced under $800.
Modern vector databases (Qdrant, Weaviate, Pinecone, Redis) support metadata filtering alongside vector search: store structured attributes with each vector, and apply filter conditions at query time. Some implement this as a pre-filter (narrow the candidate set first, then rank by similarity); others use post-filtering or integrated filtering during ANN traversal.
Hybrid search takes this further by combining a vector similarity score with a traditional keyword (BM25) score. This recovers precise matches that vector search might rank lower — for example, rare product codes or technical identifiers — while still surfacing semantically related results.
The ecosystem has consolidated around a few clear options:
| Option | Best for |
|---|---|
| pgvector | Teams already on Postgres who want to avoid another service |
| Pinecone | Fully managed, serverless; least operational overhead |
| Qdrant | High-performance, open-source, strong filtering |
| Weaviate | Built-in hybrid search and multi-modal support |
| FAISS | In-process library; good for offline batch search |
For most applications, the embedding model choice matters more than the vector database choice. Use the same model for indexing and querying, keep embeddings versioned alongside your data, and plan for re-indexing when you upgrade models — dimensions and similarity distributions change between versions.
Vector databases do not replace relational databases. They solve a specific problem: finding semantically similar items in a large corpus at query speed. The core mechanism is straightforward — embed data into high-dimensional vectors, build an ANN index for fast retrieval, and measure closeness by vector distance. Where they get interesting is in the engineering tradeoffs: recall vs. latency, filtering strategies, hybrid scoring, and embedding model selection.
If you are building anything that requires understanding intent rather than matching keywords — search, recommendations, RAG pipelines, duplicate detection — vector search belongs in your toolbox.