A traditional database stores a row and retrieves it by key. You ask for the exact thing; you get the exact thing. Vector databases do something stranger. They store meaning, and retrieve by resemblance.
That shift sounds philosophical. The mechanics are not.
Text, images, and audio cannot be compared directly. Machines need numbers. A machine learning model called an encoder takes a piece of content and produces a vector: a list of floating-point numbers, typically hundreds or thousands of values long.
These numbers encode semantic relationships. Two sentences with similar meanings land near each other in that high-dimensional space. "The cat sat on the mat" and "A feline rested on the rug" produce vectors close together. "Corporate tax restructuring in 2019" lands somewhere else entirely.
The embedding is not a hash. A hash is arbitrary; two similar inputs produce wildly different hashes. An embedding is a learned geometry. Proximity means something.
Modern embeddings come from models like OpenAI's text-embedding-3-small, Cohere's Embed, or open-source alternatives from HuggingFace. You pass in data; they return vectors of fixed dimensionality. Those vectors go into the database.
A naive search across millions of vectors computes the distance from the query vector to every stored vector. That is exact. It is also slow. For a million 1536-dimensional vectors, brute force costs real time.
Vector databases trade a small amount of accuracy for large gains in speed. The technique is called Approximate Nearest Neighbor search, or ANN. Several data structures make this work.
HNSW (Hierarchical Navigable Small World graphs) organizes vectors in a layered graph. Higher layers are sparse, lower layers are dense. Queries start at the top and descend, following edges toward the nearest neighbors. Search skips most of the space entirely.
IVF (Inverted File Index) clusters vectors using k-means. At query time, the database checks only the clusters closest to the query vector, ignoring the rest.
Both approaches return approximate results. In practice, recall rates above 95% are achievable at millisecond latency. For most applications, the lost five percent is invisible.
The tradeoff parameters are tunable. Higher recall costs more compute. Lower recall costs less. You choose.
The query is itself a vector. You embed the incoming text or image with the same model used to embed the stored data, then ask the database: which stored vectors are closest to this one?
Distance has more than one definition. Cosine similarity measures the angle between two vectors, ignoring magnitude. Dot product measures both direction and length. Euclidean distance measures the straight-line gap. The right choice depends on how the embedding model was trained. Most modern text models expect cosine similarity.
A working query looks like this:
import openai
import pinecone
# Embed the query
response = openai.embeddings.create(
model="text-embedding-3-small",
input="How do I handle database connection pooling?"
)
query_vector = response.data[0].embedding
# Search the index
index = pinecone.Index("docs")
results = index.query(vector=query_vector, top_k=5, include_metadata=True)
for match in results.matches:
print(match.score, match.metadata["text"])
The database returns the five stored chunks most semantically similar to the query. Score is the cosine similarity. Metadata carries the original text or a reference to it.
This is the backbone of retrieval-augmented generation. You embed a user's question, find the most relevant documents in your corpus, and hand those documents to a language model with the question. The model answers from context rather than from memory alone.
Vectors alone are not enough for production systems. You need to combine similarity with hard filters. Find the five documents most similar to this query, but only from documents published after January 2024, and only those tagged "security."
All major vector databases handle this: Pinecone, Weaviate, Qdrant, Milvus. They store metadata alongside vectors and execute pre-filter or post-filter strategies. Pre-filtering restricts the candidate set before ANN search. Post-filtering runs ANN first and discards results that fail the filter afterward. Each has latency and recall tradeoffs.
Hybrid search combines dense vector similarity with sparse keyword scoring. BM25 handles exact term matches. The dense retriever handles semantic matches. Reciprocal rank fusion merges the two result sets. When a user types an unusual proper noun, sparse search earns its place.
Vector databases are infrastructure for systems that reason over language. They solve one narrow problem well: given a query, find what is close in meaning.
Embeddings compress semantics into geometry. ANN indexes make search fast at scale. Similarity metrics define what "close" means for a given domain. Put them together and you can build document search that survives paraphrase, recommendation engines that handle cold starts, and retrieval layers that give language models a memory longer than their context window.
The data structures are not exotic. The geometry is not mysterious. What changes is the unit of retrieval. Rows become meanings, and queries become positions in space.