How Vector Databases Work: Embeddings, Indexes, and Similarity Search Explained

Step 1: Converting Data into Embeddings

Before anything enters a vector database, it must be converted into a fixed-length array of floating-point numbers called an embedding. This conversion happens outside the database, in an embedding model. The model reads the input, whether it is a sentence, a paragraph, an image, or a code snippet, and outputs a vector that captures its semantic meaning in numerical form.

For text, the most common embedding models in 2026 are OpenAI's text-embedding-3-small (1536 dimensions, $0.02 per million tokens), text-embedding-3-large (3072 dimensions, $0.13 per million tokens), Cohere's embed-v4 (1024 dimensions), and open-source options like BGE-M3 (1024 dimensions) and Nomic Embed (768 dimensions). The dimension count determines both the expressiveness of the embedding and the storage cost. Higher dimensions capture more nuance but require more memory and slower search.

The embedding process is deterministic for a given model. The same input always produces the same output vector. This is critical because search works by comparing query embeddings to stored embeddings, and they must come from the same model to be comparable. Mixing embeddings from different models in the same collection produces garbage search results because different models map concepts to entirely different coordinates in vector space.

A practical example: the sentence "The cat sat on the mat" might become a 1536-dimensional vector where position 0 is 0.0234, position 1 is -0.0891, position 2 is 0.1547, and so on for all 1536 values. The sentence "A kitten rested on a rug" would produce a different vector, but one that is mathematically close to the first because the two sentences have similar meanings. The sentence "Quarterly revenue exceeded projections by 12%" would produce a vector that is far away from both because its meaning is completely unrelated.

Step 2: Storing Vectors with Metadata

When you insert data into a vector database, you provide the embedding vector plus metadata, structured key-value pairs that describe the original content. The metadata might include the source document ID, the original text, a timestamp, a category tag, a user ID, or any other field you might want to filter on later.

A typical insert operation looks like this conceptually: store vector [0.0234, -0.0891, 0.1547, ...] with metadata {source: "support-kb", doc_id: "KB-2847", text: "To cancel your subscription, go to Settings and click Billing", category: "billing", created: "2026-07-15"}.

The database stores the vector in its index structure and the metadata in a separate key-value store or columnar format. The separation matters because the vector index is optimized for distance calculations while the metadata store is optimized for filter evaluation. Qdrant uses a custom storage engine that keeps vectors in memory-mapped files and metadata in a RocksDB-like structure. Pinecone abstracts all storage details behind its managed API. ChromaDB stores everything in SQLite with vectors serialized as BLOBs.

Most databases organize vectors into collections (Qdrant, ChromaDB), indexes (Pinecone), or classes (Weaviate). A collection is analogous to a table in a relational database. You might have one collection for customer support documents, another for product descriptions, and another for conversation history. Each collection can have its own distance metric, index configuration, and metadata schema.

Step 3: Building the Search Index

The index is the data structure that makes fast search possible. Without an index, finding the nearest neighbors to a query vector requires computing the distance to every stored vector, which is O(n) and becomes impossibly slow at scale. With a million vectors and 1536 dimensions, a brute-force scan involves 1.5 billion floating-point multiplications per query.

HNSW (Hierarchical Navigable Small World). The dominant index type in 2026, used by Qdrant, Weaviate, pgvector, and ChromaDB. HNSW builds a multi-layer graph where each node is a vector and edges connect nodes to their nearest neighbors. The top layer is sparse with long-range connections for fast navigation. Each subsequent layer adds more nodes and shorter connections for precision. A query enters at the top layer, follows edges toward the closest node, drops to the next layer, and repeats until reaching the bottom layer where it finds the true nearest neighbors.

HNSW has two critical configuration parameters. M controls how many connections each node maintains (higher M means better recall but more memory). ef_construction controls how many candidates are examined when building the graph (higher values produce a better index but slower build times). For most workloads, M=16 and ef_construction=200 provide a good balance.

At query time, the ef_search parameter controls how many candidates are examined during search. Higher ef_search increases recall (the percentage of true nearest neighbors found) at the cost of higher latency. Setting ef_search=100 typically achieves 95-98% recall with sub-5ms latency on million-scale datasets.

IVF (Inverted File Index). IVF partitions the vector space into clusters using k-means clustering, then stores each vector in its assigned cluster. At query time, the algorithm identifies the closest clusters to the query vector and only searches within those clusters. IVF uses less memory than HNSW but requires periodic retraining as data changes. Milvus supports IVF-based indexes and combines them with product quantization for memory efficiency at billion-scale datasets.

Flat Index (Brute Force). Computes exact distances to all stored vectors. Guarantees 100% recall but has O(n) query time. Useful for datasets under 50,000 vectors or as a baseline for testing index quality. ChromaDB uses a flat index by default for small collections.

Step 4: Executing a Similarity Search

When a query arrives, the database performs these operations in sequence. First, the query vector enters the index. For HNSW, the algorithm starts at a random entry point in the top graph layer and greedily moves to the closest neighbor at each step. When it can no longer find a closer node, it drops to the next layer and repeats. This process continues through all layers, maintaining a list of the best candidates found so far.

Second, the distance metric scores each candidate. The three main metrics are cosine similarity (measures the angle between vectors, ranging from -1 to 1), dot product (equivalent to cosine for normalized vectors, unbounded), and Euclidean distance (measures straight-line distance, where lower is more similar). Text embeddings almost universally use cosine similarity because it is scale-invariant and the most intuitive for semantic comparison.

Third, metadata filters are applied. This is where databases differ significantly. Pre-filtering evaluates metadata conditions before vector search, which guarantees the correct number of results but can slow queries when the filter is very selective (only 1% of vectors match). Post-filtering runs vector search first, then removes non-matching results, which is faster but might return fewer results than requested. Qdrant implements a hybrid approach that automatically chooses the faster strategy based on filter selectivity.

Fourth, the top-k results are assembled and returned, each with the vector ID, similarity score, and associated metadata. The caller receives a ranked list of the most relevant stored items and uses them for whatever downstream task is required, typically injecting them as context into an LLM prompt.

Distance Metrics in Detail

Cosine Similarity is calculated as the dot product of two vectors divided by the product of their magnitudes. It ranges from -1 to 1, where 1 means identical direction, 0 means orthogonal (unrelated), and -1 means opposite. Most text embedding models produce vectors where cosine similarity above 0.8 indicates strong semantic similarity, 0.5 to 0.8 indicates moderate relatedness, and below 0.5 indicates weak or no relation. In practice, thresholds vary by model, so you should calibrate based on your specific embedding model and data.

Dot Product is mathematically simpler, just the sum of element-wise products. When vectors are normalized to unit length (which many embedding models do by default), dot product is equivalent to cosine similarity. The advantage of dot product is computational speed, requiring fewer operations than full cosine calculation. Pinecone recommends dot product for normalized embeddings and cosine for unnormalized ones.

Euclidean Distance measures the straight-line distance between two points in the vector space. Lower values mean more similar. In high-dimensional spaces (above 100 dimensions), Euclidean distance tends to concentrate, meaning the difference between the nearest and farthest points becomes proportionally small, a phenomenon called the curse of dimensionality. This makes Euclidean distance less discriminating than cosine similarity for high-dimensional text embeddings, which is why cosine is preferred for most NLP applications.

How Metadata Filtering Interacts with Vector Search

Real-world queries almost always combine vector similarity with metadata conditions. An AI agent searching a customer support knowledge base does not just want "the most similar documents." It wants "the most similar documents that are still current, from the billing department, and applicable to enterprise customers."

The implementation of filtered search varies significantly across databases and directly affects both result quality and performance.

Qdrant uses payload indexes, inverted indexes on metadata fields that are integrated directly into the HNSW traversal. During graph navigation, the algorithm skips nodes that do not match the filter conditions, ensuring that all returned results satisfy both the vector similarity and metadata requirements. This approach maintains consistent performance regardless of filter selectivity.

Pinecone applies filters during the vector search phase for serverless indexes, using its internal partitioning to efficiently narrow the search space. For pod-based indexes, filtering happens after the initial vector search, which can result in fewer results than requested if many high-similarity vectors are excluded by the filter.

Weaviate combines a BM25 inverted index with its HNSW vector index, supporting hybrid queries that merge keyword matching scores with vector similarity scores using a configurable alpha parameter. This gives it the unique ability to handle queries where the user provides both a semantic question and specific keywords that must appear in results.

pgvector relies on PostgreSQL's native query planner to combine vector search with standard WHERE clauses. For queries with selective filters, PostgreSQL might choose to evaluate the filter first and then run vector search on the subset, or vice versa. The planner's decisions are not always optimal for vector workloads, which is why pgvector performance can be less predictable than purpose-built databases.

Write Operations: Inserts, Updates, and Deletes

Inserting a new vector into an HNSW index requires finding its nearest neighbors and adding edges, an operation that costs O(log n) and temporarily locks a portion of the graph. Qdrant and Weaviate support real-time inserts where new vectors become searchable immediately after insertion. Pinecone has a short propagation delay of 1 to 5 seconds for new vectors to become queryable.

Updating a vector is typically implemented as a delete-then-insert operation. The old vector is marked as deleted (soft delete) and a new vector is inserted with the same ID. Soft-deleted vectors remain in the index and consume memory until the index is compacted or rebuilt. Frequent updates without compaction lead to index bloat, where the effective dataset size in the index exceeds the actual data size by 2 to 5x.

Deletes also use soft deletion in most implementations. The vector is marked as invisible to queries but remains in the index structure. Periodic compaction or vacuum operations physically remove deleted vectors and rebuild affected portions of the graph. Qdrant runs automatic optimization that triggers compaction when the ratio of deleted to active vectors exceeds a configurable threshold.

Batch operations are significantly more efficient than individual operations. Inserting 10,000 vectors in a single batch call is typically 5 to 10x faster than 10,000 individual insert calls because the database can amortize index updates and network overhead. All major databases support batch operations, and using them is one of the simplest performance optimizations available.