Generative AI & LLM Engineering · Chapter 5 · ~18 min · Hands-on Visuals
Vector Databases & Similarity Search
In Chapter 4 we saw how text becomes vectors. But when your app has 10 million documents, how do you find the 3 most relevant paragraphs in under 10 milliseconds? Enter the vector database.
- Why SQL B-tree indexes fail for vectors
- See HNSW highway graphs navigate high dimensions
- Master Approximate Nearest Neighbors (ANN)
- Combine semantic search with metadata security filters
1The High-Dimensional Scaling Problem
In a normal relational database, indexing numbers is easy. If you search WHERE age > 25, a B-tree index sorts numbers along a 1D line and finds your row in microseconds.
Vector embeddings have 1,536 dimensions. There is no such thing as "greater than" or alphabetical order in 1,536 dimensions! To find the closest match with standard SQL, you would have to calculate cosine distance against every single row in the database.
Why traditional SQL table scans choke on millions of 1,536-dimensional vectors.
Illustrative simulation only: displayed operation counts and latency are estimates, not measurements from a vector database. Real results depend on the data, index, hardware, and search settings.
Calculates cosine distance against every single vector in the database one by one.
Navigates a multi-layer graph and may examine fewer vectors than a full scan. Recall and work vary with the index, dataset, and search settings.
Why can't traditional SQL B-tree indexes speed up vector similarity search?
2How ANN Works: HNSW Highways
To make search faster at scale, vector databases can use approximate search, trading some recall for lower query cost. The speed and accuracy tradeoff depends on the index, data, and configuration. This is called Approximate Nearest Neighbors (ANN).
The most famous algorithm is HNSW (Hierarchical Navigable Small World). It organizes vectors into multi-layer graphs like an express highway system:
Hierarchical Navigable Small World (HNSW) is the gold standard indexing algorithm behind Pinecone, Qdrant, and pgvector.
Layer 2: Express Highway
Few sparse nodes with long-distance links. The query enters at the top entry point and makes giant leaps across meaning space.
Just like driving across the country: you take the interstate highway across states, exit onto arterial roads, then take local streets to the specific house address.
What does the top layer (Layer 2) of an HNSW graph do?
3Metadata Filtering: Role & Tenant Security
In production apps, semantic similarity alone is dangerous. An intern asking "What are our company plans for next quarter?"should never retrieve confidential Board of Directors payroll documents, even if they are a 99% vector match!
We attach metadata payloads (JSON attributes like tenant_id, department, role, date) to each vector:
A vector database must combine semantic similarity with hard tenant, department, and role permissions.
1=1 AND 1=1Matching documents: 7 of 7Why is 'single-stage pre-filtering' better than 'post-filtering' in vector search?
4Choosing your vector database
The vector landscape exploded with options. Which one should you actually pick for your project?
Select your project stage and architecture constraints to see the recommended approach.
pgvector (PostgreSQL)
Best for 80% of startups & existing stacksAdvantages
- Zero new infrastructure if you already run Postgres
- ACID transactions, relational joins, and vectors in one database
- Available on most managed PostgreSQL hosts
Trade-offs
- Needs index tuning (HNSW / IVFFlat) at scale > 2M vectors
pgvector with one command. You can write normal SQL queries with the <=> cosine distance operator without leaving your database!You are building a startup app that already runs on PostgreSQL with 200,000 document chunks. What is the most practical choice?
5Distance metrics: Cosine vs Dot Product vs L2
Measures the angle between vectors (0 = identical, 1 = orthogonal). Best default for text and documents where length varies.
Fastest mathematical calculation. When vectors are normalized to unit length (length = 1.0), Dot Product is identical to Cosine!
Measures straight-line ruler distance between coordinates. Commonly used in computer vision, image classification, and facial recognition.
text-embedding-3 and modern embedding models return vectors that are already normalized. That means you can use Dot Product for maximum search speed with exact cosine accuracy!6Chapter challenge
Vector Databases & ANN Mastery
Answer all four questions correctly to complete Chapter 5.
1What does 'ANN' stand for in vector search?
2How does the HNSW algorithm achieve sub-10ms search over millions of vectors?
3In pgvector (PostgreSQL), which operator calculates Cosine Distance between two vectors?
4Why should multi-tenant SaaS apps always use metadata filtering during vector retrieval?
7Recap
- Brute Force vs ANN: A full scan compares against every vector (O(N)). ANN indexes can reduce query work substantially; actual speed and recall depend on the index, dataset, and settings.
- HNSW Graph: A multi-layer graph that can navigate toward nearby vectors with fewer comparisons; speed and recall depend on index and search settings.
- Metadata Filtering: Combine semantic vector similarity with strict SQL-like filters for tenant and role-based security.
- pgvector vs Dedicated: If you already run PostgreSQL, start with pgvector. Turn to Pinecone or Qdrant when you need dedicated serverless scale.
Solve the challenge above to complete the chapter, or