What Is Vector Quantization? Compressing Embeddings

Vector quantization is a compression technique that maps high-dimensional vectors to a small set of representative points, so each original vector can be stored as a short code instead of hundreds of floating-point numbers. It’s how large-scale embedding indexes fit in memory and stay fast to search, trading a controlled amount of precision for a large reduction in size.

Why embeddings need compression

Vector embeddings are typically 384 to 1,536 floating-point numbers per item. That’s manageable for a few thousand vectors, but at tens or hundreds of millions — the scale of a real semantic search or recommendation index — the raw storage and the memory bandwidth needed to scan it become the bottleneck. A vector database built on raw floats either needs enormous RAM or has to spend time paging vectors in from disk on every query.

Quantization addresses this directly: instead of storing the exact vector, store a compact approximation of it, built from a much smaller table of reference points.

How it works

The core idea is a codebook — a set of representative vectors (centroids) learned from the data, usually with a clustering algorithm like k-means. To quantize a new vector, you find its nearest centroid and store just the centroid’s index instead of the full vector. Reconstructing an approximate version later just means looking that index up in the codebook.

The compression ratio depends entirely on how many bits you spend per index versus how many bits the original vector took. If a codebook has 256 centroids, each index fits in a single byte — a massive reduction from, say, 1,536 floats (6 kilobytes) down to one byte, at the cost of only approximating each vector as its nearest centroid.

Product quantization: splitting the problem up

A single codebook covering an entire high-dimensional vector needs an enormous number of centroids to represent it accurately — the “curse of dimensionality” makes a coarse codebook too lossy. Product quantization (PQ) fixes this by splitting each vector into several smaller sub-vectors (say, 8 chunks of 192 dimensions each) and quantizing each chunk independently with its own small codebook.

A vector is then represented as a sequence of sub-codes — one index per chunk — which can represent an enormous number of combinations from a small number of per-chunk centroids. This is the technique behind most production-scale approximate nearest neighbor systems, often combined with an inverted file structure like IVF to narrow the search before any distance calculation happens.

Scalar, product, and binary quantization compared

	Scalar quantization	Product quantization	Binary quantization
What’s compressed	Each dimension independently	Vector split into sub-vector chunks	Each dimension to a single bit
Typical compression	4x (float32 → int8)	8x–64x	32x (float32 → 1 bit)
Accuracy loss	Low	Moderate, tunable via chunk count	Highest, needs re-ranking
Distance computation	Fast, simple	Requires precomputed lookup tables	Extremely fast (Hamming distance)
Best for	A safe default reduction	Large-scale ANN indexes	Massive-scale first-pass filtering

Where it fits in a search pipeline

Quantization rarely stands alone. A typical pipeline uses a coarse index — an inverted file or an HNSW graph — to narrow a query down to a shortlist of candidates, then computes distances against quantized vectors for speed, and finally re-ranks the top handful of candidates against their full-precision originals to recover any accuracy lost in the approximation. This two-stage pattern — quantized for breadth, exact for the final cut — is how systems keep both speed and quality at scale.

It also interacts with semantic search systems more broadly: quantization only affects how the vector index stores and compares embeddings, not how those embeddings were generated or what they’re being searched against.

Recall versus memory: the real trade-off

Every quantization scheme accepts some approximation error in exchange for space savings, and the practical question is always how much recall you’re willing to give up. A useful mental model: more centroids (or more sub-vector chunks) means less compression but higher recall; fewer means more compression but a real risk of returning the wrong nearest neighbors, not just approximate distances to the right ones. Because the loss is a distortion of distance, not a corruption of the result, most systems mitigate it with the re-ranking step above rather than avoiding quantization altogether. This general accuracy-for-efficiency trade-off shows up throughout machine learning systems — see what quantization means for model weights for the analogous idea applied to a model itself rather than its output embeddings.

The takeaway

Vector quantization replaces exact, high-dimensional vectors with compact codes drawn from a learned codebook, shrinking both memory footprint and search latency at the cost of controlled approximation error. Product quantization does this by splitting vectors into chunks and quantizing each independently, which is what lets modern approximate nearest neighbor indexes scale into the hundreds of millions of vectors. If you’re building a large embedding index, quantization usually isn’t optional — it’s the difference between an index that fits in memory and one that doesn’t.