Cosine Similarity: The Math Behind Semantic Search

Search for "cheap computers" in an old keyword search engine, and a page titled "Affordable Laptops Under $500" might not show up at all. Not one word matches.

A semantic search system finds it immediately. It doesn't compare words. It turns both texts into vectors and measures the angle between them. The number it computes, cosine similarity, comes straight from high school trigonometry, and it's one of the most widely used formulas in modern AI.

Direction Matters, Length Doesn't

Suppose one document mentions "machine learning" 3 times and another, much longer one mentions it 30 times. Their vectors might point in the same direction but differ hugely in length.

If you measured plain distance between the points, they'd look very different. Cosine similarity ignores length entirely and compares only direction, so two texts about the same topic score as similar regardless of how long they are.

The Formula

For vectors a and b:

cos θ = (a · b) / (|a| × |b|)

Where:

• a · b is the dot product: a₁b₁ + a₂b₂ + … + aₙbₙ
• |a| is the length: √(a₁² + a₂² + … + aₙ²)

The result ranges from −1 to 1:

Convert a similarity score to an angle with the arc cosine calculator.

A Worked Example

Imagine a tiny embedding model with 3 dimensions. Three texts become:

"cheap computers"            q = (0.8, 0.6, 0.1)
"affordable laptops"         d₁ = (0.7, 0.7, 0.2)
"chocolate cake recipe"      d₂ = (0.1, 0.1, 0.9)

Query vs. affordable laptops:

q · d₁ = 0.56 + 0.42 + 0.02 = 1.00
|q| = √(0.64 + 0.36 + 0.01) = √1.01 ≈ 1.005
|d₁| = √(0.49 + 0.49 + 0.04) = √1.02 ≈ 1.010
cos θ ≈ 1.00 / (1.005 × 1.010) ≈ 0.985

Query vs. chocolate cake:

q · d₂ = 0.08 + 0.06 + 0.09 = 0.23
|d₂| = √(0.01 + 0.01 + 0.81) = √0.83 ≈ 0.911
cos θ ≈ 0.23 / (1.005 × 0.911) ≈ 0.251

The laptop page scores 0.985, and the recipe scores 0.251. The search engine ranks results by this score. Check square roots with the square roots list.

A Speed Trick: Normalize First

If every vector is normalized to length 1 ahead of time, the denominator is always 1 × 1, and cosine similarity is just the dot product:

cos θ = a · b     (when |a| = |b| = 1)

Many embedding models output normalized vectors for exactly this reason. Dot products are extremely fast, and a whole batch of them is a single matrix multiplication.

The Semantic Search Pipeline

1. Embed every document once, and store the vectors
2. Embed the user's query when it arrives
3. Compute cosine similarity between the query and documents
4. Return the documents with the highest scores

This is also the retrieval step of retrieval-augmented generation (RAG), described by Patrick Lewis and colleagues in 2020, where a chatbot looks up the most similar passages before answering.

An Insider Reference: Searching Millions of Vectors Fast

Comparing a query with every document works for thousands of vectors. With hundreds of millions, it's too slow. The solution is approximate nearest neighbor (ANN) search, which trades a tiny amount of accuracy for enormous speed.

A widely used method is HNSW (Hierarchical Navigable Small World graphs), described by Yury Malkov and Dmitry Yashunin in 2016. It links vectors into a multi-layer graph: sparse "highway" layers for long jumps, and dense lower layers for fine search. A query starts at the top, greedily moves toward closer neighbors, and drops down layers, typically examining only a small fraction of the data. Libraries like Meta's FAISS and most vector databases implement methods like this.

A Warning: High Dimensions Are Strange

In high-dimensional spaces, randomly chosen vectors are almost always nearly perpendicular, with cosine similarity close to 0. Distances between random points also become surprisingly similar to each other. This curse of dimensionality is why embeddings must be learned so that meaningful relationships stand out from random noise.

Two Concepts Worth Knowing

Dot Product

The dot product multiplies matching components and adds them up. Geometrically it equals |a||b| cos θ, which is where cosine similarity comes from.

Orthogonality

Two vectors are orthogonal when their dot product is 0, meaning they're at 90°. In semantic search, orthogonal embeddings represent unrelated content.

Quick Answer: What Is Cosine Similarity?

Cosine similarity measures how similar two vectors are by the cosine of the angle between them: (a · b) / (|a| × |b|). It ranges from −1 (opposite) to 1 (identical direction) and ignores vector length. Semantic search embeds queries and documents as vectors and ranks documents by their cosine similarity to the query.

Try Them Yourself

• Cosine Calculator: see how cosine changes with angle
• Arc Cosine Calculator: turn a similarity score into degrees
• Square Roots List: compute vector lengths
• Matrix Multiplication Calculator: batch dot products
• Trigonometric Formulas: the geometry of angles
• What Is a Vector? The Math Behind AI Embeddings: where the vectors come from

Rate three movies on "action," "comedy" and "romance" from 0 to 1, then rate a fourth. Compute cosine similarity to find which of the three it's most like.