What Is a Vector? The Math Behind AI Embeddings

In 2013, a team at Google led by Tomas Mikolov released word2vec, a method that turns words into lists of numbers. They noticed something startling. Take the numbers for "king," subtract "man," add "woman," and the result lands closest to the numbers for "queen."

vector("king") − vector("man") + vector("woman") ≈ vector("queen")

Nobody programmed that relationship in. It emerged from reading text. Those lists of numbers are called embeddings, and they're one of the core ideas behind modern AI search, recommendations and chatbots. To understand them, start with a simpler question: what is a vector?

Meaning Can Be Measured as Distance

Words feel like symbols, not measurements. But if you represent each word as a point in space, arranged so that words used in similar ways are near each other, then meaning becomes geometry. Similar ideas are nearby points. Relationships, like male-to-female or country-to-capital, become consistent directions.

What Is a Vector?

A vector is an ordered list of numbers:

v = (3, 4)
w = (0.21, −0.54, 0.88, 0.02)

You can think of it two ways:

• As a point: (3, 4) is a location on a plane
• As an arrow: from the origin to that point, with a direction and a length

The length of (3, 4) comes from the Pythagorean theorem: √(3² + 4²) = 5. Check with the right triangle calculator.

Vectors can have any number of dimensions. Two or three are easy to draw. AI embeddings typically have hundreds or thousands. We can't picture 768 dimensions, but the math of distance and direction works exactly the same way.

Three Operations That Matter

Addition combines vectors component by component:

(1, 2) + (3, 1) = (4, 3)

Subtraction gives the direction from one point to another:

king − man = the "royalty" direction, roughly

Dot product measures alignment:

a · b = a₁b₁ + a₂b₂ + … + aₙbₙ

A large positive dot product means the vectors point in similar directions. Divide by both lengths and you get the cosine similarity, a score from −1 to 1. See Cosine Similarity: The Math Behind Semantic Search.

From Words to Vectors

Here's a toy example with three made-up dimensions, "animal-ness," "pet-ness" and "vehicle-ness":

The cosine similarity of cat and dog is about 0.996; cat and car only about 0.29. The geometry matches our sense of meaning.

Real embeddings aren't designed by hand like this. Their dimensions are learned, and individual dimensions usually don't have neat human labels. Meaning is spread across many dimensions at once.

An Insider Reference: "The Company It Keeps"

How can a computer learn meaning without a dictionary? The idea goes back to linguist J. R. Firth, who wrote in 1957:

You shall know a word by the company it keeps.

This is the distributional hypothesis: words that appear in similar contexts have similar meanings. "Coffee" and "tea" both appear near "cup," "hot" and "drink."

Word2vec turned this into an optimization problem. A small neural network reads billions of words and adjusts each word's vector so it can predict the words around it. After training, words that share contexts end up with similar vectors. The original word2vec vectors had 300 dimensions.

Embeddings Beyond Words

The same idea now applies to almost anything:

• Sentences and documents: whole passages become single vectors, often with 384 to 3,072 dimensions
• Images: photos of similar objects get similar vectors
• Products and songs: recommendation systems embed items so similar ones sit close together
• Code, proteins, molecules: specialized models embed them too

Because embeddings from some models share a space across modalities, a text query like "a dog on a beach" can be compared directly with image vectors.

What Embeddings Power

• Semantic search: find documents whose vectors are nearest the query's vector, even without matching keywords
• Retrieval-augmented generation (RAG): look up relevant passages by vector similarity and give them to a chatbot
• Clustering: group similar customer reviews or support tickets automatically
• Recommendations: suggest items near what you've liked

Vector databases exist specifically to store millions of embeddings and find nearest neighbors fast.

Two Concepts Worth Knowing

Dimension

A vector's dimension is how many numbers it contains. More dimensions let an embedding capture more distinctions, at the cost of more memory and computation.

Vector Space

A vector space is a set of vectors you can add and scale. Embeddings live in a vector space where geometric operations, like subtraction to find relationships, carry meaning.

Quick Answer: What Is a Vector Embedding in AI?

A vector embedding is a list of numbers that represents a word, sentence, image or other item as a point in a high-dimensional space. Embeddings are learned so that similar items are close together, which lets AI systems compare meaning using distances and angles, for example with cosine similarity.

Try Them Yourself

• Right Triangle Calculator: vector length in 2D
• Cosine Calculator: the angle behind similarity scores
• Arc Cosine Calculator: convert similarity to an angle
• Analytic Geometry Formulas: distance between points
• Matrix Multiplication Calculator: many dot products at once
• The Mathematics Behind Large Language Models: where embeddings fit in an LLM

Invent three dimensions of your own, like "sweet," "hot" and "healthy," and score five foods. Compute cosine similarities and see whether the closest pairs match your intuition.