Q: What are embeddings?

An embedding is a list of numbers that captures the meaning of a piece of text.

Here's the idea. "Dog" and "puppy" mean similar things, so their embeddings are close together. "Dog" and "coconut" mean very different things, so their embeddings are far apart. Once meaning is turned into numbers, a computer can compare meaning with simple math: closer numbers, closer meaning. That's the trick behind semantic search and RAG.

Embeddings are created by embedding models. This is a separate type of model, not the chat model you talk to. You send it text, and it returns a vector, often around 1,500 numbers long. OpenAI's text-embedding-3 models are a common choice, and there are strong open source options too.

Two practical things to know. First, your choice of embedding model affects quality, cost, and language support. Second, you must embed your documents and your queries with the same model. Vectors from different models live in different "spaces" and can't be compared.