Embeddings Explained: What They Are and How to Use Them (with Code)

An embedding is a vector (a list of numbers) that a model produces from text, an image or another object, so that objects with similar meaning get similar vectors. “The cat is asleep on the sofa” and “A kitten naps on the couch” have no nouns or verbs in common, yet a good model places their vectors close together, while a sentence about quarterly revenue lands far away. That makes meaning comparable with arithmetic: the distance or the cosine of the angle between two vectors.

Semantic search, RAG, deduplication, clustering and recommendations all run on embeddings. Many of these systems break at the embedding step, before the language model is involved: documents and queries encoded by different models, a required prefix left out, long text silently truncated, non-English text sent to an English-only model.

This article covers how an embedding works at the level of intuition, how to measure similarity, how to pick a model as of November 2026 and how to shorten a vector without losing much quality. The Python example computes vectors locally with Ollama. The numbers comparing an English-only and a multilingual model on Russian and English sentences come from the author’s measurement on 2026-11-08 with Ollama 0.40.14.

What is an embedding?

An embedding model takes text and returns a fixed-length vector: 768, 1024, 1536 or 3072 numbers, depending on the model. One word or a full page, the length stays the same. The model is trained so that texts with similar meaning land at neighboring points in this high-dimensional space.

What an embedding is often confused with:

• Not compression. You can’t restore the original text from the vector. The model keeps what matters for comparing meaning and drops the wording.
• Not a translation. A vector isn’t text in another language. A multilingual model places a sentence and its translation side by side, but the vector itself isn’t written in any language.
• Not a dictionary of word senses. An embedding doesn’t list meanings. It gives a point in space, and the meaning shows only through which points end up nearby.

Formally, an embedding is a mapping: a rule that assigns a vector to an object. Hence the name: objects are “embedded” into a vector space.

Analogy: coordinates on a map of meaning

On a regular map a city has two coordinates, and they show which cities are close. An embedding does the same for meaning, only with hundreds of coordinates. A single coordinate means nothing on its own: there is no “about animals” axis or “about money” axis. Only the relative position of points matters. So one text’s vector is useless by itself. What you use is the comparison of two vectors.

From word2vec to sentence embeddings

The idea went mainstream with word2vec: Mikolov et al. posted Efficient Estimation of Word Representations in Vector Space on arXiv in January 2013. The paper includes the well-known example: the vector for “King” minus “Man” plus “Woman” is closest to the vector for “Queen”.

word2vec gives one vector per word. “Bank” gets a single point whether it means a river bank or a place that holds your money. Modern models embed a whole sentence or paragraph and take context into account, so “bank” in two different sentences shifts the vector in different directions. Search needs exactly this. People look for answers to questions, not for isolated words.

How it works: vectors, distance, cosine

Dimensions

The dimension is the number of values in the vector. OpenAI’s text-embedding-3-small has 1536 by default, text-embedding-3-large 3072, bge-m3 1024, nomic-embed-text 768. More dimensions usually capture more detail, and also take more space.

Storage math is simple. One float32 takes 4 bytes, a 1536-dimension vector takes 6 KB, and a million such vectors take about 6 GB before any index. So when you pick a model, weigh dimension against quality. Voyage, for example, can return int8 and binary vectors, which according to its documentation cut storage by 4x and 32x respectively.

Cosine similarity, dot product and Euclidean distance

Three measures are common:

OpenAI, Voyage and Ollama return vectors normalized to length 1. The OpenAI documentation states this directly, as does the Anthropic embeddings guide for Voyage, and Ollama says /api/embed returns L2-normalized vectors. For such vectors cosine equals the dot product, and cosine and Euclidean distance rank results identically. The choice of measure then affects speed, not results.

Normalization is lost if you truncate a vector by hand: after that, the dot product no longer equals cosine.

Where embeddings are used

• Semantic search. The query and the documents become vectors, and the nearest documents are the results. It works because vector proximity reflects similarity of meaning rather than shared words.
• RAG. The same search, except the retrieved chunks go to a language model as context for its answer.
• Clustering. Reviews, tickets or articles are grouped by meaning with an algorithm like k-means, directly on the vectors.
• Recommendations. “Related articles” and “similar products” are found through the nearest vectors of their descriptions.
• Classification. You train a simple classifier on top of the vectors, or compare a text with vectors of reference examples for each class.
• Deduplication. Pairs of texts with a very high cosine are most often duplicates or paraphrases.

Example: computing similarity by hand

The example below reproduces the author’s measurement: 10 Russian sentences and their English translations, two models in Ollama. For each model it computes three numbers: the average cosine between a sentence and its translation, the average cosine with the other translations (90 mismatched pairs), and top-1, meaning for how many Russian sentences the nearest English vector was their own translation.

You need a running Ollama and two models: nomic-embed-text (English-only, 274 MB) and bge-m3 (multilingual, 1.2 GB).

ollama pull nomic-embed-text
ollama pull bge-m3

The code uses only the Python standard library. It calls the native /api/embed endpoint: a model field and an input field with a string or an array of strings; the response contains an embeddings array.

import json, math, urllib.request

def embed(model, texts):
    req = urllib.request.Request(
        "http://localhost:11434/api/embed",
        json.dumps({"model": model, "input": texts}).encode(),
        {"Content-Type": "application/json"})
    return json.load(urllib.request.urlopen(req, timeout=600))["embeddings"]

def cos(a, b):
    dot = sum(x * y for x, y in zip(a, b))
    return dot / math.sqrt(sum(x * x for x in a) * sum(y * y for y in b))

PAIRS = [("Кот спит на диване.", "The cat is sleeping on the sofa."),
         ("Поезд опоздал на сорок минут.", "The train was forty minutes late."),
         # the remaining pairs from the table below
        ]

The cos function computes the full cosine, even though Ollama vectors are already normalized and a dot product would do. That way the code works with vectors from any source.

n = len(PAIRS)
for model, prefix in (("nomic-embed-text", "clustering: "), ("bge-m3", "")):
    ru = embed(model, [prefix + r for r, _ in PAIRS])
    en = embed(model, [prefix + e for _, e in PAIRS])
    match = [cos(ru[i], en[i]) for i in range(n)]
    other = [cos(ru[i], en[j]) for i in range(n) for j in range(n) if i != j]
    top1 = sum(max(range(n), key=lambda j: cos(ru[i], en[j])) == i
               for i in range(n))
    print(model, len(ru[0]), round(sum(match) / n, 3),
          round(sum(other) / len(other), 3), f"top-1 {top1}/{n}")

The model card makes a task prefix mandatory for nomic-embed-text. To compare two texts with each other, clustering: fits. For search you need the pair search_query: and search_document: . bge-m3 needs no prefixes, as its model card says directly.

Results and how to read them

Author’s measurement, 2026-11-08, Ollama 0.40.14 on an Apple Silicon Mac, full set of 10 pairs:

For the English-only model, a translation and a random sentence are almost indistinguishable: 0.587 versus 0.583. The ranges overlap. The worst translation (0.539) scored lower than the most similar mismatched pair (0.622), and search found the correct translation in only 2 cases out of 10. bge-m3 shows a wide gap: 0.92 versus 0.47. Even its weakest translation (0.821) is above the most similar unrelated sentence (0.630), which gives 10 out of 10.

Cosine for each pair:

Compare the gap within one model, not absolute numbers across models. A cosine of 0.6 for nomic-embed-text covers both translations and mismatched pairs. For bge-m3 the same 0.6 is the level of unrelated sentences: all its translations are above 0.82.

Limits of this measurement: 10 short everyday sentences, one language pair, one metric. It illustrates that an English-only model doesn’t fit non-English text; it isn’t a benchmark. According to ollama show, bge-m3 has 566.7M parameters and an 8192-token context, nomic-embed-text has 137M parameters and a 2048-token context in Ollama.

How to install Ollama and pick a model that fits your memory is covered in the Ollama guide and the guide to local LLMs.

Which embedding model to choose (November 2026)

Prices and specs below were taken from the vendors’ official pages on November 8, 2026.

Hosted models

OpenAI. The documentation still lists text-embedding-3-small and text-embedding-3-large; there are no models newer than v3. You reduce the dimension with the dimensions parameter.

Voyage. Anthropic has no embedding model of its own. The Claude documentation recommends Voyage AI (owned by MongoDB), with the caveat that you should compare several vendors. API keys now come from MongoDB Atlas, and the endpoint is https://ai.mongodb.com/v1/embeddings. The first 200M tokens are free according to the billing page. For search, always set input_type: it adds a service prefix to the text for a query or a document.

import numpy as np, voyageai

vo = voyageai.Client()  # reads the key from the VOYAGE_API_KEY environment variable
docs = ["The cat is sleeping on the sofa", "Quarterly revenue grew by 12 percent"]
d = vo.embed(docs, model="voyage-4", input_type="document").embeddings
q = vo.embed(["where is the cat?"], model="voyage-4", input_type="query").embeddings[0]
print(np.dot(d, q))  # vectors are normalized: dot == cosine

Atlas keys require the voyageai library version 0.3.7 or later (pip install -U voyageai). Keep the key in the environment, not in code.

Google. gemini-embedding-2 is multimodal: text, images, video, audio and PDF share one space. It has no task_type parameter. The task goes into the text itself: task: search result | query: ... for a query, title: ... | text: ... for a document. The previous gemini-embedding-001 handles text only, accepts up to 2048 tokens and costs $0.15 per 1M tokens. The two models’ spaces are incompatible: moving to gemini-embedding-2 means re-embedding the whole index.

Cohere. Embed 5 comes in two variants, embed-v5.0-pro and embed-v5.0-fast, with the same dimensions and a 128K context. It handles text, images and mixed input such as PDF pages. As of November 8, 2026, Cohere’s official pages didn’t list Embed 5 prices. Third-party aggregators quote their own figures; check them against the official price list before you budget.

Open models for your own server

BGE-M3 supports sparse and multi-vector modes in addition to the regular dense vector. multilingual-e5-large truncates text to 512 tokens and, per its model card, performs worse without the query: and passage: prefixes. The Qwen3-Embedding-8B card claims first place on the MTEB multilingual leaderboard with a mean score of 70.58 as of June 5, 2025: that is the authors’ claim at publication, not the current standing. EmbeddingGemma, with 300M parameters, appears in Ollama’s official examples, and its queries use the same format as gemini-embedding-2.

How to read MTEB

MTEB is a public leaderboard of embedding models across many tasks: retrieval, clustering, classification, pair comparison. Rankings change often, so look at the column for the task you need, usually retrieval, rather than the overall score. Some languages have their own task sets integrated into MTEB, for example ruMTEB for Russian (Snegirev et al., NAACL 2025).

Leaderboard numbers are for a shortlist, not the final pick. Make the final pick on your own data: 50 to 100 real queries with known correct documents, and the share of cases where the right document lands in the top 5.

Matryoshka: how to shorten a vector without losing quality

Matryoshka Representation Learning (Kusupati et al., 2022) trains a model so that the most important information collects at the start of the vector. The first 256 values out of 768 already work as a standalone vector, just a coarser one, like a smaller nesting doll inside a larger one.

How much coarser shows in the nomic-embed-text-v1.5 model card, which gives the mean MTEB score at each vector length:

A vector three times shorter loses a bit more than one point; twelve times shorter, about six.

The simplest route is to ask the API for the length you need: dimensions in OpenAI and in Ollama /api/embed, output_dimension in Voyage, output_dimensionality in Gemini. If the vectors are already in your index, truncate them yourself and normalize again:

import numpy as np

def shorten(v, k):
    v = np.array(v)[:k]
    return v / np.linalg.norm(v)

Without the second line, the truncated vector no longer has length 1, and the dot product stops matching cosine. gemini-embedding-2 normalizes shortened vectors itself; gemini-embedding-001 doesn’t, so manual normalization is required there.

Truncation only makes sense for models trained this way: OpenAI v3, Voyage 4, Gemini, Cohere Embed 5, Qwen3-Embedding, EmbeddingGemma, nomic-embed-text v1.5. bge-m3 and multilingual-e5 have fixed dimensions. Their model cards don’t cover truncation, and nobody guarantees the quality after it.

Common mistakes

Different models for the index and the query. Vectors from different models live in different spaces, and comparing them is meaningless. This also applies to versions from one vendor: gemini-embedding-001 and gemini-embedding-2 are incompatible. Changing the model means re-indexing everything. The Ollama documentation specifically advises using one model for both indexing and search.

Missing prefixes. nomic-embed-text expects search_query: for a query and search_document: for a document, multilingual-e5 expects query: and passage: , and EmbeddingGemma and gemini-embedding-2 expect the task: ... | query: ... format. Without the prefix the model still returns a vector with no error, but search gets worse. In hosted APIs a parameter plays the same role: input_type in Voyage.

Text that is too long. multilingual-e5 is limited to 512 tokens, EmbeddingGemma to 2048, nomic-embed-text in Ollama to 2048. By default Ollama truncates input silently (truncate: true), and the end of the document simply never reaches the vector. With truncate: false you get an error instead. Split long documents into chunks before embedding.

An English-only model on non-English text. The measurement above shows what this looks like: nomic-embed-text gave a translation and an unrelated sentence the same cosine, about 0.58. The model doesn’t crash or warn you. It just stops telling meanings apart.

Comparing cosine across models. A threshold like “similar if above 0.8” belongs to a specific model. In the measurement above, such a threshold would separate every translation from the unrelated sentences for bge-m3, while for nomic-embed-text no translation would pass: its maximum is 0.618. When you switch models, tune the threshold again on your own data.

When you don’t need embeddings

Vector search is bad at exact strings: SKUs, error codes, contract numbers, function names. A vector carries meaning, and the vectors for “ERR_4012” and “ERR_4013” will most likely end up close together. For such queries, regular full-text search like BM25 works, and production systems often combine both into hybrid search.

If the whole corpus fits in the model’s context and there are few queries, you can put the documents in the prompt without any vector search. When long context is enough and when you can’t do without RAG is covered in Context Engineering vs RAG.

You don’t always need a vector database either. A thousand vectors of 1024 values take about 4 MB, and searching them is one matrix-vector multiplication in numpy. A database pays off when you have millions of vectors and need metadata filters and updates without a rebuild. If the project already runs on Postgres, the pgvector extension is a convenient option, and it’s available in Supabase.

Where to start

1. Install Ollama and pull a multilingual model: ollama pull bge-m3 or ollama pull embeddinggemma.
2. Run the example from this article on your own texts and see how similar and unrelated pairs separate.
3. Collect 50 to 100 real queries with known correct documents and compare two or three models from the shortlists above on them.
4. Pick a dimension: start with the default and reduce it with Matryoshka if index size becomes the limit.
5. Pin the model, its version and its prefixes in one place in your indexing and search code, so they never drift apart.

How to turn embeddings into full document search with chunking, hybrid search and reranking is covered in the guide to RAG systems.