Embeddings

What Embeddings Represent

An embedding model maps text (words, sentences, passages) to fixed-dimensional vectors in a continuous space. Semantically similar texts land near each other; dissimilar texts are farther apart. Unlike generative LLMs, embedding models are trained for representation quality, not open-ended generation.

Typical dimensions range from 384 to 3072 depending on model. Higher dimensions can capture finer distinctions but increase storage and search cost in vector databases.

How Embedding Models Are Trained

Contrastive learning

Models learn by pulling positive pairs (query and relevant passage) together and pushing negatives apart in vector space. Loss functions include InfoNCE, triplet loss, and multi-negative cross-entropy across in-batch negatives.

Dual encoders

Separate encoders for queries and documents (or shared weights) produce embeddings independently. At search time you precompute document vectors and only encode the query live, enabling fast retrieval at scale.

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("BAAI/bge-small-en-v1.5")

doc_embeddings = model.encode(
    passages, normalize_embeddings=True, batch_size=64
)
query_embedding = model.encode(
    "query: " + user_query, normalize_embeddings=True
)

# cosine similarity ≈ dot product when vectors are normalized
scores = query_embedding @ doc_embeddings.T

Matryoshka and flexible dimensions

Some models support truncating vectors to fewer dimensions with modest quality loss, letting you tune the cost-quality tradeoff without retraining separate models.

Similarity Metrics

Cosine similarity is standard for normalized embeddings: it measures angle between vectors and is insensitive to magnitude. Dot product is equivalent when vectors are unit-normalized but faster on some hardware when norms vary. Euclidean distance is less common at scale but appears in clustering workflows.

Always use the same similarity metric at index time and query time. Mixing normalized cosine indexed vectors with raw dot product queries silently degrades ranking.

Choosing and Evaluating Models

General-purpose models (OpenAI text-embedding-3, Cohere embed, open models like e5, bge, gte) work across domains. Domain-specific fine-tuned embedders win on specialized corpora (legal, medical, code).

Evaluate on your data with:

• Recall@k and MRR on labeled query-document pairs
• Latency and throughput for your batch sizes
• Multilingual coverage if queries span languages
• Max sequence length vs your chunk sizes

Hybrid Search

Pure vector search misses exact keyword matches (SKUs, error codes, rare names). Hybrid search combines:

• Dense retrieval via embeddings
• Sparse retrieval via BM25 or SPLADE learned sparse vectors

Scores are fused with reciprocal rank fusion (RRF) or learned rerankers. Hybrid is the default recommendation for production RAG over heterogeneous documents.

def reciprocal_rank_fusion(dense_ranks, sparse_ranks, k=60):
    scores = {}
    for ranks in (dense_ranks, sparse_ranks):
        for rank, doc_id in enumerate(ranks):
            scores[doc_id] = scores.get(doc_id, 0) + 1 / (k + rank + 1)
    return sorted(scores, key=scores.get, reverse=True)

Pipeline Integration

Embed documents at ingestion, store vectors with metadata (source, ACL, timestamp), and embed queries at request time. Batch embedding jobs should be idempotent and versioned: when you change embedding models, re-embed the corpus or maintain separate indexes per model version.

Prefix instructions matter for models trained with them (e.g., "query: " vs "passage: " in e5 family). Apply the correct prefix or retrieval quality drops sharply.