Embeddings
Turning text into vectors - the foundation of retrieval.
What is a text embedding and what makes a good one?
An embedding is a dense numerical vector representing the semantic meaning of a piece of text such that semantically similar texts have vectors close together in the vector space (by cosine similarity or dot product). A good embedding model produces vectors where distance genuinely correlates with human-judged similarity/relevance, is robust across domains, and is efficient enough to embed at scale.
How are embedding models actually trained?
Most modern embedding models use contrastive learning: given a (query, relevant-passage) pair and many irrelevant "negative" passages, the model is trained (often via InfoNCE-style loss) to push the query's vector close to the relevant passage's vector and far from negatives. Hard negative mining (using genuinely confusable, near-miss negatives, not just random text) is key to embedding quality.
Dense embeddings vs sparse (BM25/TF-IDF) retrieval - how do they differ and when do you combine them?
Sparse methods (BM25) match on exact/near-exact term overlap weighted by term frequency and rarity - excellent for keyword, acronym, and exact-match queries (e.g. product codes, names) but blind to synonyms/paraphrase. Dense embeddings capture semantic similarity even with no lexical overlap but can miss exact keyword matches or rare terms poorly represented in training. Hybrid search - combining both and merging/reranking results - typically outperforms either alone in production, especially for domains with specialized vocabulary.
What is Matryoshka Representation Learning (Matryoshka embeddings)?
Embedding models trained so that meaningful information is concentrated in the first N dimensions of the vector, allowing you to truncate the embedding to a smaller size (e.g. 1536 → 256 dims) with graceful, controlled quality degradation - useful for cutting storage and search cost when full precision isn't needed.
How do you choose an embedding model and dimensionality for production?
Evaluate against your actual domain/queries (public benchmarks like MTEB are a starting point, not a guarantee) using metrics like recall@k on a labeled or synthetic query-document set. Higher dimensionality generally improves quality but increases storage and search latency - balance against your dataset scale; for millions+ of vectors, dimensionality and index choice materially affect cost and latency.
Why do you need to re-embed your entire corpus when switching embedding models?
Embedding spaces from different models (or even different versions of the same model) are not compatible - the geometry and meaning of "distance" differs between them. Mixing vectors from two different models in the same index produces meaningless similarity scores, so any embedding model change requires a full re-embedding and re-indexing of the corpus.