Vector Databases

Role in the Stack

Vector databases (or vector-capable stores) persist embeddings with metadata and answer "which vectors are closest to this query vector?" at low latency. They sit between embedding models and applications like RAG, recommendation, deduplication, and anomaly detection.

Some teams use dedicated vector DBs (Pinecone, Weaviate, Qdrant, Milvus). Others add vector indexes to existing databases (pgvector in PostgreSQL, Elasticsearch dense_vector). Choice depends on scale, ops maturity, filtering needs, and hybrid search requirements.

Approximate Nearest Neighbor (ANN)

Exact nearest neighbor search over millions of high-dimensional vectors is too slow for interactive queries. ANN indexes trade perfect recall for speed using data structures that probe only promising regions of vector space.

Refer: HNSW paper · FAISS index guide

Common index types

• HNSW (Hierarchical Navigable Small World) - graph-based, high recall, popular default; memory-heavy
• IVF (Inverted File) - clusters vectors into cells; fast at scale with tunable nprobe
• PQ / OPQ (Product Quantization) - compresses vectors for memory savings with some accuracy loss
• DiskANN - disk-backed graphs for billion-scale indexes

Tuning parameters (efConstruction, M, nprobe, quantizer settings) balances recall, latency, and memory. Always benchmark on representative queries, not defaults.

from qdrant_client import QdrantClient
from qdrant_client.models import Filter, FieldCondition, MatchValue

client = QdrantClient(url="http://localhost:6333")
hits = client.search(
    collection_name="docs",
    query_vector=query_embedding,
    query_filter=Filter(must=[
        FieldCondition(key="tenant_id", match=MatchValue(value="acme")),
    ]),
    limit=10,
    score_threshold=0.75,
)

Metadata Filtering

Production queries rarely are pure vector search. You filter by tenant ID, document type, date range, or permissions before or during ANN search. Pre-filtering vs post-filtering affects recall: aggressive pre-filters on small subsets may return empty results if the index is not designed for constrained search.

Design schemas with filter fields indexed alongside vectors. Multi-tenant SaaS almost always requires mandatory tenant_id filters on every query.

Operational Concerns

• Ingestion throughput - bulk upsert vs streaming updates
• Index rebuild - required when embedding model or dimension changes
• Consistency - eventual vs strong for write-read paths
• Replication and sharding - horizontal scale for vector count and QPS
• Backup and disaster recovery - re-embedding from source may be slow; snapshot indexes

Comparing Approaches

Managed vector DB: fastest time to market, hosted scaling, built-in monitoring. Vendor lock-in and per-dimension pricing matter at scale.

pgvector: good when data already lives in Postgres, moderate scale, strong transactional needs. May need careful index tuning past tens of millions of vectors.

Elasticsearch/OpenSearch: strong if you already need full-text plus hybrid fusion in one system.

In-memory (FAISS, USearch): research and single-node prototypes; you own persistence and distribution.

Monitoring Search Quality

Track recall@k on golden query sets, p50/p95 query latency, index size growth, and null-result rate after filters. Drift in document distribution or embedding model upgrades can collapse retrieval quality without obvious infrastructure alerts. Pair vector indexes with rerankers and periodic human relevance audits.