Why Eval Matters
LLM outputs are high-dimensional and subjective. Without systematic evaluation, teams ship regressions discovered only by user complaints. Evaluation spans pre-deployment benchmarking, CI gates on prompt or model changes, and ongoing production monitoring tied to human feedback.
Good eval is task-specific. A model strong on MMLU may fail your invoice extraction JSON schema or your support tone guidelines.
Benchmark Types
Public academic benchmarks
- MMLU / GPQA - broad knowledge and reasoning
- HumanEval / MBPP - code generation
- GSM8K / MATH - math word problems
- MT-Bench / Arena-style - multi-turn chat quality
Useful for coarse model selection, but vulnerable to contamination if training data overlapped benchmark questions.
Custom eval sets
Curate prompts from real logs (redacted), label expected behaviors, and score automatically or with human raters. This is the highest-signal eval for product teams.
Metrics
- Exact match / F1 - QA with short answers
- BLEU / ROUGE - n-gram overlap for summarization (limited semantic capture)
- Pass@k - code tasks with multiple samples
- Faithfulness / groundedness - answer supported by retrieved context in RAG
- Latency and cost - tokens per task, p95 response time
- Safety rates - refusal appropriateness, jailbreak success
RAG-Specific Evaluation
Split retrieval and generation eval:
- Retrieval: recall@k, MRR, nDCG on query-chunk relevance labels
- Generation: citation correctness, answer faithfulness, hallucination rate when context is empty
Synthetic QA generation from documents can bootstrap labels, but human spot-checks remain necessary to catch bad synthetic questions.
from ragas import evaluate
from ragas.metrics import faithfulness, answer_relevancy, context_recall
result = evaluate(
dataset=eval_dataset, # question, answer, contexts, ground_truth
metrics=[faithfulness, answer_relevancy, context_recall],
)
print(result.to_pandas())Eval Harness Design
A reproducible harness fixes model version, temperature, prompt template, and tool configuration. Store results per git commit or config hash. Run nightly or on PR for critical suites. Include pairwise comparison against a baseline checkpoint, not only absolute scores.
Stratify reports by task category, language, and user segment. Aggregate averages hide failures on minority slices that matter legally or commercially.
Human Evaluation
Humans rate helpfulness, correctness, and safety on side-by-side outputs. Use clear rubrics, multiple raters per item, and inter-rater agreement tracking. Human eval is expensive but essential before major launches and for disputes when automatic metrics disagree with user sentiment.
Close the loop: production thumbs-down flows into labeling queues; labeled data feeds fine-tuning and preference optimization.
Continuous Improvement
Evaluation is never finished. New failure modes emerge with features (tools, longer context, multimodal inputs). Treat eval sets as living assets with owners, versioning, and deprecation of stale prompts that no longer reflect the product.