Evaluation that survives production
Why automated benchmarks aren't enough — and how to design human-eval suites that scale with your release cadence.
A benchmark can help select a model, but it cannot guarantee that a production system will keep working. Real users bring new language, incomplete context, shifting goals, and edge cases that were absent from the original test set. Evaluation has to evolve with the product.
Evaluate the complete system
Users experience prompts, retrieval, tools, policies, latency, and interface behaviour together. Testing only the base model misses failures introduced elsewhere in the stack. A production suite should exercise end-to-end tasks and capture the intermediate evidence needed to diagnose a failure.
Build from real work
The most valuable test cases come from actual user goals, support escalations, expert corrections, and near misses. Samples should cover common tasks as well as high-risk slices. Keep a stable core for regression testing and a rotating set that reflects current traffic.
Combine automated and human review
Automated checks are useful for structure, citations, tool outcomes, policy rules, and repeatable heuristics. Human reviewers are better at nuance, usefulness, cultural fit, and ambiguous trade-offs. The two methods should be calibrated against each other rather than treated as substitutes.
Write rubrics that support decisions
A good rubric tells reviewers what evidence to inspect and how scores affect release decisions. Separate dimensions such as correctness, completeness, safety, tone, and action quality. Require comments for severe failures and use adjudication to refine unclear criteria.
Track slices, not just averages
An overall score may improve while an important language, customer group, or workflow gets worse. Report performance by relevant slices and attach minimum thresholds to high-risk categories. Monitor disagreement and confidence alongside the score.
Close the loop after release
Production monitoring should feed new examples back into evaluation. When a failure is confirmed, add a regression case, identify the responsible component, and verify the fix across related slices. Evaluation survives production when it becomes part of the release cycle rather than a gate used once.