Evaluating LLM features honestly

Every team ships its first LLM feature on judgement and a shared spreadsheet. That works until the surface area grows past what one person can hold in their head, at which point regressions start reaching users.

Start from real failures

Do not write a synthetic benchmark. Collect the actual inputs that went wrong — the support escalations, the abandoned sessions, the outputs someone screenshotted in frustration — and make those your first fifty test cases.

Grade what the user cares about

• Correctness against a reference, where a reference genuinely exists.
• Groundedness: is every claim traceable to retrieved context, or was it invented?
• Refusal behaviour: does it decline the things it should, and only those?
• Cost and latency at the ninety-fifth percentile, not the mean.

Model-as-judge, with supervision

An LLM judge is cheap and scales, and it is also biased toward verbosity and its own family of models. Calibrate it against human labels on a sample every release, and report the agreement rate alongside the score. A judge you have not audited is a number, not a measurement.

An evaluation you do not trust enough to block a release on is documentation, not a gate.

Wire the harness into CI, fail the build on regression against the golden set, and the feature stops being a research project and starts being software.