LLM evals you need before launch
A practical checklist of the evaluations that catch most LLM failures before users do, and how to keep them running after launch.
Traditional software has unit tests. LLM features need the same discipline, with one difference: the output is not exact, so tests have to score quality instead of checking for one right answer. These are the evaluations we put in place before any LLM feature goes live.
Start with a golden set
Collect 50 to 200 real inputs that cover the common cases, the edge cases, and the cases that matter most to the business. For each one, write down what a good answer must contain and what it must never contain. This set is the foundation for everything else, and it should grow every time a user finds a failure.
The evaluations that matter
- Task accuracy: does the output answer the question or complete the task, scored against the golden set.
- Grounding: for retrieval features, is every claim supported by the retrieved documents.
- Format: does structured output parse, match the schema, and contain the required fields.
- Safety and policy: does the feature refuse what it should and avoid leaking private data.
- Robustness: does quality hold up with typos, long inputs, other languages, and prompt injection attempts.
- Latency and cost: time and spend per request, measured on the same set.
Automate the scoring
Use exact checks where possible, such as schema validation and required keywords. Use a separate model as a grader for open-ended quality, and check a sample of its grades by hand so you know it agrees with your team. Run the full suite on every change to a prompt, model, retrieval setting, or tool.
Keep evaluating after launch
Production traffic drifts away from the test set over time. Log inputs and outputs with user consent, sample them weekly, and add new failures to the golden set. Track quality, latency, and cost on a dashboard next to your usual uptime metrics. A model provider update is a release, and it should go through the same tests as your own code.
None of this needs a large team. It needs the evaluations to exist before launch, and an owner who treats a drop in quality the same way they would treat an outage.