What are evals for an AI system?

Evals are systematic test suites that measure how well an LLM-based system performs on a specific task, before and after every change. They do not remove human judgment, they move it: instead of re-reading everything on every change, manual review is what builds the golden dataset and calibrates the automated checks on a sample. A well-built eval answers, with a number, the question "is this new version of the prompt, model or system better or worse than the previous one?", instead of leaving it to a subjective feeling. At their core is almost always a golden dataset, a set of real questions or inputs with the expected answer or the criteria to judge it, checked with automated metrics, judgment from a second model (LLM-as-judge) and sampled human review. Without this discipline, every prompt change or model update stays a leap in the dark: it seems to work on the handful of cases tried by hand, but nobody really knows what changed for the rest, and the "vibe check" ends up replacing a real quality strategy with its absence.

How it works

At its core is a golden dataset: a set of real questions or inputs with the expected answer or the criteria to judge it, built by hand or from already verified real cases. On that dataset you run three kinds of checks: automated metrics when the answer is verifiable by code (a field extracted correctly, a format respected), judgment from a second model when quality is more nuanced (the technique known as LLM-as-judge), and sampled human review to calibrate the other two. The result is a score you can track over time and use as a gate before every release.

Why it matters for your business

Without evals, every prompt change, every model update, every new document added to a RAG is a leap in the dark: it seems to work on the handful of examples you tried by hand, but you do not know what changed for the rest of the cases. The "vibe check", trying a couple of questions and judging by eye, is not a quality strategy, it is the absence of one. Evals are the minimum condition for putting an AI system into production with confidence: without them, you are not managing quality, you are hoping for it. A caveat on method: the same input does not necessarily produce the same output, so a score has to be read knowing how much LLM determinism is actually worth under the conditions you run in.