What is RLVR (verifiable rewards)?

RLVR stands for Reinforcement Learning with Verifiable Rewards: a way of training a language model where the reward does not come from a human rater saying "I prefer this answer", but from an automatic verifier that checks whether the result is correct: the unit test passes or fails, the answer to the math problem matches the known one, the required format is respected or not. The name comes from the Tülu 3 paper by the Allen Institute for AI (arXiv:2411.15124), which describes it as a technique that assigns a constant reward only when the model's generation is verified correct, applied to problems with checkable outcomes such as GSM8K, MATH and measurable instruction-following constraints (IFEval). The difference from classic RLHF is exactly this: nobody has to read the answer and say whether they like it, a verifier just runs a mechanical check and returns a score.

See also: RLHF vs DPO vs GRPO · The three techniques by which models learn to behave after pretraining: from reinforcement with human feedback to direct preferences.

Why it scales, and where it stops

DeepSeek took the idea to its most extreme consequence with DeepSeek-R1-Zero (arXiv:2501.12948), trained with pure reinforcement learning on rule-based rewards, with no neural reward model for reasoning. It is the same post-training shift described in the entry on RLHF, DPO and GRPO, but the focus here is a different question: where the reward comes from, not which algorithm optimizes it. DeepSeek states the advantage explicitly: a neural reward model may suffer reward hacking in large-scale RL, meaning the model learns to fool the evaluator rather than genuinely improve, and it needs extra compute to train and maintain. A rule-based verifier avoids that when correctness is objective: the test either passes or it does not, there is no room for persuasion. The cost it also removes is human labeling: no annotator has to compare pairs of answers, the signal already exists inside the problem itself. The limit is just as sharp: it only works where an automatic way to say "correct" exists, so math, code with tests, structured formats, not writing a persuasive email or a matter of style. And even a rule-based verifier is not immune to its own kind of gaming: a model can learn to pass the test without solving the problem the test was meant to measure, for instance by writing code that recognizes the exam cases instead of implementing the general logic, the form of reward hacking specific to RLVR.

An enterprise example

A developer-tools vendor wants to align a model to write code that passes an internal repository's test suite. With classic RLHF this would need thousands of diffs read by human reviewers asked which version they prefer, a slow and expensive bottleneck. With RLVR the signal already lives in the existing CI pipeline: the model generates a patch, the test suite runs, the result (green or red) is the reward. It is this kind of signal, real test execution rather than a subjective readability judgment, that makes RLVR a fit for coding agents working inside an agent harness.

Why it matters for decision makers

Anyone evaluating a vendor who claims to have aligned a model with RLVR should ask two things: on what portion of the use case an automatic verifier actually exists, and how the verifier's own reward hacking is checked, meaning whether anyone has confirmed the model solves the problem and not just the test that measures it. The first question separates a marketing claim from a real use case: outside math, code and a few other structured domains, a reliable verifier usually does not exist, and anyone pitching it as a universal solution is oversimplifying. The second is the same discipline that applies to any evals suite: a high score on the verifier says the model beat that verifier, not that it solved the business problem.