IFEval
Benchmark metadata from benchwiki — this page mirrors their record so scores can be joined to it. The original is canonical.
What it measures
~500 prompts, each carrying an explicit, machine-checkable constraint (format, content, or style) that a response must respect.
- Primary metric
- prompt- and instruction-level accuracy, strict and loose (%)
- Judge model
- not judged by a model
- Human baseline
- not reportedNo protocol-comparable human score was published. The constraints are deliberately trivial for a human to meet, so a nominal human baseline would sit near 100% and be only weakly informative.
- Languages
- English
Health
541 prompts place roughly a ±4pt 95% interval on a 50% prompt-level score; per-instruction splits are noisier but finer-grained.
- Status evidence
- Deprecated in this wiki for frontier instruction-following comparison: it remains a useful cheap regression check, but its simple verifiable prompts have become a floor and newer suites cover multi-constraint, multi-turn, and agentic instruction following more directly.
- Saturated since
- not saturated
- Refresh cycle
- static
- Test set public
- yes
The paper includes an explicit prompt-level decontamination layer that filters candidates appearing in training data. The set is static and public, so memorization remains possible for models trained after release.
Models scored on this benchmark
The reverse lookup, which is this site’s addition: every model here with a score on this benchmark. Independent measurements and vendor-reported numbers are kept apart rather than interleaved, because they are different claims.
No model on this site has a score on this benchmark yet. That usually means the models that were measured on it are not in the registry, rather than that nobody has run it.
Trajectory
No trajectory is published for this benchmark upstream, so none is shown. An empty chart would imply the scores exist and are flat.
Where results are tracked
Leaderboards benchwiki lists for this benchmark. These are links out, not sources this site ingests — no score below comes from any of them.
- Official repo (data and graders)official
- lm-evaluation-harnessthird-party
Source
This record is maintained by benchwiki and mirrored here. Read the original. Upstream last updated 2026-08-12; fetched 2026-10-19 by this site.