Omni-MATH

What it measures

A 4,428-problem Olympiad mathematics suite spanning more than 33 subdomains and ten annotated difficulty levels.

Primary metric

GPT-4o-judged answer accuracy (%)

Judge model

GPT-4o

Human baseline

—not reportedNo aggregate human baseline was published across the 33+ domains and ten difficulty levels.

Languages

English

Health

Sampling error is smaller than judge error here. Scores depend on final-answer extraction and equivalence judgment; the 2026 audit found these errors can erase real model differences.

Status evidence

A 2026 expert audit found the original Omni-Judge wrong in 96.4% of disagreements and introduced Omni-MATH-2 with a clean exact-answer subset; original leaderboard scores should not guide frontier comparisons.

Saturated since

not saturated

Refresh cycle

static

Test set public

yes

The authors curated, annotated, and categorized competition problems, but the complete questions and answers are public and many source competitions predate the benchmark.

Models scored on this benchmark

The reverse lookup, which is this site’s addition: every model here with a score on this benchmark. Independent measurements and vendor-reported numbers are kept apart rather than interleaved, because they are different claims.

No model on this site has a score on this benchmark yet. That usually means the models that were measured on it are not in the registry, rather than that nobody has run it.

Trajectory

Where results are tracked

Leaderboards benchwiki lists for this benchmark. These are links out, not sources this site ingests — no score below comes from any of them.

• Official Omni-MATH projectofficial