OlympiadBench

What it measures

Bilingual Olympiad and Gaokao mathematics and physics problems, including diagrams, open answers, and proofs.

Primary metric

average score (%)

Judge model

not judged by a model

Human baseline

—not reportedNo single aggregate human baseline is published across competition, Gaokao, math, physics, text, image, open-answer, and proof subsets.

Languages

Chinese, English

Health

The aggregate combines heterogeneous subjects and grading modes. Small score gaps are not meaningful unless systems use identical dataset versions, image handling, proof policy, and subset weighting.

Status evidence

The official full-benchmark evaluation reports GPT-4o at 25.89% overall, with materially lower physics performance; broad headroom remains across bilingual and multimodal subsets.

Saturated since

not saturated

Refresh cycle

static

Test set public

yes

The authors cleaned and deduplicated source exams, but all released questions, images, solutions, and answers are now public.

Models scored on this benchmark

The reverse lookup, which is this site’s addition: every model here with a score on this benchmark. Independent measurements and vendor-reported numbers are kept apart rather than interleaved, because they are different claims.

No model on this site has a score on this benchmark yet. That usually means the models that were measured on it are not in the registry, rather than that nobody has run it.

Trajectory

Where results are tracked

Leaderboards benchwiki lists for this benchmark. These are links out, not sources this site ingests — no score below comes from any of them.

• Official repository leaderboardofficial