MMMU

What it measures

College-level multimodal questions across 30 subjects and 183 subfields, with interleaved text and diverse images.

Primary metric

micro-averaged accuracy (%)

Judge model

not judged by a model

Human baseline

88.6Best expert validation accuracy. The paper also reports medium and lowest expert groups at 82.6% and 76.2%; no equivalent full-test human score is reported.

Languages

English

Health

The 900-example validation aggregate is much noisier than the 10,500-example test score, especially after slicing 30 subjects. Prompting and answer extraction must match across comparisons.

Status evidence

Still a common broad benchmark, but the maintainers introduced the harder MMMU-Pro in 2024 and released original test answers in 2026, weakening MMMU's value as a held-out frontier signal.

Saturated since

not saturated

Refresh cycle

static

Test set public

yes

At launch the test labels were server-held. The official repository announced their release on 2026-02-12, so local evaluation is now reproducible but the original test can no longer function as secret held-out data.

Models scored on this benchmark

The reverse lookup, which is this site’s addition: every model here with a score on this benchmark. Independent measurements and vendor-reported numbers are kept apart rather than interleaved, because they are different claims.

No model on this site has a score on this benchmark yet. That usually means the models that were measured on it are not in the registry, rather than that nobody has run it.

Trajectory

Where results are tracked

Leaderboards benchwiki lists for this benchmark. These are links out, not sources this site ingests — no score below comes from any of them.

• Official MMMU leaderboardofficial