MME

What it measures

Image benchmark with paired yes/no questions across 10 perception and 4 cognition subtasks.

Primary metric

perception score (0-2000) and cognition score (0-800)

Judge model

not judged by a model

Human baseline

—not reportedNo protocol-comparable aggregate human baseline is reported; the paired questions are manually authored and intended to be straightforward for people.

Languages

English

Health

Do not combine perception and cognition into an undocumented percentage. The two official totals have different maxima, and accuracy+ makes paired failures count differently from ordinary question accuracy.

Status evidence

Still widely supported as a compact diagnostic, but its small, public, static yes/no tasks provide limited frontier headroom; the same team now maintains harder MME-RealWorld and MME-CoT suites.

Saturated since

not saturated

Refresh cycle

static

Test set public

yes

The authors manually wrote all question-answer pairs and reused only images, not original annotations, from public datasets where applicable. Public release after launch still permits later training exposure.

Models scored on this benchmark

The reverse lookup, which is this site’s addition: every model here with a score on this benchmark. Independent measurements and vendor-reported numbers are kept apart rather than interleaved, because they are different claims.

No model on this site has a score on this benchmark yet. That usually means the models that were measured on it are not in the registry, rather than that nobody has run it.

Trajectory

No trajectory is published for this benchmark upstream, so none is shown. An empty chart would imply the scores exist and are flat.

Where results are tracked

Leaderboards benchwiki lists for this benchmark. These are links out, not sources this site ingests — no score below comes from any of them.

• Official MME leaderboardofficial