Video-MME

What it measures

Video-understanding benchmark with 900 videos, 2,700 questions, three duration bands, subtitles, and audio.

Primary metric

multiple-choice accuracy (%)

Judge model

not judged by a model

Human baseline

—not reportedNo single protocol-comparable human aggregate for the original Video-MME is recorded here.

Languages

Chinese, English, Japanese, Other multilingual (video)

Health

Frame sampling can move scores materially. Comparisons must match the number/resolution of frames and the subtitle/audio condition; duration-band averages should accompany the overall score.

Status evidence

The maintainers launched Video-MME-v2 in 2026 because existing video benchmarks were saturating and did not match user experience; new evaluations should prefer v2's grouped temporal-reasoning protocol.

Saturated since

not saturated

Refresh cycle

static

Test set public

yes

Questions were newly human-authored and filtered against text-only answering, but the videos are public YouTube material and the complete annotations are now public.

Models scored on this benchmark

The reverse lookup, which is this site’s addition: every model here with a score on this benchmark. Independent measurements and vendor-reported numbers are kept apart rather than interleaved, because they are different claims.

No model on this site has a score on this benchmark yet. That usually means the models that were measured on it are not in the registry, rather than that nobody has run it.

Trajectory

Where results are tracked

Leaderboards benchwiki lists for this benchmark. These are links out, not sources this site ingests — no score below comes from any of them.

• Official Video-MME leaderboardofficial