Skip to content
LLM Atlas

Benchmarks

OpenAI-MRCR

activeLong contextcontamination risk: highlaunched 2025-04-14

Benchmark metadata from benchwiki — this page mirrors their record so scores can be joined to it. The original is canonical.

What it measures

MRCR v2 8-needle long-context benchmark where a model must recover the requested instance from repeated similar requests in a synthetic conversation.

Primary metric
MRCR v2 8-needle mean_similarity
Judge model
not judged by a model
Human baseline
not reportedNo protocol-comparable human baseline is recorded on the dataset card.
Languages
English

Health

This page now uses the MRCR v2 (8-needle) version as tracked by LLM Stats. The visual graph deliberately plots only the LLM Stats headline score: mean_similarity converted from raw 0-1 values to percentages for this site's chart axis. It does not plot per-bin scores, other needle counts, cost, context length, or verification status. LLM Stats marks the 2026-07-22 snapshot as 21 self-reported and 0 verified results.

Status evidence
Actively tracked as MRCR v2 (8-needle) on LLM Stats, whose 2026-07-22 snapshot lists 21 self-reported model results with a wide 13.5%-91.5% displayed score spread, corresponding to raw mean_similarity values of 0.135-0.915.
Saturated since
not saturated
Refresh cycle
static
Test set public
yes

The data is public on Hugging Face; fresh synthetic generation would be needed for low-leakage private testing.

Models scored on this benchmark

The reverse lookup, which is this site’s addition: every model here with a score on this benchmark. Independent measurements and vendor-reported numbers are kept apart rather than interleaved, because they are different claims.

No model on this site has a score on this benchmark yet. That usually means the models that were measured on it are not in the registry, rather than that nobody has run it.

Trajectory

GPT-5.6 Sol (LLM Stats MRCR v2 8-needle) — 91.5 on 2026-07-22 (vendor-reported)GPT-5.6 Terra (LLM Stats MRCR v2 8-needle) — 89.6 on 2026-07-22 (vendor-reported)Claude Opus 4.6 (LLM Stats MRCR v2 8-needle) — 76 on 2026-07-22 (vendor-reported)GPT-5.5 (LLM Stats MRCR v2 8-needle) — 74 on 2026-07-22 (vendor-reported)Gemma 4 31B (LLM Stats MRCR v2 8-needle) — 66.4 on 2026-07-22 (vendor-reported)Gemini 3.1 Flash-Lite (LLM Stats MRCR v2 8-needle) — 60.1 on 2026-07-22 (vendor-reported)Gemini 3.6 Flash (LLM Stats MRCR v2 8-needle) — 54 on 2026-07-22 (vendor-reported)Gemma 4 26B-A4B (LLM Stats MRCR v2 8-needle) — 44.1 on 2026-07-22 (vendor-reported)Gemma 4 12B (LLM Stats MRCR v2 8-needle) — 43.4 on 2026-07-22 (vendor-reported)GPT-5.6 Luna (LLM Stats MRCR v2 8-needle) — 41.3 on 2026-07-22 (vendor-reported)GPT-5.4 mini (LLM Stats MRCR v2 8-needle) — 33.6 on 2026-07-22 (vendor-reported)GPT-5.4 nano (LLM Stats MRCR v2 8-needle) — 33.1 on 2026-07-22 (vendor-reported)Gemini 3.5 Flash (LLM Stats MRCR v2 8-needle) — 26.6 on 2026-07-22 (vendor-reported)Gemini 3.1 Pro (LLM Stats MRCR v2 8-needle) — 26.3 on 2026-07-22 (vendor-reported)Gemini 3 Pro (LLM Stats MRCR v2 8-needle) — 26.3 on 2026-07-22 (vendor-reported)Gemma 4 E4B (LLM Stats MRCR v2 8-needle) — 25.4 on 2026-07-22 (vendor-reported)Gemini 3 Flash (LLM Stats MRCR v2 8-needle) — 22.1 on 2026-07-22 (vendor-reported)Gemini 3.5 Flash-Lite (LLM Stats MRCR v2 8-needle) — 21.3 on 2026-07-22 (vendor-reported)Gemma 4 E2B (LLM Stats MRCR v2 8-needle) — 19.1 on 2026-07-22 (vendor-reported)Gemini 2.5 Pro Preview 06-05 (LLM Stats MRCR v2 8-needle) — 16.4 on 2026-07-22 (vendor-reported)Gemma 3 27B (LLM Stats MRCR v2 8-needle) — 13.5 on 2026-07-22 (vendor-reported)20262026920
independent · vendor-reported. Points are not joined: the scores come from different models, harnesses and protocol versions, and a line between them would claim a continuity the data does not have.

Where results are tracked

Leaderboards benchwiki lists for this benchmark. These are links out, not sources this site ingests — no score below comes from any of them.

Source

This record is maintained by benchwiki and mirrored here. Read the original. Upstream last updated 2026-08-12; fetched 2026-10-19 by this site.