Skip to content
LLM Atlas

Benchmarks

InfiniteBench

nearing saturationLong contextcontamination risk: highlaunched 2024-02-21

Benchmark metadata from benchwiki — this page mirrors their record so scores can be joined to it. The original is canonical.

What it measures

100k+ token benchmark mixing realistic and synthetic tasks across English, Chinese, code, math, and retrieval.

Primary metric
task-specific accuracy / ROUGE aggregate
Judge model
not judged by a model
Human baseline
not reportedNo single protocol-comparable human baseline is recorded.
Languages
Chinese, Code, English

Health

Input lengths and metrics vary sharply by task; quote per-task scores rather than a single aggregate. The visual graph on this page deliberately plots only the official InfiniteBench paper Table 3 Average row for the four baseline models. It does not plot the individual subset rows.

Status evidence
InfiniteBench remains valuable for 100k+ token stress testing, but its public static split and newer complementary suites make it better classified as nearing-saturation than as the freshest active frontier signal.
Saturated since
not saturated
Refresh cycle
static
Test set public
yes

The data and construction scripts are public. Synthetic tasks can be regenerated, but the released benchmark split is static.

Models scored on this benchmark

The reverse lookup, which is this site’s addition: every model here with a score on this benchmark. Independent measurements and vendor-reported numbers are kept apart rather than interleaved, because they are different claims.

No model on this site has a score on this benchmark yet. That usually means the models that were measured on it are not in the registry, rather than that nobody has run it.

Trajectory

GPT-4 (paper Table 3 Average) — 45.63 on 2024-02-21 (independent)YaRN-Mistral (paper Table 3 Average) — 19.96 on 2024-02-21 (independent)Kimi-Chat (paper Table 3 Average) — 34.73 on 2024-02-21 (independent)Claude 2 (paper Table 3 Average) — 37.06 on 2024-02-21 (independent)20242024460
independent · vendor-reported. Points are not joined: the scores come from different models, harnesses and protocol versions, and a line between them would claim a continuity the data does not have.

Where results are tracked

Leaderboards benchwiki lists for this benchmark. These are links out, not sources this site ingests — no score below comes from any of them.

Source

This record is maintained by benchwiki and mirrored here. Read the original. Upstream last updated 2026-08-12; fetched 2026-10-19 by this site.