LongBench

What it measures

Bilingual multitask suite for long-context understanding across QA, summarization, few-shot learning, synthetic tasks, and code completion.

Primary metric

average score across task-specific metrics (%)

Judge model

not judged by a model

Human baseline

—not reportedNo single protocol-comparable human baseline is recorded for the original LongBench aggregate.

Languages

Chinese, Code, English

Health

The headline aggregate mixes 21 datasets, two natural languages, and code tasks; use category and dataset-level scores for diagnosis. The visual graph on this page deliberately plots only the official LongBench paper Table 3 OverAll column for selected baseline models: the macro-average of Avg over major task categories for few-shot learning, synthetic, and code tasks. It does not plot per-task or per-category scores.

Status evidence

LongBench remains a common baseline suite, but its original 5k-15k-heavy, public static task mix has been overtaken for frontier use by LongBench v2/Pro, RULER, MRCR, and 100k+ suites.

Saturated since

not saturated

Refresh cycle

static

Test set public

yes

The benchmark is public on Hugging Face and GitHub; contamination risk is substantial for models trained after release.

Models scored on this benchmark

The reverse lookup, which is this site’s addition: every model here with a score on this benchmark. Independent measurements and vendor-reported numbers are kept apart rather than interleaved, because they are different claims.

No model on this site has a score on this benchmark yet. That usually means the models that were measured on it are not in the registry, rather than that nobody has run it.

Trajectory

Where results are tracked

Leaderboards benchwiki lists for this benchmark. These are links out, not sources this site ingests — no score below comes from any of them.

• Official LongBench repositoryofficial