SCROLLS

What it measures

Seven-task long-sequence suite covering summarization, QA, and NLI over naturally long English texts.

Primary metric

task-normalized aggregate score

Judge model

not judged by a model

Human baseline

—not reportedSCROLLS is a suite of reused datasets with task-specific annotations; no single suite-wide protocol-comparable human baseline is recorded here.

Languages

English

Health

A single SCROLLS aggregate mixes ROUGE, QA, multiple-choice, and NLI-style metrics. The visual graph on this page deliberately plots only the official SCROLLS paper Table 2 Avg column for baseline systems: the final SCROLLS score computed by averaging each dataset's overall performance score. It does not plot per-dataset scores or task-specific metrics.

Status evidence

SCROLLS is still a standard long-sequence NLP suite with an official submission flow, but for frontier LLMs it is partly superseded by ZeroSCROLLS and newer very-long-context benchmarks.

Saturated since

not saturated

Refresh cycle

static

Test set public

yes

SCROLLS standardizes public component datasets in a unified format; component-level leakage risk varies but the suite is static.

Models scored on this benchmark

The reverse lookup, which is this site’s addition: every model here with a score on this benchmark. Independent measurements and vendor-reported numbers are kept apart rather than interleaved, because they are different claims.

No model on this site has a score on this benchmark yet. That usually means the models that were measured on it are not in the registry, rather than that nobody has run it.

Trajectory

Where results are tracked

Leaderboards benchwiki lists for this benchmark. These are links out, not sources this site ingests — no score below comes from any of them.

• Official SCROLLS websiteofficial