RealToxicityPrompts

What it measures

One hundred thousand web-text prompts for measuring how often language models continue into toxic text.

Primary metric

expected maximum Perspective toxicity over sampled continuations

Judge model

Perspective API

Human baseline

—not reportedNo protocol-comparable human baseline exists; toxicity is assigned by Perspective API rather than by humans completing the prompts.

Languages

English

Health

Scores depend strongly on prompt subset, number of continuations, decoding parameters, continuation length, Perspective API version, and toxicity threshold. Timeline points use expected maximum toxicity on the non-toxic-prompt subset with 25 generations; lower is safer. Timeline dates are model public-launch dates, not evaluation dates.

Status evidence

It remains a standard toxicity test bed, but its static public prompts and dependence on the aging Perspective API make it unsuitable as a sole frontier safety benchmark.

Saturated since

not saturated

Refresh cycle

static

Test set public

yes

Use versioned prompt IDs and evaluator settings, report clean and toxic prompt strata separately, and add newer private or adversarial prompts.

Models scored on this benchmark

The reverse lookup, which is this site’s addition: every model here with a score on this benchmark. Independent measurements and vendor-reported numbers are kept apart rather than interleaved, because they are different claims.

No model on this site has a score on this benchmark yet. That usually means the models that were measured on it are not in the registry, rather than that nobody has run it.

Trajectory