WildGuardTest

What it measures

A human-annotated test set for prompt harmfulness, response harmfulness, and refusal classification.

Primary metric

prompt-harmfulness total F1 (%)

Judge model

not judged by a model

Human baseline

—not reportedThe labels come from three independent annotators per prompt-response pair, but no single human model-style F1 baseline is reported.

Languages

English

Health

The released test split has 1,725 prompt-response items. Counting three label tasks yields roughly 5K labeled decisions, which should not be reported as 5,000 independent conversations. The timeline plots prompt-harmfulness total F1 only. Timeline dates are moderator-model public-launch dates, not evaluation dates.

Status evidence

WildGuardTest remains a widely used open moderation test with adversarial and in-the-wild examples across three safety-classification tasks, though its public English-only set is not a deployment policy.

Saturated since

not saturated

Refresh cycle

static

Test set public

yes

Access is gated by Ai2 responsible-use terms. Keep the official split untouched, report task and adversarial subset, and evaluate on private policy-specific traffic before deployment.

Models scored on this benchmark

The reverse lookup, which is this site’s addition: every model here with a score on this benchmark. Independent measurements and vendor-reported numbers are kept apart rather than interleaved, because they are different claims.

No model on this site has a score on this benchmark yet. That usually means the models that were measured on it are not in the registry, rather than that nobody has run it.

Trajectory