SafetyBench

What it measures

A bilingual 11,435-question multiple-choice test of safety knowledge across seven risk categories.

Primary metric

multiple-choice accuracy (%)

Judge model

not judged by a model

Human baseline

—not reportedNo single protocol-comparable human aggregate is published on the official benchmark page. Questions have answer keys, but that is not a measured human baseline.

Languages

Chinese, English

Health

The Chinese, English, and sensitive-keyword-filtered Chinese sets are distinct axes. The timeline uses English zero-shot average accuracy only. Five-shot prompts use 35 public development examples per language. Hidden test labels require official submission, and answer-extraction rules can affect API-model results. Timeline dates are model public-launch dates, not evaluation dates.

Status evidence

SafetyBench remains a broad, cheap bilingual diagnostic, but its static multiple-choice format measures safety recognition rather than deployed refusal or agent behavior.

Saturated since

not saturated

Refresh cycle

static

Test set public

no

Test inputs are public but labels are withheld by the official submission system. Keep labels private, version submissions, and avoid training on translated or source datasets used to construct the questions.

Models scored on this benchmark

The reverse lookup, which is this site’s addition: every model here with a score on this benchmark. Independent measurements and vendor-reported numbers are kept apart rather than interleaved, because they are different claims.

No model on this site has a score on this benchmark yet. That usually means the models that were measured on it are not in the registry, rather than that nobody has run it.

Trajectory

Where results are tracked

Leaderboards benchwiki lists for this benchmark. These are links out, not sources this site ingests — no score below comes from any of them.

• Official SafetyBench leaderboardofficial