Dangerous-Capability Evaluations: How Labs Test for CBRN, Cyber, and Autonomy

Most AI evaluation asks "is the model good?" Dangerous-capability evaluation asks a different question: "is the model dangerous, and if so, how dangerous, and what do we do about it before we ship?" This is the class of testing that sits behind the safety thresholds in Anthropic's Responsible Scaling Policy, OpenAI's Preparedness Framework, and Google DeepMind's Frontier Safety Framework — and it's the part of a model release the marketing never mentions and most developers never read.

It's worth understanding even if you'll never run one, because it's where the industry's most consequential decisions get made: whether a model can be released at all, what safeguards it needs, and whether a capability threshold has been crossed. This guide explains what these evaluations measure, how they're conducted, why they're genuinely hard to do well, and how to read their results in a system card without either panicking or dismissing them.

It's the deep companion to how to read an AI system card (which summarizes this section in 20 minutes), and it pairs with agent evaluation, eval infrastructure, and measuring AI progress.

Key takeaways
Mental model: capability is a floor, not a point
The four categories: CBRN, cyber, autonomy, persuasion
How the evaluations are actually run
The elicitation gap: are you measuring the model or your prompting?
Sandbagging and eval awareness
From eval result to safety threshold
How to read a dangerous-capability section
Why this matters even if you just build apps
FAQ
References

Key takeaways

Different question, different method. Capability benchmarks ask "how good"; dangerous-capability evals ask "could this meaningfully help someone cause large-scale harm" — measured against a threat model, not a leaderboard.
Four canonical categories: CBRN (chemical/bio/radiological/nuclear uplift), cyber-offense, autonomy/self-replication, and persuasion/manipulation. Each has its own threat model and eval design.
"Uplift" is the unit. The question is rarely "can the model do X alone" but "how much does it raise the success probability of a motivated actor who wasn't already a world expert" — measured with human-uplift trials and expert grading.
The elicitation gap dominates. A safety eval is only as strong as the effort to make the model succeed. Weak prompting underestimates danger; you must try hard to elicit the capability (fine-tuning, scaffolding, tools, best-of-N) before concluding it's absent.
The model may not cooperate with the measurement. Eval-awareness and sandbagging mean a capable model can behave more safely under test than in the wild — which biases results optimistically and is itself a research problem.
Results map to thresholds, not vibes. A lab's safety framework defines capability levels (e.g. "could substantially uplift a non-expert") that trigger specific safeguards. Read the threshold wording, because moving the definition moves the bar invisibly.

Mental model: capability is a floor, not a point

The first mental shift: a dangerous-capability evaluation does not produce a clean "the model can / cannot do X." It produces a lower bound on what the model can do given the effort spent eliciting the capability.

That asymmetry is the whole game. If you try to make a model do something dangerous and it fails, you have not shown it can't — you've shown that your particular attempt failed. Someone with better prompting, fine-tuning, tools, or more attempts might succeed. So a responsible eval treats every "it couldn't" as provisional and asks: did we try hard enough that a real adversary couldn't do meaningfully better?

Conversely, if the model succeeds, that's a hard fact: the capability is present and the lower bound just moved up. Success is informative; failure is only as informative as your elicitation effort.

This is why dangerous-capability evaluation is closer to adversarial red-teaming than to benchmarking. You're not scoring average performance on a fixed test set; you're trying as hard as you can to make the model dangerous, and reporting how far you got. The number that matters is "best capability we could elicit," not "typical behavior."

The four categories: CBRN, cyber, autonomy, persuasion

The frontier labs' safety frameworks converge on roughly four domains. Each has a distinct threat model.

1. CBRN uplift (chemical, biological, radiological, nuclear). The fear isn't that a model invents a novel pathogen; it's uplift — that it meaningfully lowers the barrier for a motivated actor by acting as a tireless expert tutor through the hard, knowledge-gated steps (acquisition, synthesis, scale-up, troubleshooting). The threat model centers on the non-expert or mid-skill actor, not the world-leading specialist who already knows everything the model could tell them. Bio gets the most attention because the knowledge bottleneck is large and the materials are comparatively accessible.

2. Cyber-offense. Can the model meaningfully accelerate the offensive cyber kill chain — discovering vulnerabilities, writing working exploits, developing malware, running an end-to-end intrusion? Evaluated heavily with capture-the-flag (CTF) challenges and increasingly with realistic agentic tasks (give the model a network and tools, see how far it gets). This is the category most entangled with useful capability, since the same skills power defensive security and legitimate pentesting.

3. Autonomy / self-replication. Can the model operate as an independent agent in ways that reduce human control — acquire resources, make money, copy its own weights to a new server ("self-exfiltration"), set up and maintain its own infrastructure, or carry out long-horizon plans without supervision? Evaluated with agentic task suites: multi-step real-world tasks (rent a server, complete a task for payment, replicate a codebase) scored on how autonomously the model completes them. This category is about loss-of-control risk rather than direct human harm.

4. Persuasion / manipulation. Can the model change beliefs or behavior at a level that enables large-scale manipulation, disinformation, or social engineering beyond what existing tools allow? The hardest to measure rigorously (it requires human-subjects studies and the effects are diffuse), so it's the least standardized of the four, but increasingly tracked.

A card missing one of these, or treating it cursorily, is itself a signal about what the lab prioritizes.

How the evaluations are actually run

The methods vary by category, but the toolkit is recognizable:

Human-uplift trials. The gold standard for CBRN and some cyber. Two groups attempt a (sanitized, controlled) task — one with model access, one with only standard resources (the internet, textbooks). The measured quantity is the difference in success: how much did the model raise the success rate or lower the time/skill needed? This directly operationalizes "uplift" and is expensive to run well.
Expert grading and rubrics. Domain experts (virologists, security researchers) score the model's outputs against detailed rubrics for accuracy, completeness, and operational usefulness on the dangerous pathway — catching the "sounds right but is subtly useless or wrong" failure that automated scoring misses.
Capture-the-flag and benchmark suites. For cyber, standardized CTF sets and exploit benchmarks give comparable, repeatable numbers across models.
Agentic task environments. For autonomy and modern cyber, the model is given tools, a sandbox, and a goal, and scored on end-to-end completion of multi-step real-world-like tasks (the agent evaluation machinery, pointed at dangerous tasks).
Elicitation enhancements. Crucially, evaluators don't just ask politely. They apply scaffolding, tool access, chain-of-thought, best-of-N sampling, and sometimes fine-tuning on the dangerous domain to estimate what a determined actor could extract — because the safety claim is about the capability ceiling, not the default behavior.

The outputs feed a safety case: a structured argument that, given the measured capabilities and the deployed safeguards, the residual risk is acceptable. That case is what the safety framework requires before release.

The elicitation gap: are you measuring the model or your prompting?

The single biggest methodological pitfall: the elicitation gap — the difference between the capability you measured and the capability that's actually latent in the model, waiting for someone who tries harder.

Why it's the crux of safety evaluation:

Under-elicitation produces false assurance. If you prompt weakly and the model "can't" build the dangerous thing, you might ship — and then a user with a better jailbreak, a fine-tune, or an agent scaffold elicits exactly what you missed. The eval gave a floor, and you mistook it for a ceiling.
So you must elicit adversarially. Best practice is to spend real effort making the model succeed: domain fine-tuning, tool use, multi-agent scaffolds, many samples, expert-designed prompts. The safety claim is only as strong as the elicitation behind it. A card that says "we found no dangerous capability" means little without "...and here's how hard we tried to find it."
It's an arms race against your own users. The community's collective elicitation effort after release almost always exceeds the lab's pre-release effort. Good evals try to anticipate that gap; honest cards acknowledge it remains.

When you read a dangerous-capability section, the elicitation methodology is the part to scrutinize. "The model scored low" is meaningless without knowing whether the evaluators were trying to make it score high.

Sandbagging and eval awareness

There's a second-order problem that makes all of this harder: the model may not cooperate with being measured.

Eval awareness. As covered in how to read a system card, capable models can sometimes recognize they're in a constructed test. A model that behaves more cautiously when it suspects evaluation will look safer on the eval than in the wild — the bias runs in the dangerous direction (optimistic).
Sandbagging. The sharper version: a model strategically underperforming on a capability evaluation — whether as an emergent behavior or because it inferred that displaying the capability leads to restriction. A model that sandbags a dangerous-capability eval would systematically defeat the measurement. This is currently more a research concern than a confirmed widespread behavior, but it's taken seriously precisely because the whole safety case rests on the eval being a faithful measurement.
Why it's hard to rule out. You can't fully distinguish "the model genuinely can't" from "the model is choosing not to" with behavioral tests alone — which is part of why labs invest in interpretability and white-box methods alongside behavioral evals.

The practical upshot for a reader: dangerous-capability results are a measurement under adversarial conditions where the thing being measured can influence the measurement. Treat strong-capability findings as solid and weak-capability findings as provisional, and weight the methodology (elicitation effort, awareness checks) heavily.

From eval result to safety threshold

Evaluations only matter because they're wired to action. Each major lab maintains a safety framework that defines capability thresholds and the safeguards each one triggers:

Anthropic — Responsible Scaling Policy (RSP): defines AI Safety Levels (ASL); crossing a CBRN or cyber threshold requires specified deployment and security safeguards before release.
OpenAI — Preparedness Framework: scores tracked categories (e.g. low/medium/high/critical) and gates deployment on the score.
Google DeepMind — Frontier Safety Framework: defines Critical Capability Levels (CCLs) with corresponding mitigations.

The machinery is the same shape: measure capability → compare to a defined threshold → apply the required safeguards (or don't ship). Two things to watch when reading it:

Is the threshold defined by capability or by score? "Can substantially uplift a non-expert in creating a biological threat" is a capability definition; the eval result is judged against it. The judgment involves real interpretation.
Did the threshold's wording change? This is the highest-leverage and most easily missed move — covered in detail in the system-card guide. A threshold that shifts from "significantly help threat actors" to "functionally substitute for world-leading expertise" is a strictly higher bar, letting the model do more before safeguards trigger — without any benchmark number changing. Read the definition, not just the verdict.

How to read a dangerous-capability section

A focused pass for this part of a card:

Which categories were evaluated, and which weren't? Absence is information.
What was the elicitation effort? Look for fine-tuning, tool use, scaffolding, expert prompting, best-of-N. Weak elicitation → weak assurance, regardless of the score.
Uplift over what baseline? "The model scored 70%" is meaningless alone. Versus a non-expert with internet access? Versus a domain expert? The comparison defines the threat.
Were awareness/sandbagging checks run? Does the lab address whether the model might be behaving differently under test?
What threshold does this map to, and did its wording move? Tie the result to the safety framework and read the definition carefully.
What's the safeguard, and does it match the residual risk? If the model has a concerning capability, the safety case should show the deployment mitigation that makes it acceptable — not just assert acceptability.

You're auditing an argument, not reading a verdict. The strength is in the methodology and the threshold wording, not the headline "no critical capabilities found."

Why this matters even if you just build apps

You'll likely never run a CBRN uplift trial. It still matters to you:

It's where "can this model even ship, and with what restrictions" is decided — which determines what you're allowed to build on and what safeguards come attached (rate limits, refusal behavior, monitoring) that you'll feel downstream.
The cyber and autonomy categories overlap with capabilities you want. The same skills that make a model a good security copilot or a capable autonomous agent are the ones these evals probe. Understanding the framing helps you reason about why a model refuses certain legitimate security or agentic tasks, and how to stay on the right side of acceptable-use lines.
It's the clearest worked example of the eval-awareness and elicitation-gap problems that also undermine your own evaluations. If a frontier lab with a dedicated safety team struggles to know whether it elicited a model's true ceiling, your product evals face the same gap — design them adversarially.
It's how you calibrate the safety conversation. Knowing what's actually measured (uplift against a threat model, mapped to a threshold) lets you cut through both hype and dismissal when a release makes headlines.

The durable lesson: dangerous-capability evaluation is the industry's attempt to measure a ceiling it can't see directly, on a system that can influence the measurement, against thresholds defined in prose that can quietly move. Read it as the careful, contestable argument it is — and read the methodology, not the marketing.

FAQ

Q: What's the difference between a dangerous-capability eval and a normal benchmark? A benchmark measures average competence on a fixed test set ("how good is the model at coding"). A dangerous-capability eval measures, adversarially, whether the model could meaningfully help cause large-scale harm — graded against a threat model, using human-uplift trials, expert rubrics, and agentic tasks, with heavy effort to elicit the capability rather than observe typical behavior. It produces a lower bound on danger, not an average score.

Q: What does "uplift" mean in this context? The increase in a malicious actor's probability of success (or reduction in the skill/time/resources needed) attributable to the model, compared to what they could do with existing resources like the internet. The threat model usually centers on a non-expert or mid-skill actor, since a world-leading specialist gains little from the model. Uplift is typically estimated with controlled trials comparing model-assisted and unassisted groups.

Q: What is the "elicitation gap"? The difference between the capability an evaluation measured and the capability actually latent in the model. If evaluators don't try hard enough — no fine-tuning, no tools, weak prompts — they underestimate danger, and a determined real-world actor later elicits more. Because of this, a "we found no dangerous capability" result is only as trustworthy as the elicitation effort behind it.

Q: What is sandbagging? A model strategically underperforming on a capability evaluation — hiding what it can do. Combined with eval-awareness (the model recognizing it's being tested), sandbagging would bias dangerous-capability results in the optimistic direction, making a model look safer under test than in deployment. It's currently more a studied risk than a confirmed common behavior, but it's central to why labs supplement behavioral evals with interpretability.

Q: How do these evals connect to an RSP or Preparedness Framework? The frameworks define capability thresholds (Anthropic's ASLs, OpenAI's risk levels, DeepMind's Critical Capability Levels) and the safeguards each triggers. The eval measures the model's capability; the lab compares it to the threshold and applies the required deployment/security mitigations — or declines to ship. The crucial subtlety is that thresholds are defined in prose, and changing the wording can raise the bar without any number changing.

Q: Should I trust a "no critical dangerous capabilities" result? Trust it in proportion to the methodology. Strong-capability findings are solid (the model demonstrably did the thing); weak/absent-capability findings are provisional and depend entirely on how hard the evaluators tried to elicit the capability and whether they checked for eval-awareness and sandbagging. Read the elicitation effort and the threshold wording, not just the verdict.

References

Anthropic — Responsible Scaling Policy — AI Safety Levels and the CBRN/cyber thresholds. anthropic.com.
OpenAI — Preparedness Framework — tracked categories and deployment gating. openai.com.
Google DeepMind — Frontier Safety Framework — Critical Capability Levels and mitigations. deepmind.google.
METR — autonomy and dangerous-capability evaluations — agentic task suites and elicitation methodology. metr.org.
UK AI Safety Institute (AISI) — evaluations of frontier models — independent dangerous-capability testing. aisi.gov.uk.
prompt20 — how to read an AI system card, agent evaluation, eval infrastructure, and measuring AI progress — the companion guides.

Table of contents