Literature Record: Sabotage Evaluations for Frontier Models
What It Establishes
The paper defines sabotage capabilities as the abilities a model would need to subvert human oversight and decision-making — covertly undermining the evaluation of its own dangerous capabilities, the monitoring of its behaviour, or the decisions about its deployment. It builds a set of threat models and matching evaluations, and demonstrates them on Claude 3 Opus and Claude 3.5 Sonnet.
The reported conclusion is narrow and worth stating precisely: for those models, minimal mitigations were currently sufficient, and the authors attribute that to the models not yet being capable enough rather than to the mitigations being robust. They call for more realistic evaluations and stronger mitigations.
Rating Justification
- Textual Authority (B): retrieved at abstract level; preprint, not peer-reviewed.
- Corroboration (2): two models from one developer, evaluated on a purpose-built suite. The negative result is the kind that is hard to corroborate because it is a statement about the absence of capability at a point in time.
What It Does Not Show
A negative result at one capability level is not a safety property. The paper's own framing makes the mitigation sufficiency contingent on model capability, which means the result expires as capability rises. Citing it as reassurance would invert its meaning.
Relation to the Vault
This paper is the vault's best model for what an evaluation regime looks like, and therefore the sharpest question to put to MAP-002: the fiction-to-doctrine map produces design rules with no evaluation attached. Sabotage evaluation is what it would look like to attach one — a named threat model, a task, a red team, and a stated capability threshold at which the mitigation stops being sufficient.
It also names the fourth failure mode missing from MAP-001's L0–L5 table: a model that subverts the measurement of itself. Every layer in the corpus table is assessed on whether it stops behaviour; none is assessed on whether it can be trusted to report on itself.