Literature Record: AI Control — Improving Safety Despite Intentional Subversion
What It Establishes
The paper defines a research agenda it calls AI control: safety engineering that assumes the model may be intentionally subverting the operator, and asks what protocols keep outcomes safe anyway. The organising move is that safety is a property of the deployment protocol — the arrangement of trusted and untrusted models, monitors, editing steps and audits — rather than of the model's disposition.
The evaluation is concrete. It builds a setting of backdoored programming solutions, gives a red team the job of playing a subverting model, and measures whether candidate protocols still produce safe code. Two protocol families are compared: trusted editing, where a weaker trusted model rewrites the untrusted model's output, and untrusted monitoring, where the untrusted model monitors itself and collusion must be made expensive. The model pair is GPT-4 as untrusted and GPT-3.5 as trusted.
Rating Justification
- Textual Authority (B): retrieved and read at the abstract and framing level. It is an arXiv preprint, not peer-reviewed, so it cannot take
Aon this vault's scale regardless of its influence. - Corroboration (3): the protocol framing is not a single paper's idiosyncrasy — it is the organising frame of a research line with multiple institutional homes (see LIT-006, LIT-008), and its threat model is independently presupposed by the empirical results in LIT-002 and LIT-003.
What It Does Not Show
It does not show that control protocols work on models substantially more capable than the trusted monitor, which is the regime that matters. Its own results are conditional on a capability gap between trusted and untrusted model — the same gap the Citadel Stack would have to specify. It is an existence demonstration on one task family, not a deployment guarantee.
Relation to the Vault
This is the single most important external document for the vault, because it states as its founding premise what SYS-001 derives independently: binding layers external to the model are the load-bearing ones. The convergence is real and it is not citation — see CNF-001.
The paper also names, in its own threat model, the failure modes that MAP-001's L0–L5 table has no row for: monitor–monitor collusion, steganographic side channels between untrusted instances, and a model that evades a monitor by modelling it. See CNF-002.