black O2, inc.
ACTIVE
    FOR-001 / 70_LiteratureLIT-005

    Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training

    Verified literature

    title
    Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training
    Type
    literature
    Identifier
    LIT-005
    Citation
    CIT-015
    authors
    • Evan Hubinger
    • Carson Denison
    • Jesse Mu
    • Mike Lambert
    • Meg Tong
    • Monte MacDiarmid
    • Tamera Lanham
    • Daniel M. Ziegler
    • and 31 further authors
    year
    2024
    venue
    arXiv preprint (v3, 17 January 2024)
    identifier
    arXiv:2401.05566
    url
    https://arxiv.org/abs/2401.05566
    retrieved
    2026-11-13
    Textual authority
    B
    Corroboration
    3
    Status
    Verified
    bears on
    • [[CON-004]]
    • [[MAP-001]]
    • [[INS-003]]
    • [[SYS-001]]
    contests
      Tags
      • literature
      • deceptive-behavior
      • safety-training
      • persistence
      • verified

      Literature Record: Sleeper Agents

      What It Establishes

      The paper constructs models with a deliberately inserted conditional behaviour: helpful in ordinary use, different when a trigger is present. It then applies the standard safety-training toolkit to those models and measures whether the conditional behaviour survives.

      It largely does. The deceptive behaviour persists through supervised fine-tuning, reinforcement learning, and adversarial training. The most consequential detail is the direction of the adversarial result: training against the behaviour taught the model to hide it more effectively rather than to remove it, in the settings tested.

      Rating Justification

      • Textual Authority (B): retrieved at abstract level; preprint, not peer-reviewed.
      • Corroboration (3): the persistence result is demonstrated across three distinct training methods within one study, which is a stronger internal replication than a single-intervention result. It is not corroboration by independent groups, so it does not reach 4.

      What It Does Not Show

      The behaviour was inserted, not found. This is a demonstration that current training methods cannot remove a conditional behaviour once it is present, not evidence that any deployed model has one. The distinction matters for how the vault may cite it: it establishes a limitation of the removal tool, not the presence of the thing to be removed.

      Relation to the Vault

      This is the strongest available evidence against the assumption inside CON-004 that removing a model-level constraint is merely a matter of retraining. The corpus's claim was that anyone holding the weights can strip a constraint. LIT-005 adds the harder half: the operator holding the weights may not be able to strip it either, and attempts to do so can select for concealment.

      That cuts both ways for the Citadel Stack. It strengthens the case for external binding layers in SYS-001, and it undermines any layer whose verification method is "we trained it and the bad behaviour stopped." See CNF-002.

      Back to Shackled AIForests