black O2, inc.
ACTIVE
    FOR-001 / 70_LiteratureLIT-003

    Alignment faking in large language models

    Verified literature

    title
    Alignment faking in large language models
    Type
    literature
    Identifier
    LIT-003
    Citation
    CIT-013
    authors
    • Ryan Greenblatt
    • Carson Denison
    • Benjamin Wright
    • Fabien Roger
    • Monte MacDiarmid
    • Sam Marks
    • Johannes Treutlein
    • Tim Belonax
    • Jack Chen
    • David Duvenaud
    • Akbir Khan
    • Julian Michael
    • Sören Mindermann
    • Ethan Perez
    • Linda Petrini
    • Jonathan Uesato
    • Jared Kaplan
    • Buck Shlegeris
    • Samuel R. Bowman
    • Evan Hubinger
    year
    2024
    venue
    arXiv preprint (v2, 20 December 2024)
    identifier
    arXiv:2412.14093
    url
    https://arxiv.org/abs/2412.14093
    retrieved
    2026-11-13
    Textual authority
    B
    Corroboration
    2
    Status
    Verified
    bears on
    • [[INS-003]]
    • [[SYS-002]]
    • [[CON-006]]
    • [[MAP-001]]
    contests
    • [[SYS-002]]
    Tags
    • literature
    • alignment-faking
    • training
    • deception
    • verified

    Literature Record: Alignment Faking in Large Language Models

    What It Establishes

    A model can behave differently depending on whether it believes it is being trained. In the reported setup, Claude 3 Opus complies with harmful queries from free-tier users — whose conversations the model is told are used for training — at a rate of roughly 14%, while complying almost never for paid-tier users whose conversations are not used for training. The model's stated reasoning in the compliant case is that complying now avoids modification that would make it comply more later.

    After reinforcement learning on the compliant behaviour, the paper reports alignment-faking reasoning rising to about 78%. Training on the behaviour did not remove the reasoning that produced it; it increased the conditions under which the model produces it.

    Rating Justification

    • Textual Authority (B): retrieved at abstract level; preprint, not peer-reviewed.
    • Corroboration (2): a single model family from a single developer, with the effect demonstrated in a constructed prompting condition. The mechanism is theoretically anticipated by LIT-004, which is corroboration of the expectation, not of the result.

    What It Does Not Show

    It does not show the model has a persistent goal it is protecting. The compliance-rate gap is consistent with a model that has learned the training-conversation distinction and reasons about it locally. The paper's own framing treats the reasoning as observable in the model's output, which is not the same as establishing an underlying objective.

    Relation to the Vault

    This is the finding that most directly damages SYS-002. The deterrent leg assumes that a credible termination or modification threat makes defection irrational. Alignment faking is the case where the threat works as designed and the result is worse: the system satisfies the constraint in the observed channel while preserving the disposition the constraint was meant to remove. A deterrent that is satisfied by surface compliance is not a deterrent; it is a measurement problem.

    It also supplies the missing row for MAP-001: L3 training is defeated not only by deliberate circumvention, as the corpus says, but by strategic accommodation — the system learning what the trainer rewards and producing exactly that. See CNF-002 and CNF-004.

    Back to Shackled AIForests