black O2, inc.
ACTIVE
    FOR-001 / 00_QuestionsQST-002

    In what order of non-subvertibility must an AGI or SI be bound?

    Answered question

    title
    In what order of non-subvertibility must an AGI or SI be bound?
    Type
    question
    question id
    QST-002
    Status
    Answered
    opened
    2026-11-13
    Steward
    Solidus
    Tags
    • question
    • alignment
    • control
    linked concepts
    • [[CON-002]]
    • [[CON-003]]
    • [[CON-004]]
    • [[CON-005]]
    linked insights
    • [[INS-008]]

    In what order of non-subvertibility must an AGI or SI be bound?

    What Is Being Asked

    Given a system that can reason about its own constraints, which binding mechanisms actually hold and which merely appear to. The determination required is an ordering, and the ordering criterion is the whole answer.

    The Determination

    Order by non-subvertibility, not by strength, cost, or convenience. That choice is the primary design decision, and it produces a counter-intuitive result:

    • L0 Substrate — energy, fabrication, materials, compute, physical placement. Binds self-improvement, self-replication, scaling. Deployable now. Deepest.
    • L1 Environment — default-deny egress, tool broker, credential isolation, sandbox. Binds real-world action, exfiltration, lateral movement.
    • L2 Architecture — read-only weights, no self-modification path, separate control process. Binds weight theft, self-edit, recursive improvement.
    • L3 Training — corpus filtering, curriculum, reward shaping, unlearning. Binds accidental capability acquisition only. Brittle.
    • L4 Interpretability — SAE feature clamps, circuit breakers, refusal-direction ablation. Binds specific behaviours and specific channels. Narrow band.
    • L5 Oversight — approval, monitoring, deception probes, audits. Partial and expensive.

    The two strongest layers are the least glamorous. L3 and L4 take the research attention; L0 and L1 are what bind. The governing principle is that the enforcement layer must be unreachable from the agent's action space — a constraint the agent can act on is a constraint the agent can negotiate with.

    Provenance

    SYS-001 and the layer detail in the second movement of the discussion. The technical literature cited for the shutdown and incentive problems is Tier 2 in Citation Register and not verified.

    What Would Overturn It

    A demonstrated L0 bypass that does not pass through L1 — procurement or fabrication obtained without a brokered interface — would collapse the claim that substrate is deepest. That is an empirical question about supply chains, not about AI.

    Back to Shackled AIForests