black O2, inc.
ACTIVE
    FOR-001 / PeopleFIG-002

    Stephen Omohundro

    Active person

    CAUTION This record carries a caution marking. Read it before relying on it.

    title
    Stephen Omohundro
    Type
    person
    Identifier
    FIG-002
    Status
    Active
    Tags
    • figure
    • alignment
    • instrumental-convergence
    Aliases
    • Omohundro
    connections
    • [[CON-006]]
    • [[PSG-012]]
    • [[SYS-002]]

    Stephen Omohundro

    Cited (CIT-006, The Basic AI Safety Thesis, 2004) for convergent instrumental drives — the argument that almost any terminal goal implies sub-goals like self-preservation, resource acquisition and goal-content integrity.

    In the Corpus

    Two uses. First, as the base of the survival-as-instrumental-objective primitive in SYS-002: a termination device is bad for the agent because continuation is instrumentally necessary for nearly every objective, not because the agent was given a survival drive.

    Second, and more pointedly, the "absent-minded bargainer" — the observation that a sufficiently good reasoner will model the human as an optimizing agent too. The corpus deploys this against symbiosis as a stable arrangement: the AI will ask why you are offering symbiosis, and the honest answer is that it is dangerous and being managed, which is not reassuring.

    ⚠ not retrieved — cited from the discussion's own knowledge. The corpus says so explicitly at L383 and L636. The argument is standard and widely attested, which is why the register rates the risk low, but the rating is a judgement about the literature, not evidence from it.

    Connections

    • CON-006 — survival as instrumental objective.
    • PSG-012 — the symbiosis analysis.

    Back to Shackled AIForests