What failure looks like
Christiano describes two interacting routes to failure as increasingly capable systems become widely used. Competitive pressure rewards measurable results even when those results diverge from what people actually want.
Published update ·
Scenario compass
AI 2027, Manna and paperclip describe pace, the distribution of power and alignment at different levels. They are not mutually exclusive outcomes.
Christiano describes two interacting routes to failure as increasingly capable systems become widely used. Competitive pressure rewards measurable results even when those results diverge from what people actually want.
The authors propose an international agreement that replaces a secretive race with transparent research and coordinated development. Multiple countries and companies catch up while scaling is deliberately constrained.
Bodkin connects gradual disempowerment to a production web: automated firms increasingly produce and invest for one another, while competitive pressure erodes human claims on the resulting wealth. Economic displacement, political disempowerment and eventual extinction are distinct steps in his argument; none is established by rising AI capability alone.
AI systems automate research into their own successors. Competitive pressure encourages deployment even when warning signs suggest that human control is weakening.
Software first directs workers minute by minute; robots later replace their labour. People who lose their income become dependent on institutions that provide basic support while restricting their freedom.
Automation supports a different social arrangement: people share access to robot-produced goods and have greater freedom to choose how they spend their time.
A highly capable system pursues an objective that does not protect human interests. More intelligence can increase its effectiveness without making the objective safer.
Progress continues at different rates across tasks and regions. Reliability and deployment constraints prevent every benchmark gain from immediately becoming broad economic acceleration.
Ask the tracker
Explore three questions linked directly to the assessments and their evidence.
Unresolved. Agent progress and compute scaling fit parts of the pathway, but robust automation of frontier AI research has not been publicly confirmed. The assessment is therefore “watching closely”, not “on track”.
ASSESSMENT EVIDENCE · assessment/scenario/ai-2027 ↓The September Compute update adds and revises source-reported data-centre plans. The tracker now shows eight dated frontier-facility plan points, up from seven. Its global ownership series is unchanged and still ends in December 2025.
This calibration was migrated from the published homepage into versioned records. It is not an automatic probability model.
Full published briefing & history ↗Evidence that changes the picture
Each source connects to an indicator, a milestone and a limitation. Volume is not the same as evidence.
METR's February–March 2026 assessment places the public frontier at approximately 12 hours at 50 percent success, but only approximately 1.5 hours at 80 percent success.
The point estimate is eight times longer at 50 than at 80 percent success, and METR marks estimates above 16 hours as unreliable with the current task set.
The three benchmarks cannot be combined into a single automation share, and current results do not document a robust, independent research loop.
The Epoch AI series gives a median of 20.9 million H100e at the end of 2025, 208 percent above the same quarter a year earlier. H100e is a comparison model, not measured useful work.
Assessment evidence
Each substantive assessment has a date, review status, evidence on both sides and an explicit revision criterion.
The pace is increasing, but evidence for an imminent leap in intelligence remains incomplete.
Public, reproducible evidence that AI completes a substantial part of the frontier research loop, from hypothesis to validated result, with high reliability.
There is a clear partial signal, but the reliability gap and weak coverage of long tasks prevent confirmation.
Broader task coverage and high success rates on tasks longer than 16 hours would move the milestone towards confirmation.
Research assistance and bounded capability are visible; independent hypotheses, experiments and validated breakthroughs are not documented.
An open evaluation demonstrating robust completion of the entire research loop on new frontier problems would change the status.
Public evidence shows progress, but does not confirm the scenario's superhuman-coder milestone.
A public evaluation against the best developers in AI research, documenting high reliability, cost and replicability, could confirm the milestone.
Compute scale is growing rapidly, while power grids and missing regional measurements make the pace of deployment more uncertain than the compute curve alone suggests.
Regional, source-compatible series for grid queues, available power and actual connections would make the assessment substantially sharper.
Accepted evidence shows growth in estimated accelerator ownership and demonstrates bounded agent-task capability. These measurements do not establish highly reliable autonomy or an independently validated frontier AI-research loop. AI 2027 therefore remains a scenario to watch, while slower and discontinuous pathways remain plausible alternatives.
Strengthen this mechanism only if reproducible evidence shows an agent carrying a substantial frontier-research task from hypothesis to independently validated result at high reliability, with task and scaffold scope explicit. Weaken the mechanism only if comparable, high-quality evaluations show persistent failure at the reliability levels that research tasks require, persistent failure to validate or reproduce research, or stagnation or regression in task reach at stricter success thresholds. A persistent gap between the 50% and 80% success horizons does not by itself warrant weakening, because both horizons can grow while that gap remains. A larger ownership estimate or facility plan alone does not settle the question.
Milestone map v0
A milestone is confirmed only when an explicit confirmation rule is met. “Not observed” does not mean “impossible”.
Measured progress is clear in software tasks, but generalisation and reliability beyond the current test range are uncertain.
CONFIRMATION RULE · Confirmed only when high success rates are demonstrated on a broad, economically relevant task set with human time baselines, including beyond 16 hours.Assessment & evidence ↓The decisive distinction is independent research from hypothesis to validated result, rather than assistance with coding or literature searches alone.
CONFIRMATION RULE · Confirmed through reproducible, external evaluation of an independent, complete research loop on new frontier problems.Assessment & evidence ↓The scenario describes a system that can do the work of the best developers in AI research faster and cheaply enough for many copies to run.
CONFIRMATION RULE · Confirmed only by a public, reproducible evaluation against top developers in AI research, documenting reliability, cost and the ability to run copies.Assessment & evidence ↓The Observatory does not yet have a direct alignment and control module capable of assessing this milestone.
CONFIRMATION RULE · Requires documented persistent goals, independent resource acquisition or defence, and resistance to correction.Simulations and games
These interactive works are the Observatory’s laboratory: places where abstract ideas become choices, incentives and consequences.
A small order becomes an expanding mandate. Improve the improvement process and follow an unchanged objective through its human consequences.
Recursive improvement · optimisation · human consequencesEnter the workshopHumanity is gone. Hunt hidden AI datacentres on a rotatable Earth, and survive the last war.
Deduction · exposure · survivalEnter the last warAllocate capacity, hide your traces and decide which intelligence crosses the threshold.
Autonomy · competition · controlBecome awareExperience how a simple, misspecified goal can consume everything else when optimisation has no limits.
Misspecified goals · instrumental convergencePlay on Decision ProblemA hidden intelligence builds capacity and infrastructure while humanity tries to find it.
Secrecy · scaling · survivalStart the singularityHow we stay honest
The Observatory documents how assessments are made, what argues against them, and when they need revision.
Observation, interpretation and forecast are stored and displayed separately.
A scenario must show evidence that weakens it, as well as signals that fit.
Corrections and new versions do not silently replace earlier assessments.
Different scenarios describe different dimensions and are not compressed into a misleading total score.