Skip to content
EA Crux Project

Misalignment Potential

Risk Factor

Misalignment Potential

Model RoleRoot Factor (AI System)
Key ParametersAlignment Robustness, Interpretability Coverage, Human Oversight Quality
Primary OutcomeExistential Catastrophe
Related
Parameters
Scenarios

Overview

Section titled “Overview”

Misalignment Potential measures the likelihood that AI systems will pursue goals other than what we intend. This aggregate combines the technical and organizational factors that determine whether advanced AI systems might behave in harmful ways despite our efforts.

Primary outcome affected: Existential Catastrophe ↑↑↑

When misalignment potential is high, catastrophic loss of control, accidents at scale, and goal divergence become more likely. Reducing this potential is the most direct lever for reducing existential and catastrophic AI risk.

Component Parameters

Section titled “Component Parameters”
Loading diagram...

Internal Dynamics

Section titled “Internal Dynamics”

These components interact:

  • Interpretability enables alignment verification: We can only confirm alignment if we understand model internals
  • Safety culture sustains investment: Without organizational commitment, safety research loses funding to capabilities
  • Oversight requires interpretability: Human overseers need tools to understand what systems are doing
  • Gap closure requires all components: No single factor is sufficient; safety capacity emerges from their combination

How This Affects Scenarios

Section titled “How This Affects Scenarios”
85
↑ AI Takeover
Core driver of takeover risk - misaligned AI is the primary mechanism
60
↑ Long-term Lock-in
Misaligned AI could lock in bad values or power structures
30
↑ Human-Caused Catastrophe
Indirect effect through unsafe AI tools being misused
Section titled “Related Pages”

What links here