🎯 Understanding AI Risk
The core argument for why AI might pose existential risk, broken into key claims and cruxes.
AI Safety Wiki
Understanding AI risk, alignment, and the path to safe artificial intelligence
What Is This?
Section titled “What Is This?”This wiki maps the landscape of AI existential risk—the arguments, disagreements, organizations, and interventions that matter for ensuring advanced AI goes well for humanity.
Whether you’re new to AI safety or a researcher looking for a comprehensive reference, this site aims to be your guide.
Core Content
Section titled “Core Content”⚖️ Key Debates
Structured arguments on contested questions:
🏢 Organizations
Profiles of key players:
- Frontier labs (Anthropic, OpenAI, DeepMind)
- Safety research orgs (MIRI, ARC, Redwood)
- Funders and governance
🛡️ Safety Approaches
Technical and governance solutions:
- Technical approaches (interpretability, RLHF, AI control)
- Governance (compute governance, international coordination)
- Research agendas compared
📐 Key Parameters
Foundational variables AI affects in both directions:
- All parameters (alignment robustness, racing intensity, societal trust)
- A symmetric framework connecting risks and interventions
- Track what matters, not just what could go wrong
Analysis & Visualization
Section titled “Analysis & Visualization”📊 History
AI safety timeline from the Dartmouth conference to the present day.
🗺️ Entity Graph
Visual dependency graph showing how entities connect.
🔮 Scenarios
Future projections for how AI development might unfold.
📚 Knowledge Base
Browse all entries — risks, responses, organizations, people, and cruxes.
Learning Paths
Section titled “Learning Paths”New to AI safety? Start with:
- Introduction - What is AI safety and why it matters
- Core argument overview
- Key Parameters - The variables that matter
Want to contribute? Explore:
- Intervention analysis - What can you do?
- Funding landscape - How to get funded
- Organizations - Who’s working on this
Looking for depth? Try:
- Key Parameters - The variables that matter for AI outcomes
- Key debates - Strongest arguments on each side
- Research agendas - Compare approaches
Key Numbers
Section titled “Key Numbers”| Question | Estimates |
|---|---|
| P(transformative AI by 2040) | 40-80% (varies by source) |
| P(doom) estimates | 5-90% (wide disagreement) |
| AI safety researchers | ~300-1000 FTE |
| Annual safety funding | ~$100-500M |
| Frontier lab safety spend | ~$50-200M combined |
See the dashboard for more details.
Featured People
Section titled “Featured People”Key voices in AI safety:
- Dario Amodei - Anthropic CEO, “10-25% doom”
- Eliezer Yudkowsky - MIRI, most pessimistic public voice
- Paul Christiano - ARC founder, influential alignment researcher
- Geoffrey Hinton - “Godfather of AI”, recent safety advocate
- Stuart Russell - UC Berkeley, Human Compatible author
This Wiki Is…
Section titled “This Wiki Is…”✅ Comprehensive — Covers technical, governance, and strategic perspectives
✅ Structured — Organized by cruxes, not just topics
✅ Parameter-oriented — Tracks foundational variables, not just risks
✅ Interactive — Timeline, risk maps, argument maps
✅ Practical — Career and funding guidance
Limitations & Perspective
Section titled “Limitations & Perspective”This wiki is not neutral. It was created within the AI safety community and reflects that perspective. While we strive to present counterarguments fairly, readers should be aware:
What this wiki does well:
- Steelmans the case for AI existential risk
- Maps the landscape of AI safety arguments, organizations, and research
- Presents the range of views within the AI safety community
What this wiki does less well:
- Representing perspectives that reject the x-risk framing entirely
- Engaging deeply with AI ethics/fairness concerns (often dismissed as “near-term”)
- Covering non-Western perspectives on AI development
- Quantifying uncertainty honestly (probability estimates should be treated as rough intuitions)
Key assumptions embedded in this wiki:
- That “existential risk” is a coherent concept for AI
- That the theoretical arguments for concern (orthogonality, instrumental convergence) are basically sound
- That the AI safety community’s research agenda is on the right track
If you’re skeptical of these assumptions, this wiki may still be useful for understanding what AI safety researchers believe and why—but you should seek out alternative perspectives as well.
Recommended alternative viewpoints:
- Gary Marcus’s Substack↗✏️ blogGary Marcus's SubstackGary Marcus's Substack offers expert analysis and commentary on artificial intelligence, focusing on responsible AI development and potential risks.Source ↗Notes — AI skepticism
- Timnit Gebru et al.’s work↗🔗 webTimnit Gebru et al.'s workThe Distributed AI Research Institute (DAIR) examines AI systems' societal impacts, emphasizing harm reduction and equitable technological futures. Their work centers on exposin...Source ↗Notes — AI ethics perspective
- Yann LeCun’s posts↗🔗 webYann LeCun's postsI apologize, but the provided content appears to be an error page from X (formerly Twitter) and does not contain any substantive text from Yann LeCun's posts. Without the actual...Source ↗Notes — Skepticism of AGI/x-risk framing
- Emily Bender’s work↗🔗 webEmily Bender's workEmily Bender is a University of Washington linguistics professor who researches computational linguistics, grammar engineering, and the ethical implications of language technolo...Source ↗Notes — Linguistic critique of LLM capabilities
Read our full transparency statement →
Contributing
Section titled “Contributing”This is an open project. Key areas where contributions would be valuable:
- Adding researcher profiles
- Updating organization pages
- Improving argument maps
- Adding sources and citations
Explore by Topic
Browse the sidebar to explore specific topics, risks, and organizations.
Start Learning