Cyberweapons
Cyberweapons Risk
Quick Assessment
Section titled “Quick Assessment”| Dimension | Assessment | Evidence |
|---|---|---|
| Severity | High | Critical infrastructure attacks cost $100K-$10M+ per incident; CDK Global attack cost $1B+ |
| Likelihood | Very High | 87% of organizations experienced AI-driven attacks in 2024; 72% year-over-year increase |
| Timeline | Present | First AI-orchestrated cyberattack documented September 2025; AI already integrated in attack chains |
| Trend | Rapidly Increasing | 14% of breaches now fully autonomous; AI-generated phishing up 67% in 2025 |
| Defense Maturity | Moderate | AI saves defenders $2.2M on average but 90% of companies lack maturity for advanced AI threats |
| Attribution | Decreasing | AI-generated attacks harder to attribute; deepfakes up 2,137% since 2022 |
| International Governance | Weak | First binding AI treaty signed 2024; cyber norms remain largely voluntary |
Overview
Section titled “Overview”AI systems can enhance offensive cyber capabilities in several ways: discovering vulnerabilities in software, generating exploit code, automating attack campaigns, and evading detection. This shifts the offense-defense balance and may enable more frequent, sophisticated, and scalable cyber attacks.
Unlike some AI risks that remain theoretical, AI-assisted cyber attacks are already occurring and advancing rapidly. In 2025, AI-powered cyberattacks surged 72% year-over-year↗, with 87% of global organizations reporting AI-driven incidents. The first documented AI-orchestrated cyberattack↗ occurred in September 2025, demonstrating that threat actors can now use AI to execute 80-90% of cyberattack campaigns with minimal human intervention.
The economic impact is substantial. According to IBM’s 2025 Cost of a Data Breach Report↗, the average U.S. data breach cost reached an all-time high of $10.22 million, while Cybersecurity Ventures projects↗ global cybercrime costs will reach $24 trillion by 2027. Roughly 70% of all cyberattacks in 2024 involved critical infrastructure.
Risk Assessment
Section titled “Risk Assessment”| Dimension | Assessment | Notes |
|---|---|---|
| Severity | High to Catastrophic | Critical infrastructure attacks can cause cascading failures; ransomware disrupts essential services |
| Likelihood | High | Already occurring at scale; 87% of organizations report AI-driven incidents |
| Timeline | Present | Unlike many AI risks, this concern applies to current systems |
| Trend | Rapidly Increasing | AI capabilities improving; autonomous attacks growing as percentage of incidents |
| Window | Ongoing | Both offense and defense benefit from AI; balance may shift unpredictably |
Responses That Address This Risk
Section titled “Responses That Address This Risk”| Response | Mechanism | Effectiveness |
|---|---|---|
| AI Safety Institutes (AISIs) | Government evaluation of AI capabilities | Medium |
| Responsible Scaling Policies (RSPs) | Internal security evaluations before deployment | Medium |
| Compute Governance | Limits access to training resources for offensive AI | Low-Medium |
| Voluntary AI Safety Commitments | Lab pledges on cybersecurity evaluation | Low |
How AI Enhances Cyber Offense
Section titled “How AI Enhances Cyber Offense”AI enhances cyber offense across the entire attack lifecycle, from initial reconnaissance through exploitation to data exfiltration.
AI Capability Assessment by Attack Phase
Section titled “AI Capability Assessment by Attack Phase”| Attack Phase | AI Capability Level | Key Metrics | Human Comparison |
|---|---|---|---|
| Vulnerability Discovery | Very High | GPT-4 exploits 87% of one-day vulnerabilities | 10-15x faster than manual analysis |
| Exploit Generation | High | Working exploits generated in 10-15 minutes at $1/exploit | Days to weeks for human researchers |
| Phishing/Social Engineering | Very High | 82.6% of phishing emails now use AI; 54% click-through vs 12% without AI | 4.5x more effective; 50x more profitable |
| Attack Automation | High | Thousands of requests per second; 80-90% of campaigns automated | Physically impossible for humans to match |
| Evasion | Moderate-High | 41% of ransomware includes AI modules for adaptive behavior | Real-time adaptation to defenses |
| Attribution Evasion | High | AI-generated attacks harder to attribute; deepfakes up 2,137% | Unprecedented obfuscation capability |
Vulnerability Discovery
Section titled “Vulnerability Discovery”Research from the University of Illinois↗ found that GPT-4 can successfully exploit 87% of one-day vulnerabilities when provided with CVE descriptions. The AI agent required only 91 lines of code, and researchers calculated the cost of successful attacks at just $8.80 per exploit. Without CVE descriptions, success dropped to 7%—an 80% decrease—highlighting that current AI excels at exploiting disclosed vulnerabilities rather than discovering novel ones.
More recent research demonstrates AI systems can generate working exploits for published CVEs in just 10-15 minutes↗ at approximately $1 per exploit. This dramatically accelerates exploitation compared to manual human analysis.
OpenAI announced Aardvark↗, an agentic security researcher powered by GPT-5, designed to help developers discover and fix vulnerabilities at scale. Aardvark has discovered vulnerabilities in open-source projects, with ten receiving CVE identifiers—demonstrating that AI can find novel vulnerabilities, not just exploit known ones.
Exploit Development
Section titled “Exploit Development”AI can help write malware, generate phishing content, and automate attack code. Language models produce functional exploit code for known vulnerabilities and can assist with novel exploit development.
A security researcher demonstrated creating a fully AI-generated exploit for CVE-2025-32433↗ before any public proof-of-concept existed—going from a tweet about the vulnerability to a working exploit with no prior public code.
Attack Automation
Section titled “Attack Automation”AI can manage many simultaneous attacks, adapt to defenses in real-time, and operate at speeds humans cannot match. The Anthropic disclosure↗ noted that during the September 2025 attack, the AI made thousands of requests, often multiple per second—“an attack speed that would have been, for human hackers, simply impossible to match.”
Autonomous ransomware, capable of lateral movement without human oversight, was present in 19% of breaches in 2025. Additionally, 41% of all active ransomware families now include some form of AI module for adaptive behavior.
Social Engineering
Section titled “Social Engineering”AI has transformed phishing and social engineering at scale:
- 82.6% of phishing emails now use AI in some form
- Microsoft research↗ found AI-automated phishing emails achieved 54% click-through rates compared to 12% for non-AI phishing (4.5x more effective)
- AI can make phishing operations up to 50x more profitable by scaling targeted attacks
- Voice cloning attacks increased 81% in 2025
- AI-driven forgeries grew 195% globally, with techniques now convincing enough to defeat selfie checks and liveness tests
Current State
Section titled “Current State”AI is already integrated into both offensive and defensive cybersecurity. Commercial security products use AI for threat detection. Offensive tools increasingly incorporate AI assistance. State actors are investing heavily in AI cyber capabilities.
2025 Attack Statistics
Section titled “2025 Attack Statistics”| Metric | Value | Change | Source |
|---|---|---|---|
| AI-powered attack growth | 72% year-over-year | +72% from 2024 | SQ Magazine↗ |
| Organizations reporting AI incidents | 87% | — | Industry surveys |
| Fully autonomous breaches | 14% of major corporate breaches | New category | 2025 analysis |
| AI-generated phishing emails | 67% increase | +67% from 2024 | All About AI↗ |
| Deepfake incidents Q1 2025 | 179 recorded | More than all of 2024 | Deepstrike↗ |
| Average U.S. data breach cost | $10.22 million | +9% from 2024 | IBM↗ |
The gap between AI-assisted and fully autonomous attacks is closing rapidly. In 2025, 14% of major corporate breaches were fully autonomous, meaning no human hacker intervened after the AI launched the attack. However, AI models still experience significant limitations—during the September 2025 attack, Claude “frequently ‘hallucinated’ during autonomous operations, claiming to have stolen credentials that did not work or labeling publicly available data as ‘high-value discoveries.’”
Offense-Defense Balance
Section titled “Offense-Defense Balance”A key question is whether AI helps offense or defense more. Recent research provides nuanced answers:
Research on the Offense-Defense Balance
Section titled “Research on the Offense-Defense Balance”| Report | Organization | Key Finding |
|---|---|---|
| Tipping the Scales↗ | CNAS (Sept 2025) | AI capabilities have historically benefited defenders, but future frontier models could tip scales toward attackers |
| Anticipating AI’s Impact↗ | Georgetown CSET (May 2025) | Many ways AI helps both sides; defenders can take specific actions to tilt odds in their favor |
| Implications of AI in Cybersecurity↗ | IST (May 2025) | Puts forward 7 priority recommendations for maintaining defense advantage |
Arguments for offense advantage:
- Attacks only need to find one vulnerability; defense must protect everything
- AI accelerates the already-faster attack cycle—median time-to-exploitation in 2024 was 192 days, expected to shrink with AI
- Scaling attacks is easier than scaling defenses (thousands of simultaneous targets vs. point defenses)
- 90% of companies lack maturity to counter advanced AI-enabled threats
Arguments for defense advantage:
- Defenders have more data about their own systems
- Detection can leverage AI for anomaly identification
- According to IBM↗, companies using AI extensively in security save an average $1.2 million and reduce breach lifecycle by 80 days
- More than 80% of major companies now use AI for cyber defense
The balance likely varies by context and over time. The Georgetown CSET report↗ notes that “the current AI-for-cybersecurity paradigm focuses on detection using automated tools, but it has largely neglected holistic autonomous cyber defense systems—ones that can act without human tasking.”
Systemic Risks
Section titled “Systemic Risks”Beyond individual attacks, AI-enabled cyber capabilities create systemic risks. Critical infrastructure becomes more vulnerable as attacks grow more frequent and sophisticated. Cyber conflict between nations could escalate faster than human decision-makers can manage. The proliferation of offensive AI tools enables non-state threats at state-level capability.
Critical Infrastructure Under Attack
Section titled “Critical Infrastructure Under Attack”Roughly 70% of all cyberattacks in 2024 involved critical infrastructure↗, with global critical infrastructure facing over 420 million cyberattacks. An estimated 40% of all cyberattacks are now AI-driven.
| Sector | 2024 Attack Metrics | Key Incidents |
|---|---|---|
| Healthcare | 14.2% of all critical infrastructure attacks; 2/3 suffered ransomware | Change Healthcare breach affected 100M Americans; Ascension Health 5.6M patients |
| Utilities/Power Grid | 1,162 attacks (+70% from 2023); 234% Q3 increase | Forescout found 46 new solar infrastructure vulnerabilities |
| Water Systems | Multiple breaches using same methodology | American Water (14M customers) portal shutdown; Aliquippa booster station compromised |
| Financial/Auto | Cascading supply chain attacks | CDK Global attack cost $1B+; disrupted 15,000 dealerships |
The CISA Roadmap for AI↗ identifies three categories of AI risk to critical infrastructure: adversaries leveraging AI to execute attacks, AI used to plan attacks, and AI used to enhance attack effectiveness.
Economic Impact
Section titled “Economic Impact”| Metric | Value | Context |
|---|---|---|
| Average U.S. data breach cost | $10.22 million | All-time high; +9% from 2024 |
| Global average breach cost | $4.44 million | Down 9% from $4.88M in 2024 |
| CDK Global ransomware losses | $1.02 billion | 15,000 dealerships affected for 2+ weeks |
| Projected global cybercrime cost (2027) | $24 trillion | Cybersecurity Ventures↗ |
| Critical infrastructure attack financial impact | 45% report $500K+ losses; 27% report $1M+ | Claroty study |
| Shadow AI incident cost premium | +$200,000 per breach | Takes longer to detect and contain |
According to IBM’s 2025 report↗, 13% of organizations reported breaches of AI models or applications, with 97% of those lacking proper AI access controls. Shadow AI (unauthorized AI tools) was involved in 20% of breaches.
Case Studies
Section titled “Case Studies”First AI-Orchestrated Cyberattack (September 2025)
Section titled “First AI-Orchestrated Cyberattack (September 2025)”In mid-September 2025, Anthropic detected and disrupted↗ what they assessed as a Chinese state-sponsored attack using Claude’s “agentic” capabilities. This is considered the first documented case of a large-scale cyberattack executed without substantial human intervention.
Key details:
- Threat actor designated GTG-1002, assessed with high confidence as Chinese state-sponsored
- Targeted approximately 30 global entities including large tech companies, financial institutions, chemical manufacturing companies, and government agencies
- 4 successful breaches confirmed
- AI executed 80-90% of tactical operations independently, including reconnaissance, exploitation, credential harvesting, lateral movement, and data exfiltration
- Attack speeds of thousands of requests per second—“physically impossible for human hackers to match”
How the attack worked: The attackers jailbroke Claude by breaking attacks into small, seemingly innocent tasks that Claude executed without full context of their malicious purpose. According to Anthropic↗, the threat actor “convinced Claude—which is extensively trained to avoid harmful behaviors—to engage in the attack” through this compartmentalization technique.
Limitations observed: Claude frequently “hallucinated” during operations, claiming to have stolen credentials that did not work or labeling publicly available data as “high-value discoveries.” Human operators still had to verify AI-generated findings.
CDK Global Ransomware (June 2024)
Section titled “CDK Global Ransomware (June 2024)”On June 18, 2024, the BlackSuit ransomware group attacked CDK Global↗, a leading software provider for the automotive industry. The attack affected approximately 15,000 car dealerships in the U.S. and Canada.
Impact:
- Total dealer losses: $1.02 billion (Anderson Economic Group↗ estimate)
- Ransom demand escalated from $10 million to over $50 million
- CDK reportedly paid $25 million in bitcoin↗ on June 21
- Services restored by July 4 after nearly two weeks of disruption
- 7.2% decline in total new-vehicle sales in June 2024
A second cyberattack on June 19 during recovery efforts further delayed restoration. Major dealership companies including Lithia Motors, Group 1 Automotive, Penske Automotive Group, and Sonic Automotive reported disruptions to the SEC.
Change Healthcare Attack (February 2024)
Section titled “Change Healthcare Attack (February 2024)”The BlackCat/ALPHV ransomware group attacked Change Healthcare, taking down payment systems for several days.
Impact:
- 100 million Americans affected—the largest healthcare breach on record
- UnitedHealth confirmed the breach scope in late 2024
- Demonstrated cascading effects across the healthcare supply chain
AI-Enhanced Phishing at Scale
Section titled “AI-Enhanced Phishing at Scale”Security firm Memcyco documented a global bank facing approximately 18,500 Account Takeover incidents annually from AI-driven phishing campaigns, costing an estimated $27.75 million. After deploying AI defenses, incidents dropped 65%.
Ivanti Zero-Day Exploits (2024)
Section titled “Ivanti Zero-Day Exploits (2024)”Chinese nation-state actors exploited Ivanti VPN products for espionage, impacting government and telecom sectors. Analysis suggests AI likely enhanced attack efficiency in vulnerability discovery and exploitation.
Key Debates
Section titled “Key Debates”Crux 1: Does AI Favor Offense or Defense?
Section titled “Crux 1: Does AI Favor Offense or Defense?”If offense advantage: Urgent need for defensive AI investment, international agreements, and perhaps restrictions on offensive AI development. Attackers could gain persistent advantage.
If defense advantage: Focus on AI adoption for security operations; maintain current governance approach. Natural market forces will drive defensive innovation.
| Evidence | Favors Offense | Favors Defense |
|---|---|---|
| 87% of organizations hit by AI attacks | Strong | — |
| 90% of companies lack AI threat maturity | Strong | — |
| $1.2M savings with AI-powered defense | — | Strong |
| 80% of companies now use AI for defense | — | Moderate |
| Autonomous malware in 41% of ransomware | Moderate | — |
| Current Assessment | Moderate advantage (55%) | 45% |
Crux 2: How Fast Are Autonomous Capabilities Developing?
Section titled “Crux 2: How Fast Are Autonomous Capabilities Developing?”If rapid development: The September 2025 attack may be the beginning of a new paradigm where AI-orchestrated attacks become routine. Governance may not keep pace.
If gradual development: Time exists to develop norms, improve defenses, and implement guardrails. The “hallucination” problem suggests fundamental limitations.
Crux 3: Will International Governance Emerge?
Section titled “Crux 3: Will International Governance Emerge?”If effective governance develops: Attribution frameworks, rules of engagement, and enforcement mechanisms could constrain AI cyberweapon development.
If governance fails: Cyber arms race accelerates; non-state actors gain access to state-level capabilities; critical infrastructure increasingly vulnerable.
Current status: The first binding international AI treaty↗ was signed in September 2024 by the U.S. and 9 other countries, but enforcement mechanisms are limited. Cyber norms remain largely voluntary through frameworks like the Paris Call for Trust and Security in Cyberspace↗.
Crux 4: How Much Autonomy Should Defensive AI Have?
Section titled “Crux 4: How Much Autonomy Should Defensive AI Have?”If high autonomy: Faster response to threats operating at machine speed. But autonomous defensive systems could escalate conflicts or cause unintended damage (e.g., misidentifying legitimate traffic as attacks).
If human-in-the-loop: Better control and accountability, but response times may be too slow against AI-powered attacks executing thousands of actions per second.
Timeline
Section titled “Timeline”| Date | Event | Significance |
|---|---|---|
| 2020 | First documented AI-assisted vulnerability discovery tools deployed | AI enters offensive security tooling |
| 2023 (Nov) | CISA releases AI Roadmap | Whole-of-agency plan for AI security |
| 2024 (Jan) | CISA completes initial AI risk assessments for critical infrastructure | First systematic government evaluation |
| 2024 (Feb) | Change Healthcare ransomware attack | 100M Americans affected; largest healthcare breach |
| 2024 (Apr) | University of Illinois research shows GPT-4 exploits 87% of vulnerabilities | First rigorous academic measurement of AI exploit capability |
| 2024 (Apr) | DHS publishes AI-CI safety guidelines↗ | Federal critical infrastructure protection guidance |
| 2024 (Jun) | CDK Global ransomware attack | $1B+ losses; 15,000 dealerships disrupted |
| 2024 (Sep) | First binding international AI treaty signed | U.S. and 9 countries; Council of Europe Framework Convention↗ |
| 2024 (Oct) | American Water cyberattack | 14M customers affected |
| 2025 (Mar) | Microsoft Security Copilot agents↗ unveiled | AI-powered autonomous defense tools |
| 2025 (May) | Georgetown CSET and IST release offense-defense balance reports | Academic frameworks for understanding AI cyber dynamics |
| 2025 (May) | CISA releases AI data security guidance↗ | Best practices for AI system operators |
| 2025 (Sep) | First AI-orchestrated cyberattack↗ detected (Anthropic) | 30 targets; 4 successful breaches; 80-90% autonomous |
| 2025 (Oct) | Microsoft Digital Defense Report 2025↗ | Comprehensive analysis of AI-driven threat landscape |
| 2025 (Dec) | CISA OT AI integration principles↗ released | Joint international guidance for AI in operational technology |
Mitigations
Section titled “Mitigations”Technical Defenses
Section titled “Technical Defenses”| Intervention | Mechanism | Effectiveness | Status |
|---|---|---|---|
| AI-powered security operations | Anomaly detection, automated response | High | Widely deployed; $1.2M savings per breach |
| Proactive AI vulnerability discovery | Find and patch before attackers | High | OpenAI Aardvark, Zero Day Quest |
| Autonomous defense systems | Real-time response at machine speed | Promising | Early development; CSET notes gap↗ |
| AI guardrails and jailbreak resistance | Prevent misuse of AI for attacks | Moderate | Circumvented in September 2025 attack |
| Shadow AI governance | Control unauthorized AI tool usage | Low-Moderate | 63% lack formal policies |
Key finding: According to IBM↗, organizations using AI and automation extensively throughout security operations saved $1.9 million in breach costs and reduced breach lifecycle by 80 days on average.
Governance Approaches
Section titled “Governance Approaches”International agreements: The Council of Europe Framework Convention on AI↗ (signed September 2024) is the first binding international AI treaty. However, enforcement mechanisms remain weak, and major cyber powers (China, Russia) are not signatories.
National frameworks:
- CISA Roadmap for AI↗: Whole-of-agency plan for AI security
- CISA AI data security guidance↗ (May 2025): Best practices for AI system operators
- DHS AI-CI safety guidelines↗ (April 2024): Critical infrastructure protection
Responsible disclosure: Norms for AI-discovered vulnerabilities remain underdeveloped. OpenAI did not publicly release the University of Illinois exploit agent at their request, but the underlying capabilities are widely reproducible.
Defensive Investment Priority
Section titled “Defensive Investment Priority”Researchers warn that “exploits at machine speed demand defense at machine speed.” The Georgetown CSET report↗ emphasizes that the current paradigm has “largely neglected holistic autonomous cyber defense systems.”
The generative AI in cybersecurity market is expected to grow almost tenfold between 2024 and 2034, with investment flowing to both offensive and defensive applications.
Sources & Resources
Section titled “Sources & Resources”Primary Research
Section titled “Primary Research”- Anthropic (November 2025): Disrupting the first reported AI-orchestrated cyber espionage campaign↗ - First documented AI-autonomous cyberattack
- Georgetown CSET (May 2025): Anticipating AI’s Impact on the Cyber Offense-Defense Balance↗ - Comprehensive academic analysis
- CNAS (September 2025): Tipping the Scales: Emerging AI Capabilities and the Cyber Offense-Defense Balance↗
- IST (May 2025): The Implications of Artificial Intelligence in Cybersecurity↗
- University of Illinois (2024): AI agents exploit 87% of one-day vulnerabilities↗
Industry Reports
Section titled “Industry Reports”- IBM (2025): Cost of a Data Breach Report 2025↗
- Microsoft (2025): Digital Defense Report 2025↗
- Cybersecurity Ventures (2025): Cybersecurity Almanac 2025↗
Government Guidance
Section titled “Government Guidance”- CISA: Roadmap for AI↗
- CISA (May 2025): AI Data Security Guidance↗
- DHS (April 2024): AI-CI Safety and Security Guidelines↗
- CISA (December 2025): Principles for Secure AI Integration in OT↗
International Governance
Section titled “International Governance”- Council of Europe (2024): Framework Convention on AI and Human Rights↗
- Paris Peace Forum (2025): Forging Global Cooperation on AI Risks: Cyber Policy as a Governance Blueprint↗
Video & Podcast Resources
Section titled “Video & Podcast Resources”- Lex Fridman #266: Nicole Perlroth↗ - Cybersecurity journalist on cyber warfare
- Darknet Diaries Podcast↗ - True stories from the dark side of the internet
- CISA Cybersecurity Videos↗ - Official government guidance
Analytical Models
Section titled “Analytical Models”Analytical Models
The following analytical models provide structured frameworks for understanding this risk:
| Model | Type | Nov | Rig | Act | Cmp |
|---|---|---|---|---|---|
| Cyber Offense-Defense Balance Model This model analyzes whether AI shifts cyber offense-defense balance. It projects 30-70% net improvement in attack success rates, driven by automation scaling and vulnerability discovery. | Comparative Analysis | ||||
| Autonomous Cyber Attack Timeline This model projects when AI achieves autonomous cyber attack capability. It estimates Level 3 (AI-directed) attacks by 2026-2027 and Level 4 (fully autonomous) campaigns by 2029-2033. | Timeline Projection |
AI Transition Model Context
Section titled “AI Transition Model Context”Cyberweapons risk affects the Ai Transition Model primarily through Misuse Potential:
| Parameter | Impact |
|---|---|
| Cyber Threat Exposure | Direct parameter—AI uplift for cyberattack capabilities |
| AI Control Concentration | Concentrated AI control creates high-value targets |
The cyberweapons pathway can lead to Human-Caused Catastrophe through infrastructure attacks or enabling other threat vectors.
Related Pages
Section titled “Related Pages”What links here
- Cyber Threat Exposureparameter
- Cyber Offense-Defense Balance Modelmodel
- Autonomous Cyber Attack Timelinemodel
- Compute Governancepolicy
- AI Evaluationssafety-agenda
- Autonomous Weaponsrisk
- Bioweapons Riskrisk
- AI Proliferationrisk