HUMHUM-003 — Human-AI Collaboration Design Flaws
Automated Military AI Systems Risk Unintended Escalation to Armed Conflict
4/5Sector: DefenceGeography: GlobalStage: OperateIngested: —
Executive Summary
AI systems operating without human oversight in tactical and strategic military roles can trigger unintended escalation, including nuclear exchange, through faulty threat assessment or autonomous engagement. Boards supplying defence technology must address liability exposure and governance frameworks for human-in-the-loop requirements.
Domain
Human Factors
Blindspots in change management, skills, human-AI collaboration, trust, workforce, and culture.
Source
MIT AI Risk Repository — A Survey of the Potential Long-term Impacts of AI: How AI Could Lead to Long-term Changes in Science, Cooperation, Power, Epistemics and Values (Clarke2023) ↗https://airisk.mit.edu/
Could this happen in your organisation?
A Velinor AI Audit maps your active AI portfolio against the 50+ blindspots and benchmarks against documented sector failures like this one. A board-ready foresight document in 5 weeks.