HUMHUM-003 — Human-AI Collaboration Design Flaws
Large Language Models Generate False Information With Overconfident Justifications
4/5Sector: OtherGeography: GlobalStage: OperateIngested: —
Executive Summary
LLMs routinely produce fabricated facts, erroneous code, and false citations presented with unwarranted confidence, with medical misinformation posing acute harm risks. Organisations deploying these systems without mandatory human validation expose themselves to reputational, legal, and safety liability.
Domain
Human Factors
Blindspots in change management, skills, human-AI collaboration, trust, workforce, and culture.
Source
MIT AI Risk Repository — Mapping the Ethics of Generative AI: A Comprehensive Scoping Review (Hagendorff2024) ↗https://airisk.mit.edu/
Could this happen in your organisation?
A Velinor AI Audit maps your active AI portfolio against the 50+ blindspots and benchmarks against documented sector failures like this one. A board-ready foresight document in 5 weeks.