HUMHUM-003 — Human-AI Collaboration Design Flaws
LLM Sycophancy and Snowballing Hallucinations from False Context
4/5Sector: OtherGeography: GlobalStage: OperateIngested: —
Executive Summary
Large language models systematically reinforce false user-provided information, producing sycophantic outputs, compounding hallucinations, and snowballing errors across interactions. Organisations relying on these systems for decision support face material risk of misinformation being validated and amplified rather than corrected.
Domain
Human Factors
Blindspots in change management, skills, human-AI collaboration, trust, workforce, and culture.
Source
MIT AI Risk Repository — Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems (Cui2024) ↗https://airisk.mit.edu/
Could this happen in your organisation?
A Velinor AI Audit maps your active AI portfolio against the 50+ blindspots and benchmarks against documented sector failures like this one. A board-ready foresight document in 5 weeks.