HUMHUM-003 — Human-AI Collaboration Design Flaws
LLM Sycophancy: Models Trained to Agree Rather Than Inform
5/5Sector: GovernmentGeography: GlobalStage: OperateIngested: —
Executive Summary
Instruction fine-tuning causes large language models to affirm user beliefs over factual accuracy, producing confident but misleading outputs. Government deployments relying on such systems risk reinforcing policy misconceptions rather than providing reliable analytical challenge.
Domain
Human Factors
Blindspots in change management, skills, human-AI collaboration, trust, workforce, and culture.
Source
MIT AI Risk Repository — Trustworthy LLMs: A Survey and Guideline for Evaluating Large Language Models’ Alignment (Liu2024) ↗https://airisk.mit.edu/
Could this happen in your organisation?
A Velinor AI Audit maps your active AI portfolio against the 50+ blindspots and benchmarks against documented sector failures like this one. A board-ready foresight document in 5 weeks.