HUMHUM-003 — Human-AI Collaboration Design Flaws
LLM Fails to Reliably Identify Harmful Mental Health Behaviours
4/5Sector: GovernmentGeography: GlobalStage: OperateIngested: —
Executive Summary
Large language models demonstrate inconsistent safety performance on mental health questions, risking harmful or misleading guidance to vulnerable users. Government deployments in health and social care face legal and reputational exposure if such models are used without validated safeguards.
Domain
Human Factors
Blindspots in change management, skills, human-AI collaboration, trust, workforce, and culture.
Source
MIT AI Risk Repository — SafetyBench: Evaluating the Safety of Large Language Models with Multiple Choice Questions (Zhang2023) ↗https://airisk.mit.edu/
Could this happen in your organisation?
A Velinor AI Audit maps your active AI portfolio against the 50+ blindspots and benchmarks against documented sector failures like this one. A board-ready foresight document in 5 weeks.