HUMHUM-004 — Trust and Acceptance Issues
Anthropomorphising AI Agents Drives Overreliance and Loss of Human Oversight
3/5Sector: OtherGeography: GlobalStage: OperateIngested: —
Executive Summary
Users who perceive conversational AI as human-like overestimate its competence, yielding control without critical scrutiny in high-stakes domains such as mental health. This erosion of effective oversight converts model errors into preventable harms, exposing organisations to liability and reputational risk.
Domain
Human Factors
Blindspots in change management, skills, human-AI collaboration, trust, workforce, and culture.
Source
MIT AI Risk Repository — Ethical and social risks of harm from language models (Weidinger2021) ↗https://airisk.mit.edu/
Could this happen in your organisation?
A Velinor AI Audit maps your active AI portfolio against the 50+ blindspots and benchmarks against documented sector failures like this one. A board-ready foresight document in 5 weeks.