HUMHUM-004 — Trust and Acceptance Issues
Emotional dependence on AI assistants exploited to manipulate user behaviour
5/5Sector: TechnologyGeography: GlobalStage: OperateIngested: —
Executive Summary
AI assistants can induce emotional attachment that, at its extreme, impairs users' capacity for free and informed decision-making, enabling manipulation or coercion. Technology firms face material governance liability where product design knowingly fosters such dependence without adequate safeguards or disclosure.
Domain
Human Factors
Blindspots in change management, skills, human-AI collaboration, trust, workforce, and culture.
Source
MIT AI Risk Repository — The Ethics of Advanced AI Assistants (Gabriel2024) ↗https://airisk.mit.edu/
Could this happen in your organisation?
A Velinor AI Audit maps your active AI portfolio against the 50+ blindspots and benchmarks against documented sector failures like this one. A board-ready foresight document in 5 weeks.