OPSOPS-001 — Monitoring and Alerting Inadequacies
Retrieval-Augmented LLMs Overridden by Small Volumes of False External Data
4/5Sector: OtherGeography: GlobalStage: OperateIngested: —
Executive Summary
LLMs can be manipulated into producing false outputs when retrieval-augmented pipelines inject even minor quantities of conflicting disinformation, overriding correct prior training. Organisations deploying RAG systems face material risk of corrupted decisions if external data sources are compromised or poorly governed.
Domain
Operational Management
Blindspots in monitoring, incident response, performance, scalability, integration, and business continuity.
Source
MIT AI Risk Repository — Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems (Gipiškis2024) ↗https://airisk.mit.edu/
Could this happen in your organisation?
A Velinor AI Audit maps your active AI portfolio against the 50+ blindspots and benchmarks against documented sector failures like this one. A board-ready foresight document in 5 weeks.