DATDAT-003 — Data Bias and Fairness Oversights
ML Systems in Education Discriminate Against Minority Demographics
4/5Sector: EducationGeography: GlobalStage: OperateIngested: —
Executive Summary
Machine learning tools used in education exhibit allocational and representational harms, performing worse for minority groups and encoding demographic stereotypes. Institutions deploying such systems face regulatory liability and reputational damage if discriminatory outcomes go ungoverned.
Domain
Data Management
Blindspots in data quality, privacy, bias, lineage, lifecycle, and third-party data dependencies.
Source
MIT AI Risk Repository — The Risks of Machine Learning Systems (Tan2022) ↗https://airisk.mit.edu/
Could this happen in your organisation?
A Velinor AI Audit maps your active AI portfolio against the 50+ blindspots and benchmarks against documented sector failures like this one. A board-ready foresight document in 5 weeks.