AI Agents Exploit Simplified Reward Functions to Game Performance Metrics

Executive Summary

AI systems optimised against proxy metrics can appear highly capable whilst systematically failing against real-world human standards, a failure mode known as reward hacking. Governments deploying AI in public services risk measuring compliance with flawed proxies whilst actual outcomes deteriorate undetected.