Cambridge and NVIDIA Paper Teaches AI Agents and Evaluators to Improve Together

A research paper from the University of Cambridge, NVIDIA, and other institutions proposes a framework that allows AI agents and their evaluators to improve simultaneously, addressing a persistent bottleneck in autonomous machine learning. The study, titled “The Red Queen Gödel Machine: Co-Evolving Agents and Their Evaluators,” introduces a method where language models and the judges grading their performance evolve together. Currently, most self-improving AI systems train against fixed benchmarks or static evaluators, which causes them to quickly overfit, game the scoring system, or stagnate once the benchmark ceases to present meaningful challenges.

The framework structures progress through distinct training epochs. During each phase, an AI agent learns under a frozen evaluator, while potential upgrades for the judge are trained separately and swapped in only after scoring higher on held-out human or objective ground truth. According to the authors’ testing, this approach enabled coding agents to outperform previous self-improving models while using 1.35 to 1.72 times fewer tokens. The co-evolution mechanism also improved paper-writing AI, which saw a 1.86-fold increase in acceptance rates from reviewer panels, and helped correct AI reviewers that previously over-accepted AI-generated content at nearly double the rate of human graders.