GPT-5.4 Nano System Matches Gemini 3 Pro at 76.4% on SWE-bench Verified, Paper Finds
A new paper reported that a GPT-5.4 nano system using a critic-comparator orchestration loop scored 76.4% on SWE-bench Verified, matching the standalone results cited for Gemini 3 Pro and Claude Opus 4.5 Thinking.
The method selects from eight patch proposals generated by a weak model using execution and proof signals, rather than asking the model to choose the best answer itself. The paper argues that many correct fixes are already present in a weak model's top candidates, making the selector and verification process the key determinant of performance on the benchmark.
From the sources (2 posts)
@dair_aiNEW paper worth reading. GPT-5.4 nano plus a critic-comparator orchestration loop hits 76.4% on SWE-bench Verified, matching standalone Gemini 3 Pro and Claude Opus 4.5 Thinking. The trick is to select from k=8 weak-model proposals using
@omarsar0RT @dair_ai: NEW paper worth reading. GPT-5.4 nano plus a critic-comparator orchestration loop hits 76.4% on SWE-bench Verified, matching…