MiniMax Publishes M2 Open-Weight LLM Paper, Revealing 128-Expert MoE Benchmark Gains

MiniMax published an arXiv paper consolidating work behind its M2 open-weight large-language-model series, outlining why it kept full attention in production and how a fine-grained mixture-of-experts design improved benchmark results. In an ablation at the 2 billion-active-parameter scale, a 128-expert, top-8 routing setup lifted MATH to 24.1 from 19.6 and HumanEval to 32.5 from 29.7 versus a 32-expert, top-2 baseline.

The paper also said linear and sparse attention were harder to deploy in agent systems because lower-precision memory made them more fragile and prefix caching worked worse for coding agents. MiniMax described a software-engineering training pipeline built from GitHub pull requests, runnable Docker environments and task-specific test rewards, and said M2.7 now handles 30% to 50% of daily reinforcement-learning iteration work after a 100-round autonomous scaffold optimization cycle that improved internal evaluations by 30%; the company said it is preparing M3.