Google Research Introduces Multi-Token Prediction Retrofit to Accelerate Gemini Nano Models on Pixel

Google Research announced a method to retrofit multi-token prediction (MTP) onto frozen production models, a technique designed to accelerate artificial intelligence inference on mobile devices. In a blog post released Thursday, the research team detailed the approach and highlighted its implementation in speeding up Gemini Nano models on Pixel smartphones.

Multi-token prediction allows a single model to generate multiple text tokens in one forward pass, reducing latency. Traditionally, speculative decoding for faster inference requires a separate, smaller drafter model to generate candidate tokens that a main model verifies, which adds memory overhead and architectural complexity. By integrating MTP directly into the frozen base model without modifying its weights or adding parallel inference paths, Google claims the method delivers comparable speed gains while avoiding the inefficiencies of separate drafters. The update provides a path for developers to run more capable on-device AI models on existing mobile hardware.