Nvidia Blackwell Ultra GPUs Hit 1,648 TFLOPs Per GPU In DeepSeek Pretraining Record

aiai-infrastructureai-compute-chips 1 posts · 1 accounts

Nvidia Blackwell Ultra GPUs delivered a world-record 1,648 TFLOPs per GPU on the pretraining of DeepSeek-V3 671B, achieving about 3x the performance of the previous generation.

The company attributed the throughput gain to hardware and software co-design, noting optimizations across Megatron-Core, TorchTitan and JAX frameworks.

From the sources (1 posts)

@nvidiaai

NVIDIA Blackwell Ultra just set a world record for pre-training DeepSeek-V3 671B, achieving 1,648 TFLOPs per GPU. This is about 3x the earlier delivered performance of the previous generation, and is a result of our extreme co-design and c

U.S. Spot Bitcoin ETFs End Seven-Day Inflow Streak With $225 Million in Outflows as Ether Funds Add $26 Million DeepSeek Enters Funding Talks Valuing AI Lab at $71 Billion Less Than a Month After $7 Billion Raise