Meta Cuts AMD MI400 Chip Memory to 144GB for Recommendation Systems
Meta is developing a custom version of AMD’s MI400-series graphics processor that halves the standard package size and reduces high-bandwidth memory to approximately 144 gigabytes from 432 gigabytes. The design replaces the usual 12 HBM4 stacks with six, prioritizing memory bandwidth cost per server rack.
SemiAnalysis reported that the configuration is tuned for recommendation system workloads and CPU-to-GPU compute ratios. The firm noted the reduced memory and compute make the chip less optimal for large language model training and inference, a design choice finalized before TBD Lab’s formation. The half-size packaging may also make the hardware harder to rent to external clients, according to the analyst’s thread.
From the sources (6 posts)
@semianalysis_ALERT🚨🚨: META's CUSTOM AMD MI400-series chip will be half the size of a normal MI455X chip. It is "optimized" for recsys workloads and $/Memory Bandwidth. It will use ~144GB of HBM instead of 432GB. We break it down below👇️ 1/7🧵 https://t.c
@semianalysis_Compared to a normal MI455X package, it will use six HBM4 8i stacks instead of 12 HBM4 12Hi stacks. The reasoning is that Meta’s recsys infrastructure strategy wanted to have a CPU compute-to-GPU compute ratio tuned for recsys and to optimi
@semianalysis_But the issue is that Meta’s custom MI400-series SKU is not as optimized for LLM inference and training. The decision was made before TBD Lab was formed or could have its say. Given the significant decreases in compute and HBM in its custom
@semianalysis_Another example of Meta's overengineering obsession is GB200 NVL72 Ariel, which caused massive infrastructure issues with its cross-rack NVLink ACC cables due to signal integrity problems, as Meta was obsessed with having the Grace CPU at a
@semianalysis_Another issue that Meta's custom half-size package design will cause is that it will be harder for Zuck to rent them out, following his strategy of copying @elonmusk's neocloud strategy. 6/7🧵
@semianalysis_@elonmusk We think that the MI455X will be a great chip as long as @AnushElangovan invests enough in software and automated testing capabilities to fix AMD's long history of poor software quality, but Meta's overengineering leads to less fl