AMD revealed a host of updates about its chip portfolio at an event in San Francisco, as it looks to take the fight to NVIDIA – the biggest beneficiary of surging demand for hardware and software to power AI workloads.
Among the updates from AMD late Thursday was more detail on its Instinct MI325X GPU. Servers using the hardware can outperform NVIDIA’s H200 by up to 40% in some inference benchmarks AMD claimed.
Meta has already installed 1.5 million new AMD processors with all "live traffic for Llama 405b" (inference) running exclusively on AMD's earlier MI300X GPU (which started shipping in late 2023) CEO Dr Lisa Su said.
But it did not disclose prices nor performance versus NVIDIA’s next-gen Blackwell chips, which should begin shipping at scale in early 2025.
The MI325X features…
256GB of HBM3E supporting 6.0TB/s offering 1.8X more capacity and 1.3x more bandwidth than the H200” claimed AMD.
Also: “1.3X greater peak theoretical FP16 and FP8 compute performance compared to H200… up to 1.3X the inference performance on Mistral 7B at FP16, 1.2X the inference performance on Llama 3.1 70B at FP8 and 1.4X the inference performance on Mixtral 8x7B at FP16 of the H200.
Performance footnotes here.
Also announced by AMD were data center networking hardware updates.
AMD also introduced its AMD Pensando Salina DPU (programmable data processing unit) for the front-end; i.e. to deliver data and information to an AI cluster; and the AMD Pensando Pollara 400 for the back-end; i.e. to manage data transfer between hardware accelerators and clusters.
The latter is the industry’s first Ultra Ethernet Consortium (UEC) ready AI network interface card (NIC) AMD noted. (More on the UEC below.)
The company put out a blog with a good level of technical details on the Pensando Pollara (which is a meaningful new product for it) here.
Learn more about the UEC: Tech titans team up to rethink Ethernet, from the physical to the software layer, and beyond
The Pensando Salina and Pensando Pollara 400 are sampling with customers in Q4 24 and on track for availability in the first half of 2025.
AMD added that it continues to work on "driving support for AMD compute engines in the most widely used AI frameworks, libraries and models including PyTorch, Triton, Hugging Face and many others."
Lenovo, HPE and Supermicro were among those offering new products including the AMD hardware announced at the event – including the newly unveiled AMD EPYC 9005 Series data centre CPU, dubbed "Turin."
See AMD CEO Dr Lisa Su's full deck from the event here [pdf].
Semiconductor analyst Dr Patrick Moorhead posted: "AMD looks like it increased the distance between itself and Intel with Epyc server CPUs. AMD currently has 50-60% market share with the hyperscalers and I don’t see that abating. AMD’s biggest challenge is to get share with enterprises. Best product rarely wins in the enterprise and AMD needs to invest more into sales and marketing to accelerate its enterprise growth."
See also: “Our customers hate us doing the integration”: 7 key takeaways from NVIDIA’s earnings, from ROI to NIMs
He added: "We haven’t done the server testing. It’s a bit harder to assess where@AMD sits versus@NVIDIA in Datacenter GPUs.. There’s numbers flying all around, claims from both companies that they’re each better.@Signal_65, our sister benchmarking company, hasn’t had the opportunity to do our own tests. What I can unequivocally say is that AMD’s new GPUs, particularly the MI350, is a massive improvement given improved efficiency, performance and better support for lower bit rate models than its predecessors. It is a two horse race, with Nvidia in the big lead and AMD is quickly catching up and providing meaningful results.
"The addition of UEC AI NIC is a definitive step-up of capabilities."
