AMD Instinct MI300C: A custom Epyc processor with 128 GB HBM3 for Microsoft 40 comments
Image: AMD
At SC24/Ignite, Microsoft Azure launched the first Zen 4 processor with HBM3 under the name Epyc. Behind this is the product most recently known as the Instinct MI300C: AMD had already mentioned this variant as a future option in 2023. For now, the custom 4th generation Epyc processor remains quite exclusive.
On Sunday, when the head of AMD’s data center division was present at the presentation of the fastest new supercomputer, El Capitan, AMD briefly mentioned the MI300C as a possible option. He probably didn’t want to and wasn’t allowed to reveal that it was official until two days later and simply said “you might hear something about it later” – so an announcement in the future close was already mentioned there.
What is Instinct MI300C?
MI300 (MI300X & MI300A (Details)) is an accelerator for the high-performance computing environment and in particular for the currently booming artificial intelligence (AI) sector. This does not only involve adaptations to already known architectures (CDNA); Everything is also packaged in a so-called 3.5D package. To do this, AMD uses the well-known 2.5D chip design, available for years in the form of graphics chips with HBM right next to it, but not only does it literally go even further with the additional 3D stacking, so that The finished product rests on a 3.5D stack.
Architecture of the AMD Instinct MI300 family (Image: AMD)
The base plate, if you want to put it that way, is the I/O die. Four of them are side by side. Either the actual GPU/XCD dies or the CPU chips are installed on them. In a MI300A, there are six XCDs and three CCDs.
AMD Instinct MI300 Family Architecture (Image: AMD)
Replacing GPU with CPU Chipsets
For a MI300C, the XCDs (GPU units) are now swapped for additional CCD blocks, since in the MI300A there are only three with eight cores each. 24 Zen 4 cores become a total of 96 cores. The rest remains pretty much the same. This also includes the 128 GB HBM3 memory included in the package. The MI300C is essentially an Epyc processor with HBM3.
In practice with actually 88 cores and without SMT
In practice, however, the theory easily becomes different parameters depending on the intended use. In the Azure HBv5 virtual machine, Microsoft uses four of these solutions in a quartet. However, only 88 cores are available per CPU; eight cores each take care of the operating system and management to keep everything running smoothly. There are four of these processors on a single board, clocked at a maximum of 4 GHz, or 352 cores in total.
HBv5 virtual machine (Image: Azure)
The fact that SMT has also been disabled depends on the intended use in the VM area, but also on the target for the maximum possible memory bandwidth. Almost 7 TB per second is rare on the market. These are implemented using a maximum of 400 to 450 GB of HBM3, Azure writes, and here too they remain below the theoretical maximum of 512 GB.
The basic technical data includes:
6.9 TB/s memory bandwidth (STREAM Triad) on 400 to 450 GB RAM (HBM3) Up to 9 GB memory per core (customer configurable) Up to 352 AMD EPYC “Zen4” processor cores “, 4 GHz peak frequencies (customer configurable) Infinity Fabric 2X total bandwidth across processors versus any platform of AMD EPYC™ servers to date. SMT disabled, single-tenant design only (1 VM per server)800 Gbps NVIDIA Quantum-2 InfiniBand, balanced to 200 Gbps per SoCAAzure VMSS Flex processor to scale MPI workloads to hundreds of thousands of CPU cores powered by HBM160 Gbps of Azure accelerated networking via a 2nd Gen Azure Boost NIC14 TB local NVMe SSD delivering up to 50 GB/s read and 30 GB/s write bandwidth
It was announced at the SC24 show in Atlanta that these AMD processors would not be available as a general solution; for now, they will remain exclusive to Azure.
Topics: AMD AMD Genoa Epyc HBM Instinct Processors SC24 Server Supercomputer Economy Zen Source: Microsoft
Marc deciphers processors by testing their performance for gaming, content creation, and artificial intelligence.
