High bandwidth flash: Sandisk plans giant storage for AI accelerator 16 comments
Image: Sanisk
Sandisk will soon be digitally independent after splitting from Western and will have big plans for the future. One of them is High Bandwidth Flash (HBF), which should provide GPUs with much more memory than would be possible with previous High Bandwidth Memory (HBM).
HBF should complement and replace HBM
At first the idea may seem absurd, because as is well known, NAND-Flash is considerably slower than DRAM, as is stacked in the HBM. Nevertheless, Sander with the high bandwidth flash envisages the (additional) use of NAND flash, which is cut at a high bandwidth, which is supposed to use the GPUs when using AI models, the so-called AI inference, with plenty of storage space.
At least 8 times more memory than with HBM
With each new generation of high-bandwidth memory, there are higher storage capacities that reach 192 GB, for example, by AMD and NVIDIA’s AI. But with High Bandwidth Flash (HBF), Sandisk wants to enable 8 to 16 times uniform storage capacity at comparable costs.
Two scenarios are illustrated graphically and a conventional solution is compared with 8 HBM packages and a total of 192 GB of storage volume. After the GPU can access 6 HBF packages and 2 HBM building blocks, the high bandwidth flash complements the fast HBM, which continues to a small extent. The entire storage volume increases to 3,120 GB or approximately 3 TB. In the second scenario, the HBM is even completely replaced by HBF, so the GPU can even access the 4,096 GB (4 TB) memory. Then even a 1.8 trillion llm heavy setting with 3.6TB memory requirements.
Sandisk High Bandwidth Flash (HBF) (Image: Sandisk) Image 1 of 2
In the examples, a single HBM package has a storage capacity of 24 GB, while an HBF package reaches a total of 512 GB—thanks to the large area of Nand-Flash.
Flash Schema IM Flash High Bandwidth
A diagram shows that the stacked HBF dies on logic. This HBF package is just as much as the HBM on an interposer next to the GPU or CPU or TPU, depending on where the memory is needed. The HBF is intended to use the same electrical interface with “smaller logs”. HBF will not be compatible with HBM.
Sandisk High Bandwidth Flash (HBF) (Bild: Sandisk)
Same flow rate as with HBM?
The big question remains, how fast can the HBF run? According to Sandisk, it should be equal to the HBM throughput (bandwidth). Normal NAND flash is hopelessly inferior during access hours, but continues to approach throughout throughput.
Another chart indicates that Sandisk relies on a special NAND architecture at HBF, which is divided into multiple domains with more data lines and faster access. The parallels with the Flash XL come to mind. Ultimately, such a successful landing of the storage class memory of the kind could make it the lucrative AI accelerator market, provided the plan opens up.
Sandisk High Bandwidth Flash (HBF) (Bild: Sandisk)
No standard yet, but already a roadmap
Until then, the idea must first be implemented. According to its own statements, Sandisk developed the HBF architecture last year with “Contributions from Big IA Playern”. Now it is important to shape a technical advisory board of partners and industry sizes and get an open standard.
Sandisk High Bandwidth Flash (HBF) (Bild: Sandisk)
How much HBF has in common with the concept (PDF file) of high bandwidth NAND (HBN), formulated years ago by Japanese researchers, will soon be shown.
While this is all still future music, Sandisk is already planning for subsequent generations with more storage and performance in the form of a roadmap.
SANDISK High Bandwidth Flash (HBF) (Image: Sandisk) Topics: RAM Flash Memory
Alice guides you through the best storage solutions, from ultra-fast SSDs to secure cloud options.
