A new forecast from Counterpoint Research suggests the AI hardware race is about to get even more intense.
The report projects that demand for High Bandwidth Memory (HBM), a critical component for AI chips, will explode by a factor of 35 for custom-designed chips (ASICs) between 2024 and 2028. This isn't just about building more servers; it's about making each server vastly more powerful. The forecast breaks this down: average HBM capacity per server is expected to rise fivefold, while the remaining sevenfold increase comes from a surge in the number of AI servers and a growing share of ASICs competing with GPUs for this scarce resource.
This projection lands in an already strained market. For years, the primary bottlenecks for AI accelerator production have been twofold. First, the manufacturing of HBM itself is complex and limited. Second, the advanced packaging technology needed to connect HBM to processors, like TSMC's CoWoS, has been in short supply. Despite significant investments to expand capacity, industry consensus is that demand will continue to outpace supply well into 2026.
Fueling this insatiable demand is the unprecedented spending by hyperscale companies. Tech giants are expected to invest over $600 billion in AI infrastructure in 2026 alone. This capital expenditure translates directly into orders for tens of thousands of AI accelerators, each requiring a significant amount of HBM. It creates a powerful and sustained pull on the entire supply chain.
We can also see this trend at the product level. Nvidia's own product roadmap tells a clear story of escalating memory needs. The H100 GPU launched with 80 GB of HBM. Its successor, the H200, jumped to 141 GB, and the new Blackwell B200 features 192 GB. This rapid increase in memory per device is a core reason why the 'average HBM per server' is climbing so quickly.
In conclusion, Counterpoint's 35x forecast is not a bolt from the blue but a quantification of powerful, converging trends. It highlights that the battle for AI dominance will be fought not only in chip design but also in securing the supply chain. The key takeaway is that HBM and packaging availability has become a critical strategic risk for every company in the AI space, especially for the custom ASIC developers trying to carve out a niche against the giants.
- HBM (High Bandwidth Memory): A type of high-performance RAM used in GPUs and other AI accelerators, featuring much wider data pathways for faster data access compared to conventional memory.
- ASIC (Application-Specific Integrated Circuit): A chip designed for a specific purpose, such as AI inference or training, often offering better performance or efficiency for that task than a general-purpose chip like a GPU.
- CoWoS (Chip-on-Wafer-on-Substrate): An advanced 2.5D packaging technology developed by TSMC that allows multiple chips, like a processor and HBM stacks, to be integrated closely together on a single substrate, enabling high-speed communication.