The HBM4 technology competition is entering a white-hot stage

Categories : tech

HBM (High Bandwidth Memory) is a new type of CPU/GPU memory chip, essentially a stacked array of many DDR chips integrated with the GPU to achieve large capacity and high bit-width DDR combinations. This memory technology has broken through the bottlenecks of memory capacity and bandwidth, and is seen as a new generation of DRAM solutions, aligning with the development trend of miniaturization and integration in semiconductor technology.

Over the past decade, the performance of HBM technology has been continuously upgraded and iterated, becoming one of the important technical foundations in the field of high-performance computing. Since the beginning of 2023, the huge demand for computing power driven by AI large models represented by ChatGPT has made HBM one of the few relatively prosperous sub-markets in the entire memory chip industry.

There are two hot topics in the current HBM market: one is "HBM3 shortage" and the other is "HBM4 technology competition".

The horn of HBM3 shortage has been blowing for a long time. Recently, SK Hynix CEO Gu Lu Zheng revealed that the company's HBM production capacity for this year has been completely sold out, and next year's orders are basically sold out. Samsung Electronics also revealed that the HBM production capacity has been sold out. Micron Technology also disclosed in the recent earnings call that its HBM products have been completely sold out, and most of the production capacity for 2025 has also been booked.

Despite the increasingly prominent tight situation of HBM production capacity, the three major original factories have not slowed down the pace of new technology development because of this, but instead have pushed the HBM4 technology competition to a higher heat.

So what are the differences between HBM4 and HBM3? What surprises can HBM4 bring us? Will the promotion of HBM4 affect the future production capacity of HBM3? Before that, let's first understand HBM4.

What are the highlights of HBM4?It is reported that since 2015, from HBM1 to HBM3e, various updates and improvements have been made, but HBM has maintained the same 1024-bit (per stack) interface in all iterations, which is a super-wide interface running at a relatively moderate clock speed. However, as the requirements for memory transfer rates continue to increase, especially when the basic physical principles of DRAM cells have not changed, this speed will not meet the data transfer requirements in future AI scenarios. Therefore, the next generation of HBM4 needs to make more substantial changes to high-bandwidth memory technology, starting with a wider 2048-bit memory interface.

The interface width has been increased from 1024 bits per stack to 2048 bits per stack, bringing a breakthrough change to HBM4. With a 2048-bit memory interface, the transfer rate can theoretically double again. For example, NVIDIA's flagship Hopper H100 GPU, equipped with six HBM3, reaches a 6144-bit width. If the memory interface is doubled to 2048 bits, NVIDIA could theoretically reduce the number of chips to three and achieve the same performance.

HBM4 also has some changes in the number of stacks. In addition to the initial 12-layer vertical stacking, it is expected that memory manufacturers will bring 16-layer vertical stacking in 2027. The advancement of stacking technology means that more memory cells can be accommodated in the same physical space, thereby significantly increasing memory capacity and bandwidth. This high-density stacking technology is a huge advantage for applications that require large capacity and high-speed access, especially in data centers and supercomputers. In contrast, the number of stacks for HBM3 is mainly 8 layers/12 layers. In addition, HBM will also develop towards a more customized direction, not only arranged next to the SoC main chip, but some will also turn to stacking on the SoC main chip.

With the advancement of technology, HBM4E is expected to bring lower power consumption. This will help reduce system energy consumption, reduce heat generation, and extend the battery life of devices. At the same time, HBM4E memory is expected to be applied in various fields, including artificial intelligence, high-performance computing, data centers, graphics processing, and other fields, bringing higher performance and efficiency to these fields.

Storage giants show their skills

SK Hynix: Using advanced packaging technologies such as MR-MUF, in cooperation with TSMC

NB-IoT and LoRa become the kings of the Internet of Things

AI PCs are hot selling, Lenovo copies Apple's strategy

Facing "patent trolls," Micron Technology faces a 480 million U.S. dollar infrin

The future of AI chips may not be GPUs

TSMC's Nanjing factory receives an unlimited exemption permit from the United St

Mitsui Chemicals will mass-produce a new photoresist film to support ASML's next

$3.34 trillion! Nvidia's market value tops the world

Dialogue with GAC Aion's Gu Huinan: discussing going overseas, technology, and i

The rise of storage power in the AI era

Temperature: An increasingly concerning issue for chip security experts

In terms of technology, SK Hynix will continue to use advanced MR-MUF technology to achieve 16-layer stacking.

In MR-MUF (Massive Reflow Underfill Mold), Massive Reflow Soldering (MR) is a technology that connects chips to each other by melting the bumps between stacked chips. Underfill Mold (MUF) is a technology that fills protective material between stacked chips to improve durability and heat dissipation effects. With MR-MUF, multiple layers of DRAM can be packaged at the same time.At the same time, SK Hynix is also committed to the development of next-generation advanced packaging technologies such as Chiplet and Hybrid bonding, to support heterogeneous integration between semiconductor memory and logic chips, and at the same time promote the development of new types of semiconductors.

According to the latest information released by SK Hynix, HBM4 will have a 40% speed increase compared to the fifth-generation HBM3E, and the power consumption is only 70% of the latter.

SK Hynix also plans to use TSMC's advanced logic process on the foundation chip of HBM4, so that additional functions can be packaged into a limited space. This also helps SK Hynix to produce customized HBM to meet customers' extensive needs for performance and efficiency. SK Hynix and TSMC have also agreed to cooperate in optimizing the integration of SK Hynix's HBM and TSMC's CoWoS (Chip on Wafer on Substrate) technology, and to cooperate in responding to common customer requirements related to HBM.

SK Hynix President and AI Infrastructure Head Justin Kim said: "We hope to establish a strong partnership with TSMC to help accelerate our open cooperation with customers and develop the industry's best-performing HBM4."

Last month, the two companies signed a memorandum of understanding.

Samsung: Synchronous development of hybrid bonding and traditional TC-NCF process

Samsung Electronics adopts a two-pronged strategy in the HBM4 memory bonding technology, developing hybrid bonding and traditional TC-NCF (thermal compression with non-conductive film) process synchronously.

Hybrid bonding technology, as a new type of memory bonding method, has significant advantages compared to traditional bonding processes. It abandons the cumbersome step of adding bumps between DRAM memory layers and directly connects the upper and lower layers through a copper-to-copper connection. This innovative approach not only improves signal transmission speed, better meeting the urgent demand for high bandwidth in AI computing, but also reduces the spacing between DRAM layers, reducing the overall height of the HBM module.

The maturity and application cost of hybrid bonding technology have always been the focus of attention in the industry. To address this issue, Samsung Electronics has adopted a diversified strategy in the HBM4 memory bonding technology. In addition to actively promoting the research and application of hybrid bonding technology, Samsung Electronics is also developing the traditional TC-NCF process synchronously to achieve technological diversification, reduce risks, and enhance overall competitiveness.

From a technical point of view, TC-NCF is a technology slightly different from MR-MUF. When stacking chips each time, a layer of non-conductive adhesive film is placed between each layer. This film is a polymer material used to insulate chips from each other and protect the connection points from impact. The advantage of this method is that it can minimize warpage that may occur as the number of layers increases and the thickness of the chips decreases, making it more suitable for building higher stacks.It is reported that SK Hynix also used NCF before the second generation of HBM, but starting from the third generation (HBM2E), it switched to MUF (especially MR-MUF). Industry insiders believe that MUF is the reason why SK Hynix can stand out in the HBM market. Because of this, many people are skeptical about Samsung's technical route.

However, Samsung Vice President Kim Dae-woo said that when stacking up to 8 layers, the production efficiency of MR-MUF is higher than that of TC-NCF, but once the stacking reaches 12 or more, the latter will have more advantages. The vice president also pointed out that when HBM4 is introduced, it is expected that custom requests will increase.

Micron: HBMnext may surprise

In terms of HBM chips, Micron Technology is also accelerating its pace to catch up with the two Korean storage giants. However, Micron has not disclosed too much information on technical details.

Regarding future layout, Micron disclosed the next-generation HBM memory, temporarily named HBMnext, which the industry speculates may be its HBM 4. Micron estimates that HBMNext will provide 36 GB and 64 GB capacities, which means various configurations, such as a 12-Hi 24 Gb stack (36 GB) or a 16-Hi 32 Gb stack (64 GB). As for performance, Micron claims a bandwidth of 1.5 TB/s-2+TB/s per stack, which means a data transfer rate of over 11.5 GT/s/pin.

Unlike Samsung and SK Hynix, Micron seems not to plan to integrate HBM and logic chips into one chip. In the development of the next generation of HBM products, Micron may want to obtain faster memory access speed through the combination chip form of HBM-GPU. However, the American media said that as the machine learning training model becomes larger and the training time is extended, the pressure to shorten the running time by accelerating memory access speed and increasing the memory capacity of each GPU will also increase. In order to obtain the locked HBM-GPU combination chip design (although with better speed and capacity) and give up the competitive supply advantage of standardized DRAM, it may not be the right way forward.

Mass production time of HBM4 from the three major original factories

Regarding the mass production time, SK Hynix said at a press conference held in May that its HBM4 memory mass production time has been advanced to 2025. Specifically, SK Hynix plans to launch the first HBM4 products with 12-layer DRAM stacking in the second half of 2025, and the 16-layer stacking HBM will be launched slightly later in 2026.According to Samsung's plan, HBM 4 will produce samples in 2025 and mass-produce in 2026. Micron is expected to launch 12 and 16 stack HBM4 in 2026, with a bandwidth of over 1.5TB/s; by 2027-2028, it will also release 12 and 16 stack HBM4E, with a bandwidth of more than 2TB/s.

If the current plans of each company are advanced, then SK Hynix will take the lead.

TSMC is also gearing up for HBM4

In addition, TSMC is actively developing and optimizing its packaging technology to support the integration of HBM4.

At the North American Technology Seminar in April this year, TSMC launched the next-generation wafer system platform - CoW-SoW - which will achieve 3D integration with wafer-level design. This technology is based on TSMC's InFO_SoW wafer-level system integration technology launched in 2020, which enables it to build wafer-level logic processors. It is understood that TSMC's CoW-SoW focuses on integrating wafer-level processors with HBM4 memory. The next-generation memory stack will use a 2048-bit interface, which makes it possible to directly integrate HBM4 on top of the logic chip. At the same time, it may also make sense to stack additional logic on the wafer-level processor to optimize costs.

Subsequently, at the European Technology Seminar in mid-May 2024, TSMC stated that it will manufacture HBM4 chips using its 12FFC+ (12nm level) and N5 (5nm level) process technologies.

The senior director of TSMC's design and technology platform said: "We are working with major HBM memory partners (Micron, Samsung, SK Hynix) to develop advanced processes for full-stack integration of HBM4. The N5 process can provide more logic functions for HBM4 with lower power consumption."

The N5 process allows more logic functions to be packaged into HBM4 and achieves very fine interconnect spacing, which is crucial for direct bonding on the logic chip, and can improve the memory performance of AI and HPC processors.Compared to N5, TSMC's 12FFC+ process (derived from the company's 16nm FinFET technology) is more cost-effective, and the base chips manufactured can build 12-layer and 16-layer HBM4 memory stacks, providing capacities of 48GB and 64GB respectively.

TSMC is also optimizing packaging technologies, especially CoWoS-L and CoWoS-R, to support HBM4 integration. These advanced packaging technologies help assemble up to 12 layers of HBM4 memory stacks. The new interposer ensures efficient routing of more than 2000 interconnections while maintaining signal integrity. According to TSMC, the experimental HBM4 memory has achieved a data transfer rate of 6 GT/s at 14mA so far.

TSMC is also working with EDA companies such as Cadence, Synopsys, and Ansys to certify HBM4 channel signal integrity, IR/EM, and thermal accuracy.

So, should we be concerned about the future development progress of HBM4 encroaching on the production capacity of HBM3? After all, the current production capacity of HBM3 is already quite tight.

Will HBM4 squeeze the production capacity of HBM3?

It is well known that the main competitors in the HBM3 market are only SK Hynix, Samsung, and Micron.

Mass production progress of each company's HBM3 series products

It is reported that Micron, SK Hynix, and Samsung successively provided Nvidia with 8-layer vertically stacked HBM3E (24GB) samples at the end of July last year, mid-August, and early October. Among them, Micron and SK Hynix's HBM3E have passed Nvidia's verification at the beginning of this year and have received orders.In March of this year, SK Hynix announced that the company has successfully mass-produced the ultra-high-performance memory product HBM3E for AI ahead of schedule, and has started supplying to customers from the end of March. Micron also announced that it has started mass production of its HBM3E solution, and its first 24GB 8H HBM3E product will be used for NVIDIA H200 GPU, which will start shipping in the second quarter of 2024.

However, recently, according to a report by DigiTimes, Samsung's HBM3E has not yet passed NVIDIA's test and still requires further verification. It is understood that the main reason Samsung has not passed NVIDIA's verification is the approval process of TSMC. As the manufacturer and packaging factory of NVIDIA's data center GPU, TSMC is also an important participant in NVIDIA's verification process. It is rumored that the detection standard is set based on SK Hynix's HBM3E product, and Samsung's HBM3E product has some differences in the manufacturing process, such as SK Hynix using MR-RUF technology, while Samsung uses TC-NCF technology, which will have some impact on some parameters.

However, on May 24, Samsung denied reports that its high-bandwidth memory chips had not passed NVIDIA's test for use in the American chip giant's artificial intelligence processors.

Samsung said that its HBM supply tests with partners around the world are progressing "smoothly."

In the first quarter earnings report for 2024 released last month, Samsung said that the 8-layer vertically stacked HBM3E has been mass-produced in April, and plans to mass-produce the 12-layer vertically stacked HBM3E in the second quarter, ahead of the second half of the year as originally planned. According to Samsung, this is to better meet the growing demand for generative AI, so the project schedule for the new HBM product was accelerated.

The tight supply of HBM is mainly due to the strong demand for HPC and GPU from chip giants such as NVIDIA and AMD. Against this backdrop, major storage leaders have invested heavily to promote the expansion of HBM production plans.

Expansion actions are frequent

In April this year, SK Hynix announced plans to expand the production capacity of next-generation DRAM, including HBM, to meet the rapidly growing demand for AI. The company will invest about 53 trillion won (about 279.84 billion yuan) to build the M15X wafer factory as a new DRAM production base. The company plans to start construction at the end of April, with the goal of completing and mass-producing as soon as possible by November 2025. With the gradual increase of equipment investment plans, the total investment in the construction of the new production base will exceed 200 trillion won in the long run.

It is worth noting that in a previous telephone meeting, SK Hynix said it would expand the investment in HBE production facilities, and the investment in facilities related to through-silicon via (TSV) will more than double compared to 2023, aiming to double the production capacity.

Recently, SK Hynix's production head Kwon Jae-soon said that the company's HBM3E memory yield has approached 80%. In addition, Kwon Jae-soon also mentioned that SK Hynix has now reduced the production cycle of HBM3E by 50%. Shorter production time means higher production efficiency, which can provide more sufficient supply for downstream customers such as NVIDIA. The executive once again confirmed that the main focus of SK Hynix this year is to produce 8-layer stacked HBM3E, as this specification is currently the core of customer demand.Samsung's Executive Vice President and head of the DRAM Product and Technology Division, Hwang Sang-joong, revealed Samsung's capacity expansion plan at the "Memcon 2024" conference held in San Jose, California. Hwang stated that Samsung expects this year's HBM chip production to increase by 2.9 times compared to last year, a figure that even exceeds the 2.4 times growth that Samsung had earlier predicted at CES 2024. In addition, Samsung also disclosed its HBM technology roadmap, forecasting that by 2026, its HBM shipments will be 13.8 times higher than in 2023, and by 2028, this number will further climb to 23.1 times that of 2023. To meet the strong demand in the HBM field, Micron also slightly increased its capital expenditure budget for this fiscal year, adjusting from the original plan of $7.5 to $8 billion to $8 billion. Micron estimates that in the next few years, the annual growth rate of its HBM memory bit capacity will reach 50%.

When will the capacity be released? Will HBM4 conflict with HBM3?

Observations indicate that the introduction of the next-generation HBM4 by the three major memory giants is expected to be at the earliest in the second half of 2025, and the full-scale mass production may need to wait until 2026. Looking more closely at the preparation of each company's new capacity, SK Hynix's new factory is expected to be completed in November 2025, and if production is immediately started after completion, the start of the new capacity will also follow closely, expected by the end of 2025.

Samsung predicts that by 2026, its HBM shipments will surge by 13.8 times compared to 2023. Although Micron Technology appears to be more cautious in action, judging from the capacity expansion trend of the first two companies, the capacity tension problem of HBM may have been alleviated by then, and the mass production timing of HBM4 and the start of new capacity are expected to echo each other.

However, considering the adaptation cycle required for the launch of new products and the process of improving yield, as well as the migration of some orders from HBM3 to HBM4, the promotion of HBM4 may not have a significant impact on the market of HBM3 in the short term.