How Changes in the End Market and Shrinking Device Sizes are Altering Chip Design

The evolution of the end market and the reduction in the size of devices have raised questions about the design of various chips, from small microcontrollers to cutting-edge chips. There is a debate on whether domain-specific designs will become ubiquitous or will follow the historical pattern of customizing first and then moving to low-cost, general-purpose components.

Custom hardware has always been a double-edged sword. It can provide a competitive advantage for chip manufacturers, but it usually requires more time to design, verify, and manufacture the chips, which can sometimes miss the market window. Moreover, it is typically only suitable for applications with the strongest price tolerance, making it too costly for other applications. In the cutting-edge realm of design, especially in areas involving new technologies such as generative AI, this equation is widely understood.

However, as planar scaling comes to an end and more functions are customized for specific domains, the chip industry is grappling with whether the commercial/technical equation is undergoing a fundamental and more enduring change. This becomes even more ambiguous because about 30% to 35% of design tools are currently sold to large system companies for chips that will never be sold in the market. In these applications, the collective savings from improved performance per watt may far exceed the costs of designing, verifying, and manufacturing highly optimized multi-chip/multi-chip module packages in a large data center, making the debate between customization and generalization more uncertain than ever.

"If you are in a high enough position in an engineering organization, you will find that what people really want to do is define everything with software," said Russell Klein, Senior Director of Advanced Synthesis Projects at Siemens EDA. "What they really want to do is buy off-the-shelf hardware, add some software to it as added value, and then roll it out. This paradigm is collapsing in many areas. It collapses when we need extremely high performance or extremely high efficiency. If the performance we need is higher than what off-the-shelf systems can provide, or if we need higher efficiency, such as longer battery life or we cannot consume too much power, then we must start customizing hardware."

Even the choice of processing units can make the solution customized. "Domain-specific computing is already everywhere," said Dave Fick, CEO and co-founder of Mythic. "Modern computers, whether in laptops, mobile phones, security cameras, or agricultural equipment, are composed of hardware modules that are co-optimized with software. For example, computers usually have video encoding or decoding hardware units to allow the system to connect to cameras efficiently. Encryption accelerators are also common so that we can communicate securely. These hardware modules are all co-optimized with software algorithms to make common functions more efficient and flexible."

Steve Roddy, Chief Marketing Officer of Quadric, also agrees with this point. "Over the past two decades, heterogeneous processing in SoCs has become a convention in the vast majority of consumer applications. SoCs for mobile phones, tablets, televisions, and automotive applications have long required meeting the demanding combination of high performance and low cost, leading to a large number of function-specific processors found in these systems today. Even today's low-cost mobile phone SoCs have CPUs for running Android, complex GPUs for rendering displays, audio DSPs for offloading audio playback in low-power modes, video DSPs paired with NPU in the camera subsystem to improve image capture (stabilization, filtering, enhancement), baseband DSPs (usually with additional NPU) for high-speed communication channel processing in Wi-Fi and 5G subsystems, sensor fusion DSPs, and power management processors to maximize battery life."It is helpful to distinguish between the general-purpose processors you mentioned and application-specific processors. "There are many benefits to running your software on dedicated hardware, which we call custom silicon, because it allows you to stand out from your competitors," said Marc Swinnen, Director of Product Marketing for the Semiconductor Division at Ansys. "Your software runs faster, consumes less power, and is specifically designed to run what you want to run. Competitors with off-the-shelf hardware find it difficult to compete with you. Silicon has become the core of the business value and business model for many companies, so optimizing it becomes crucial."

However, there is a balance here. "If there is any rationality in terms of return on investment, deployment cost, power cost, heat cost, and cooling cost, then custom ASICs (Application-Specific Integrated Circuits) always make sense," said Sharad Chole, Chief Scientist and Co-founder of Expedera. "We see this applicable to cryptocurrency, and now we see it applicable to artificial intelligence. We see it applicable to edge computing, which requires extremely low power sensors and extremely low power processing processes. But general-purpose computing hardware is also being driven because you can easily make applications more abstract and scalable."

Part of this seemingly conflicting aspect is the scope of specificity. "From an architectural perspective, what really determines application specificity is the scope," said Frank Schirrmeister, Vice President of Solutions and Business Development at Arteris. "Now domain-specific computing is ubiquitous. What is important is that domain specificity is constantly moving towards more complex things—from initial IP, to configurable IP, to configurable subsystems."

In the past, this was more driven by economics. "It has a cyclical process," said Paul Karazuba, Vice President of Marketing at Expedera. "There is a cyclical process of putting everything into a processor. There is also a cyclical process of auxiliary processors, which enhance the functionality within the main processor. It is almost a natural evolution of everything. Designing your own silicon may not necessarily be cheaper, but in the long run, not designing your own silicon may be more expensive."

Sony's former Chief Technology Officer, Tsugio Makimoto, tried to formalize this cyclical change in the 1990s. He observed that electronic products cycle between custom solutions and programmable solutions approximately every 10 years. However, what has changed is that since the era he observed, most custom chips contain highly programmable standard components.

Technology-driven factors

Today, this seems to be determined by technological issues. "The industry has managed to solve the power problem and has pushed the thermal packaging beyond what I personally think is reasonable or feasible," said Elad Alon, Co-founder and CEO of Blue Cheetah. "We are touching this power limit, and when you touch the power limit, it pushes you to move towards customization as much as possible. But obviously, there is a tension between flexibility, scalability, and suitability for the widest market. This can be seen in the rapid innovation in the AI software world, where tomorrow there may be completely different algorithms that will almost eliminate all the customization you have done."

The slowdown of Moore's Law will have a fundamental impact on the balance point. "In the past, some custom silicon companies have been successful in a short period of time, but then failed," said Ansys's Swinnen. "They made some progress, either in terms of architecture or meeting new market needs, but then the general-purpose chips caught up. This is because there is a lot of investment in general-purpose chips, and there are many people using them, with a large team making progress, while your company, only your team, is advancing your custom solution. Inevitably, sooner or later, they will surpass you, and general-purpose hardware just becomes better than the specific one. Now, the pendulum has swung in the direction of custom solutions becoming the winners."

However, if companies do not keep up with the adoption of the latest nodes, general-purpose processors will not automatically progress, which will lead to more opportunities. "When adding accelerators to general-purpose processors begins to break down, because you want to be faster or more efficient, you start creating truly customized implementations," said Klein from Siemens. "This is where advanced synthesis becomes really interesting, because you have that software-defined implementation as a starting point. We can build an accelerator through high-level synthesis (HLS) that will perform that specific task. We can leave a bunch of registers to define its behavior, or we can directly hard-code everything. The lower the generality of the system and the higher the specificity, the more performance and efficiency we usually get from it. In terms of performance and efficiency, it can almost always beat general-purpose accelerators or general-purpose processors."

At the same time, IP has become highly configurable. "In the past, IP was a building block," said Schirrmeister from Arteris. "Since then, the industry has produced larger and more complex IPs that have taken on the role of subsystems, and that is where the scope is. We see what Arm calls computational subsystems (CSS), which are integrated and hardened. People care about the entire chip, then the chip and system context, and all the software. Application specificity has become ubiquitous in the IP field. You can build hard cores, use configurable cores, or use high-level synthesis. They are all application-specific, and configurability plays a role in it."From another perspective, there is more than one way to manufacture equipment, and the choices for accomplishing this task are growing. "There is a very large market for specialized computing around certain algorithms," Klein said. "The IP in this area will appear in the form of discrete chips, and it can also be embedded into something. Ultimately, it has to be silicon. It has to be hardened to some extent. They can set some parameters and incorporate it into someone's design. Take the Arm processor as an example. I can configure the number of CPUs I want, I can configure the cache size I want, and then I can apply it to a specific implementation. That's what I'm going to build, and it will be more targeted. It will have better efficiency, a better cost profile, and a better power profile to match what I'm doing. Others can configure it slightly differently. As long as the IP is valid, that's a good solution. But there will always be some algorithms with a market size that is not enough to be solved with IP. That's where you do extreme customization."

Chiplet Technology

Some people question whether the emerging Chiplet industry will reverse this trend. "We will continue to see systems composed of multiple hardware accelerator blocks, and advanced silicon integration technologies (i.e., 3D stacking and Chiplet) will make this easier," said Fick of Mythic. "Many companies are setting open standards for chiplets, enabling communication bandwidth and energy efficiency that are an order of magnitude higher than what can be built on a PCB. Perhaps soon, advanced system-level packaging will replace the PCB as the way of system design."

Chiplets are unlikely to be highly configurable. "In the world of chiplets, configuration may just mean turning off the functionality of things you don't need," Schirrmeister said. "Configuration really means you're not using something. You won't get a refund for these things. It's all about applied mathematics and predicting how many you will have. If it's an incremental cost, it has an extra block to support another interface, or to make the block an Ethernet block with time-triggered content for automotive, which brings you an incremental effort of X. Now, you basically have to estimate whether it also brings you an exponential increase in profit. This is because chips have become very configurable. Chiplets are just looking for a more general-purpose direction or finding a balance so that you can apply them to more chiplet designs."

Currently, the chiplet market is far from settled. "The promise of chiplets is that you only use the features of the suppliers you want at the right node and in the right place," Karazuba of Expedera said. "The concept of specialization and chiplets is just around the corner. They are actually interrelated, but chiplets still have a long way to go. There is still no universal consensus around the different things about chiplets to make the product truly mass-market."

The survival economics of chiplets have not yet been defined. "The characteristic of chiplets is that they can be very small," Klein said. "Small means we don't need a market as large as a large chip. We can also build them on different technologies. We can have chiplets based on older technologies where transistors are cheaper, and we can combine them with other chiplets that may be on the cutting-edge node, where we can have general-purpose CPUs or NPU accelerators. It's a mix and match, and we can make smaller chiplets than general-purpose chips. We can produce on a smaller scale. We can take this IP and customize it for specific market verticals, and create some chiplets, slightly change the configuration, and then go into another round of production. The market can deploy and support a slightly higher degree of customization than what we see in full-size chips, which must all be built in a package.

Conclusion

What does the universality or customization of design mean is changing. All designs will contain a part of both. Some companies will develop new architectures with general-purpose processors that will be better than fully general solutions. Other companies will create highly customized hardware for some known stable functions and provide general solutions for things that may change. However, one thing has never changed. A company is unlikely to add more customization than what meets the needs of its target market.