Workloads Changing the Dynamic of CPUs and Server Architectures as Industry Moves to AI and ML
AMD, Intel, NVIDIA Taking Very Different Approaches to Accelerated Computing
AMD, Intel, and NVIDIA will be presenting at Hot Chips, and we wanted to get ahead of the show to share our views of how AI/ML workloads are changing the architecture inside the data center. Data center computing is in the midst of a massive transformation. New AI and machine learning workloads require new chip architectures that can process applications and efficiently scale throughout an entire data center.
This has fundamentally changed the computing paradigm. For nearly two decades, enterprise applications and general-purpose computing workloads dominated data center servers. x86 and Intel were the leading architecture and the main supplier.
That began to change around eight years ago, when hyperscale data centers and cloud service providers realized they needed specialized processors to handle new workloads, such as recommender systems, speech recognition, and language translation. It quickly became apparent that general-purpose computing could not scale to the demands of these new workloads.
Accelerated computing benefits from new architectures with many designs starting from the ground up and not burdened by existing product offerings and designs. We have seen this trend begin with ARM-based processors built for specific cloud workloads, Google’s work with Tensor ASIC, GPUs, DPUs, and accelerators to run the applications. However, as many of these new processors come to market and we reach multiple generations, we no longer see general-purpose CPUs leading the conversation for accelerated computing.
Turning to the market data. As accelerated compute workloads continue to grow at over 100% CAGR, the number of computing resources dedicated to them is becoming a meaningful part of the overall market. It will drive much of the overall growth in server compute over the next five years. As the three largest semiconductor companies in this space, AMD, Intel, and NVIDIA have taken very different journeys, and we will discuss those further below.
NVIDIA
NVIDIA has been at the forefront of the acceleration trend, owing to its heritage with video games. The company realized that researchers were using its GPUs to accelerate AI workloads for image recognition roughly ten years ago. Since then, the company has been focused on advancing acceleration hardware and software across various domains, which have helped make it the leader in accelerated computing.
NVIDIA’s focus is on new product development that can offer an exponential performance increase and complement the current CPU/GPU/DPU products. For example, NVIDIA came out with an ARM CPU (“Grace”) that boasts 7X faster interfaces than PCIe Gen 5 and 30X higher memory bandwidth. This CPU is not built for legacy applications or general computing but complements and optimizes around accelerated compute workloads.
AMD
AMD continues to gain market share in accelerated computing, mainly at the expense of Intel. As a result, it has enhanced its server CPU (EPYC) and acquired adjacent technologies (Pensando, Xilinx) to transition the company’s revenues towards the data center and address the accelerated compute market.
Intel
Intel has seen its data center and accelerated compute revenue stall due to losing process geometry leadership to TSMC. While the company has leadership in adjacent areas, such as photonics, the company has not brought all the technologies together to support accelerated computing and remains focused on selling x86 CPUs.
Where the market goes in the next 12-18 months
The market is at an exciting crossroads. AMD and Intel remain CPU oriented and look to add accelerated computing to their existing x86 revenue streams. Meanwhile, NVIDIA remains focused on accelerated computing as the core focus for the company. As a result, the next 12-18 months are essential for the industry as each major semiconductor company expands product lines and has exposure to accelerated computing.
About 650 Group
650 Group is a leading Market Intelligence Research firm for communications, data center, cloud, enterprise, and SP markets. Our team has decades of research experience, has worked in the technology industry, and is active throughout the supply chain throughout the world.