OPINION: A Touch of CPU Banter in AI
Do AI agents require a unique type of CPU? That’s the narrative that Arm, Nvidia, and a group of chip creators would like you to accept. Arm has officially dubbed its initial datacenter chip the “AGI CPU.” Nvidia’s chief, Jensen Huang, referred to Vera as a “CPU for agents,” and AWS’s Graviton 5 advertising overflows with discussions about agentic AI. But don’t be fooled – these processors are still merely general-purpose chips, adorned with a trendy AI label.
The Truth About AI Agents
Certainly, AI agents and their frameworks necessitate CPUs. No one is disputing that. However, agents don’t represent a singular workload. They merely function as intermediaries between the AI model and the conventional applications we’ve been tinkering with for ages. The tools that these agents utilize can vary widely. Some may demand greater memory bandwidth rather than computational power, others might perform better on chips with substantial unified caches or specialized compression engines, and some might prioritize high frequency over core count, or vice versa. This is why AMD and Intel don’t just produce a single Epyc or Xeon SKU – and why these so-called ‘purpose-built’ agent CPUs are significantly diverse.
Nvidia’s Sleek Vera CPU
Examining Nvidia’s 88-core Vera CPU, this chip offers exceptional single-threaded performance with plenty of memory and interconnect bandwidth. As Huang elaborated during his GTC Taiwan keynote, this combination of speed and bandwidth is crucial for minimizing latency. “There will be billions of agents, and these agents are going to use the CPUs with very little patience because the cost of the GPUs they operate alongside is far too high,” he stated. But let’s be candid, Huang will always present it this way – he’s entrenched in the GPU sector. Vera, similar to Grace, was designed to facilitate smooth data transfer between the CPU and GPU. Data movement is fundamentally Vera’s specialty.
Arm’s AGI CPU and Amazon’s Graviton 5
Conversely, Arm’s AGI CPU appears to be a conventional Neoverse V3 processor with 136 cores, devoid of any additional features that an agent doesn’t require, all aimed at conserving power. No simultaneous multithreading or dedicated accelerators, minimal vector extensions, yet abundant memory bandwidth. Amazon’s 192-core Graviton 5 processors, showcased at Re:Invent last winter, are essentially an enhanced variant of Arm’s AGI CPU, utilizing the same Neoverse V3 cores, but even more generic, in my opinion.
Cease the ‘AI Chips’ Nonsense
Let’s not neglect Intel and AMD in this spectacle. They are also eager to promote their top-tier Xeons and Epycs as ideal for operating AI agents. At Computex earlier this month, Intel unveiled a couple of reference rack designs integrating up to 36,864 x86 cores into a 100 kW rack. Meanwhile, AMD, following an initial series of Vera CPU benchmarks, adopted a defensive stance last week, asserting that concurrency, rather than latency, is what truly matters when deploying agents at scale. The House of Zen estimates that for a 100 kW power envelope, its forthcoming 256-core Venice Epycs, set to debut later this year, would achieve 3.3 times the throughput per rack than Vera.
Varied Approaches for Varying Needs
If it seems like everyone has their own perspective on what the optimal agent CPU should entail, it’s because, similar to any datacenter workload, there’s rarely a singular correct response. This becomes evident in the early benchmarks of Nvidia’s Vera CPU. Late last month, FOSS-friendly outlet Phoronix received early access to the chip and conducted a subset of its test suite that Nvidia presumably considered representative of its target audience. The chip registered a geo-mean score that was 10 percent superior to AMD’s 128-core Epyc 9575F and 55 percent higher than Intel’s 128-core Xeon 6980P. A commendable performance, indeed. Yet, a closer look at the outcomes reveals that Vera excels in some applications more than others. And this touches on the crux of the matter. There has never been one CPU to dominate them all, and as the AI hype train accelerates into its agentic age, there certainly isn’t one now.
Summary: “AI Chips? More Like AI Chips and Dip!”
Nothing has changed – branding these CPUs as ‘AI chips’ is akin to labeling a regular cuppa as “gourmet.” They’re inflating old technology with a sprinkle of AI flair. So, let’s stay grounded and call it as it is.
