""It's more about model management than about model building - and the CPU is critical in providing workflow management," said Jack Gold, principal analyst at J. Gold Associates."
""GPUs in training use more electricity generically due to near 100% utilization in a training workload, whereas in general-purpose compute, servers and CPUs run more like 40% to 60% utilization," he said."
""But it's highly variable depending on what the agent is doing.""
""CPUs take on a major significance in making everything work. It's why all the hyperscalers are now loading up on CPUs, not just GPUs," Gold said."
Agentic AI requires different computing models than traditional generative AI, necessitating careful consideration of hardware and pricing from cloud providers. CPU utilization is crucial for workflow management, while GPU usage incurs higher costs due to electricity consumption during training. Predictions indicate a significant shift towards inference workloads in the coming years, highlighting the growing importance of CPUs as hyperscalers increasingly invest in them alongside GPUs.
Read at Computerworld
Unable to calculate read time
Collection
[
|
...
]