Nvidia's Groq 3 LPU targets agentic AI inference at GTC 2026

"Groq built its Language Processing Units specifically for AI inference, running AI models, rather than training them. The LPU's architecture functions as a software-defined assembly line for AI workloads, moving data directly between on-chip memory modules without the overhead inherent to Nvidia's general-purpose GPU design. That makes it very fast indeed, and sidesteps memory bandwidth bottlenecks inherent to GPUs with separate memory modules."

"The Groq 3 carries that philosophy further. Its memory, while smaller than what Nvidia's GPUs offer, delivers 40 petabytes per second of bandwidth, enabling inference speeds that outpace anything a GPU can manage. The chip ships in dedicated Groq 3 LPX server racks, each holding 256 LPUs with 128 gigabytes of solid-state random access memory."

"Ian Buck, Nvidia's vice president of hyperscale and high-performance computing, framed the Nvidia-Groq partnership in clear terms. Groq 3 acts as a coprocessor to the Rubin GPUs, boosting performance at 'every layer of the AI model on every token.'"

Nvidia announced the Groq 3 LPU at GTC 2026, the first product from its $20 billion acquisition of Groq. The LPU is specifically designed for AI inference rather than training, using a software-defined architecture that moves data directly between on-chip memory modules, avoiding GPU memory bandwidth bottlenecks. The Groq 3 delivers 40 petabytes per second of bandwidth and ships in dedicated server racks containing 256 LPUs with 128 gigabytes of solid-state RAM each. Nvidia positions the Groq 3 as a coprocessor to its Rubin GPUs, enhancing performance across AI model layers and tokens.

Read at Techzine Global