Santa clara: Nvidia announced on Monday that its Groq 3 LPX artificial intelligence racks have entered full production, marking the commercialization of technology acquired via the chipmaker's record $20 billion purchase of Groq assets. The systems will be deployed alongside Nvidia's Vera central processing units and Rubin graphics processing units at cloud infrastructure provider Nebius, with expectations to become operational later this year.
According to Anadolu Agency, Nvidia's acquisition of Groq assets in December was the company's largest-ever transaction. The Groq 3 LPX racks are specifically designed for low-latency inference, a critical process where trained AI models generate responses. This capability is particularly important for applications like AI agents and coding assistants, where any delays can negatively impact the user experience.
Each Groq 3 LPX rack contains 256 Groq 3 chips and is capable of generating approximately 3,400 tokens per second, based on a benchmark cited by Nvidia. The Groq architecture includes 500 megabytes of high-speed static random-access memory on each chip to minimize memory-related bottlenecks. Samsung Electronics manufactures the Groq chips, while Taiwan Semiconductor Manufacturing Company produces Nvidia's graphics processors.
Nvidia has stated that these specialized systems are meant to complement, not replace, GPUs. While GPUs are capable of performing both AI model training and inference, Groq chips are primarily focused on the latency-sensitive 'decode' phase of running models. The company is also boosting shipments of its Vera Rubin systems, which began production earlier this year.
Nvidia CEO Jensen Huang remarked in March that the company anticipates cumulative sales from its Blackwell and Vera Rubin platforms to reach $1 trillion by 2027. He also noted that a quarter of the data center capacity allocated to coding applications would utilize Groq chips. The competition in specialized inference hardware is intensifying as technology companies strive to enhance the speed and cost-effectiveness of AI services. Nvidia's competitor, Advanced Micro Devices, has also announced plans to integrate rack-scale systems with chips produced by Cerebras.