Everything You Need to Know About the NVIDIA Blackwell GPUs

Highlights

• Announced on 18 March 2024 by NVIDIA CEO Jensen Huang at GTC 2024

• NVLink Enable Trillion-Parametre-Scale AI Models

• Latest Blackwell Tensor Cores and TensorRT- LLM Compiler Reduce LLM Inference Operating Cost and Energy by up to 25 times

• New Accelerators Enable Innovation in Data Processing, Electronic Design Automation, Computer-Aided Engineering and Quantum Computing

After months of anticipation, NVIDIA CEO Jensen Huang finally took the stage today at “The #1 AI Conference for Developers” GTC 2024. His highly-anticipated keynote kicked off the event, promising to reveal cutting-edge innovations that will power the new era of Generative AI. 

The announcement revealed NVIDIA's latest breakthrough called the “Blackwell”. It is the new architecture succeeding the NVIDIA Hopper architecture launched on September 20, 2022. The NVIDIA Blackwell GPU is named after David Harold Blackwell, a statistician and mathematician specialising in game theory and statistics. 

The NVIDIA Blackwell GPU will enable organisations across the globe to build and run real-time inference on trillion-parameter large language models at 25x less cost and energy consumption than its predecessor. The NVIDIA Blackwell architecture features six transformative technologies for generative AI and accelerated computing, which will help in breakthroughs in data processing, electronic design automation, computer-aided engineering and quantum computing. 

NVIDIA Blackwell Features to Power Generative AI

NVIDIA Blackwell GPUs are powered by six revolutionary technologies that will enable AI training and real-time LLM inference for models scaling up to 10 trillion parametres. The ground-breaking GPU will include the following features: 

1. World’s Most Powerful Chip: The Blackwell processor packs 208 billion transistors and is manufactured using a custom-built 4NP TSMC process. All NVIDIA Blackwell products feature two dies that are linked via a 900GB/s chip-to-chip interconnect, providing the power of two GPUs in one fully cache-coherent, full-stack-ready chip.
2. Generative AI Engine: Custom Tensor Core technology, combined with NVIDIA TensorRT-LLM and NeMo framework innovations, accelerates inference and training for LLMs, including a mixture-of-experts models. Enterprises can optimise their business with the latest expert parallelism and quantisation techniques and deploy these models by using new precision formats.
3. Secure AI: Advanced confidential computing capabilities protect AI models and customer data with uncompromised performance, with support for new native interface encryption protocols, which are critical for data-sensitive industries like healthcare and financial services.
4. Fifth-Generation NVLink: To accelerate performance for multitrillion-parameter AI models, NVLink’s latest iteration delivers ground-breaking 1.8 Tb/s throughput per GPU, ensuring seamless high-speed communication among up to 576 GPUs for today’s most complex large language models.
5. Decompression Engines: Dedicated decompression engines support the latest formats, accelerating database queries to deliver the highest performance in data analytics and data science. 
6. RAS Engine: Only Blackwell-powered GPUs include a dedicated engine for reliability, availability and serviceability. 

Also, the Blackwell architecture adds capabilities at the chip level to utilise AI-based preventative maintenance to run diagnostics and forecast reliability issues. This maximises system uptime and improves resiliency for massive-scale AI deployments to run uninterrupted for weeks or even months at a time and to reduce operating costs.

About NVIDIA Blackwell GPUs: B100 and B200

Based on the Blackwell architecture, the NVIDIA B200 Tensor Core GPU delivers a massive leap forward in speeding up inference workloads, making real-time performance a possibility for resource-intensive and multitrillion-parameter language models.

Two B200 GPUs are combined in Blackwell’s flagship accelerator, the NVIDIA GB200 Grace Blackwell Superchip, which also utilises an NVIDIA Grace CPU. The GB200 provides a 30x performance increase compared to the NVIDIA H100 Tensor Core GPU for LLM inference workloads and reduces cost and energy consumption by 25x.

For the highest AI performance, GB200 supports the NVIDIA Quantum-X800 InfiniBand and Spectrum™-X800 Ethernet platforms which deliver advanced networking options at speeds up to 800 Gb/s. The GB200 NVL72 also includes NVIDIA BlueField®-3 data processing units to enable cloud network acceleration, composable storage, zero-trust security and GPU compute elasticity in hyperscale AI clouds.

The GB200 is a key component of the NVIDIA GB200 NVL72, a multi-node, liquid-cooled, rack-scale platform for the most compute-intensive workloads. It combines 36 Grace Blackwell  Superchips, which include 72 B200 GPUs and 36 Grace CPUs interconnected by fifth-generation NVLink.

To help accelerate the development of Blackwell-based servers from its partner network, NVIDIA announced NVIDIA HGX B200, a server board that links eight B200 GPUs through high-speed interconnects to develop the world’s most powerful x86 generative AI platforms. HGX B200 supports networking speeds up to 400 Gb/s through the Quantum-2 InfiniBand and Spectrum-X Ethernet platforms, along with support for BlueField-3 DPUs.

NVIDIA Blackwell GPU Specifications 

Check out the latest NVIDIA Blackwell GPU specifications for NVIDIA Blackwell GB200, B200 and B100: 

FAQs

What is the NVIDIA Blackwell?

NVIDIA Blackwell is the latest ground-breaking GPU architecture announced on 18 March by NVIDIA CEO Jensen Huang at the “The #1 AI Conference for Developers” GTC 2024 in San Jose. 

What are the features of NVIDIA Blackwell?

The Blackwell Architecture by NVIDIA is powered by six revolutionary technologies that will enable AI training and real-time LLM inference for models scaling up to 10 trillion parametres:

1. World’s Most Powerful Chip: The Blackwell processor packs 208 billion transistors and is manufactured using a custom-built 4NP TSMC process. All NVIDIA Blackwell products feature two dies that are linked via a 10 Tb/s chip-to-chip interconnect, providing the power of two GPUs in one fully cache-coherent, full-stack-ready chip.
2. Generative AI Engine: Custom Tensor Core technology, combined with NVIDIA TensorRT-LLM and NeMo framework innovations, accelerates inference and training for LLMs, including a mixture-of-experts models. 
3. Secure AI: Advanced confidential computing capabilities protect AI models and customer data with uncompromised performance, with support for new native interface encryption protocols, which are critical for privacy-sensitive industries like healthcare and financial services.
4. Fifth-Generation NVLink: To accelerate performance for multitrillion-parameter AI models, NVLink’s latest iteration delivers ground-breaking 1.8Tb/s throughput per GPU, ensuring seamless high-speed communication among up to 576 GPUs for today’s most complex large language models.
5. Decompression Engines: Dedicated decompression engines support the latest formats, accelerating database queries to deliver the highest performance in data analytics and data science.
6. RAS Engine: Only Blackwell-powered GPUs include a dedicated engine for reliability, availability and serviceability. 

What is the NVIDIA GB200?

The GB200 is a key component of the NVIDIA GB200 NVL72, a multi-node, liquid-cooled, rack-scale platform for the most compute-intensive workloads. It combines 36 Grace Blackwell  Superchips, which include 72 B200 GPUs and 36 Grace CPUs interconnected by fifth-generation NVLink.