All About the NVIDIA Blackwell GPUs

The NVIDIA Blackwell GPUs represent the biggest architectural shift since Hopper, built for trillion-parameter models and extreme performance per watt. This article explains exactly what’s new, faster training, improved inference efficiency, and advanced AI-native capabilities right from the start. We break down the architecture, expected performance gains and real-world implications with clear examples. 

Highlights

• Announced on 18 March 2024 by NVIDIA CEO Jensen Huang at GTC 2024

• NVLink Enable Trillion-Parametre-Scale AI Models

• Latest Blackwell Tensor Cores and TensorRT- LLM Compiler Reduce LLM Inference Operating Cost and Energy by up to 25 times

• New Accelerators Enable Innovation in Data Processing, Electronic Design Automation, Computer-Aided Engineering and Quantum Computing

After months of anticipation, NVIDIA CEO Jensen Huang finally took the stage today at “The #1 AI Conference for Developers” GTC 2024. His highly-anticipated keynote kicked off the event, promising to reveal cutting-edge innovations that will power the new era of Generative AI. Check out the full blog to learn about Blackwell specs, Blackwell GPU architecture and more.

Blackwell GPU Architecture

NVIDIA's Blackwell is the next-generation NVIDIA GPU built on the revolutionary NVIDIA Blackwell GPU architecture, featuring advanced NVLink, confidential computing, and the powerful NVIDIA Grace Blackwell Superchip for massive AI workloads.

The announcement revealed NVIDIA's latest breakthrough called the “Blackwell”. It is the NVIDIA Blackwell architecture succeeding the NVIDIA Hopper architecture launched on September 20, 2022. The NVIDIA Blackwell GPU is named after David Harold Blackwell, a statistician and mathematician specialising in game theory and statistics. The NVIDIA Blackwell GPU will enable organisations across the globe to build and run real-time inference on trillion-parameter large language models at 25x less cost and energy consumption than its predecessor. The NVIDIA Blackwell architecture features six transformative technologies for generative AI and accelerated computing, which will help in breakthroughs in data processing, electronic design automation, computer-aided engineering and quantum computing. 

NVIDIA Blackwell Features to Power Generative AI

NVIDIA Blackwell GPUs are powered by six revolutionary technologies that will enable AI training and real-time LLM inference for models scaling up to 10 trillion parametres. The ground-breaking GPU will include the following features: 

1. Blackwell chip: The Blackwell includes the new NVIDIA chip processor packs 208 billion transistors and is manufactured using a custom-built 4NP TSMC process. All NVIDIA Blackwell products feature two dies that are linked via a 900GB/s chip-to-chip interconnect, providing the power of two GPUs in one fully cache-coherent, full-stack-ready Blackwell chip.
2. Generative AI Engine: Custom Tensor Core technology, combined with NVIDIA TensorRT-LLM and NeMo framework innovations, accelerates inference and training for LLMs, including a mixture-of-experts models. Enterprises can optimise their business with the latest expert parallelism and quantisation techniques and deploy these models by using new precision formats.
3. Secure AI: Advanced confidential computing capabilities protect AI models and customer data with uncompromised performance, with support for new native interface encryption protocols, which are critical for data-sensitive industries like healthcare and financial services.
4. Fifth-Generation NVLink: To accelerate performance for multitrillion-parameter AI models, NVLink’s latest iteration delivers ground-breaking 1.8 Tb/s throughput per GPU, ensuring seamless high-speed communication among up to 576 GPUs for today’s most complex large language models.
5. Decompression Engines: Dedicated decompression engines support the latest formats, accelerating database queries to deliver the highest performance in data analytics and data science. 
6. RAS Engine: Only Blackwell-powered GPUs include a dedicated engine for reliability, availability and serviceability. 

Also, this architecture adds capabilities at the Blackwell chip level to utilise AI-based preventative maintenance to run diagnostics and forecast reliability issues. This maximises system uptime and improves resiliency for massive-scale AI deployments to run uninterrupted for weeks or even months at a time and to reduce operating costs.

About NVIDIA Blackwell GPUs: B100 and B200

Based on the Blackwell architecture NVIDIA, the NVIDIA B200 Tensor Core GPU delivers a massive leap forward in speeding up inference workloads, making real-time performance a possibility for resource-intensive and multitrillion-parameter language models.

Two B200 GPUs are combined in Blackwell’s flagship accelerator, the NVIDIA GB200 Grace Blackwell chip, which also utilises an NVIDIA Grace CPU. The GB200 provides a 30x performance increase compared to the NVIDIA H100 Tensor Core GPU for LLM inference workloads and reduces cost and energy consumption by 25x.

For the highest AI performance, GB200 supports the NVIDIA Quantum-X800 InfiniBand and Spectrum™-X800 Ethernet platforms which deliver advanced networking options at speeds up to 800 Gb/s. The GB200 NVL72 also includes NVIDIA BlueField®-3 data processing units to enable cloud network acceleration, composable storage, zero-trust security and GPU compute elasticity in hyperscale AI clouds.

The GB200 is a key component of the NVIDIA GB200 NVL72, a multi-node, liquid-cooled, rack-scale platform for the most compute-intensive workloads. It combines 36 Grace Blackwell  chip, which include 72 B200 GPUs and 36 Grace CPUs interconnected by fifth-generation NVLink.

To help accelerate the development of Blackwell-based servers from its partner network, NVIDIA announced NVIDIA Blackwell B200, a server board that links eight B200 GPUs through high-speed interconnects to develop the world’s most powerful x86 generative AI platforms. HGX B200 supports networking speeds up to 400 Gb/s through the Quantum-2 InfiniBand and Spectrum-X Ethernet platforms, along with support for BlueField-3 DPUs.

NVIDIA Blackwell GPU Specs

Check out the latest NVIDIA Blackwell specs for NVIDIA Blackwell GB200, B200 and B100: 

FAQs

What is the NVIDIA Blackwell architecture?

Blackwell is NVIDIA's GPU architecture announced March 2024, succeeding Hopper. It is built on a dual-die chip (208 billion transistors, 10 TB/s chip-to-chip interconnect), fifth-generation NVLink at 1.8 TB/s per GPU, and new Tensor Core precision formats including FP4 and FP6 — designed for trillion-parameter model training and real-time LLM inference.

Should I choose NVIDIA B200 or NVIDIA H100 for my workload?

Choose B200 for inference on models above 30B parameters, large-scale training, or full-parameter fine-tuning of 70B+ models. The 192 GB HBM3e and 8 TB/s bandwidth are decisive at that scale. For smaller models or cost-sensitive batch jobs already running well on H100, the H100 SXM remains a strong option on Hyperstack. The team can run a workload assessment if you are unsure.

Are B200 and GB200 available on demand or only by reservation on Hyperstack?

Both B200 and GB200 NVL72 are available for reservation on Hyperstack.

What is the chip-to-chip interconnect speed?

All NVIDIA Blackwell products feature two dies connected via a 10 TB/s chip-to-chip interconnect, presenting as a single fully cache-coherent GPU to the software stack.

Is Blackwell better than Hopper for training, not just inference?

Yes. For training, the key gains are higher HBM bandwidth, new FP8/FP6 precision formats for larger effective batch sizes, and fifth-generation NVLink reducing all-reduce overhead at scale. The GB200 NVL72's 72-GPU NVLink domain is particularly effective for tensor-parallel training of dense models above 100B parameters.