Data Parallelism: How to Train Deep Learning Models on Multiple GPUs

About this course

Train deep learning models faster by spreading the work across GPUs. You will see how batch size affects stochastic gradient descent, distribute training with PyTorch Distributed Data Parallel and keep accuracy high as you scale.

What you'll learn

• Understand how data parallel deep learning training is performed using multiple GPUs
• Achieve maximum throughput when training, for the best use of multiple GPUs
• Distribute training to multiple GPUs using Pytorch
• Distributed Data Parallel
• Understand and utilize algorithmic considerations specific to multi-GPU training performance and accuracy

Curriculum

Who it's for

Deep learning practitioners who train large models and need more throughput.

Before you start

• Experience with deep learning training using Python