Loading timeline…
20152010s
Batch Normalization
A mechanism to accelerate the training of deep neural networks by normalizing layer inputs.
Why It Was Important
As networks grew deeper, managing internal covariate shifts became unstable. Sergey Ioffe and Christian Szegedy introduced Batch Norm to normalize activation matrices between layers, allowing for much higher learning rates and drastically lowering the time it took to train monumental networks.
Who Invented It
Sergey Ioffe & Christian Szegedy
Google researchers focused on model training efficiency.
Applications
- Deep Network Optimization
- Training Acceleration
Key Papers
- Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Sergey Ioffe, Christian Szegedy · ICML 2015
Videos
Why Does Batch Norm Work? (C2W3L06)
DeepLearningAI