Loading timeline…
20062000s
Deep Belief Networks
Generative graphical models trained greedily, one layer at a time.
Why It Was Important
Geoffrey Hinton showed that a 'deep' (multi-layer) neural network could be effectively trained if each layer was initially pre-trained without supervision as a Restricted Boltzmann Machine, before fine-tuning the whole stack with backpropagation. This 'layer-wise pre-training' breakthrough proved deep networks could finally be optimized.
Who Invented It
Geoffrey Hinton, Simon Osindero, Yee-Whye Teh
The pioneers who persisted through the 'AI Winter' of neural networks.
Applications
- Early Deep Learning
- Feature Extraction
- Dimensionality Reduction
Key Papers
- A Fast Learning Algorithm for Deep Belief Nets
Geoffrey E. Hinton, Simon Osindero, Yee-Whye Teh · Neural Computation · 2006