Diffusion Models
Thermodynamics-inspired networks that generate data by reversing a noise-addition process.
In Plain Language
Take a photo and add a speck of static, again and again, until it is nothing but noise. Now teach a program to undo one single step of that — to make a noisy picture slightly less noisy. Once it can do that, start it on pure static and let it tidy up, over and over, until a brand new picture appears.
Diffusion models are trained on a deliberately easy question. Take a real image, add a known amount of random noise, and ask the network to estimate the noise it was just given. That has a correct answer you can check against, so training is stable in a way GAN training is not. To generate something new, you start from pure noise and apply the same denoising step repeatedly, each pass removing a little more static until an image emerges — and because the starting noise is random, you get a different image every time. Steering the result with a text prompt is a matter of conditioning each step on the words, which is how Stable Diffusion and DALL·E turn a sentence into a picture.
Why It Was Important
Surpassing GANs in stability and quality, Diffusion models iteratively 'denoise' an image starting from static. Denoising Diffusion Probabilistic Models (DDPMs) powered the meteoric rise of Midjourney, DALL-E, and Stable Diffusion, completely democratizing high-fidelity graphic design and video generation.
Who Invented It
Jonathan Ho, Ajay Jain, Pieter Abbeel
Researchers at UC Berkeley.
Applications
- Text-to-Image
- Text-to-Video (Sora)
- Medical Image Synthesis
Key Papers
- Denoising Diffusion Probabilistic Models
Jonathan Ho, Ajay Jain, Pieter Abbeel · NeurIPS 2020