Loading timeline…
20102010s
Self-Supervised Learning
Training methods where the data itself provides the supervision, eliminating manual labeling.
Why It Was Important
By randomly masking words in a sentence or patches in an image and forcing the model to predict the missing parts, the model learns the intrinsic structure of the world. This completely bypassed the crippling bottleneck of human data labeling, accelerating the rise of Foundation Models.
Who Invented It
Yann LeCun (Advocate) & Community
Champions of moving away from purely supervised bounds.
Applications
- Foundation Models
- Masked Language Modeling (BERT)
- Predictive Coding
Videos
Supervised vs. Unsupervised Learning
IBM Technology