Loading timeline…
20232020s
Synthetic Data Generation
Using frontier AI models to perfectly generate complex training data for smaller models.
Why It Was Important
The world rapidly approached the 'data wall'—running out of high-quality human text on the internet to train next-generation models. The solution was having GPT-4 write billions of high-quality coding tutorials and philosophical debates to train smaller, specialized open-source models, shifting the paradigm of data science.
Who Invented It
AI scaling labs
Researchers circumventing the limits of human-generated internet data.
Applications
- Alpaca/Vicuna Models
- Coding LLM training
- Data-wall bypass
Videos
What is Synthetic Data? No, It's Not "Fake" Data
IBM Technology