Loading timeline…
20012000s
Gradient Boosting
Building a strong predictor by adding simple models one at a time, each correcting the errors of those before it.
Why It Was Important
Jerome Friedman's formulation cast boosting as gradient descent in function space, turning a clever trick into general theory. Its descendants — XGBoost, LightGBM, CatBoost — became the default winners of tabular machine-learning competitions and remain the method of choice for structured business data, the one major domain where deep learning has not taken over.
Who Invented It
Jerome Friedman
Stanford statistician, also a co-author of the CART framework.
Applications
- Fraud Detection
- Ranking and Search
- Risk Modelling
- Tabular Prediction
Key Papers
- Greedy Function Approximation: A Gradient Boosting Machine
Jerome H. Friedman · The Annals of Statistics · 2001