Loading timeline…
20212020s
Low-Rank Adaptation (LoRA)
A wildly efficient parameter-tuning method allowing individuals to fine-tune massive models on consumer hardware.
Why It Was Important
Standard fine-tuning requires updating billions of numbers, taking weeks on expensive data centers. LoRA mathematically freezes the original model and introduces tiny 'rank decomposition matrices' that inject the new knowledge. This democratized AI, allowing community developers to custom-train local LLMs on a single RTX gaming computer.
Who Invented It
Edward Hu et al. (Microsoft)
Researchers focusing on scalable and democratized knowledge injection.
Applications
- Open Source Fine Tuniing
- Stable Diffusion Custom Checkpoints
- Hugging Face integration
Key Papers
- LoRA: Low-Rank Adaptation of Large Language Models
Edward J. Hu et al. · ICLR 2022 · 2021