Loading timeline…
20232020s
State Space Models (Mamba)
A linear-time sequence architecture positioned as a highly efficient alternative to Transformers.
Why It Was Important
Transformers scale quadratically—processing a 100k-word book takes exponentially more memory than a 10k-word essay. Albert Gu and Tri Dao developed Mamba using selective State Space Models, achieving Transformer-level reasoning but scaling linearly. This allowed models to natively process millions of tokens context loops on minimal hardware.
Who Invented It
Albert Gu, Tri Dao
Researchers at CMU and Princeton focusing on recurrent sequence limits.
Applications
- Infinite Context Windows
- Genomic Sequence Parsing
- Audio Analysis
Key Papers
- Mamba: Linear-Time Sequence Modeling with Selective State Spaces
Albert Gu, Tri Dao · arXiv · 2023
Videos
What are State Space Models? Redefining AI & Machine Learning with Data
IBM Technology