Loading timeline…
20222020s
Organizations
Stanford UniversityPrinceton UniversityTogether AI
Major Achievements
- •Authored the FlashAttention algorithm, a drastically faster, memory-efficient exact attention mechanism.
- •FlashAttention became an absolute staple in deploying virtually all massive language models.
- •Co-authored Mamba with Albert Gu.
Key Papers
- Mamba: Linear-Time Sequence Modeling with Selective State Spaces
Albert Gu, Tri Dao · arXiv · 2023
- FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness
Tri Dao et al. · NeurIPS 2022