Loading timeline…
20142010s
Dzmitry Bahdanau
First author of the 2014 attention mechanism — the idea the Transformer was built on three years later.
Organizations
Université de MontréalElement AIServiceNow ResearchMcGill University
Major Achievements
- •Led 'Neural Machine Translation by Jointly Learning to Align and Translate' (2014) with Kyunghyun Cho and Yoshua Bengio.
- •Introduced attention: letting a decoder look back over the whole input instead of a single fixed-length vector.
- •Removed the bottleneck that had capped sequence-to-sequence translation quality.
- •'Attention Is All You Need' (2017) took this mechanism and discarded recurrence entirely.
Key Papers
- Neural Machine Translation by Jointly Learning to Align and Translate
Dzmitry Bahdanau, Kyunghyun Cho, Yoshua Bengio · ICLR 2015 · 2014