Loading timeline…
20032000s
Word Embeddings (Pre-cursors)
Representing words as continuous vectors in a dense mathematical space.
Why It Was Important
Yoshua Bengio published 'A Neural Probabilistic Language Model', introducing the idea of learning distributed representations for words concurrently with the language model itself. Rather than treating words as isolated discrete symbols, similar words mapped to similar multi-dimensional mathematical coordinates. This paved the way for word2vec in the 2010s.
Who Invented It
Yoshua Bengio et al.
Canadian computer scientist and deep learning pioneer.
Applications
- Language Modeling
- Semantic Search
- Foundations of LLMs
Key Papers
- A Neural Probabilistic Language Model
Yoshua Bengio et al. · Journal of Machine Learning Research · 2003