Mila (Quebec AI Institute)

Contribution to AI

For most of the 1990s a recurrent network trained by gradient descent simply did not work, and the Montreal lab was among the few groups willing to explain why rather than change subject. The 1994 analysis of vanishing gradients showed the failure was structural rather than a matter of tuning, and defined the problem that gating and later attention were built to answer. The 2003 neural probabilistic language model was the second durable move: learning a distributed vector for each word jointly with the predictor that used it, instead of treating words as unrelated symbols. Every embedding since rests on that assumption. Theano, in use from 2007, let a researcher write a mathematical expression in Python and receive compiled GPU code with the derivatives worked out symbolically — the graph-and-autodiff arrangement that TensorFlow and PyTorch later inherited, and the reason a graduate student could try an architecture in an afternoon. Two 2014 papers from the same corridors reshaped what people trained: additive attention, which let a translation decoder consult the whole source sentence rather than one compressed vector, and adversarial training, which turned generative modelling into a contest between two networks needing no Markov chains. The institutional result matters as much. Concentrating deep learning researchers in one city gave Canada a case for the 2017 national strategy, the first of its kind, and made Montreal a place where ethics consultations happened alongside the training runs rather than after them.

Drafted with AI and edited by hand (claude-opus-5, reviewed 2026-08).