NVIDIA (CUDA Era)

Contribution to AI

Before 2006 a matrix multiplication on a graphics card had to be disguised as a drawing operation: data packed into textures, the computation written as a pixel shader, the answer read back out of a rendered frame. CUDA removed the disguise. Announced alongside the G80 architecture in November 2006 — whose 128 shader units replaced the separate vertex and pixel pipelines with one programmable array — it let a researcher write C with a few extensions and target the whole chip directly. The consequence that mattered for AI was economic as much as technical. CUDA ran on ordinary consumer GeForce cards, so parallel compute at hundreds of gigaflops became something a graduate student could buy, and in 2012 AlexNet was trained on two GTX 580s in a bedroom, roughly six thousand lines of hand-written CUDA doing the work that a cluster would otherwise have required. cuDNN, released in 2014, moved the hard kernels — convolution, pooling, normalisation — below the frameworks, so Caffe, Torch and everything after inherited fast GPU training without anyone reimplementing it. What followed was a lock-in with few parallels in computing: a decade of AI research was written against one vendor's software stack, and competing accelerators have had to argue not about arithmetic throughput but about the libraries, compilers and habits accumulated on top of it. The bottleneck on progress became chip supply.

Drafted with AI and edited by hand (claude-opus-5, reviewed 2026-08).