Gods of AI

Co-authored the 1986 paper that made backpropagation practical, then spent the AI winter insisting neural networks would work while the field moved on. Was proved right in 2012 when his students won ImageNet by a margin nobody could argue with. Shared the 2018 Turing Award and the 2024 Nobel Prize in Physics, and left Google to speak freely about the risks.

Built LeNet in the late 1980s and had convolutional networks reading cheques at industrial scale while the rest of the field doubted them. Shared the 2018 Turing Award. As Meta’s chief AI scientist he has been the loudest senior voice for releasing weights rather than gatekeeping them, which is a large part of why open-weight models exist at all.

Third of the 2018 Turing Award trio, and the one who stayed in the university. Founded MILA, whose lab produced Theano — the framework a generation of researchers learned on before TensorFlow and PyTorch existed. Co-authored the Deep Learning textbook, and now chairs international scientific reporting on AI risk.

With Sepp Hochreiter published long short-term memory in 1997, the architecture that made sequence learning work and which ran speech recognition and translation on a billion phones before transformers arrived. Also did early work on meta-learning, artificial curiosity and GAN-adjacent ideas, and is famous for pointing out, at length, when the field forgets its own history.

Invented temporal-difference learning and co-wrote Reinforcement Learning: An Introduction, the textbook the whole field learned from and which is free online. His 2019 essay “The Bitter Lesson” — that general methods plus compute beat handcrafted human knowledge — has become the most-cited two pages in modern AI. Shared the 2024 Turing Award.

With Sutton, built reinforcement learning into a coherent field from the actor-critic architecture onward, and co-wrote the textbook that defined it. His work tied machine learning back to animal learning and dopamine-driven reward prediction, which turned out to describe both. Shared the 2024 Turing Award.

Made probabilistic reasoning tractable with Bayesian networks, then built the formal machinery of causal inference — do-calculus, counterfactuals, the ladder of causation. Turing Award, 2011. Spends much of his time arguing that systems which only fit correlations are missing the thing that makes reasoning work, an argument the scaling era has not settled.

Built ImageNet when the field believed better algorithms mattered more than more data, and gave it away. The 2012 competition on that dataset is the moment deep learning stopped being a minority position. Founded Stanford HAI, served as chief scientist of AI at Google Cloud, and has pushed hard on who gets a say in how this technology is built.

Lead author of the 2017 paper that threw out recurrence and convolution and kept only attention. Every large language model since is a transformer, which makes those eight pages the most consequential architecture paper of the century so far. Later co-founded Adept and then Essential AI.

Transformer co-author whose fingerprints are on most of what made the architecture fast: multi-query attention, sparsely-gated mixture-of-experts, the Mesh-TensorFlow work on model parallelism, and the T5 line. Left Google to found Character.AI, then returned to lead Gemini. Widely regarded as the best engineer in the room on making big models cheap.

Transformer co-author who also built and released Tensor2Tensor, the open library that put a working transformer implementation in everyone’s hands within weeks of the paper, and later Trax. Moved to OpenAI to work on the reasoning-model line. The gap between publishing an architecture and everyone being able to run it was largely his to close.

Was an intern when he became a co-author on Attention Is All You Need. Co-founded FOR.ai and then Cohere, building large language models for enterprises, and Cohere Labs has released open-weight multilingual models including the Aya family with a strong emphasis on languages the field usually ignores.

Wrote the CUDA code that trained AlexNet on two consumer GPUs and won ImageNet 2012 by a margin so large it settled the question of whether deep learning worked. Released cuda-convnet so others could reproduce it. Also assembled CIFAR-10 and CIFAR-100, which remain the default first datasets for anyone learning the field.

Co-author of AlexNet, then of sequence-to-sequence learning, then co-founder and chief scientist of OpenAI through the GPT series. The through-line of his career is a conviction that scale plus the right objective gets you most of the way, held earlier and harder than almost anyone. Left in 2024 to found Safe Superintelligence.

Proposed generative adversarial networks in 2014 — two networks trained against each other — and gave image generation its first decade. Co-wrote the Deep Learning textbook with Bengio and Courville, which is free to read online, and did foundational work on adversarial examples that became its own subfield.

Wrote FlashAttention, an IO-aware exact attention kernel that cut memory use enough to make long contexts practical, and released it openly — it is now in essentially every training stack in the world. Co-created the Mamba state space architecture with Albert Gu, also open. Chief scientist at Together AI and a Princeton professor.

Developed structured state space models — S4 and its successors — showing that sequence modelling with linear-time scaling could compete with quadratic attention on long inputs. Co-created Mamba with Tri Dao and released it openly. The most serious architectural alternative to the transformer to come out of the last decade.

First author on the original GPT paper, on GPT-2, on CLIP and on Whisper — an unusual concentration of foundational work in one person. DCGAN before that. Whisper was released openly with weights and remains the default speech recognition model for anyone who does not want to send audio to an API.

Led the AlphaGo work that beat Lee Sedol in 2016, a result the field had expected to be a decade away, then AlphaZero — which threw away the human games entirely and taught itself Go, chess and shogi from the rules. His RL lecture series, free on YouTube, taught the subject to a generation.

Co-authored sequence-to-sequence learning and Pointer Networks, led AlphaStar to grandmaster level at StarCraft II — a far messier problem than board games — and is now a lead on Gemini. One of the few researchers whose career spans the pre-transformer, game-playing and frontier-model eras at the top of each.

Led AlphaFold 2, which solved a fifty-year-old grand challenge in biology to experimental accuracy. DeepMind then released the code and predicted structures for essentially every known protein, free, which is the part biologists actually talk about. Shared the 2024 Nobel Prize in Chemistry.

Co-founded DeepMind with Hassabis and Suleyman and is chief AGI scientist there. His doctoral work with Marcus Hutter produced a formal definition of machine intelligence, and he popularised the term AGI at a point when saying it out loud was a career risk. Has publicly held a rough timeline for human-level AI since 2011.

Chess prodigy, then games developer, then neuroscientist, then founder of DeepMind — the lab that produced DQN, AlphaGo, AlphaZero and AlphaFold. Shared the 2024 Nobel Prize in Chemistry for the last of those. Has been consistent for two decades that the point of building intelligence is to use it on science.

Berkeley professor who moved deep RL out of simulators and onto physical robots, with apprenticeship learning, domain randomisation and the BAIR lab’s long run of open releases. Co-founded Covariant, and his Deep RL Bootcamp and course materials are freely available and are how a lot of people learned the subject.

Founding member of OpenAI, then director of AI at Tesla, but the reason he is on this card is the teaching: char-rnn, CS231n, micrograd, nanoGPT, llm.c and the Zero to Hero series, all free, all readable. He has probably taught more working engineers how a transformer actually works than every university course combined.

Built much of the distributed systems layer modern computing runs on — MapReduce, BigTable, Spanner — then co-founded Google Brain and drove DistBelief, TensorFlow and the TPU programme. TensorFlow being released openly in 2015 is a large part of why the deep learning boom was not confined to a handful of labs.

Led TensorFlow engineering at Google, taking the internal DistBelief system and shipping it as an open-source framework with a public API, documentation and a community. For several years TensorFlow was simply how machine learning was done outside a handful of labs, and that was an engineering and release-management achievement as much as a research one.

Co-created and led PyTorch, which won the research community by being straightforwardly pleasant to debug when the alternative was not. Ran it as a genuinely open project with an unusually visible maintainer, shepherded it into the Linux Foundation, and co-authored DCGAN along the way. Almost every model in this set was trained in his framework.

Wrote the original PyTorch autograd engine as a student and is first author on the PyTorch paper, then moved to Google and became a core author of JAX. Two of the three frameworks serious research runs on have his design decisions inside them, which is an unusual thing to be able to say.

Wrote XGBoost, which won a remarkable share of Kaggle competitions and is still the default for tabular data. Then co-created MXNet, then Apache TVM for compiling models to any hardware, then MLC-LLM for running models on phones and browsers. Every one of them open, and each solved a bottleneck the field was about to hit.

Wrote Caffe as a Berkeley PhD student and released it, giving computer vision its first widely usable deep learning framework at exactly the moment the field needed one. Later built Caffe2 at Facebook, co-created ONNX so models could move between frameworks, and led large-scale AI infrastructure at Alibaba before founding Lepton AI.

Wrote Keras, whose entire premise was that building a neural network should not require a PhD in the framework, and it became the on-ramp for an enormous number of practitioners. Also created the ARC benchmark and its prize, a deliberate test of fluid reasoning that current models still find hard — a useful corrective to benchmark saturation.

Co-founder and chief scientist of Hugging Face, and the driving force behind the transformers library, which turned "reproduce this paper" from a week of work into an import statement. Led the BigScience collaboration that produced BLOOM, one of the first genuinely open large multilingual models, built by a thousand researchers in the open.

Co-founded Hugging Face and turned it from a chatbot app into the default distribution point for open models, datasets and demos. Whatever you think of the company, the Hub is the reason an open-weight release can reach everyone the day it lands, and the reason a small lab can ship a model at all.

Wrote ggml, whisper.cpp and llama.cpp — dependency-free C implementations that run large models on laptops, phones and Raspberry Pis. The GGUF format and its quantisation schemes are how most people actually run a model locally. Arguably did more for practical access to open models than any lab release.

Worked out how to run and fine-tune large models in 8-bit and then 4-bit without wrecking them, shipped it as bitsandbytes, and followed with QLoRA — which put fine-tuning a 65B model on a single consumer GPU. His hardware guides are the reference for anyone deciding what to buy.

Co-authored ULMFiT, which brought transfer learning to NLP before the GPT line, and founded fast.ai, whose free courses and library have trained an enormous number of practitioners on the premise that you do not need a PhD to do this. Has been one of the most persistent public voices against regulating open-weight models out of existence.

Executive director of EleutherAI, the volunteer collective that trained and released GPT-Neo, GPT-J and GPT-NeoX when nothing comparable was public, and built the Pile. Led Pythia, a suite of models released with every training checkpoint and the data order — which made a whole class of interpretability and memorisation research possible for the first time.

Wrote Cosmopolitan Libc, which produces a single binary that runs natively on Linux, macOS, Windows and BSD, then used it to build llamafile — a whole language model and its runtime in one executable file you can copy to a machine and run. Also contributed substantial CPU inference speedups back to llama.cpp.

Led LLaMA at Meta, showing that a smaller model trained on far more tokens beat much larger ones — and then it reached the public. Nearly every open fine-tune, quantisation and local-inference project of the following two years traces to those weights. Co-founded Mistral, which has kept releasing open-weight models under permissive licences.

Worked on Chinchilla and Flamingo at DeepMind, then co-founded Mistral and released Mistral 7B and Mixtral under Apache 2.0 — models good enough that "open weights" stopped being a compromise. Made the case, commercially rather than rhetorically, that a frontier lab can give the weights away and still have a business.

Lead author of the latent diffusion paper — run the diffusion in a compressed latent space and image generation suddenly fits on a consumer GPU — which became Stable Diffusion. Its open release in 2022 put image generation in everyone’s hands overnight and set off both an explosion of tooling and a still-unresolved argument about training data.

Co-authored VQGAN — the taming-transformers work that made high-resolution generation tractable — and then latent diffusion and Stable Diffusion. Later led the rectified-flow architecture behind Stable Diffusion 3 and moved to Black Forest Labs for the FLUX models, which were also released with open weights.

A schoolteacher who organised a volunteer effort to assemble LAION-400M and LAION-5B, the open image-text datasets that Stable Diffusion and OpenCLIP were trained on. Before LAION, the data behind multimodal models was private by default; afterwards, anyone could inspect it — including the researchers who found serious problems in it.

McGill professor and long-time head of Meta’s AI research, during which FAIR released PyTorch, LLaMA, and a long run of open models and code. Created the machine learning reproducibility checklist adopted by NeurIPS, forcing the field to state its assumptions and publish enough to be checked.

Did the feature-visualisation and circuits work that turned "what is the network doing" from a shrug into a research programme, and co-founded Distill to publish it in a form people could actually understand. Co-founded Anthropic, where the interpretability team continues to publish its findings openly.

Lead author of deep reinforcement learning from human preferences, the technique that became RLHF and turned raw language models into things that follow instructions — the single largest reason chat assistants work at all. Later founded the Alignment Research Center and moved to the US AI Safety Institute.

Co-led the work that turned RLHF into InstructGPT and then ChatGPT, and headed OpenAI’s superalignment effort. Resigned publicly in 2024 saying safety culture had lost out to shipping, which is a rare thing for someone at that level to say on the record, and moved to Anthropic to continue the work.

Co-wrote Artificial Intelligence: A Modern Approach with Peter Norvig, the textbook that has taught the subject in essentially every university for thirty years. Then spent the later part of his career arguing that optimising a fixed objective is the wrong design, and that systems should be uncertain about what humans want.

Co-authored Concrete Problems in AI Safety, which gave safety research a concrete agenda, and led the GPT-2 and GPT-3 work where the scaling laws were established. Left OpenAI to co-found Anthropic, whose interpretability and Constitutional AI research is published openly even though the models are not.