Noam Shazeer

Rating

Domains

Scouting report

Transformer co-author whose fingerprints are on most of what made the architecture fast: multi-query attention, sparsely-gated mixture-of-experts, the Mesh-TensorFlow work on model parallelism, and the T5 line. Left Google to found Character.AI, then returned to lead Gemini. Widely regarded as the best engineer in the room on making big models cheap.

Curator’s note

Designed it, then spent a decade making it affordable to run.

Sources

• 01Attention Is All You Need (2017) arxiv.org
• 02Outrageously Large Neural Networks: Sparsely-Gated Mixture-of-Experts (2017) arxiv.org