How Open Frontier Labs Actually Train Their Models
Training a large language model is an exercise in tradeoffs you didn't expect. Should you spend a week optimizing infrastructure and architecture, or just start training? This talk covers how to think about pre-training decisions: why architecture changes are rarely about accuracy and almost always about performance, how frontier open labs actually design models, and how to make principled calls when everything is a tradeoff. We'll walk through real examples of decisions that looked obvious in retrospect, and the ones that still don't have clean answers.