How Open Frontier Labs Actually Train Their Models

Training a large language model is an exercise in tradeoffs you didn’t expect, where choices made for pre-training can shape post-training, agentic RL, and inference much later. Should you spend a week optimizing infrastructure and architecture, or just start training when every design choice affects rollout speed, memory use, and serving cost? This talk covers how to think about mdeol decisions: why architecture changes are rarely just about accuracy and almost always about performance, how inference choices determine what is feasible for post-training, and how frontier open labs design models for RL-heavy, agentic workloads. We’ll walk through the full lifecycle of a model, from pre-training to mid-training to post-training RL, examining how decisions at each stage shape the next and why tradeoffs around efficiency, capability, and inference cost rarely have clean answers