The World Is Not Enough: RL’s Environment Problem

While researchers may still opine about access to compute, most are hitting a new bottleneck: access to high-quality environments in which to post-train agents. Expanding agent capabilities will depend on how we curate and utilize data, much of which exists only in the minds of human experts today. This panel will explore the challenge of building environments, including the need to improve infrastructure reliability, the obstacles posed by subjective domains, and the challenge of achieving high variety. We’ll also discuss future opportunities, such as environments for long-horizon reasoning and continual learning, and what it will take to evolve environment design from a bespoke craft into a more automated, scalable discipline that consistently drives generalization.