Five years of OpenLineage: How we built an industry standard and why agents need it
Over the past five years, OpenLineage has become the de facto standard for data lineage metadata, adopted across the industry by leading platforms and enterprises. In this talk, we'll trace the journey of building an open standard. You'll learn what changed in the ecosystem that made standardization possible, the critical features that drove adoption (column-level lineage, streaming support, unified facets), and where OpenLineage stands today - five years since its initial release. Most importantly, we'll explore why this matters now: as AI agents increasingly make decisions about data - where to read from, what to trust, how fresh it is - they need a shared understanding of data context. Lineage metadata is the knowledge graph that transforms agents from black boxes into informed decision-makers. The talk covers the standards perspective, the pragmatic integration challenges, and a forward-looking vision for how great metadata enables intelligent data systems.