Stream Processing Beyond Streaming Data

Stream processing is becoming something like a ""grand unifying paradigm"" for data processing. Outgrowing its original space of real-time data processing, stream processing is becoming a technology that offers new approaches to data processing (including batch processing), real-time applications, and even distributed transactions. We will take a look at these developments from the view of Apache Flink and present some of the major efforts in the Flink community to build a unified stream processor data processing and data-driven applications. Flink already powers many of the world's most demanding stream processing applications. We present the approach of Flink's next generation streaming runtime that also offers a state-of-the-art batch processing experience and performance. A new Machine Learning library, built on top of a unique new API supports many algorithms to train dynamically across static and real-time data. Finally, we look at new building blocks stream processing offers for data-driven applications that open a new direction to solve application consistency. With use cases from different users, we show how companies apply this broader streaming paradigm in practice.