Hi all, Various discussion in the mailing list & JIRA tickets [2] had been brought up in the past regarding the windowing operation performance. As we experiment internally with some of our extreme use cases, we found out that using a slice-based implementation can optimize Flink's windowing mechanism and provide a better performance in most cases.
We've put together a preliminary enhancement and performance optimization plan [1] for the current windowing operation in Flink. This is largely inspired by stream slicing research shared in recent Flink Forward conference [3] by Philip and Jonas, and the discussion in the main JIRA ticket [2]. The initial design and POC implementations consider optimizing the performance for the category of overlapping windows as well as allowing chaining of cascade window operators. It will be great to hear the feedbacks and suggestions from the community. Please kindly share your comments and suggestions. Thanks, Rong [1] https://docs.google.com/document/d/1ziVsuW_HQnvJr_4a9yKwx_LEnhVkdlde2Z5l6sx5HlY/edit?usp=sharing [2] https://issues.apache.org/jira/browse/FLINK-7001 [3] https://data-artisans.com/flink-forward-berlin/resources/efficient-window-aggregation-with-stream-slicing
