Re: Slide Window Compute Optimization

+1. Yes your use case would probably fit best in the OVER aggregate use case. I actually created for myself a complimentary note for some of the complex aggregate components on top of Flink

Re: Slide Window Compute Optimization

Hi Yennie, You might want to have a look at the OVER windows of Flink's Table API or SQL [1]. An OVER window computes an aggregate (such as a count) for each incoming record over a range of previous events. For example the query: SELECT ip, successful, COUNT(*) OVER (PARTITION BY ip, successful

Re: Slide Window Compute Optimization

Hi Kostas and Rong， Thank you for your reply. As both of you ask for more info about my use case, I now reply in unison. My case is used for counting the number of successful login and failures within one hour， keyBy other login related attributes （e.g. ip, device, login type ...）.

Re: Slide Window Compute Optimization

Hi Kostas and Rong， Thank you for your reply. As both of you ask for more info about my use case, I now reply in unison. My case is used for counting the number of successful login and failures within one hour， keyBy other login related attributes （e.g. ip, device, login type ...）.

Re: Slide Window Compute Optimization

Hi Yennie, AFAIK, the sliding window will in fact duplicate elements into multiple different streams. There's a discussion thread regarding this [1]. We are looking into some performance improvement, can you provide some more info regarding your use case? -- Rong [1]

Re: Slide Window Compute Optimization

Hi, You are correct that with sliding windows you will have 3600 “open windows” at any point. Could you describe a bit more what you want to do? If you simply want to have an update of something like a counter every second, then you can implement your own logic with a ProcessFunction that

Slide Window Compute Optimization

Hi， I want to use slide windows of 1 hour window size and 1 second step size. I found that once a element arrives, it will be processed in 3600 windows serially through one thread. It takes serveral seconds to finish one element processing，much more than my expection. Do I have any way to