Re: TimeWindow overload?

2016-05-03 Thread Stephan Ewen
Just had a quick chat with Aljoscha... The first version of the aligned window code will still duplicate the elements, later versions should be able to get rid of that. On Tue, May 3, 2016 at 11:10 AM, Aljoscha Krettek wrote: > Hi, > even with the optimized operator for

Re: TimeWindow overload?

2016-05-03 Thread Aljoscha Krettek
Hi, even with the optimized operator for aligned time windows I would advice against using long sliding windows with a small slide. The system will internally create a lot of "buckets", i.e. each sliding window is treated separately and the element is put into 1,440 buckets, in your case. With a

Re: TimeWindow overload?

2016-05-03 Thread Stephan Ewen
Hi Elias! There is a feature pending that uses an optimized version for aligned time windows. In that case, elements would go into a single window pane, and the full window would be composed of all panes it spans (in the case of sliding windows). That should help a lot in those cases. The

TimeWindow overload?

2016-05-02 Thread Elias Levy
Looking over the code, I see that Flink creates a TimeWindow object each time the WindowAssigner is created. I have not yet tested this, but I am wondering if this can become problematic if you have a very long sliding window with a small slide, such as a 24 hour window with a 1 minute slide. It