Hi Elias! There is a feature pending that uses an optimized version for aligned time windows. In that case, elements would go into a single window pane, and the full window would be composed of all panes it spans (in the case of sliding windows). That should help a lot in those cases.
The default window mechanism does it that way, because is supports unaligned windows (where each key has a different window start and endpoint) and it supports completely custom window assigners. Greetings, Stephan On Tue, May 3, 2016 at 4:07 AM, Elias Levy <fearsome.lucid...@gmail.com> wrote: > Looking over the code, I see that Flink creates a TimeWindow object each > time the WindowAssigner is created. I have not yet tested this, but I am > wondering if this can become problematic if you have a very long sliding > window with a small slide, such as a 24 hour window with a 1 minute slide. > It seems this would create 1,440 TimeWindow objects per event. Event a low > event rates this would seem to result in an explosion of TimeWindow > objects: at 1,000 events per second, you'd be creating 1,440,000 TImeWindow > objects. After 24 hours you'd have nearly 125 billion TM objects that > would just begin to be purged. > > Does this analysis seem right? > > I suppose that means you should not use long length sliding window with > small slides. > >
