Hi All, I'm currently using a tumbling window of 5 seconds using TumblingTimeWindow but due to change in requirements I would not have to window every incoming data. With that said I'm planning to use process function to achieve this selective windowing.
I looked at the example provided in the documentation and I'm not clear on how I can implement the windowing. https://ci.apache.org/projects/flink/flink-docs-stable/dev/stream/operators/process_function.html Basically what I want is keep collecting data until it reaches 5 seconds from the time the first data came in for the key and then forward it to the next operator. I will be using ListState to add the entries and then register a timer when the list is empty. When the timer runs then collect all entries and forward it, also remove entries from the list. Do you guys think this will suffice or anything else has to be done? Also I will have about 1M keys, then would there be any performance impact in creating these many timers? I believe the timers are automatically removed after they are fired or should I do anything extra to remove these timers? Thanks
