Hi everyone, we were testing a Flink streaming job (1.0.0), with a GlobalWindow on a KeyedStream with custom Trigger.
On each element the trigger function registers a processing time timer and deletes the currently registered processing time timer. So we are registering a lot of timers, but also deleting most of them right away. The desired functionality is, that the window is purged (and all state is set to null) after a timeout (last event for this key + timeout). The performance tests showed, that after a short time (5mins or so) all the time went to garbage collection. From the heap dumnps, we can tell that the problem were retained TriggerTasks (with reference to the TriggerContext) off all the registered processing time timers. The problems seems to be that when deleting the TriggerTasks the corresponding Callables are not removed form the queue, the deleteProcessingTimeTimer-method only removes the Timer from the set/queues of the TriggerContext itself, but not from the RuntimeContext. Is this a bug? Are we using ProcessingTimeTimers in a fundamentally wrong way? If so, is there any other way to achieve the desired functionality? We have a workaround in place now (basically just a timeout starting with the first element in window instead of the last element in the window). Cheers, Konstantin -- Konstantin Knauf * konstantin.kn...@tngtech.com * +49-174-3413182 TNG Technology Consulting GmbH, Betastr. 13a, 85774 Unterföhring Geschäftsführer: Henrik Klagges, Christoph Stock, Dr. Robert Dahlke Sitz: Unterföhring * Amtsgericht München * HRB 135082
