[
https://issues.apache.org/jira/browse/FLINK-3669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15219845#comment-15219845
]
Aljoscha Krettek commented on FLINK-3669:
-----------------------------------------
Yes, you're right that's still a problem. The issue with removing the
TriggerTasks is, that a) right now, the WindowOperator does not have a handle
to the ScheduledFuture and b) if we combine the triggers for a certain
timestamp into one trigger we cannot simply delete that TriggerTask until we
know that all timers in the WindowOperator for that timestamp have been
removed. By now, I see basically three solutions for our problem:
1. Create one TriggerTask per {{registerProcessingTimeTimer()}} call of the
user (as is the case now), keep the {{ScheduledFuture}}, remove the timer when
the user calls {{deleteProcessingTimeTimer()}}
2. Coalesce multiple {{registerProcessingTimeTimer()}} calls to only create one
TriggerTask to reduce the burden on the {{ScheduledThreadPoolExecutor}}, keep
track of how many user triggers we have for a timestamp, remove using the
{{ScheduledFuture}} once the number drops to zero
3. Change the timer service to call {{trigger()}} of the {{WindowOperator}} on
a fixed interval, say 10 ms, the logic in {{WindowOperator.trigger()}} already
internally checks which triggers should fire based on the timestamp given. This
would mean that the burden on {{ScheduledThreadPoolExecutor}} is constant does
not depend on the number of keys or number of user timers registered but it is
potentially wasteful.
(when I say user above, I mean the {{Trigger}} implementation)
I think I would go with option 3. the granularity can be user configurable, of
course. There is, of course, a tradeoff. If the user has very large windows
firing all those timers is a huge waste, but the other options require to hold
a data structure to keep track of things.
What do you think?
> WindowOperator registers a lot of timers at StreamTask
> ------------------------------------------------------
>
> Key: FLINK-3669
> URL: https://issues.apache.org/jira/browse/FLINK-3669
> Project: Flink
> Issue Type: Bug
> Components: Streaming
> Affects Versions: 1.0.1
> Reporter: Aljoscha Krettek
> Priority: Blocker
>
> Right now, the WindowOperator registers a timer at the StreamTask for every
> processing-time timer that a Trigger registers. We should combine several
> registered trigger timers to only register one low-level timer (timer
> coalescing).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
