[GitHub] flink pull request: Timer coalescing across keys and cleanup of un...

[aljoscha]

Github user aljoscha commented on the pull request:

https://github.com/apache/flink/pull/1944#issuecomment-216535365
  
I merged it. Thanks a lot for your work! 😃 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] flink pull request: Timer coalescing across keys and cleanup of un...

[aljoscha]

Github user aljoscha commented on a diff in the pull request:

https://github.com/apache/flink/pull/1944#discussion_r61567670
  
--- Diff: 
flink-streaming-java/src/main/java/org/apache/flink/streaming/runtime/operators/windowing/WindowOperator.java
 ---
@@ -542,14 +561,24 @@ public void registerEventTimeTimer(long time) {
// immediately schedule a trigger, so that we 
don't wait for the next
// watermark update to fire the watermark 
trigger

getRuntimeContext().registerTimer(System.currentTimeMillis(), 
WindowOperator.this);
+   //No need to put it in 
processingTimeTimerFutures as this timer is never removed
}
}
 
@Override
public void deleteProcessingTimeTimer(long time) {
Timer timer = new Timer<>(time, key, window);
-   if (processingTimeTimers.remove(timer)) {
-   processingTimeTimersQueue.remove(timer);
+
+   processingTimeTimers.remove(timer);
+   processingTimeTimersQueue.remove(timer);
+
+   //If there are no timers left for this timestamp, 
remove it from queue and cancel TriggerTask
+   if (processingTimeTimerTimestamps.remove(time,1) == 1) {
+   ScheduledFuture triggerTaskFuture = 
processingTimeTimerFutures.get(timer);
--- End diff --

This should be replaced by `processingTimeTimerFutures.remove(timer.time)`. 
The key in `processingTimeTimerFutures` is a `long` timestamp. This way, we 
also don't need to call remove inside the if and only have one lookup.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] flink pull request: Timer coalescing across keys and cleanup of un...

[knaufk]

Github user knaufk closed the pull request at:

https://github.com/apache/flink/pull/1944


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] flink pull request: Timer coalescing across keys and cleanup of un...

[knaufk]

GitHub user knaufk reopened a pull request:

https://github.com/apache/flink/pull/1944

Timer coalescing across keys and cleanup of unused trigger tasks

Thanks for contributing to Apache Flink. Before you open your pull request, 
please take the following check list into consideration.
If your changes take all of the items into account, feel free to open your 
pull request. For more information and/or questions please refer to the [How To 
Contribute guide](http://flink.apache.org/how-to-contribute.html).
In addition to going through the list, please provide a meaningful 
description of your changes.

- [ ] General
  - The pull request references the related JIRA issue
  - The pull request addresses only one issue
  - Each commit in the PR has a meaningful commit message

- [ ] Documentation
  - Documentation has been added for new functionality
  - Old documentation affected by the pull request has been updated
  - JavaDoc for public methods has been added

- [ ] Tests & Build
  - Functionality added by the pull request is covered by tests
  - `mvn clean verify` has been executed successfully locally or a Travis 
build has passed

Per timestamp only one TriggerTask is registered at
the runtime context. When the first timer is registered a new TriggerTask
is sheduled. When no timer is registered anymore for a specific timestamp
the corresponding trigger task is canceled and hence removed.

The ScheduledFutures to cancel trigger tasks are not checkpointed. So
cleanup of trigger tasks will not work after a failure.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/knaufk/flink FLINK-3669

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/flink/pull/1944.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1944


commit e1f7a5b0c448516e7c42bfc4938755a43e3bcae0
Author: Konstantin Knauf 
Date:   2016-04-05T17:59:19Z

Timer coalescing across keys and cleanup of unused trigger tasks

Per timestamp only one TriggerTask is registered at
the runtime context. When the first timer is registered a new TriggerTask
is sheduled. When no timer is registered anymore for a specific timestamp
the corresponding trigger task is canceled and hence removed.

The ScheduledFutures to cancel trigger tasks are not checkpointed. So
cleanup of trigger tasks will not work after a failure.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] flink pull request: Timer coalescing across keys and cleanup of un...

[knaufk]

GitHub user knaufk opened a pull request:

https://github.com/apache/flink/pull/1944

Timer coalescing across keys and cleanup of unused trigger tasks

Thanks for contributing to Apache Flink. Before you open your pull request, 
please take the following check list into consideration.
If your changes take all of the items into account, feel free to open your 
pull request. For more information and/or questions please refer to the [How To 
Contribute guide](http://flink.apache.org/how-to-contribute.html).
In addition to going through the list, please provide a meaningful 
description of your changes.

- [ ] General
  - The pull request references the related JIRA issue
  - The pull request addresses only one issue
  - Each commit in the PR has a meaningful commit message

- [ ] Documentation
  - Documentation has been added for new functionality
  - Old documentation affected by the pull request has been updated
  - JavaDoc for public methods has been added

- [ ] Tests & Build
  - Functionality added by the pull request is covered by tests
  - `mvn clean verify` has been executed successfully locally or a Travis 
build has passed

Per timestamp only one TriggerTask is registered at
the runtime context. When the first timer is registered a new TriggerTask
is sheduled. When no timer is registered anymore for a specific timestamp
the corresponding trigger task is canceled and hence removed.

The ScheduledFutures to cancel trigger tasks are not checkpointed. So
cleanup of trigger tasks will not work after a failure.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/knaufk/flink FLINK-3669

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/flink/pull/1944.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1944


commit e1f7a5b0c448516e7c42bfc4938755a43e3bcae0
Author: Konstantin Knauf 
Date:   2016-04-05T17:59:19Z

Timer coalescing across keys and cleanup of unused trigger tasks

Per timestamp only one TriggerTask is registered at
the runtime context. When the first timer is registered a new TriggerTask
is sheduled. When no timer is registered anymore for a specific timestamp
the corresponding trigger task is canceled and hence removed.

The ScheduledFutures to cancel trigger tasks are not checkpointed. So
cleanup of trigger tasks will not work after a failure.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---