[GitHub] flink pull request #5790: [FLINK-9107][docs] document timer coalescing for P...

2018-04-04 Thread asfgit 
Github user asfgit closed the pull request at:

https://github.com/apache/flink/pull/5790


---

[GitHub] flink pull request #5790: [FLINK-9107][docs] document timer coalescing for P...

2018-04-03 Thread aljoscha 
Github user aljoscha commented on a diff in the pull request:

https://github.com/apache/flink/pull/5790#discussion_r178754487
  
--- Diff: docs/dev/stream/operators/process_function.md ---
@@ -269,4 +269,38 @@ override def onTimer(timestamp: Long, ctx: 
OnTimerContext, out: Collector[OUT]):
 }
 {% endhighlight %}
 
-
\ No newline at end of file
+
+
+## Optimisations
+
+### Timer Coalescing
+
+Every timer registered at the `TimerService` via 
`ctx.timerService().registerEventTimeTimer()` will
+be stored on heap and enqueued for execution. There is, however, a maximum 
of one timer per key and
+timestamp at a millisecond resolution and thus, in the worst case, every 
key may have a timer for
+each upcoming millisecond. Even if you do not do any processing for 
outdated timers in `onTimer`
+(as above), this may put a significant burden on the Flink runtime.
+
+Since there is only one timer per key and timestamp, however, you may 
coalesc timers by reducing the
--- End diff --

typo: coalesc -> coalesce


---

[GitHub] flink pull request #5790: [FLINK-9107][docs] document timer coalescing for P...

2018-03-29 Thread NicoK 
GitHub user NicoK opened a pull request:

https://github.com/apache/flink/pull/5790

[FLINK-9107][docs] document timer coalescing for ProcessFunction

## What is the purpose of the change

In a ProcessFunction, registering timers for each event via 
`ctx.timerService().registerEventTimeTimer()` using timestamps like 
`ctx.timestamp() + timeout` will get a millisecond accuracy and may thus create 
one timer per millisecond which may lead to some overhead in the `TimerService`.

This problem can be mitigated by using timer coalescing if the desired 
accuracy of the timer can be larger than 1ms. A timer firing at full seconds 
only, for example, can be realised like this:

```
long coalescedTime = ((ctx.timestamp() + timeout) / 1000) * 1000;
ctx.timerService().registerEventTimeTimer(coalescedTime);
```

As a result, only a single timer may exist for every second since we do not 
add timers for timestamps that are already there.

Please note that this PR includes #5788 and should also be merged into 1.3 
and 1.4 docs to which it applies as well.

## Brief change log

- document timer coalescing for `ProcessFunction`


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/NicoK/flink flink-9107

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/flink/pull/5790.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #5790


commit b6258697aabb8688aac19b5b228ddf8926e518f3
Author: Nico Kruber 
Date:   2018-03-29T13:55:47Z

[FLINK-9110][docs] fix local bundler installation

commit 540485a5620f6dcdd98e751a086076fb80997f65
Author: Nico Kruber 
Date:   2018-03-29T13:56:23Z

[hotfix][docs] remove duplicate bundle installation

commit f8274cf3ba06af19a4ac31e784803723e449336a
Author: Nico Kruber 
Date:   2018-03-29T14:20:00Z

[FLINK-9107][docs] document timer coalescing for ProcessFunction




---