Hi everyone,
me again :) Let's say you have a stream, and for every window and key
you compute some aggregate value, like this:
DataStream.keyBy(..)
.timeWindow(..)
.apply(...)
Now I want to get the maximum aggregate value for every window over the
keys. This feels like a pretty natural use case. How can I achieve this
with Flink in the most compact way?
The options I thought of so far are:
* Use an allTimeWindow, obviously. Drawback is, that the WindowFunction
would not be distributed by keys anymore.
* use a windowAll after the WindowFunction to create windows of the
aggregates, which originated from the same timeWindow. This could be
done either with a TimeWindow or with a GlobalWindow with DeltaTrigger.
Drawback: Seems unnecessarily complicated and doubles the latency (at
least in my naive implementation ;)).
* Of course, you could also just keyBy the start time of the window
after the WindowFunction, but then you get more than one event for each
window.
Is there some easy way I am missing? If not, is there a technical
reasons, why such an "reduceByKeyAndWindow"-operator is not available in
Flink?
Cheers,
Konstantin
