Hey all,
I'm exploring the generalized watermark functionality in the DS V2 API.
This feature looks very useful to me, I have a few questions around it
though:
1) The examples shown in the FLIP [1] use instance variables to
exchange state between processRecord() and onWatermark() ("use variable
instead of state for simplicity"). How can I work with persistent state
instead? In particular, I don't see how I can access the Flink state
manager from within onWatermark().
2) The FLIP is missing an image in the "Combine Watermarks" section. Could
this be added?
3) Do watermark combiners only work to combine watermarks of multiple
instances of one and the same process function, or can they also be used to
combine watermarks from two separate streams processed
by TwoInputNonBroadcastStreamProcessFunction? For my use case, I'd like to
emit the same watermark type within the sources of two streams and then
forward only the minimum watermark from a TINBSPF which joins these two
streams. Is this possible?
Thanks a lot for any insights,
--Gunnar
[1]
https://cwiki.apache.org/confluence/display/FLINK/FLIP-467%3A+Introduce+Generalized+Watermarks
