Thanks for starting the conversation. The two approaches both look good
to me. Probably we want to start with approach #1 for all Runners to be
able to support delaying bundles. Flink supports cycles and thus
approach #2 would also be applicable and could be used to implement
dynamic splitting.
-Max
On 05.10.20 23:13, Luke Cwik wrote:
Thanks Boyuan, I left a few comments.
On Mon, Oct 5, 2020 at 11:12 AM Boyuan Zhang <[email protected]
<mailto:[email protected]>> wrote:
Hi team,
I'm looking at adding self-checkpoint support to portable Flink
runner(BEAM-10940
<https://issues.apache.org/jira/browse/BEAM-10940>) for both batch
and streaming. I summarized the problem that we want to solve and
proposed 2 potential approaches in this doc
<https://docs.google.com/document/d/1372B7HYxtcUYjZOnOM7OBTfSJ4CyFg_gaPD_NUxWClo/edit?usp=sharing>.
I want to collect feedback on which approach is preferred and
anything that I have not taken into consideration yet but I should.
Many thanks to all your help!
Boyuan
