With Async checkpointing the checkpoint callback in CheckpointPoint
listener is called for a previous window, that is,
beginWindow (x) -> endWindow(x) -> checkpointed (x-1 )

This feature was newly introduced. With synchronous checkpointing, the
behavior was always
beginWindow(x) -> endWindow(x) -> checkpointed (x)

A lot of operators were written before asynchronous checkpointing was
introduced and few of them can rely on the sequencing guaranteed by
synchronous checkpointing.

So why was Async Checkpointed made default?

With how Async checkpoint is today, the complexity to handle transient
state in checkpointed callback falls on every operator. For eg, lets say
earlier I had a transient map which I cleared every time the checkpointed
was called, with async checkpointing this simple task will be a lot more
complicated.

I think Async checkpointing broke the semantics of operator callbacks and
should NOT be the default.

Reply via email to