The GitHub Actions job "Nightly (beta)" on flink.git/release-2.1 has failed.
Run started by GitHub user github-actions[bot] (triggered by 
github-actions[bot]).

Head commit for run:
7025a3f1ec19b8ca9b537d2dce1805a2624c5953 / Maximilian Michels <[email protected]>
[FLINK-38370] Ensure CommitterOperator commits all pending committables in 
batch mode (#27015)

In #26433, we removed the EOI marker in the form of Long.MAX_VALUE as the 
checkpoint id. Since
streaming pipelines can continue to checkpoint even after their respective 
operators have been shut
down, it is not safe to use a constant as this can lead to duplicate commits.

However, in batch pipelines we only have one commit on job shutdown. Using any 
checkpoint id should
suffice in this scenario. Any pending committables should be processed by the 
ComitterOperator when
the operator shuts down. No further checkpoints will take place.

There are various connectors which rely on this behavior. I don't see any 
drawbacks from keeping
this behavior for batch pipelines.

Report URL: https://github.com/apache/flink/actions/runs/17933996520

With regards,
GitHub Actions via GitBox

Reply via email to