Martijn Visser created FLINK-41415:
--------------------------------------
Summary: CommonTestUtils.getLatestCompletedCheckpointPath can
return a subsumed checkpoint
Key: FLINK-41415
URL: https://issues.apache.org/jira/browse/FLINK-41415
Project: Flink
Issue Type: Bug
Components: Runtime / Coordination, Test Infrastructure
Affects Versions: 2.5.0
Reporter: Martijn Visser
With FLINK-30644 fixed on release-1.20 by retaining more checkpoints in
ChangelogCompatibilityITCase, the underlying problem in
{{CommonTestUtils.getLatestCompletedCheckpointPath}} remains on all branches.
It reads the checkpoint stats of the archived execution graph, but
{{CheckpointStatsTracker#createSnapshot}} returns the previous snapshot when
its lock is held by a concurrent update. After a cancel, the returned path can
then point to a checkpoint that a newer one has already subsumed and deleted,
and the restore fails with a FileNotFoundException. This happened in
https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=79931,
where chk-2 completed before the cancel and the restore still used chk-1.
Most callers keep the default of one retained checkpoint. We should make the
utility return the latest checkpoint that completed before the job reached its
terminal state, instead of relying on each test to raise
MAX_RETAINED_CHECKPOINTS.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
