Martijn Visser created FLINK-41073:
--------------------------------------
Summary: Checkpoint acknowledgement that fails shared state
registration is dropped and the checkpoint waits for its timeout
Key: FLINK-41073
URL: https://issues.apache.org/jira/browse/FLINK-41073
Project: Flink
Issue Type: Improvement
Components: Runtime / Coordination
Affects Versions: 2.4.0
Reporter: Martijn Visser
In CheckpointCoordinator#receiveAcknowledgeMessage the subtask state registers
its shared handles before the task is acknowledged. When registerSharedStates
throws, as it did in FLINK-38574,
ExecutionGraphHandler#processCheckpointCoordinatorMessage only logs the
exception and the acknowledgement is lost. The pending checkpoint then waits
for its full timeout (ten minutes by default) before it fails, the handles
registered up to the failing one stay registered under a checkpoint that never
completes, and the handles after it are neither registered nor discarded.
FLINK-40663 shows what that looks like, a 600 second hang per occurrence.
We could consider failing the pending checkpoint right away with its own
CheckpointFailureReason and discard the state of that acknowledgement, so the
job sees the failure within seconds and nothing leaks.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
