zhangshenghang opened a new issue, #13324:
URL: https://github.com/apache/seatunnel/issues/13324
### What happened
Terminal coordinator status writes to the IMap are best-effort (failures
swallowed with a warning), while correctness decisions on master failover trust
exactly that IMap entry (dev @ 89f91a89af):
1. `CheckpointCoordinator.updateStatus`
(`CheckpointCoordinator.java:1547-1578`) retries the
`runningJobStateIMap.set(...)` write, and when retries are exhausted it only
logs `"Set {} state {} to IMap failed, skip do it"` and continues — the
terminal status (e.g. FINISHED at `completePendingCheckpoint`,
`CheckpointCoordinator.java:1469`) is never persisted.
2. `isNoErrorCompleted` (`CheckpointCoordinator.java:1510-1520`) decides
"this pipeline already completed" purely from the IMap status:
`latestCompletedCheckpoint.getCheckpointType().isFinalCheckpoint() && (FINISHED
|| SUSPEND equals status)`.
If the FINISHED write was swallowed, a later master failover concludes the
pipeline never completed → `restoreCoordinator(alreadyStarted=true)`
re-triggers the final COMPLETED_POINT checkpoint and re-runs completion work
for a pipeline whose tasks already finished (duplicate final checkpoint;
re-execution side effects). Symmetrically, a swallowed FAILED write can make a
failed pipeline look recoverable.
### Trigger
Transient IMap unavailability exactly when a coordinator terminal status is
written (job finishing during cluster degradation).
### Impact
Failover can re-run or re-finish already-completed pipelines; terminal
pipeline states are not durable.
### Suggested fix
Terminal status writes (FINISHED/SUSPEND/FAILED/CANCELED) must not be
best-effort: retry to success with backoff, or fail loudly so the pipeline
end-state computation knows the state could not be persisted, instead of
swallowing and letting `isNoErrorCompleted` read stale data.
### SeaTunnel Version
dev (89f91a89af), Zeta engine
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]