CryoThrust commented on issue #12558: URL: https://github.com/apache/seatunnel/issues/12558#issuecomment-5902375265
I traced the current `dev` paths, and the missing first status write and the `get`/`set` cleanup race both look valid. Two boundaries seem worth pinning before implementing the per-job lock: 1. `JobMaster#createJobCleanupRecord` explicitly includes the checkpoint and ready-to-close keys, but `CoordinatorService#createTerminalZombieCleanupRecord` snapshots only keys currently in the IMap. In the terminal-zombie path, a status key created after that snapshot but before cleanup would not be in `record.getStateKeys()`. If that path shares the lock, the key enumeration needs to be inside the same critical section as removal (or the known coordinator keys need explicit removal). 2. Savepoint-start resubmission can reuse the same job ID and pipeline IDs: `cleanupPendingJobStateForRestore` removes the old keys, then a new `JobMaster` initializes. A late callback from the *old* coordinator could acquire the lock after the new job key exists and write its old status to the new round's `checkpoint_state_<job>_<pipeline>` key. Checking only that `runningJobStateIMap.get(jobId)` is non-null serializes the write but does not identify the execution round. Could the update be guarded by an owner/generation token (for example the initialization timestamp), with a test for an old callback arriving after same-ID resubmission? Adjacent path to consider: `updateReadyToCloseStartingTask` also writes a cleanup-owned key via `compute` without a job-liveness gate. It may need the same invariant, even if handled in a follow-up PR. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
