SEZ9 commented on issue #12558: URL: https://github.com/apache/seatunnel/issues/12558#issuecomment-5903493787
Thanks for the detailed analysis. Could you confirm the SeaTunnel version/commit you observed this on, and whether you can reproduce the second symptom (checkpointCoordinatorStatus == null after master switch for a freshly started job) with a simple test or log snippet? That would help confirm the regression from #10687. The proposed fix sounds reasonable: gate persistence on the job key in runningJobStateIMap rather than the checkpoint_state key, and wrap both updateStatus's check-and-write and the cleanup's removal of job key + checkpoint_state_* keys in the same per-job IMap lock with a bounded tryLock. Would you be willing to open a PR? A unit test that races a late updateStatus callback against cleanup and asserts no orphaned checkpoint_state_* key would be very valuable. Note for reviewers: since #10687 was itself a race-guard fix, please double-check that gating on the job key still covers the original node-failure cleanup scenario (a cleaned job has no job key, so late callbacks should still be dropped), and that the bounded lock wait cannot stall cleanup indefinitely if a coordinator thread is stuck. <!-- streview-comment:1429 --> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
