SEZ9 commented on issue #11679: URL: https://github.com/apache/seatunnel/issues/11679#issuecomment-5215602984
Thanks for the detailed report and for confirming this reproduces on pristine `dev` independently of #11648. The key symptom seems to be that after `SubPlan.handleCheckpointError()` moves the pipeline to CANCELING and the pipeline is repeatedly resubmitted/failed, the job never transitions to a terminal state (FAILED/CANCELED) that would complete the client's future — could you share the coordinator logs around the pipeline restore/retry loop so we can see whether the restore count is being exhausted or the retry loop never terminates? Two areas worth investigating: (1) why repeated pipeline resubmission after a non-retryable exception like `NoPrivilegeException` does not eventually fail the job (exceptions from connector authorization checks should arguably be treated as non-recoverable), and (2) why the coordinator's status dump shows 0 jobs in every state while the job future held by `ClientJobProxy.waitForJobComplete()` is never completed — that suggests the job state was dropped from IMap without notifying waiters. As a mitigation, the client side could add a safeguard: if the cluster no longer reports the submitted job id in any state, `waitForJobComplete()` should fail fast with a clear error instead of blocking indefinitely. A PR addressing either the retry-loop termination or the client-side detection would be welcome — marking as help wanted. <!-- streview-comment:67 --> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
