jjj-n commented on issue #12117: URL: https://github.com/apache/seatunnel/issues/12117#issuecomment-5611871496
I'd like to work on this issue. Before making a substantial executor change, could a maintainer confirm the implementation scope below? I checked the current `dev` source at `a9cda80`. In addition to the `JobMaster.run()` lifetime wait and pipeline-end callbacks described here, the shared executor also runs the permanent pending-job scheduler, the restore fan-out/join, and savepoint waits. Pipeline completion waits for checkpoint work, and checkpoint triggering itself contains nested asynchronous submission followed by a synchronous wait. This is source analysis; I have not reproduced a runtime deadlock. My proposed approach is to remove the job-lifetime wait and explicitly preserve completion ownership: connector-JAR cleanup, running-master removal, exceptional completion, and master step-down must retain correct semantics. I would isolate admission from lifecycle/control progress and keep the permanent scheduler out of the admission pool. Before bounding any shared lifecycle executor, I would audit and resolve its checkpoint/restore dependencies rather than just move the blocking work to another pool. The regression coverage would deliberately saturate admission while checking completion, cancellation, savepoints/checkpoints, and recovery. It would also cover submission overload, many restored jobs, and identical executor settings after master reactivation. Only after those checks would I introduce a finite admission default with documented queue/rejection behavior and compatibility guidance, retaining the existing configuration keys. Does this scope look appropriate, and would you prefer the non-blocking lifecycle work and the bounded admission default in separate PRs? I will keep the independent checkpoint-trigger scheduling change in #12165 out of scope. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
