JeremyXin commented on issue #12119: URL: https://github.com/apache/seatunnel/issues/12119#issuecomment-6399222065
@DanielLeens I’d like to work on this issue. Here is an initial design for discussion: 1. Persist a lightweight acceptance record when a job is submitted, including its job ID, immutable configuration/artifact identity, enqueue order, and master epoch. Keep queued jobs visible as PENDING. 2. Separate resource-demand derivation from full JobMaster initialization. Derive and persist the demand needed for admission, then release planning objects while the job waits. Construct the execution plan and checkpoint state when resources become available. 3. Before starting the job, verify that the reconstructed plan matches the accepted configuration and resource demand. Treat a mismatch or initialization error as a visible job failure. 4. Make cancellation and master recovery operate on the durable record. Use its version/epoch to prevent a cancelled job from being initialized later or a job from being admitted twice. I would preserve existing JobStatus values and test submission error timing, cancel-before-admission, failover during admission, and memory use with a backlog under WAIT. I see that #12201 addresses queue ordering and lazy initialization during recovery. I’d build on that work while keeping this issue focused on the broader submission and admission lifecycle. I’m happy to take this on. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
