jerryshao commented on issue #10271: URL: https://github.com/apache/gravitino/issues/10271#issuecomment-6678577939
I checked current `main` and this issue no longer reproduces, though it was fixed by reordering rather than by compensation. `JobManager.runJob` used to submit the job to the executor first and persist the `JobEntity` afterwards, so a failing `entityStore.put(...)` left an untracked execution running. Since #14278 (for #13180), the job is registered before it is submitted: - `runJob` first persists the `JobEntity` with a placeholder execution id, then submits to the executor (`JobManager.registerAndSubmit`). - If `put` fails, nothing has been submitted yet. The staging directory is deleted and the failure is rethrown, so there is nothing to cancel and no orphaned execution. - If the submission is not recorded in time, the status pull settles the placeholder job. - Ownership is set before the submission, and a failure there cleans up the unsubmitted job. So the "submit succeeded but persist failed" window described here no longer exists. I suggest closing this issue as completed, along with #10814, since it implements the cancel-on-failure approach for the old ordering. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
