OmarFarooq908 commented on PR #56399: URL: https://github.com/apache/airflow/pull/56399#issuecomment-3419126521
> Yet another discrepancy between deferrable and non-deferrable mode! > > We already have many differences in behavior between executing the operator in deferrable mode and executing it in normal mode, and your fix here adds a new one — where we won’t support cleaning up the created pod if the task is executed in deferrable mode. A better solution would be to refactor the code and check what needs to be recreated (launcher, hook, etc.) to support deleting the pod after resuming the task **if it should be deleted**. > > > if it should be deleted > > If the user set `delete_on_termination=True` (which is the default value), they expect the driver pod to be deleted when the job finishes — regardless of the execution mode of the Airflow task. > > A general note: to accept the PR as a fix for the bug, we expect a unit test that reproduces the issue without the fix/change, and that passes after applying the patch. @hussein-awala Thanks for the feedback! I've fixed the issues: - Both deferrable and non-deferrable modes now behave the same. - Pods are cleaned up properly when `delete_on_termination=True`. - `hook` and `launcher` are recreated as needed. - Added tests that reproduce the bug and confirm the fix. PR is ready for review! Let me know if anything else needs changing -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
