sgoel2be24-cyber opened a new pull request, #73323: URL: https://github.com/apache/airflow/pull/73323
`EmrServerlessDeleteApplicationOperator(deferrable=True)` never deleted the application. The inherited stop step defers, so the code after `super().execute()` that calls `DeleteApplication` was never reached. The task then resumed in the delete operator's `execute_complete` with the *stop* trigger's success event, logged "deleted successfully" and went green, leaving the application in place (only stopped). The non-deferrable path was unaffected. The stop operator now defers to an overridable `stop_complete_method_name` (default `execute_complete`, so its behaviour is unchanged) in both places it waits on `EmrServerlessStopApplicationTrigger`, including the `force_stop` path. The delete operator sets it to `delete_stopped_application`, which validates the stop event, then calls `DeleteApplication` and defers on `EmrServerlessDeleteApplicationTrigger` as before. Deferrable and non-deferrable modes now share the same delete code. The existing `test_delete_application_deferrable` only asserted that `TaskDeferred` was raised, which is why it passed with the bug. It now also checks the trigger, the resume method and that delete isn't called before the stop completes. New tests cover the resume-then-delete step, the `force_stop` chain and a failed stop event. All of them fail on `main`. Note: task instances already deferred on the stop trigger by an older provider version during an upgrade will still resume in `execute_complete`, since the stop and delete trigger events are identical and can't be told apart. Tested locally: - `pytest` on the amazon provider's EMR operator and trigger tests: 166 passed - The reproduction script from the issue now shows stop → defer → delete → defer → complete - `prek` pre-commit stage passes, apart from `check-template-fields-valid`, which needs Docker and wasn't run locally (no template fields changed); `mypy` on `emr.py` passes closes: #72123 --- ##### Was generative AI tooling used to co-author this PR? - [X] Yes — Claude Code (Opus 5) Generated-by: Claude Code (Opus 5) following [the guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
