luc-pimentel opened a new pull request, #73849: URL: https://github.com/apache/airflow/pull/73849
Raising `AirflowFailException` in `on_retry_callback` did not stop the retry. The exception means "fail without retrying", but `_run_task_state_change_callbacks` in `task-sdk/src/airflow/sdk/execution_time/task_runner.py` treated it like any other callback error: it logged "Failed to run task callback", and the retry that `run()` had already reported went ahead. Since #73253 the supervisor holds the retry report until the task process exits, and reports the last terminal state it received. So when a retry callback raises `AirflowFailException`, `finalize()` now sends `TaskState(FAILED)` in place of the pending retry, then finishes as a failed task: the remaining retry callbacks are skipped, and `on_failure_callback`, the failure listener and `email_on_failure` run with the original task exception. It also marks the runtime task instance failed, because `dag.test()` stores that state after the run. Other exceptions from retry callbacks are still logged and ignored. Earlier attempts (#64198, #66781) deferred the retry report inside `run()`, which #73253 makes unnecessary. Reproduced with a Dag of `retries=1` tasks, using `airflow dags test` on 3.3.1, 3.3.2 and main, and the scheduler with LocalExecutor on 3.3.2 and main: | Task | Before | With this change | | --- | --- | --- | | `on_retry_callback` raises `AirflowFailException` | try 2 runs | fails after try 1, then `on_failure_callback` runs | | `on_retry_callback` raises `ValueError` | retries | retries | | `execute` raises `AirflowFailException` | fails after try 1 | fails after try 1 | The two new tests in `task-sdk/tests/task_sdk/execution_time/test_task_runner.py` fail without the change. The rest of `task-sdk/tests/task_sdk/execution_time/` passes, apart from 11 tests that fail locally with and without this change (in `TestTriggerDagRunOperator`, and three `test_supervisor.py` tests that spawn subprocesses). The prek pre-commit hooks, including `mypy-task-sdk`, and the manual stage pass. closes: #60172 --- ##### Was generative AI tooling used to co-author this PR? - [X] Yes — Claude Code (Opus 5.5) Generated-by: Claude Code (Opus 5.5) following [the guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
