sandeep-paluru opened a new pull request, #75317: URL: https://github.com/apache/airflow/pull/75317
A deferred task was given a fresh `execution_timeout` every time it resumed. The task runner always used the full configured duration, so time already spent, including the deferral, did not count and the task could run longer than its timeout. When a task resumes (`next_method` is set), the runner now keeps only the time still left: `execution_timeout` minus the time since the task's original `start_date`. That remaining time is used both for the in-process timeout and for the `SetExecutionTimeout` message. If the timeout has already elapsed, the task raises `AirflowTaskTimeout` before the callable runs. A task that is not resuming still gets the full timeout. The original `start_date` is available on resume: the server sends it on `TIRunContext` only when `next_method` is set, and the supervisor copies it onto the runtime task. This does not change how long a trigger may run while the task is still deferred. Tested in `task-sdk/tests/task_sdk/execution_time/test_task_runner.py` with `time_machine`. A resumed task gets the reduced timeout, an already expired resume fails as a timeout, and a non-deferred run keeps the full timeout even when `start_date` is hours in the past. prek pre-commit checks passed, including Task SDK mypy, and the manual prek stage passed. Computing the timeout when one is set takes about 0.14 microseconds instead of 0.07. Resuming, which also subtracts the elapsed time, takes about 0.5 microseconds. There is an older open attempt, #69734, which reloads the serialized Dag on every deferral. This one only uses the start_date the server already sends on resume. closes: #73475 --- ##### Was generative AI tooling used to co-author this PR? - [x] Yes (AI coding assistant) Generated-by: AI coding assistant following [the guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions) --- * Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#pull-request-guidelines)** for more information. Note: commit author/co-author name and email in commits become permanently public when merged. * For fundamental code changes, an Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvement+Proposals)) is needed. * When adding dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x). * For significant user-facing changes create newsfragment: `{pr_number}.significant.rst`, in [airflow-core/newsfragments](https://github.com/apache/airflow/tree/main/airflow-core/newsfragments). You can add this file in a follow-up commit after the PR is created so you know the PR number. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
