developer-rpai commented on code in PR #74567:
URL: https://github.com/apache/airflow/pull/74567#discussion_r4321908536
##########
airflow-core/src/airflow/jobs/scheduler_job_runner.py:
##########
@@ -3266,7 +3266,20 @@ def _schedule_dag_run(
default=None,
)
for task_instance in unfinished_task_instances:
- task_instance.set_state(TaskInstanceState.SKIPPED,
session=session)
+ if task_instance.state in (
+ TaskInstanceState.RUNNING,
+ TaskInstanceState.DEFERRED,
+ TaskInstanceState.UP_FOR_RESCHEDULE,
+ TaskInstanceState.AWAITING_INPUT,
+ ):
+ # The timeout interrupted a running task instance; mark it
FAILED so a
+ # mapped task with unfinished inner tasks reflects the
failure instead
+ # of appearing successful.
+ task_instance.set_state(TaskInstanceState.FAILED,
session=session)
+ elif not dag.task_dict[task_instance.task_id].is_teardown:
+ # Task instances that never started stay SKIPPED.
Teardowns are left
+ # alone so they can still run.
+ task_instance.set_state(TaskInstanceState.SKIPPED,
session=session)
Review Comment:
Thanks for looking closely. You are right that the teardown never executes
once the run has timed out; that is inherent to the timeout path and this
change does not affect it either way. On the state it ends up in: the timeout
branch in _schedule_dag_run collects all unfinished task instances of the run,
and the loop that marks them does not filter by task type, so a pending
teardown (state None) is included and is set to SKIPPED in the same pass that
fails the run. The run only reaches FAILED on a timeout through this same
branch, so there is no ordering where the run is already finished and the
teardown is visited later and left dangling. That end state is what the test
pins. If the test comment reads as claiming more than that, I am happy to
reword it or split the teardown assertion out.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
