dzhigimont commented on PR #32701: URL: https://github.com/apache/airflow/pull/32701#issuecomment-1647371922
> Why are the upstream tasks calculated incorrectly before the task is expanded? Task expansion happens on the task, why would it affect the upstream tasks? Apologies for any confusion in my previous explanation. Upstream tasks were calculated or executed correctly. What I meant is that for tasks belonging to a mapped task group, the method `get_closest_mapped_task_group()` returns True, and for these tasks, relevant upstream tasks are getting with a corresponding index. The problem arose because `are_dependencies_met `is called before `_expand_mapped_task_if_needed` in the `_get_ready_tis()` function. As a result, upstream tasks were already expanded, but the current task with `TriggerRule.ONE_FAILED` was not. When a task has `get_closest_mapped_task_group() == True`, the algorithm in the method `_get_relevant_upstream_map_indexes()` attempts to find the corresponding index in our case returns -1 but the upstream tasks were mapped, there are no tasks with an index of -1. Since the expected number of completed and failed tasks is 0, the upstream tasks are considered done, leading to the optimization that skips tasks with `TriggerRule. ONE_FAILED` when there were 0 failed tasks upstream and all tasks were completed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
