SameerMesiah97 opened a new pull request, #59561:
URL: https://github.com/apache/airflow/pull/59561

   **Description**
   
   Ensure that tasks within a `MappedTaskGroup` correctly inherit downstream 
dependencies from the mapping source, even when they are not explicitly defined 
as consumers of the mapping input.
   
   This change ensures implicit dependencies within a mapped TaskGroup are 
consistently reflected at the task level.
   
   **Rationale**
   
   Previously, only tasks within a `MappedTaskGroup` that explicitly consumed 
the mapping input were registered as downstream dependencies of the mapping 
source.
   
   For example, in the task group definition below, the mapped task created by 
calling the mapping function explicitly consumes the mapping function passed 
into the argument `y`:
   
   `task_group_task = process_something(x, y)`
   
   As a result, this task was correctly registered as downstream of the mapping 
source.
   
   However, this was not the case for other tasks within the same 
MappedTaskGroup (but not immediately downstream of the mapping source) that did 
not directly consume the mapping input:
   
   `task_group_end = EmptyOperator (task_id ='end')`
   
   These relied solely on task-to-task relationships within the group:
   
   'task_group_task >> task_group_end `
   
   Even though the entire task group was downstream of the mapping source:
   
   `dag_level_start >> the_list >> mapped_group >> dag_level_end`
   
   Tasks such `task_group_end ` did not inherit an implicit task-level 
dependency on the mapping source. Their dependency on the mapping input was 
only represented at the group level.
   
   This could cause implicit dependencies to be skipped during DAG evaluation 
under certain trigger rules (such as `NONE_FAILED_MIN_ONE_SUCCESS`), 
particularly when multiple parallel task streams within the `MappedTaskGroup 
`converge on a single downstream task outside the group such as `dag_level_end`
   
   As a result, DAG runs could be incorrectly marked as failed or skipped 
despite valid upstream execution.
   
   Please refer to issue #59289 for more context. This PR was opened in 
response to that. The author of the issue reported the bug in Airflow 3.0.6 but 
I can confirm that the same issue is present in Airflow 3.1.5 after reproducing 
the bug. 
   
   **Tests**
   
   Added a test covering `MappedTaskGroup` behavior to ensure all tasks within 
the group are registered as downstream dependencies of the mapping source, 
including tasks that do not explicitly consume the mapping input.
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to