uranusjr commented on PR #34337:
URL: https://github.com/apache/airflow/pull/34337#issuecomment-1721584961

   I think the fix to this would be we should check whether an upstream is a 
dependency used for task expansion. If it is, we go through the normal 
“relevant map index” check; if not, we need to depend on its entirety i.e. 
`_get_relevant_upstream_map_indexes` should return None.
   
   This change seems to work for me:
   
   ```python
   def _iter_expansion_dependencies() -> Iterator[str]:
       from airflow.models.mappedoperator import MappedOperator
   
       if isinstance(ti.task, MappedOperator):
           for op in ti.task.iter_mapped_dependencies():
               yield op.task_id
       for tg in ti.task.task_group.iter_mapped_task_groups():
           for op in tg.iter_mapped_dependencies():
               yield op.task_id
   
   def _get_relevant_upstream_map_indexes(upstream_id: str) -> int | range | 
None:
       if upstream_id not in set(_iter_expansion_dependencies()):
           return None
       # Same as before...
   ```
   
   This is obviously very unoptimised and I want to add a better abstraction 
for this. But @ephraimbuddy maybe you can help check if this can fix the actual 
DAG in the reported issue.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to