uranusjr commented on PR #34337:
URL: https://github.com/apache/airflow/pull/34337#issuecomment-1721584961
I think the fix to this would be we should check whether an upstream is a
dependency used for task expansion. If it is, we go through the normal
“relevant map index” check; if not, we need to depend on its entirety i.e.
`_get_relevant_upstream_map_indexes` should return None.
This change seems to work for me:
```python
def _iter_expansion_dependencies() -> Iterator[str]:
from airflow.models.mappedoperator import MappedOperator
if isinstance(ti.task, MappedOperator):
for op in ti.task.iter_mapped_dependencies():
yield op.task_id
for tg in ti.task.task_group.iter_mapped_task_groups():
for op in tg.iter_mapped_dependencies():
yield op.task_id
def _get_relevant_upstream_map_indexes(upstream_id: str) -> int | range |
None:
if upstream_id not in set(_iter_expansion_dependencies()):
return None
# Same as before...
```
This is obviously very unoptimised and I want to add a better abstraction
for this. But @ephraimbuddy maybe you can help check if this can fix the actual
DAG in the reported issue.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]