uranusjr commented on issue #30333: URL: https://github.com/apache/airflow/issues/30333#issuecomment-1650399474
OK, so I went on a debug adventure, and here’s my conclusion. The analysis in https://github.com/apache/airflow/pull/32701#issuecomment-1647371922 is spot on, but the problem is deeper. Currently, we first check whether a task has all its dependencies satisfied, and only expand it if so: https://github.com/apache/airflow/blob/9f3af9c862faec73cd477e8398b7607d2d045b37/airflow/models/dagrun.py#L853-L869 (Notice how `are_dependencies_met` must satisfy to reach `_expand_mapped_task_if_needed`) Whether a trigger rule dependency is satisfied, however, is not known before expansion, since we only want to consider “relevant” upstreams. This is why #32802 doesn’t work. I actually don’t think #32701 completely fixes the issue either—if I understand the code correctly, it will fail when we add nested task mapping (i.e. a mapped task inside a mapped task group). But I guess that’s a problem we can address when (if) we ever reach there. I’ll slightly tweak #32701 to enhance the test case and add some comments for later viewers and approve it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
