raunak2007 commented on issue #55187:
URL: https://github.com/apache/airflow/issues/55187#issuecomment-6181976465
Still reproduces on 3.3.2 with a minimal DAG: one `@task.branch` (`branch1`)
into `problematic` and `branch2`, `branch2` into `deadend` and `dummy`, plus
`dummy >> problematic`, with `problematic` on `NONE_FAILED_MIN_ONE_SUCCESS`.
When `branch1` goes to `branch2`, `problematic` runs instead of being skipped.
Here is what happens in the code. `problematic` ends up with two upstreams:
`branch1` (success) and `dummy` (skipped). In
`TriggerRuleDep._evaluate_trigger_rule`, the `NONE_FAILED_MIN_ONE_SUCCESS`
branch only marks the task skipped when every upstream is skipped:
```python
elif trigger_rule == TR.NONE_FAILED_MIN_ONE_SUCCESS:
...
elif skipped == upstream:
new_state = TaskInstanceState.SKIPPED
```
Here `upstream = 2`, `skipped = 1`, `success = 1`, so that condition is
false and the task runs.
The harder part: `_UpstreamTIStates` only carries aggregate counts
(success/skipped/failed/...), so at this point there is no way to tell that the
one success is a branch that branched away from this task. `branch1` never
skipped `problematic`, because `problematic` is reachable from the followed
branch via `branch2 -> dummy -> problematic` and the branch operator protects
that join from being skipped (`branch2` does skip `dummy`). So `dummy` is
skipped, `problematic` has no live path, but `branch1` still shows as a success
and nothing catches it.
This looks less like a pure trigger-rule bug and more like a branch-skip vs
join interaction. Before I open a PR, where would you prefer the fix: inside
the `NONE_FAILED_MIN_ONE_SUCCESS` handling in `TriggerRuleDep`, or in how
branch/skip propagates to a join whose other paths are all skipped? Happy to
implement whichever direction you want and add a `TriggerRuleDep` test for it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
