luc-pimentel opened a new issue, #76170:
URL: https://github.com/apache/airflow/issues/76170
### Under which category would you file this issue?
Task SDK
### Apache Airflow version
3.3.2 (same on 3.4.0rc2)
### What happened and how to reproduce it?
With `clean` inside a TaskGroup and `load` outside it, `clean >>
Label("rows") >> load` makes `load` depend on every leaf of the group, not only
on `clean`. The unlabelled edge, `load << Label("rows") << clean` and
`clean.set_downstream(load, Label("rows"))` all keep the single dependency.
```python
from airflow.sdk import DAG, Label, TaskGroup, task
with DAG("label_grouped_task", schedule=None):
with TaskGroup("transform"):
@task
def clean():
return "ok"
@task
def check():
raise RuntimeError("check fails on purpose")
clean_t = clean()
check()
@task
def load():
return "loaded"
clean_t >> Label("rows") >> load()
```
`airflow dags test label_grouped_task -f <file>` on a fresh SQLite metadata
DB ends with `load` in `upstream_failed`. With `clean_t >> load()` or
`clean_t.set_downstream(load(), Label("rows"))` instead, `load` succeeds.
At definition time, with `EmptyOperator` tasks of the same shape:
| Edge | `transform.check.downstream_task_ids` |
`transform.downstream_task_ids` | `edge_info` |
| --- | --- | --- | --- |
| `clean >> load` | `[]` | `[]` | `{}` |
| `clean >> Label("rows") >> load` | `['load']` | `['load']` |
`{'transform.downstream_join_id': {'load': {'label': 'rows'}}}` |
| `load << Label("rows") << clean` | `[]` | `[]` | `{'transform.clean':
{'load': {'label': 'rows'}}}` |
| `clean.set_downstream(load, Label("rows"))` | `[]` | `[]` |
`{'transform.clean': {'load': {'label': 'rows'}}}` |
Output is identical on 3.3.2, 3.4.0rc2, 2.10.5, 2.6.0 and 2.5.3. On 2.5.1
all four spellings give the single dependency, so this started with #29410
(2.5.2).
### What you think should happen instead?
A label should not change which tasks `load` waits for. The docs present
`>>` with a Label and `set_downstream(..., Label(...))` as the same thing, and
#17469, which #29410 fixed, stated that task dependencies with or without
labels should be the same. `load` should depend on `transform.clean` only, with
the label on that edge.
As far as I can tell, `EdgeModifier.set_downstream` in
`task-sdk/src/airflow/sdk/definitions/edges.py` converts the grouped task into
its TaskGroup before wiring the dependency, so every leaf of the group gets
connected; `set_upstream` passes the unconverted task on, which is why `<<`
behaves differently.
### Operating System
macOS (Darwin 25.2), Python 3.12.15, venv
### Deployment
Other
### Apache Airflow Provider(s)
standard
### Versions of Apache Airflow Providers
apache-airflow-providers-standard as pinned by the 3.3.2 and 3.4.0rc2
constraints files
### Official Helm Chart version
Not Applicable
### Kubernetes Version
Not Applicable
### Helm Chart configuration
Not Applicable
### Docker Image customizations
Not Applicable
### Anything else?
Searched issues and PRs for Label with TaskGroup dependencies, EdgeModifier,
and `upstream_failed` with Label; no duplicate found in those searches. #17469
is the closest and covers a different case (a label pointing at a group).
I can open a PR once the intended behaviour is confirmed (keep the
dependency on the task itself and put the label on that edge).
---
Drafted-by: Claude Code (Fable 5.1); reviewed by @luc-pimentel before posting
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [x] I agree to follow this project's [Code of
Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
