uranusjr opened a new pull request, #27876:
URL: https://github.com/apache/airflow/pull/27876
This allows the scheduler to correctly identify the dependencies, run things
in the right order, and write TaskMap metadata as needed.
This also helped catch a bug in BackfillJob where children of a mapped task
group is not correctly identified and expanded.
Bug identified by Jyotsana Namdev. Reproduction:
```python
from datetime import datetime
from airflow import DAG
from airflow.decorators import task, task_group
from airflow.operators.empty import EmptyOperator
with DAG(
dag_id="taskmap_taskgroup",
tags=["AIP_42"],
start_date=datetime(1970, 1, 1),
schedule=None,
) as dag:
@task
def onetwothree():
return [1, 2, 3]
@task
def hello_there(arg):
print(arg)
@task_group
def tg(x):
hello_there(x)
increment_and_verify.expand(x=onetwothree()) >>
EmptyOperator(task_id="done")
```
when executed, `tg` is currently unable to correctly identify `onetwothree`
as an upstream, and `hellow_there` fails to expand with error
```
Cannot expand <Task(_PythonDecoratedOperator): tg.hello_there> for run
my_custom_run; missing upstream values: ['x']
```
since the task group cannot “provide” it the value.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
