brunorigal opened a new issue, #26271:
URL: https://github.com/apache/airflow/issues/26271
### Description
I would like to build a graph like this:
┌───► task_2_1 ─────► task_2_1
│
│
│
task 1 ─────► task_2_2 ─────► task_3_2
│
│
│
│
└──► task_2_N ─────► task_3_N
Where the number of vertical tasks (N) is controlled dynamically by the
first step computation. A possible solution would be:
```python
from airflow import DAG
from airflow.decorators import task, task_group
from pendulum import datetime, now
@task
def task_1():
return list(range(5))
@task
def task_2(task_num):
return task_num
@task
def task_3(task_num):
return task_num
with DAG(dag_id="my_dag", start_date=now(), schedule_interval=None) as dag:
task_3.expand(task_num=task_2.expand(task_num=task_1()))
```
But the task_3 instances start to run only when all the task_2 instance have
finished. I would rather group tasks by instances of task_2 and task_3 sharing
the same task_num.
An intuitive solution would have been to use a taskgroup, but these do not
have the expand method.
### Use case/motivation
_No response_
### Related issues
_No response_
### Are you willing to submit a PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of
Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
