GitHub user olk edited a discussion: branching + dyn. task mapping + passing
arguments: how to filter passed arguments
Unfortunately **all** branches get the **same data/arguments** passed.
Based on the branch task-group 'consumer_a' should only process parameters with
"flag" = True and 'consumer_b' only data with "flag" = False.
How can I filter the argument list?
```
from airflow.decorators import dag, task, task_group
from pendulum import datetime
@task
def producer():
return [ { "flag": True, "url": "abc" }, { "flag": False, "url": "xyz "} ]
@task.branch(task_id="branching")
def random_choice(data):
if data["flag"]:
return "consumer_a"
else:
return "consumer_b"
@task
def print_flag(data):
print(f"flag: {data['flag']}")
return data
@task
def print_url(data):
print(f"url: {data['url']}")
@task_group
def consumer_a(data):
print_url(print_flag(data))
@task_group
def consumer_b(data):
@dag(
start_date=datetime(2023, 1, 1),
catchup=False,
schedule="@daily"
)
def branch_2():
data = producer()
random_choice.expand(data=data) >> [ consumer_a.expand(data=data),
consumer_b.expand(data=data) ] # filtering 'data' here?
branch_2()
```
GitHub link: https://github.com/apache/airflow/discussions/46035
----
This is an automatically sent email for [email protected].
To unsubscribe, please send an email to: [email protected]
