jedcunningham commented on issue #26256:
URL: https://github.com/apache/airflow/issues/26256#issuecomment-1242223978
So there are a couple of things going on here.
The first is that the "triggered dagruns" column is empty sometimes. This is
being fixed in #26256.
Next is that you expect greater than 1 with the DAGs you are testing with.
You have many updates to a single dataset, and a single downstream consumer.
With this setup, by definition each dataset event can only trigger a single
downstream run - even though there are many events, each specific event can
only trigger the one downstream DAG once. In contrast, the follow setup where
there are multiple consumers of the dataset will properly show more than 1
triggered dagrun per event.
```python
from datetime import datetime
from airflow import DAG, Dataset
from airflow.operators.empty import EmptyOperator
dataset = Dataset("hello")
with DAG(
dag_id="dataset_many_upstream",
start_date=datetime(2022, 8, 1),
schedule="@daily",
):
EmptyOperator(task_id="upstream_task", outlets=[dataset])
with DAG(
dag_id="dataset_many_downstream1",
start_date=datetime(2022, 8, 1),
schedule=[dataset],
):
EmptyOperator(task_id="downstream")
with DAG(
dag_id="dataset_many_downstream2",
start_date=datetime(2022, 8, 1),
schedule=[dataset],
):
EmptyOperator(task_id="downstream")
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
