Lee-W commented on code in PR #41022: URL: https://github.com/apache/airflow/pull/41022#discussion_r1692408965
########## docs/apache-airflow/authoring-and-scheduling/datasets.rst: ########## @@ -321,6 +321,34 @@ Example: Note that this example is using `(.values() | first | first) <https://jinja.palletsprojects.com/en/3.1.x/templates/#jinja-filters.first>`_ to fetch the first of one dataset given to the DAG, and the first of one DatasetEvent for that dataset. An implementation can be quite complex if you have multiple datasets, potentially with multiple DatasetEvents. +Manipulating queued dataset events through REST API +--------------------------------------------------- + +.. versionadded:: 2.9 + +In this example, the DAG ``waiting_for_dataset_1_and_2`` will be triggered when tasks update both datasets "dataset-1" and "dataset-2". Once "dataset-1" is updated, Airflow creates a record. This ensures that Airflow knows to trigger the DAG when "dataset-2" is updated. We call such records queued dataset events. + +.. code-block:: python + + with DAG( + dag_id="waiting_for_dataset_1_and_2", + schedule=[Dataset("dataset-1"), Dataset("dataset-2")], + ..., + ): + ... Review Comment: I'm trying to explain the term "queued dataset events" here. Under the hook, it's DatasetDagRunQueue, but ASAIK, we don't want to expose this internal thing -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
