karenbraganz opened a new issue, #49525: URL: https://github.com/apache/airflow/issues/49525
### Description Currently, a dataset is updated only when the producing task completes in the success state. I propose adding an `allowed_states` parameter to datasets, which would allow datasets to trigger the downstream consuming DAG even if the producing task is not successful. This would provide more flexibility with dataset scheduling. Proposed Changes: - The consuming DAG should include a list of `allowed_states` with the dataset used in the schedule. - The dataset should be updated once the producing task completes irrespective of the state of the producing task (not only when the producing task succeeds). - This update should trigger the consuming DAG if the state of the producing task is one of the states included in the `allowed_states` list. - The `allowed_states` will default to including the `success` state only. ### Use case/motivation - User wants consuming DAG to be triggered irrespective of the state of the producing task. - User wants different consuming DAGs to be triggered depending on the state of the producing task. ### Related issues _No response_ ### Are you willing to submit a PR? - [x] Yes I am willing to submit a PR! ### Code of Conduct - [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
