hugochinchilla opened a new pull request, #28333: URL: https://github.com/apache/airflow/pull/28333
This comes from the discussion in https://github.com/apache/airflow/discussions/28253 > Currently the behavior for a DAG which depends on multiple DataSets is to be executed only when all datasets have been updated since the last DAG execution. This feature would enable a DAG to run every time any of the multiple DataSets change. > > The motivation is to allow better traceability of changes in a derived dataset, being able to track down changes along all the DAGs chain for every change in every dataset. > This adds a new field to the DAG and DagModel to allow for a DAG to be scheduled anytime any of it's datasets passed in the schedule field is updated, instead of waiting for all datasets to be updated. Still to do: - [ ] pass community review - [ ] update the UI to remove the "x of y datasets updated" in the column for next schedule ping @ashb @potiuk -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
