thejens opened a new issue #18317: URL: https://github.com/apache/airflow/issues/18317
### Description I don't believe backfilling (of data) is not well handled in Airflow at the moment. I believe the current backfill CLI command should have a UI component on the webserver - not the least as many deployments of Airflow doesn't expose a CLI interface to the users - only to admins. I would also like it to handle the following cases: 1. Backfill a DAG for execution dates where it has already run. - This essentially is the same as clearing all DAG-runs between those dates. The current UI forces me to manually clear the runs, I can toggle "Future" or "Past", but nothing like a "Range Between". 2. Backfill a TASK for execution dates where it has already run - This is essentially the same as above, except instead of clearing the whole DAG, it would clear a task, and potentially all downstreams from that task. A common use case here is when a new task is added to an existing DAG and you want to re-trigger it for historical dates. Currently the new task will have no execution, but the DAG it was created into will have successful runs for those dates. 3. The above, including execution dates where it has not run - In both the above examples, specifying a start_date before pre-existing DAG runs should insert those DAG runs and execute them. It should also insert any missing dag-runs between the dates where it has already run, in case those have gone missing. I'd be happy to provide some of the functionality in a PR, but I am not a skilled frontend developer. ### Use case/motivation Altering a DAG or task is a common use-case, for instance if an improvement to some business logic has been made - or a bug has been found and fixed. Restoring the status of those produced datasets easily is then important. Here's both the case where you want to re-run DAG executions, and insert new DAG executions for historical dates that may have been cleaned up to save space in the DB. ### Related issues I am certain this has been raised and discussed in the past. It is the number one feature I miss from working with Luigi - where I could easily re-trigger historical task-runs. ### Are you willing to submit a PR? - [X] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
