thejens edited a comment on issue #18317: URL: https://github.com/apache/airflow/issues/18317#issuecomment-923758407
It seems backfilling comes with some baggage in terms of definitions. What I'm after is a way to insert multiple dag-runs for historical dates in bulk from the UI, possibly with some tasks already marked as complete/skipped, as well as clearing tasks/dagruns between certain dates. I don't see the need for a dedicated backfill process to run, the scheduler could take care of that I believe, if tasks and dags are idempotent they don't even need to care about execution order, if order matters I guess `depends_on_past` should be set on the tasks(?) and the scheduler should handle it(?) I also don't consider backfill an afterthought. We often instrument new DAGs and want to execute them for historical dates to create historical data. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
