GitHub user peter added a comment to the discussion: Add the ability to backfill a DAG based on past Asset Events
@hnomichith Good question. I don't know how to do the "downstream clear with all future" that you are talking about here. Conceptually I don't understand why a backfill means clearing out historical runs. What I am doing for now as a workaround is that I am triggering the asset based downstream DAG using the Airflow REST API by invoking the "trigger DAG Run" endpoint and passing `logical_date` to be the date after the date I want to backfill. I then loop over all the dates in a script. The only problem there though is that there is a unique constraint on `logical_date` (which is not really a date at all but a timestamp) in the database so I have to keep coming up with new timestamp increments to work around this. It's very messy. I guess clearing out runs first would avoid this, but I don't know if that is any better? https://airflow.apache.org/docs/apache-airflow/stable/stable-rest-api-ref.html#operation/trigger_dag_run GitHub link: https://github.com/apache/airflow/discussions/59886#discussioncomment-15556919 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
