GitHub user hnomichith added a comment to the discussion: Add the ability to backfill a DAG based on past Asset Events
Hello! If so, then we could write an AIP [here](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvement+Proposals). To simplify, I wouldn't call it neither "backfill" or "catch-up", but something like "process past asset events", but I don't think it even need to bear a name if we limit the proposal to the API. What I would really want to do is: - POST /api/v2/dags/:dag_id/assets/queuedEvents endpoint; giving it an existing Asset Event ID - More filters on GET /api/v2/assets/events to find relevant Asset Event IDs Reading your usecase bellow, I think it would also answer your need. It still a script, but it would not rely on faking an Asset update. As me, you would then search the Asset Event IDs for the date range for am interested in, then queue them in the downstream DAG. Reading the AIP page, what I describe doesn't qualify as "a major change that needs an AIP" since it's only touching Airflow 3 REST endpoints. However, since there seems to be some constraints or complexity I don't see, it would justify it. If we add a UI part, then it falls into major change, and it would need a name. I can also see that "searching for the asset events" could be hard for DAGs with complexe dependencies. So it could come later, depending on the need, and requiring more design. GitHub link: https://github.com/apache/airflow/discussions/59886#discussioncomment-15508840 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
