GitHub user potiuk edited a comment on the discussion: Add the ability to backfill a DAG based on past Asset Events
> Not exactly. If you take the scenario I described above, when I want to > "catch-up" DAG_2, it does not have any DAG runs yet. There's nothing I can > select and clear. Therefore, the two options you are mentioning does not > answer my need. You are contradicting yourself here. In this case you just start a New dag. It does not "catch up" with anything. Because it will be created NOW not in the past. Again - you have no linking of the asset-triggered dag with "time". This is where your logic is completely broken. If you want to "catch up" with say "5 missing past events" - those will be just **5** times NEW asset events that will create new dag runs. Those dag runs will have NO "past" meaning **at all**. They will be triggered "now" - and there is no way you can "catch-up" with anything. You will have simply 5 new runs of the same dag. There is no "concept" of time or time interval associated with dag run triggered by asset. Those new 5 dag runs will not be created in the past (also backfill does not do it - because it makes no sense whatsoever - when you backfill (timed) dags - each of them will be associated with some past "data interval", but they will be triggered NOW. So you are not "catching up" with anything - you are just creating 5 new dags. In order to somehow associate your dags with some "time" to catch up, you would need to specify that time somehow as "extra" of your asset - in the way that is specific to your asset, because - as I already explained several times - asset-triggered dag has "NO" time relation. It's just a dag run triggered by asset - but it tells nothing about "Time" of that event, except the time when it was triggered - but tha time changes, if you want to run the dag run again, it will be triggered "now" and it's triggering time will be NOW - not in May. You can "clear" past run, yes, but you cannot create a past-time asset triggered dag run unless you specify time in asset-specific way. And it makes absolutely no sense to catch-up old dags if you do not not want to trigger downstream dags. If you want to trigger new Dag Runs that have no downstream dags, you should define them like that and run. I see zero reason for > /api/v2/dags/:dag_id/assets/queuedEvents I have no idea what that event would do in term of the "catchup" described above. There are no past dagruns so whatever queud events relates to past dags - especially - that (as I explained in detail several times) there are no asset triggered dag runs connected with some time - they have "create time" and if you create a new dag run - they are always created NOW. GitHub link: https://github.com/apache/airflow/discussions/59886#discussioncomment-15381240 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
