dabeeru commented on issue #55956:
URL: https://github.com/apache/airflow/issues/55956#issuecomment-3329956664
@cmarteepants I think we could introduce this parameter at the DAG level
while still allowing it to be used in any scenario. Then `max_asset_events`
parameter would just limit how many asset events can be polled for single DAG
run, always assuming that we should prioritize polling the oldest items first.
So for example:
```
Timeline: DAG with max_asset_events=1, scheduled on Assets A, B, C
Time Asset A Queue Asset B Queue Asset C Queue DAG Action
----- ------------- ------------- ------------- ----------
10:00 [E1] [] [] Wait
10:05 [E1] [E1] [] Wait
10:10 [E1, E2] [E1] [] Wait
10:15 [E1, E2] [E1, E2] [] Wait
10:20 [E1, E2] [E1, E2] [E1] 🚀 TRIGGER!
Consumed Events: A-E1, B-E1, C-E1 (oldest from each)
10:20 [E2] [E2] [] Wait
10:25 [E2, E3] [E2] [E2] 🚀 TRIGGER!
Consumed Events: A-E2, B-E2, C-E2 (oldest from each)
10:25 [E3] [] [] Wait
10:30 [E3, E4] [E3] [] Wait
```
Not sure how much it would impact current implementation, but conceptually
it makes sense for me. What do you think?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
