uranusjr commented on issue #38826: URL: https://github.com/apache/airflow/issues/38826#issuecomment-2044327568
On the implementation front—the `catchup=True` case is not too difficult to implement, but `catchup=False` presents problems. When scheduling against time, we can simply create a DAG run in the future because we know when that’ll happen, and that future run also acts as the cutoff point in subsequent iterations for how long the scheduler should look back. With event-based scheduling, however, we don’t know when the next run will happen, and need another way to tell the scheduler when to stop looking into the past. We could add a new field on `DagModel` for this (say a datetime field `last_unpause`). Alternatively… maybe we can cheat instead and create one run from the most recent past events? Once that one run exists, the scheduler would know not to look past it in later iterations. Are there other solutions? I would love some ideas. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
