uranusjr opened a new pull request, #33404: URL: https://github.com/apache/airflow/pull/33404
Previously, with catchup=False, CronTriggerTimetable would aggressively cut off a run if the scheduler doesn't ask to schedule the next run immediately. This causes DAGs to seemingly mysteriously "miss" a run from time to time due to the scheduler inevitably having a very slight hiccup. This change makes the timetable's non-catchup cutoff logic a little more lax, and only activate when the scheduler misses at least an entire interval. For example, for a daily cron, if the previous run happened on midnight of 2nd Jun (to cover 1st Jun), the timetable would still allow scheduling a run covering 2nd Jun if the scheduler asks for it some time during the 2nd, and would skip the 2nd Jun run entirely only if the scheduler fails to ask for a run on the entirety of the 2nd and only asks after midnight on the 3rd. As discussed in #27399, I feel this is the more reasonable fix than #32921, even though it slightly changes the behaviour (specifically, the first ever run would start one interval earlier than previously). Quoting from the linked issue: > > I think a reasonable logic would be to change the catchup=False logic to cover one schedule before the current time instead, so in the above scenario, the timetable would make the next run cover 4am, and only skip the 4am run if the current time is pas 5am. > > Sounds good. It's natural to those who are used to Cron and Anacron. > > I think we need to document the new behavior. […] Fix #27339. Close #32921. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
