Just to add to that: there are already tests that tests this behaviour: https://github.com/PolideaInternal/airflow/blob/master/tests/models/test_dag.py#L749 and indeed - the schedule follows DST changes rather than discards the DST time.
I think it is generally a good idea to follow DST (for clarity), but I might miss some context/cases. Maybe it is indeed better to have always 24 hour intervals for daily schedules for example rather than 23/25 sometimes). BTW. I fixed one of those tests that was failing recently on Python 3.6 CI. Python 3.6 behaved a bit differently than 3.5 at DST change: https://issues.apache.org/jira/browse/AIRFLOW-4308 J. On Mon, May 13, 2019 at 11:39 PM Maxime Beauchemin < [email protected]> wrote: > It would be great if people can provide failing unit tests as PR with clear > expectations stated out as code. It makes it easier for people to get > consensus on expectations and for anyone to jump in and implement a fix. > > Max > > On Mon, May 13, 2019 at 12:48 PM David Klosowski <[email protected]> > wrote: > > > Damian is correct. We've observed that exact behavior and noticed the > > timedelta logic is dubiously broken for DST but works for CRON. > > > > On Mon, May 13, 2019 at 12:38 PM Shaw, Damian P. < > > [email protected]> wrote: > > > > > Hi, > > > > > > I’m not part of the Airflow team but I came to the same conclusion, > that > > > the behavior is the opposite of what the documentation specifies. When > > > using the cron notation then DST is properly honored, when using > > timedelta > > > it is not. I played around with the DAG.following_schedule method to > > > satisfy this was the case. > > > > > > I’ve had production instance of airflow based on this that successfully > > > respected the March/April DST changes in many timezones. > > > > > > Regards > > > Damian > > > > > > From: Jiahao Chen [mailto:[email protected]] > > > Sent: Monday, May 13, 2019 2:08 PM > > > To: [email protected] > > > Subject: Cron schedule with DST-aware timezone > > > > > > Hi team, > > > > > > I have a question about the expected behavior of Airflow scheduler when > > > the schedule_interval is a cron expression and the start_date is in a > > > timezone with DST. > > > > > > Based on the Airflow documentation > > > https://airflow.apache.org/timezone.html#cron-schedules, the DST > change > > > will be ignored if schedule_interval is a cron expresion (e.g. '0 17 * > * > > > *'). And it gives an example that the GMT offset will not change > > regardless > > > how DST changes. If I'm understanding it correctly, that means if I > > upload > > > a DAG with a schedule_interval of "0 17 * * *" and a start_date of > > > 2019-03-15 17:00 PST(GMT-8) which is before the DST change on March 10, > > the > > > Airflow scheduler will always start the DAG on 5 pm everyday GMT-8 even > > > after the DST change on March 10. > > > > > > However, that is not the behavior I've seen with my experimental code > > (see > > > attachments). It looks like the the Airflow is actually taking the DST > > into > > > account, since the execution time is always 17:00 locally, which is 1 > > hour > > > off on the GMT after the DST change. > > > > > > Could you please confirm the behavior of Airflow scheduler in this use > > > case? > > > > > > Thank you! > > > Jiahao > > > > > > > > > > > > > > > > > > =============================================================================== > > > > > > Please access the attached hyperlink for an important electronic > > > communications disclaimer: > > > http://www.credit-suisse.com/legal/en/disclaimer_email_ib.html > > > > > > =============================================================================== > > > > > > > > > -- Jarek Potiuk Polidea <https://www.polidea.com/> | Principal Software Engineer M: +48 660 796 129 <+48660796129> E: [email protected]
