The distinction could come from being non-UTC VS UTC. You can schedule a DAG at any time UTC and it would follow the 24-hour intervals but if it is non-UTC it would follow DST changes when relevant (by timezone). You could technically make this behavior configurable and follow the former path as well (not sure the use cases).
On Mon, May 13, 2019 at 9:17 PM Jarek Potiuk <[email protected]> wrote: > Just to add to that: there are already tests that tests this behaviour: > > https://github.com/PolideaInternal/airflow/blob/master/tests/models/test_dag.py#L749 > and > indeed - the schedule follows DST changes rather than discards the DST > time. > > I think it is generally a good idea to follow DST (for clarity), but I > might miss some context/cases. Maybe it is indeed better to have always 24 > hour intervals for daily schedules for example rather than 23/25 > sometimes). > > BTW. I fixed one of those tests that was failing recently on Python 3.6 CI. > Python 3.6 behaved a bit differently than 3.5 at DST change: > https://issues.apache.org/jira/browse/AIRFLOW-4308 > > J. > > > > On Mon, May 13, 2019 at 11:39 PM Maxime Beauchemin < > [email protected]> wrote: > > > It would be great if people can provide failing unit tests as PR with > clear > > expectations stated out as code. It makes it easier for people to get > > consensus on expectations and for anyone to jump in and implement a fix. > > > > Max > > > > On Mon, May 13, 2019 at 12:48 PM David Klosowski <[email protected]> > > wrote: > > > > > Damian is correct. We've observed that exact behavior and noticed the > > > timedelta logic is dubiously broken for DST but works for CRON. > > > > > > On Mon, May 13, 2019 at 12:38 PM Shaw, Damian P. < > > > [email protected]> wrote: > > > > > > > Hi, > > > > > > > > I’m not part of the Airflow team but I came to the same conclusion, > > that > > > > the behavior is the opposite of what the documentation specifies. > When > > > > using the cron notation then DST is properly honored, when using > > > timedelta > > > > it is not. I played around with the DAG.following_schedule method to > > > > satisfy this was the case. > > > > > > > > I’ve had production instance of airflow based on this that > successfully > > > > respected the March/April DST changes in many timezones. > > > > > > > > Regards > > > > Damian > > > > > > > > From: Jiahao Chen [mailto:[email protected]] > > > > Sent: Monday, May 13, 2019 2:08 PM > > > > To: [email protected] > > > > Subject: Cron schedule with DST-aware timezone > > > > > > > > Hi team, > > > > > > > > I have a question about the expected behavior of Airflow scheduler > when > > > > the schedule_interval is a cron expression and the start_date is in a > > > > timezone with DST. > > > > > > > > Based on the Airflow documentation > > > > https://airflow.apache.org/timezone.html#cron-schedules, the DST > > change > > > > will be ignored if schedule_interval is a cron expresion (e.g. '0 17 > * > > * > > > > *'). And it gives an example that the GMT offset will not change > > > regardless > > > > how DST changes. If I'm understanding it correctly, that means if I > > > upload > > > > a DAG with a schedule_interval of "0 17 * * *" and a start_date of > > > > 2019-03-15 17:00 PST(GMT-8) which is before the DST change on March > 10, > > > the > > > > Airflow scheduler will always start the DAG on 5 pm everyday GMT-8 > even > > > > after the DST change on March 10. > > > > > > > > However, that is not the behavior I've seen with my experimental code > > > (see > > > > attachments). It looks like the the Airflow is actually taking the > DST > > > into > > > > account, since the execution time is always 17:00 locally, which is 1 > > > hour > > > > off on the GMT after the DST change. > > > > > > > > Could you please confirm the behavior of Airflow scheduler in this > use > > > > case? > > > > > > > > Thank you! > > > > Jiahao > > > > > > > > > > > > > > > > > > > > > > > > > > =============================================================================== > > > > > > > > Please access the attached hyperlink for an important electronic > > > > communications disclaimer: > > > > http://www.credit-suisse.com/legal/en/disclaimer_email_ib.html > > > > > > > > > > =============================================================================== > > > > > > > > > > > > > > > > -- > > Jarek Potiuk > Polidea <https://www.polidea.com/> | Principal Software Engineer > > M: +48 660 796 129 <+48660796129> > E: [email protected] >
