pankajkoti commented on PR #33464:
URL: https://github.com/apache/airflow/pull/33464#issuecomment-1681786153
@uranusjr The zip file contains two DAGs parent & child both have timezone
as pendulum.timezone("Pacific/Auckland")
and the execution delta is 1 day.
So for the parent DAGs execution date of 2023-04-01 (which is the logical
date in UTC), which is the DST switch day, it cannot find a matching DAG.
Below are some logs which show how the calculations happen with the change
in this PR and without this change how it looks for the DST switched time for
the child DAG(2023-03-31 15:00hrs) instead of the executed time (pre-DST) on
2023-03-31 (14:00 hrs local time)
```
[2023-08-17, 12:57:57 IST] {external_task.py:235} INFO - Below are the DST
transition calculations for execution_delta
[2023-08-17, 12:57:57 IST] {external_task.py:236} INFO -
'context['logical_date']' called as logical_date: : 2023-04-01T15:00:00+00:00
[2023-08-17, 12:57:57 IST] {external_task.py:237} INFO -
context['logical_date'].in_timezone(dag_tz) called as logical_date_dag_tz:
2023-04-02T03:00:00+12:00
[2023-08-17, 12:57:57 IST] {external_task.py:238} INFO -
'make_naive(logical_date_dag_tz, dag_tz)' called as logical_date_dag_tz_naive:
2023-04-02 03:00:00
[2023-08-17, 12:57:57 IST] {external_task.py:239} INFO -
'self.execution_delta': 1 day, 0:00:00
[2023-08-17, 12:57:57 IST] {external_task.py:240} INFO -
'make_naive(logical_date_dag_tz, dag_tz) - self.execution_delta' called as
dttm_dag_naive: 2023-04-01 03:00:00
[2023-08-17, 12:57:57 IST] {external_task.py:241} INFO -
'make_aware(make_naive(logical_date_dag_tz, dag_tz) - self.execution_delta,
dag_tz)' called as dttm_dag_tz: 2023-04-01 03:00:00+13:00
[2023-08-17, 12:57:57 IST] {external_task.py:242} INFO -
'pendulum.instance(make_aware(make_naive(logical_date_dag_tz, dag_tz) -
self.execution_delta, dag_tz)).in_timezone('UTC')' called as dttm:
2023-03-31T14:00:00+00:00
[2023-08-17, 12:57:57 IST] {external_task.py:244} INFO - Without all the
above calculations, currently the value of 'context['logical_date'] -
self.execution_delta' called as dttm is: 2023-03-31T15:00:00+00:00
[2023-08-17, 12:57:57 IST] {external_task.py:264} INFO - Poking for tasks
['EOJ'] in dag END_OF_BATCH_MILESTONE_1 on 2023-03-31T14:00:00+00:00 ...
```
How would you suggest handling this better?
[Archive.zip](https://github.com/apache/airflow/files/12366684/Archive.zip)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
