yuqian90 edited a comment on issue #7038: [AIRFLOW-4495] Allow externally
triggered dags to run for future exec dates
URL: https://github.com/apache/airflow/pull/7038#issuecomment-577471036
> Sorry for the comment spam but I thought it would be important for
@yuqian90
>
> We have the exact same problem you have, we solve it by setting the DAG
with a start date of the timezones it should be scheduled in, e.g. Tokyo time.
We then create a macro which is a modified version of this:
https://stackoverflow.com/a/54922776/2958068
>
> The macro above provides the execution date in the same time zone as the
DAG. Further we modify it to provide the actual scheduled date e.g. if
scheduled to run on 2019-01-01 08:00 that's the datetime it provides (which in
the default Airlow variables is the 'next execution date').
@notatallshaw thanks for the comment. Did setting `start_date` to the
timezone you want work? We definitely remember trying something like that. But
the airflow scheduler still did not run tasks before UTC midnight. Looking at
the original code before this PR, it's not hard to see why. No matter what
timezone we set the `start_date` to, this line was always skipping tasks as
long as the execution_date is greater than `timezone.utcnow()`. So if the
timezone we need is in Asia, this does not work till UTC midnight.
But this PR fixed / improved this. Now we no longer have to put tasks for
date T on the DAG for execution_date T-1.
```
airflow/jobs/scheduler_job.py
...
# don't consider runs that are executed in the future
if run.execution_date > timezone.utcnow():
self.log.error(
"Execution date is in future: %s",
run.execution_date
)
continue
```
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
