krisuj edited a comment on issue #19343:
URL: https://github.com/apache/airflow/issues/19343#issuecomment-956278498
Yeah @andydennehy, I see that as well. The DAG that the scheduler is trying
to update is a SubDAG in my case; and it fails to update correctly (once
completed) and causes the error in scheduler. The DAG has an hourly interval
with a cron schedule of "1 * * * *". I wonder if using the `timetable` instead
will resolve this issue? I tried what you mentioned in manually editing the
fields in the `dag` table in the airflow database (postgres). Seems to help
scheduler startup but fails again in completing the next DAG run and the issue
persists therafter.
```
____________ _____________
____ |__( )_________ __/__ /________ __
____ /| |_ /__ ___/_ /_ __ /_ __ \_ | /| / /
___ ___ | / _ / _ __/ _ / / /_/ /_ |/ |/ /
_/_/ |_/_/ /_/ /_/ /_/ \____/____/|__/
[2021-11-01 07:08:34,346] {scheduler_job.py:596} INFO - Starting the
scheduler
[2021-11-01 07:08:34,347] {scheduler_job.py:601} INFO - Processing each file
at most -1 times
[2021-11-01 07:08:34,356] {manager.py:163} INFO - Launched
DagFileProcessorManager with pid: 30671
[2021-11-01 07:08:34,360] {scheduler_job.py:1115} INFO - Resetting orphaned
tasks for active dag runs
[2021-11-01 07:08:34 -0700] [30670] [INFO] Starting gunicorn 20.1.0
[2021-11-01 07:08:34 -0700] [30670] [INFO] Listening at: http://0.0.0.0:8793
(30670)
[2021-11-01 07:08:34 -0700] [30670] [INFO] Using worker: sync
[2021-11-01 07:08:34,369] {settings.py:52} INFO - Configured default
timezone Timezone('UTC')
[2021-11-01 07:08:34 -0700] [30672] [INFO] Booting worker with pid: 30672
[2021-11-01 07:08:34,403] {manager.py:434} WARNING - Because we cannot use
more than 1 thread (parsing_processes = 2 ) when using sqlite. So we set
parallelism to 1.
[2021-11-01 07:08:34 -0700] [30673] [INFO] Booting worker with pid: 30673
[2021-11-01 07:08:36,919] {dagrun.py:511} INFO - Marking run <DagRun
load_coincompare_historical_prices_hour_interval_intraday_run.load_coincompare_historical_subdag
@ 2021-11-01 05:01:00+00:00: scheduled__2021-11-01T05:01:00+00:00, externally
triggered: True> successful
[2021-11-01 07:08:36,919] {dagrun.py:571} INFO - DagRun Finished:
dag_id=load_coincompare_historical_prices_hour_interval_intraday_run.load_coincompare_historical_subdag,
execution_date=2021-11-01 05:01:00+00:00,
run_id=scheduled__2021-11-01T05:01:00+00:00, run_start_date=2021-11-01
06:40:54.866411+00:00, run_end_date=2021-11-01 14:08:36.919724+00:00,
run_duration=26862.053313, state=success, external_trigger=True,
run_type=scheduled, data_interval_start=2021-11-01 05:01:00+00:00,
data_interval_end=2021-11-01 06:01:00+00:00, dag_hash=None
[2021-11-01 07:08:36,923] {scheduler_job.py:644} ERROR - Exception when
executing SchedulerJob._run_scheduler_loop
Traceback (most recent call last):
File
"/Users/krishan/opt/miniconda3/envs/jumpman-dev/lib/python3.7/site-packages/airflow/jobs/scheduler_job.py",
line 628, in _execute
self._run_scheduler_loop()
File
"/Users/krishan/opt/miniconda3/envs/jumpman-dev/lib/python3.7/site-packages/airflow/jobs/scheduler_job.py",
line 709, in _run_scheduler_loop
num_queued_tis = self._do_scheduling(session)
File
"/Users/krishan/opt/miniconda3/envs/jumpman-dev/lib/python3.7/site-packages/airflow/jobs/scheduler_job.py",
line 792, in _do_scheduling
callback_to_run = self._schedule_dag_run(dag_run, session)
File
"/Users/krishan/opt/miniconda3/envs/jumpman-dev/lib/python3.7/site-packages/airflow/jobs/scheduler_job.py",
line 1044, in _schedule_dag_run
self._update_dag_next_dagruns(dag, dag_model, active_runs)
File
"/Users/krishan/opt/miniconda3/envs/jumpman-dev/lib/python3.7/site-packages/airflow/jobs/scheduler_job.py",
line 935, in _update_dag_next_dagruns
data_interval = dag.get_next_data_interval(dag_model)
File
"/Users/krishan/opt/miniconda3/envs/jumpman-dev/lib/python3.7/site-packages/airflow/models/dag.py",
line 629, in get_next_data_interval
return self.infer_automated_data_interval(dag_model.next_dagrun)
File
"/Users/krishan/opt/miniconda3/envs/jumpman-dev/lib/python3.7/site-packages/airflow/models/dag.py",
line 667, in infer_automated_data_interval
end = cast(CronDataIntervalTimetable, self.timetable)._get_next(start)
File
"/Users/krishan/opt/miniconda3/envs/jumpman-dev/lib/python3.7/site-packages/airflow/timetables/interval.py",
line 171, in _get_next
naive = make_naive(current, self._timezone)
File
"/Users/krishan/opt/miniconda3/envs/jumpman-dev/lib/python3.7/site-packages/airflow/utils/timezone.py",
line 143, in make_naive
if is_naive(value):
File
"/Users/krishan/opt/miniconda3/envs/jumpman-dev/lib/python3.7/site-packages/airflow/utils/timezone.py",
line 50, in is_naive
return value.utcoffset() is None
AttributeError: 'NoneType' object has no attribute 'utcoffset'
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
