krisuj edited a comment on issue #19343:
URL: https://github.com/apache/airflow/issues/19343#issuecomment-955946153
I am also seeing the same error, also running airflow 2.2.1.
In fact the error even occurs when a subdag completes; and seems to be
related to parsing the correct end-time.
[2021-10-31 22:33:42,297] {dagrun.py:715} WARNING - Failed to record
duration of <DagRun rebalance_portfolio.run_strategy @ 2021-11-01
04:10:00+00:00: scheduled__2021-11-01T04:10:00+00:00, externally triggered:
True>: start_date is not set.
[2021-10-31 22:33:42,439] {scheduler_job.py:644} ERROR - Exception when
executing SchedulerJob._run_scheduler_loop
Traceback (most recent call last):
File
"/Users/krishan/opt/miniconda3/envs/jumpman-dev/lib/python3.7/site-packages/airflow/jobs/scheduler_job.py",
line 628, in _execute
self._run_scheduler_loop()
File
"/Users/krishan/opt/miniconda3/envs/jumpman-dev/lib/python3.7/site-packages/airflow/jobs/scheduler_job.py",
line 709, in _run_scheduler_loop
num_queued_tis = self._do_scheduling(session)
File
"/Users/krishan/opt/miniconda3/envs/jumpman-dev/lib/python3.7/site-packages/airflow/jobs/scheduler_job.py",
line 792, in _do_scheduling
callback_to_run = self._schedule_dag_run(dag_run, session)
File
"/Users/krishan/opt/miniconda3/envs/jumpman-dev/lib/python3.7/site-packages/airflow/jobs/scheduler_job.py",
line 1044, in _schedule_dag_run
self._update_dag_next_dagruns(dag, dag_model, active_runs)
File
"/Users/krishan/opt/miniconda3/envs/jumpman-dev/lib/python3.7/site-packages/airflow/jobs/scheduler_job.py",
line 935, in _update_dag_next_dagruns
data_interval = dag.get_next_data_interval(dag_model)
File
"/Users/krishan/opt/miniconda3/envs/jumpman-dev/lib/python3.7/site-packages/airflow/models/dag.py",
line 629, in get_next_data_interval
return self.infer_automated_data_interval(dag_model.next_dagrun)
File
"/Users/krishan/opt/miniconda3/envs/jumpman-dev/lib/python3.7/site-packages/airflow/models/dag.py",
line 667, in infer_automated_data_interval
end = cast(CronDataIntervalTimetable, self.timetable)._get_next(start)
File
"/Users/krishan/opt/miniconda3/envs/jumpman-dev/lib/python3.7/site-packages/airflow/timetables/interval.py",
line 171, in _get_next
naive = make_naive(current, self._timezone)
File
"/Users/krishan/opt/miniconda3/envs/jumpman-dev/lib/python3.7/site-packages/airflow/utils/timezone.py",
line 143, in make_naive
if is_naive(value):
File
"/Users/krishan/opt/miniconda3/envs/jumpman-dev/lib/python3.7/site-packages/airflow/utils/timezone.py",
line 50, in is_naive
return value.utcoffset() is None
AttributeError: 'NoneType' object has no attribute 'utcoffset'
Once the error occurs, it re-occurs when restarting the scheduler; as it
seems to go through similar steps in parsing the end time for completed
dags/tasks.
(jumpman-dev) krishan@Krishans-MacBook-Pro jumpman % airflow scheduler
____________ _____________
____ |__( )_________ __/__ /________ __
____ /| |_ /__ ___/_ /_ __ /_ __ \_ | /| / /
___ ___ | / _ / _ __/ _ / / /_/ /_ |/ |/ /
_/_/ |_/_/ /_/ /_/ /_/ \____/____/|__/
[2021-10-31 22:34:57 -0700] [25584] [INFO] Starting gunicorn 20.1.0
[2021-10-31 22:34:57 -0700] [25584] [INFO] Listening at: http://0.0.0.0:8793
(25584)
[2021-10-31 22:34:57 -0700] [25584] [INFO] Using worker: sync
[2021-10-31 22:34:57 -0700] [25585] [INFO] Booting worker with pid: 25585
[2021-10-31 22:34:57 -0700] [25586] [INFO] Booting worker with pid: 25586
[2021-10-31 22:34:57,619] {scheduler_job.py:596} INFO - Starting the
scheduler
[2021-10-31 22:34:57,620] {scheduler_job.py:601} INFO - Processing each file
at most -1 times
[2021-10-31 22:34:57,727] {manager.py:163} INFO - Launched
DagFileProcessorManager with pid: 25620
[2021-10-31 22:34:57,729] {scheduler_job.py:1115} INFO - Resetting orphaned
tasks for active dag runs
[2021-10-31 22:34:57,742] {settings.py:52} INFO - Configured default
timezone Timezone('UTC')
[2021-10-31 22:34:58,468] {scheduler_job.py:1181} INFO - Reset the following
1 orphaned TaskInstances:
<TaskInstance: rebalance_portfolio.run_strategy
scheduled__2021-11-01T04:10:00+00:00 [running]>
[2021-10-31 22:35:00,108] {dagrun.py:511} INFO - Marking run <DagRun
rebalance_portfolio.run_strategy @ 2021-11-01 04:10:00+00:00:
scheduled__2021-11-01T04:10:00+00:00, externally triggered: True> successful
[2021-10-31 22:35:00,108] {dagrun.py:571} INFO - DagRun Finished:
dag_id=rebalance_portfolio.run_strategy, execution_date=2021-11-01
04:10:00+00:00, run_id=scheduled__2021-11-01T04:10:00+00:00,
run_start_date=None, run_end_date=2021-11-01 05:35:00.108711+00:00,
run_duration=None, state=success, external_trigger=True, run_type=scheduled,
data_interval_start=2021-11-01 04:10:00+00:00, data_interval_end=2021-11-01
05:10:00+00:00, dag_hash=None
[2021-10-31 22:35:00,109] {dagrun.py:715} WARNING - Failed to record
duration of <DagRun rebalance_portfolio.run_strategy @ 2021-11-01
04:10:00+00:00: scheduled__2021-11-01T04:10:00+00:00, externally triggered:
True>: start_date is not set.
[2021-10-31 22:35:00,244] {scheduler_job.py:644} ERROR - Exception when
executing SchedulerJob._run_scheduler_loop
Traceback (most recent call last):
File
"/Users/krishan/opt/miniconda3/envs/jumpman-dev/lib/python3.7/site-packages/airflow/jobs/scheduler_job.py",
line 628, in _execute
self._run_scheduler_loop()
File
"/Users/krishan/opt/miniconda3/envs/jumpman-dev/lib/python3.7/site-packages/airflow/jobs/scheduler_job.py",
line 709, in _run_scheduler_loop
num_queued_tis = self._do_scheduling(session)
File
"/Users/krishan/opt/miniconda3/envs/jumpman-dev/lib/python3.7/site-packages/airflow/jobs/scheduler_job.py",
line 792, in _do_scheduling
callback_to_run = self._schedule_dag_run(dag_run, session)
File
"/Users/krishan/opt/miniconda3/envs/jumpman-dev/lib/python3.7/site-packages/airflow/jobs/scheduler_job.py",
line 1044, in _schedule_dag_run
self._update_dag_next_dagruns(dag, dag_model, active_runs)
File
"/Users/krishan/opt/miniconda3/envs/jumpman-dev/lib/python3.7/site-packages/airflow/jobs/scheduler_job.py",
line 935, in _update_dag_next_dagruns
data_interval = dag.get_next_data_interval(dag_model)
File
"/Users/krishan/opt/miniconda3/envs/jumpman-dev/lib/python3.7/site-packages/airflow/models/dag.py",
line 629, in get_next_data_interval
return self.infer_automated_data_interval(dag_model.next_dagrun)
File
"/Users/krishan/opt/miniconda3/envs/jumpman-dev/lib/python3.7/site-packages/airflow/models/dag.py",
line 667, in infer_automated_data_interval
end = cast(CronDataIntervalTimetable, self.timetable)._get_next(start)
File
"/Users/krishan/opt/miniconda3/envs/jumpman-dev/lib/python3.7/site-packages/airflow/timetables/interval.py",
line 171, in _get_next
naive = make_naive(current, self._timezone)
File
"/Users/krishan/opt/miniconda3/envs/jumpman-dev/lib/python3.7/site-packages/airflow/utils/timezone.py",
line 143, in make_naive
if is_naive(value):
File
"/Users/krishan/opt/miniconda3/envs/jumpman-dev/lib/python3.7/site-packages/airflow/utils/timezone.py",
line 50, in is_naive
return value.utcoffset() is None
AttributeError: 'NoneType' object has no attribute 'utcoffset'
[2021-10-31 22:35:00,248] {local_executor.py:389} INFO - Shutting down
LocalExecutor; waiting for running tasks to finish. Signal again if you don't
want to wait.
[2021-10-31 22:35:01,309] {process_utils.py:100} INFO - Sending
Signals.SIGTERM to GPID 25620
[2021-10-31 22:35:02,655] {process_utils.py:212} INFO - Waiting up to 5
seconds for processes to exit...
[2021-10-31 22:35:02,678] {process_utils.py:66} INFO - Process
psutil.Process(pid=25629, status='terminated', started='22:35:00') (25629)
terminated with exit code None
[2021-10-31 22:35:02,679] {process_utils.py:66} INFO - Process
psutil.Process(pid=25628, status='terminated', started='22:35:00') (25628)
terminated with exit code None
[2021-10-31 22:35:02,679] {process_utils.py:66} INFO - Process
psutil.Process(pid=25620, status='terminated', exitcode=0, started='22:34:57')
(25620) terminated with exit code 0
[2021-10-31 22:35:02,680] {scheduler_job.py:655} INFO - Exited execute loop
Traceback (most recent call last):
File "/Users/krishan/opt/miniconda3/envs/jumpman-dev/bin/airflow", line 8,
in <module>
sys.exit(main())
File
"/Users/krishan/opt/miniconda3/envs/jumpman-dev/lib/python3.7/site-packages/airflow/__main__.py",
line 48, in main
args.func(args)
File
"/Users/krishan/opt/miniconda3/envs/jumpman-dev/lib/python3.7/site-packages/airflow/cli/cli_parser.py",
line 48, in command
return func(*args, **kwargs)
File
"/Users/krishan/opt/miniconda3/envs/jumpman-dev/lib/python3.7/site-packages/airflow/utils/cli.py",
line 92, in wrapper
return f(*args, **kwargs)
File
"/Users/krishan/opt/miniconda3/envs/jumpman-dev/lib/python3.7/site-packages/airflow/cli/commands/scheduler_command.py",
line 75, in scheduler
_run_scheduler_job(args=args)
File
"/Users/krishan/opt/miniconda3/envs/jumpman-dev/lib/python3.7/site-packages/airflow/cli/commands/scheduler_command.py",
line 46, in _run_scheduler_job
job.run()
File
"/Users/krishan/opt/miniconda3/envs/jumpman-dev/lib/python3.7/site-packages/airflow/jobs/base_job.py",
line 245, in run
self._execute()
File
"/Users/krishan/opt/miniconda3/envs/jumpman-dev/lib/python3.7/site-packages/airflow/jobs/scheduler_job.py",
line 628, in _execute
[2021-10-31 22:35:03 -0700] [25584] [INFO] Handling signal: term
self._run_scheduler_loop()
File
"/Users/krishan/opt/miniconda3/envs/jumpman-dev/lib/python3.7/site-packages/airflow/jobs/scheduler_job.py",
line 709, in _run_scheduler_loop
num_queued_tis = self._do_scheduling(session)
File
"/Users/krishan/opt/miniconda3/envs/jumpman-dev/lib/python3.7/site-packages/airflow/jobs/scheduler_job.py",
line 792, in _do_scheduling
callback_to_run = self._schedule_dag_run(dag_run, session)
File
"/Users/krishan/opt/miniconda3/envs/jumpman-dev/lib/python3.7/site-packages/airflow/jobs/scheduler_job.py",
line 1044, in _schedule_dag_run
[2021-10-31 22:35:03 -0700] [25586] [INFO] Worker exiting (pid: 25586)
[2021-10-31 22:35:03 -0700] [25585] [INFO] Worker exiting (pid: 25585)
self._update_dag_next_dagruns(dag, dag_model, active_runs)
File
"/Users/krishan/opt/miniconda3/envs/jumpman-dev/lib/python3.7/site-packages/airflow/jobs/scheduler_job.py",
line 935, in _update_dag_next_dagruns
data_interval = dag.get_next_data_interval(dag_model)
File
"/Users/krishan/opt/miniconda3/envs/jumpman-dev/lib/python3.7/site-packages/airflow/models/dag.py",
line 629, in get_next_data_interval
return self.infer_automated_data_interval(dag_model.next_dagrun)
File
"/Users/krishan/opt/miniconda3/envs/jumpman-dev/lib/python3.7/site-packages/airflow/models/dag.py",
line 667, in infer_automated_data_interval
end = cast(CronDataIntervalTimetable, self.timetable)._get_next(start)
File
"/Users/krishan/opt/miniconda3/envs/jumpman-dev/lib/python3.7/site-packages/airflow/timetables/interval.py",
line 171, in _get_next
naive = make_naive(current, self._timezone)
File
"/Users/krishan/opt/miniconda3/envs/jumpman-dev/lib/python3.7/site-packages/airflow/utils/timezone.py",
line 143, in make_naive
if is_naive(value):
File
"/Users/krishan/opt/miniconda3/envs/jumpman-dev/lib/python3.7/site-packages/airflow/utils/timezone.py",
line 50, in is_naive
return value.utcoffset() is None
AttributeError: 'NoneType' object has no attribute 'utcoffset'
[2021-10-31 22:35:03 -0700] [25584] [INFO] Shutting down: Master
(jumpman-dev) krishan@Krishans-MacBook-Pro jumpman %
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
