zhang699 commented on issue #13805:
URL: https://github.com/apache/airflow/issues/13805#issuecomment-781793430
I also encounter this problem, but curious about another questions, why load
same DAG three times before the exception happen, and the last two of loaded
DAG path seems incorrect.
```
[2021-02-19 03:35:50,204] {dagbag.py:413} DEBUG - Loaded DAG <DAG: etl_dag1>
[2021-02-19 03:35:50,205] {dagbag.py:287} DEBUG - Importing
/home/airflow/dags/..data/code-dag.py
[2021-02-19 03:35:50,222] {dagbag.py:413} DEBUG - Loaded DAG <DAG: etl_dag1>
[2021-02-19 03:35:50,223] {dagbag.py:287} DEBUG - Importing
/home/airflow/dags/..2021_02_19_03_05_19.306169647/code-dag.py
[2021-02-19 03:35:50,239] {dagbag.py:413} DEBUG - Loaded DAG <DAG: etl_dag1>
```
```
airflow@webserver-6b975954d-pf6h8:/opt/airflow$ airflow dags backfill
etl_dag1 -s 2020-01-20 -e 2020-01-21
[2021-02-19 03:35:48,536] {settings.py:210} DEBUG - Setting up DB connection
pool (PID 214)
[2021-02-19 03:35:48,537] {settings.py:281} DEBUG -
settings.prepare_engine_args(): Using pool settings. pool_size=5,
max_overflow=10, pool_recycle=1800, pid=214
[2021-02-19 03:35:48,636] {cli_action_loggers.py:40} DEBUG - Adding
<function default_action_log at 0x7f7b77054320> to pre execution callback
[2021-02-19 03:35:50,016] {cli_action_loggers.py:66} DEBUG - Calling
callbacks: [<function default_action_log at 0x7f7b77054320>]
/home/airflow/.local/lib/python3.7/site-packages/airflow/cli/commands/dag_command.py:62
PendingDeprecationWarning: --ignore-first-depends-on-past is deprecated as the
value is always set to True
[2021-02-19 03:35:50,050] {dagbag.py:448} INFO - Filling up the DagBag from
/home/airflow/dags
[2021-02-19 03:35:50,051] {dagbag.py:287} DEBUG - Importing
/home/airflow/dags/code-dag.py
/home/airflow/.local/lib/python3.7/site-packages/airflow/providers/cncf/kubernetes/backcompat/backwards_compat_converters.py:26
DeprecationWarning: This module is deprecated. Please use
`kubernetes.client.models.V1Volume`.
/home/airflow/.local/lib/python3.7/site-packages/airflow/providers/cncf/kubernetes/backcompat/backwards_compat_converters.py:27
DeprecationWarning: This module is deprecated. Please use
`kubernetes.client.models.V1VolumeMount`.
[2021-02-19 03:35:50,204] {dagbag.py:413} DEBUG - Loaded DAG <DAG: etl_dag1>
[2021-02-19 03:35:50,205] {dagbag.py:287} DEBUG - Importing
/home/airflow/dags/..data/code-dag.py
[2021-02-19 03:35:50,222] {dagbag.py:413} DEBUG - Loaded DAG <DAG: etl_dag1>
[2021-02-19 03:35:50,223] {dagbag.py:287} DEBUG - Importing
/home/airflow/dags/..2021_02_19_03_05_19.306169647/code-dag.py
[2021-02-19 03:35:50,239] {dagbag.py:413} DEBUG - Loaded DAG <DAG: etl_dag1>
[2021-02-19 03:35:50,240] {executor_loader.py:82} DEBUG - Loading core
executor: KubernetesExecutor
[2021-02-19 03:35:50,398] {kubernetes_executor.py:473} INFO - Start
Kubernetes executor
[2021-02-19 03:35:50,415] {cli_action_loggers.py:84} DEBUG - Calling
callbacks: []
Traceback (most recent call last):
File "/home/airflow/.local/bin/airflow", line 8, in <module>
sys.exit(main())
File
"/home/airflow/.local/lib/python3.7/site-packages/airflow/__main__.py", line
40, in main
args.func(args)
File
"/home/airflow/.local/lib/python3.7/site-packages/airflow/cli/cli_parser.py",
line 48, in command
return func(*args, **kwargs)
File
"/home/airflow/.local/lib/python3.7/site-packages/airflow/utils/cli.py", line
89, in wrapper
return f(*args, **kwargs)
File
"/home/airflow/.local/lib/python3.7/site-packages/airflow/cli/commands/dag_command.py",
line 116, in dag_backfill
run_backwards=args.run_backwards,
File
"/home/airflow/.local/lib/python3.7/site-packages/airflow/models/dag.py", line
1706, in run
job.run()
File
"/home/airflow/.local/lib/python3.7/site-packages/airflow/jobs/base_job.py",
line 237, in run
self._execute()
File
"/home/airflow/.local/lib/python3.7/site-packages/airflow/utils/session.py",
line 65, in wrapper
return func(*args, session=session, **kwargs)
File
"/home/airflow/.local/lib/python3.7/site-packages/airflow/jobs/backfill_job.py",
line 788, in _execute
executor.start()
File
"/home/airflow/.local/lib/python3.7/site-packages/airflow/executors/kubernetes_executor.py",
line 475, in start
raise AirflowException("Could not get scheduler_job_id")
airflow.exceptions.AirflowException: Could not get scheduler_job_id
[2021-02-19 03:35:50,429] {settings.py:292} DEBUG - Disposing DB connection
pool (PID 214)
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
