MaxTaggart opened a new pull request #14160:
URL: https://github.com/apache/airflow/pull/14160
related: #13805
This is to address issue #13805 where when using the Kubernetes executor
backfill jobs or jobs triggered from the UI will produce this error:
```
[2021-01-27 06:35:50,376] {kubernetes_executor.py:491} INFO - Start
Kubernetes executor
Traceback (most recent call last):
File "/home/airflow/.local/bin/airflow", line 8, in <module>
sys.exit(main())
File
"/home/airflow/.local/lib/python3.7/site-packages/airflow/__main__.py", line
40, in main
args.func(args)
File
"/home/airflow/.local/lib/python3.7/site-packages/airflow/cli/cli_parser.py",
line 48, in command
return func(*args, **kwargs)
File
"/home/airflow/.local/lib/python3.7/site-packages/airflow/utils/cli.py", line
89, in wrapper
return f(*args, **kwargs)
File
"/home/airflow/.local/lib/python3.7/site-packages/airflow/cli/commands/dag_command.py",
line 116, in dag_backfill
run_backwards=args.run_backwards,
File
"/home/airflow/.local/lib/python3.7/site-packages/airflow/models/dag.py", line
1701, in run
job.run()
File
"/home/airflow/.local/lib/python3.7/site-packages/airflow/jobs/base_job.py",
line 237, in run
self._execute()
File
"/home/airflow/.local/lib/python3.7/site-packages/airflow/utils/session.py",
line 65, in wrapper
return func(*args, session=session, **kwargs)
File
"/home/airflow/.local/lib/python3.7/site-packages/airflow/jobs/backfill_job.py",
line 788, in _execute
executor.start()
File
"/home/airflow/.local/lib/python3.7/site-packages/airflow/executors/kubernetes_executor.py",
line 493, in start
raise AirflowException("Could not get scheduler_job_id")
airflow.exceptions.AirflowException: Could not get scheduler_job_id
```
The fix is to simply add a hardcoded executor `job_id` in
`airflow/jobs/backfill_job.py` and `airflow/www/views.py`.
The changes still need tests, but I figured that this PR would be a good
place to discuss where those tests should live (I'm not familiar with the
testing landscape yet).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
