JeonChangWon created AIRFLOW-7060:
-------------------------------------
Summary: EXECUTOR variable don't work in airflow.cfg in breeze
shell
Key: AIRFLOW-7060
URL: https://issues.apache.org/jira/browse/AIRFLOW-7060
Project: Apache Airflow
Issue Type: Bug
Components: celery, cli
Affects Versions: 2.0.0
Reporter: JeonChangWon
Assignee: JeonChangWon
Because the default breeze environment set `AIRFLOW__CORE__EXECUTOR` to
`SequentialExecutor`, I can't change it to 'CeleryExecutor' through
airflow.cfg.
Problem 1:
When I execute the command `airflow celery` in the breeze default environment,
an error occurred. Because the `celery` command is added dynamically when the
executor is set to CeleryExecutor by airflow.cfg.
{code:java}
if conf.get("core", "EXECUTOR") == ExecutorLoader.CELERY_EXECUTOR or
BUILD_DOCS: ...{code}
So, I changed airflow.cfg file from SequentialExecutor to CeleryExecutor.
but above error still exist because of the environment variable
`AIRFLOW__CORE__EXECUTOR` which is set to "SequentialExecutor" by
'backend-sqlite.yml' file
Problem 2:
In addition, the below error can occur after I changed
`AIRFLOW__CORE__EXECUTOR` to "CeleryExecutor" because "Celery Executor" don't
use sqlite.
{code:java}
Traceback (most recent call last): File "/usr/local/bin/airflow", line 7, in
<module> exec(compile(f.read(), __file__, 'exec')) File
"/opt/airflow/airflow/bin/airflow", line 26, in <module> from airflow.bin.cli
import CLIFactory File "/opt/airflow/airflow/__init__.py", line 34, in <module>
from airflow import settings File "/opt/airflow/airflow/settings.py", line 34,
in <module> from airflow.configuration import AIRFLOW_HOME, WEBSERVER_CONFIG,
conf # NOQA F401 File "/opt/airflow/airflow/configuration.py", line 602, in
<module> conf.read(AIRFLOW_CONFIG) File
"/opt/airflow/airflow/configuration.py", line 309, in read self._validate()
File "/opt/airflow/airflow/configuration.py", line 185, in _validate
self.get('core', 'executor'))) airflow.exceptions.AirflowConfigException:
error: cannot use sqlite with the CeleryExecutor{code}
Solution 2:
I execute breeze command with an option like
{code:java}
./breeze -i redis -b postgres{code}
And change airflow.cfg SequentialExecutor to CeleryExecutor.
I finally could use the celery command and the celery worker works well.
I will send PR for Problem 1
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
