Rafael Gomes Fernandes created AIRFLOW-668:
----------------------------------------------
Summary: Configuration parsing doesn't work properly with python 3
Key: AIRFLOW-668
URL: https://issues.apache.org/jira/browse/AIRFLOW-668
Project: Apache Airflow
Issue Type: Bug
Environment: Airflow version: v1.7.1.3
- Airflow components: webserver and scheduler with a postgres database and
CeleryExecutor
- Python Version: 3.4.5
Reporter: Rafael Gomes Fernandes
The problem is: if you use python3 and the '_cmd' on the config file airflow
will not start due the error:
File
"~/test/env/airflow3/lib/python3.4/site-packages/airflow/configuration.py",
line 447, in _validate
"sqlite" in self.get('core', 'sql_alchemy_conn')):
TypeError: 'str' does not support the buffer interface
To reproduce the problem change the following line on airflow.cfg:
sql_alchemy_conn_cmd = echo sqlite:////~/airflow/airflow.db
The solution is change the following run_command method's line on
configuration.py:
command.split(), stdout=subprocess.PIPE, stderr=subprocess.PIPE,
universal_newlines=True)
By setting the universal_newlines to true the file objects stdout and stderr
are opened as text files and treated as string in python 2 and python 3
avoiding the error.
run_command with universal_newlines=True:
When using python 3 output type: <class 'str'> and no error.
When using python 2 output type: <type 'str'> and no error.
run_command as it is :
When using python 3 output type: <class 'bytes'> and TypeError.
When using python 2 output type: <type 'str'> and no error.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
