Vianney FOUCAULT created AIRFLOW-1184:
-----------------------------------------
Summary: Contrib Spark Submit Hook does not split argument and
argument value
Key: AIRFLOW-1184
URL: https://issues.apache.org/jira/browse/AIRFLOW-1184
Project: Apache Airflow
Issue Type: Bug
Components: contrib, hooks
Affects Versions: Airflow 2.0, Airflow 1.8
Reporter: Vianney FOUCAULT
Assignee: Vianney FOUCAULT
Fix For: Airflow 2.0, Airflow 1.8
Python Popen expect a list as command. Spark submit too, as:
* ['--option value']
is not the same as
* ['--option', 'value']
in regards of spark. eg spark logs : (yarn logs)
Error: Unknown option --end 2017-05-08
Error: Unknown option --begin 2017-05-07
Error: Unknown option --db_name mydb
Error: Missing option --begin
Error: Missing option --end
Error: Missing option --db_name
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
