[
https://issues.apache.org/jira/browse/AIRFLOW-1191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16747996#comment-16747996
]
ASF subversion and git services commented on AIRFLOW-1191:
----------------------------------------------------------
Commit 39dab6f4019f8107814104257274f3c7e78efeaa in airflow's branch
refs/heads/master from Holden Karau
[ https://gitbox.apache.org/repos/asf?p=airflow.git;h=39dab6f ]
[AIRFLOW-1191] Simplify override of spark submit command. (#4360)
Adds a spark_binary param to the spark submit operator to
allow folks to more easily configure the operator to use a
different binary, as is needed for some distros of the
Hadoop ecosystem which ship multiple version of Spark.
> Contrib Spark Submit hook should permit override of spark-submit cmd
> --------------------------------------------------------------------
>
> Key: AIRFLOW-1191
> URL: https://issues.apache.org/jira/browse/AIRFLOW-1191
> Project: Apache Airflow
> Issue Type: Improvement
> Components: contrib, hooks
> Affects Versions: 1.8.1
> Environment: Cloudera based Spark parcel
> Reporter: Vianney FOUCAULT
> Assignee: Vianney FOUCAULT
> Priority: Major
> Fix For: 1.10.0
>
>
> Using Cloudera based cluster with spark 2 parcel that rename spark-submit to
> spark2-submit
> It should be possible to change the spark submit cmd without specifying a env
> var
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
