[
https://issues.apache.org/jira/browse/AIRFLOW-1191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16030030#comment-16030030
]
ASF subversion and git services commented on AIRFLOW-1191:
----------------------------------------------------------
Commit d06ab68f2c83ad5dce3cae1c5aa9a9a9f32cf934 in incubator-airflow's branch
refs/heads/master from [~vfoucault]
[ https://git-wip-us.apache.org/repos/asf?p=incubator-airflow.git;h=d06ab68 ]
[AIRFLOW-1191] : SparkSubmitHook custom cmd
Add the capability to set the spark-submit binary to call.
The default behaviour set the spark-submit command to
'spark-submit', or to set it via a Spark env var.
the spark binary can now be set in the spark connection.
Test coverage extended for the new settings.
> Contrib Spark Submit hook should permit override of spark-submit cmd
> --------------------------------------------------------------------
>
> Key: AIRFLOW-1191
> URL: https://issues.apache.org/jira/browse/AIRFLOW-1191
> Project: Apache Airflow
> Issue Type: Improvement
> Components: contrib, hooks
> Affects Versions: Airflow 2.0, Airflow 1.8
> Environment: Cloudera based Spark parcel
> Reporter: Vianney FOUCAULT
> Assignee: Vianney FOUCAULT
> Fix For: Airflow 2.0, 1.8.1
>
>
> Using Cloudera based cluster with spark 2 parcel that rename spark-submit to
> spark2-submit
> It should be possible to change the spark submit cmd without specifying a env
> var
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
