[
https://issues.apache.org/jira/browse/AIRFLOW-2380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on AIRFLOW-2380 started by Cristòfol Torrens.
--------------------------------------------------
> Add support for environment variables in Spark submit operator
> --------------------------------------------------------------
>
> Key: AIRFLOW-2380
> URL: https://issues.apache.org/jira/browse/AIRFLOW-2380
> Project: Apache Airflow
> Issue Type: Improvement
> Components: contrib, operators
> Reporter: Cristòfol Torrens
> Assignee: Cristòfol Torrens
> Priority: Minor
>
> Add support for environment variables in Spark submit operator.
> For example, to pass the *HADOOP_CONF_DIR* in case of use same Spark cluster
> with multiple HDFS.
> The idea is to pass as a dict, and resolve it in the case of using
> *yarn-*_client/cluster_*,* and *standalone-*_client_ mode.
> In *standalone-*_cluster_ mode is not possible to do this.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
