Cristòfol Torrens created AIRFLOW-2380:
------------------------------------------
Summary: Add support for environment variables in Spark submit
operator
Key: AIRFLOW-2380
URL: https://issues.apache.org/jira/browse/AIRFLOW-2380
Project: Apache Airflow
Issue Type: Improvement
Components: contrib, operators
Reporter: Cristòfol Torrens
Assignee: Cristòfol Torrens
Add support for environment variables in Spark submit operator.
For example, to pass the *HADOOP_CONF_DIR* in case of use same Spark cluster
with multiple HDFS.
The idea is to pass as a dict, and resolve it in the case of using
*yarn-*_client/cluster_*,* and *standalone-*_client_ mode.
In *standalone-*_cluster_ mode is not possible to do this.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
