Repository: spark Updated Branches: refs/heads/master 5a3371883 -> bdcd6e4c6
[SPARK-20995][CORE] Spark-env.sh.template' should add 'YARN_CONF_DIR' configuration instructions. ## What changes were proposed in this pull request? Ensure that `HADOOP_CONF_DIR` or `YARN_CONF_DIR` points to the directory which contains the (client side) configuration files for the Hadoop cluster. These configs are used to write to HDFS and connect to the YARN ResourceManager. The configuration contained in this directory will be distributed to the YARN cluster so that all containers used by the application use the same configuration. Sometimes, `HADOOP_CONF_DIR` is set to the hdfs configuration file path. So, YARN_CONF_DIR should be set to the yarn configuration file path. My project configuration item of 'spark-env.sh ' is as follows:  'HADOOP_CONF_DIR' configuration file path. List the relevant documents below:  'YARN_CONF_DIR' configuration file path. List the relevant documents below:  So, 'Spark-env.sh.template' should add 'YARN_CONF_DIR' configuration instructions. ## How was this patch tested? manual tests Please review http://spark.apache.org/contributing.html before opening a pull request. Author: guoxiaolong <guo.xiaolo...@zte.com.cn> Author: éå°é¾ 10207633 <guo.xiaolo...@zte.com.cn> Author: guoxiaolongzte <guo.xiaolo...@zte.com.cn> Closes #18212 from guoxiaolongzte/SPARK-20995. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/bdcd6e4c Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/bdcd6e4c Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/bdcd6e4c Branch: refs/heads/master Commit: bdcd6e4c680ebd3ddf5c1baaeba31134b143dfb4 Parents: 5a33718 Author: guoxiaolong <guo.xiaolo...@zte.com.cn> Authored: Fri Jun 9 09:26:30 2017 +0100 Committer: Sean Owen <so...@cloudera.com> Committed: Fri Jun 9 09:26:30 2017 +0100 ---------------------------------------------------------------------- conf/spark-env.sh.template | 1 + 1 file changed, 1 insertion(+) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/spark/blob/bdcd6e4c/conf/spark-env.sh.template ---------------------------------------------------------------------- diff --git a/conf/spark-env.sh.template b/conf/spark-env.sh.template index b7c985a..b9aab5a 100755 --- a/conf/spark-env.sh.template +++ b/conf/spark-env.sh.template @@ -34,6 +34,7 @@ # Options read in YARN client mode # - HADOOP_CONF_DIR, to point Spark towards Hadoop configuration files +# - YARN_CONF_DIR, to point Spark towards YARN configuration files when you use YARN # - SPARK_EXECUTOR_CORES, Number of cores for the executors (Default: 1). # - SPARK_EXECUTOR_MEMORY, Memory per Executor (e.g. 1000M, 2G) (Default: 1G) # - SPARK_DRIVER_MEMORY, Memory for Driver (e.g. 1000M, 2G) (Default: 1G) --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
