[ https://issues.apache.org/jira/browse/SPARK-2051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14019560#comment-14019560 ]
Guoqiang Li commented on SPARK-2051: ------------------------------------ The PR: https://github.com/apache/spark/pull/969 > In yarn.ClientBase spark.yarn.dist.* do not work > ------------------------------------------------ > > Key: SPARK-2051 > URL: https://issues.apache.org/jira/browse/SPARK-2051 > Project: Spark > Issue Type: Bug > Components: YARN > Reporter: Guoqiang Li > > Spark configuration > {{conf/spark-defaults.conf}}: > {quote} > spark.yarn.dist.archives /toona/conf > spark.executor.extraClassPath ./conf > spark.driver.extraClassPath ./conf > {quote} > ---- > HDFS directory > {{hadoop dfs -cat /toona/conf/toona.conf}} : > {quote} > redis.num=4 > {quote} > ---- > The following command execution fails > {code} > YARN_CONF_DIR=/etc/hadoop/conf ./bin/spark-submit --num-executors 2 > --driver-memory 2g --executor-memory 2g --master yarn-cluster --class > toona.DeployTest toona-assembly.jar > {code} > ---- > The following is the test code > {code} > package toona > import com.typesafe.config.Config > import com.typesafe.config.ConfigFactory > object DeployTest { > def main(args: Array[String]) { > val conf = ConfigFactory.load("toona.conf") > val redisNum = conf.getInt("redis.num") // Here will throw an > `ConfigException` exception > assert(redisNum == 4) > } > } > {code} -- This message was sent by Atlassian JIRA (v6.2#6252)