Guoqiang Li created SPARK-2051: ---------------------------------- Summary: In yarn.ClientBase spark.yarn.dist.* do not work Key: SPARK-2051 URL: https://issues.apache.org/jira/browse/SPARK-2051 Project: Spark Issue Type: Bug Components: YARN Reporter: Guoqiang Li
Spark configuration {{conf/spark-defaults.conf}}: {quote} spark.yarn.dist.archives /toona/conf spark.executor.extraClassPath ./conf spark.driver.extraClassPath ./conf {quote} ---- HDFS directory {{hadoop dfs -cat /toona/conf/toona.conf}} : {quote} redis.num=4 {quote} ---- The following command execution fails {code} YARN_CONF_DIR=/etc/hadoop/conf ./bin/spark-submit --num-executors 2 --driver-memory 2g --executor-memory 2g --master yarn-cluster --class toona.DeployTest toona-assembly.jar {code} ---- The following is testing the code {code} package toona import com.typesafe.config.Config import com.typesafe.config.ConfigFactory object DeployTest { def main(args: Array[String]) { val conf = ConfigFactory.load("toona.conf") val redisNum = conf.getInt("redis.num") // Here will throw an `ConfigException` exception assert(redisNum == 4) } } {code} -- This message was sent by Atlassian JIRA (v6.2#6252)