Github user witgo commented on the pull request:
https://github.com/apache/spark/pull/969#issuecomment-45224988
Spark configuration
`conf/spark-defaults.conf` =>
```
spark.yarn.dist.archives /toona/conf
spark.executor.extraClassPath ./conf
spark.driver.extraClassPath ./conf
```
----------------------------
HDFS directory
`hadoop dfs -cat /toona/conf/toona.conf` =>
```
redis.num=4
```
-----------------------------
The following command execution fails
```shell
YARN_CONF_DIR=/etc/hadoop/conf ./bin/spark-submit --num-executors 2
--driver-memory 2g --executor-memory 2g --master yarn-cluster --class
toona.DeployTest toona-assembly.jar
```
--------------------------------
The following is testing the code
```scala
package toona
import com.typesafe.config.Config
import com.typesafe.config.ConfigFactory
object DeployTest {
def main(args: Array[String]) {
val conf = ConfigFactory.load("toona.conf")
val redisNum = conf.getInt("redis.num") // Here will throw an
`ConfigException` exception
assert(redisNum == 4)
}
}
```
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
