I need to tell YARN a JAVA_HOME to use when spawning containers (to run a Java 8 app on Java 7 YARN).
The only way I've found that works is setting SPARK_YARN_USER_ENV="JAVA_HOME=/path/to/java8". The code <https://github.com/apache/spark/blob/b72bb62d421840f82d663c6b8e3922bd14383fbb/yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala#L762> implies that this is deprecated and users should use "the config", but I can't figure out what config is being referenced. Passing "--conf spark.yarn.appMasterEnv.JAVA_HOME=/path/to/java8" seems to set it for the AM but not for executors. Likewise, spark.executor.extraLibraryPath and spark.driver.extraLibraryPath don't appear to set JAVA_HOME (and maybe aren't even supposed to?). The 1.0.1 docs <https://spark.apache.org/docs/1.0.1/running-on-yarn.html#environment-variables> are the last ones to reference the SPARK_YARN_USER_ENV var, afaict. What's the preferred way of passing YARN a custom JAVA_HOME that will be applied to executors' containers? Thanks!
