Hi Greg, I'm not sure exactly what it is that you're trying to achieve, but I'm pretty sure those variables are not supposed to be set by users. You should take a look at the documentation for "spark.driver.extraClassPath" and "spark.driver.extraLibraryPath", and the equivalent options for executors. (The driver ones also have direct equivalents in the spark-submit command line, check its help output.)
Since you're running on Yarn, you might also want to take a look at https://issues.apache.org/jira/browse/SPARK-1719 (and SPARK-1720). On Tue, Oct 14, 2014 at 11:57 AM, Greg Hill <greg.h...@rackspace.com> wrote: > It seems to me that SPARK_SUBMIT_CLASSPATH does not follow the same ability > as other tools to put wildcards in the paths you add. For some reason it > doesn't pick up the classpath information from yarn-site.xml either, it > seems, when running on YARN. I'm having to manually add every single > dependency JAR. There must be a better way, so what am I missing? > > Greg > -- Marcelo --------------------------------------------------------------------- To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
