Xiangrui Meng created SPARK-1870:
------------------------------------
Summary: Jars specified via --jars in spark-submit are not added
to executor classpath for YARN
Key: SPARK-1870
URL: https://issues.apache.org/jira/browse/SPARK-1870
Project: Spark
Issue Type: Bug
Components: YARN
Affects Versions: 1.0.0
Reporter: Xiangrui Meng
Priority: Critical
With `spark-submit`, jars specified via `--jars` are added to distributed cache
in `yarn-cluster` mode. The executor should add cached jars to classpath.
However,
{code}
sc.parallelize(0 to 10, 10).map { i =>
System.getProperty("java.class.path")
}.collect().foreach(println)
{code}
shows only system jars, `app.jar`, and `spark.jar` but not other jars in the
distributed cache.
The workaround is using assembly jar.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
