[
https://issues.apache.org/jira/browse/SPARK-1604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13979994#comment-13979994
]
Kan Zhang edited comment on SPARK-1604 at 4/24/14 5:38 PM:
-----------------------------------------------------------
Ah, that could be the reason. I was using sbt assemble-deps and then package to
build. Just verified, when building the normal sbt assembly jar, problem
disappears. Could be an issue with the former build sequence.
Moving this to BUILD.
was (Author: kzhang):
Ah, that could be the reason. I was using sbt assemble-deps and then package to
build. Just verified, when building the normal sbt assembly jar, problem
disappears. Could be a problem with the former build sequence.
Moving this BUILD.
> Couldn't run spark-submit with yarn cluster mode when using deps jar
> --------------------------------------------------------------------
>
> Key: SPARK-1604
> URL: https://issues.apache.org/jira/browse/SPARK-1604
> Project: Spark
> Issue Type: Bug
> Components: Build
> Affects Versions: 1.0.0
> Reporter: Kan Zhang
> Priority: Blocker
>
> SPARK_JAR=./assembly/target/scala-2.10/spark-assembly-1.0.0-SNAPSHOT-hadoop2.3.0-deps.jar
> ./bin/spark-submit
> ./examples/target/scala-2.10/spark-examples_2.10-1.0.0-SNAPSHOT.jar --master
> yarn --deploy-mode cluster --class org.apache.spark.examples.sql.JavaSparkSQL
> Exception in thread "main" java.lang.ClassNotFoundException:
> org.apache.spark.deploy.yarn.Client
> at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:270)
> at org.apache.spark.deploy.SparkSubmit$.launch(SparkSubmit.scala:234)
> at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:47)
> at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
--
This message was sent by Atlassian JIRA
(v6.2#6252)
