Hi,
I'm trying the spark on yarn example at
https://spark.incubator.apache.org/docs/latest/running-on-yarn.html
When I try to run the SparkPi example using the spark-class command, the
job fails and In the stderr file of the job logs, I see the following error.
java.lang.ClassNotFoundException:
org.apache.spark.examples.SparkPi$$anonfun$1
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:249)
at
org.apache.spark.serializer.JavaDeserializationStream$$anon$1.resolveClass(JavaSerializer.scala:37)
at
java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1589)
at
java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1494)
at
java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1748)
at
java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1327)
at
java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1969)
at
java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1893)
at
java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1775)
at
java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1327)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:349)
at
org.apache.spark.serializer.JavaDeserializationStream.readObject(JavaSerializer.scala:40)
I think the job is not able to find the
examples/target/scala-2.10/spark-examples-assembly-0.9.0-incubating.jar jar.
Do I need to upload this jar to a specific location?
Thanks,
Terance.
