Hi Hussam, Have you (1) generated Spark jar using sbt/sbt assembl, (2) distributed the Spark jar to the worker machines? It could be that the system expects that Spark jar to be present in /opt/spark-0.8.0/conf:/opt/ spark-0.8.0/assembly/target/scala-2.9.3/spark-assembly_2. 9.3-0.8.0-incubating-hadoop1.0.4.jar in one of the worker machines, but its not finding the jar and hence not finding the necessary class. Can you double-check whether the jar exists in that location in all the worker nodes?
TD On Mon, Jan 20, 2014 at 4:44 PM, <[email protected]> wrote: > Hi, > > > > I am using spark 0.8.0 when hadoop 1.2.1 on Standalone cluster mode with 3 > worker nodes and 1 master. > > > > Can someone help me on this error I am getting when running my app in a > spark cluster ? > > Error: Could not find or load main class > org.apache.spark.executor.CoarseGrainedExecutorBackend > > > > Command on the worker node is > > Spark Executor Command: "java" "-cp" > ":/opt/spark-0.8.0/conf:/opt/spark-0.8.0/assembly/target/scala-2.9.3/spark-assembly_2.9.3-0.8.0-incubating-hadoop1.0.4.jar" > "-Dspark.local.dir=/home/hadoop/spark" > "-Dspark.local.dir=/home/hadoop/spark" "-Dspark.local.dir=/home/hadoop/spark" > "-Dspark.local.dir=/home/hadoop/spark" "-Xms49152M" "-Xmx49152M" > "org.apache.spark.executor.CoarseGrainedExecutorBackend" > "akka://spark@poc1:54483/user/CoarseGrainedScheduler" "2" "poc3" "16" > > > > I checked logs on spark master as well spark workers but not much info > except above error. > > > > Thanks, > > Hussam >
