It says that it could be due to incompatible version of scala. Are you using the latest version of scala? I just build spark 0.9.0 yesterday and I installed latest version of scala and sbt and I didn't use this option - "SPARK_HADOOP_VERSION=2.2.0 SPARK_YARN=true", I just did sbt/sbt assembly. After this I was able to start the spark-shell and run a simple example.
Regards, Manu On Wed, Apr 2, 2014 at 3:26 AM, Francis.Hu <francis...@reachjunction.com>wrote: > Hi, All > > > > I stuck in a NoClassDefFoundError. Any helps that would be appreciated. > > I download spark 0.9.0 source, and then run this command to build it : > SPARK_HADOOP_VERSION=2.2.0 SPARK_YARN=true sbt/sbt assembly > > then no error during the build of spark. > > After that I run the spark-shell for testing, it always say below: > > > > ------------------------error after run spark-shell > ------------------------------ > > testuser@ubuntu-1:~/softs/spark-0.9.0-incubating$ ./bin/spark-shell > > > > 14/04/02 00:11:52 INFO HttpServer: Using Spark's default log4j profile: > org/apache/spark/log4j-defaults.properties > > 14/04/02 00:11:52 INFO HttpServer: Starting HTTP Server > > error: > > while compiling: <init> > > during phase: uncurry > > library version: version 2.10.3 > > compiler version: version 2.10.3Z > > reconstructed args: > > > > last tree to typer: EmptyTree > > symbol: null > > symbol definition: null > > tpe: <notype> > > symbol owners: > > context owners: constructor $repl_$init -> class $repl_$init -> > package <empty> > > > > == Enclosing template or block == > > > > "scala" // final package scala, tree.tpe=scala.type > > > > == Expanded type of tree == > > > > <notype> > > > > uncaught exception during compilation: java.lang.NoClassDefFoundError > > > > Failed to initialize compiler: NoClassDefFoundError. > > This is most often remedied by a full clean and recompile. > > Otherwise, your classpath may continue bytecode compiled by > > different and incompatible versions of scala. > > > > java.lang.NoClassDefFoundError: > scala/tools/nsc/transform/UnCurry$UnCurryTransformer$$anonfun$14$$anonfun$apply$5$$anonfun$scala$tools$nsc$transform$UnCurry$UnCurryTransformer$$anonfun$$anonfun$$transformInConstructor$1$1 > > at > scala.tools.nsc.transform.UnCurry$UnCurryTransformer$$anonfun$14$$anonfun$apply$5.scala$tools$nsc$transform$UnCurry$UnCurryTransformer$$anonfun$$anonfun$$transformInConstructor$1(UnCurry.scala:601) > > at > scala.tools.nsc.transform.UnCurry$UnCurryTransformer$$anonfun$14$$anonfun$apply$5$$anonfun$16.apply(UnCurry.scala:604) > > at > scala.tools.nsc.transform.UnCurry$UnCurryTransformer$$anonfun$14$$anonfun$apply$5$$anonfun$16.apply(UnCurry.scala:604) > > at > scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244) > > at > scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244) > > at scala.collection.immutable.List.foreach(List.scala:318) > > at > scala.collection.TraversableLike$class.map(TraversableLike.scala:244) > > at scala.collection.AbstractTraversable.map(Traversable.scala:105) > > at > scala.tools.nsc.transform.UnCurry$UnCurryTransformer$$anonfun$14$$anonfun$apply$5.apply(UnCurry.scala:604) > > at > scala.tools.nsc.transform.UnCurry$UnCurryTransformer$$anonfun$14$$anonfun$apply$5.apply(UnCurry.scala:597) > > at scala.reflect.api.Trees$Transformer.atOwner(Trees.scala:2936) > > at > scala.tools.nsc.transform.TypingTransformers$TypingTransformer.atOwner(TypingTransformers.scala:34) > > at > scala.tools.nsc.transform.UnCurry$UnCurryTransformer.mainTransform(UnCurry.scala:595) > > at > scala.tools.nsc.transform.UnCurry$UnCurryTransformer.transform(UnCurry.scala:122) > > at > scala.tools.nsc.transform.UnCurry$UnCurryTransformer.transform(UnCurry.scala:82) > > at > scala.reflect.api.Trees$Transformer$$anonfun$transformStats$1.apply(Trees.scala:2927) > > at > scala.reflect.api.Trees$Transformer$$anonfun$transformStats$1.apply(Trees.scala:2925) > > at scala.collection.immutable.List.loop$1(List.scala:170) > > at scala.collection.immutable.List.mapConserve(List.scala:186) > > at > scala.reflect.api.Trees$Transformer.transformStats(Trees.scala:2925) > > at scala.reflect.internal.Trees$class.itransform(Trees.scala:1276) > > at > scala.reflect.internal.SymbolTable.itransform(SymbolTable.scala:13) > > at > scala.reflect.internal.SymbolTable.itransform(SymbolTable.scala:13) > > at scala.reflect.api.Trees$Transformer.transform(Trees.scala:2897) > > at > scala.tools.nsc.transform.TypingTransformers$TypingTransformer.scala$tools$nsc$transform$TypingTransformers$TypingTransformer$$super$transform(TypingTransformers.scala:44) > > at > scala.tools.nsc.transform.TypingTransformers$TypingTransformer$$anonfun$transform$1.apply(TypingTransformers.scala:44) > > at > scala.tools.nsc.transform.TypingTransformers$TypingTransformer$$anonfun$transform$1.apply(TypingTransformers.scala:44) > > at scala.reflect.api.Trees$Transformer.atOwner(Trees.scala:2936) > > at > scala.tools.nsc.transform.TypingTransformers$TypingTransformer.atOwner(TypingTransformers.scala:34) > > at > scala.tools.nsc.transform.TypingTransformers$TypingTransformer.transform(TypingTransformers.scala:44) > > at > scala.tools.nsc.transform.UnCurry$UnCurryTransformer.scala$tools$nsc$transform$UnCurry$UnCurryTransformer$$super$transform(UnCurry.scala:613) > > at > scala.tools.nsc.transform.UnCurry$UnCurryTransformer.mainTransform(UnCurry.scala:670) > > at > scala.tools.nsc.transform.UnCurry$UnCurryTransformer.transform(UnCurry.scala:122) > > at > scala.tools.nsc.transform.UnCurry$UnCurryTransformer.transform(UnCurry.scala:82) > > at > scala.reflect.api.Trees$Transformer.transformTemplate(Trees.scala:2904) > > at > scala.reflect.internal.Trees$$anonfun$itransform$4.apply(Trees.scala:1280) > > at > scala.reflect.internal.Trees$$anonfun$itransform$4.apply(Trees.scala:1279) > > at scala.reflect.api.Trees$Transformer.atOwner(Trees.scala:2936) > > at > scala.tools.nsc.transform.TypingTransformers$TypingTransformer.atOwner(TypingTransformers.scala:34) > > at > scala.tools.nsc.transform.TypingTransformers$TypingTransformer.atOwner(TypingTransformers.scala:28) > > at > scala.tools.nsc.transform.TypingTransformers$TypingTransformer.atOwner(TypingTransformers.scala:19) > > at scala.reflect.internal.Trees$class.itransform(Trees.scala:1278) > > at > scala.reflect.internal.SymbolTable.itransform(SymbolTable.scala:13) > > at > scala.reflect.internal.SymbolTable.itransform(SymbolTable.scala:13) > > at scala.reflect.api.Trees$Transformer.transform(Trees.scala:2897) > > at > scala.tools.nsc.transform.TypingTransformers$TypingTransformer.transform(TypingTransformers.scala:48) > > at > scala.tools.nsc.transform.UnCurry$UnCurryTransformer.mainTransform(UnCurry.scala:673) > > at > scala.tools.nsc.transform.UnCurry$UnCurryTransformer.transform(UnCurry.scala:122) > > at > scala.tools.nsc.transform.UnCurry$UnCurryTransformer.transform(UnCurry.scala:82) > > at > scala.reflect.api.Trees$Transformer$$anonfun$transformStats$1.apply(Trees.scala:2927) > > at > scala.reflect.api.Trees$Transformer$$anonfun$transformStats$1.apply(Trees.scala:2925) > > at scala.collection.immutable.List.loop$1(List.scala:170) > > at scala.collection.immutable.List.mapConserve(List.scala:186) > > at > scala.reflect.api.Trees$Transformer.transformStats(Trees.scala:2925) > > at > scala.reflect.internal.Trees$$anonfun$itransform$7.apply(Trees.scala:1298) > > at > scala.reflect.internal.Trees$$anonfun$itransform$7.apply(Trees.scala:1298) > > at scala.reflect.api.Trees$Transformer.atOwner(Trees.scala:2936) > > at > scala.tools.nsc.transform.TypingTransformers$TypingTransformer.atOwner(TypingTransformers.scala:34) > > at > scala.tools.nsc.transform.TypingTransformers$TypingTransformer.atOwner(TypingTransformers.scala:28) > > at > scala.tools.nsc.transform.TypingTransformers$TypingTransformer.atOwner(TypingTransformers.scala:19) > > at scala.reflect.internal.Trees$class.itransform(Trees.scala:1297) > > at > scala.reflect.internal.SymbolTable.itransform(SymbolTable.scala:13) > > at > scala.reflect.internal.SymbolTable.itransform(SymbolTable.scala:13) > > at scala.reflect.api.Trees$Transformer.transform(Trees.scala:2897) > > at > scala.tools.nsc.transform.TypingTransformers$TypingTransformer.scala$tools$nsc$transform$TypingTransformers$TypingTransformer$$super$transform(TypingTransformers.scala:44) > > at > scala.tools.nsc.transform.TypingTransformers$TypingTransformer$$anonfun$transform$2.apply(TypingTransformers.scala:46) > > at > scala.tools.nsc.transform.TypingTransformers$TypingTransformer$$anonfun$transform$2.apply(TypingTransformers.scala:46) > > at scala.reflect.api.Trees$Transformer.atOwner(Trees.scala:2936) > > at > scala.tools.nsc.transform.TypingTransformers$TypingTransformer.atOwner(TypingTransformers.scala:34) > > at > scala.tools.nsc.transform.TypingTransformers$TypingTransformer.transform(TypingTransformers.scala:46) > > at > scala.tools.nsc.transform.UnCurry$UnCurryTransformer.mainTransform(UnCurry.scala:673) > > at > scala.tools.nsc.transform.UnCurry$UnCurryTransformer.transform(UnCurry.scala:122) > > at > scala.tools.nsc.transform.UnCurry$UnCurryTransformer.transform(UnCurry.scala:82) > > at > scala.tools.nsc.ast.Trees$Transformer.transformUnit(Trees.scala:227) > > at > scala.tools.nsc.transform.Transform$Phase.apply(Transform.scala:30) > > at scala.tools.nsc.Global$GlobalPhase.applyPhase(Global.scala:464) > > at > scala.tools.nsc.Global$GlobalPhase$$anonfun$run$1.apply(Global.scala:431) > > at > scala.tools.nsc.Global$GlobalPhase$$anonfun$run$1.apply(Global.scala:431) > > at scala.collection.Iterator$class.foreach(Iterator.scala:727) > > at scala.collection.AbstractIterator.foreach(Iterator.scala:1157) > > at scala.tools.nsc.Global$GlobalPhase.run(Global.scala:431) > > at > scala.tools.nsc.Global$Run.compileUnitsInternal(Global.scala:1583) > > at scala.tools.nsc.Global$Run.compileUnits(Global.scala:1557) > > at scala.tools.nsc.Global$Run.compileSources(Global.scala:1553) > > at org.apache.spark.repl.SparkIMain.org > $apache$spark$repl$SparkIMain$$_initialize(SparkIMain.scala:175) > > at > org.apache.spark.repl.SparkIMain.initializeSynchronous(SparkIMain.scala:196) > > at > org.apache.spark.repl.SparkILoop$$anonfun$process$1.apply$mcZ$sp(SparkILoop.scala:919) > > at > org.apache.spark.repl.SparkILoop$$anonfun$process$1.apply(SparkILoop.scala:876) > > at > org.apache.spark.repl.SparkILoop$$anonfun$process$1.apply(SparkILoop.scala:876) > > at > scala.tools.nsc.util.ScalaClassLoader$.savingContextLoader(ScalaClassLoader.scala:135) > > at org.apache.spark.repl.SparkILoop.process(SparkILoop.scala:876) > > at org.apache.spark.repl.SparkILoop.process(SparkILoop.scala:968) > > at org.apache.spark.repl.Main$.main(Main.scala:31) > > at org.apache.spark.repl.Main.main(Main.scala) > > Caused by: java.lang.ClassNotFoundException: > scala.tools.nsc.transform.UnCurry$UnCurryTransformer$$anonfun$14$$anonfun$apply$5$$anonfun$scala$tools$nsc$transform$UnCurry$UnCurryTransformer$$anonfun$$anonfun$$transformInConstructor$1$1 > > at java.net.URLClassLoader$1.run(URLClassLoader.java:366) > > at java.net.URLClassLoader$1.run(URLClassLoader.java:355) > > at java.security.AccessController.doPrivileged(Native Method) > > at java.net.URLClassLoader.findClass(URLClassLoader.java:354) > > at java.lang.ClassLoader.loadClass(ClassLoader.java:423) > > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) > > at java.lang.ClassLoader.loadClass(ClassLoader.java:356) > > ... 94 more > > > > > > Thanks, > > Franics.Hu > -- Manu Suryavansh
