Hi , I was able to successfully build the project(source code), from intellij. But when i try to run any of the examples present in $SPARK_HOME/examples folder , i am getting different errors for different example jobs.
example: for structuredkafkawordcount example, Exception in thread "main" java.lang.NoClassDefFoundError: scala/collection/Seq at org.apache.spark.examples.sql.streaming.StructuredKafkaWordCount.main(StructuredKafkaWordCount.scala) Caused by: java.lang.ClassNotFoundException: scala.collection.Seq at java.net.URLClassLoader.findClass(URLClassLoader.java:381) at java.lang.ClassLoader.loadClass(ClassLoader.java:424) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335) at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ... 1 more for LogQuery job, objc[21879]: Class JavaLaunchHelper is implemented in both /Library/Java/JavaVirtualMachines/jdk1.8.0_131.jdk/Contents/Home/bin/java (0x106ff54c0) and /Library/Java/JavaVirtualMachines/jdk1.8.0_131.jdk/Contents/Home/jre/lib/libinstrument.dylib (0x1070bd4e0). One of the two will be used. Which one is undefined. Error: A JNI error has occurred, please check your installation and try again Exception in thread "main" java.lang.NoClassDefFoundError: scala/collection/immutable/List at java.lang.Class.getDeclaredMethods0(Native Method) at java.lang.Class.privateGetDeclaredMethods(Class.java:2701) at java.lang.Class.privateGetMethodRecursive(Class.java:3048) at java.lang.Class.getMethod0(Class.java:3018) at java.lang.Class.getMethod(Class.java:1784) at sun.launcher.LauncherHelper.validateMainClass(LauncherHelper.java:544) at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:526) Caused by: java.lang.ClassNotFoundException: scala.collection.immutable.List at java.net.URLClassLoader.findClass(URLClassLoader.java:381) at java.lang.ClassLoader.loadClass(ClassLoader.java:424) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335) at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ... 7 more On Wed, Jun 28, 2017 at 5:21 PM, Dongjoon Hyun <dongjoon.h...@gmail.com> wrote: > Did you follow the guide in `IDE Setup` -> `IntelliJ` section of > http://spark.apache.org/developer-tools.html ? > > Bests, > Dongjoon. > > On Wed, Jun 28, 2017 at 5:13 PM, satyajit vegesna < > satyajit.apas...@gmail.com> wrote: > >> Hi All, >> >> When i try to build source code of apache spark code from >> https://github.com/apache/spark.git, i am getting below errors, >> >> Error:(9, 14) EventBatch is already defined as object EventBatch >> public class EventBatch extends org.apache.avro.specific.SpecificRecordBase >> implements org.apache.avro.specific.SpecificRecord { >> Error:(9, 14) EventBatch is already defined as class EventBatch >> public class EventBatch extends org.apache.avro.specific.SpecificRecordBase >> implements org.apache.avro.specific.SpecificRecord { >> /Users/svegesna/svegesna/dev/scala/spark/external/flume-sink >> /target/scala-2.11/src_managed/main/compiled_avro/org/ >> apache/spark/streaming/flume/sink/SparkFlumeProtocol.java >> Error:(26, 18) SparkFlumeProtocol is already defined as object >> SparkFlumeProtocol >> public interface SparkFlumeProtocol { >> Error:(26, 18) SparkFlumeProtocol is already defined as trait >> SparkFlumeProtocol >> public interface SparkFlumeProtocol { >> /Users/svegesna/svegesna/dev/scala/spark/external/flume-sink >> /target/scala-2.11/src_managed/main/compiled_avro/org/ >> apache/spark/streaming/flume/sink/SparkSinkEvent.java >> Error:(9, 14) SparkSinkEvent is already defined as object SparkSinkEvent >> public class SparkSinkEvent extends >> org.apache.avro.specific.SpecificRecordBase >> implements org.apache.avro.specific.SpecificRecord { >> Error:(9, 14) SparkSinkEvent is already defined as class SparkSinkEvent >> public class SparkSinkEvent extends >> org.apache.avro.specific.SpecificRecordBase >> implements org.apache.avro.specific.SpecificRecord { >> >> Would like to know , if i can successfully build the project, so that i >> can test and debug some of spark's functionalities. >> >> Regards, >> Satyajit. >> > >
