Based on the jar name you used and the errors you are getting it looks like you may have not built the topology as a fat jar? The jar you pass to `storm jar` should contain all of your dependencies. When you run `mvn package` in that project it will produce two jars in the target dir: storm-kafka-0.8-plus-test-0.1.0-SNAPSHOT.jar and storm-kafka-0.8-plus-test-0.1.0-SNAPSHOT-jar-with-dependencies.jar. You want the "jar-with-dependencies" one. See the instructions in the readme:
run locally https://github.com/wurstmeister/storm-kafka-0.8-plus-test#build-for-running-locally deploy to a cluster https://github.com/wurstmeister/storm-kafka-0.8-plus-test#build-for-running-on-a-storm-cluster On Wed, Apr 16, 2014 at 10:32 AM, M. Solaimani <[email protected]>wrote: > Hi, > > I have copied all the scala file > both scala-library-2.9.2.jar, scala-library.jar into storm's lib folder. > But still get the error :( > > Can anyone tell me if there is any version mismatch issue with storm and > kafka integration? > > > > Best regards, > M Solaimani > > > On Tue, Apr 15, 2014 at 8:44 PM, Xing Yong <[email protected]> wrote: > >> maybe you need scala-library.jar, it will work. >> >> >> 2014-04-16 5:37 GMT+08:00 M. Solaimani <[email protected]>: >> >> Hi, >>> >>> I have copied those jar files in storm lib folder. That errors are gone >>> but I have new error. >>> >>> java.lang.NoClassDefFoundError: scala/runtime/AbstractFunction1$mcLI$sp >>> at java.lang.ClassLoader.defineClass1(Native Method) >>> at java.lang.ClassLoader.defineClass(ClassLoader.java:800) >>> at >>> java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) >>> at java.net.URLClassLoader.defineClass(URLClassLoader.java:449) >>> at java.net.URLClassLoader.access$100(URLClassLoader.java:71) >>> at java.net.URLClassLoader$1.run(URLClassLoader.java:361) >>> at java.net.URLClassLoader$1.run(URLClassLoader.java:355) >>> at java.security.AccessController.doPrivileged(Native Method) >>> at java.net.URLClassLoader.findClass(URLClassLoader.java:354) >>> at java.lang.ClassLoader.loadClass(ClassLoader.java:425) >>> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) >>> at java.lang.ClassLoader.loadClass(ClassLoader.java:358) >>> at >>> kafka.api.OffsetResponse$$anonfun$2$$anonfun$apply$1.apply(OffsetResponse.scala:37) >>> at >>> kafka.api.OffsetResponse$$anonfun$2$$anonfun$apply$1.apply(OffsetResponse.scala:33) >>> at >>> scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:233) >>> at >>> scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:233) >>> at scala.collection.immutable.Range.foreach(Range.scala:81) >>> at scala.collection.TraversableLike$class.map(TraversableLike.scala:233) >>> at scala.collection.immutable.Range.map(Range.scala:46) >>> at kafka.api.OffsetResponse$$anonfun$2.apply(OffsetResponse.scala:33) >>> at kafka.api.OffsetResponse$$anonfun$2.apply(OffsetResponse.scala:30) >>> at >>> scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:239) >>> at >>> scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:239) >>> at scala.collection.immutable.Range.foreach(Range.scala:81) >>> at >>> scala.collection.TraversableLike$class.flatMap(TraversableLike.scala:239) >>> at scala.collection.immutable.Range.flatMap(Range.scala:46) >>> at kafka.api.OffsetResponse$.readFrom(OffsetResponse.scala:30) >>> >>> at >>> kafka.consumer.SimpleConsumer.getOffsetsBefore(SimpleConsumer.scala:125) >>> >>> >>> It seems that this error is related with scala. I copied the scala 2.9.2 >>> jar files inside the storm lib directory but still have this error. Do you >>> guys have any idea to resolve it? >>> >>> With best regards, >>> M Solaimani >>> >>> >>> On Mon, Apr 14, 2014 at 8:51 PM, Xing Yong <[email protected]> wrote: >>> >>>> as the Exception "NoClassDefFoundError: storm/kafka/BrokerHos", i >>>> think the jar of storm-kafka-0.8-plus and jar of kafka should be >>>> placed in the lib of the storm >>>> >>>> >>>> 2014-04-15 5:21 GMT+08:00 M. Solaimani <[email protected]>: >>>> >>>> Hi, >>>>> >>>>> I saw this mail and tried to run the Wurstmiesters >>>>> storm-kafka-0.8-plus test. Here what I did: >>>>> >>>>> 1. I downloaded the Wurstmiesters storm-kafka-0.8-plus test and >>>>> build it by using mvn clean install. >>>>> 2. I had my storm cluster running. (Storm 0.9.0-rc2, Kakfka 0.8) >>>>> 3. I had kafka 0.8 and I run kafka servers, with zoo-keeper and >>>>> created a topic storm-sentence >>>>> >>>>> All the above steps running well. >>>>> >>>>> But When I run Wurstmiesters storm-kafka-0.8-plus test by the >>>>> following commands >>>>> >>>>> storm jar target/storm-kafka-0.8-plus-test-0.1.0-SNAPSHOT.jar >>>>> storm.kafka.KafkaSpoutTestTopology 10.176.148.51 >>>>> sentences 10.176.148.51 >>>>> >>>>> I got the following errors: >>>>> >>>>> Exception in thread "main" java.lang.NoClassDefFoundError: >>>>> storm/kafka/BrokerHos >>>>> ts >>>>> at java.lang.Class.getDeclaredMethods0(Native Method) >>>>> at java.lang.Class.privateGetDeclaredMethods(Class.java:2531) >>>>> at java.lang.Class.getMethod0(Class.java:2774) >>>>> at java.lang.Class.getMethod(Class.java:1663) >>>>> at >>>>> sun.launcher.LauncherHelper.getMainMethod(LauncherHelper.java:494) >>>>> at >>>>> sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:486) >>>>> Caused by: java.lang.ClassNotFoundException: storm.kafka.BrokerHosts >>>>> at java.net.URLClassLoader$1.run(URLClassLoader.java:366) >>>>> at java.net.URLClassLoader$1.run(URLClassLoader.java:355) >>>>> at java.security.AccessController.doPrivileged(Native Method) >>>>> at java.net.URLClassLoader.findClass(URLClassLoader.java:354) >>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:425) >>>>> at >>>>> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) >>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:358) >>>>> ... 6 more >>>>> >>>>> >>>>> At this point I am stuck. I searched the internet and it says that >>>>> there is some version mismatch issue? Can anyone help me please? >>>>> >>>>> With best regards, >>>>> M Solaimani >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> On Mon, Apr 14, 2014 at 11:58 AM, Kashyap Mhaisekar < >>>>> [email protected]> wrote: >>>>> >>>>>> Wurstmiesters seems to work for me. I used storm 0.9.0 rc3 >>>>>> >>>>>> Thanks, >>>>>> Kashyap >>>>>> >>>>>> >>>>>> On Monday, April 14, 2014, Trevor Madge <[email protected]> wrote: >>>>>> >>>>>>> I didn't have any luck with wurstmeister's but I've found this one ( >>>>>>> https://github.com/HolmesNL/kafka-spout) and it works with storm >>>>>>> 0.9.1 and kafka 0.8.1. >>>>>>> >>>>>>> >>>>>>> -Trevor >>>>>>> >>>>>>> >>>>>>> >>>>>>> On Fri, Apr 11, 2014 at 7:22 AM, Kashyap Mhaisekar < >>>>>>> [email protected]> wrote: >>>>>>> >>>>>>>> Thanks Carlos. >>>>>>>> >>>>>>>> >>>>>>>> On Friday, April 11, 2014, Carlos Rodriguez <[email protected]> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> Hi Kashyap, >>>>>>>>> >>>>>>>>> The kafka spout provided in the storm-contrib project ( >>>>>>>>> https://github.com/nathanmarz/storm-contrib/tree/master/storm-kafka) >>>>>>>>> doesnt >>>>>>>>> work with kafka 0.8 >>>>>>>>> >>>>>>>>> The version provided by wurstmeister at >>>>>>>>> https://github.com/wurstmeister/storm-kafka-0.8-plus is supposed >>>>>>>>> to work with storm 0.9.1 and kafka 0.8. >>>>>>>>> >>>>>>>>> Regards, >>>>>>>>> Carlos >>>>>>>>> >>>>>>>>> >>>>>>>>> 2014-04-11 4:17 GMT+02:00 Kashyap Mhaisekar <[email protected]>: >>>>>>>>> >>>>>>>>>> Hi, >>>>>>>>>> What is the version of Kafka spout that works with storm 0.9 and >>>>>>>>>> kafka 0.8? >>>>>>>>>> The standard example doesn't throw exceptions but doesn't seem to >>>>>>>>>> consume messages either. >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> Kashyap >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>> >>>>> >>>> >>> >> >
