Hi Manu, I do not see any 0.10.0 versions being downloaded in my ~/.m2 repo:
ls ~/.m2/repository/org/apache/storm/* ~/.m2/repository/org/apache/storm/flux: 1.0.2 ~/.m2/repository/org/apache/storm/flux-core: 1.0.2 ~/.m2/repository/org/apache/storm/storm: 1.0.2 ~/.m2/repository/org/apache/storm/storm-kafka: 1.0.2 And yet surprisingly, when I do `mvn clean install` and explore the jar file, I see two versions of KeyValueSchemeAsMultiScheme.class jar -xvf my_shaded_fat_jar.jar find . -name KeyValueSchemeAsMultiScheme.class ./org/apache/storm/kafka/KeyValueSchemeAsMultiScheme.class ./storm/kafka/KeyValueSchemeAsMultiScheme.class On Mon, Sep 12, 2016 at 5:56 PM, Manu Zhang <[email protected]> wrote: > Hi, > > Could you share your maven file ? When running `mvn package`, you should > see what and which version of jar files are included ? > That may give you a hint. > > Thanks, > Manu > > > On Tue, Sep 13, 2016 at 6:50 AM S G <[email protected]> wrote: > > > From the > > https://github.com/apache/storm/tree/1.0.x-branch/external/storm-kafka > > docs, > > > > "This means that pre 1.0 kafka spouts will not work with Storm versions > 1.0 > > and higher. While running topologies in Storm version 1.0 and higher, it > > must be ensured that the storm-kafka version is at least 1.0. Pre 1.0 > > shaded topology jars that bundles storm-kafka classes must be rebuilt > with > > storm-kafka version 1.0 for running in clusters with storm 1.0 and > higher." > > > > Any idea how do I ensure that? > > I have done a thorough checking of my pom.xmls and also ran grep > > expressions. > > > > On Mon, Sep 12, 2016 at 3:44 PM, S G <[email protected]> wrote: > > > > > Can anyone help with this? > > > > > > I tried relocating classes too during shading as: > > > <relocations> > > > <relocation> > > > <pattern>backtype.storm</pattern> > > > <shadedPattern>org.apache.storm</shadedPattern> > > > </relocation> > > > <relocation> > > > <pattern>storm.trident</pattern> > > > <shadedPattern>org.apache.storm.trident</shadedPattern> > > > </relocation> > > > <relocation> > > > <pattern>org.apache.thrift7</pattern> > > > <shadedPattern>org.apache.storm.thrift</shadedPattern> > > > </relocation> > > > </relocations> > > > > > > But then the deployed topology began to complain about: > > > java.lang.NoSuchMethodError: org.apache.storm.spout. > > > MultiScheme.deserialize([B)Ljava/lang/Iterable; at > > storm.kafka.KafkaUtils. > > > generateTuples(KafkaUtils.java:206) > > > > > > So it seems that I really need to get the 1.0.2 storm-kafka into my > > > project to make it work. > > > > > > I even tried to exclude storm-kafka everywhere from all the > dependencies > > > my project had and then including it as an explicit dependency, but it > > > still did not work. > > > > > > mvn dependency:tree shows that storm-kafka is nowhere being included > as a > > > transitive dependency in my project. > > > > > > What to do now? > > > > > > Thx, > > > SG > > > > > > On Mon, Sep 12, 2016 at 12:24 PM, S G <[email protected]> > wrote: > > > > > >> Hi, > > >> > > >> We are upgrading to Storm 1.0.2 and have done all the package name > > >> changes. > > >> > > >> grep -ri backtype in our code repo returns nothing. > > >> > > >> Yet on deploying the jar to Storm, following error is seen: > > >> > > >> > > >> storm/storm_installed/apache-storm-1.0.2/bin/storm \ > > >> jar \ > > >> $jarFile \ > > >> org.apache.storm.flux.Flux \ > > >> $yamlFileTarget \ > > >> --remote > > >> > > >> > > >> > > >> +- Apache Storm -+ > > >> +- data FLow User eXperience -+ > > >> Version: 1.0.2 > > >> Parsing file: topology-config.yaml > > >> 333 [main] INFO o.a.s.f.p.FluxParser - loading YAML from input > > stream... > > >> 335 [main] INFO o.a.s.f.p.FluxParser - Not performing property > > >> substitution. > > >> 335 [main] INFO o.a.s.f.p.FluxParser - Not performing environment > > >> variable substitution. > > >> Exception in thread "main" java.lang.NoClassDefFoundError: > > >> backtype/storm/spout/MultiScheme > > >> at java.lang.Class.forName0(Native Method) > > >> at java.lang.Class.forName(Class.java:264) > > >> at org.apache.storm.flux.FluxBuilder.buildObject( > FluxBuilder.java:291) > > >> at org.apache.storm.flux.FluxBuilder.buildComponents(FluxBuilde > > >> r.java:350) > > >> at org.apache.storm.flux.FluxBuilder.buildTopology( > FluxBuilder.java:75) > > >> at org.apache.storm.flux.Flux.runCli(Flux.java:153) > > >> at org.apache.storm.flux.Flux.main(Flux.java:98) > > >> Caused by: java.lang.ClassNotFoundException: > > >> backtype.storm.spout.MultiScheme > > >> at java.net.URLClassLoader.findClass(URLClassLoader.java:381) > > >> at java.lang.ClassLoader.loadClass(ClassLoader.java:424) > > >> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331) > > >> at java.lang.ClassLoader.loadClass(ClassLoader.java:357) > > >> ... 7 more > > >> > > >> > > >> > > >> This used to work fine with 0.10.0 > > >> > > >> I have checked all the places in my code including mvn > dependency:tree's > > >> output > > >> All versions are 1.0.2 (storm and flux) and there is no mention of > > >> backtype anywhere in the code, flux-file or the jar we are deploying. > > >> > > >> What is wrong here? > > >> > > >> Thanks > > >> SG > > >> > > > > > > > > >
