I have seen this as well and thought I was going nuts. In my testing I could reliably reproduce it in local mode against 0.9.1-incubating.
What I noticed in my testing: - It only occurred when a custom logback.xml was on the class path. - More specifically, it only happened when there was an “appender-ref” child element in the “root”. Commenting out the “appender-ref” element made it go away. - It only happened with Trident topologies. It did not happen with regular storm topologies. It is clearly a classloader issue, but I have no idea what the root cause is. My theory is that it is related to the fact that in 0.9.1 we had both the clojure source code as well as the AOT-compiled clojure classes in the storm-core jar. In 0.9.2-SNAPSHOT (latest master branch), we no longer package the clojure source code alongside the AOT-compiled classes. In 0.9.2, with the patch for STORM-290 applied, this problem goes away. Not absolutely sure why, but I’m glad it does. ;) - Taylor On Apr 21, 2014, at 9:09 PM, Adam Lewis <[email protected]> wrote: > The tests are in the test tree (which I don't think eclipse cares about, but > should get treated properly on the command line). The scope wasn't provided, > it was defaulted to runtime...and I've now corrected that and it doesn't > help. > > Strangely, if I explicitly exclude logback-classic within the storm > dependency, it does work as long as I also add a dependency on > slf4j-simple...so I am still seeing weird logger induced classpath > issues...so this is what is working for me: > > <dependency> > <groupId>org.apache.storm</groupId> > <artifactId>storm-core</artifactId> > <exclusions> > <exclusion> > <artifactId>logback-classic</artifactId> > <groupId>ch.qos.logback</groupId> > </exclusion> > </exclusions> > </dependency> > <dependency> > <groupId>org.slf4j</groupId> > <artifactId>slf4j-simple</artifactId> > </dependency> > > WEIRD! > > > > On Mon, Apr 21, 2014 at 8:54 PM, Jon Logan <[email protected]> wrote: > Are your maven scopes right? The scope of the Storm dependency should be > provided -- not runtime. Also be sure that your main method / unit test is > under your test/ classpath, not your main/ classpath. > > > On Mon, Apr 21, 2014 at 8:49 PM, Adam Lewis <[email protected]> wrote: > Are there other things that could cause this error? Since upgrading to > 0.9.1-incubating, I've hit it twice. The first time I resolved it (in one > project) by fixing an issue where two slf4j bindings were on the classpath > together (strange, but it worked)...now I'm hitting the problem again in a > different project and can't figure out what is causing the problem. This is > for a test which is submitting a topology to a LocalCluster; the full trace > follows (happens launching JUnit from Eclipse and from Maven command line) > > java.lang.RuntimeException: java.lang.ClassNotFoundException: > backtype.storm.daemon.nimbus$normalize_conf$get_merged_conf_val__3916$fn__3917 > at backtype.storm.utils.Utils.deserialize(Utils.java:88) > at backtype.storm.daemon.nimbus$read_storm_conf.invoke(nimbus.clj:89) > at backtype.storm.daemon.nimbus$start_storm.invoke(nimbus.clj:724) > at > backtype.storm.daemon.nimbus$eval3974$exec_fn__1459__auto__$reify__3987.submitTopologyWithOpts(nimbus.clj:962) > at > backtype.storm.daemon.nimbus$eval3974$exec_fn__1459__auto__$reify__3987.submitTopology(nimbus.clj:971) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at clojure.lang.Reflector.invokeMatchingMethod(Reflector.java:93) > at clojure.lang.Reflector.invokeInstanceMethod(Reflector.java:28) > at backtype.storm.testing$submit_local_topology.invoke(testing.clj:253) > at > backtype.storm.LocalCluster$_submitTopology.invoke(LocalCluster.clj:34) > at backtype.storm.LocalCluster.submitTopology(Unknown Source) > at > com.acuitysds.trident.TestTimeseriesAssembly.testBasicTopology(TestTimeseriesAssembly.java:108) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at > org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47) > at > org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) > at > org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44) > at > org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) > at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271) > at > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70) > at > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50) > at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238) > at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63) > at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236) > at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53) > at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229) > at org.junit.runners.ParentRunner.run(ParentRunner.java:309) > at > org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:50) > at > org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38) > at > org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:467) > at > org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:683) > at > org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:390) > at > org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:197) > Caused by: java.lang.ClassNotFoundException: > backtype.storm.daemon.nimbus$normalize_conf$get_merged_conf_val__3916$fn__3917 > at java.net.URLClassLoader$1.run(URLClassLoader.java:366) > at java.net.URLClassLoader$1.run(URLClassLoader.java:355) > at java.security.AccessController.doPrivileged(Native Method) > at java.net.URLClassLoader.findClass(URLClassLoader.java:354) > at java.lang.ClassLoader.loadClass(ClassLoader.java:424) > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) > at java.lang.ClassLoader.loadClass(ClassLoader.java:357) > at java.lang.Class.forName0(Native Method) > at java.lang.Class.forName(Class.java:270) > at java.io.ObjectInputStream.resolveClass(ObjectInputStream.java:623) > at > java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1610) > at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1515) > at > java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1769) > at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1348) > at > java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1989) > at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1913) > at > java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1796) > at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1348) > at > java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1989) > at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1913) > at > java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1796) > at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1348) > at > java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1989) > at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1913) > at > java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1796) > at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1348) > at > java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1989) > at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1913) > at > java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1796) > at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1348) > at > java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1989) > at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1913) > at > java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1796) > at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1348) > at > java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1989) > at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1913) > at > java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1796) > at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1348) > at > java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1989) > at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1913) > at > java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1796) > at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1348) > at > java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1989) > at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1913) > at > java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1796) > at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1348) > at > java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1989) > at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1913) > at > java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1796) > at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1348) > at > java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1989) > at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1913) > at > java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1796) > at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1348) > at > java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1989) > at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1913) > at > java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1796) > at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1348) > at java.io.ObjectInputStream.readArray(ObjectInputStream.java:1704) > at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1342) > at > java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1989) > at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1913) > at > java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1796) > at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1348) > at java.io.ObjectInputStream.readObject(ObjectInputStream.java:370) > at backtype.storm.utils.Utils.deserialize(Utils.java:82) > ... 37 more > > > > On Mon, Mar 10, 2014 at 12:34 PM, James Hardy <[email protected]> wrote: > It was indeed a classpath/maven issue. Thank you Nathan! > > > On Mon, Mar 10, 2014 at 2:36 PM, Nathan Leung <[email protected]> wrote: > These appear to classes generated from clojure. Are you building against the > same version of storm that you are using to run the application? > > > On Mon, Mar 10, 2014 at 8:07 AM, James Hardy <[email protected]> wrote: > Hi > > I'm hoping someone can help. I've been learning Storm for the past few hours > and everything has been great however I'm getting a > java.lang.ClassNotFoundException: > backtype.storm.daemon.nimbus$normalize_conf$get_merged_conf_val__3916$fn__3917 > exception when trying to submit my topology to a local cluster. > > I have a simple spout which emits the string "go" and the ExclamationBolt > taken from the examples. I wanted to create a pipeline of bolts and so > created a topology: > > TopologyBuilder builder = new TopologyBuilder(); > builder.setSpout("test", new Spout()); > builder.setBolt("ex1", new ExclamationBolt()).shuffleGrouping("test"); > builder.setBolt("ex2", new ExclamationBolt()).shuffleGrouping("ex1"); > > This works fine no problem at all however when I add another bolt I get the > "Topology submission exception": > > TopologyBuilder builder = new TopologyBuilder(); > builder.setSpout("test", new Spout()); > builder.setBolt("ex1", new ExclamationBolt()).shuffleGrouping("test"); > builder.setBolt("ex2", new ExclamationBolt()).shuffleGrouping("ex1"); > builder.setBolt("ex3", new ExclamationBolt()).shuffleGrouping("ex2"); > > If anyone has any ideas they would be really appreciated. I feel like I've > hit a brick wall :/ > > Thanks > > James > > > > >
signature.asc
Description: Message signed with OpenPGP using GPGMail
