Hi, Tested by applying your patch commit over the 1.0.2 storm branch and can confirm that it did fix the ClassDefNotFound and Serailisation issue we was seeing.
Thanks Rama On 23 May 2016 10:34, "Abhishek Agarwal" <[email protected]> wrote: > can you try with the patch I have put in JIRA and see if it resolves the > issue? > > On Fri, May 20, 2016 at 6:49 PM, Abhishek Agarwal <[email protected]> > wrote: > >> I have a solution in mind which will make serialization/de-serialization >> consistent with storm 0.10. Patch will come by today or tomorrow. You can >> track it here >> https://issues.apache.org/jira/browse/STORM-1853 >> >> On Fri, May 20, 2016 at 6:27 PM, Rama Nallamilli <[email protected]> >> wrote: >> >>> I have managed to work around this issue for now, I am not really sure >>> what the serialVersionUID of Scala Map is, but found some reports that >>> Scala Map may not be fully serializable so I switched it out for >>> java.util.HashMap. >>> >>> The issue has now changed to a ClassDefNotFoundException for our bolts >>> and spouts. It seems that the Storm LocalCluster class loader cannot see >>> the locally build source classes that are expelled during the compile phase >>> to the */target/scala-2.11/classes* directory. Is anybody aware of any >>> changes that may have stopped this working in the newer versions of Storm >>> as it could resolve source built class files fine in version 0.10.0? >>> >>> ERROR org.apache.storm.daemon.worker - Error on initialization of server >>> mk-worker >>> >>> java.lang.RuntimeException: java.lang.ClassNotFoundException: >>> aggregator.spouts.MySpout >>> >>> at org.apache.storm.utils.Utils.javaDeserialize(Utils.java:181) >>> ~[storm-core-1.0.1.jar:1.0.1] >>> >>> at org.apache.storm.utils.Utils.getSetComponentObject(Utils.java:430) >>> ~[storm-core-1.0.1.jar:1.0.1] >>> >>> at org.apache.storm.daemon.task$get_task_object.invoke(task.clj:74) >>> ~[storm-core-1.0.1.jar:1.0.1] >>> >>> at >>> org.apache.storm.daemon.task$mk_task_data$fn__7601.invoke(task.clj:177) >>> ~[storm-core-1.0.1.jar:1.0.1] >>> >>> at org.apache.storm.util$assoc_apply_self.invoke(util.clj:930) >>> ~[storm-core-1.0.1.jar:1.0.1] >>> >>> >>> >>> On 20 May 2016 at 05:02, Abhishek Agarwal <[email protected]> wrote: >>> >>>> This is a different issue possibly caused by the same library. What is >>>> the actual serialVersionUID of this class? >>>> On May 20, 2016 4:43 AM, "Rajasekhar" <[email protected]> >>>> wrote: >>>> >>>>> Hi, >>>>> >>>>> Its known issue with latest version of storm. >>>>> >>>>> https://issues.apache.org/jira/browse/STORM-1773 >>>>> >>>>> On Thu, May 19, 2016 at 2:57 AM, Rama Nallamilli <[email protected]> >>>>> wrote: >>>>> >>>>>> Hi there, >>>>>> >>>>>> I attempted an upgrade from Storm 0.10.0 -> 1.0.1, most of it went >>>>>> smoothly up until the last hurdle of submitting the topology, we are >>>>>> trying >>>>>> to submit to a Local Cluster in memory (our tests do this) but are get >>>>>> the >>>>>> following serialization errors when running. >>>>>> >>>>>> We are using Scala 2.11 and build with sbt, the tests do work fine >>>>>> from IntelliJ which is unusual. Java version for building topology is >>>>>> 1.8, >>>>>> Scala version is 2_11. Works fine on Storm 0.10.0. >>>>>> >>>>>> Anybody else had this issue or could point to any possible causes? >>>>>> >>>>>> >>>>>> 2016-05-18 16:18:42,587 ERROR org.apache.storm.daemon.worker - Error >>>>>> on initialization of server mk-worker >>>>>> java.lang.RuntimeException: java.io.InvalidClassException: >>>>>> scala.collection.immutable.Map$Map1; local class incompatible: stream >>>>>> classdesc serialVersionUID = -331641707862204208, local class >>>>>> serialVersionUID = 6416318198292458305 >>>>>> at org.apache.storm.utils.Utils.javaDeserialize(Utils.java:179) >>>>>> ~[storm-core-1.0.1.jar:1.0.1] >>>>>> at org.apache.storm.utils.Utils.getSetComponentObject(Utils.java:430) >>>>>> ~[storm-core-1.0.1.jar:1.0.1] >>>>>> at org.apache.storm.daemon.task$get_task_object.invoke(task.clj:74) >>>>>> ~[storm-core-1.0.1.jar:1.0.1] >>>>>> at >>>>>> org.apache.storm.daemon.task$mk_task_data$fn__7601.invoke(task.clj:177) >>>>>> ~[storm-core-1.0.1.jar:1.0.1] >>>>>> at org.apache.storm.util$assoc_apply_self.invoke(util.clj:930) >>>>>> ~[storm-core-1.0.1.jar:1.0.1] >>>>>> at org.apache.storm.daemon.task$mk_task_data.invoke(task.clj:170) >>>>>> ~[storm-core-1.0.1.jar:1.0.1] >>>>>> at org.apache.storm.daemon.task$mk_task.invoke(task.clj:181) >>>>>> ~[storm-core-1.0.1.jar:1.0.1] >>>>>> at >>>>>> org.apache.storm.daemon.executor$mk_executor$fn__7820.invoke(executor.clj:371) >>>>>> ~[storm-core-1.0.1.jar:1.0.1] >>>>>> at clojure.core$map$fn__4553.invoke(core.clj:2622) >>>>>> ~[clojure-1.7.0.jar:na] >>>>>> at clojure.lang.LazySeq.sval(LazySeq.java:40) [clojure-1.7.0.jar:na] >>>>>> at clojure.lang.LazySeq.seq(LazySeq.java:49) [clojure-1.7.0.jar:na] >>>>>> at clojure.lang.RT.seq(RT.java:507) [clojure-1.7.0.jar:na] >>>>>> at clojure.core$seq__4128.invoke(core.clj:137) [clojure-1.7.0.jar:na] >>>>>> at clojure.core.protocols$seq_reduce.invoke(protocols.clj:30) >>>>>> ~[clojure-1.7.0.jar:na] >>>>>> at clojure.core.protocols$fn__6506.invoke(protocols.clj:101) >>>>>> ~[clojure-1.7.0.jar:na] >>>>>> at >>>>>> clojure.core.protocols$fn__6452$G__6447__6465.invoke(protocols.clj:13) >>>>>> ~[clojure-1.7.0.jar:na] >>>>>> at clojure.core$reduce.invoke(core.clj:6519) ~[clojure-1.7.0.jar:na] >>>>>> at clojure.core$into.invoke(core.clj:6600) ~[clojure-1.7.0.jar:na] >>>>>> at >>>>>> org.apache.storm.daemon.executor$mk_executor.invoke(executor.clj:372) >>>>>> ~[storm-core-1.0.1.jar:1.0.1] >>>>>> at >>>>>> org.apache.storm.daemon.worker$fn__8450$exec_fn__2461__auto__$reify__8452$iter__8457__8461$fn__8462.invoke(worker.clj:637) >>>>>> ~[storm-core-1.0.1.jar:1.0.1] >>>>>> at clojure.lang.LazySeq.sval(LazySeq.java:40) [clojure-1.7.0.jar:na] >>>>>> at clojure.lang.LazySeq.seq(LazySeq.java:49) [clojure-1.7.0.jar:na] >>>>>> at clojure.lang.RT.seq(RT.java:507) [clojure-1.7.0.jar:na] >>>>>> at clojure.core$seq__4128.invoke(core.clj:137) [clojure-1.7.0.jar:na] >>>>>> at clojure.core$dorun.invoke(core.clj:3009) [clojure-1.7.0.jar:na] >>>>>> at clojure.core$doall.invoke(core.clj:3025) [clojure-1.7.0.jar:na] >>>>>> at >>>>>> org.apache.storm.daemon.worker$fn__8450$exec_fn__2461__auto__$reify__8452.run(worker.clj:637) >>>>>> ~[storm-core-1.0.1.jar:1.0.1] >>>>>> at java.security.AccessController.doPrivileged(Native Method) >>>>>> ~[na:1.8.0_66] >>>>>> at javax.security.auth.Subject.doAs(Subject.java:422) ~[na:1.8.0_66] >>>>>> at >>>>>> org.apache.storm.daemon.worker$fn__8450$exec_fn__2461__auto____8451.invoke(worker.clj:609) >>>>>> ~[storm-core-1.0.1.jar:1.0.1] >>>>>> at clojure.lang.AFn.applyToHelper(AFn.java:178) [clojure-1.7.0.jar:na] >>>>>> at clojure.lang.AFn.applyTo(AFn.java:144) [clojure-1.7.0.jar:na] >>>>>> at clojure.core$apply.invoke(core.clj:630) ~[clojure-1.7.0.jar:na] >>>>>> at >>>>>> org.apache.storm.daemon.worker$fn__8450$mk_worker__8545.doInvoke(worker.clj:583) >>>>>> [storm-core-1.0.1.jar:1.0.1] >>>>>> at clojure.lang.RestFn.invoke(RestFn.java:512) [clojure-1.7.0.jar:na] >>>>>> at >>>>>> org.apache.storm.daemon.supervisor$fn__9365.invoke(supervisor.clj:1200) >>>>>> [storm-core-1.0.1.jar:1.0.1] >>>>>> at clojure.lang.MultiFn.invoke(MultiFn.java:251) >>>>>> [clojure-1.7.0.jar:na] >>>>>> at >>>>>> org.apache.storm.daemon.supervisor$get_valid_new_worker_ids$iter__8926__8930$fn__8931.invoke(supervisor.clj:380) >>>>>> [storm-core-1.0.1.jar:1.0.1] >>>>>> at clojure.lang.LazySeq.sval(LazySeq.java:40) [clojure-1.7.0.jar:na] >>>>>> at clojure.lang.LazySeq.seq(LazySeq.java:49) [clojure-1.7.0.jar:na] >>>>>> at clojure.lang.RT.seq(RT.java:507) [clojure-1.7.0.jar:na] >>>>>> at clojure.core$seq__4128.invoke(core.clj:137) [clojure-1.7.0.jar:na] >>>>>> at clojure.core$dorun.invoke(core.clj:3009) [clojure-1.7.0.jar:na] >>>>>> at clojure.core$doall.invoke(core.clj:3025) [clojure-1.7.0.jar:na] >>>>>> at >>>>>> org.apache.storm.daemon.supervisor$get_valid_new_worker_ids.invoke(supervisor.clj:367) >>>>>> [storm-core-1.0.1.jar:1.0.1] >>>>>> at >>>>>> org.apache.storm.daemon.supervisor$sync_processes.invoke(supervisor.clj:428) >>>>>> [storm-core-1.0.1.jar:1.0.1] >>>>>> at clojure.core$partial$fn__4527.invoke(core.clj:2492) >>>>>> [clojure-1.7.0.jar:na] >>>>>> at org.apache.storm.event$event_manager$fn__8630.invoke(event.clj:40) >>>>>> [storm-core-1.0.1.jar:1.0.1] >>>>>> at clojure.lang.AFn.run(AFn.java:22) [clojure-1.7.0.jar:na] >>>>>> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_66] >>>>>> Caused by: java.io.InvalidClassException: >>>>>> scala.collection.immutable.Map$Map1; local class incompatible: stream >>>>>> classdesc serialVersionUID = -331641707862204208, local class >>>>>> serialVersionUID = 6416318198292458305 >>>>>> at java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:616) >>>>>> ~[na:1.8.0_66] >>>>>> at >>>>>> java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1623) >>>>>> ~[na:1.8.0_66] >>>>>> at >>>>>> java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1518) >>>>>> ~[na:1.8.0_66] >>>>>> at >>>>>> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1774) >>>>>> ~[na:1.8.0_66] >>>>>> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351) >>>>>> ~[na:1.8.0_66] >>>>>> at >>>>>> java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2000) >>>>>> ~[na:1.8.0_66] >>>>>> at >>>>>> java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1891) >>>>>> ~[na:1.8.0_66] >>>>>> at >>>>>> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801) >>>>>> ~[na:1.8.0_66] >>>>>> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351) >>>>>> ~[na:1.8.0_66] >>>>>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:371) >>>>>> ~[na:1.8.0_66] >>>>>> at org.apache.storm.utils.Utils.javaDeserialize(Utils.java:175) >>>>>> ~[storm-core-1.0.1.jar:1.0.1] >>>>>> ... 49 common frames omitted >>>>>> >>>>>> >>>>>> Kind Regards >>>>>> Rama >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> Thanks & Regards >>>>> Rajasekhar >>>>> >>>> >>> >> >> >> -- >> Regards, >> Abhishek Agarwal >> >> > > > -- > Regards, > Abhishek Agarwal > >
