I have a solution in mind which will make serialization/de-serialization consistent with storm 0.10. Patch will come by today or tomorrow. You can track it here https://issues.apache.org/jira/browse/STORM-1853
On Fri, May 20, 2016 at 6:27 PM, Rama Nallamilli <[email protected]> wrote: > I have managed to work around this issue for now, I am not really sure > what the serialVersionUID of Scala Map is, but found some reports that > Scala Map may not be fully serializable so I switched it out for > java.util.HashMap. > > The issue has now changed to a ClassDefNotFoundException for our bolts and > spouts. It seems that the Storm LocalCluster class loader cannot see the > locally build source classes that are expelled during the compile phase to > the */target/scala-2.11/classes* directory. Is anybody aware of any > changes that may have stopped this working in the newer versions of Storm > as it could resolve source built class files fine in version 0.10.0? > > ERROR org.apache.storm.daemon.worker - Error on initialization of server > mk-worker > > java.lang.RuntimeException: java.lang.ClassNotFoundException: > aggregator.spouts.MySpout > > at org.apache.storm.utils.Utils.javaDeserialize(Utils.java:181) > ~[storm-core-1.0.1.jar:1.0.1] > > at org.apache.storm.utils.Utils.getSetComponentObject(Utils.java:430) > ~[storm-core-1.0.1.jar:1.0.1] > > at org.apache.storm.daemon.task$get_task_object.invoke(task.clj:74) > ~[storm-core-1.0.1.jar:1.0.1] > > at > org.apache.storm.daemon.task$mk_task_data$fn__7601.invoke(task.clj:177) > ~[storm-core-1.0.1.jar:1.0.1] > > at org.apache.storm.util$assoc_apply_self.invoke(util.clj:930) > ~[storm-core-1.0.1.jar:1.0.1] > > > > On 20 May 2016 at 05:02, Abhishek Agarwal <[email protected]> wrote: > >> This is a different issue possibly caused by the same library. What is >> the actual serialVersionUID of this class? >> On May 20, 2016 4:43 AM, "Rajasekhar" <[email protected]> wrote: >> >>> Hi, >>> >>> Its known issue with latest version of storm. >>> >>> https://issues.apache.org/jira/browse/STORM-1773 >>> >>> On Thu, May 19, 2016 at 2:57 AM, Rama Nallamilli <[email protected]> >>> wrote: >>> >>>> Hi there, >>>> >>>> I attempted an upgrade from Storm 0.10.0 -> 1.0.1, most of it went >>>> smoothly up until the last hurdle of submitting the topology, we are trying >>>> to submit to a Local Cluster in memory (our tests do this) but are get the >>>> following serialization errors when running. >>>> >>>> We are using Scala 2.11 and build with sbt, the tests do work fine from >>>> IntelliJ which is unusual. Java version for building topology is 1.8, >>>> Scala version is 2_11. Works fine on Storm 0.10.0. >>>> >>>> Anybody else had this issue or could point to any possible causes? >>>> >>>> >>>> 2016-05-18 16:18:42,587 ERROR org.apache.storm.daemon.worker - Error >>>> on initialization of server mk-worker >>>> java.lang.RuntimeException: java.io.InvalidClassException: >>>> scala.collection.immutable.Map$Map1; local class incompatible: stream >>>> classdesc serialVersionUID = -331641707862204208, local class >>>> serialVersionUID = 6416318198292458305 >>>> at org.apache.storm.utils.Utils.javaDeserialize(Utils.java:179) >>>> ~[storm-core-1.0.1.jar:1.0.1] >>>> at org.apache.storm.utils.Utils.getSetComponentObject(Utils.java:430) >>>> ~[storm-core-1.0.1.jar:1.0.1] >>>> at org.apache.storm.daemon.task$get_task_object.invoke(task.clj:74) >>>> ~[storm-core-1.0.1.jar:1.0.1] >>>> at >>>> org.apache.storm.daemon.task$mk_task_data$fn__7601.invoke(task.clj:177) >>>> ~[storm-core-1.0.1.jar:1.0.1] >>>> at org.apache.storm.util$assoc_apply_self.invoke(util.clj:930) >>>> ~[storm-core-1.0.1.jar:1.0.1] >>>> at org.apache.storm.daemon.task$mk_task_data.invoke(task.clj:170) >>>> ~[storm-core-1.0.1.jar:1.0.1] >>>> at org.apache.storm.daemon.task$mk_task.invoke(task.clj:181) >>>> ~[storm-core-1.0.1.jar:1.0.1] >>>> at >>>> org.apache.storm.daemon.executor$mk_executor$fn__7820.invoke(executor.clj:371) >>>> ~[storm-core-1.0.1.jar:1.0.1] >>>> at clojure.core$map$fn__4553.invoke(core.clj:2622) >>>> ~[clojure-1.7.0.jar:na] >>>> at clojure.lang.LazySeq.sval(LazySeq.java:40) [clojure-1.7.0.jar:na] >>>> at clojure.lang.LazySeq.seq(LazySeq.java:49) [clojure-1.7.0.jar:na] >>>> at clojure.lang.RT.seq(RT.java:507) [clojure-1.7.0.jar:na] >>>> at clojure.core$seq__4128.invoke(core.clj:137) [clojure-1.7.0.jar:na] >>>> at clojure.core.protocols$seq_reduce.invoke(protocols.clj:30) >>>> ~[clojure-1.7.0.jar:na] >>>> at clojure.core.protocols$fn__6506.invoke(protocols.clj:101) >>>> ~[clojure-1.7.0.jar:na] >>>> at >>>> clojure.core.protocols$fn__6452$G__6447__6465.invoke(protocols.clj:13) >>>> ~[clojure-1.7.0.jar:na] >>>> at clojure.core$reduce.invoke(core.clj:6519) ~[clojure-1.7.0.jar:na] >>>> at clojure.core$into.invoke(core.clj:6600) ~[clojure-1.7.0.jar:na] >>>> at >>>> org.apache.storm.daemon.executor$mk_executor.invoke(executor.clj:372) >>>> ~[storm-core-1.0.1.jar:1.0.1] >>>> at >>>> org.apache.storm.daemon.worker$fn__8450$exec_fn__2461__auto__$reify__8452$iter__8457__8461$fn__8462.invoke(worker.clj:637) >>>> ~[storm-core-1.0.1.jar:1.0.1] >>>> at clojure.lang.LazySeq.sval(LazySeq.java:40) [clojure-1.7.0.jar:na] >>>> at clojure.lang.LazySeq.seq(LazySeq.java:49) [clojure-1.7.0.jar:na] >>>> at clojure.lang.RT.seq(RT.java:507) [clojure-1.7.0.jar:na] >>>> at clojure.core$seq__4128.invoke(core.clj:137) [clojure-1.7.0.jar:na] >>>> at clojure.core$dorun.invoke(core.clj:3009) [clojure-1.7.0.jar:na] >>>> at clojure.core$doall.invoke(core.clj:3025) [clojure-1.7.0.jar:na] >>>> at >>>> org.apache.storm.daemon.worker$fn__8450$exec_fn__2461__auto__$reify__8452.run(worker.clj:637) >>>> ~[storm-core-1.0.1.jar:1.0.1] >>>> at java.security.AccessController.doPrivileged(Native Method) >>>> ~[na:1.8.0_66] >>>> at javax.security.auth.Subject.doAs(Subject.java:422) ~[na:1.8.0_66] >>>> at >>>> org.apache.storm.daemon.worker$fn__8450$exec_fn__2461__auto____8451.invoke(worker.clj:609) >>>> ~[storm-core-1.0.1.jar:1.0.1] >>>> at clojure.lang.AFn.applyToHelper(AFn.java:178) [clojure-1.7.0.jar:na] >>>> at clojure.lang.AFn.applyTo(AFn.java:144) [clojure-1.7.0.jar:na] >>>> at clojure.core$apply.invoke(core.clj:630) ~[clojure-1.7.0.jar:na] >>>> at >>>> org.apache.storm.daemon.worker$fn__8450$mk_worker__8545.doInvoke(worker.clj:583) >>>> [storm-core-1.0.1.jar:1.0.1] >>>> at clojure.lang.RestFn.invoke(RestFn.java:512) [clojure-1.7.0.jar:na] >>>> at >>>> org.apache.storm.daemon.supervisor$fn__9365.invoke(supervisor.clj:1200) >>>> [storm-core-1.0.1.jar:1.0.1] >>>> at clojure.lang.MultiFn.invoke(MultiFn.java:251) [clojure-1.7.0.jar:na] >>>> at >>>> org.apache.storm.daemon.supervisor$get_valid_new_worker_ids$iter__8926__8930$fn__8931.invoke(supervisor.clj:380) >>>> [storm-core-1.0.1.jar:1.0.1] >>>> at clojure.lang.LazySeq.sval(LazySeq.java:40) [clojure-1.7.0.jar:na] >>>> at clojure.lang.LazySeq.seq(LazySeq.java:49) [clojure-1.7.0.jar:na] >>>> at clojure.lang.RT.seq(RT.java:507) [clojure-1.7.0.jar:na] >>>> at clojure.core$seq__4128.invoke(core.clj:137) [clojure-1.7.0.jar:na] >>>> at clojure.core$dorun.invoke(core.clj:3009) [clojure-1.7.0.jar:na] >>>> at clojure.core$doall.invoke(core.clj:3025) [clojure-1.7.0.jar:na] >>>> at >>>> org.apache.storm.daemon.supervisor$get_valid_new_worker_ids.invoke(supervisor.clj:367) >>>> [storm-core-1.0.1.jar:1.0.1] >>>> at >>>> org.apache.storm.daemon.supervisor$sync_processes.invoke(supervisor.clj:428) >>>> [storm-core-1.0.1.jar:1.0.1] >>>> at clojure.core$partial$fn__4527.invoke(core.clj:2492) >>>> [clojure-1.7.0.jar:na] >>>> at org.apache.storm.event$event_manager$fn__8630.invoke(event.clj:40) >>>> [storm-core-1.0.1.jar:1.0.1] >>>> at clojure.lang.AFn.run(AFn.java:22) [clojure-1.7.0.jar:na] >>>> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_66] >>>> Caused by: java.io.InvalidClassException: >>>> scala.collection.immutable.Map$Map1; local class incompatible: stream >>>> classdesc serialVersionUID = -331641707862204208, local class >>>> serialVersionUID = 6416318198292458305 >>>> at java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:616) >>>> ~[na:1.8.0_66] >>>> at >>>> java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1623) >>>> ~[na:1.8.0_66] >>>> at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1518) >>>> ~[na:1.8.0_66] >>>> at >>>> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1774) >>>> ~[na:1.8.0_66] >>>> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351) >>>> ~[na:1.8.0_66] >>>> at >>>> java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2000) >>>> ~[na:1.8.0_66] >>>> at >>>> java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1891) >>>> ~[na:1.8.0_66] >>>> at >>>> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801) >>>> ~[na:1.8.0_66] >>>> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351) >>>> ~[na:1.8.0_66] >>>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:371) >>>> ~[na:1.8.0_66] >>>> at org.apache.storm.utils.Utils.javaDeserialize(Utils.java:175) >>>> ~[storm-core-1.0.1.jar:1.0.1] >>>> ... 49 common frames omitted >>>> >>>> >>>> Kind Regards >>>> Rama >>>> >>> >>> >>> >>> -- >>> Thanks & Regards >>> Rajasekhar >>> >> > -- Regards, Abhishek Agarwal
