I have managed to work around this issue for now, I am not really sure what
the serialVersionUID of Scala Map is, but found some reports that Scala Map
may not be fully serializable so I switched it out for java.util.HashMap.
The issue has now changed to a ClassDefNotFoundException for our bolts and
spouts. It seems that the Storm LocalCluster class loader cannot see the
locally build source classes that are expelled during the compile phase to
the */target/scala-2.11/classes* directory. Is anybody aware of any
changes that may have stopped this working in the newer versions of Storm
as it could resolve source built class files fine in version 0.10.0?
ERROR org.apache.storm.daemon.worker - Error on initialization of
server mk-worker
java.lang.RuntimeException: java.lang.ClassNotFoundException:
aggregator.spouts.MySpout
at org.apache.storm.utils.Utils.javaDeserialize(Utils.java:181)
~[storm-core-1.0.1.jar:1.0.1]
at org.apache.storm.utils.Utils.getSetComponentObject(Utils.java:430)
~[storm-core-1.0.1.jar:1.0.1]
at org.apache.storm.daemon.task$get_task_object.invoke(task.clj:74)
~[storm-core-1.0.1.jar:1.0.1]
at org.apache.storm.daemon.task$mk_task_data$fn__7601.invoke(task.clj:177)
~[storm-core-1.0.1.jar:1.0.1]
at org.apache.storm.util$assoc_apply_self.invoke(util.clj:930)
~[storm-core-1.0.1.jar:1.0.1]
On 20 May 2016 at 05:02, Abhishek Agarwal <[email protected]> wrote:
> This is a different issue possibly caused by the same library. What is the
> actual serialVersionUID of this class?
> On May 20, 2016 4:43 AM, "Rajasekhar" <[email protected]> wrote:
>
>> Hi,
>>
>> Its known issue with latest version of storm.
>>
>> https://issues.apache.org/jira/browse/STORM-1773
>>
>> On Thu, May 19, 2016 at 2:57 AM, Rama Nallamilli <[email protected]>
>> wrote:
>>
>>> Hi there,
>>>
>>> I attempted an upgrade from Storm 0.10.0 -> 1.0.1, most of it went
>>> smoothly up until the last hurdle of submitting the topology, we are trying
>>> to submit to a Local Cluster in memory (our tests do this) but are get the
>>> following serialization errors when running.
>>>
>>> We are using Scala 2.11 and build with sbt, the tests do work fine from
>>> IntelliJ which is unusual. Java version for building topology is 1.8,
>>> Scala version is 2_11. Works fine on Storm 0.10.0.
>>>
>>> Anybody else had this issue or could point to any possible causes?
>>>
>>>
>>> 2016-05-18 16:18:42,587 ERROR org.apache.storm.daemon.worker - Error on
>>> initialization of server mk-worker
>>> java.lang.RuntimeException: java.io.InvalidClassException:
>>> scala.collection.immutable.Map$Map1; local class incompatible: stream
>>> classdesc serialVersionUID = -331641707862204208, local class
>>> serialVersionUID = 6416318198292458305
>>> at org.apache.storm.utils.Utils.javaDeserialize(Utils.java:179)
>>> ~[storm-core-1.0.1.jar:1.0.1]
>>> at org.apache.storm.utils.Utils.getSetComponentObject(Utils.java:430)
>>> ~[storm-core-1.0.1.jar:1.0.1]
>>> at org.apache.storm.daemon.task$get_task_object.invoke(task.clj:74)
>>> ~[storm-core-1.0.1.jar:1.0.1]
>>> at
>>> org.apache.storm.daemon.task$mk_task_data$fn__7601.invoke(task.clj:177)
>>> ~[storm-core-1.0.1.jar:1.0.1]
>>> at org.apache.storm.util$assoc_apply_self.invoke(util.clj:930)
>>> ~[storm-core-1.0.1.jar:1.0.1]
>>> at org.apache.storm.daemon.task$mk_task_data.invoke(task.clj:170)
>>> ~[storm-core-1.0.1.jar:1.0.1]
>>> at org.apache.storm.daemon.task$mk_task.invoke(task.clj:181)
>>> ~[storm-core-1.0.1.jar:1.0.1]
>>> at
>>> org.apache.storm.daemon.executor$mk_executor$fn__7820.invoke(executor.clj:371)
>>> ~[storm-core-1.0.1.jar:1.0.1]
>>> at clojure.core$map$fn__4553.invoke(core.clj:2622)
>>> ~[clojure-1.7.0.jar:na]
>>> at clojure.lang.LazySeq.sval(LazySeq.java:40) [clojure-1.7.0.jar:na]
>>> at clojure.lang.LazySeq.seq(LazySeq.java:49) [clojure-1.7.0.jar:na]
>>> at clojure.lang.RT.seq(RT.java:507) [clojure-1.7.0.jar:na]
>>> at clojure.core$seq__4128.invoke(core.clj:137) [clojure-1.7.0.jar:na]
>>> at clojure.core.protocols$seq_reduce.invoke(protocols.clj:30)
>>> ~[clojure-1.7.0.jar:na]
>>> at clojure.core.protocols$fn__6506.invoke(protocols.clj:101)
>>> ~[clojure-1.7.0.jar:na]
>>> at
>>> clojure.core.protocols$fn__6452$G__6447__6465.invoke(protocols.clj:13)
>>> ~[clojure-1.7.0.jar:na]
>>> at clojure.core$reduce.invoke(core.clj:6519) ~[clojure-1.7.0.jar:na]
>>> at clojure.core$into.invoke(core.clj:6600) ~[clojure-1.7.0.jar:na]
>>> at org.apache.storm.daemon.executor$mk_executor.invoke(executor.clj:372)
>>> ~[storm-core-1.0.1.jar:1.0.1]
>>> at
>>> org.apache.storm.daemon.worker$fn__8450$exec_fn__2461__auto__$reify__8452$iter__8457__8461$fn__8462.invoke(worker.clj:637)
>>> ~[storm-core-1.0.1.jar:1.0.1]
>>> at clojure.lang.LazySeq.sval(LazySeq.java:40) [clojure-1.7.0.jar:na]
>>> at clojure.lang.LazySeq.seq(LazySeq.java:49) [clojure-1.7.0.jar:na]
>>> at clojure.lang.RT.seq(RT.java:507) [clojure-1.7.0.jar:na]
>>> at clojure.core$seq__4128.invoke(core.clj:137) [clojure-1.7.0.jar:na]
>>> at clojure.core$dorun.invoke(core.clj:3009) [clojure-1.7.0.jar:na]
>>> at clojure.core$doall.invoke(core.clj:3025) [clojure-1.7.0.jar:na]
>>> at
>>> org.apache.storm.daemon.worker$fn__8450$exec_fn__2461__auto__$reify__8452.run(worker.clj:637)
>>> ~[storm-core-1.0.1.jar:1.0.1]
>>> at java.security.AccessController.doPrivileged(Native Method)
>>> ~[na:1.8.0_66]
>>> at javax.security.auth.Subject.doAs(Subject.java:422) ~[na:1.8.0_66]
>>> at
>>> org.apache.storm.daemon.worker$fn__8450$exec_fn__2461__auto____8451.invoke(worker.clj:609)
>>> ~[storm-core-1.0.1.jar:1.0.1]
>>> at clojure.lang.AFn.applyToHelper(AFn.java:178) [clojure-1.7.0.jar:na]
>>> at clojure.lang.AFn.applyTo(AFn.java:144) [clojure-1.7.0.jar:na]
>>> at clojure.core$apply.invoke(core.clj:630) ~[clojure-1.7.0.jar:na]
>>> at
>>> org.apache.storm.daemon.worker$fn__8450$mk_worker__8545.doInvoke(worker.clj:583)
>>> [storm-core-1.0.1.jar:1.0.1]
>>> at clojure.lang.RestFn.invoke(RestFn.java:512) [clojure-1.7.0.jar:na]
>>> at
>>> org.apache.storm.daemon.supervisor$fn__9365.invoke(supervisor.clj:1200)
>>> [storm-core-1.0.1.jar:1.0.1]
>>> at clojure.lang.MultiFn.invoke(MultiFn.java:251) [clojure-1.7.0.jar:na]
>>> at
>>> org.apache.storm.daemon.supervisor$get_valid_new_worker_ids$iter__8926__8930$fn__8931.invoke(supervisor.clj:380)
>>> [storm-core-1.0.1.jar:1.0.1]
>>> at clojure.lang.LazySeq.sval(LazySeq.java:40) [clojure-1.7.0.jar:na]
>>> at clojure.lang.LazySeq.seq(LazySeq.java:49) [clojure-1.7.0.jar:na]
>>> at clojure.lang.RT.seq(RT.java:507) [clojure-1.7.0.jar:na]
>>> at clojure.core$seq__4128.invoke(core.clj:137) [clojure-1.7.0.jar:na]
>>> at clojure.core$dorun.invoke(core.clj:3009) [clojure-1.7.0.jar:na]
>>> at clojure.core$doall.invoke(core.clj:3025) [clojure-1.7.0.jar:na]
>>> at
>>> org.apache.storm.daemon.supervisor$get_valid_new_worker_ids.invoke(supervisor.clj:367)
>>> [storm-core-1.0.1.jar:1.0.1]
>>> at
>>> org.apache.storm.daemon.supervisor$sync_processes.invoke(supervisor.clj:428)
>>> [storm-core-1.0.1.jar:1.0.1]
>>> at clojure.core$partial$fn__4527.invoke(core.clj:2492)
>>> [clojure-1.7.0.jar:na]
>>> at org.apache.storm.event$event_manager$fn__8630.invoke(event.clj:40)
>>> [storm-core-1.0.1.jar:1.0.1]
>>> at clojure.lang.AFn.run(AFn.java:22) [clojure-1.7.0.jar:na]
>>> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_66]
>>> Caused by: java.io.InvalidClassException:
>>> scala.collection.immutable.Map$Map1; local class incompatible: stream
>>> classdesc serialVersionUID = -331641707862204208, local class
>>> serialVersionUID = 6416318198292458305
>>> at java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:616)
>>> ~[na:1.8.0_66]
>>> at
>>> java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1623)
>>> ~[na:1.8.0_66]
>>> at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1518)
>>> ~[na:1.8.0_66]
>>> at
>>> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1774)
>>> ~[na:1.8.0_66]
>>> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351)
>>> ~[na:1.8.0_66]
>>> at
>>> java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2000)
>>> ~[na:1.8.0_66]
>>> at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1891)
>>> ~[na:1.8.0_66]
>>> at
>>> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801)
>>> ~[na:1.8.0_66]
>>> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351)
>>> ~[na:1.8.0_66]
>>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:371)
>>> ~[na:1.8.0_66]
>>> at org.apache.storm.utils.Utils.javaDeserialize(Utils.java:175)
>>> ~[storm-core-1.0.1.jar:1.0.1]
>>> ... 49 common frames omitted
>>>
>>>
>>> Kind Regards
>>> Rama
>>>
>>
>>
>>
>> --
>> Thanks & Regards
>> Rajasekhar
>>
>
