Hi All,
We made a couple of failover tests on our Storm cluster and some supervisors
won`t restart at all.
Here is the error found in the supervisor log. Apart from this, our topology
processed as if nothing happened which is really neat (it slowed down a bit but
that`s ok).
Any idea?
Thanks
2014-02-06 09:31:20 o.a.z.ClientCnxn [DEBUG] Reading reply
sessionid:0x34402aa3f530175, packet:: clientPath:null serverPath:null
finished:false header:: 11,5 replyHeader:: 11,4295785979,0 request::
'/storm/supervisors/f90ceddf-4a39-
44c9-b549-f41057688d47,#ffffffacffffffed05737202b6261636b747970652e73746f726d2e6461656d6f6e2e636f6d6d6f6e2e53757065727669736f72496e666f6a24ffffffa04effffff9f2affffffa4ffffff862094c085f5f6578746d6170740124c6a6176612f6c616e672f4f626a656
3743b4c065f5f6d6574617107e014c0d61737369676e6d656e745f69647107e014c08686f73746e616d657107e014c046d6574617107e014c0e7363686564756c65725f6d6574617107e014c0974696d655f736563737107e014c0b757074696d655f736563737107e014c0a757365645f706f7274
737107e01787070707402466393063656464662d346133392d343463392d623534392d6634313035373638386434377401d50756c6c65723034412d70726f642e77616e746564746563682e636f6d7372014636c6f6a7572652e6c616e672e4c617a795365716affffffdc2e13ffffff8bffffffda
ffffff9fffffffa02034c02666e740124c636c6f6a7572652f6c616e672f49466e3b4c0173740134c636c6f6a7572652f6c616e672f495365713b4c0273767107e017872010636c6f6a7572652e6c616e672e4f626ab21613772ffffffd4ffffff83ffffffee2014c055f6d6574617401d4c636c6f
6a7572652f6c616e672f4950657273697374656e744d61703b787070707372018636c6f6a7572652e6c616e672e4368756e6b6564436f6e73267efffffff1ffffffb6fffffff2ffffffc8fffffff2ffffffcf2024c055f6d6f72657107e074c056368756e6b740154c636c6f6a7572652f6c616e67
2f494368756e6b3b7872011636c6f6a7572652e6c616e672e415365715cffffffb73effffffee56ffffff9effffffe2ffffffda200787107e0870737107e05707070707372017636c6f6a7572652e6c616e672e41727261794368756e6bffffff8cffffffc8ffffffdd924bffffffc2ffffffda203
4903656e6449036f66665b056172726179740135b4c6a6176612f6c616e672f4f626a6563743b7870000c000075720135b4c6a6176612e6c616e672e4f626a6563743bffffff90ffffffce58ffffff9f1073296c2007870000c73720116a6176612e6c616e672e496e746567657212ffffffe2ffff
ffa0ffffffa4fffffff7ffffff81ffffff8738201490576616c756578720106a6176612e6c616e672e4e756d626572ffffff86ffffffacffffff951dbffffff94ffffffe0ffffff8b2007870001a2c737107e015001a2d737107e015001a2e737107e015001a2f737107e015001a30737107e01500
1a31737107e015001a32737107e015001a33737107e015001a34737107e015001a35737107e015001a36737107e015001a377070737107e01552fffffff3ffffff9cffffffb873720e6a6176612e6c616e672e4c6f6e673bffffff8bffffffe4ffffff90ffffffccffffff8f23ffffffdf2014a057
6616c7565787107e0160000000170,-1 response::
s{4295785967,4295785979,1391697079310,1391697080148,1,0,0,235316010760208757,994,0,4295785967}
2014-02-06 09:31:20 o.a.z.ClientCnxn [DEBUG] Reading reply
sessionid:0x34402aa3f530175, packet:: clientPath:null serverPath:null
finished:false header:: 12,12 replyHeader:: 12,4295785979,0 request::
'/storm/assignments,T response::
v{'Wed14h25-7-1391628195},s{4294967301,4294967301,1391614136616,1391614136616,0,29,0,0,1,1,4295202953}
2014-02-06 09:31:20 b.s.event [ERROR] Error when processing event
java.lang.RuntimeException: java.io.EOFException
at backtype.storm.utils.Utils.deserialize(Utils.java:69)
~[storm-core-0.9.0.1.jar:na]
at backtype.storm.utils.LocalState.snapshot(LocalState.java:28)
~[storm-core-0.9.0.1.jar:na]
at backtype.storm.utils.LocalState.get(LocalState.java:39)
~[storm-core-0.9.0.1.jar:na]
at
backtype.storm.daemon.supervisor$sync_processes.invoke(supervisor.clj:187)
~[storm-core-0.9.0.1.jar:na]
at clojure.lang.AFn.applyToHelper(AFn.java:161) [clojure-1.4.0.jar:na]
at clojure.lang.AFn.applyTo(AFn.java:151) [clojure-1.4.0.jar:na]
at clojure.core$apply.invoke(core.clj:603) ~[clojure-1.4.0.jar:na]
at clojure.core$partial$fn__4070.doInvoke(core.clj:2343)
~[clojure-1.4.0.jar:na]
at clojure.lang.RestFn.invoke(RestFn.java:397) ~[clojure-1.4.0.jar:na]
at backtype.storm.event$event_manager$fn__3072.invoke(event.clj:24)
~[storm-core-0.9.0.1.jar:na]
at clojure.lang.AFn.run(AFn.java:24) [clojure-1.4.0.jar:na]
at java.lang.Thread.run(Thread.java:662) [na:1.6.0_45]
Caused by: java.io.EOFException: null
at
java.io.ObjectInputStream$PeekInputStream.readFully(ObjectInputStream.java:2303)
~[na:1.6.0_45]
at
java.io.ObjectInputStream$BlockDataInputStream.readShort(ObjectInputStream.java:2772)
~[na:1.6.0_45]
at
java.io.ObjectInputStream.readStreamHeader(ObjectInputStream.java:778)
~[na:1.6.0_45]
at java.io.ObjectInputStream.<init>(ObjectInputStream.java:278)
~[na:1.6.0_45]
at backtype.storm.utils.Utils.deserialize(Utils.java:64)
~[storm-core-0.9.0.1.jar:na]
... 11 common frames omitted
2014-02-06 09:31:20 o.a.z.ClientCnxn [DEBUG] Reading reply
sessionid:0x34402aa3f530175, packet:: clientPath:null serverPath:null
finished:false header:: 13,3 replyHeader:: 13,4295785979,0 request::
'/storm/assignments/Wed14h25-7-139
1628195,T response::
s{4295120264,4295162822,1391628195689,1391631258777,17,0,0,0,47100,0,4295120264}
2014-02-06 09:31:20 b.s.util [INFO] Halting process: ("Error when processing an
event")
