Andres Gomez Ferrer created STORM-434:
-----------------------------------------
Summary: Supervisor dies with "[ERROR] Error when processing even"
Key: STORM-434
URL: https://issues.apache.org/jira/browse/STORM-434
Project: Apache Storm (Incubating)
Issue Type: Bug
Affects Versions: 0.9.1-incubating
Environment: CentOS 1.6
storm-core-0.9.1-incubating
Apache Kafka 0.8.1
Zookeeper 3.4.6
Reporter: Andres Gomez Ferrer
I have a one topology running on my production cluster. This topology has run
for some weeks without fails, but few days ago my supervisor died with this
error:
tail /var/log/storm/supervisor.log
2014-07-27 23:15:26 b.s.event [ERROR] Error when processing event
java.lang.RuntimeException: java.io.EOFException
at backtype.storm.utils.Utils.deserialize(Utils.java:86)
~[storm-core-0.9.1-incubating-mmx2.jar:0.9.1-incubating-mmx2]
at backtype.storm.utils.LocalState.snapshot(LocalState.java:45)
~[storm-core-0.9.1-incubating-mmx2.jar:0.9.1-incubating-mmx2]
at backtype.storm.utils.LocalState.get(LocalState.java:56)
~[storm-core-0.9.1-incubating-mmx2.jar:0.9.1-incubating-mmx2]
at
backtype.storm.daemon.supervisor$read_worker_heartbeat.invoke(supervisor.clj:77)
~[storm-core-0.9.1-incubating-mmx2.jar:0.9.1-incubating-mmx2]
at
backtype.storm.daemon.supervisor$read_worker_heartbeats$iter__4842__4846$fn__4847.invoke(supervisor.clj:90)
~[na:na]
at clojure.lang.LazySeq.sval(LazySeq.java:42) ~[clojure-1.4.0.jar:na]
at clojure.lang.LazySeq.seq(LazySeq.java:60) ~[clojure-1.4.0.jar:na]
at clojure.lang.RT.seq(RT.java:473) ~[clojure-1.4.0.jar:na]
at clojure.core$seq.invoke(core.clj:133) ~[clojure-1.4.0.jar:na]
at clojure.core$dorun.invoke(core.clj:2725) ~[clojure-1.4.0.jar:na]
at clojure.core$doall.invoke(core.clj:2741) ~[clojure-1.4.0.jar:na]
at
backtype.storm.daemon.supervisor$read_worker_heartbeats.invoke(supervisor.clj:89)
~[storm-core-0.9.1-incubating-mmx2.jar:0.9.1-incubating-mmx2]
at
backtype.storm.daemon.supervisor$read_allocated_workers.invoke(supervisor.clj:106)
~[storm-core-0.9.1-incubating-mmx2.jar:0.9.1-incubating-mmx2]
at
backtype.storm.daemon.supervisor$sync_processes.invoke(supervisor.clj:209)
~[storm-core-0.9.1-incubating-mmx2.jar:0.9.1-incubating-mmx2]
at clojure.lang.AFn.applyToHelper(AFn.java:161) [clojure-1.4.0.jar:na]
at clojure.lang.AFn.applyTo(AFn.java:151) [clojure-1.4.0.jar:na]
at clojure.core$apply.invoke(core.clj:603) ~[clojure-1.4.0.jar:na]
at clojure.core$partial$fn__4070.doInvoke(core.clj:2343)
~[clojure-1.4.0.jar:na]
at clojure.lang.RestFn.invoke(RestFn.java:397) ~[clojure-1.4.0.jar:na]
at backtype.storm.event$event_manager$fn__2593.invoke(event.clj:39) ~[na:na]
at clojure.lang.AFn.run(AFn.java:24) [clojure-1.4.0.jar:na]
at java.lang.Thread.run(Unknown Source) [na:1.7.0_03]
Caused by: java.io.EOFException: null
at java.io.ObjectInputStream$PeekInputStream.readFully(Unknown Source)
~[na:1.7.0_03]
at java.io.ObjectInputStream$BlockDataInputStream.readShort(Unknown Source)
~[na:1.7.0_03]
at java.io.ObjectInputStream.readStreamHeader(Unknown Source) ~[na:1.7.0_03]
at java.io.ObjectInputStream.<init>(Unknown Source) ~[na:1.7.0_03]
at backtype.storm.utils.Utils.deserialize(Utils.java:81)
~[storm-core-0.9.1-incubating-mmx2.jar:0.9.1-incubating-mmx2]
... 21 common frames omitted
2014-07-27 23:15:26 b.s.util [INFO] Halting process: ("Error when processing an
event")
He tried wake up but I died with the same error all time.
I have fixed the problem when I delete my temporally storm directory
"/tmp/storm" But the next day, I found the same problem again. I deleted the
directory again and now the topology runs fine but I think the error "[ERROR]
Error when processing even" isn't normal and I have decided report it.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
