We upgraded to 0.9.2-incubating from storm 0.8.2 and using kafka 0.8.1.1 to
read messages in the storm. We are using higher level consumer api to read
messages from kafka.
We saw following only once at the startup of topology, where one worker
seems to died and got restarted. Things were working after that and did not
see this error again. Not sure why it would happen only at the start up.
Does anyone know why we get these two exceptions.
We are emitting tuples in our kafka spout in a single seperate thread, but
that should be fine as far as I know.
2014-08-28 22:47:29 k.c.ClientUtils$ [INFO] Fetching metadata from broker
id:0,host:10.9.134.87,port:9092 with correlation id 2910 for 1 topic(s)
Set(alert_events)
2014-08-28 22:47:29 k.p.SyncProducer [INFO] Connected to 10.9.134.87:9092
for producing
2014-08-28 22:47:29 k.p.SyncProducer [INFO] Disconnecting from
10.9.134.87:9092
2014-08-28 22:47:29 k.p.SyncProducer [INFO] Disconnecting from
kafka1.r6.03c.snaa.gaikai.org:9092
2014-08-28 22:47:29 k.p.SyncProducer [INFO] Connected to
kafka1.r6.03c.snaa.gaikai.org:9092 for producing
*2014-08-28 22:47:55 b.s.util [ERROR] Async loop died!*
*java.util.NoSuchElementException: null*
at java.util.LinkedList.removeFirst(LinkedList.java:268)
~[na:1.7.0_67]
* at
backtype.storm.daemon.executor$fn__5573$fn__5588$fn__5617$fn__5619.invoke(executor.clj:547)
~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]*
at
backtype.storm.daemon.executor$fn__5573$fn__5588$fn__5617.invoke(executor.clj:543)
~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
at backtype.storm.util$async_loop$fn__457.invoke(util.clj:431)
~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
at clojure.lang.AFn.run(AFn.java:24) [clojure-1.5.1.jar:na]
at java.lang.Thread.run(Thread.java:745) [na:1.7.0_67]
2014-08-28 22:47:55 b.s.d.executor [ERROR]
java.util.NoSuchElementException: null
at java.util.LinkedList.removeFirst(LinkedList.java:268)
~[na:1.7.0_67]
at
backtype.storm.daemon.executor$fn__5573$fn__5588$fn__5617$fn__5619.invoke(executor.clj:547)
~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
at
backtype.storm.daemon.executor$fn__5573$fn__5588$fn__5617.invoke(executor.clj:543)
~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
at backtype.storm.util$async_loop$fn__457.invoke(util.clj:431)
~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
at clojure.lang.AFn.run(AFn.java:24) [clojure-1.5.1.jar:na]
at java.lang.Thread.run(Thread.java:745) [na:1.7.0_67]
2014-08-28 22:47:55 b.s.util [ERROR] Async loop died!
*java.lang.RuntimeException: java.lang.NullPointerException at
backtype.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:128)
*~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
at
backtype.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:99)
~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
at
backtype.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:80)
~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
at
backtype.storm.disruptor$consume_loop_STAR_$fn__758.invoke(disruptor.clj:94)
~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
at backtype.storm.util$async_loop$fn__457.invoke(util.clj:431)
~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
at clojure.lang.AFn.run(AFn.java:24) [clojure-1.5.1.jar:na]
at java.lang.Thread.run(Thread.java:745) [na:1.7.0_67]
Caused by: java.lang.NullPointerException: null
at clojure.lang.RT.intCast(RT.java:1087) ~[clojure-1.5.1.jar:na]
at
backtype.storm.daemon.worker$mk_transfer_fn$fn__5748.invoke(worker.clj:128)
~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
at
backtype.storm.daemon.executor$start_batch_transfer__GT_worker_handler_BANG_$fn__5483.invoke(executor.clj:256)
~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
at
backtype.storm.disruptor$clojure_handler$reify__745.onEvent(disruptor.clj:58)
~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
at
backtype.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:125)
~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
... 6 common frames omitted
2014-08-28 22:47:55 b.s.d.executor [ERROR]
java.lang.RuntimeException: java.lang.NullPointerException
at
backtype.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:128)
~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
at
backtype.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:99)
~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
at
backtype.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:80)
~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
at
backtype.storm.disruptor$consume_loop_STAR_$fn__758.invoke(disruptor.clj:94)
~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
at backtype.storm.util$async_loop$fn__457.invoke(util.clj:431)
~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
at clojure.lang.AFn.run(AFn.java:24) [clojure-1.5.1.jar:na]
at java.lang.Thread.run(Thread.java:745) [na:1.7.0_67]
Caused by: java.lang.NullPointerException: null
at clojure.lang.RT.intCast(RT.java:1087) ~[clojure-1.5.1.jar:na]
at
backtype.storm.daemon.worker$mk_transfer_fn$fn__5748.invoke(worker.clj:128)
~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
at
backtype.storm.daemon.executor$start_batch_transfer__GT_worker_handler_BANG_$fn__5483.invoke(executor.clj:256)
~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
at
backtype.storm.disruptor$clojure_handler$reify__745.onEvent(disruptor.clj:58)
~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
at
backtype.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:125)
~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
... 6 common frames omitted
*2014-08-28 22:47:55 b.s.util [INFO] Halting process: ("Worker died")*
