Hi all,
I've deployed storm 0.9.1 on EC2, with KafkaSpout and every time a
message is received from kafka, the worker dies with the below stacktrace.
Ive patched my 0.9.1 with https://issues.apache.org/jira/browse/STORM-187.
Im using netty, with the below config on supervisor. Has anyone run into
this issue?
storm.messaging.transport: "backtype.storm.messaging.netty.Context"
supervisor.worker.timeout.secs: 5
storm.messaging.netty.server_worker_threads: 1
storm.messaging.netty.client_worker_threads: 1
storm.messaging.netty.buffer_size: 67108864 #5MB buffer
storm.messaging.netty.max_retries: 5
storm.messaging.netty.max_wait_ms: 10000
storm.messaging.netty.min_wait_ms: 5000
2014-06-07 06:43:27 b.s.d.task [INFO] Emitting: processMunchkin
__ack_ack [2264770826798969496 7102317654394246142]
2014-06-07 06:43:27 b.s.util [ERROR] Async loop died!
java.lang.RuntimeException: java.lang.RuntimeException: Client is
being closed, and does not take requests any more
at
backtype.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:107)
~[storm-core-0.9.1.0-incubating.jar:0.9.1.0-incubating]
at
backtype.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:78)
~[storm-core-0.9.1.0-incubating.jar:0.9.1.0-incubating]
at
backtype.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:77)
~[storm-core-0.9.1.0-incubating.jar:0.9.1.0-incubating]
at
backtype.storm.disruptor$consume_loop_STAR_$fn__1577.invoke(disruptor.clj:89)
~[na:na]
at backtype.storm.util$async_loop$fn__384.invoke(util.clj:433) ~[na:na]
at clojure.lang.AFn.run(AFn.java:24) [clojure-1.4.0.jar:na]
at java.lang.Thread.run(Thread.java:744) [na:1.7.0_55]
Caused by: java.lang.RuntimeException: Client is being closed, and
does not take requests any more
at backtype.storm.messaging.netty.Client.send(Client.java:125)
~[storm-core-0.9.1.0-incubating.jar:0.9.1.0-incubating]
at
backtype.storm.daemon.worker$mk_transfer_tuples_handler$fn__4398$fn__4399.invoke(worker.clj:319)
~[na:na]
at
backtype.storm.daemon.worker$mk_transfer_tuples_handler$fn__4398.invoke(worker.clj:308)
~[na:na]
at
backtype.storm.disruptor$clojure_handler$reify__1560.onEvent(disruptor.clj:58)
~[na:na]
at
backtype.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:104)
~[storm-core-0.9.1.0-incubating.jar:0.9.1.0-incubating]
... 6 common frames omitted
2014-06-07 06:43:27 b.s.util [INFO] Halting process: ("Async loop died!")
