I think, you may want to start kafka and then give some time for its initialisation and then submit topology.
On Sat, Nov 19, 2016 at 12:55 AM, Cuneo, Nicholas <ncu...@tycoint.com> wrote: > The cause of the null pointer exception is IN kafka spout during an ACK > > > > My guess is there’s a race condition where it tries to retrieve the topic > partition from the map but it hasn’t been loaded yet. Which results in a > NULL entry during the GET. When the topology starts up it will immediately > receive messages but could be slow to startup due to rebalancing of the > kafka offsets or other tasks it does when it subscribes. > > > > Like I said, this happens occasionally during startup but not reliably and > has nothing to do with my code other than I’m probably acking the received > message faster than the kafka spout can finish initialization. > > > > Thanks, > > Nick > > > > *From:* Harsh Choudhary [mailto:shry.ha...@gmail.com] > *Sent:* Friday, November 18, 2016 11:18 AM > *To:* user@storm.apache.org > *Subject:* Re: NullPointerException on startup > > > > Hi > > This happens when there is some code in bolt or spout which throws Null > Pointer Exception. I suggest you to use Debugger in your IDE, to find out > where is this happening. You can try making a Local Cluster and runs it in > IDE to figure it out easily. > > It never happens because of Storm, so do not raise ticket for now. > > > > > > > On Sat, Nov 19, 2016 at 12:40 AM, Cuneo, Nicholas <ncu...@tycoint.com> > wrote: > > Hello, > > > > We are working with storm 1.0.2 and using Kafka client to subscribe to > kafka topics to retrieve data. Randomly when our topologies are starting > up we receive a null pointer exception which is killing the topology. We > can’t seem to identify why this happens, or what measures we can take to > prevent it. I’m considering raising a ticket with storm to check for null > to prevent the topology from crashing. > > > > 2016-11-17 23:11:05.366 o.a.s.util [ERROR] Async loop died! > > java.lang.RuntimeException: java.lang.NullPointerException > > at > org.apache.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:464) > ~[storm-core-1.0.2.jar:1.0.2] > > at > org.apache.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:430) > ~[storm-core-1.0.2.jar:1.0.2] > > at > org.apache.storm.utils.DisruptorQueue.consumeBatch(DisruptorQueue.java:420) > ~[storm-core-1.0.2.jar:1.0.2] > > at org.apache.storm.disruptor$consume_batch.invoke(disruptor.clj:69) > ~[storm-core-1.0.2.jar:1.0.2] > > at > org.apache.storm.daemon.executor$fn__7990$fn__8005$fn__8036.invoke(executor.clj:628) > ~[storm-core-1.0.2.jar:1.0.2] > > at org.apache.storm.util$async_loop$fn__624.invoke(util.clj:484) > [storm-core-1.0.2.jar:1.0.2] > > at clojure.lang.AFn.run(AFn.java:22) [clojure-1.7.0.jar:?] > > at java.lang.Thread.run(Thread.java:745) [?:1.8.0_91] > > Caused by: java.lang.NullPointerException > > at org.apache.storm.kafka.spout.KafkaSpout.ack(KafkaSpout.java:316) > ~[stormjar.jar:?] > > at org.apache.storm.daemon.executor$ack_spout_msg.invoke(executor.clj:448) > ~[storm-core-1.0.2.jar:1.0.2] > > at org.apache.storm.daemon.executor$fn__7990$tuple_ > action_fn__7996.invoke(executor.clj:536) ~[storm-core-1.0.2.jar:1.0.2] > > at > org.apache.storm.daemon.executor$mk_task_receiver$fn__7979.invoke(executor.clj:464) > ~[storm-core-1.0.2.jar:1.0.2] > > at > org.apache.storm.disruptor$clojure_handler$reify__7492.onEvent(disruptor.clj:40) > ~[storm-core-1.0.2.jar:1.0.2] > > at > org.apache.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:451) > ~[storm-core-1.0.2.jar:1.0.2] > > ... 7 more > > 2016-11-17 23:11:05.379 o.a.s.d.executor [ERROR] > > java.lang.RuntimeException: java.lang.NullPointerException > > at > org.apache.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:464) > ~[storm-core-1.0.2.jar:1.0.2] > > at > org.apache.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:430) > ~[storm-core-1.0.2.jar:1.0.2] > > at > org.apache.storm.utils.DisruptorQueue.consumeBatch(DisruptorQueue.java:420) > ~[storm-core-1.0.2.jar:1.0.2] > > at org.apache.storm.disruptor$consume_batch.invoke(disruptor.clj:69) > ~[storm-core-1.0.2.jar:1.0.2] > > at > org.apache.storm.daemon.executor$fn__7990$fn__8005$fn__8036.invoke(executor.clj:628) > ~[storm-core-1.0.2.jar:1.0.2] > > at org.apache.storm.util$async_loop$fn__624.invoke(util.clj:484) > [storm-core-1.0.2.jar:1.0.2] > > at clojure.lang.AFn.run(AFn.java:22) [clojure-1.7.0.jar:?] > > at java.lang.Thread.run(Thread.java:745) [?:1.8.0_91] > > Caused by: java.lang.NullPointerException > > at org.apache.storm.kafka.spout.KafkaSpout.ack(KafkaSpout.java:316) > ~[stormjar.jar:?] > > at org.apache.storm.daemon.executor$ack_spout_msg.invoke(executor.clj:448) > ~[storm-core-1.0.2.jar:1.0.2] > > at org.apache.storm.daemon.executor$fn__7990$tuple_ > action_fn__7996.invoke(executor.clj:536) ~[storm-core-1.0.2.jar:1.0.2] > > at > org.apache.storm.daemon.executor$mk_task_receiver$fn__7979.invoke(executor.clj:464) > ~[storm-core-1.0.2.jar:1.0.2] > > at > org.apache.storm.disruptor$clojure_handler$reify__7492.onEvent(disruptor.clj:40) > ~[storm-core-1.0.2.jar:1.0.2] > > at > org.apache.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:451) > ~[storm-core-1.0.2.jar:1.0.2] > > ... 7 more > > 2016-11-17 23:11:05.473 o.a.s.util [ERROR] Halting process: ("Worker died") > > java.lang.RuntimeException: ("Worker died") > > at org.apache.storm.util$exit_process_BANG_.doInvoke(util.clj:341) > [storm-core-1.0.2.jar:1.0.2] > > at clojure.lang.RestFn.invoke(RestFn.java:423) [clojure-1.7.0.jar:?] > > at org.apache.storm.daemon.worker$fn__8663$fn__8664.invoke(worker.clj:765) > [storm-core-1.0.2.jar:1.0.2] > > at org.apache.storm.daemon.executor$mk_executor_data$fn__ > 7875$fn__7876.invoke(executor.clj:274) [storm-core-1.0.2.jar:1.0.2] > > at org.apache.storm.util$async_loop$fn__624.invoke(util.clj:494) > [storm-core-1.0.2.jar:1.0.2] > > at clojure.lang.AFn.run(AFn.java:22) [clojure-1.7.0.jar:?] > > at java.lang.Thread.run(Thread.java:745) [?:1.8.0_91] > > > > The method and line number in question below: > > > > @Override > > public void ack(Object messageId) { > > final KafkaSpoutMessageId msgId = (KafkaSpoutMessageId) messageId; > > if (!consumerAutoCommitMode) { // Only need to keep track of > acked tuples if commits are not done automatically > > *acked.get(msgId.getTopicPartition()).add(msgId);* > > } > > emitted.remove(msgId); > > } > > > > Restarting the topology sometimes fixes the issue but sometimes does not. > I don’t know if there’s some kind of race condition occurring. We have > about 15 topologies all connecting to kafka the same way, and this doesn’t > happen consistently between them. > > > > Does anyone have any insight as to why this occurs or how to prevent it on > our end? > > > > > > Thanks, > > > > *Nick Cuneo* */ *Software Engineer, Cloud */ Enterprise Software* > > Tel: +1 949 517 4802 / Mobile: +1 949 243 4952 > 3 Ada / Irvine, CA 92618 / USA > > *ncu...@tycoint.com <ncu...@tycoint.com>* / *www.tyco.com > <http://www.tyco.com>* > > [image: cid:image003.png@01D2418C.07311610] > > > > This email (including any attachments) may contain information that is > private or business confidential. If you received this email in error, > please delete it from your system without copying it and notify sender by > reply email so that our records can be corrected > > > > > ------------------------------ > > > This e-mail contains privileged and confidential information intended for > the use of the addressees named above. If you are not the intended > recipient of this e-mail, you are hereby notified that you must not > disseminate, copy or take any action in respect of any information > contained in it. If you have received this e-mail in error, please notify > the sender immediately by e-mail and immediately destroy this e-mail and > its attachments. > > > > ------------------------------ > > This e-mail contains privileged and confidential information intended for > the use of the addressees named above. If you are not the intended > recipient of this e-mail, you are hereby notified that you must not > disseminate, copy or take any action in respect of any information > contained in it. If you have received this e-mail in error, please notify > the sender immediately by e-mail and immediately destroy this e-mail and > its attachments. >
