[
https://issues.apache.org/jira/browse/STORM-770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14496199#comment-14496199
]
Jungtaek Lim edited comment on STORM-770 at 4/15/15 2:05 PM:
-------------------------------------------------------------
https://github.com/apache/storm/commit/861a92eab8740cfc0f83ac4d7ade9a2ab04a8b9b
Maybe it wasn't an issue before this commit (introduced on 0.9.2-incubating)
because
- transfer-fn could publish pair of null task value and serialized tuples
without issue (no conversion is being processed)
-- Please refer mk-transfer-fn
- transfer-tuple-handler finds node-port with null task value, maybe fail to
find, then silently discard tuple
-- Please refer mk-transfer-tuples-handler
I still don't get why task becomes null (It may be related to grouping, I found
some scenario which tasks-fn may return null or [null]), but if it's possible,
and there's no issue to discard tuples with null task value, we can easily
apply it by modifying mk-transfer-fn.
Though null task value is not acceptable, we need to check null task value
before encountering NPE (maybe we can log it when emitting), and log it with
various informations for debug.
[~clockfly]
Since you're author of commit, I wish to hear your motivation / intention of
these changeset.
to. committers
Please clarify when null task value occurs and whether null task value is
acceptable or not.
Thanks in advance!
was (Author: kabhwan):
https://github.com/apache/storm/commit/861a92eab8740cfc0f83ac4d7ade9a2ab04a8b9b
Maybe it wasn't an issue before this commit (introduced on 0.9.2-incubating)
because
- transfer-fn could publish pair of null task value and serialized tuples
without issue (no conversion is being processed)
-- Please refer mk-transfer-fn
- transfer-tuple-handler find node-port with null task value, maybe fail to
find, then silently discard tuple
-- Please refer mk-transfer-tuples-handler
I still don't get why task becomes null (It may be related to grouping, I found
some scenario which tasks-fn may return null or [null]), but if it's possible,
and there's no issue to discard tuples with null task value, we can easily
apply it by modifying mk-transfer-fn.
Though null task value is not acceptable, we need to check null task value
before encountering NPE (maybe we can log it when emitting), and log it with
various informations for debug.
[~clockfly]
Since you're author of commit, I wish to hear your motivation / intention of
these changeset.
to. committers
Please clarify when null task value occurs and whether null task value is
acceptable or not.
Thanks in advance!
> NullPointerException in consumeBatchToCursor
> --------------------------------------------
>
> Key: STORM-770
> URL: https://issues.apache.org/jira/browse/STORM-770
> Project: Apache Storm
> Issue Type: Bug
> Affects Versions: 0.9.2-incubating
> Reporter: Stas Levin
>
> We got the following exception after our topology had been up for ~2 days,
> and I was wondering if it might be related.
> Looks like "task" in "mk-transfer-fn" is null, making "(.add remote
> (TaskMessage. task (.serialize serializer tuple)))" fail on NPE
> (worker.clj:128, storm-core-0.9.2-incubating.jar)
> java.lang.RuntimeException: java.lang.NullPointerException
> at
> backtype.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:128)
> ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
> at
> backtype.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:99)
> ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
> at
> backtype.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:80)
> ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
> at
> backtype.storm.disruptor$consume_loop_STAR_$fn__758.invoke(disruptor.clj:94)
> ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
> at backtype.storm.util$async_loop$fn__457.invoke(util.clj:431)
> ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
> at clojure.lang.AFn.run(AFn.java:24) [clojure-1.5.1.jar:na]
> at java.lang.Thread.run(Thread.java:745) [na:1.7.0_72]
> Caused by: java.lang.NullPointerException: null
> at clojure.lang.RT.intCast(RT.java:1087) ~[clojure-1.5.1.jar:na]
> at
> backtype.storm.daemon.worker$mk_transfer_fn$fn__5748.invoke(worker.clj:128)
> ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
> at
> backtype.storm.daemon.executor$start_batch_transfer_GT_worker_handler_BANG$fn__5483.invoke(executor.clj:256)
> ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
> at
> backtype.storm.disruptor$clojure_handler$reify__745.onEvent(disruptor.clj:58)
> ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
> at
> backtype.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:125)
> ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
> ... 6 common frames omitted,java.lang.RuntimeException:
> java.lang.NullPointerException
> Any ideas?
> P.S.
> Also saw it here:
> http://mail-archives.apache.org/mod_mbox/storm-user/201501.mbox/%3CCABcMBhCusXXU=v1e66wfuatgyh1euqnd1siog65-tp8xlwx...@mail.gmail.com%3E
> https://mail-archives.apache.org/mod_mbox/storm-user/201408.mbox/%3ccajuqm_4kxhsh2_x08ujuqr76m2c+dswp0fcijbmfcaeyqgs...@mail.gmail.com%3E
> Comment from Bobby
> http://mail-archives.apache.org/mod_mbox/storm-user/201501.mbox/%3c574363643.2791948.1420470097280.javamail.ya...@jws10027.mail.ne1.yahoo.com%3E
> {quote}
> What version of storm are you using? Are any of the bolts shell bolts?
> There is a known
> issue where this can happen if two shell bolts share an executor, because
> they are multi-threaded.
> - Bobby
> {quote}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
