[
https://issues.apache.org/jira/browse/STORM-3422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stig Rohde Døssing resolved STORM-3422.
---------------------------------------
Resolution: Fixed
Assignee: Petr Janeček
Fix Version/s: 2.0.1
Thanks [~JanecekPetr], great job. Merged to master.
> TupleCaptureBolt is not thread-safe
> -----------------------------------
>
> Key: STORM-3422
> URL: https://issues.apache.org/jira/browse/STORM-3422
> Project: Apache Storm
> Issue Type: Bug
> Components: storm-client
> Affects Versions: 2.0.0, 1.2.2
> Reporter: Petr Janeček
> Assignee: Petr Janeček
> Priority: Major
> Labels: pull-request-available
> Fix For: 2.0.1
>
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> Marking this as Major because it's a crash. That said, the problem lies in
> testing code. This makes integration testing hard, but the issue does not
> affect any production code.
>
> First, let me show you a stack trace for Storm 2.0.0:
> {{java.lang.RuntimeException: java.lang.NullPointerException}}
> {{at org.apache.storm.executor.Executor.accept(Executor.java:282)
> ~[storm-client-2.0.0.jar:2.0.0]}}
> {{at org.apache.storm.utils.JCQueue.consumeImpl(JCQueue.java:133)
> ~[storm-client-2.0.0.jar:2.0.0]}}
> {{at org.apache.storm.utils.JCQueue.consume(JCQueue.java:110)
> ~[storm-client-2.0.0.jar:2.0.0]}}
> {{at
> org.apache.storm.executor.bolt.BoltExecutor$1.call(BoltExecutor.java:171)
> ~[storm-client-2.0.0.jar:2.0.0]}}
> {{at
> org.apache.storm.executor.bolt.BoltExecutor$1.call(BoltExecutor.java:158)
> ~[storm-client-2.0.0.jar:2.0.0]}}
> {{at org.apache.storm.utils.Utils$1.run(Utils.java:388)
> [storm-client-2.0.0.jar:2.0.0]}}
> {{at java.lang.Thread.run(Thread.java:745) [?:1.8.0_121]}}
> {{Caused by: java.lang.NullPointerException}}
> {{at
> org.apache.storm.testing.TupleCaptureBolt.execute(TupleCaptureBolt.java:45)
> ~[storm-client-2.0.0.jar:2.0.0]}}
> {{at
> org.apache.storm.executor.bolt.BoltExecutor.tupleActionFn(BoltExecutor.java:234)
> ~[storm-client-2.0.0.jar:2.0.0]}}
> {{at org.apache.storm.executor.Executor.accept(Executor.java:275)
> ~[storm-client-2.0.0.jar:2.0.0]}}
> {{... 6 more}}
>
> Here's the same for Storm 1.2.2:
> {{java.lang.RuntimeException: java.lang.NullPointerException}}
> {{at
> org.apache.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:522)
> ~[storm-core-1.2.2.jar:1.2.2]}}
> {{at
> org.apache.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:487)
> ~[storm-core-1.2.2.jar:1.2.2]}}
> {{at
> org.apache.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:74)
> ~[storm-core-1.2.2.jar:1.2.2]}}
> {{at
> org.apache.storm.daemon.executor$fn__10795$fn__10808$fn__10861.invoke(executor.clj:861)
> ~[storm-core-1.2.2.jar:1.2.2]}}
> {{at org.apache.storm.util$async_loop$fn__553.invoke(util.clj:484)
> [storm-core-1.2.2.jar:1.2.2]}}
> {{at clojure.lang.AFn.run(AFn.java:22) [clojure-1.7.0.jar:?]}}
> {{at java.lang.Thread.run(Thread.java:745) [?:1.8.0_121]}}
> {{Caused by: java.lang.NullPointerException}}
> {{at
> org.apache.storm.testing.TupleCaptureBolt.execute(TupleCaptureBolt.java:50)
> ~[storm-core-1.2.2.jar:1.2.2]}}
> {{at
> org.apache.storm.daemon.executor$fn__10795$tuple_action_fn__10797.invoke(executor.clj:739)
> ~[storm-core-1.2.2.jar:1.2.2]}}
> {{at
> org.apache.storm.daemon.executor$mk_task_receiver$fn__10716.invoke(executor.clj:468)
> ~[storm-core-1.2.2.jar:1.2.2]}}
> {{at
> org.apache.storm.disruptor$clojure_handler$reify__10135.onEvent(disruptor.clj:41)
> ~[storm-core-1.2.2.jar:1.2.2]}}
> {{at
> org.apache.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:509)
> ~[storm-core-1.2.2.jar:1.2.2]}}
> {{... 6 more}}
>
> This is a topology running as our integration test using
> {{Testing.completeTopology()}}. Both the stack traces point to the same code
> in the {{TupleCaptureBolt}} - its {{name}} field is not safely published (it
> should be marked {{final}}), and the internal {{HashMap}} does not safely
> store the data put in it. Perhaps it should be a {{ConcurrentHashMap}}?
> Would you accept a PR with a more detailed analysis, or are you going to
> investigate on your side?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
