[
https://issues.apache.org/jira/browse/STORM-3620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17081732#comment-17081732
]
Julien Nioche commented on STORM-3620:
--------------------------------------
IMHO it would be preferable to make the outputCollector threadsafe - this is
the case in 1.x and apart from solving the problem experienced in STORM-3582 ,
it would greatly simplify things. Having to change the user code to enforce
synchronization would be a pain. As for eliminating multithreading inside a
bolt, well, again this would be a regression compared to 1.x and simply not
possible in some cases.
> OutputCollector in Storm 2.x is not thread-safe
> -----------------------------------------------
>
> Key: STORM-3620
> URL: https://issues.apache.org/jira/browse/STORM-3620
> Project: Apache Storm
> Issue Type: Bug
> Reporter: Ethan Li
> Priority: Major
>
> OutputCollector is not thread-safe in 2.x.
> It can cause data corruption if multiple threads in the same executor calls
> OutputCollector to emit data at the same time:
> 1. Every executor has an instance of ExecutorTransfer
> https://github.com/apache/storm/blob/00f48d60e75b28e11a887baba02dc77876b2bb3d/storm-client/src/jvm/org/apache/storm/executor/Executor.java#L146
> 2. Every ExecutorTransfer has its own serializer
> https://github.com/apache/storm/blob/00f48d60e75b28e11a887baba02dc77876b2bb3d/storm-client/src/jvm/org/apache/storm/executor/ExecutorTransfer.java#L44
> 3. Every executor has its own outputCollector
> https://github.com/apache/storm/blob/00f48d60e75b28e11a887baba02dc77876b2bb3d/storm-client/src/jvm/org/apache/storm/executor/bolt/BoltExecutor.java#L146-L147
> 4. When outputCollector is called to emit to remote workers, it uses
> ExecutorTransfer to transfer data
> https://github.com/apache/storm/blob/00f48d60e75b28e11a887baba02dc77876b2bb3d/storm-client/src/jvm/org/apache/storm/executor/ExecutorTransfer.java#L66
> 5. which will try to serialize data
> https://github.com/apache/storm/blob/00f48d60e75b28e11a887baba02dc77876b2bb3d/storm-client/src/jvm/org/apache/storm/daemon/worker/WorkerTransfer.java#L116
> 6. But serializer is not thread-safe
> https://github.com/apache/storm/blob/00f48d60e75b28e11a887baba02dc77876b2bb3d/storm-client/src/jvm/org/apache/storm/serialization/KryoTupleSerializer.java#L33-L43
> ----
> But in the doc, http://storm.apache.org/releases/2.1.0/Concepts.html, it says
> outputCollector is thread-safe.
> {code:java}
> Its perfectly fine to launch new threads in bolts that do processing
> asynchronously. OutputCollector is thread-safe and can be called at any time.
> {code}
> We should either fix it to make it thread-safe, or update the document to not
> mislead users
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
