[
https://issues.apache.org/jira/browse/STORM-737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14547031#comment-14547031
]
ASF GitHub Bot commented on STORM-737:
--------------------------------------
Github user HeartSaVioR commented on a diff in the pull request:
https://github.com/apache/storm/pull/521#discussion_r30466949
--- Diff: storm-core/src/clj/backtype/storm/daemon/worker.clj ---
@@ -139,12 +139,12 @@
(.add local pair)
;;Using java objects directly to avoid performance
issues in java code
- (let [node+port (get @task->node+port task)]
- (when (not (.get remoteMap node+port))
- (.put remoteMap node+port (ArrayList.)))
- (let [remote (.get remoteMap node+port)]
+ (let []
+ (when (not (.get remoteMap task))
+ (.put remoteMap task (ArrayList.)))
+ (let [remote (.get remoteMap task)]
(.add remote (TaskMessage. task (.serialize
serializer tuple)))
- ))))
+ ))))
--- End diff --
@d2r
Think about situation like
* tuples which consist of
* 3 tuples corresponding to task 1 (worker A) -- (A)
* 5 tuples corresponding to task 2 (worker A) -- (B)
* 2 tuples corresponding to task 3 (worker B) -- (C)
We can ensure we send tuples to proper worker since we check
task->node+port with read lock.
But if we don't re-group messages by worker, we'll send tuples batched by
task, not worker.
It means, though we can send 10 tuples at a moment, we send batched tuples
(A), (B) to worker A separately.
> Workers may try to send to closed connections
> ---------------------------------------------
>
> Key: STORM-737
> URL: https://issues.apache.org/jira/browse/STORM-737
> Project: Apache Storm
> Issue Type: Bug
> Affects Versions: 0.9.2-incubating
> Reporter: Derek Dagit
>
> There is a race condition in the worker code that can allow for a send() to
> be called on a closed connection.
> [Discussion|https://github.com/apache/storm/pull/349#issuecomment-87778672]
> The assignment mapping from task -> node+port needs to be read and used in
> the read lock when sending, so that an accurate mapping is used that does not
> include any connections that are closed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
