GitHub user vanzin opened a pull request:
https://github.com/apache/spark/pull/5234
Fix thread-safety issue in outbound path of network library.
While the inbound path of a netty pipeline is thread-safe, the outbound
path is not. That means that multiple threads can compete to write messages
to the next stage of the pipeline.
The network library sometimes breaks a single RPC message into multiple
buffers internally to avoid copying data (see MessageEncoder). This can
result in the following scenario (where "FxBy" means "frame x, buffer y"):
T1 F1B1 F1B2
\ \
\ \
socket F1B1 F2B1 F1B2 F2B2
/ /
/ /
T2 F2B1 F2B2
And the frames now cannot be rebuilt on the receiving side because the
different messages have been mixed up on the wire.
The fix introduces a new stage in the pipeline that acts as a
"demultiplexer":
multi-buffer frames are expected to navigate the pipeline as a List, and the
new handler will write all buffers in the List to the next stage in a
thread-safe manner.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/vanzin/spark SPARK-6578
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/5234.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #5234
----
commit 8d70e6091a78478387dfc9e1b121efbfb6240487
Author: Marcelo Vanzin <[email protected]>
Date: 2015-03-27T17:57:38Z
Fix thread-safety issue in outbound path of network library.
While the inbound path of a netty pipeline is thread-safe, the outbound
path is not. That means that multiple threads can compete to write messages
to the next stage of the pipeline.
The network library sometimes breaks a single RPC message into multiple
buffers internally to avoid copying data (see MessageEncoder). This can
result in the following scenario (where "FxBy" means "frame x, buffer y"):
T1 F1B1 F1B2
\ \
\ \
socket F1B1 F2B1 F1B2 F2B2
/ /
/ /
T2 F2B1 F2B2
And the frames now cannot be rebuilt on the receiving side because the
different messages have been mixed up on the wire.
The fix introduces a new stage in the pipeline that acts as a
"demultiplexer":
multi-buffer frames are expected to navigate the pipeline as a List, and the
new handler will write all buffers in the List to the next stage in a
thread-safe manner.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
