GitHub user squito opened a pull request:
https://github.com/apache/spark/pull/21346
[SPARK-6237][NETWORK] Network-layer changes to allow stream upload.
These changes allow an RPCHandler to receive an upload as a stream of
data, without having to buffer the entire message in the FrameDecoder.
The primary use case is for replicating large blocks.
Added unit tests for handling streaming data, including successfully
sending data, and failures in reading the stream with concurrent requests.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/squito/spark upload_stream
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/21346.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #21346
----
commit 49e0a80f89433368d3a3116eb9fcd7854ceecb62
Author: Imran Rashid <irashid@...>
Date: 2018-05-02T14:55:15Z
[SPARK-6237][NETWORK] Network-layer changes to allow stream upload.
These changes allow an RPCHandler to receive an upload as a stream of
data, without having to buffer the entire message in the FrameDecoder.
The primary use case is for replicating large blocks.
Added unit tests.
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
