Github user squito commented on a diff in the pull request:
https://github.com/apache/spark/pull/21346#discussion_r195464933
--- Diff:
common/network-common/src/main/java/org/apache/spark/network/server/RpcHandler.java
---
@@ -38,15 +38,28 @@
*
* This method will not be called in parallel for a single
TransportClient (i.e., channel).
*
+ * The rpc *might* included a data stream in <code>streamData</code>
(eg. for uploading a large
+ * amount of data which should not be buffered in memory here). An
error while reading data from
+ * the stream ({@link
org.apache.spark.network.client.StreamCallback#onData(String, ByteBuffer)})
+ * will fail the entire channel. A failure in "post-processing" the
stream in
+ * {@link
org.apache.spark.network.client.StreamCallback#onComplete(String)} will result
in an
+ * rpcFailure, but the channel will remain active.
+ *
+ * If streamData is not null, you *must* call
<code>streamData.registerStreamCallback</code>
+ * before this method returns.
+ *
* @param client A channel client which enables the handler to make
requests back to the sender
* of this RPC. This will always be the exact same object
for a particular channel.
* @param message The serialized bytes of the RPC.
+ * @param streamData StreamData if there is data which is meant to be
read via a StreamCallback;
--- End diff --
I've done this refactoring, and I agree it made the change significantly
simpler.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
