[
https://issues.apache.org/jira/browse/HADOOP-18536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17637039#comment-17637039
]
Shilun Fan edited comment on HADOOP-18536 at 11/22/22 7:35 AM:
---------------------------------------------------------------
Thank you very much for your feedback. Because it is widely used, we should pay
more attention to risks. It is difficult to judge whether the modified code and
the original code are idempotent, and just to avoid the apply of 1024 bytes in
the ResponseBuffer constructor. The code modification I saw is to copy some
code directly from other places and put it outside, it is difficult to agree
with this modification. Current Hadoop RPC has very good performance, this
modification seems to be over-engineered.
was (Author: slfan1989):
Thank you very much for your feedback. Because it is widely used, we should pay
more attention to risks. It is difficult to judge whether the modified code and
the original code are idempotent, and just to avoid the application of 1024
bytes in the ResponseBuffer constructor. The code modification I saw is to copy
some code directly from other places and put it outside, it is difficult to
agree with this modification. Current Hadoop RPC has very good performance,
this modification seems to be over-engineered.
> RPC Client Improvement
> ----------------------
>
> Key: HADOOP-18536
> URL: https://issues.apache.org/jira/browse/HADOOP-18536
> Project: Hadoop Common
> Issue Type: Improvement
> Components: rpc-server
> Reporter: xinqiu.hu
> Priority: Minor
>
> In the RPC Client, before a request (including RpcRequestHeaderProto,
> RequestHeaderProto, Message Payload) is sent, they will be copied to the three
> CodedOutputStream internal byte arrays, and then aggregated to the
> ResponseBuffer
> internal byte array. Then the ResponseBuffer byte array is written to a
> BufferedOutputStream and finally to a SocketOutputStream.
> To simplify the writing process, Maybe we can copy them directly to a big
> CodedOutputStream and send them directly to the IpcStreams#out. To achieve
> this, I
> propose HADOOP-18533. But it brings the following two side effects.
> # The generic declaration of rpcRequestQueue inside Client has been changed
> to Object
> # The serialization of protobuf has been moved to rpcRequestThread, because
> rpcRequestThread is a single thread for each connection, which may have a
> performance impact.
> For the above reasons, I propose this. This pr brings the following benefits
> # For each rpc request, avoid creating a ResponseBuffer of 1024 bytes
> # For each rpc request, reduce one copy
> # For each rpc request, combine the three fragmented CodedOutputStreams into
> one
> # No side effects like HADOOP-18533
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
