[
https://issues.apache.org/jira/browse/HBASE-23113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17076796#comment-17076796
]
Mark Robert Miller commented on HBASE-23113:
--------------------------------------------
One short term option to consider is to only enable it for tests as well.
Tests put a very high number of connections into TIME_WAIT - I see a larger
issue with it the faster I run tests. I think making that less common is not
likely super short term, but more SO_REUSEPORT usage can alleviate it a bit.
(lots of sources on this stuff, but here is one
[http://www.serverframework.com/asynchronousevents/2011/01/time-wait-and-its-design-implications-for-protocols-and-scalable-servers.html])
> IPC Netty Optimization
> ----------------------
>
> Key: HBASE-23113
> URL: https://issues.apache.org/jira/browse/HBASE-23113
> Project: HBase
> Issue Type: Improvement
> Components: IPC/RPC
> Reporter: Nicholas Jiang
> Assignee: Nicholas Jiang
> Priority: Minor
> Attachments: Decoder.jpeg
>
>
> Netty options in IPC Server/Client optimization:
> 1.SO_BACKLOG setting:Two queues are maintained in the Linux system kernel:
> syns queue and accept queue. The first is a semi-join queue that saves the
> connections to the synrecv state after receiving the client syn. The default
> netty is 128,io.netty.util.NetUtil#SOMAXCONN , and then read
> /proc/sys/net/core /somaxconn to continue to determine, and then there are
> some system level coverage logic.In some scenarios, if the client is far
> redundant to the server and the connection is established, it may not be
> enough. This value should not be too large, otherwise it will not prevent
> SYN-Flood attacks. The current value has been changed to 1024. After setting,
> the value set by yourself is equivalent to setting the upper limit because of
> the setting of the system and the size of the system. If some settings of the
> Linux system operation and maintenance are wrong, it can be avoided at the
> code level.At present, our Linux level is usually set to 128, and the final
> calculation will be set to 128.
> 2.WRITE_BUFFER_WATER_MARK setting:After WRITEBUFFERWATERMARK sets the maximum
> and minimum Buffer that can be temporarily stored on a connection, isWritable
> returns unwritable if the amount of data waiting to be sent for the
> connection is greater than the set value. In this way, the client can no
> longer send, preventing this amount of continuous backlog, and eventually the
> client may hang. If this happens, it is usually caused by slow processing on
> the server side. This value can effectively protect the client. At this point
> the data was not sent.
> 3.SO_REUSEADDR - Port multiplexing (allowing multiple sockets to listen on
> the same IP+ port): For time-wait links, it ensures that the server restarts
> successfully. In the case where some servers start up very quickly, it can
> prevent startup failure.
> Netty decoder in IPC Server optimization:
> Netty provides a convenient decoding tool class ByteToMessageDecoder, as
> shown in the top half of the figure, this class has accumulate bulk unpacking
> capability, can read bytes from the socket as much as possible, and then
> synchronously call the decode method to decode the business object. And
> compose a List. Finally, the traversal traverses the List and submits it to
> ChannelPipeline for processing. Here we made a small change, as shown in the
> bottom half of the figure, the content to be submitted is changed from a
> single command to the entire List, which reduces the number of pipeline
> executions and improves throughput. This mode has no advantage in
> low-concurrency scenarios, and has a significant performance boost in boost
> throughput in high-concurrency scenarios.
> !Decoder.jpeg!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
