[
https://issues.apache.org/jira/browse/HBASE-14490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14970334#comment-14970334
]
Hiroshi Ikeda commented on HBASE-14490:
---------------------------------------
{quote}
I know you doubling idea.If we use Pool, we hope fewer number of buffer service
more Connection.If Pool can't reduce amount of buffer dynamically, it will
increase to N ("N" is number of Connection ).In that case, there is no
different between Pool and non-pool.I'm thinking about Pool that can control
the number of buffer and each size of buffer.
{quote}
Sorry, could you be more specific? The pool can hold much more number of
buffers than connections on a heavy load, and lower on a low load. It is easy
and it would be not bad to leave that adjustment to weak/soft references (I
still recommend to use weak one). Leaving it aside, from the point of a view of
GC (and creating cost), what we should pay attention to is, not how many
buffers we are currently keeping, but how many buffers we have discarded (and
created). Non-pooling creates and discards a buffer each time and pooling would
have an advantage.
{quote}
It maintain average size.It expect hitting 50% req.
{quote}
If it is much more possibility that coming data has a different size, it nearly
discards the buffer each time, which depends on the protocol HBase uses. For
example, receiving data whose size of 100, 1000, 100, 1000, ..., the average is
500-600 and your logic creates a buffer of the average size or 1000 for each
data.
> [RpcServer] reuse request read buffer
> -------------------------------------
>
> Key: HBASE-14490
> URL: https://issues.apache.org/jira/browse/HBASE-14490
> Project: HBase
> Issue Type: Improvement
> Components: IPC/RPC
> Affects Versions: 2.0.0, 1.0.2
> Reporter: Zephyr Guo
> Assignee: Zephyr Guo
> Labels: performance
> Fix For: 2.0.0, 1.0.2
>
> Attachments: ByteBufferPool.java, HBASE-14490-v1.patch,
> HBASE-14490-v10.patch, HBASE-14490-v11.patch, HBASE-14490-v2.patch,
> HBASE-14490-v3.patch, HBASE-14490-v4.patch, HBASE-14490-v5.patch,
> HBASE-14490-v6.patch, HBASE-14490-v7.patch, HBASE-14490-v8.patch,
> HBASE-14490-v9.patch
>
>
> Reuse buffer to read request.It's not necessary to every request free
> buffer.The idea of optimization is to reduce the times that allocate
> ByteBuffer.
> *Modification*
> 1. {{saslReadAndProcess}} ,{{processOneRpc}}, {{processUnwrappedData}},
> {{processConnectionHeader}} accept a ByteBuffer instead of byte[].They can
> move {{ByteBuffer.position}} correctly when we have read the data.
> 2. {{processUnwrappedData}} no longer use any extra memory.
> 3. Maintaining a reused ByteBuffer in each {{Connection}}.
> a) Size is dynamic to fit specific client.
> b) If size deviate far from average size, we replace a new one.
> c) Using a SoftReference to reference the buffer.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
