[
https://issues.apache.org/jira/browse/HBASE-14490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14988799#comment-14988799
]
Hiroshi Ikeda commented on HBASE-14490:
---------------------------------------
{code}
+ data = reqBufPool.getBuffer();
+ if (data.capacity() < dataLength) {
+ data = ByteBuffer.allocate(dataLength);
{code}
Is that right usage to throw away a buffer gotten from BoundedByteBufferPool?
What does dumping information means?
{code}
read create total bytes: 1344303
read discard total bytes: 1347255
{code}
What should I interpret the information that discarded total bytes is larger
than created total bytes? It would be better to show the patches that you used.
Did you check the frequency of GC? I'm interested in how much these patches
reduce it, though it also requires a result of the code without patches.
ByteBufferPool I attached can be tuned. It might reduce GC to lower the
threshold to pool buffers .
To tell the truth, checking the size is required before memory allocation in
order to prevent OutOfMemoryError caused by a malicious client.
> [RpcServer] reuse request read buffer
> -------------------------------------
>
> Key: HBASE-14490
> URL: https://issues.apache.org/jira/browse/HBASE-14490
> Project: HBase
> Issue Type: Improvement
> Components: IPC/RPC
> Affects Versions: 2.0.0, 1.0.2
> Reporter: Zephyr Guo
> Assignee: Zephyr Guo
> Labels: performance
> Fix For: 2.0.0, 1.0.2
>
> Attachments: ByteBufferPool.java, HBASE-14490-v1.patch,
> HBASE-14490-v10.patch, HBASE-14490-v11.patch, HBASE-14490-v12.patch,
> HBASE-14490-v2.patch, HBASE-14490-v3.patch, HBASE-14490-v4.patch,
> HBASE-14490-v5.patch, HBASE-14490-v6.patch, HBASE-14490-v7.patch,
> HBASE-14490-v8.patch, HBASE-14490-v9.patch
>
>
> Reuse buffer to read request.It's not necessary to every request free
> buffer.The idea of optimization is to reduce the times that allocate
> ByteBuffer.
> *Modification*
> 1. {{saslReadAndProcess}} ,{{processOneRpc}}, {{processUnwrappedData}},
> {{processConnectionHeader}} accept a ByteBuffer instead of byte[].They can
> move {{ByteBuffer.position}} correctly when we have read the data.
> 2. {{processUnwrappedData}} no longer use any extra memory.
> 3. Maintaining a reused ByteBuffer in each {{Connection}}.
> a) Size is dynamic to fit specific client.
> b) If size deviate far from average size, we replace a new one.
> c) Using a SoftReference to reference the buffer.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
