[
https://issues.apache.org/jira/browse/HBASE-14490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14962485#comment-14962485
]
Zephyr Guo commented on HBASE-14490:
------------------------------------
[~stack]
{quote}What is wrong w/ BoundedByteBufferPool ? {quote}
{{BoundedByteBufferPool.getBuffer()}} provide a buffer, but it can't assure
that size is enough.For example, if dataLen=10 and {{getBuffer()}} return a
buffer that size less than 10, we have to allocate again.Patch-11 maintain a
dynamic buffer in {{Connection}}.I think BufferPool is not different with
Patch-11.I do some test and it seems Patch-10 is better.Patch-10 use a fixed
buffer for most small request(such as get operation).
YCSB test result.
{code}
*No reusable buffer:*
[OVERALL], RunTime(ms), 18583.0
[OVERALL], Throughput(ops/sec), 1614.3787332508207
[INSERT], Operations, 30000.0
[INSERT], AverageLatency(us), 563.6561
[OVERALL], RunTime(ms), 7997.0
[OVERALL], Throughput(ops/sec), 1250.4689258471926
[READ], Operations, 10000.0
[READ], AverageLatency(us), 629.6487
*Patch-11:*
[OVERALL], RunTime(ms), 18441.0
[OVERALL], Throughput(ops/sec), 1626.8098259313485
[INSERT], Operations, 30000.0
[INSERT], AverageLatency(us), 540.7597
[OVERALL], RunTime(ms), 7668.0
[OVERALL], Throughput(ops/sec), 1304.1210224308816
[READ], Operations, 10000.0
[READ], AverageLatency(us), 620.7383
*Patch-10(3K size):*
[OVERALL], RunTime(ms), 17313.0
[OVERALL], Throughput(ops/sec), 1732.8019407381737
[INSERT], Operations, 30000.0
[INSERT], AverageLatency(us), 526.9979333333333
[OVERALL], RunTime(ms), 7516.0
[OVERALL], Throughput(ops/sec), 1330.4949441192123
[READ], Operations, 10000.0
[READ], AverageLatency(us), 604.255
{code}
Seems like anoop said older version is better...
stack, what's your suggestion?
> [RpcServer] reuse request read buffer
> -------------------------------------
>
> Key: HBASE-14490
> URL: https://issues.apache.org/jira/browse/HBASE-14490
> Project: HBase
> Issue Type: Improvement
> Components: IPC/RPC
> Affects Versions: 2.0.0, 1.0.2
> Reporter: Zephyr Guo
> Assignee: Zephyr Guo
> Labels: performance
> Fix For: 2.0.0, 1.0.2
>
> Attachments: ByteBufferPool.java, HBASE-14490-v1.patch,
> HBASE-14490-v10.patch, HBASE-14490-v11.patch, HBASE-14490-v2.patch,
> HBASE-14490-v3.patch, HBASE-14490-v4.patch, HBASE-14490-v5.patch,
> HBASE-14490-v6.patch, HBASE-14490-v7.patch, HBASE-14490-v8.patch,
> HBASE-14490-v9.patch
>
>
> Reuse buffer to read request.It's not necessary to every request free
> buffer.The idea of optimization is to reduce the times that allocate
> ByteBuffer.
> *Modification*
> 1. {{saslReadAndProcess}} ,{{processOneRpc}}, {{processUnwrappedData}},
> {{processConnectionHeader}} accept a ByteBuffer instead of byte[].They can
> move {{ByteBuffer.position}} correctly when we have read the data.
> 2. {{processUnwrappedData}} no longer use any extra memory.
> 3. Maintaining a reused ByteBuffer in each {{Connection}}.
> a) Size is dynamic to fit specific client.
> b) If size deviate far from average size, we replace a new one.
> c) Using a SoftReference to reference the buffer.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
