[
https://issues.apache.org/jira/browse/HBASE-14490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14950402#comment-14950402
]
Zephyr Guo commented on HBASE-14490:
------------------------------------
{quote}We keep on adding to the reqTotalSize. Possible overflow?{quote}
It can hold 8000TB.For more correct, we can calculate average by other way.
{quote}If we do to this extend and handle both smaller sized reqs and larger
sized (write req) we better use a BufferPool?We already have it and using it
while sending back response.{quote}
I know {{BoundedByteBufferPool}} and I think it does not fit request.
Look at the {{getBuffer()}}.
{code}
public ByteBuffer getBuffer() {
ByteBuffer bb = null;
// It can cause blocking. We don't need lock because Reader is single-thread.
// If we use a BoundedByteBufferPool in all Reader, we will increase possible
of blocking.
// If we maintain a buffer in Connection, we can get most suitable size for
specific client.
lock.lock();
try {
bb = this.buffers.poll();
if (bb != null) {
this.totalReservoirCapacity -= bb.capacity();
}
} finally {
lock.unlock();
}
if (bb != null) {
bb.clear();
} else {
// It can not provide enough capacity.We will allocate again.
bb = ByteBuffer.allocate(this.runningAverage);
this.allocations.incrementAndGet();
}
if (LOG.isTraceEnabled()) {
LOG.trace("runningAverage=" + this.runningAverage +
", totalCapacity=" + this.totalReservoirCapacity + ", count=" +
this.buffers.size() +
", alloctions=" + this.allocations.get());
}
return bb;
}
{code}
How about other better BufferPool?I think is not necessary and that'd work
too.So simple way is better.
{quote}So the older version for reusing smaller sized BB is better? Any test
result numbers? {quote}
I'm trying to test.
> [RpcServer] reuse request read buffer
> -------------------------------------
>
> Key: HBASE-14490
> URL: https://issues.apache.org/jira/browse/HBASE-14490
> Project: HBase
> Issue Type: Improvement
> Components: IPC/RPC
> Affects Versions: 2.0.0, 1.0.2
> Reporter: Zephyr Guo
> Assignee: Zephyr Guo
> Labels: performance
> Fix For: 2.0.0, 1.0.2
>
> Attachments: ByteBufferPool.java, HBASE-14490-v1.patch,
> HBASE-14490-v10.patch, HBASE-14490-v11.patch, HBASE-14490-v2.patch,
> HBASE-14490-v3.patch, HBASE-14490-v4.patch, HBASE-14490-v5.patch,
> HBASE-14490-v6.patch, HBASE-14490-v7.patch, HBASE-14490-v8.patch,
> HBASE-14490-v9.patch
>
>
> Reuse buffer to read request.It's not necessary to every request free
> buffer.The idea of optimization is to reduce the times that allocate
> ByteBuffer.
> *Modification*
> 1. {{saslReadAndProcess}} ,{{processOneRpc}}, {{processUnwrappedData}},
> {{processConnectionHeader}} accept a ByteBuffer instead of byte[].They can
> move {{ByteBuffer.position}} correctly when we have read the data.
> 2. {{processUnwrappedData}} no longer use any extra memory.
> 3. Maintaining a reused ByteBuffer in each {{Connection}}.
> a) Size is dynamic to fit specific client.
> b) If size deviate far from average size, we replace a new one.
> c) Using a SoftReference to reference the buffer.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
