[
https://issues.apache.org/jira/browse/HBASE-9535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13770622#comment-13770622
]
Liang Xie commented on HBASE-9535:
----------------------------------
to me, seems the RPC's "ByteBuffer.allocate" is not in the top contributor list
for gc.
I added logs below
{code}
data = ByteBuffer.allocate(dataLength);
{code}
in SecureServer.java (my test env is security enabled)
for a 100%read ycsb scenario, yes, this log shows the data lengths are similar
with each other(that means it's very easier to reuse/pool the byte buffers),
but after a raw estimation:
my config: Xmn512m Xmx4096m
my result: SecureServer.readAndProcess, dataLength:162;read throughput 7000ops;
"jstat -gcutil" shows about 5ygc per second
"7000 * 162" is just 1MB more or less, is just a little portion of Xmn size, so
per my view, seems it doesn't very helpful to improve gc...
> Try a pool of direct byte buffers handling incoming ipc requests
> ----------------------------------------------------------------
>
> Key: HBASE-9535
> URL: https://issues.apache.org/jira/browse/HBASE-9535
> Project: HBase
> Issue Type: Brainstorming
> Reporter: stack
> Assignee: stack
>
> ipc takes in a query by allocating a ByteBuffer of the size of the request
> and then reading off the socket into this on-heap BB.
> Experiment with keeping a pool of BBs so we have some buffer reuse to cut on
> garbage generated. Could checkout from pool in RpcServer#Reader. Could
> check back into the pool when Handler is done just before it queues the
> response on the Responder's queue. We should be good since, at least for
> now, kvs get copied up into MSLAB (not references) when data gets stuffed
> into MemStore; this should make it so no references left over when we check
> the BB back into the pool for use next time around.
> If on-heap BBs work, we could then try direct BBs (Allocation of DBBs takes
> time so if already allocated, should be good. GC of DBBs is a pain but if in
> a pool, we shouldn't be wanting this to happen). The copy from socket to the
> DBB will be off-heap (should be fast).
> Could start w/ the HDFS DirectBufferPool. It is unbounded and keeps items by
> size (we might want to bypass the pool if an object is > size N).
> DBBs for this task would contend w/ offheap BBs used in BlockReadLocal when
> short-circuit reading. It'd be a bummer if we had to allocate big objects
> on-heap. Would still be an improvement.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
