[
https://issues.apache.org/jira/browse/HBASE-9535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13834219#comment-13834219
]
Nicolas Liochon commented on HBASE-9535:
----------------------------------------
v2 is still incomplete, but does work now. I think I have to use some guava
stuff to use a fifo policy on the pool however.
I've got a 7% improvement on this one, with my YCSB + empty table test. It's
not bad, and there is another advantage: as we allocate the memory off heap, it
does not interfere with the GC, so it's good when we have a server with a lot
of memory that is so difficult to use in Java.
It's not a full test, I need to look more at what the gc is doing.
bq. So it is quite important to use BBs of the same size only.
The hadoop pool has a very simple policy for this: it takes the first buffer
"big enough" instead of "at the same size". This seems smart. It could be even
smarter to use the "smallest big enough one".
> Try a pool of direct byte buffers handling incoming ipc requests
> ----------------------------------------------------------------
>
> Key: HBASE-9535
> URL: https://issues.apache.org/jira/browse/HBASE-9535
> Project: HBase
> Issue Type: Brainstorming
> Reporter: stack
> Assignee: stack
> Attachments: 9535.v1.patch, 9535.v2.patch
>
>
> ipc takes in a query by allocating a ByteBuffer of the size of the request
> and then reading off the socket into this on-heap BB.
> Experiment with keeping a pool of BBs so we have some buffer reuse to cut on
> garbage generated. Could checkout from pool in RpcServer#Reader. Could
> check back into the pool when Handler is done just before it queues the
> response on the Responder's queue. We should be good since, at least for
> now, kvs get copied up into MSLAB (not references) when data gets stuffed
> into MemStore; this should make it so no references left over when we check
> the BB back into the pool for use next time around.
> If on-heap BBs work, we could then try direct BBs (Allocation of DBBs takes
> time so if already allocated, should be good. GC of DBBs is a pain but if in
> a pool, we shouldn't be wanting this to happen). The copy from socket to the
> DBB will be off-heap (should be fast).
> Could start w/ the HDFS DirectBufferPool. It is unbounded and keeps items by
> size (we might want to bypass the pool if an object is > size N).
> DBBs for this task would contend w/ offheap BBs used in BlockReadLocal when
> short-circuit reading. It'd be a bummer if we had to allocate big objects
> on-heap. Would still be an improvement.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
