[
https://issues.apache.org/jira/browse/HBASE-9535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13767927#comment-13767927
]
stack commented on HBASE-9535:
------------------------------
[~vrodionov] Thread local could work. There is a bit of a mismatch though
since we read in one thread and the processing is done in another altogether
(There are 10 Reader threads. We'd need the buffer available at read time.
They read the request, put it on a queue from which the Handler threads pick it
up. There are usually many more handlers than Readers (50 Handlers by default
but could be hundreds). Maybe we should purge the Reader+queuing step and just
have Handlers do the read ([~nkeywal] what you think?)
> Try a pool of direct byte buffers handling incoming ipc requests
> ----------------------------------------------------------------
>
> Key: HBASE-9535
> URL: https://issues.apache.org/jira/browse/HBASE-9535
> Project: HBase
> Issue Type: Brainstorming
> Reporter: stack
> Assignee: stack
>
> ipc takes in a query by allocating a ByteBuffer of the size of the request
> and then reading off the socket into this on-heap BB.
> Experiment with keeping a pool of BBs so we have some buffer reuse to cut on
> garbage generated. Could checkout from pool in RpcServer#Reader. Could
> check back into the pool when Handler is done just before it queues the
> response on the Responder's queue. We should be good since, at least for
> now, kvs get copied up into MSLAB (not references) when data gets stuffed
> into MemStore; this should make it so no references left over when we check
> the BB back into the pool for use next time around.
> If on-heap BBs work, we could then try direct BBs (Allocation of DBBs takes
> time so if already allocated, should be good. GC of DBBs is a pain but if in
> a pool, we shouldn't be wanting this to happen). The copy from socket to the
> DBB will be off-heap (should be fast).
> Could start w/ the HDFS DirectBufferPool. It is unbounded and keeps items by
> size (we might want to bypass the pool if an object is > size N).
> DBBs for this task would contend w/ offheap BBs used in BlockReadLocal when
> short-circuit reading. It'd be a bummer if we had to allocate big objects
> on-heap. Would still be an improvement.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
