[
https://issues.apache.org/jira/browse/HBASE-9535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-9535:
-------------------------
Resolution: Implemented
Status: Resolved (was: Patch Available)
Closing as 'implemented' by HBASE-13142.
The approach in HBASE-13142 differs from what is here in that rather than have
a pool of buffers of various sizes, instead we have a pool of buffers whose
sizes all climb to some upper bound. If the RPC is of a size beyond this buffer
size, we allocate a buffer just for this rpc.
HBASE-13142 has numbers on benefit most apparent in CPU and GC when rowsize is
10s of ks.
> Try a pool of direct byte buffers handling incoming ipc requests
> ----------------------------------------------------------------
>
> Key: HBASE-9535
> URL: https://issues.apache.org/jira/browse/HBASE-9535
> Project: HBase
> Issue Type: Brainstorming
> Reporter: stack
> Assignee: Nicolas Liochon
> Attachments: 9535.v1.patch, 9535.v2-trunk.patch, 9535.v2.patch,
> 9535.v3.patch
>
>
> ipc takes in a query by allocating a ByteBuffer of the size of the request
> and then reading off the socket into this on-heap BB.
> Experiment with keeping a pool of BBs so we have some buffer reuse to cut on
> garbage generated. Could checkout from pool in RpcServer#Reader. Could
> check back into the pool when Handler is done just before it queues the
> response on the Responder's queue. We should be good since, at least for
> now, kvs get copied up into MSLAB (not references) when data gets stuffed
> into MemStore; this should make it so no references left over when we check
> the BB back into the pool for use next time around.
> If on-heap BBs work, we could then try direct BBs (Allocation of DBBs takes
> time so if already allocated, should be good. GC of DBBs is a pain but if in
> a pool, we shouldn't be wanting this to happen). The copy from socket to the
> DBB will be off-heap (should be fast).
> Could start w/ the HDFS DirectBufferPool. It is unbounded and keeps items by
> size (we might want to bypass the pool if an object is > size N).
> DBBs for this task would contend w/ offheap BBs used in BlockReadLocal when
> short-circuit reading. It'd be a bummer if we had to allocate big objects
> on-heap. Would still be an improvement.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
