[
https://issues.apache.org/jira/browse/HBASE-13819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16279648#comment-16279648
]
Anoop Sam John commented on HBASE-13819:
----------------------------------------
The Direct memory (DBB) we allocate for only pooling purpose. The pool is
having a fixed max size and as u see, the BBs are not freed. Put back just put
the DBB back to pool for later use. When the requests read can not find a
free BB in pool, we dont create one DBB. We just create on heap only. On demand
DBB create, we are NOT doing any where. That will be more dangerous IMO.. So
the DBB exhaust ya possible.. Because as u know, the NIO , if we pass a HBB,
it creates a DBB on demand (it is having its own pooling also) and read to that
1st and then copy to passed HBB. ThreadLocal based pooling in NIO is again
more problematic. Seeing more on how Netty handles things. FOr reading reqs,
Netty can read into HBB directly? In all , there are some possible cases of
Direct memory exhaust. Or more GC time. Which all can kill the RS..
> Make RPC layer CellBlock buffer a DirectByteBuffer
> --------------------------------------------------
>
> Key: HBASE-13819
> URL: https://issues.apache.org/jira/browse/HBASE-13819
> Project: HBase
> Issue Type: Sub-task
> Components: Scanners
> Reporter: Anoop Sam John
> Assignee: Anoop Sam John
> Fix For: 2.0.0, 1.3.0
>
> Attachments: HBASE-13819.patch, HBASE-13819_branch-1.patch,
> HBASE-13819_branch-1.patch, HBASE-13819_branch-1.patch, q.png
>
>
> In RPC layer, when we make a cellBlock to put as RPC payload, we will make an
> on heap byte buffer (via BoundedByteBufferPool). The pool will keep upto
> certain number of buffers. This jira aims at testing possibility for making
> this buffers off heap ones. (DBB) The advantages
> 1. Unsafe based writes to off heap is faster than that to on heap. Now we are
> not using unsafe based writes at all. Even if we add, DBB will be better
> 2. When Cells are backed by off heap (HBASE-11425) off heap to off heap
> writes will be better
> 3. When checked the code in SocketChannel impl, if we pass a HeapByteBuffer
> to the socket channel, it will create a temp DBB and copy data to there and
> only DBBs will be moved to Sockets. If we make DBB 1st hand itself, we can
> avoid this one more level of copying.
> Will do different perf testing with changed and report back.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
