[
https://issues.apache.org/jira/browse/HBASE-13819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15208009#comment-15208009
]
deepankar commented on HBASE-13819:
-----------------------------------
What [~anoop.hbase] said is correct, in our case avg response sizes are around
10 -16 kb but the max size is 512kb if you run for significantly longer time
the running avg goest toward 512kb. Another point its not like that there are
lot of request if you have around 200 clients and lets say you have around 5 %
are GCing then 10 clients with around 400 pending reqs will lead to 4000
pending requests and this leads to exhaustion of the direct memory we allocated
(with buffer sizes 512 kb) but in general the overall pending response size for
these 4000 requests is only 82 MB its atleast 1 - 2 orders of magnitude of
space getting occupied extra.
I think fixed size BBs is an excellent idea that will definitely be useful to
get out of this issue.
> Make RPC layer CellBlock buffer a DirectByteBuffer
> --------------------------------------------------
>
> Key: HBASE-13819
> URL: https://issues.apache.org/jira/browse/HBASE-13819
> Project: HBase
> Issue Type: Sub-task
> Components: Scanners
> Reporter: Anoop Sam John
> Assignee: Anoop Sam John
> Fix For: 2.0.0, 1.3.0
>
> Attachments: HBASE-13819.patch, HBASE-13819_branch-1.patch,
> HBASE-13819_branch-1.patch, HBASE-13819_branch-1.patch
>
>
> In RPC layer, when we make a cellBlock to put as RPC payload, we will make an
> on heap byte buffer (via BoundedByteBufferPool). The pool will keep upto
> certain number of buffers. This jira aims at testing possibility for making
> this buffers off heap ones. (DBB) The advantages
> 1. Unsafe based writes to off heap is faster than that to on heap. Now we are
> not using unsafe based writes at all. Even if we add, DBB will be better
> 2. When Cells are backed by off heap (HBASE-11425) off heap to off heap
> writes will be better
> 3. When checked the code in SocketChannel impl, if we pass a HeapByteBuffer
> to the socket channel, it will create a temp DBB and copy data to there and
> only DBBs will be moved to Sockets. If we make DBB 1st hand itself, we can
> avoid this one more level of copying.
> Will do different perf testing with changed and report back.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
