[
https://issues.apache.org/jira/browse/HBASE-15525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15209785#comment-15209785
]
Anoop Sam John commented on HBASE-15525:
----------------------------------------
For writing the CellBlock response, we really dont need the whole of it in one
BB. While writing the response to socket, we have a BufferChain. The Cell
block can also be backed by N BBs rather than 1. Even within BufferChain, can
see that the actual write to socket happens as chunks of 64 KB size. So if we
have a cell block of 512 KB and instead of one BB, we have 4 BBs of size 128 KB
each, the socket write wont take more time as the total #writes in both case
will be same.
With a model of writing cell block to N smaller sized BBs, we can avoid the
need for regrow (In BBOutputStream) and copy from old smaller sized BB to new
BB. We need a new BBOS which can do this way of write. This one will passed
to Encoder for writing the Cells to form cell block.
> OutOfMemory could occur when using BoundedByteBufferPool during RPC bursts
> --------------------------------------------------------------------------
>
> Key: HBASE-15525
> URL: https://issues.apache.org/jira/browse/HBASE-15525
> Project: HBase
> Issue Type: Bug
> Components: IPC/RPC
> Reporter: deepankar
> Assignee: Anoop Sam John
> Priority: Critical
>
> After HBASE-13819 the system some times run out of direct memory whenever
> there is some network congestion or some client side issues.
> This was because of pending RPCs in the RPCServer$Connection.responseQueue
> and since all the responses in this queue hold a buffer for cellblock from
> BoundedByteBufferPool this could takeup a lot of memory if the
> BoundedByteBufferPool's moving average settles down towards a higher value
> See the discussion here
> [HBASE-13819-comment|https://issues.apache.org/jira/browse/HBASE-13819?focusedCommentId=15207822&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15207822]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
