[
https://issues.apache.org/jira/browse/HBASE-15525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15215397#comment-15215397
]
Anoop Sam John commented on HBASE-15525:
----------------------------------------
Yes Stack, while reading request into the BB and writing response (Using
ByteBufferChain), we do read/write from/to socket channel in 64 KB chunks. So
ya while reading also, we can read into N buffers. This is some thing I
discussed with you offline some time back :-)
Ya the pool will be having all fixed sized BBs and off heap. The max #BBs in
the pool can be defined by customer. (Default to 2 * handlers).. We do we have
this config already. The name of the config is bit odd to me .. We will
discuss that.
There is no initial buffer size and max buffer size.. Only one config using
which one can define the size of the BBs in the pool. Default of this is set
as 64KB same as the socket chunk size. If for a use case it is sure that most
of the time the response size will be lesser than this, they can reduce it.
For scan kind of usage advisable not to change this size. When the response
size is expected to be more than 64KB also, better not change as this is the
chunk size. So even if the buffer size is 256KB we will do 4 writes to socket.
And ya the main another diff is when pool is not able to find a free BB (after
reaching the max capacity), pool will return null.. And the response will
allocate new BB in on heap area.
> OutOfMemory could occur when using BoundedByteBufferPool during RPC bursts
> --------------------------------------------------------------------------
>
> Key: HBASE-15525
> URL: https://issues.apache.org/jira/browse/HBASE-15525
> Project: HBase
> Issue Type: Bug
> Components: IPC/RPC
> Reporter: deepankar
> Assignee: Anoop Sam John
> Priority: Critical
> Attachments: WIP.patch
>
>
> After HBASE-13819 the system some times run out of direct memory whenever
> there is some network congestion or some client side issues.
> This was because of pending RPCs in the RPCServer$Connection.responseQueue
> and since all the responses in this queue hold a buffer for cellblock from
> BoundedByteBufferPool this could takeup a lot of memory if the
> BoundedByteBufferPool's moving average settles down towards a higher value
> See the discussion here
> [HBASE-13819-comment|https://issues.apache.org/jira/browse/HBASE-13819?focusedCommentId=15207822&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15207822]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
