[
https://issues.apache.org/jira/browse/HBASE-13819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15209427#comment-15209427
]
stack commented on HBASE-13819:
-------------------------------
Good to hear you don't think there leaking going on.
bq. No concrete evidence that responder is not able to keep up, but the bound
in pool does not help this case because we create a new BB when one is not
present in the pool and occasionally (we are observing once in 2 - 3 days)
there will be spew when returns to pool grows above the configured threshold.
Messages saying ... LOG.warn("Overflowed total capacity."); or ....
LOG.warn("At capacity: " + countOfBuffers); ?
They don't look that useful. Might be good though to report detail around the
burst in traffic so could adjust sizing....
What you think of Anoop's idea of the BB being allocated onheap rather than
offheap if we can't get it from the pool? The allocation would be faster...
So, growing because of a burst in traffic.. but no leaking... because when the
burst passes, we seem to shrink down again (if noisy about it...)
[~dvdreddy] Would you mind opening a new issue describing how you would like
this to work? Its great that you fellas are running it and uncovering this
stuff. It would be easy enough to do you up patches for you to try since it so
contained. Thanks.
> Make RPC layer CellBlock buffer a DirectByteBuffer
> --------------------------------------------------
>
> Key: HBASE-13819
> URL: https://issues.apache.org/jira/browse/HBASE-13819
> Project: HBase
> Issue Type: Sub-task
> Components: Scanners
> Reporter: Anoop Sam John
> Assignee: Anoop Sam John
> Fix For: 2.0.0, 1.3.0
>
> Attachments: HBASE-13819.patch, HBASE-13819_branch-1.patch,
> HBASE-13819_branch-1.patch, HBASE-13819_branch-1.patch, q.png
>
>
> In RPC layer, when we make a cellBlock to put as RPC payload, we will make an
> on heap byte buffer (via BoundedByteBufferPool). The pool will keep upto
> certain number of buffers. This jira aims at testing possibility for making
> this buffers off heap ones. (DBB) The advantages
> 1. Unsafe based writes to off heap is faster than that to on heap. Now we are
> not using unsafe based writes at all. Even if we add, DBB will be better
> 2. When Cells are backed by off heap (HBASE-11425) off heap to off heap
> writes will be better
> 3. When checked the code in SocketChannel impl, if we pass a HeapByteBuffer
> to the socket channel, it will create a temp DBB and copy data to there and
> only DBBs will be moved to Sockets. If we make DBB 1st hand itself, we can
> avoid this one more level of copying.
> Will do different perf testing with changed and report back.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
