[
https://issues.apache.org/jira/browse/HBASE-13819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15209401#comment-15209401
]
deepankar commented on HBASE-13819:
-----------------------------------
bq. Because we are not returning BB to the pool? The pool is growing w/o bound?
I think there are no leaks in BB from the analysis on the heap dump, all the
objects were accounted
bq. We should add these if not present at TRACE level.
Sorry My comment was misled, by debug statements I meant by enabling TRACE
(these loggings were most useful stuff in many debugging scenarios)
bq. So 2G instead of 1G? But the pool is bounded?
bq. The responder is not keeping up? It is not moving stuff out of the server
fast enough?
bq. I am interested in leaks; how they are happening.
No concrete evidence that responder is not able to keep up, but the bound in
pool does not help this case because we create a new BB when one is not present
in the pool and occasionally (we are observing once in 2 - 3 days) there will
be spew when returns to pool grows above the configured threshold.
>From the analysis we did there are no leaks
bq. Where is pendingCallsQueue?
The queue per connection object RpcServer$Connection.responseQueue
bq. Did you observe the offheap size used growing? There s a metric IIRC.
Yes we saw this in the metric (hbase.regionserver.direct.MemoryUsed)
bq. Where would the fixed size be? In BBBP they eventually reach fixed size?
Yes they eventually reach fixed size, but the size is that of the larger
response sizes rather than median or some smaller number
> Make RPC layer CellBlock buffer a DirectByteBuffer
> --------------------------------------------------
>
> Key: HBASE-13819
> URL: https://issues.apache.org/jira/browse/HBASE-13819
> Project: HBase
> Issue Type: Sub-task
> Components: Scanners
> Reporter: Anoop Sam John
> Assignee: Anoop Sam John
> Fix For: 2.0.0, 1.3.0
>
> Attachments: HBASE-13819.patch, HBASE-13819_branch-1.patch,
> HBASE-13819_branch-1.patch, HBASE-13819_branch-1.patch
>
>
> In RPC layer, when we make a cellBlock to put as RPC payload, we will make an
> on heap byte buffer (via BoundedByteBufferPool). The pool will keep upto
> certain number of buffers. This jira aims at testing possibility for making
> this buffers off heap ones. (DBB) The advantages
> 1. Unsafe based writes to off heap is faster than that to on heap. Now we are
> not using unsafe based writes at all. Even if we add, DBB will be better
> 2. When Cells are backed by off heap (HBASE-11425) off heap to off heap
> writes will be better
> 3. When checked the code in SocketChannel impl, if we pass a HeapByteBuffer
> to the socket channel, it will create a temp DBB and copy data to there and
> only DBBs will be moved to Sockets. If we make DBB 1st hand itself, we can
> avoid this one more level of copying.
> Will do different perf testing with changed and report back.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
