[jira] [Comment Edited] (HBASE-14978) Don't allow Multi to retain too many blocks
[ https://issues.apache.org/jira/browse/HBASE-14978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15947372#comment-15947372 ] stack edited comment on HBASE-14978 at 3/29/17 3:46 PM: Makes sense [~anoop.hbase] Should be easy enough to verify. We should just not do this trick when Cell is backed by BucketCache? Open new issue? Can we get a metric in here on how effective (or frustrating) this change is? was (Author: stack): Makes sense [~anoop.hbase] Should be easy enough to verify. We should just not do this trick when Cell is backed by BucketCache? > Don't allow Multi to retain too many blocks > --- > > Key: HBASE-14978 > URL: https://issues.apache.org/jira/browse/HBASE-14978 > Project: HBase > Issue Type: Improvement > Components: io, IPC/RPC, regionserver >Affects Versions: 2.0.0, 1.2.0, 1.3.0 >Reporter: Elliott Clark >Assignee: Elliott Clark >Priority: Blocker > Fix For: 2.0.0, 1.2.0, 1.3.0 > > Attachments: HBASE-14978-branch-1.2.patch, > HBASE-14978-branch-1.2.patch, HBASE-14978-branch-1.2.patch, > HBASE-14978-branch-1.2.patch, HBASE-14978.patch, HBASE-14978-v1.patch, > HBASE-14978-v2.patch, HBASE-14978-v3.patch, HBASE-14978-v4.patch > > > Scans and Multi's have limits on the total size of cells that can be > returned. However if those requests are not all pointing at the same blocks > then the KeyValues can keep alive a lot more data than their size. > Take the following example: > A multi with a list of 1 gets to a fat row. Each column being returned in > in a different block. Each column is small 32 bytes or so. > So the total cell size will be 32 * 1 = ~320kb. However if each block is > 128k then total retained heap size will be almost 2gigs. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Comment Edited] (HBASE-14978) Don't allow Multi to retain too many blocks
[
https://issues.apache.org/jira/browse/HBASE-14978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15057602#comment-15057602
]
Anoop Sam John edited comment on HBASE-14978 at 12/15/15 8:33 AM:
--
So with off heap Cells, withe very cell, we will treat it as coming from a new
block?
In fact with off heap Cell, which are delivered from L2 off heap cache, we wont
get any OOME issue whatever be the #blocks one request touches. We avoid
copying of block data from L2 off heap space to temp byte[]. So these Cells
refer to the L2 cache space only.
bq.if (c.getRowArray() != lastBlock) {
Even with array backed cells also this wont work as expected when we use DBE.
In that case every Cell will have new byte[] backing the key parts at least..
Now with Cells in place every where, this kind of assumption taking is not
correct IMHO.
was (Author: anoop.hbase):
So with off heap Cells, withe very cell, we will treat it as coming from a new
block?
bq.if (c.getRowArray() != lastBlock) {
Even with array backed cells also this wont work as expected when we use DBE.
In that case every Cell will have new byte[] backing the key parts at least..
Now with Cells in place every where, this kind of assumption taking is not
correct IMHO.
> Don't allow Multi to retain too many blocks
> ---
>
> Key: HBASE-14978
> URL: https://issues.apache.org/jira/browse/HBASE-14978
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 2.0.0, 1.2.0, 1.3.0
>Reporter: Elliott Clark
>Assignee: Elliott Clark
>Priority: Critical
> Attachments: HBASE-14978-v1.patch, HBASE-14978.patch
>
>
> Scans and Multi's have limits on the total size of cells that can be
> returned. However if those requests are not all pointing at the same blocks
> then the KeyValues can keep alive a lot more data than their size.
> Take the following example:
> A multi with a list of 1 gets to a fat row. Each column being returned in
> in a different block. Each column is small 32 bytes or so.
> So the total cell size will be 32 * 1 = ~320kb. However if each block is
> 128k then total retained heap size will be almost 2gigs.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
