[
https://issues.apache.org/jira/browse/HBASE-18294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16304674#comment-16304674
]
Anoop Sam John commented on HBASE-18294:
----------------------------------------
Yes Stack. That is why we have || check. We have a limit on both. We wont
allow any of the size to grow above the limit.
bq.What about the case where overhead is more than data size where key is
larger than value?
In off heap memstore, even the key is in off heap. The key , value and tag
parts. WHat is there in on heap (java heap) is the POJO overhead and CSLM
overhead. Ya even that can be larger than the key+value+tags. But will that
be that normal? May be rare IMO. U can correct me. Any way we have the ||
check.. We can do like what u proposed above that even for the off heap also
we can consider the dataSize+ overhead. But then that way have to continue for
ever. Now the on heap case we can not change because that is a drastic behave
change. At first (when impl other jira) we all were thinking that change will
make /align things to intended way.. But later feedbacks are not that. So we
will have to continue with that for ever/ For off heap also, if we do now, we
have to continue for ever IMO. Ya if u strongly feel that is to be the case
(we say it loudly that 128 MB region flush size means 128 MB size of cells data
+ all their overhead), I will stand by that.
> Reduce global heap pressure: flush based on heap occupancy
> ----------------------------------------------------------
>
> Key: HBASE-18294
> URL: https://issues.apache.org/jira/browse/HBASE-18294
> Project: HBase
> Issue Type: Improvement
> Affects Versions: 3.0.0
> Reporter: Eshcar Hillel
> Assignee: Eshcar Hillel
> Attachments: HBASE-18294.01.patch, HBASE-18294.02.patch,
> HBASE-18294.03.patch, HBASE-18294.04.patch, HBASE-18294.05.patch,
> HBASE-18294.06.patch, HBASE-18294.07.patch, HBASE-18294.07.patch,
> HBASE-18294.08.patch, HBASE-18294.09.patch, HBASE-18294.10.patch,
> HBASE-18294.11.patch, HBASE-18294.11.patch, HBASE-18294.12.patch,
> HBASE-18294.13.patch
>
>
> A region is flushed if its memory component exceed a threshold (default size
> is 128MB).
> A flush policy decides whether to flush a store by comparing the size of the
> store to another threshold (that can be configured with
> hbase.hregion.percolumnfamilyflush.size.lower.bound).
> Currently the implementation (in both cases) compares the data size
> (key-value only) to the threshold where it should compare the heap size
> (which includes index size, and metadata).
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
