[
https://issues.apache.org/jira/browse/HBASE-18294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16268600#comment-16268600
]
Anoop Sam John commented on HBASE-18294:
----------------------------------------
Ya as I said in above comments, we can continue to have decision for flush
based on data size OR heap Size. FOr on heap cases (which is the one what we
have till now) there will be no differences at all. Ya I agree to the concern
that people will see diff wrt blocked writes and so overall throughput.
For off heap there is no need to do so many separate accounting and configs.
Because there also any way we will have heap size based checks. Data size based
check is what mainly govern the flushes there. Because the whole work moves the
heap overhead because of the data size to off heap area. I dont know why this
is not acceptable.
bq.There are claims pro reducing memstore size like for reducing GC, but there
are also claims pro increasing the size to reduce number of flushes, reduce
number of compactions and reduce write amplification.
Ya that is what. You can not say faster flush is better always. It all depends
on the global heap size available for the all regions.
My strong concern is against adding public APIs to set off heap flush size on
heap flush size. We were having only flush size been configured by user. As
every thing in on heap, we ended up deciding based on heap size. That same can
not be equated to exposing heap flush size (flushSize ) and off heap flush size
now.
> Reduce global heap pressure: flush based on heap occupancy
> ----------------------------------------------------------
>
> Key: HBASE-18294
> URL: https://issues.apache.org/jira/browse/HBASE-18294
> Project: HBase
> Issue Type: Improvement
> Affects Versions: 3.0.0
> Reporter: Eshcar Hillel
> Assignee: Eshcar Hillel
> Attachments: HBASE-18294.01.patch, HBASE-18294.02.patch,
> HBASE-18294.03.patch, HBASE-18294.04.patch, HBASE-18294.05.patch,
> HBASE-18294.06.patch
>
>
> A region is flushed if its memory component exceed a threshold (default size
> is 128MB).
> A flush policy decides whether to flush a store by comparing the size of the
> store to another threshold (that can be configured with
> hbase.hregion.percolumnfamilyflush.size.lower.bound).
> Currently the implementation (in both cases) compares the data size
> (key-value only) to the threshold where it should compare the heap size
> (which includes index size, and metadata).
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
