[
https://issues.apache.org/jira/browse/HBASE-18294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16264974#comment-16264974
]
ramkrishna.s.vasudevan commented on HBASE-18294:
------------------------------------------------
bq.First, I noticed that master suffer major performance degradation w.r.t.
branch-2
Can you file a JIRA for this if you have not already done.
bq. We see that looking at the heap size instead of data size causes more disk
flushes, since each store trigger flushes more frequently. However, the
throughput increases significantly as we never reach global heap pressure.
So when you say considering only the delta of the current patch - you are just
trying to see per region flush as data + heap and so the global pressure
reduces. Because we do more region level flushes.
>From code one thing I noticed is that in the current master branch - when we
>decide there is a lower mark breach for the memstore we decide based on both
>data and heap size. (for onheap memstores). But the region that we select is
>only by region data size. and I think that is why we are suggesting a change
>there.
So probably for onheap cases we should check (data + heap > 128MB) while
selecting the best region and take it for flush.
I think this is why the argument of increasing the existing default values
comes. Because our decision metric has changed.
> Reduce global heap pressure: flush based on heap occupancy
> ----------------------------------------------------------
>
> Key: HBASE-18294
> URL: https://issues.apache.org/jira/browse/HBASE-18294
> Project: HBase
> Issue Type: Improvement
> Affects Versions: 3.0.0
> Reporter: Eshcar Hillel
> Assignee: Eshcar Hillel
> Attachments: HBASE-18294.01.patch, HBASE-18294.02.patch,
> HBASE-18294.03.patch, HBASE-18294.04.patch, HBASE-18294.05.patch,
> HBASE-18294.06.patch
>
>
> A region is flushed if its memory component exceed a threshold (default size
> is 128MB).
> A flush policy decides whether to flush a store by comparing the size of the
> store to another threshold (that can be configured with
> hbase.hregion.percolumnfamilyflush.size.lower.bound).
> Currently the implementation (in both cases) compares the data size
> (key-value only) to the threshold where it should compare the heap size
> (which includes index size, and metadata).
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
