[
https://issues.apache.org/jira/browse/HBASE-18294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16300507#comment-16300507
]
Eshcar Hillel commented on HBASE-18294:
---------------------------------------
Hey [~jmspaggi],
We are definitely going to use the 128MB limit, only that we will compare it to
the total heap size occupancy of the store/region and not just to the size of
the data.
For example, consider a store in which the data of each cell is 100B. We have
noticed in our experiments that the (average) heap size of each such cell is
*150B*.
Which means -- if we have 1M records in memory the data size is only 100MB,
while the heap size is 150MB.
If we compare the data size against the threshold we do not flush yet, while if
we compare the heap size this store was flushed to disk when it exceeded 128MB.
> Reduce global heap pressure: flush based on heap occupancy
> ----------------------------------------------------------
>
> Key: HBASE-18294
> URL: https://issues.apache.org/jira/browse/HBASE-18294
> Project: HBase
> Issue Type: Improvement
> Affects Versions: 3.0.0
> Reporter: Eshcar Hillel
> Assignee: Eshcar Hillel
> Attachments: HBASE-18294.01.patch, HBASE-18294.02.patch,
> HBASE-18294.03.patch, HBASE-18294.04.patch, HBASE-18294.05.patch,
> HBASE-18294.06.patch, HBASE-18294.07.patch, HBASE-18294.07.patch
>
>
> A region is flushed if its memory component exceed a threshold (default size
> is 128MB).
> A flush policy decides whether to flush a store by comparing the size of the
> store to another threshold (that can be configured with
> hbase.hregion.percolumnfamilyflush.size.lower.bound).
> Currently the implementation (in both cases) compares the data size
> (key-value only) to the threshold where it should compare the heap size
> (which includes index size, and metadata).
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
