[
https://issues.apache.org/jira/browse/HBASE-18294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16305040#comment-16305040
]
Anoop Sam John commented on HBASE-18294:
----------------------------------------
bq.but Eshcar Hillel's patch needs to include offheap datasize as Anoop Sam
John says.
And for that the solution said was to change heapSize impl of the ByteBufferKV.
May be not that apt IMO. Ya heapSize() is intended to track the on heap
sizes.. Yes we were all on heap till now. All these naming confuses us now.
For simplicity we can even keep like for off heap cases we consider data size
only. That is what the per region flush decision is based on an || condition.
The change is supposed to be same. We dont have off heap usage as default.
Any one trying that would thoroughly tune things and test. We can correct it
always , that the off heap also follow same way of data size + overhead. At
least I believe that all agree to the point that the flush size would best mean
the sum of data size when to flush. In theory at least. Practical wise we
could not follow this as we were always under the limits of java heap.
> Reduce global heap pressure: flush based on heap occupancy
> ----------------------------------------------------------
>
> Key: HBASE-18294
> URL: https://issues.apache.org/jira/browse/HBASE-18294
> Project: HBase
> Issue Type: Improvement
> Affects Versions: 3.0.0
> Reporter: Eshcar Hillel
> Assignee: Eshcar Hillel
> Fix For: 2.0.0-beta-2
>
> Attachments: HBASE-18294.01.patch, HBASE-18294.02.patch,
> HBASE-18294.03.patch, HBASE-18294.04.patch, HBASE-18294.05.patch,
> HBASE-18294.06.patch, HBASE-18294.07.patch, HBASE-18294.07.patch,
> HBASE-18294.08.patch, HBASE-18294.09.patch, HBASE-18294.10.patch,
> HBASE-18294.11.patch, HBASE-18294.11.patch, HBASE-18294.12.patch,
> HBASE-18294.13.patch
>
>
> A region is flushed if its memory component exceed a threshold (default size
> is 128MB).
> A flush policy decides whether to flush a store by comparing the size of the
> store to another threshold (that can be configured with
> hbase.hregion.percolumnfamilyflush.size.lower.bound).
> Currently the implementation (in both cases) compares the data size
> (key-value only) to the threshold where it should compare the heap size
> (which includes index size, and metadata).
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
