[
https://issues.apache.org/jira/browse/HBASE-21029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16576479#comment-16576479
]
Ted Yu edited comment on HBASE-21029 at 8/10/18 3:59 PM:
---------------------------------------------------------
I applied the test change only in master branch and ran:
-Dtest=TestDefaultMemStore#testPutSameCell
The test passed.
Can you come up with test which fails without change to Segment.java ?
Thanks
was (Author: yuzhih...@gmail.com):
I applied the test change only and ran:
-Dtest=TestDefaultMemStore#testPutSameCell
The test passed.
Can you come up with test which fails without change to Segment.java ?
Thanks
> Miscount of memstore's heap/offheap size if same cell was put
> -------------------------------------------------------------
>
> Key: HBASE-21029
> URL: https://issues.apache.org/jira/browse/HBASE-21029
> Project: HBase
> Issue Type: Bug
> Affects Versions: 2.1.0, 2.0.1
> Reporter: Allan Yang
> Assignee: Allan Yang
> Priority: Major
> Attachments: HBASE-21029.branch-2.0.001.patch,
> HBASE-21029.branch-2.0.002.patch
>
>
> We are now using memstore.heapSize() + memstore.offheapSize() to decide
> whether a flush is needed. But, if a same cell was put again in memstore,
> only the memstore's dataSize will be increased, the heap/offheap size won't.
> We encountered the problem that a user was putting the same kv again and
> again, but the memstore won't flush since the heap size was not counted
> properly. The RS was killed by system since not enough memory in the end.
> Actually, if MSLAB is used, the heap/offheap will increase no matter the cell
> is added or not. IIRC, memstore's heap/offheap size should always bigger than
> data size. We introduced heap/offheap size besides data size in order to
> reflect memory footprint more precisely.
> {code}
> // If there's already a same cell in the CellSet and we are using MSLAB,
> we must count in the
> // MSLAB allocation size as well, or else there will be memory leak
> (occupied heap size larger
> // than the counted number)
> if (succ || mslabUsed) {
> cellSize = getCellLength(cellToAdd);
> }
> // heap/offheap size is changed only if the cell is truly added in the
> cellSet
> long heapSize = heapSizeChange(cellToAdd, succ);
> long offHeapSize = offHeapSizeChange(cellToAdd, succ);
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
