Yu Li commented on HBASE-21029:

Patch v2 LGTM, +1

> Miscount of memstore's heap/offheap size if same cell was put
> -------------------------------------------------------------
>                 Key: HBASE-21029
>                 URL: https://issues.apache.org/jira/browse/HBASE-21029
>             Project: HBase
>          Issue Type: Bug
>    Affects Versions: 2.1.0, 2.0.1
>            Reporter: Allan Yang
>            Assignee: Allan Yang
>            Priority: Major
>         Attachments: HBASE-21029.branch-2.0.001.patch, 
> HBASE-21029.branch-2.0.002.patch
> We are now using memstore.heapSize() + memstore.offheapSize() to decide 
> whether a flush is needed. But, if a same cell was put again in memstore, 
> only the memstore's dataSize will be increased, the heap/offheap size won't. 
> We encountered the problem that a user was putting the same kv again and 
> again, but the memstore won't flush since the heap size was not counted 
> properly. The RS was killed by system since not enough memory in the end.
> Actually, if MSLAB is used, the heap/offheap will increase no matter the cell 
> is added or not. IIRC, memstore's heap/offheap size should always bigger than 
> data size. We introduced heap/offheap size besides data size in order to 
> reflect memory footprint more precisely. 
> {code}
>     // If there's already a same cell in the CellSet and we are using MSLAB, 
> we must count in the
>     // MSLAB allocation size as well, or else there will be memory leak 
> (occupied heap size larger
>     // than the counted number)
>     if (succ || mslabUsed) {
>       cellSize = getCellLength(cellToAdd);
>     }
>     // heap/offheap size is changed only if the cell is truly added in the 
> cellSet
>     long heapSize = heapSizeChange(cellToAdd, succ);
>     long offHeapSize = offHeapSizeChange(cellToAdd, succ);
> {code}

This message was sent by Atlassian JIRA

Reply via email to