[
https://issues.apache.org/jira/browse/HBASE-17343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15984480#comment-15984480
]
Anoop Sam John commented on HBASE-17343:
----------------------------------------
On the MSLAB, let us do some more tests.. Will discuss that in another issue.
I believe ur tests were with MSLAB on but no pool. With pool things are not
that bad. Some cases it is bit better than no pool only.. But with G1,things
do not show very useful (like CMS)
On the long GC pause issue which I saw with compaction.
This is coming in a test case where I have 42 GB of xmx and 60% as Global
memstore size. I have MSLAB and pool ON. The Initial Heap Occupancy
Percentage is 65%. The cell value size is 100 bytes with each row having 10
cols. In case of compacting memstore, I can see long GC pause and it even
exceeds 30 secs so that HM thinks RS is down and will get YouAreDeadException
and RS gets killed! MSLAB / pool is good or not is another topic. But IMO in
any case, we should not land in such issues.. So this is some thing we should
investigate.
{code}
public boolean flatten(MemstoreSize memstoreSize) {
if (isFlat()) return false;
CellSet oldCellSet = getCellSet();
int numOfCells = getCellsCount();
// build the new (CellSet CellArrayMap based)
CellSet newCellSet = recreateCellArrayMapSet(numOfCells);
type = Type.ARRAY_MAP_BASED;
setCellSet(oldCellSet,newCellSet);
// arrange the meta-data size, decrease all meta-data sizes related to
SkipList
// (recreateCellArrayMapSet doesn't take the care for the sizes)
long newSegmentSizeDelta = -(numOfCells *
ClassSize.CONCURRENT_SKIPLISTMAP_ENTRY);
// add size of CellArrayMap and meta-data overhead per Cell
newSegmentSizeDelta = newSegmentSizeDelta + numOfCells *
ClassSize.CELL_ARRAY_MAP_ENTRY;
incSize(0, newSegmentSizeDelta);
if (memstoreSize != null) {
memstoreSize.incMemstoreSize(0, newSegmentSizeDelta);
}
return true;
}
{code}
we have the CSLM and we create the CellArray also and then finally change the
size (This changes the global RS level size also). We reduce the CSLM entry
sizes and increase CellArray entry sizes. Finally this will come as reduction
in size only.. But temp the size is going up. I just did a change here like
1st I change the size with increasing (numOfCells *
ClassSize.CELL_ARRAY_MAP_ENTRY) and then only create CellArrayMap and then
reduce as in this position. Testing with that patch on the same settings, I
did never get this YouAreDeadException and RS killed.
Also when the tests are with 42% as global MS size, and IHOP as 50%, I did not
see RS killed issue any time.. But the other case, it is very frequent (once
in 3 tries)
> Make Compacting Memstore default in 2.0 with BASIC as the default type
> ----------------------------------------------------------------------
>
> Key: HBASE-17343
> URL: https://issues.apache.org/jira/browse/HBASE-17343
> Project: HBase
> Issue Type: Bug
> Affects Versions: 2.0.0
> Reporter: ramkrishna.s.vasudevan
> Priority: Blocker
> Fix For: 2.0.0
>
> Attachments: HBASE-17343-V01.patch, HBASE-17343-V02.patch,
> HBASE-17343-V04.patch, HBASE-17343-V05.patch
>
>
> FYI [~anastas], [~eshcar] and [~ebortnik].
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
