[
https://issues.apache.org/jira/browse/HBASE-18010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16015614#comment-16015614
]
Anastasia Braginsky commented on HBASE-18010:
---------------------------------------------
While writing the integration I've encountered the following issue with the
sizes. Generally, the issue is that KeyValue.FIXED_OVERHEAD, meaning the size
of the Cell object is not counted as part of the any size. I am looking how
size of the cells in segment is calculated:
{code}
protected void updateMetaInfo(Cell cellToAdd, boolean succ, boolean mslabUsed,
MemstoreSize memstoreSize) {
long cellSize = 0;
// If there's already a same cell in the CellSet and we are using MSLAB, we
must count in the
// MSLAB allocation size as well, or else there will be memory leak
(occupied heap size larger
// than the counted number)
if (succ || mslabUsed) {
cellSize = getCellLength(cellToAdd);
}
long heapSize = heapSizeChange(cellToAdd, succ);
incSize(cellSize, heapSize);
if (memstoreSize != null) {
memstoreSize.incMemstoreSize(cellSize, heapSize);
}
getTimeRangeTracker().includeTimestamp(cellToAdd);
minSequenceId = Math.min(minSequenceId, cellToAdd.getSequenceId());
// In no tags case this NoTagsKeyValue.getTagsLength() is a cheap call.
// When we use ACL CP or Visibility CP which deals with Tags during
// mutation, the TagRewriteCell.getTagsLength() is a cheaper call. We do not
// parse the byte[] to identify the tags length.
if (cellToAdd.getTagsLength() > 0) {
tagsPresent = true;
}
}
{code}
However, getCellLength() is implemented just as KeyValueUtil.length(cell) which
doesn't include counting Cell object size. I remember I was told that
KeyValue.heapSize() is responsible for counting both Cell "true" length and
Cell object size. This is true, but KeyValue.heapSize() is not on the path of
updating the segment size. From here I see that the size of CellArrayMap
overhead is extremely small...
{code}
// REFERENCE in the CellArrayMap all the rest is counted in KeyValue.heapSize()
CELL_ARRAY_MAP_ENTRY = align(REFERENCE);
{code}
[~anoop.hbase], [~ramkrishna], what do you think about it? Am I missing
something?
> Connect CellChunkMap to be used for flattening in CompactingMemStore
> --------------------------------------------------------------------
>
> Key: HBASE-18010
> URL: https://issues.apache.org/jira/browse/HBASE-18010
> Project: HBase
> Issue Type: Sub-task
> Reporter: Anastasia Braginsky
>
> The CellChunkMap helps to create a new type of ImmutableSegment, where the
> index (CellSet's delegatee) is going to be CellChunkMap. No big cells or
> upserted cells are going to be supported here.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
