[
https://issues.apache.org/jira/browse/HBASE-16229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15418395#comment-15418395
]
Anoop Sam John commented on HBASE-16229:
----------------------------------------
Gone through it.. That is what exactly this is trying to do. Each class is
responsible for its own size. Also added a keySize() and size/heapSize. In
segment also we have this 2 variants. Former just tracks the size of the
cells. heapSize will be like keySize() + heap overhead.
The keySize includes cells heapsize. - We plan to split this also.. (I
guess you are suggesting that too)- This will be one part will have cell data
size (key size + val size + tags size) and other will be heap overhead. Even
at global level also we will have this split. This is required when we have off
heap memstore (The MSLAB will be off heap based). So there we will need clear
separate tracking for what is memstore size in off heap area and what is heap
overhead. The flush decisions to consider both. ( Pls see
https://docs.google.com/document/d/1fj5P8JeutQ-Uadb29ChDscMuMaJqaMNRI86C4k5S1rQ/edit
for discussion abt this. See the section "Memstore sizing" )
Once 14921 is in, we will go ahead with this jira and later with the split of
cell data size and heap overhead split.
> Cleaning up size and heapSize calculation
> -----------------------------------------
>
> Key: HBASE-16229
> URL: https://issues.apache.org/jira/browse/HBASE-16229
> Project: HBase
> Issue Type: Sub-task
> Affects Versions: 2.0.0
> Reporter: Anoop Sam John
> Assignee: Anoop Sam John
> Fix For: 2.0.0
>
> Attachments: HBASE-16229.patch, HBASE-16229_V2.patch,
> HBASE-16229_V3.patch
>
>
> It is bit ugly now. For eg:
> AbstractMemStore
> {code}
> public final static long FIXED_OVERHEAD = ClassSize.align(
> ClassSize.OBJECT +
> (4 * ClassSize.REFERENCE) +
> (2 * Bytes.SIZEOF_LONG));
> public final static long DEEP_OVERHEAD = ClassSize.align(FIXED_OVERHEAD +
> (ClassSize.ATOMIC_LONG + ClassSize.TIMERANGE_TRACKER +
> ClassSize.CELL_SKIPLIST_SET + ClassSize.CONCURRENT_SKIPLISTMAP));
> {code}
> We include the heap overhead of Segment also here. It will be better the
> Segment contains its overhead part and the Memstore impl uses the heap size
> of all of its segments to calculate its size.
> Also this
> {code}
> public long heapSize() {
> return getActive().getSize();
> }
> {code}
> HeapSize to consider all segment's size not just active's. I am not able to
> see an override method in CompactingMemstore.
> This jira tries to solve some of these.
> When we create a Segment, we seems pass some initial heap size value to it.
> Why? The segment object internally has to know what is its heap size not
> like some one else dictate it.
> More to add when doing this cleanup
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
