[
https://issues.apache.org/jira/browse/HBASE-14921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15247653#comment-15247653
]
Anastasia Braginsky commented on HBASE-14921:
---------------------------------------------
{quote}
It will be bad, if we have to continue with Cell ref in case of flat map (No
extra copy to new chunks as not much of the cells getting expired/removed).
Cells having ref to chunk data (byte[] now). Can we make the meta data here as
ref + offset ( 8 = 4 = 12 bytes per Cell)..Ya it is 4 bytes more but its ok and
better than 40 bytes per cell overhead. We need to mark the Cells created out
of copy to MSLAB in a special way so as to retrieve the byte[] ref.
{quote}
Agree with you that it is better to flatten into the CellChunkMap instead of
CellArrayMap. But as you have said, in order to do that “We need to mark the
Cells created out of copy to MSLAB in a special way so as to retrieve the
byte[] ref.”. This is planned to be done in the future :)
bq. in case of Cells in CSLM, the seqId is a state (long) on Cell object. That
will be an issue in above approach. So when the Cell is recreated after copying
to MSLAB (mayCloneWithAllocator), we can keep append the seqId 8 bytes after
Cell key, value, tags.
I am probably not familiar enough, so I don’t quite understand the problem. If
the Cell is copied to another Chunk in MSLAB, then everything is copied: Cell
key, value, tags, seqId, isn’t it? Or do we care to “waste” those 8 bytes of
seqId? Or would it give us wrong length? [~anoop.hbase], can you please try to
explain the problem once again? Thanking you in advance!
bq. Am I making the explanation here? Any help needed, I can do. :)
Thank you so much again for suggesting your help! If you have extra time, you
can deal with “We need to mark the Cells created out of copy to MSLAB in a
special way so as to retrieve the byte[] ref.” But probably the merging it all
together would be a nightmare… So at least it makes sense to wait to more
stable version of this version of the code… What do you think?
> Memory optimizations
> --------------------
>
> Key: HBASE-14921
> URL: https://issues.apache.org/jira/browse/HBASE-14921
> Project: HBase
> Issue Type: Sub-task
> Affects Versions: 2.0.0
> Reporter: Eshcar Hillel
> Assignee: Anastasia Braginsky
> Attachments: CellBlocksSegmentInMemStore.pdf,
> CellBlocksSegmentinthecontextofMemStore(1).pdf, HBASE-14921-V01.patch,
> HBASE-14921-V02.patch, HBASE-14921-V03.patch,
> IntroductiontoNewFlatandCompactMemStore.pdf
>
>
> Memory optimizations including compressed format representation and offheap
> allocations
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
