[
https://issues.apache.org/jira/browse/HBASE-14921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15208212#comment-15208212
]
Anastasia Braginsky commented on HBASE-14921:
---------------------------------------------
Hi [~stack],
Thanks for taking a look. It looks like the name “CellBlock" made a confusion.
Please pay attention that we distinguish between the “Cell Data” and the “Cell
Reference” as it was explained in the attached diagrams.
The “Cell Data” is key, value, tags, sequence ID, etc., it is de facto
information explaining the given Cell.
However, the “Cell Reference” is any ability to reach this “Cell Data”. The
“Cell Reference” holds no information explaining the given Cell, but a shortcut
to where this information can be found. For example, in current MemStore, the
skip-list holds “Cell References” and MSLAB holds “Cell Data”.
We didn’t mean to change the way the Cell Data is represented, it is done very
well via MSLAB. Alternatively, CellData can be allocated directly from Java
heap or using any Cell Codec/Encoding/Decoding etc.
As it was explained also in the attached documents, all what we meant to do was
to change the way the shortcut to the Cell in MemStore is working. If it is
applicable anywhere beside MemStore - great!
The suggested structure uses 4 times less memory than the previous
ConcurrentSkipListMap, provides no fragmentation and it is ready for
off-heaping.
Soon, I’ll upload the continuation of this patch, where you’ll see how this
flat structure is integrated in ImmutableSegment and works well with the
existing scanners.
Surely, we are also preparing the benchmark to measure the potential.
Thanks,
Anastasia
> Memory optimizations
> --------------------
>
> Key: HBASE-14921
> URL: https://issues.apache.org/jira/browse/HBASE-14921
> Project: HBase
> Issue Type: Sub-task
> Affects Versions: 2.0.0
> Reporter: Eshcar Hillel
> Assignee: Anastasia Braginsky
> Attachments: CellBlocksSegmentInMemStore.pdf,
> CellBlocksSegmentinthecontextofMemStore(1).pdf, HBASE-14921-V01.patch,
> HBASE-14921-V02.patch
>
>
> Memory optimizations including compressed format representation and offheap
> allocations
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
