[
https://issues.apache.org/jira/browse/HBASE-14921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15214273#comment-15214273
]
Anastasia Braginsky commented on HBASE-14921:
---------------------------------------------
Thanks [~ram_krish], you raise the really good questions about ChunkPool!
bq. So the CellBlockSerialized (I think we can have a better name) - will work
only if the MSLAB Chunk pool is ON right? If there is no MSLAB chunk pool still
it can work? I may be missing something.
Indeed this requires ChunkPool to be ON. Please pay attention that we can
design ChunkPool not to pre-allocate Chunks if this is what configuration
wants. It can be just a global bookkeeper for all allocated Chunks. We just
require ChunkPool not to be NULL.
bq. Now in your shallowCellsToBuffer even the shallowBuffer is created with the
same 'chunkSize' as we create the deepbuffer also? Is that really needed?
The “chunkSize” used in shallowCellsToBuffer holds the size of any Chunk
created by ChunkPool. I assume (as it is now in MemStoreChunkPool) that all
Chunks created from the same instance of Pool are of same size. This is why one
can reuse the Chunks. If Chunks are "tailor-made" with different sizes, they
almost cannot be reused.
bq. How does things work when the chunkpool gets reset and initialized?
This would be a problem. What is the case when ChunkPool need to be reset and
initialized?
And BTW regarding the names, do you like the names that I have suggested above?
{quote}
Regarding the names, as “CellBlock” is already in some other use, we suggest
the following names (almost as you suggested).
CellBlock -> CellFlatMap
CellBlockObjectArray -> CellArrayMap
CellBlockSerialized -> CellChunkMap
Why “flat”? Because this is how we call the process of changing the
ConcurrentSkipListMap to CellMap. When we have in-memory-flash and there is no
need to compact (unnecessary copies) we flatten the segment. Meaning we create
ImmutableSegment with the same MSLAB, but with different Map, replacing
ConcurrentSkipListMap to CellMap. What do you think?
{quote}
> Memory optimizations
> --------------------
>
> Key: HBASE-14921
> URL: https://issues.apache.org/jira/browse/HBASE-14921
> Project: HBase
> Issue Type: Sub-task
> Affects Versions: 2.0.0
> Reporter: Eshcar Hillel
> Assignee: Anastasia Braginsky
> Attachments: CellBlocksSegmentInMemStore.pdf,
> CellBlocksSegmentinthecontextofMemStore(1).pdf, HBASE-14921-V01.patch,
> HBASE-14921-V02.patch
>
>
> Memory optimizations including compressed format representation and offheap
> allocations
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
