[
https://issues.apache.org/jira/browse/HBASE-14921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15214437#comment-15214437
]
stack commented on HBASE-14921:
-------------------------------
bq. As you have mentioned, CellBlock is implementing ConcurrentNavigbleMap
because delegatee of the CellSet is of type ConcurrentNavigbleMap and we want
just to give CellSet another delegatee instead of ConcurrentSkipListMap.
Please say more on this. CellSet is a NavigableMap (not a
ConcurrentNavigableMap) so I'm missing where we need the 'Concurrent' (is it in
this patch?)
Your new names are better. I considered 'flat' Map but shied away given its
meaning over in spark/scala; I think it will be ok as long as you stick why its
a 'flat' map in the javadoc on CellFlatMap.
bq. ... it is Cells per chunk and it should be constant for all Chunks in a
CellBlockSerialized, because MSLAB always allocates Chunks with same size
How do you see this working? We do not control the size of inbound Cells. They
could have some regularity and they could also be erratic to the extreme (What
to do when a 1G cell arrives into a column family that up to this has been
taking on metrics?)
I still do not see how the 3 * int is BYTES_IN_CELL. Not important.
bq. But we are not sure what are the configurations where ChunkPool is off on
purpose and what we can do with such configurations. Do you know?
It was introduced and off by default as is usual when new features. But as also
happens this is our practice, the facility was 'forgotten'. It came up then
when our Lars noticed it and wanted to remove it since it was not being used.
It came up again recently in HBASE-15513
It would seem to make sense enabling it by default if we come up w/ a proper
sizing. Having it on seems to mess w/ G1GC too. Would need to figure that.
We need to do up a memory management doc. Between your work on Segments,
Segment pipelines, MSLAB chunks, chunk pools and bytebufferpools to host
requests read from sockets, bucket cache and reference counting bucketcache
bucket blocks at read time, it would be good if we had a map so we could trace
a Cell on its travels.
> Memory optimizations
> --------------------
>
> Key: HBASE-14921
> URL: https://issues.apache.org/jira/browse/HBASE-14921
> Project: HBase
> Issue Type: Sub-task
> Affects Versions: 2.0.0
> Reporter: Eshcar Hillel
> Assignee: Anastasia Braginsky
> Attachments: CellBlocksSegmentInMemStore.pdf,
> CellBlocksSegmentinthecontextofMemStore(1).pdf, HBASE-14921-V01.patch,
> HBASE-14921-V02.patch
>
>
> Memory optimizations including compressed format representation and offheap
> allocations
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
