[
https://issues.apache.org/jira/browse/HBASE-3455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12986698#action_12986698
]
Todd Lipcon commented on HBASE-3455:
------------------------------------
There's one big flaw in this which I've now realized: upserts
The issue is that, even after we upsert a KV and drop its reference from the KV
set, we don't actually free any memory, since that KV's byte[] was just a
pointer to a larger chunk. So assumedly there is at least one other KV pointing
into that same chunk. Let's say a 2M chunk supports 20000 incrementable cells.
If we increment 19999 of them, we still have one KV pointing to it which uses
up 2MB of RAM.
Not sure exactly how to solve this... will sleep on it a few days...
One thought is that the Memstore can keep an IdentityHashMap<byte[],
AtomicInteger> chunk->referred_size map around. When we upsert something (and
hence remove its KV from kvset) we decrement the value in this map for the
chunk it referred to. If the amount of referred-to data in a chunk falls below
some threshold (say 50%) then we run some "compaction" code which relocates the
remaining values to a new contiguous chunk. Performance impact of this might be
too much, but maybe addressable with striping the counters or something.
> Heap fragmentation in region server
> -----------------------------------
>
> Key: HBASE-3455
> URL: https://issues.apache.org/jira/browse/HBASE-3455
> Project: HBase
> Issue Type: Brainstorming
> Components: performance, regionserver
> Reporter: Todd Lipcon
> Priority: Critical
> Attachments: collapse-arrays.patch, HBasefragmentation.pdf,
> parse-fls-statistics.py, with-kvallocs.png
>
>
> Stop-the-world GC pauses have long been a problem in HBase. "Concurrent mode
> failures" can usually be tuned around by setting the initiating occupancy
> fraction low, but eventually the heap becomes fragmented and a promotion
> failure occurs.
> This JIRA is to do research/experiments about the heap fragmentation issue
> and possible solutions.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
