[
https://issues.apache.org/jira/browse/HBASE-14613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15212814#comment-15212814
]
Yanping Wang commented on HBASE-14613:
--------------------------------------
Hi, Elliott
>>-XX:InitiatingHeapOccupancyPercent
>>Setting it lower doesn't help since this is all about the promotion from
>>young to old an g1 preferring not to do mixed. But we tried it.
No, no, not set to lower, instead, calculate the size of pinned in Old and set
this number higher.
for example, if you have 100GB heap, you calculate those "pinned old forever"
are about 30GB, then most likely the default 45% (45GB) will cause trouble for
you. if after Mixed GC completes, the heap Occupancy is above 45GB, concurrent
marking cycle will start immediately to prepare next series of Mixed GC, which
gives you the symptom of "get pauses even though there is a lot of memory
left."
so if for 100GB heap and 30GB pinned Old, you might have to increase
-XX:InitiatingHeapOccupancyPercent to 60 or 65. so you are able to use more
heap memory. you can find those info by using -XX:+PrintAdaptiveSizePolicy
flag.
>>-XX:ConcGCThreads
>>Setting this higher helped a lot but we have enough happening on these boxes
>>that it started to hurt performance.
after you let more heap to be used, the total number of Mixed GC is likely will
be reduced, which could cause full GC if we are not careful. so increase
ConcGCThread helps concurrent marking completes faster. just consider which one
will hurt performance more? (1) take risk of Full GC (2) 80 Mixed GCs using 4
concurrent threads to do 10 concurrent marking each last 12 seconds (3) 40
Mixed GCs with 12 concurrent threads to do 5 concurrent marking each last 4
seconds.
>>-XX:G1MixedGCCountTarget
>> Setting this lower just moved what caused the long gc's
The most expensive steps in GC are scrub the remember set and COPY live objects
from one place to another. after each concurrent marking cycle completes, by
default G1 will try to do 8 mixed GC, first 1-5 might be easy for G1 so takes
less time to complete, but the deeper it goes, the harder it gets, the longer
it takes. so I'd recommend HBase don't need to go that deep. The reason is
other than pinned forever objects, most HBase objects are short lived. if this
group of mixed GC find too hard to clean and copy some regions, maybe wait a
while, in next cycle of mixed GC, some objects will die and makes their regions
easy to clean up (no need copy any more).
> Remove MemStoreChunkPool?
> -------------------------
>
> Key: HBASE-14613
> URL: https://issues.apache.org/jira/browse/HBASE-14613
> Project: HBase
> Issue Type: Brainstorming
> Reporter: Lars Hofhansl
> Priority: Minor
> Attachments: 14613-0.98.txt, gc.png, writes.png
>
>
> I just stumbled across MemStoreChunkPool. The idea behind is to reuse chunks
> of allocations rather than letting the GC handle this.
> Now, it's off by default, and it seems to me to be of dubious value. I'd
> recommend just removing it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
