[jira] [Commented] (HBASE-27805) The chunk created by mslab may cause memory fragement and lead to fullgc
[
https://issues.apache.org/jira/browse/HBASE-27805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17748312#comment-17748312
]
Hudson commented on HBASE-27805:
Results for branch master
[build #877 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/877/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/877/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/877/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/877/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> The chunk created by mslab may cause memory fragement and lead to fullgc
>
>
> Key: HBASE-27805
> URL: https://issues.apache.org/jira/browse/HBASE-27805
> Project: HBase
> Issue Type: Improvement
> Components: documentation
>Reporter: Zheng Wang
>Assignee: Zheng Wang
>Priority: Major
> Fix For: 4.0.0-alpha-1
>
> Attachments: chunksize-2047k.png, chunksize-2048k-fullgc.png
>
>
> The default size of chunk is 2m, when we use G1, if heapRegionSize equals 4m,
> these chunks are allocated as humongous objects, exclusively allocating one
> region, then the remaining 2m become memory fragement.
> Lots of memory fragement may lead to fullgc even if the percent of used heap
> not high enough.
> I have tested to reduce the chunk size to 2047k(2m-1k, a bit lesser than half
> of heapRegionSize), there was no repeat of the above.
> BTW, in G1, humongous objects are objects larger or equal the size of half a
> region, and the heapRegionSize is automatically calculated based on the heap
> size parameter if not explicitly specified.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-27805) The chunk created by mslab may cause memory fragement and lead to fullgc
[ https://issues.apache.org/jira/browse/HBASE-27805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17747710#comment-17747710 ] Zheng Wang commented on HBASE-27805: In this issue, we just updated the documentation and provided a way to workaround. > The chunk created by mslab may cause memory fragement and lead to fullgc > > > Key: HBASE-27805 > URL: https://issues.apache.org/jira/browse/HBASE-27805 > Project: HBase > Issue Type: Improvement > Components: documentation >Reporter: Zheng Wang >Assignee: Zheng Wang >Priority: Major > Attachments: chunksize-2047k.png, chunksize-2048k-fullgc.png > > > The default size of chunk is 2m, when we use G1, if heapRegionSize equals 4m, > these chunks are allocated as humongous objects, exclusively allocating one > region, then the remaining 2m become memory fragement. > Lots of memory fragement may lead to fullgc even if the percent of used heap > not high enough. > I have tested to reduce the chunk size to 2047k(2m-1k, a bit lesser than half > of heapRegionSize), there was no repeat of the above. > BTW, in G1, humongous objects are objects larger or equal the size of half a > region, and the heapRegionSize is automatically calculated based on the heap > size parameter if not explicitly specified. > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HBASE-27805) The chunk created by mslab may cause memory fragement and lead to fullgc
[ https://issues.apache.org/jira/browse/HBASE-27805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17714630#comment-17714630 ] Zheng Wang commented on HBASE-27805: Ok. [~zhangduo] > The chunk created by mslab may cause memory fragement and lead to fullgc > > > Key: HBASE-27805 > URL: https://issues.apache.org/jira/browse/HBASE-27805 > Project: HBase > Issue Type: Improvement > Components: regionserver >Reporter: Zheng Wang >Assignee: Zheng Wang >Priority: Major > Attachments: chunksize-2047k.png, chunksize-2048k-fullgc.png > > > The default size of chunk is 2m, when we use G1, if heapRegionSize equals 4m, > these chunks are allocated as humongous objects, exclusively allocating one > region, then the remaining 2m become memory fragement. > Lots of memory fragement may lead to fullgc even if the percent of used heap > not high enough. > I have tested to reduce the chunk size to 2047k(2m-1k, a bit lesser than half > of heapRegionSize), there was no repeat of the above. > BTW, in G1, humongous objects are objects larger or equal the size of half a > region, and the heapRegionSize is automatically calculated based on the heap > size parameter if not explicitly specified. > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HBASE-27805) The chunk created by mslab may cause memory fragement and lead to fullgc
[ https://issues.apache.org/jira/browse/HBASE-27805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17714598#comment-17714598 ] Duo Zhang commented on HBASE-27805: --- Better start a discussion thread on dev list. > The chunk created by mslab may cause memory fragement and lead to fullgc > > > Key: HBASE-27805 > URL: https://issues.apache.org/jira/browse/HBASE-27805 > Project: HBase > Issue Type: Improvement > Components: regionserver >Reporter: Zheng Wang >Assignee: Zheng Wang >Priority: Major > Attachments: chunksize-2047k.png, chunksize-2048k-fullgc.png > > > The default size of chunk is 2m, when we use G1, if heapRegionSize equals 4m, > these chunks are allocated as humongous objects, exclusively allocating one > region, then the remaining 2m become memory fragement. > Lots of memory fragement may lead to fullgc even if the percent of used heap > not high enough. > I have tested to reduce the chunk size to 2047k(2m-1k, a bit lesser than half > of heapRegionSize), there was no repeat of the above. > BTW, in G1, humongous objects are objects larger or equal the size of half a > region, and the heapRegionSize is automatically calculated based on the heap > size parameter if not explicitly specified. > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HBASE-27805) The chunk created by mslab may cause memory fragement and lead to fullgc
[ https://issues.apache.org/jira/browse/HBASE-27805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17714381#comment-17714381 ] Zheng Wang commented on HBASE-27805: Not sure if we should change the default chunk size to 2047k, any suggestions are welcomed. > The chunk created by mslab may cause memory fragement and lead to fullgc > > > Key: HBASE-27805 > URL: https://issues.apache.org/jira/browse/HBASE-27805 > Project: HBase > Issue Type: Improvement > Components: regionserver >Reporter: Zheng Wang >Assignee: Zheng Wang >Priority: Major > Attachments: chunksize-2047k.png, chunksize-2048k-fullgc.png > > > The default size of chunk is 2m, when we use G1, if heapRegionSize equals 4m, > these chunks are allocated as humongous objects, exclusively allocating one > region, then the remaining 2m become memory fragement. > Lots of memory fragement may lead to fullgc even if the percent of used heap > not high enough. > I have tested to reduce the chunk size to 2047k(2m-1k, a bit lesser than half > of heapRegionSize), there was no repeat of the above. > BTW, in g1, humongous objects are objects larger or equal the size of half a > region. > -- This message was sent by Atlassian Jira (v8.20.10#820010)
