[jira] [Commented] (HBASE-8163) MemStoreChunkPool: An improvement for JAVA GC when using MSLAB
[ https://issues.apache.org/jira/browse/HBASE-8163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13825601#comment-13825601 ] Hudson commented on HBASE-8163: --- SUCCESS: Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #842 (See [https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/842/]) Add note on https://issues.apache.org/jira/browse/HBASE-8163 -- config for write-heavy situation (stack: rev 1542903) * /hbase/trunk/src/main/docbkx/performance.xml > MemStoreChunkPool: An improvement for JAVA GC when using MSLAB > -- > > Key: HBASE-8163 > URL: https://issues.apache.org/jira/browse/HBASE-8163 > Project: HBase > Issue Type: New Feature > Components: Performance, regionserver >Reporter: chunhui shen >Assignee: chunhui shen > Fix For: 0.98.0, 0.95.0 > > Attachments: hbase-0.95-8163v6.patch, hbase-8163v1.patch, > hbase-8163v2.patch, hbase-8163v3.patch, hbase-8163v4.patch, > hbase-8163v5.patch, hbase-8163v6.patch > > > *Usage:* > Disable pool(default): configure "hbase.hregion.memstore.chunkpool.maxsize" > as 0 > Enable pool: configure "hbase.hregion.memstore.chunkpool.maxsize" as a > percentage of global memstore size (between 0.0 and 1.0, recommend to set it > as the gap between min and max sizes of memstore, e.g. 0.5) > *Background*： > When we use mslab,we will copy the keyvalue together in a structure called > *MemStoreLAB$Chunk*, therefore we could decrease the heap fragment. > *Problem*： > When one chunk is full, we will create a new chunk, and then the old chunk > will be reclaimed by JVM if no reference to it. > Mostly the chunk object would be promoted when doing Young GC, cause > increasing the cost of YGC > When a Chunk object has no reference? It should meet two following condition: > 1.Memstore which this chunk belongs to is flushed > 2.No scanner is opening on the memstore which this chunk belongs to > *Solution:* > 1.Create a chunk pool to manage the no-reference chunks, instead of being > reclaimed by JVM > 2.When a Chunk has no reference, put it back to the pool > 3.The pool has a max capacity, it will skip the chunks when achieve the max > size > 4.When we need new Chunk to store KeyValue, get it from the pool if exists, > else create new one by pool, so we could be able to reuse the old chunks > *Test results:* > Environment: > hbase-version:0.94 > -Xms4G -Xmx4G -Xmn2G > Row size=50 bytes, Value size=1024 bytes > 50 concurrent theads per client, insert 10,000,000 rows > Before: > Avg write request per second:12953 > After testing, final result of jstat -gcutil : > YGC YGCT FGC FGCT GCT > 747 36.503 48 2.492 38.995 > After: > Avg write request per second:14025 > After testing, final result of jstat -gcutil : > YGC YGCT FGC FGCT GCT > 711 20.344 4 0.284 20.628 > *Improvement: YGC 40+%; WPS 5%+* > review board : > https://reviews.apache.org/r/10056/ -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HBASE-8163) MemStoreChunkPool: An improvement for JAVA GC when using MSLAB
[ https://issues.apache.org/jira/browse/HBASE-8163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13825325#comment-13825325 ] Hudson commented on HBASE-8163: --- SUCCESS: Integrated in HBase-TRUNK #4685 (See [https://builds.apache.org/job/HBase-TRUNK/4685/]) Add note on https://issues.apache.org/jira/browse/HBASE-8163 -- config for write-heavy situation (stack: rev 1542903) * /hbase/trunk/src/main/docbkx/performance.xml > MemStoreChunkPool: An improvement for JAVA GC when using MSLAB > -- > > Key: HBASE-8163 > URL: https://issues.apache.org/jira/browse/HBASE-8163 > Project: HBase > Issue Type: New Feature > Components: Performance, regionserver >Reporter: chunhui shen >Assignee: chunhui shen > Fix For: 0.98.0, 0.95.0 > > Attachments: hbase-0.95-8163v6.patch, hbase-8163v1.patch, > hbase-8163v2.patch, hbase-8163v3.patch, hbase-8163v4.patch, > hbase-8163v5.patch, hbase-8163v6.patch > > > *Usage:* > Disable pool(default): configure "hbase.hregion.memstore.chunkpool.maxsize" > as 0 > Enable pool: configure "hbase.hregion.memstore.chunkpool.maxsize" as a > percentage of global memstore size (between 0.0 and 1.0, recommend to set it > as the gap between min and max sizes of memstore, e.g. 0.5) > *Background*： > When we use mslab,we will copy the keyvalue together in a structure called > *MemStoreLAB$Chunk*, therefore we could decrease the heap fragment. > *Problem*： > When one chunk is full, we will create a new chunk, and then the old chunk > will be reclaimed by JVM if no reference to it. > Mostly the chunk object would be promoted when doing Young GC, cause > increasing the cost of YGC > When a Chunk object has no reference? It should meet two following condition: > 1.Memstore which this chunk belongs to is flushed > 2.No scanner is opening on the memstore which this chunk belongs to > *Solution:* > 1.Create a chunk pool to manage the no-reference chunks, instead of being > reclaimed by JVM > 2.When a Chunk has no reference, put it back to the pool > 3.The pool has a max capacity, it will skip the chunks when achieve the max > size > 4.When we need new Chunk to store KeyValue, get it from the pool if exists, > else create new one by pool, so we could be able to reuse the old chunks > *Test results:* > Environment: > hbase-version:0.94 > -Xms4G -Xmx4G -Xmn2G > Row size=50 bytes, Value size=1024 bytes > 50 concurrent theads per client, insert 10,000,000 rows > Before: > Avg write request per second:12953 > After testing, final result of jstat -gcutil : > YGC YGCT FGC FGCT GCT > 747 36.503 48 2.492 38.995 > After: > Avg write request per second:14025 > After testing, final result of jstat -gcutil : > YGC YGCT FGC FGCT GCT > 711 20.344 4 0.284 20.628 > *Improvement: YGC 40+%; WPS 5%+* > review board : > https://reviews.apache.org/r/10056/ -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HBASE-8163) MemStoreChunkPool: An improvement for JAVA GC when using MSLAB
[ https://issues.apache.org/jira/browse/HBASE-8163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13825125#comment-13825125 ] stack commented on HBASE-8163: -- Are there values we could use that would make it so we could just turn this on? Or, if we are letting go too many Chunks, could we start up a pool if only for a short while to get over a write hump? Then turn it off again after? > MemStoreChunkPool: An improvement for JAVA GC when using MSLAB > -- > > Key: HBASE-8163 > URL: https://issues.apache.org/jira/browse/HBASE-8163 > Project: HBase > Issue Type: New Feature > Components: Performance, regionserver >Reporter: chunhui shen >Assignee: chunhui shen > Fix For: 0.98.0, 0.95.0 > > Attachments: hbase-0.95-8163v6.patch, hbase-8163v1.patch, > hbase-8163v2.patch, hbase-8163v3.patch, hbase-8163v4.patch, > hbase-8163v5.patch, hbase-8163v6.patch > > > *Usage:* > Disable pool(default): configure "hbase.hregion.memstore.chunkpool.maxsize" > as 0 > Enable pool: configure "hbase.hregion.memstore.chunkpool.maxsize" as a > percentage of global memstore size (between 0.0 and 1.0, recommend to set it > as the gap between min and max sizes of memstore, e.g. 0.5) > *Background*： > When we use mslab,we will copy the keyvalue together in a structure called > *MemStoreLAB$Chunk*, therefore we could decrease the heap fragment. > *Problem*： > When one chunk is full, we will create a new chunk, and then the old chunk > will be reclaimed by JVM if no reference to it. > Mostly the chunk object would be promoted when doing Young GC, cause > increasing the cost of YGC > When a Chunk object has no reference? It should meet two following condition: > 1.Memstore which this chunk belongs to is flushed > 2.No scanner is opening on the memstore which this chunk belongs to > *Solution:* > 1.Create a chunk pool to manage the no-reference chunks, instead of being > reclaimed by JVM > 2.When a Chunk has no reference, put it back to the pool > 3.The pool has a max capacity, it will skip the chunks when achieve the max > size > 4.When we need new Chunk to store KeyValue, get it from the pool if exists, > else create new one by pool, so we could be able to reuse the old chunks > *Test results:* > Environment: > hbase-version:0.94 > -Xms4G -Xmx4G -Xmn2G > Row size=50 bytes, Value size=1024 bytes > 50 concurrent theads per client, insert 10,000,000 rows > Before: > Avg write request per second:12953 > After testing, final result of jstat -gcutil : > YGC YGCT FGC FGCT GCT > 747 36.503 48 2.492 38.995 > After: > Avg write request per second:14025 > After testing, final result of jstat -gcutil : > YGC YGCT FGC FGCT GCT > 711 20.344 4 0.284 20.628 > *Improvement: YGC 40+%; WPS 5%+* > review board : > https://reviews.apache.org/r/10056/ -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HBASE-8163) MemStoreChunkPool: An improvement for JAVA GC when using MSLAB
[ https://issues.apache.org/jira/browse/HBASE-8163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13823200#comment-13823200 ] chunhui shen commented on HBASE-8163: - Liang, thanks for the tests. The new recommend config seems be another tunning method when we face the GC. Nice:) > MemStoreChunkPool: An improvement for JAVA GC when using MSLAB > -- > > Key: HBASE-8163 > URL: https://issues.apache.org/jira/browse/HBASE-8163 > Project: HBase > Issue Type: New Feature > Components: Performance, regionserver >Reporter: chunhui shen >Assignee: chunhui shen > Fix For: 0.98.0, 0.95.0 > > Attachments: hbase-0.95-8163v6.patch, hbase-8163v1.patch, > hbase-8163v2.patch, hbase-8163v3.patch, hbase-8163v4.patch, > hbase-8163v5.patch, hbase-8163v6.patch > > > *Usage:* > Disable pool(default): configure "hbase.hregion.memstore.chunkpool.maxsize" > as 0 > Enable pool: configure "hbase.hregion.memstore.chunkpool.maxsize" as a > percentage of global memstore size (between 0.0 and 1.0, recommend to set it > as the gap between min and max sizes of memstore, e.g. 0.5) > *Background*： > When we use mslab,we will copy the keyvalue together in a structure called > *MemStoreLAB$Chunk*, therefore we could decrease the heap fragment. > *Problem*： > When one chunk is full, we will create a new chunk, and then the old chunk > will be reclaimed by JVM if no reference to it. > Mostly the chunk object would be promoted when doing Young GC, cause > increasing the cost of YGC > When a Chunk object has no reference? It should meet two following condition: > 1.Memstore which this chunk belongs to is flushed > 2.No scanner is opening on the memstore which this chunk belongs to > *Solution:* > 1.Create a chunk pool to manage the no-reference chunks, instead of being > reclaimed by JVM > 2.When a Chunk has no reference, put it back to the pool > 3.The pool has a max capacity, it will skip the chunks when achieve the max > size > 4.When we need new Chunk to store KeyValue, get it from the pool if exists, > else create new one by pool, so we could be able to reuse the old chunks > *Test results:* > Environment: > hbase-version:0.94 > -Xms4G -Xmx4G -Xmn2G > Row size=50 bytes, Value size=1024 bytes > 50 concurrent theads per client, insert 10,000,000 rows > Before: > Avg write request per second:12953 > After testing, final result of jstat -gcutil : > YGC YGCT FGC FGCT GCT > 747 36.503 48 2.492 38.995 > After: > Avg write request per second:14025 > After testing, final result of jstat -gcutil : > YGC YGCT FGC FGCT GCT > 711 20.344 4 0.284 20.628 > *Improvement: YGC 40+%; WPS 5%+* > review board : > https://reviews.apache.org/r/10056/ -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HBASE-8163) MemStoreChunkPool: An improvement for JAVA GC when using MSLAB
[ https://issues.apache.org/jira/browse/HBASE-8163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13822310#comment-13822310 ] Liang Xie commented on HBASE-8163: -- Of cause, per theory, the chunkpool solution should be a little better than my recommend config solution, since it'll reuse the chunks to avoid more "new byte[]" invoke, more memset clear operation from hotspot. > MemStoreChunkPool: An improvement for JAVA GC when using MSLAB > -- > > Key: HBASE-8163 > URL: https://issues.apache.org/jira/browse/HBASE-8163 > Project: HBase > Issue Type: New Feature > Components: Performance, regionserver >Reporter: chunhui shen >Assignee: chunhui shen > Fix For: 0.98.0, 0.95.0 > > Attachments: hbase-0.95-8163v6.patch, hbase-8163v1.patch, > hbase-8163v2.patch, hbase-8163v3.patch, hbase-8163v4.patch, > hbase-8163v5.patch, hbase-8163v6.patch > > > *Usage:* > Disable pool(default): configure "hbase.hregion.memstore.chunkpool.maxsize" > as 0 > Enable pool: configure "hbase.hregion.memstore.chunkpool.maxsize" as a > percentage of global memstore size (between 0.0 and 1.0, recommend to set it > as the gap between min and max sizes of memstore, e.g. 0.5) > *Background*： > When we use mslab,we will copy the keyvalue together in a structure called > *MemStoreLAB$Chunk*, therefore we could decrease the heap fragment. > *Problem*： > When one chunk is full, we will create a new chunk, and then the old chunk > will be reclaimed by JVM if no reference to it. > Mostly the chunk object would be promoted when doing Young GC, cause > increasing the cost of YGC > When a Chunk object has no reference? It should meet two following condition: > 1.Memstore which this chunk belongs to is flushed > 2.No scanner is opening on the memstore which this chunk belongs to > *Solution:* > 1.Create a chunk pool to manage the no-reference chunks, instead of being > reclaimed by JVM > 2.When a Chunk has no reference, put it back to the pool > 3.The pool has a max capacity, it will skip the chunks when achieve the max > size > 4.When we need new Chunk to store KeyValue, get it from the pool if exists, > else create new one by pool, so we could be able to reuse the old chunks > *Test results:* > Environment: > hbase-version:0.94 > -Xms4G -Xmx4G -Xmn2G > Row size=50 bytes, Value size=1024 bytes > 50 concurrent theads per client, insert 10,000,000 rows > Before: > Avg write request per second:12953 > After testing, final result of jstat -gcutil : > YGC YGCT FGC FGCT GCT > 747 36.503 48 2.492 38.995 > After: > Avg write request per second:14025 > After testing, final result of jstat -gcutil : > YGC YGCT FGC FGCT GCT > 711 20.344 4 0.284 20.628 > *Improvement: YGC 40+%; WPS 5%+* > review board : > https://reviews.apache.org/r/10056/ -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HBASE-8163) MemStoreChunkPool: An improvement for JAVA GC when using MSLAB
[ https://issues.apache.org/jira/browse/HBASE-8163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13822308#comment-13822308 ] Liang Xie commented on HBASE-8163: -- I did a apple-to-apple test with this patch. Here is the test result just for refer: Env: Xmx=Xms=8G,Xmn1G, memestore lower/upper limit = 0.2/0.3 each test began with empty table, then write 20million records each with three fields, and each field has 200 bytes: 1)original config(-XX:PretenureSizeThreshold=4m) YGC YGCTFGCFGCT GCT 6970 318.592 80.884 319.476 2)set hbase.hregion.memstore.mslab.chunksize to 4194320(-XX:PretenureSizeThreshold=4m) YGC YGCTFGCFGCT GCT 6973 253.891 80.522 254.413 3)set -XX:PretenureSizeThreshold=2097088(hbase.hregion.memstore.mslab.chunksize is by default 2M) 6960 260.642 81.427 262.069 4)set hbase.hregion.memstore.chunkpool.maxsize=0.6, means enable MemStoreChunkPool feature(log said maxCount=706), -XX:PretenureSizeThreshold=2097088(hbase.hregion.memstore.mslab.chunksize is by default 2M) 7028 258.598 20.401 258.999 To me, this MemStorechunkPool feature is useful for heavy FGC scenarios caused by write request. If the YGC is the big hurt by write request, personally i recommend to config "hbase.hregion.memstore.mslab.chunksize" or "-XX:PretenureSizeThreshold", considering risk:) Hope this test result will be helpful for some guys > MemStoreChunkPool: An improvement for JAVA GC when using MSLAB > -- > > Key: HBASE-8163 > URL: https://issues.apache.org/jira/browse/HBASE-8163 > Project: HBase > Issue Type: New Feature > Components: Performance, regionserver >Reporter: chunhui shen >Assignee: chunhui shen > Fix For: 0.98.0, 0.95.0 > > Attachments: hbase-0.95-8163v6.patch, hbase-8163v1.patch, > hbase-8163v2.patch, hbase-8163v3.patch, hbase-8163v4.patch, > hbase-8163v5.patch, hbase-8163v6.patch > > > *Usage:* > Disable pool(default): configure "hbase.hregion.memstore.chunkpool.maxsize" > as 0 > Enable pool: configure "hbase.hregion.memstore.chunkpool.maxsize" as a > percentage of global memstore size (between 0.0 and 1.0, recommend to set it > as the gap between min and max sizes of memstore, e.g. 0.5) > *Background*： > When we use mslab,we will copy the keyvalue together in a structure called > *MemStoreLAB$Chunk*, therefore we could decrease the heap fragment. > *Problem*： > When one chunk is full, we will create a new chunk, and then the old chunk > will be reclaimed by JVM if no reference to it. > Mostly the chunk object would be promoted when doing Young GC, cause > increasing the cost of YGC > When a Chunk object has no reference? It should meet two following condition: > 1.Memstore which this chunk belongs to is flushed > 2.No scanner is opening on the memstore which this chunk belongs to > *Solution:* > 1.Create a chunk pool to manage the no-reference chunks, instead of being > reclaimed by JVM > 2.When a Chunk has no reference, put it back to the pool > 3.The pool has a max capacity, it will skip the chunks when achieve the max > size > 4.When we need new Chunk to store KeyValue, get it from the pool if exists, > else create new one by pool, so we could be able to reuse the old chunks > *Test results:* > Environment: > hbase-version:0.94 > -Xms4G -Xmx4G -Xmn2G > Row size=50 bytes, Value size=1024 bytes > 50 concurrent theads per client, insert 10,000,000 rows > Before: > Avg write request per second:12953 > After testing, final result of jstat -gcutil : > YGC YGCT FGC FGCT GCT > 747 36.503 48 2.492 38.995 > After: > Avg write request per second:14025 > After testing, final result of jstat -gcutil : > YGC YGCT FGC FGCT GCT > 711 20.344 4 0.284 20.628 > *Improvement: YGC 40+%; WPS 5%+* > review board : > https://reviews.apache.org/r/10056/ -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HBASE-8163) MemStoreChunkPool: An improvement for JAVA GC when using MSLAB
[ https://issues.apache.org/jira/browse/HBASE-8163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13615225#comment-13615225 ] Hudson commented on HBASE-8163: --- Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #466 (See [https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/466/]) HBASE-8163 MemStoreChunkPool: An improvement for JAVA GC when using MSLAB (Revision 1461398) Result = FAILURE zjushch : Files : * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MemStore.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MemStoreChunkPool.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MemStoreFlusher.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MemStoreLAB.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestMemStoreChunkPool.java > MemStoreChunkPool: An improvement for JAVA GC when using MSLAB > -- > > Key: HBASE-8163 > URL: https://issues.apache.org/jira/browse/HBASE-8163 > Project: HBase > Issue Type: New Feature > Components: regionserver >Reporter: chunhui shen >Assignee: chunhui shen > Fix For: 0.95.0, 0.98.0 > > Attachments: hbase-0.95-8163v6.patch, hbase-8163v1.patch, > hbase-8163v2.patch, hbase-8163v3.patch, hbase-8163v4.patch, > hbase-8163v5.patch, hbase-8163v6.patch > > > *Usage:* > Disable pool(default): configure "hbase.hregion.memstore.chunkpool.maxsize" > as 0 > Enable pool: configure "hbase.hregion.memstore.chunkpool.maxsize" as a > percentage of global memstore size (between 0.0 and 1.0, recommend to set it > as the gap between min and max sizes of memstore, e.g. 0.5) > *Background*： > When we use mslab,we will copy the keyvalue together in a structure called > *MemStoreLAB$Chunk*, therefore we could decrease the heap fragment. > *Problem*： > When one chunk is full, we will create a new chunk, and then the old chunk > will be reclaimed by JVM if no reference to it. > Mostly the chunk object would be promoted when doing Young GC, cause > increasing the cost of YGC > When a Chunk object has no reference? It should meet two following condition: > 1.Memstore which this chunk belongs to is flushed > 2.No scanner is opening on the memstore which this chunk belongs to > *Solution:* > 1.Create a chunk pool to manage the no-reference chunks, instead of being > reclaimed by JVM > 2.When a Chunk has no reference, put it back to the pool > 3.The pool has a max capacity, it will skip the chunks when achieve the max > size > 4.When we need new Chunk to store KeyValue, get it from the pool if exists, > else create new one by pool, so we could be able to reuse the old chunks > *Test results:* > Environment: > hbase-version:0.94 > -Xms4G -Xmx4G -Xmn2G > Row size=50 bytes, Value size=1024 bytes > 50 concurrent theads per client, insert 10,000,000 rows > Before: > Avg write request per second:12953 > After testing, final result of jstat -gcutil : > YGC YGCT FGC FGCT GCT > 747 36.503 48 2.492 38.995 > After: > Avg write request per second:14025 > After testing, final result of jstat -gcutil : > YGC YGCT FGC FGCT GCT > 711 20.344 4 0.284 20.628 > *Improvement: YGC 40+%; WPS 5%+* > review board : > https://reviews.apache.org/r/10056/ -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8163) MemStoreChunkPool: An improvement for JAVA GC when using MSLAB
[ https://issues.apache.org/jira/browse/HBASE-8163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13615214#comment-13615214 ] Hudson commented on HBASE-8163: --- Integrated in hbase-0.95-on-hadoop2 #45 (See [https://builds.apache.org/job/hbase-0.95-on-hadoop2/45/]) HBASE-8163 MemStoreChunkPool: An improvement for JAVA GC when using MSLAB (Revision 1461399) Result = FAILURE zjushch : Files : * /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MemStore.java * /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MemStoreChunkPool.java * /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MemStoreFlusher.java * /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MemStoreLAB.java * /hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestMemStoreChunkPool.java > MemStoreChunkPool: An improvement for JAVA GC when using MSLAB > -- > > Key: HBASE-8163 > URL: https://issues.apache.org/jira/browse/HBASE-8163 > Project: HBase > Issue Type: New Feature > Components: regionserver >Reporter: chunhui shen >Assignee: chunhui shen > Fix For: 0.95.0, 0.98.0 > > Attachments: hbase-0.95-8163v6.patch, hbase-8163v1.patch, > hbase-8163v2.patch, hbase-8163v3.patch, hbase-8163v4.patch, > hbase-8163v5.patch, hbase-8163v6.patch > > > *Usage:* > Disable pool(default): configure "hbase.hregion.memstore.chunkpool.maxsize" > as 0 > Enable pool: configure "hbase.hregion.memstore.chunkpool.maxsize" as a > percentage of global memstore size (between 0.0 and 1.0, recommend to set it > as the gap between min and max sizes of memstore, e.g. 0.5) > *Background*： > When we use mslab,we will copy the keyvalue together in a structure called > *MemStoreLAB$Chunk*, therefore we could decrease the heap fragment. > *Problem*： > When one chunk is full, we will create a new chunk, and then the old chunk > will be reclaimed by JVM if no reference to it. > Mostly the chunk object would be promoted when doing Young GC, cause > increasing the cost of YGC > When a Chunk object has no reference? It should meet two following condition: > 1.Memstore which this chunk belongs to is flushed > 2.No scanner is opening on the memstore which this chunk belongs to > *Solution:* > 1.Create a chunk pool to manage the no-reference chunks, instead of being > reclaimed by JVM > 2.When a Chunk has no reference, put it back to the pool > 3.The pool has a max capacity, it will skip the chunks when achieve the max > size > 4.When we need new Chunk to store KeyValue, get it from the pool if exists, > else create new one by pool, so we could be able to reuse the old chunks > *Test results:* > Environment: > hbase-version:0.94 > -Xms4G -Xmx4G -Xmn2G > Row size=50 bytes, Value size=1024 bytes > 50 concurrent theads per client, insert 10,000,000 rows > Before: > Avg write request per second:12953 > After testing, final result of jstat -gcutil : > YGC YGCT FGC FGCT GCT > 747 36.503 48 2.492 38.995 > After: > Avg write request per second:14025 > After testing, final result of jstat -gcutil : > YGC YGCT FGC FGCT GCT > 711 20.344 4 0.284 20.628 > *Improvement: YGC 40+%; WPS 5%+* > review board : > https://reviews.apache.org/r/10056/ -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8163) MemStoreChunkPool: An improvement for JAVA GC when using MSLAB
[ https://issues.apache.org/jira/browse/HBASE-8163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13614911#comment-13614911 ] Hudson commented on HBASE-8163: --- Integrated in hbase-0.95 #109 (See [https://builds.apache.org/job/hbase-0.95/109/]) HBASE-8163 MemStoreChunkPool: An improvement for JAVA GC when using MSLAB (Revision 1461399) Result = FAILURE zjushch : Files : * /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MemStore.java * /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MemStoreChunkPool.java * /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MemStoreFlusher.java * /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MemStoreLAB.java * /hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestMemStoreChunkPool.java > MemStoreChunkPool: An improvement for JAVA GC when using MSLAB > -- > > Key: HBASE-8163 > URL: https://issues.apache.org/jira/browse/HBASE-8163 > Project: HBase > Issue Type: New Feature > Components: regionserver >Reporter: chunhui shen >Assignee: chunhui shen > Fix For: 0.95.0, 0.98.0 > > Attachments: hbase-0.95-8163v6.patch, hbase-8163v1.patch, > hbase-8163v2.patch, hbase-8163v3.patch, hbase-8163v4.patch, > hbase-8163v5.patch, hbase-8163v6.patch > > > *Usage:* > Disable pool(default): configure "hbase.hregion.memstore.chunkpool.maxsize" > as 0 > Enable pool: configure "hbase.hregion.memstore.chunkpool.maxsize" as a > percentage of global memstore size (between 0.0 and 1.0, recommend to set it > as the gap between min and max sizes of memstore, e.g. 0.5) > *Background*： > When we use mslab,we will copy the keyvalue together in a structure called > *MemStoreLAB$Chunk*, therefore we could decrease the heap fragment. > *Problem*： > When one chunk is full, we will create a new chunk, and then the old chunk > will be reclaimed by JVM if no reference to it. > Mostly the chunk object would be promoted when doing Young GC, cause > increasing the cost of YGC > When a Chunk object has no reference? It should meet two following condition: > 1.Memstore which this chunk belongs to is flushed > 2.No scanner is opening on the memstore which this chunk belongs to > *Solution:* > 1.Create a chunk pool to manage the no-reference chunks, instead of being > reclaimed by JVM > 2.When a Chunk has no reference, put it back to the pool > 3.The pool has a max capacity, it will skip the chunks when achieve the max > size > 4.When we need new Chunk to store KeyValue, get it from the pool if exists, > else create new one by pool, so we could be able to reuse the old chunks > *Test results:* > Environment: > hbase-version:0.94 > -Xms4G -Xmx4G -Xmn2G > Row size=50 bytes, Value size=1024 bytes > 50 concurrent theads per client, insert 10,000,000 rows > Before: > Avg write request per second:12953 > After testing, final result of jstat -gcutil : > YGC YGCT FGC FGCT GCT > 747 36.503 48 2.492 38.995 > After: > Avg write request per second:14025 > After testing, final result of jstat -gcutil : > YGC YGCT FGC FGCT GCT > 711 20.344 4 0.284 20.628 > *Improvement: YGC 40+%; WPS 5%+* > review board : > https://reviews.apache.org/r/10056/ -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8163) MemStoreChunkPool: An improvement for JAVA GC when using MSLAB
[ https://issues.apache.org/jira/browse/HBASE-8163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13614904#comment-13614904 ] Hudson commented on HBASE-8163: --- Integrated in HBase-TRUNK #3996 (See [https://builds.apache.org/job/HBase-TRUNK/3996/]) HBASE-8163 MemStoreChunkPool: An improvement for JAVA GC when using MSLAB (Revision 1461398) Result = FAILURE zjushch : Files : * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MemStore.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MemStoreChunkPool.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MemStoreFlusher.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MemStoreLAB.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestMemStoreChunkPool.java > MemStoreChunkPool: An improvement for JAVA GC when using MSLAB > -- > > Key: HBASE-8163 > URL: https://issues.apache.org/jira/browse/HBASE-8163 > Project: HBase > Issue Type: New Feature > Components: regionserver >Reporter: chunhui shen >Assignee: chunhui shen > Fix For: 0.95.0, 0.98.0 > > Attachments: hbase-0.95-8163v6.patch, hbase-8163v1.patch, > hbase-8163v2.patch, hbase-8163v3.patch, hbase-8163v4.patch, > hbase-8163v5.patch, hbase-8163v6.patch > > > *Usage:* > Disable pool(default): configure "hbase.hregion.memstore.chunkpool.maxsize" > as 0 > Enable pool: configure "hbase.hregion.memstore.chunkpool.maxsize" as a > percentage of global memstore size (between 0.0 and 1.0, recommend to set it > as the gap between min and max sizes of memstore, e.g. 0.5) > *Background*： > When we use mslab,we will copy the keyvalue together in a structure called > *MemStoreLAB$Chunk*, therefore we could decrease the heap fragment. > *Problem*： > When one chunk is full, we will create a new chunk, and then the old chunk > will be reclaimed by JVM if no reference to it. > Mostly the chunk object would be promoted when doing Young GC, cause > increasing the cost of YGC > When a Chunk object has no reference? It should meet two following condition: > 1.Memstore which this chunk belongs to is flushed > 2.No scanner is opening on the memstore which this chunk belongs to > *Solution:* > 1.Create a chunk pool to manage the no-reference chunks, instead of being > reclaimed by JVM > 2.When a Chunk has no reference, put it back to the pool > 3.The pool has a max capacity, it will skip the chunks when achieve the max > size > 4.When we need new Chunk to store KeyValue, get it from the pool if exists, > else create new one by pool, so we could be able to reuse the old chunks > *Test results:* > Environment: > hbase-version:0.94 > -Xms4G -Xmx4G -Xmn2G > Row size=50 bytes, Value size=1024 bytes > 50 concurrent theads per client, insert 10,000,000 rows > Before: > Avg write request per second:12953 > After testing, final result of jstat -gcutil : > YGC YGCT FGC FGCT GCT > 747 36.503 48 2.492 38.995 > After: > Avg write request per second:14025 > After testing, final result of jstat -gcutil : > YGC YGCT FGC FGCT GCT > 711 20.344 4 0.284 20.628 > *Improvement: YGC 40+%; WPS 5%+* > review board : > https://reviews.apache.org/r/10056/ -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8163) MemStoreChunkPool: An improvement for JAVA GC when using MSLAB
[ https://issues.apache.org/jira/browse/HBASE-8163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13614877#comment-13614877 ] chunhui shen commented on HBASE-8163: - Committed to trunk and 0.95, thanks for the review, Ted, Stack, Sergey > MemStoreChunkPool: An improvement for JAVA GC when using MSLAB > -- > > Key: HBASE-8163 > URL: https://issues.apache.org/jira/browse/HBASE-8163 > Project: HBase > Issue Type: New Feature > Components: regionserver >Reporter: chunhui shen >Assignee: chunhui shen > Attachments: hbase-8163v1.patch, hbase-8163v2.patch, > hbase-8163v3.patch, hbase-8163v4.patch, hbase-8163v5.patch, hbase-8163v6.patch > > > *Usage:* > Disable pool(default): configure "hbase.hregion.memstore.chunkpool.maxsize" > as 0 > Enable pool: configure "hbase.hregion.memstore.chunkpool.maxsize" as a > percentage of global memstore size (between 0.0 and 1.0, recommend to set it > as the gap between min and max sizes of memstore, e.g. 0.5) > *Background*： > When we use mslab,we will copy the keyvalue together in a structure called > *MemStoreLAB$Chunk*, therefore we could decrease the heap fragment. > *Problem*： > When one chunk is full, we will create a new chunk, and then the old chunk > will be reclaimed by JVM if no reference to it. > Mostly the chunk object would be promoted when doing Young GC, cause > increasing the cost of YGC > When a Chunk object has no reference? It should meet two following condition: > 1.Memstore which this chunk belongs to is flushed > 2.No scanner is opening on the memstore which this chunk belongs to > *Solution:* > 1.Create a chunk pool to manage the no-reference chunks, instead of being > reclaimed by JVM > 2.When a Chunk has no reference, put it back to the pool > 3.The pool has a max capacity, it will skip the chunks when achieve the max > size > 4.When we need new Chunk to store KeyValue, get it from the pool if exists, > else create new one by pool, so we could be able to reuse the old chunks > *Test results:* > Environment: > hbase-version:0.94 > -Xms4G -Xmx4G -Xmn2G > Row size=50 bytes, Value size=1024 bytes > 50 concurrent theads per client, insert 10,000,000 rows > Before: > Avg write request per second:12953 > After testing, final result of jstat -gcutil : > YGC YGCT FGC FGCT GCT > 747 36.503 48 2.492 38.995 > After: > Avg write request per second:14025 > After testing, final result of jstat -gcutil : > YGC YGCT FGC FGCT GCT > 711 20.344 4 0.284 20.628 > *Improvement: YGC 40+%; WPS 5%+* > review board : > https://reviews.apache.org/r/10056/ -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8163) MemStoreChunkPool: An improvement for JAVA GC when using MSLAB
[ https://issues.apache.org/jira/browse/HBASE-8163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13613634#comment-13613634 ] chunhui shen commented on HBASE-8163: - Will commit tomorrow if no objection > MemStoreChunkPool: An improvement for JAVA GC when using MSLAB > -- > > Key: HBASE-8163 > URL: https://issues.apache.org/jira/browse/HBASE-8163 > Project: HBase > Issue Type: New Feature > Components: regionserver >Reporter: chunhui shen >Assignee: chunhui shen > Attachments: hbase-8163v1.patch, hbase-8163v2.patch, > hbase-8163v3.patch, hbase-8163v4.patch, hbase-8163v5.patch, hbase-8163v6.patch > > > *Usage:* > Disable pool(default): configure "hbase.hregion.memstore.chunkpool.maxsize" > as 0 > Enable pool: configure "hbase.hregion.memstore.chunkpool.maxsize" as a > percentage of global memstore size (between 0.0 and 1.0, recommend to set it > as the gap between min and max sizes of memstore, e.g. 0.5) > *Background*： > When we use mslab,we will copy the keyvalue together in a structure called > *MemStoreLAB$Chunk*, therefore we could decrease the heap fragment. > *Problem*： > When one chunk is full, we will create a new chunk, and then the old chunk > will be reclaimed by JVM if no reference to it. > Mostly the chunk object would be promoted when doing Young GC, cause > increasing the cost of YGC > When a Chunk object has no reference? It should meet two following condition: > 1.Memstore which this chunk belongs to is flushed > 2.No scanner is opening on the memstore which this chunk belongs to > *Solution:* > 1.Create a chunk pool to manage the no-reference chunks, instead of being > reclaimed by JVM > 2.When a Chunk has no reference, put it back to the pool > 3.The pool has a max capacity, it will skip the chunks when achieve the max > size > 4.When we need new Chunk to store KeyValue, get it from the pool if exists, > else create new one by pool, so we could be able to reuse the old chunks > *Test results:* > Environment: > hbase-version:0.94 > -Xms4G -Xmx4G -Xmn2G > Row size=50 bytes, Value size=1024 bytes > 50 concurrent theads per client, insert 10,000,000 rows > Before: > Avg write request per second:12953 > After testing, final result of jstat -gcutil : > YGC YGCT FGC FGCT GCT > 747 36.503 48 2.492 38.995 > After: > Avg write request per second:14025 > After testing, final result of jstat -gcutil : > YGC YGCT FGC FGCT GCT > 711 20.344 4 0.284 20.628 > *Improvement: YGC 40+%; WPS 5%+* > review board : > https://reviews.apache.org/r/10056/ -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8163) MemStoreChunkPool: An improvement for JAVA GC when using MSLAB
[
https://issues.apache.org/jira/browse/HBASE-8163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13613607#comment-13613607
]
Hadoop QA commented on HBASE-8163:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12575485/hbase-8163v6.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:red}-1 site{color}. The patch appears to cause mvn site goal to
fail.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.master.TestZKBasedOpenCloseRegion
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/5015//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5015//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5015//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5015//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5015//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5015//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5015//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5015//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5015//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/5015//console
This message is automatically generated.
> MemStoreChunkPool: An improvement for JAVA GC when using MSLAB
> --
>
> Key: HBASE-8163
> URL: https://issues.apache.org/jira/browse/HBASE-8163
> Project: HBase
> Issue Type: New Feature
> Components: regionserver
>Reporter: chunhui shen
>Assignee: chunhui shen
> Attachments: hbase-8163v1.patch, hbase-8163v2.patch,
> hbase-8163v3.patch, hbase-8163v4.patch, hbase-8163v5.patch, hbase-8163v6.patch
>
>
> *Usage:*
> Disable pool(default): configure "hbase.hregion.memstore.chunkpool.maxsize"
> as 0
> Enable pool: configure "hbase.hregion.memstore.chunkpool.maxsize" as a
> percentage of global memstore size (between 0.0 and 1.0, recommend to set it
> as the gap between min and max sizes of memstore, e.g. 0.5)
> *Background*：
> When we use mslab,we will copy the keyvalue together in a structure called
> *MemStoreLAB$Chunk*, therefore we could decrease the heap fragment.
> *Problem*：
> When one chunk is full, we will create a new chunk, and then the old chunk
> will be reclaimed by JVM if no reference to it.
> Mostly the chunk object would be promoted when doing Young GC, cause
> increasing the cost of YGC
> When a Chunk object has no reference? It should meet two following condition:
> 1.Memstore which this chunk belongs to is flushed
> 2.No scanner is opening on the memstore which this chunk belongs to
> *Solution:*
> 1.Create a chunk pool to manage the no-reference chunks, instead of being
> reclaimed by JVM
> 2.When a Chunk has no reference, put it back to the pool
> 3.The pool has a max capacity, it will skip the chunks when achieve the max
> size
> 4.When we need new Chunk to store KeyValue, get it from the pool if exists,
> else create new one by pool, so we could be able to reuse the old chunks
> *Test results:*
> Environment:
> hbase-version:0.94
> -Xms4G -Xmx4G -Xmn2G
> Row size=50 bytes, Value size=1024 bytes
> 50 concurrent theads per client, insert 10,000,000 r
[jira] [Commented] (HBASE-8163) MemStoreChunkPool: An improvement for JAVA GC when using MSLAB
[
https://issues.apache.org/jira/browse/HBASE-8163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13613545#comment-13613545
]
Hadoop QA commented on HBASE-8163:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12575473/hbase-8163v5.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 1
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:red}-1 site{color}. The patch appears to cause mvn site goal to
fail.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.regionserver.TestAtomicOperation
org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/5014//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5014//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5014//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5014//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5014//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5014//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5014//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5014//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5014//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/5014//console
This message is automatically generated.
> MemStoreChunkPool: An improvement for JAVA GC when using MSLAB
> --
>
> Key: HBASE-8163
> URL: https://issues.apache.org/jira/browse/HBASE-8163
> Project: HBase
> Issue Type: New Feature
> Components: regionserver
>Reporter: chunhui shen
>Assignee: chunhui shen
> Attachments: hbase-8163v1.patch, hbase-8163v2.patch,
> hbase-8163v3.patch, hbase-8163v4.patch, hbase-8163v5.patch
>
>
> *Usage:*
> Disable pool(default): configure "hbase.hregion.memstore.chunkpool.maxsize"
> as 0
> Enable pool: configure "hbase.hregion.memstore.chunkpool.maxsize" as a
> percentage of global memstore size (between 0.0 and 1.0, recommend to set it
> as the gap between min and max sizes of memstore, e.g. 0.5)
> *Background*：
> When we use mslab,we will copy the keyvalue together in a structure called
> *MemStoreLAB$Chunk*, therefore we could decrease the heap fragment.
> *Problem*：
> When one chunk is full, we will create a new chunk, and then the old chunk
> will be reclaimed by JVM if no reference to it.
> Mostly the chunk object would be promoted when doing Young GC, cause
> increasing the cost of YGC
> When a Chunk object has no reference? It should meet two following condition:
> 1.Memstore which this chunk belongs to is flushed
> 2.No scanner is opening on the memstore which this chunk belongs to
> *Solution:*
> 1.Create a chunk pool to manage the no-reference chunks, instead of being
> reclaimed by JVM
> 2.When a Chunk has no reference, put it back to the pool
> 3.The pool has a max capacity, it will skip the chunks when achieve the max
> size
> 4.When we need new Chunk to store KeyValue, get it from the pool if exists,
> else create new one by pool, so we could be able to reuse the old chunks
> *Test results:*
> Environment:
> hbase-version:0.94
> -Xms4G -Xmx4G -Xmn2G
> Row size=50 bytes, Value size=1024 by
[jira] [Commented] (HBASE-8163) MemStoreChunkPool: An improvement for JAVA GC when using MSLAB
[ https://issues.apache.org/jira/browse/HBASE-8163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13613517#comment-13613517 ] chunhui shen commented on HBASE-8163: - bq.Are you fellas running this and if so, what is your experience? We have tested this for some time, it could notability decrease the cost of YGC as I mentioned in the description. For online application, JAVA GC is troublesome when using HBase as a Database... Since MSLab,MSChunkPool and BucketCache, HBase could manage the main memory itself without depending on JVM > MemStoreChunkPool: An improvement for JAVA GC when using MSLAB > -- > > Key: HBASE-8163 > URL: https://issues.apache.org/jira/browse/HBASE-8163 > Project: HBase > Issue Type: New Feature > Components: regionserver >Reporter: chunhui shen >Assignee: chunhui shen > Attachments: hbase-8163v1.patch, hbase-8163v2.patch, > hbase-8163v3.patch, hbase-8163v4.patch, hbase-8163v5.patch > > > *Usage:* > Disable pool(default): configure "hbase.hregion.memstore.chunkpool.maxsize" > as 0 > Enable pool: configure "hbase.hregion.memstore.chunkpool.maxsize" as a > percentage of global memstore size (between 0.0 and 1.0, recommend to set it > as the gap between min and max sizes of memstore, e.g. 0.5f) > *Background*： > When we use mslab,we will copy the keyvalue together in a structure called > *MemStoreLAB$Chunk*, therefore we could decrease the heap fragment. > *Problem*： > When one chunk is full, we will create a new chunk, and then the old chunk > will be reclaimed by JVM if no reference to it. > Mostly the chunk object would be promoted when doing Young GC, cause > increasing the cost of YGC > When a Chunk object has no reference? It should meet two following condition: > 1.Memstore which this chunk belongs to is flushed > 2.No scanner is opening on the memstore which this chunk belongs to > *Solution:* > 1.Create a chunk pool to manage the no-reference chunks, instead of being > reclaimed by JVM > 2.When a Chunk has no reference, put it back to the pool > 3.The pool has a max capacity, it will skip the chunks when achieve the max > size > 4.When we need new Chunk to store KeyValue, get it from the pool if exists, > else create new one by pool, so we could be able to reuse the old chunks > *Test results:* > Environment: > hbase-version:0.94 > -Xms4G -Xmx4G -Xmn2G > Row size=50 bytes, Value size=1024 bytes > 50 concurrent theads per client, insert 10,000,000 rows > Before: > Avg write request per second:12953 > After testing, final result of jstat -gcutil : > YGC YGCT FGC FGCT GCT > 747 36.503 48 2.492 38.995 > After: > Avg write request per second:14025 > After testing, final result of jstat -gcutil : > YGC YGCT FGC FGCT GCT > 711 20.344 4 0.284 20.628 > *Improvement: YGC 40+%; WPS 5%+* > review board : > https://reviews.apache.org/r/10056/ -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8163) MemStoreChunkPool: An improvement for JAVA GC when using MSLAB
[ https://issues.apache.org/jira/browse/HBASE-8163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13613507#comment-13613507 ] stack commented on HBASE-8163: -- [~zjushch] Are you fellas running this and if so, what is your experience? The patch looks good to me. In release note you should be clear that it is off by default and explain how to turn it on (it is not that obvious). I'd be +1 on this coming into trunk and 0.95. > MemStoreChunkPool: An improvement for JAVA GC when using MSLAB > -- > > Key: HBASE-8163 > URL: https://issues.apache.org/jira/browse/HBASE-8163 > Project: HBase > Issue Type: New Feature > Components: regionserver >Reporter: chunhui shen >Assignee: chunhui shen > Attachments: hbase-8163v1.patch, hbase-8163v2.patch, > hbase-8163v3.patch, hbase-8163v4.patch, hbase-8163v5.patch > > > *Usage:* > Disable pool(default): configure "hbase.hregion.memstore.chunkpool.maxsize" > as 0 > Enable pool: configure "hbase.hregion.memstore.chunkpool.maxsize" as a > percentage of global memstore size (between 0.0 and 1.0, recommend to set it > as the gap between min and max sizes of memstore, e.g. 0.5f) > *Background*： > When we use mslab,we will copy the keyvalue together in a structure called > *MemStoreLAB$Chunk*, therefore we could decrease the heap fragment. > *Problem*： > When one chunk is full, we will create a new chunk, and then the old chunk > will be reclaimed by JVM if no reference to it. > Mostly the chunk object would be promoted when doing Young GC, cause > increasing the cost of YGC > When a Chunk object has no reference? It should meet two following condition: > 1.Memstore which this chunk belongs to is flushed > 2.No scanner is opening on the memstore which this chunk belongs to > *Solution:* > 1.Create a chunk pool to manage the no-reference chunks, instead of being > reclaimed by JVM > 2.When a Chunk has no reference, put it back to the pool > 3.The pool has a max capacity, it will skip the chunks when achieve the max > size > 4.When we need new Chunk to store KeyValue, get it from the pool if exists, > else create new one by pool, so we could be able to reuse the old chunks > *Test results:* > Environment: > hbase-version:0.94 > -Xms4G -Xmx4G -Xmn2G > Row size=50 bytes, Value size=1024 bytes > 50 concurrent theads per client, insert 10,000,000 rows > Before: > Avg write request per second:12953 > After testing, final result of jstat -gcutil : > YGC YGCT FGC FGCT GCT > 747 36.503 48 2.492 38.995 > After: > Avg write request per second:14025 > After testing, final result of jstat -gcutil : > YGC YGCT FGC FGCT GCT > 711 20.344 4 0.284 20.628 > *Improvement: YGC 40+%; WPS 5%+* > review board : > https://reviews.apache.org/r/10056/ -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8163) MemStoreChunkPool: An improvement for JAVA GC when using MSLAB
[ https://issues.apache.org/jira/browse/HBASE-8163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13612894#comment-13612894 ] Sergey Shelukhin commented on HBASE-8163: - LGTM > MemStoreChunkPool: An improvement for JAVA GC when using MSLAB > -- > > Key: HBASE-8163 > URL: https://issues.apache.org/jira/browse/HBASE-8163 > Project: HBase > Issue Type: New Feature > Components: regionserver >Reporter: chunhui shen >Assignee: chunhui shen > Attachments: hbase-8163v1.patch, hbase-8163v2.patch, > hbase-8163v3.patch, hbase-8163v4.patch > > > *Background*： > When we use mslab,we will copy the keyvalue together in a structure called > *MemStoreLAB$Chunk*, therefore we could decrease the heap fragment. > *Problem*： > When one chunk is full, we will create a new chunk, and then the old chunk > will be reclaimed by JVM if no reference to it. > Mostly the chunk object would be promoted when doing Young GC, cause > increasing the cost of YGC > When a Chunk object has no reference? It should meet two following condition: > 1.Memstore which this chunk belongs to is flushed > 2.No scanner is opening on the memstore which this chunk belongs to > *Solution:* > 1.Create a chunk pool to manage the no-reference chunks, instead of being > reclaimed by JVM > 2.When a Chunk has no reference, put it back to the pool > 3.The pool has a max capacity, it will skip the chunks when achieve the max > size > 4.When we need new Chunk to store KeyValue, get it from the pool if exists, > else create new one by pool, so we could be able to reuse the old chunks > *Test results:* > Environment: > hbase-version:0.94 > -Xms4G -Xmx4G -Xmn2G > Row size=50 bytes, Value size=1024 bytes > 50 concurrent theads per client, insert 10,000,000 rows > Before: > Avg write request per second:12953 > After testing, final result of jstat -gcutil : > YGC YGCT FGC FGCT GCT > 747 36.503 48 2.492 38.995 > After: > Avg write request per second:14025 > After testing, final result of jstat -gcutil : > YGC YGCT FGC FGCT GCT > 711 20.344 4 0.284 20.628 > *Improvement: YGC 40+%; WPS 5%+* > review board : > https://reviews.apache.org/r/10056/ -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8163) MemStoreChunkPool: An improvement for JAVA GC when using MSLAB
[ https://issues.apache.org/jira/browse/HBASE-8163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13612856#comment-13612856 ] Ted Yu commented on HBASE-8163: --- I think the test failure above was due to: https://issues.apache.org/jira/browse/HBASE-7871?focusedCommentId=13612855&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13612855 > MemStoreChunkPool: An improvement for JAVA GC when using MSLAB > -- > > Key: HBASE-8163 > URL: https://issues.apache.org/jira/browse/HBASE-8163 > Project: HBase > Issue Type: New Feature > Components: regionserver >Reporter: chunhui shen >Assignee: chunhui shen > Attachments: hbase-8163v1.patch, hbase-8163v2.patch, > hbase-8163v3.patch, hbase-8163v4.patch > > > *Background*： > When we use mslab,we will copy the keyvalue together in a structure called > *MemStoreLAB$Chunk*, therefore we could decrease the heap fragment. > *Problem*： > When one chunk is full, we will create a new chunk, and then the old chunk > will be reclaimed by JVM if no reference to it. > Mostly the chunk object would be promoted when doing Young GC, cause > increasing the cost of YGC > When a Chunk object has no reference? It should meet two following condition: > 1.Memstore which this chunk belongs to is flushed > 2.No scanner is opening on the memstore which this chunk belongs to > *Solution:* > 1.Create a chunk pool to manage the no-reference chunks, instead of being > reclaimed by JVM > 2.When a Chunk has no reference, put it back to the pool > 3.The pool has a max capacity, it will skip the chunks when achieve the max > size > 4.When we need new Chunk to store KeyValue, get it from the pool if exists, > else create new one by pool, so we could be able to reuse the old chunks > *Test results:* > Environment: > hbase-version:0.94 > -Xms4G -Xmx4G -Xmn2G > Row size=50 bytes, Value size=1024 bytes > 50 concurrent theads per client, insert 10,000,000 rows > Before: > Avg write request per second:12953 > After testing, final result of jstat -gcutil : > YGC YGCT FGC FGCT GCT > 747 36.503 48 2.492 38.995 > After: > Avg write request per second:14025 > After testing, final result of jstat -gcutil : > YGC YGCT FGC FGCT GCT > 711 20.344 4 0.284 20.628 > *Improvement: YGC 40+%; WPS 5%+* > review board : > https://reviews.apache.org/r/10056/ -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8163) MemStoreChunkPool: An improvement for JAVA GC when using MSLAB
[
https://issues.apache.org/jira/browse/HBASE-8163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13612389#comment-13612389
]
Hadoop QA commented on HBASE-8163:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12575273/hbase-8163v4.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 1
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:red}-1 site{color}. The patch appears to cause mvn site goal to
fail.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.util.TestMergeTable
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/4992//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4992//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4992//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4992//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4992//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4992//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4992//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4992//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4992//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/4992//console
This message is automatically generated.
> MemStoreChunkPool: An improvement for JAVA GC when using MSLAB
> --
>
> Key: HBASE-8163
> URL: https://issues.apache.org/jira/browse/HBASE-8163
> Project: HBase
> Issue Type: New Feature
> Components: regionserver
>Reporter: chunhui shen
>Assignee: chunhui shen
> Attachments: hbase-8163v1.patch, hbase-8163v2.patch,
> hbase-8163v3.patch, hbase-8163v4.patch
>
>
> *Background*：
> When we use mslab,we will copy the keyvalue together in a structure called
> *MemStoreLAB$Chunk*, therefore we could decrease the heap fragment.
> *Problem*：
> When one chunk is full, we will create a new chunk, and then the old chunk
> will be reclaimed by JVM if no reference to it.
> Mostly the chunk object would be promoted when doing Young GC, cause
> increasing the cost of YGC
> When a Chunk object has no reference? It should meet two following condition:
> 1.Memstore which this chunk belongs to is flushed
> 2.No scanner is opening on the memstore which this chunk belongs to
> *Solution:*
> 1.Create a chunk pool to manage the no-reference chunks, instead of being
> reclaimed by JVM
> 2.When a Chunk has no reference, put it back to the pool
> 3.The pool has a max capacity, it will skip the chunks when achieve the max
> size
> 4.When we need new Chunk to store KeyValue, get it from the pool if exists,
> else create new one by pool, so we could be able to reuse the old chunks
> *Test results:*
> Environment:
> hbase-version:0.94
> -Xms4G -Xmx4G -Xmn2G
> Row size=50 bytes, Value size=1024 bytes
> 50 concurrent theads per client, insert 10,000,000 rows
> Before:
> Avg write request per second:12953
> After testing, final result of jstat -gcutil :
> YGC YGCT FGC FGCT GCT
> 747 36.503 48 2.492 38.995
> After:
> Avg write request per second:14025
> After testing, final result of jstat -gcutil :
> YGC YGCT FGC FGCT GCT
> 711 20.344 4 0.284 20.628
> *Improvement: YGC 40+%; WPS 5%+*
> review board :
> https:
[jira] [Commented] (HBASE-8163) MemStoreChunkPool: An improvement for JAVA GC when using MSLAB
[ https://issues.apache.org/jira/browse/HBASE-8163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13609881#comment-13609881 ] stack commented on HBASE-8163: -- Rather than choose a hard size for the pool, as as suggested above, making it a percentage of global memstore size would be easier on the operator. How do I know the pool is working effectively? I see no metrics emissions added. I'd do some math on flush sizes and infer whether or not it effective? Good stuff. > MemStoreChunkPool: An improvement for JAVA GC when using MSLAB > -- > > Key: HBASE-8163 > URL: https://issues.apache.org/jira/browse/HBASE-8163 > Project: HBase > Issue Type: New Feature > Components: regionserver >Reporter: chunhui shen >Assignee: chunhui shen > Attachments: hbase-8163v1.patch, hbase-8163v2.patch, > hbase-8163v3.patch > > > *Background*： > When we use mslab,we will copy the keyvalue together in a structure called > *MemStoreLAB$Chunk*, therefore we could decrease the heap fragment. > *Problem*： > When one chunk is full, we will create a new chunk, and then the old chunk > will be reclaimed by JVM if no reference to it. > Mostly the chunk object would be promoted when doing Young GC, cause > increasing the cost of YGC > When a Chunk object has no reference? It should meet two following condition: > 1.Memstore which this chunk belongs to is flushed > 2.No scanner is opening on the memstore which this chunk belongs to > *Solution:* > 1.Create a chunk pool to manage the no-reference chunks, instead of being > reclaimed by JVM > 2.When a Chunk has no reference, put it back to the pool > 3.The pool has a max capacity, it will skip the chunks when achieve the max > size > 4.When we need new Chunk to store KeyValue, get it from the pool if exists, > else create new one by pool, so we could be able to reuse the old chunks > *Test results:* > Environment: > hbase-version:0.94 > -Xms4G -Xmx4G -Xmn2G > Row size=50 bytes, Value size=1024 bytes > 50 concurrent theads per client, insert 10,000,000 rows > Before: > Avg write request per second:12953 > After testing, final result of jstat -gcutil : > YGC YGCT FGC FGCT GCT > 747 36.503 48 2.492 38.995 > After: > Avg write request per second:14025 > After testing, final result of jstat -gcutil : > YGC YGCT FGC FGCT GCT > 711 20.344 4 0.284 20.628 > *Improvement: YGC 40+%; WPS 5%+* > review board : > https://reviews.apache.org/r/10056/ -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8163) MemStoreChunkPool: An improvement for JAVA GC when using MSLAB
[
https://issues.apache.org/jira/browse/HBASE-8163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13609853#comment-13609853
]
Hadoop QA commented on HBASE-8163:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12574939/hbase-8163v3.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 1
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/4959//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4959//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4959//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4959//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4959//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4959//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4959//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4959//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4959//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/4959//console
This message is automatically generated.
> MemStoreChunkPool: An improvement for JAVA GC when using MSLAB
> --
>
> Key: HBASE-8163
> URL: https://issues.apache.org/jira/browse/HBASE-8163
> Project: HBase
> Issue Type: New Feature
> Components: regionserver
>Reporter: chunhui shen
>Assignee: chunhui shen
> Attachments: hbase-8163v1.patch, hbase-8163v2.patch,
> hbase-8163v3.patch
>
>
> *Background*：
> When we use mslab,we will copy the keyvalue together in a structure called
> *MemStoreLAB$Chunk*, therefore we could decrease the heap fragment.
> *Problem*：
> When one chunk is full, we will create a new chunk, and then the old chunk
> will be reclaimed by JVM if no reference to it.
> Mostly the chunk object would be promoted when doing Young GC, cause
> increasing the cost of YGC
> When a Chunk object has no reference? It should meet two following condition:
> 1.Memstore which this chunk belongs to is flushed
> 2.No scanner is opening on the memstore which this chunk belongs to
> *Solution:*
> 1.Create a chunk pool to manage the no-reference chunks, instead of being
> reclaimed by JVM
> 2.When a Chunk has no reference, put it back to the pool
> 3.The pool has a max capacity, it will skip the chunks when achieve the max
> size
> 4.When we need new Chunk to store KeyValue, get it from the pool if exists,
> else create new one by pool, so we could be able to reuse the old chunks
> *Test results:*
> Environment:
> hbase-version:0.94
> -Xms4G -Xmx4G -Xmn2G
> Row size=50 bytes, Value size=1024 bytes
> 50 concurrent theads per client, insert 10,000,000 rows
> Before:
> Avg write request per second:12953
> After testing, final result of jstat -gcutil :
> YGC YGCT FGC FGCT GCT
> 747 36.503 48 2.492 38.995
> After:
> Avg write request per second:14025
> After testing, final result of jstat -gcutil :
> YGC YGCT FGC FGCT GCT
> 711 20.344 4 0.284 20.628
> *Improvement: YGC 40+%; WPS 5%+*
> review board :
> https://reviews.apache.org/r/10056/
--
This message is automatically generated by JIRA.
If you think
[jira] [Commented] (HBASE-8163) MemStoreChunkPool: An improvement for JAVA GC when using MSLAB
[ https://issues.apache.org/jira/browse/HBASE-8163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13609793#comment-13609793 ] chunhui shen commented on HBASE-8163: - bq.don't we offset it period if successful reuse We will offset GC mostly, unless the memstore size bound seriously, As my above comments, if memstore will be from full to empty to full periodly, the pool is not appropriate > MemStoreChunkPool: An improvement for JAVA GC when using MSLAB > -- > > Key: HBASE-8163 > URL: https://issues.apache.org/jira/browse/HBASE-8163 > Project: HBase > Issue Type: New Feature > Components: regionserver >Reporter: chunhui shen >Assignee: chunhui shen > Attachments: hbase-8163v1.patch, hbase-8163v2.patch > > > *Background*： > When we use mslab,we will copy the keyvalue together in a structure called > *MemStoreLAB$Chunk*, therefore we could decrease the heap fragment. > *Problem*： > When one chunk is full, we will create a new chunk, and then the old chunk > will be reclaimed by JVM if no reference to it. > Mostly the chunk object would be promoted when doing Young GC, cause > increasing the cost of YGC > When a Chunk object has no reference? It should meet two following condition: > 1.Memstore which this chunk belongs to is flushed > 2.No scanner is opening on the memstore which this chunk belongs to > *Solution:* > 1.Create a chunk pool to manage the no-reference chunks, instead of being > reclaimed by JVM > 2.When a Chunk has no reference, put it back to the pool > 3.The pool has a max capacity, it will skip the chunks when achieve the max > size > 4.When we need new Chunk to store KeyValue, get it from the pool if exists, > else create new one by pool, so we could be able to reuse the old chunks > *Test results:* > Environment: > hbase-version:0.94 > -Xms4G -Xmx4G -Xmn2G > Row size=50 bytes, Value size=1024 bytes > 50 concurrent theads per client, insert 10,000,000 rows > Before: > Avg write request per second:12953 > After testing, final result of jstat -gcutil : > YGC YGCT FGC FGCT GCT > 747 36.503 48 2.492 38.995 > After: > Avg write request per second:14025 > After testing, final result of jstat -gcutil : > YGC YGCT FGC FGCT GCT > 711 20.344 4 0.284 20.628 > *Improvement: YGC 40+%; WPS 5%+* > review board : > https://reviews.apache.org/r/10056/ -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8163) MemStoreChunkPool: An improvement for JAVA GC when using MSLAB
[ https://issues.apache.org/jira/browse/HBASE-8163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13609790#comment-13609790 ] chunhui shen commented on HBASE-8163: - bq.Can we guarantee that pool size is 0 when memstores are full or near full Yes, we can guarantee that (pool size + memstore siz)< memstore upper limit bq.pool should not be above and beyond memstore allocation It wouldn't, pool data come from the memstore > MemStoreChunkPool: An improvement for JAVA GC when using MSLAB > -- > > Key: HBASE-8163 > URL: https://issues.apache.org/jira/browse/HBASE-8163 > Project: HBase > Issue Type: New Feature > Components: regionserver >Reporter: chunhui shen >Assignee: chunhui shen > Attachments: hbase-8163v1.patch, hbase-8163v2.patch > > > *Background*： > When we use mslab,we will copy the keyvalue together in a structure called > *MemStoreLAB$Chunk*, therefore we could decrease the heap fragment. > *Problem*： > When one chunk is full, we will create a new chunk, and then the old chunk > will be reclaimed by JVM if no reference to it. > Mostly the chunk object would be promoted when doing Young GC, cause > increasing the cost of YGC > When a Chunk object has no reference? It should meet two following condition: > 1.Memstore which this chunk belongs to is flushed > 2.No scanner is opening on the memstore which this chunk belongs to > *Solution:* > 1.Create a chunk pool to manage the no-reference chunks, instead of being > reclaimed by JVM > 2.When a Chunk has no reference, put it back to the pool > 3.The pool has a max capacity, it will skip the chunks when achieve the max > size > 4.When we need new Chunk to store KeyValue, get it from the pool if exists, > else create new one by pool, so we could be able to reuse the old chunks > *Test results:* > Environment: > hbase-version:0.94 > -Xms4G -Xmx4G -Xmn2G > Row size=50 bytes, Value size=1024 bytes > 50 concurrent theads per client, insert 10,000,000 rows > Before: > Avg write request per second:12953 > After testing, final result of jstat -gcutil : > YGC YGCT FGC FGCT GCT > 747 36.503 48 2.492 38.995 > After: > Avg write request per second:14025 > After testing, final result of jstat -gcutil : > YGC YGCT FGC FGCT GCT > 711 20.344 4 0.284 20.628 > *Improvement: YGC 40+%; WPS 5%+* > review board : > https://reviews.apache.org/r/10056/ -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8163) MemStoreChunkPool: An improvement for JAVA GC when using MSLAB
[ https://issues.apache.org/jira/browse/HBASE-8163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13609788#comment-13609788 ] chunhui shen commented on HBASE-8163: - bq.Does it make more sense to define maxsize as percentage of (total) memstore size ? It's good idea, but setting pool size 0 by default means we close the function of pool, so it wouldn't affect current logic > MemStoreChunkPool: An improvement for JAVA GC when using MSLAB > -- > > Key: HBASE-8163 > URL: https://issues.apache.org/jira/browse/HBASE-8163 > Project: HBase > Issue Type: New Feature > Components: regionserver >Reporter: chunhui shen >Assignee: chunhui shen > Attachments: hbase-8163v1.patch, hbase-8163v2.patch > > > *Background*： > When we use mslab,we will copy the keyvalue together in a structure called > *MemStoreLAB$Chunk*, therefore we could decrease the heap fragment. > *Problem*： > When one chunk is full, we will create a new chunk, and then the old chunk > will be reclaimed by JVM if no reference to it. > Mostly the chunk object would be promoted when doing Young GC, cause > increasing the cost of YGC > When a Chunk object has no reference? It should meet two following condition: > 1.Memstore which this chunk belongs to is flushed > 2.No scanner is opening on the memstore which this chunk belongs to > *Solution:* > 1.Create a chunk pool to manage the no-reference chunks, instead of being > reclaimed by JVM > 2.When a Chunk has no reference, put it back to the pool > 3.The pool has a max capacity, it will skip the chunks when achieve the max > size > 4.When we need new Chunk to store KeyValue, get it from the pool if exists, > else create new one by pool, so we could be able to reuse the old chunks > *Test results:* > Environment: > hbase-version:0.94 > -Xms4G -Xmx4G -Xmn2G > Row size=50 bytes, Value size=1024 bytes > 50 concurrent theads per client, insert 10,000,000 rows > Before: > Avg write request per second:12953 > After testing, final result of jstat -gcutil : > YGC YGCT FGC FGCT GCT > 747 36.503 48 2.492 38.995 > After: > Avg write request per second:14025 > After testing, final result of jstat -gcutil : > YGC YGCT FGC FGCT GCT > 711 20.344 4 0.284 20.628 > *Improvement: YGC 40+%; WPS 5%+* > review board : > https://reviews.apache.org/r/10056/ -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8163) MemStoreChunkPool: An improvement for JAVA GC when using MSLAB
[ https://issues.apache.org/jira/browse/HBASE-8163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13609321#comment-13609321 ] stack commented on HBASE-8163: -- [~zjushch] Can we guarantee that pool size is 0 when memstores are full or near full? [~lhofhansl] "If it is too big you'll waste space, if it is too small you'll just offset the GC for a bit." Agree that pool should not be above and beyond memstore allocation. Regards your comment that we just offset GC for a bit, don't we offset it period if successful reuse? > MemStoreChunkPool: An improvement for JAVA GC when using MSLAB > -- > > Key: HBASE-8163 > URL: https://issues.apache.org/jira/browse/HBASE-8163 > Project: HBase > Issue Type: New Feature > Components: regionserver >Reporter: chunhui shen >Assignee: chunhui shen > Attachments: hbase-8163v1.patch, hbase-8163v2.patch > > > *Background*： > When we use mslab,we will copy the keyvalue together in a structure called > *MemStoreLAB$Chunk*, therefore we could decrease the heap fragment. > *Problem*： > When one chunk is full, we will create a new chunk, and then the old chunk > will be reclaimed by JVM if no reference to it. > Mostly the chunk object would be promoted when doing Young GC, cause > increasing the cost of YGC > When a Chunk object has no reference? It should meet two following condition: > 1.Memstore which this chunk belongs to is flushed > 2.No scanner is opening on the memstore which this chunk belongs to > *Solution:* > 1.Create a chunk pool to manage the no-reference chunks, instead of being > reclaimed by JVM > 2.When a Chunk has no reference, put it back to the pool > 3.The pool has a max capacity, it will skip the chunks when achieve the max > size > 4.When we need new Chunk to store KeyValue, get it from the pool if exists, > else create new one by pool, so we could be able to reuse the old chunks > *Test results:* > Environment: > hbase-version:0.94 > -Xms4G -Xmx4G -Xmn2G > Row size=50 bytes, Value size=1024 bytes > 50 concurrent theads per client, insert 10,000,000 rows > Before: > Avg write request per second:12953 > After testing, final result of jstat -gcutil : > YGC YGCT FGC FGCT GCT > 747 36.503 48 2.492 38.995 > After: > Avg write request per second:14025 > After testing, final result of jstat -gcutil : > YGC YGCT FGC FGCT GCT > 711 20.344 4 0.284 20.628 > *Improvement: YGC 40+%; WPS 5%+* > review board : > https://reviews.apache.org/r/10056/ -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8163) MemStoreChunkPool: An improvement for JAVA GC when using MSLAB
[
https://issues.apache.org/jira/browse/HBASE-8163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13609308#comment-13609308
]
Hadoop QA commented on HBASE-8163:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12574757/hbase-8163v2.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 1
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
{color:red}-1 core zombie tests{color}. There are 1 zombie test(s):
at
org.apache.hadoop.hbase.util.TestHBaseFsck.testHbckFixOrphanTable(TestHBaseFsck.java:427)
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/4944//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4944//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4944//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4944//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4944//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4944//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4944//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4944//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4944//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/4944//console
This message is automatically generated.
> MemStoreChunkPool: An improvement for JAVA GC when using MSLAB
> --
>
> Key: HBASE-8163
> URL: https://issues.apache.org/jira/browse/HBASE-8163
> Project: HBase
> Issue Type: New Feature
> Components: regionserver
>Reporter: chunhui shen
>Assignee: chunhui shen
> Attachments: hbase-8163v1.patch, hbase-8163v2.patch
>
>
> *Background*：
> When we use mslab,we will copy the keyvalue together in a structure called
> *MemStoreLAB$Chunk*, therefore we could decrease the heap fragment.
> *Problem*：
> When one chunk is full, we will create a new chunk, and then the old chunk
> will be reclaimed by JVM if no reference to it.
> Mostly the chunk object would be promoted when doing Young GC, cause
> increasing the cost of YGC
> When a Chunk object has no reference? It should meet two following condition:
> 1.Memstore which this chunk belongs to is flushed
> 2.No scanner is opening on the memstore which this chunk belongs to
> *Solution:*
> 1.Create a chunk pool to manage the no-reference chunks, instead of being
> reclaimed by JVM
> 2.When a Chunk has no reference, put it back to the pool
> 3.The pool has a max capacity, it will skip the chunks when achieve the max
> size
> 4.When we need new Chunk to store KeyValue, get it from the pool if exists,
> else create new one by pool, so we could be able to reuse the old chunks
> *Test results:*
> Environment:
> hbase-version:0.94
> -Xms4G -Xmx4G -Xmn2G
> Row size=50 bytes, Value size=1024 bytes
> 50 concurrent theads per client, insert 10,000,000 rows
> Before:
> Avg write request per second:12953
> After testing, final result of jstat -gcutil :
> YGC YGCT FGC FGCT GCT
> 747 36.503 48 2.492 38.995
> After:
> Avg write request per second:14025
> After testing, final result of jstat -gcutil :
> YGC YGCT FGC FGCT GCT
> 711 20.344 4 0.284 20
[jira] [Commented] (HBASE-8163) MemStoreChunkPool: An improvement for JAVA GC when using MSLAB
[ https://issues.apache.org/jira/browse/HBASE-8163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13609273#comment-13609273 ] Sergey Shelukhin commented on HBASE-8163: - some r comments > MemStoreChunkPool: An improvement for JAVA GC when using MSLAB > -- > > Key: HBASE-8163 > URL: https://issues.apache.org/jira/browse/HBASE-8163 > Project: HBase > Issue Type: New Feature > Components: regionserver >Reporter: chunhui shen >Assignee: chunhui shen > Attachments: hbase-8163v1.patch, hbase-8163v2.patch > > > *Background*： > When we use mslab,we will copy the keyvalue together in a structure called > *MemStoreLAB$Chunk*, therefore we could decrease the heap fragment. > *Problem*： > When one chunk is full, we will create a new chunk, and then the old chunk > will be reclaimed by JVM if no reference to it. > Mostly the chunk object would be promoted when doing Young GC, cause > increasing the cost of YGC > When a Chunk object has no reference? It should meet two following condition: > 1.Memstore which this chunk belongs to is flushed > 2.No scanner is opening on the memstore which this chunk belongs to > *Solution:* > 1.Create a chunk pool to manage the no-reference chunks, instead of being > reclaimed by JVM > 2.When a Chunk has no reference, put it back to the pool > 3.The pool has a max capacity, it will skip the chunks when achieve the max > size > 4.When we need new Chunk to store KeyValue, get it from the pool if exists, > else create new one by pool, so we could be able to reuse the old chunks > *Test results:* > Environment: > hbase-version:0.94 > -Xms4G -Xmx4G -Xmn2G > Row size=50 bytes, Value size=1024 bytes > 50 concurrent theads per client, insert 10,000,000 rows > Before: > Avg write request per second:12953 > After testing, final result of jstat -gcutil : > YGC YGCT FGC FGCT GCT > 747 36.503 48 2.492 38.995 > After: > Avg write request per second:14025 > After testing, final result of jstat -gcutil : > YGC YGCT FGC FGCT GCT > 711 20.344 4 0.284 20.628 > *Improvement: YGC 40+%; WPS 5%+* > review board : > https://reviews.apache.org/r/10056/ -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8163) MemStoreChunkPool: An improvement for JAVA GC when using MSLAB
[ https://issues.apache.org/jira/browse/HBASE-8163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13609063#comment-13609063 ] Ted Yu commented on HBASE-8163: --- Does it make more sense to define maxsize as percentage of (total) memstore size ? > MemStoreChunkPool: An improvement for JAVA GC when using MSLAB > -- > > Key: HBASE-8163 > URL: https://issues.apache.org/jira/browse/HBASE-8163 > Project: HBase > Issue Type: New Feature > Components: regionserver >Reporter: chunhui shen >Assignee: chunhui shen > Attachments: hbase-8163v1.patch, hbase-8163v2.patch > > > *Background*： > When we use mslab,we will copy the keyvalue together in a structure called > *MemStoreLAB$Chunk*, therefore we could decrease the heap fragment. > *Problem*： > When one chunk is full, we will create a new chunk, and then the old chunk > will be reclaimed by JVM if no reference to it. > Mostly the chunk object would be promoted when doing Young GC, cause > increasing the cost of YGC > When a Chunk object has no reference? It should meet two following condition: > 1.Memstore which this chunk belongs to is flushed > 2.No scanner is opening on the memstore which this chunk belongs to > *Solution:* > 1.Create a chunk pool to manage the no-reference chunks, instead of being > reclaimed by JVM > 2.When a Chunk has no reference, put it back to the pool > 3.The pool has a max capacity, it will skip the chunks when achieve the max > size > 4.When we need new Chunk to store KeyValue, get it from the pool if exists, > else create new one by pool, so we could be able to reuse the old chunks > *Test results:* > Environment: > hbase-version:0.94 > -Xms4G -Xmx4G -Xmn2G > Row size=50 bytes, Value size=1024 bytes > 50 concurrent theads per client, insert 10,000,000 rows > Before: > Avg write request per second:12953 > After testing, final result of jstat -gcutil : > YGC YGCT FGC FGCT GCT > 747 36.503 48 2.492 38.995 > After: > Avg write request per second:14025 > After testing, final result of jstat -gcutil : > YGC YGCT FGC FGCT GCT > 711 20.344 4 0.284 20.628 > *Improvement: YGC 40+%; WPS 5%+* > review board : > https://reviews.apache.org/r/10056/ -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8163) MemStoreChunkPool: An improvement for JAVA GC when using MSLAB
[ https://issues.apache.org/jira/browse/HBASE-8163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13608727#comment-13608727 ] chunhui shen commented on HBASE-8163: - [~te...@apache.org] The gap can be got through the metrics. e.g ganglia > MemStoreChunkPool: An improvement for JAVA GC when using MSLAB > -- > > Key: HBASE-8163 > URL: https://issues.apache.org/jira/browse/HBASE-8163 > Project: HBase > Issue Type: New Feature > Components: regionserver >Reporter: chunhui shen >Assignee: chunhui shen > Attachments: hbase-8163v1.patch > > > *Background*： > When we use mslab,we will copy the keyvalue together in a structure called > *MemStoreLAB$Chunk*, therefore we could decrease the heap fragment. > *Problem*： > When one chunk is full, we will create a new chunk, and then the old chunk > will be reclaimed by JVM if no reference to it. > Mostly the chunk object would be promoted when doing Young GC, cause > increasing the cost of YGC > When a Chunk object has no reference? It should meet two following condition: > 1.Memstore which this chunk belongs to is flushed > 2.No scanner is opening on the memstore which this chunk belongs to > *Solution:* > 1.Create a chunk pool to manage the no-reference chunks, instead of being > reclaimed by JVM > 2.When a Chunk has no reference, put it back to the pool > 3.The pool has a max capacity, it will skip the chunks when achieve the max > size > 4.When we need new Chunk to store KeyValue, get it from the pool if exists, > else create new one by pool, so we could be able to reuse the old chunks > *Test results:* > Environment: > hbase-version:0.94 > -Xms4G -Xmx4G -Xmn2G > Row size=50 bytes, Value size=1024 bytes > 50 concurrent theads per client, insert 10,000,000 rows > Before: > Avg write request per second:12953 > After testing, final result of jstat -gcutil : > YGC YGCT FGC FGCT GCT > 747 36.503 48 2.492 38.995 > After: > Avg write request per second:14025 > After testing, final result of jstat -gcutil : > YGC YGCT FGC FGCT GCT > 711 20.344 4 0.284 20.628 > *Improvement: YGC 40+%; WPS 5%+* > review board : > https://reviews.apache.org/r/10056/ -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8163) MemStoreChunkPool: An improvement for JAVA GC when using MSLAB
[ https://issues.apache.org/jira/browse/HBASE-8163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13608684#comment-13608684 ] Ted Yu commented on HBASE-8163: --- Can we calculate the gap between min and max sizes of memstore and log it so that user gets to know the optimal value ? > MemStoreChunkPool: An improvement for JAVA GC when using MSLAB > -- > > Key: HBASE-8163 > URL: https://issues.apache.org/jira/browse/HBASE-8163 > Project: HBase > Issue Type: New Feature > Components: regionserver >Reporter: chunhui shen >Assignee: chunhui shen > Attachments: hbase-8163v1.patch > > > *Background*： > When we use mslab,we will copy the keyvalue together in a structure called > *MemStoreLAB$Chunk*, therefore we could decrease the heap fragment. > *Problem*： > When one chunk is full, we will create a new chunk, and then the old chunk > will be reclaimed by JVM if no reference to it. > Mostly the chunk object would be promoted when doing Young GC, cause > increasing the cost of YGC > When a Chunk object has no reference? It should meet two following condition: > 1.Memstore which this chunk belongs to is flushed > 2.No scanner is opening on the memstore which this chunk belongs to > *Solution:* > 1.Create a chunk pool to manage the no-reference chunks, instead of being > reclaimed by JVM > 2.When a Chunk has no reference, put it back to the pool > 3.The pool has a max capacity, it will skip the chunks when achieve the max > size > 4.When we need new Chunk to store KeyValue, get it from the pool if exists, > else create new one by pool, so we could be able to reuse the old chunks > *Test results:* > Environment: > hbase-version:0.94 > -Xms4G -Xmx4G -Xmn2G > Row size=50 bytes, Value size=1024 bytes > 50 concurrent theads per client, insert 10,000,000 rows > Before: > Avg write request per second:12953 > After testing, final result of jstat -gcutil : > YGC YGCT FGC FGCT GCT > 747 36.503 48 2.492 38.995 > After: > Avg write request per second:14025 > After testing, final result of jstat -gcutil : > YGC YGCT FGC FGCT GCT > 711 20.344 4 0.284 20.628 > *Improvement: YGC 40+%; WPS 5%+* > review board : > https://reviews.apache.org/r/10056/ -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8163) MemStoreChunkPool: An improvement for JAVA GC when using MSLAB
[ https://issues.apache.org/jira/browse/HBASE-8163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13608659#comment-13608659 ] chunhui shen commented on HBASE-8163: - bq.How big would size the pool? The pool size depends on the bounce of memstore size, the memstore size would be balanceable normally. Suppose the scenario as following step: 1.Current state:memstore size is 3GB, pool size is 500MB 2.Memstore size is decreased to 2.5GB because of flush, and then pool size is increased to 1GB 3.Memstore size is increased to 2.9GB because of writing, and then pool size is decreased to 600MB It means we could set pool size to the gap between the min size and max size of memstore for a running cluster. If the gap is very large, e.g. memstore would be flushed to empty when running, it is not appropriate to use pool. Hope I answered your question > MemStoreChunkPool: An improvement for JAVA GC when using MSLAB > -- > > Key: HBASE-8163 > URL: https://issues.apache.org/jira/browse/HBASE-8163 > Project: HBase > Issue Type: New Feature > Components: regionserver >Reporter: chunhui shen >Assignee: chunhui shen > Attachments: hbase-8163v1.patch > > > *Background*： > When we use mslab,we will copy the keyvalue together in a structure called > *MemStoreLAB$Chunk*, therefore we could decrease the heap fragment. > *Problem*： > When one chunk is full, we will create a new chunk, and then the old chunk > will be reclaimed by JVM if no reference to it. > Mostly the chunk object would be promoted when doing Young GC, cause > increasing the cost of YGC > When a Chunk object has no reference? It should meet two following condition: > 1.Memstore which this chunk belongs to is flushed > 2.No scanner is opening on the memstore which this chunk belongs to > *Solution:* > 1.Create a chunk pool to manage the no-reference chunks, instead of being > reclaimed by JVM > 2.When a Chunk has no reference, put it back to the pool > 3.The pool has a max capacity, it will skip the chunks when achieve the max > size > 4.When we need new Chunk to store KeyValue, get it from the pool if exists, > else create new one by pool, so we could be able to reuse the old chunks > *Test results:* > Environment: > hbase-version:0.94 > -Xms4G -Xmx4G -Xmn2G > Row size=50 bytes, Value size=1024 bytes > 50 concurrent theads per client, insert 10,000,000 rows > Before: > Avg write request per second:12953 > After testing, final result of jstat -gcutil : > YGC YGCT FGC FGCT GCT > 747 36.503 48 2.492 38.995 > After: > Avg write request per second:14025 > After testing, final result of jstat -gcutil : > YGC YGCT FGC FGCT GCT > 711 20.344 4 0.284 20.628 > *Improvement: YGC 40+%; WPS 5%+* > review board : > https://reviews.apache.org/r/10056/ -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8163) MemStoreChunkPool: An improvement for JAVA GC when using MSLAB
[ https://issues.apache.org/jira/browse/HBASE-8163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13608636#comment-13608636 ] Lars Hofhansl commented on HBASE-8163: -- How big would size the pool? If it is too big you'll waste space, if it is too small you'll just offset the GC for a bit. > MemStoreChunkPool: An improvement for JAVA GC when using MSLAB > -- > > Key: HBASE-8163 > URL: https://issues.apache.org/jira/browse/HBASE-8163 > Project: HBase > Issue Type: New Feature > Components: regionserver >Reporter: chunhui shen >Assignee: chunhui shen > Attachments: hbase-8163v1.patch > > > *Background*： > When we use mslab,we will copy the keyvalue together in a structure called > *MemStoreLAB$Chunk*, therefore we could decrease the heap fragment. > *Problem*： > When one chunk is full, we will create a new chunk, and then the old chunk > will be reclaimed by JVM if no reference to it. > Mostly the chunk object would be promoted when doing Young GC, cause > increasing the cost of YGC > When a Chunk object has no reference? It should meet two following condition: > 1.Memstore which this chunk belongs to is flushed > 2.No scanner is opening on the memstore which this chunk belongs to > *Solution:* > 1.Create a chunk pool to manage the no-reference chunks, instead of being > reclaimed by JVM > 2.When a Chunk has no reference, put it back to the pool > 3.The pool has a max capacity, it will skip the chunks when achieve the max > size > 4.When we need new Chunk to store KeyValue, get it from the pool if exists, > else create new one by pool, so we could be able to reuse the old chunks > *Test results:* > Environment: > hbase-version:0.94 > -Xms4G -Xmx4G -Xmn2G > Row size=50 bytes, Value size=1024 bytes > 50 concurrent theads per client, insert 10,000,000 rows > Before: > Avg write request per second:12953 > After testing, final result of jstat -gcutil : > YGC YGCT FGC FGCT GCT > 747 36.503 48 2.492 38.995 > After: > Avg write request per second:14025 > After testing, final result of jstat -gcutil : > YGC YGCT FGC FGCT GCT > 711 20.344 4 0.284 20.628 > *Improvement: YGC 40+%; WPS 5%+* > review board : > https://reviews.apache.org/r/10056/ -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8163) MemStoreChunkPool: An improvement for JAVA GC when using MSLAB
[ https://issues.apache.org/jira/browse/HBASE-8163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13608577#comment-13608577 ] Liang Xie commented on HBASE-8163: -- nice staff, cl~ > MemStoreChunkPool: An improvement for JAVA GC when using MSLAB > -- > > Key: HBASE-8163 > URL: https://issues.apache.org/jira/browse/HBASE-8163 > Project: HBase > Issue Type: New Feature > Components: regionserver >Reporter: chunhui shen >Assignee: chunhui shen > Attachments: hbase-8163v1.patch > > > *Background*： > When we use mslab,we will copy the keyvalue together in a structure called > *MemStoreLAB$Chunk*, therefore we could decrease the heap fragment. > *Problem*： > When one chunk is full, we will create a new chunk, and then the old chunk > will be reclaimed by JVM if no reference to it. > Mostly the chunk object would be promoted when doing Young GC, cause > increasing the cost of YGC > When a Chunk object has no reference? It should meet two following condition: > 1.Memstore which this chunk belongs to is flushed > 2.No scanner is opening on the memstore which this chunk belongs to > *Solution:* > 1.Create a chunk pool to manage the no-reference chunks, instead of being > reclaimed by JVM > 2.When a Chunk has no reference, put it back to the pool > 3.The pool has a max capacity, it will skip the chunks when achieve the max > size > 4.When we need new Chunk to store KeyValue, get it from the pool if exists, > else create new one by pool, so we could be able to reuse the old chunks > *Test results:* > Environment: > hbase-version:0.94 > -Xms4G -Xmx4G -Xmn2G > Row size=50 bytes, Value size=1024 bytes > 50 concurrent theads per client, insert 10,000,000 rows > Before: > Avg write request per second:12953 > After testing, final result of jstat -gcutil : > YGC YGCT FGC FGCT GCT > 747 36.503 48 2.492 38.995 > After: > Avg write request per second:14025 > After testing, final result of jstat -gcutil : > YGC YGCT FGC FGCT GCT > 711 20.344 4 0.284 20.628 > *Improvement: YGC 40+%; WPS 5%+* > review board : > https://reviews.apache.org/r/10056/ -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
