[
https://issues.apache.org/jira/browse/HBASE-20188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16409795#comment-16409795
]
stack edited comment on HBASE-20188 at 3/22/18 4:16 PM:
--------------------------------------------------------
I set in-memory compaction to NONE
({{hbase.hregion.compacting.memstore.type}}). This helped some w/ the reads but
we are still far down from hbase1 with workloadb, 95% reads. Looking in log I
can't tell that in-memory compaction if off. I still see logging.
{code}
...
260444 2018-03-21 22:46:35,800 INFO
[StoreOpener-e2e92d8b2112d5103bbdba11c6164874-1]
regionserver.CompactingMemStore: Setting in-memory flush size threshold to
12.80 MB and immutable segments index to type=CHUNK_MAP
260445 2018-03-21 22:46:35,800 DEBUG
[StoreOpener-e2e92d8b2112d5103bbdba11c6164874-1] regionserver.HStore: Memstore
type=org.apache.hadoop.hbase.regionserver.CompactingMemStore
260446 2018-03-21 22:46:35,800 INFO
[StoreOpener-730bdcdd82b4c5ebff04d371061e787f-1]
regionserver.CompactingMemStore: Setting in-memory flush size threshold to
12.80 MB and immutable segments index to type=CHUNK_MAP
260447 2018-03-21 22:46:35,801 DEBUG
[StoreOpener-730bdcdd82b4c5ebff04d371061e787f-1] regionserver.HStore: Memstore
type=org.apache.hadoop.hbase.regionserver.CompactingMemStore
...
349621 2018-03-22 09:04:40,098 DEBUG
[regionserver/ve0528:16020-MemStoreChunkPool Statistics]
regionserver.ChunkCreator: data stats (chunk size=2097152): current pool
size=92, created chunk count=458, reused chunk count=19139, reuseRatio=97.66%
349622 2018-03-22 09:04:40,099 DEBUG
[regionserver/ve0528:16020-MemStoreChunkPool Statistics]
regionserver.ChunkCreator: index stats (chunk size=209715): current pool
size=0, created chunk count=0, reused chunk count=0, reuseRatio=0
....
{code}
Let me make the signal clean around what in-memory compaction is doing.
So, in-memory compaction by default helps a little when pure loading but when
reads in the mix, writes are slightly less and reads are pulled down some. Let
me post a graph (See YCSB...NONE.ops.png). We are missing a bunch of read perf
in hbase2.
was (Author: stack):
I set in-memory compaction to NONE
({{hbase.hregion.compacting.memstore.type}}). This helped some w/ the reads but
we are still far down from hbase1 with workloadb, 95% reads. Looking in log I
can't tell that in-memory compaction if off. I still see logging.
{code}
...
260444 2018-03-21 22:46:35,800 INFO
[StoreOpener-e2e92d8b2112d5103bbdba11c6164874-1]
regionserver.CompactingMemStore: Setting in-memory flush size threshold to
12.80 MB and immutable segments index to type=CHUNK_MAP
260445 2018-03-21 22:46:35,800 DEBUG
[StoreOpener-e2e92d8b2112d5103bbdba11c6164874-1] regionserver.HStore: Memstore
type=org.apache.hadoop.hbase.regionserver.CompactingMemStore
260446 2018-03-21 22:46:35,800 INFO
[StoreOpener-730bdcdd82b4c5ebff04d371061e787f-1]
regionserver.CompactingMemStore: Setting in-memory flush size threshold to
12.80 MB and immutable segments index to type=CHUNK_MAP
260447 2018-03-21 22:46:35,801 DEBUG
[StoreOpener-730bdcdd82b4c5ebff04d371061e787f-1] regionserver.HStore: Memstore
type=org.apache.hadoop.hbase.regionserver.CompactingMemStore
...
349621 2018-03-22 09:04:40,098 DEBUG
[regionserver/ve0528:16020-MemStoreChunkPool Statistics]
regionserver.ChunkCreator: data stats (chunk size=2097152): current pool
size=92, created chunk count=458, reused chunk count=19139, reuseRatio=97.66%
349622 2018-03-22 09:04:40,099 DEBUG
[regionserver/ve0528:16020-MemStoreChunkPool Statistics]
regionserver.ChunkCreator: index stats (chunk size=209715): current pool
size=0, created chunk count=0, reused chunk count=0, reuseRatio=0
....
{code}
Let me make the signal clean around what in-memory compaction is doing.
So, in-memory compaction by default helps a little when pure loading but when
reads in the mix, writes are slightly less and reads are pulled down some. Let
me post a graph. We are missing a bunch of read perf in hbase2.
> [TESTING] Performance
> ---------------------
>
> Key: HBASE-20188
> URL: https://issues.apache.org/jira/browse/HBASE-20188
> Project: HBase
> Issue Type: Umbrella
> Components: Performance
> Reporter: stack
> Priority: Critical
> Fix For: 2.0.0
>
> Attachments: ITBLL2.5B_1.2.7vs2.0.0_cpu.png,
> ITBLL2.5B_1.2.7vs2.0.0_gctime.png, ITBLL2.5B_1.2.7vs2.0.0_iops.png,
> ITBLL2.5B_1.2.7vs2.0.0_load.png, ITBLL2.5B_1.2.7vs2.0.0_memheap.png,
> ITBLL2.5B_1.2.7vs2.0.0_memstore.png, ITBLL2.5B_1.2.7vs2.0.0_ops.png,
> ITBLL2.5B_1.2.7vs2.0.0_ops_NOT_summing_regions.png, YCSB_CPU.png,
> YCSB_GC_TIME.png, YCSB_MEMSTORE.png, YCSB_OPs.png,
> YCSB_in-memory-compaction=NONE.ops.png, YCSB_load.png, flamegraph-1072.1.svg,
> flamegraph-1072.2.svg, tree.txt
>
>
> How does 2.0.0 compare to old versions? Is it faster, slower? There is rumor
> that it is much slower, that the problem is the asyncwal writing. Does
> in-memory compaction slow us down or speed us up? What happens when you
> enable offheaping?
> Keep notes here in this umbrella issue. Need to be able to say something
> about perf when 2.0.0 ships.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
