[
https://issues.apache.org/jira/browse/HBASE-20188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16441945#comment-16441945
]
stack edited comment on HBASE-20188 at 4/18/18 6:41 AM:
--------------------------------------------------------
Report exploring the [~anoop.hbase] postulate above that 'flushes are taking
longer' by comparing flush histories of 1.2.7 vs 2.0.0 with client batching
enabled (2MB):
https://docs.google.com/spreadsheets/d/1sihTxb4aCplR3Rr_GGXkPlwhMIm-CbB9j_5339AS0Zc/edit#gid=1016758826
Findings are that 1.2.7 flushes twice as often during the YCSB load phase (400
vs 200 times) carrying less in memory riding at around 128M spiking up on
occasion with 2.0.0 carrying ~256M on average.
I suppose this is what we'd expect when IMC is enabled.
2.0.0 does ride well above the 128MB threshold. If the CSLM is correspondingly
large, that could be part explanation for why writes are slower in this mode in
2.0.0 -- almost 25% slower -- with reads about 3 or 4% slower when data is
batched in by client as here. Perhaps the new suggested defaults for IMC will
help (will try them).
My YCSB graphs as it happens are too coarse; they miss most flushes so don't
reveal what is really going on here. The attached report was made from parsing
the log. This load phase was done with client-side buffering enabled so batches
were coming in in 2MB lots instead of the default 1.7k which is what all
previous YCSB runs in this JIRA were doing.
was (Author: stack):
Report exploring the [~anoop.hbase] postulate above that 'flushes are taking
longer' by comparing flush histories of 1.2.7 vs 2.0.0 with client batching
enabled (2MB):
https://docs.google.com/spreadsheets/d/1sihTxb4aCplR3Rr_GGXkPlwhMIm-CbB9j_5339AS0Zc/edit#gid=1016758826
Findings are that 1.2.7 flushes twice as often during the YCSB load phase (400
vs 200 times) carrying less in memory riding at around 128M spiking up on
occasion with 2.0.0 carrying ~256M on average.
I suppose this is what we'd expect when IMC is enabled.
2.0.0 does ride well above the 128MB threshold. If the CSLM is correspondingly
large, that could be part explanation for why writes are slower in this mode in
2.0.0 -- almost 25% slower ... with reads about 3 or 4% slower when data is
batched in by client as here.
My YCSB graphs as it happens are too coarse; they miss most flushes so don't
reveal what is really going on here. The attached report was made from parsing
the log. This load phase was done with client-side buffering enabled so batches
were coming in in 2MB lots instead of the default 1.7k which is what all
previous YCSB runs in this JIRA were doing.
> [TESTING] Performance
> ---------------------
>
> Key: HBASE-20188
> URL: https://issues.apache.org/jira/browse/HBASE-20188
> Project: HBase
> Issue Type: Umbrella
> Components: Performance
> Reporter: stack
> Assignee: stack
> Priority: Blocker
> Fix For: 2.0.0
>
> Attachments: CAM-CONFIG-V01.patch, HBASE-20188-xac.sh,
> HBASE-20188.sh, HBase 2.0 performance evaluation - 8GB(1).pdf, HBase 2.0
> performance evaluation - 8GB.pdf, HBase 2.0 performance evaluation - Basic vs
> None_ system settings.pdf, ITBLL2.5B_1.2.7vs2.0.0_cpu.png,
> ITBLL2.5B_1.2.7vs2.0.0_gctime.png, ITBLL2.5B_1.2.7vs2.0.0_iops.png,
> ITBLL2.5B_1.2.7vs2.0.0_load.png, ITBLL2.5B_1.2.7vs2.0.0_memheap.png,
> ITBLL2.5B_1.2.7vs2.0.0_memstore.png, ITBLL2.5B_1.2.7vs2.0.0_ops.png,
> ITBLL2.5B_1.2.7vs2.0.0_ops_NOT_summing_regions.png, YCSB_CPU.png,
> YCSB_GC_TIME.png, YCSB_IN_MEMORY_COMPACTION=NONE.ops.png, YCSB_MEMSTORE.png,
> YCSB_OPs.png, YCSB_in-memory-compaction=NONE.ops.png, YCSB_load.png,
> flamegraph-1072.1.svg, flamegraph-1072.2.svg, hbase-env.sh, hbase-site.xml,
> hbase-site.xml, hits.png, lock.127.workloadc.20180402T200918Z.svg,
> lock.2.memsize2.c.20180403T160257Z.svg, perregion.png, run_ycsb.sh,
> total.png, tree.txt, workloadx, workloadx
>
>
> How does 2.0.0 compare to old versions? Is it faster, slower? There is rumor
> that it is much slower, that the problem is the asyncwal writing. Does
> in-memory compaction slow us down or speed us up? What happens when you
> enable offheaping?
> Keep notes here in this umbrella issue. Need to be able to say something
> about perf when 2.0.0 ships.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
