[
https://issues.apache.org/jira/browse/HBASE-20188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16419557#comment-16419557
]
stack commented on HBASE-20188:
-------------------------------
[~eshcar]
# You run the code currently committed to branch-2.0 <= YES
# run on a single machine, namely no replication at the HDFS <= NO. My single
node runs on top of an HDFS 7 node cluster of machines..
# master and RS are on the same machine <= NO. Master and YCSB client are on
one machine, the RegionServer is another.
# Heap size is 8GB? <= NO 16G
# What is the operationcount in the experiments? (defined as
operationcount=${INSERT_COUNT}) <= I attached the script I've been using.
bq. I would like to run the tests until they are completed and not to cap them
at 20minutes if that's ok.
I can see the point of doing this at load time. Less so for workloads. I just
made a change so the loading finishes loading all 100M keys (my recordcount)
and then runs workloads for 20minutes. Will update my report with these new
runs where there will be no failed reads (after [~ram_krish]'s comment).
ASYNC_WAL does not work. SYNC_WAL is default. My configuration is default
except for the site specifics like hostnames.
[~ram_krish] On your numbers, they generally agree w/ attached report; i.e.
default (in-memory-compaction) is better than no in-memory-compaction when
writing (but worse than 1.2.7 and when in-memory-compaction and
in-memory-compaction ups load/GC). Am doing reruns so eliminate variance
because of requests for non-existent keys.
[~anastas]
bq. So we should at least consider to take MSLAB out from default and not
CompactingMemStore.
While CMS is default GC, we can't turn off MSLAB (See HBASE-3455 "Add
memstore-local allocation buffers to combat heap fragmentation in the region
server."). When G1GC, its possible we could do w/o MSLAB but would need to do
the long-running tests described in HBASE-3455. MSLAB on/off is a little
orthogonal. I did it just because [~eshcar] suggested she did it in her test
runs.
bq. There are not enough keys written and for read-only workload we have 3-4
times more reads that didn't found their keys then reads that found.
Fixing this at the moment.
bq. I believe we can allow ourselves to run the experiments 3-4 days more and
to come to the conclusion which is best for the HBase product.
That'd be great. I'll do a run w/ your suggestion of a run with one Segment
only in the pipeline. Will report back.
Thanks.
> [TESTING] Performance
> ---------------------
>
> Key: HBASE-20188
> URL: https://issues.apache.org/jira/browse/HBASE-20188
> Project: HBase
> Issue Type: Umbrella
> Components: Performance
> Reporter: stack
> Priority: Blocker
> Fix For: 2.0.0
>
> Attachments: ITBLL2.5B_1.2.7vs2.0.0_cpu.png,
> ITBLL2.5B_1.2.7vs2.0.0_gctime.png, ITBLL2.5B_1.2.7vs2.0.0_iops.png,
> ITBLL2.5B_1.2.7vs2.0.0_load.png, ITBLL2.5B_1.2.7vs2.0.0_memheap.png,
> ITBLL2.5B_1.2.7vs2.0.0_memstore.png, ITBLL2.5B_1.2.7vs2.0.0_ops.png,
> ITBLL2.5B_1.2.7vs2.0.0_ops_NOT_summing_regions.png, YCSB_CPU.png,
> YCSB_GC_TIME.png, YCSB_IN_MEMORY_COMPACTION=NONE.ops.png, YCSB_MEMSTORE.png,
> YCSB_OPs.png, YCSB_in-memory-compaction=NONE.ops.png, YCSB_load.png,
> flamegraph-1072.1.svg, flamegraph-1072.2.svg, tree.txt
>
>
> How does 2.0.0 compare to old versions? Is it faster, slower? There is rumor
> that it is much slower, that the problem is the asyncwal writing. Does
> in-memory compaction slow us down or speed us up? What happens when you
> enable offheaping?
> Keep notes here in this umbrella issue. Need to be able to say something
> about perf when 2.0.0 ships.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
