[
https://issues.apache.org/jira/browse/HBASE-5699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated HBASE-5699:
-------------------------------
Attachment:
HBASE-5699_write_iops_multiwal-6_10,50,120,190,260,330,400_threads.tiff
HBASE-5699_write_iops_multiwal-4_10,50,120,190,260,330,400_threads.tiff
HBASE-5699_write_iops_multiwal-2_10,50,120,190,260,330,400_threads.tiff
HBASE-5699_write_iops_multiwal-1_1_to_200_threads.tiff
HBASE-5699_write_iops_upstream_1_to_200_threads.tiff
results-hbase5699-upstream.txt.bz2
results-hbase5699-wals-1.txt.bz2
results-updated-hbase5699-wals-2.txt.bz2
results-updated-hbase5699-wals-4.txt.bz2
results-updated-hbase5699-wals-6.txt.bz2
hbase-5699_total_throughput_sync_heavy.txt
hbase-5699_multiwal_400-threads_stats_sync_heavy.txt
Attaching results from some initial testing using WALPerformanceEvaluation with
a sync-heavy workload.
If anyone has other measurements they'd like to see, or a different workload
expressed (perhaps to look at limits for pushing bytes given larger edits with
fewer syncs), please let me know.
h5. Overview
command used was
{code}
bin/hbase org.apache.hadoop.hbase.wal.WALPerformanceEvaluation -threads
${threads} -regions $(((threads+1)/2)) -roll 100000 -iterations 1000000 -verify
{code}
with # threads varied and # regions at ceil(threads/2). default is
sync-per-write.
Test rig is a physical cluster with 4 data nodes and a total of 20 data disks
(5 per node). Test client was run on a separate non-loaded host. HDFS
2.5.0-cdh5.2.0
* {{.bz2}} files are the complete logs from the described group of runs.
* "upstream" and "wals-1" data is from prior to HBASE-12655, so run metrics
other than the final benchmark results aren't comparable to later.
* There's an image showing total write iops across the cluster for each of the
sets of test runs.
* hbase-5699_total_throughput_sync_heavy.txt has the final benchmark log from
each of the runs, so you can quickly look across successive runs.
* hbase-5699_multiwal_400-threads_stats_sync_heavy.txt has the run metrics from
just the final test of the multiwal options
h5. upstream vs multiwal-1
If you look at the two charts
"HBASE-5699_write_iops_upstream_1_to_200_threads.tiff" and
"HBASE-5699_write_iops_multiwal-1_1_to_200_threads.tiff", they behave roughly
the same modulo noise. (the only difference in the two happens during region
set up, which shouldn't be reflected here.)
They also level off on ability to push more through the pipeline at near the
limit for iops given the 3 disks in the single pipeline.
h5. increasing number of pipelines
If you look at each of the
HBASE-5699_write_iops_multiwal-X_10,50,120,190,260,330,400_threads.tiff charts,
as we ramp up the number of writers we manage to push more overall activity
through the cluster.
It's not a linear gain because splitting out the pipelines means that we do
more overall syncs since fewer of them get obviated by our sync grouping.
In this test, expanding from 2 to 4 or 6 pipelines didn't provide much benefit
because at up to 400 concurrent sync-heavy writers we just get to maxing out
the number of iops that can be done with 2 pipelines.
> Run with > 1 WAL in HRegionServer
> ---------------------------------
>
> Key: HBASE-5699
> URL: https://issues.apache.org/jira/browse/HBASE-5699
> Project: HBase
> Issue Type: Improvement
> Components: Performance, wal
> Reporter: binlijin
> Assignee: Sean Busbey
> Priority: Critical
> Attachments: HBASE-5699.3.patch.txt,
> HBASE-5699_write_iops_multiwal-1_1_to_200_threads.tiff,
> HBASE-5699_write_iops_multiwal-2_10,50,120,190,260,330,400_threads.tiff,
> HBASE-5699_write_iops_multiwal-4_10,50,120,190,260,330,400_threads.tiff,
> HBASE-5699_write_iops_multiwal-6_10,50,120,190,260,330,400_threads.tiff,
> HBASE-5699_write_iops_upstream_1_to_200_threads.tiff, PerfHbase.txt,
> hbase-5699_multiwal_400-threads_stats_sync_heavy.txt,
> hbase-5699_total_throughput_sync_heavy.txt,
> results-hbase5699-upstream.txt.bz2, results-hbase5699-wals-1.txt.bz2,
> results-updated-hbase5699-wals-2.txt.bz2,
> results-updated-hbase5699-wals-4.txt.bz2,
> results-updated-hbase5699-wals-6.txt.bz2
>
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
