[
https://issues.apache.org/jira/browse/HBASE-16890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16281039#comment-16281039
]
stack edited comment on HBASE-16890 at 12/6/17 10:38 PM:
---------------------------------------------------------
I ran a compare using WALPE. Setup was multi-DNs. asyncfs does better until we
get to 100 concurrent threads; then old WAL does better (Slightly more ops per
second, less clock-cycles and branch-misses, etc.). Latest branch-2.
+1 on this is good enough to enable by default IMO. We can work on improving
perf in-context.
||Threads||Attribute||FSHLog||AsyncFSWAL||
|1|Duration|144|58|
|1|Ops/s|6956|17032|
|1|Context-Switches|890k|490k|
|1|page-faults|1.7M|1.5M|
|3|Duration|151|74|
|3|Ops/s|19833|40423|
|3|Context-Switches|1.6M|.7M|
|3|page-faults|3.3M|3M|
|3|Duration|285|220|
|3|Ops/s|87663|113233|
|3|Context-Switches|10M|5M|
|3|page-faults|11M|13M|
|100|Duration|736|761|
|100|Ops/s|135865|131304|
|100|Context-Switches|33M|22.5M|
|100|page-faults|36.7M|36.4M|
Here is how I ran test:
{code}
export HBASE_HOME=/home/stack/hbase
for i in 1 3 5 25 100; do
for j in 1; do
log="log-really-async${i}.${j}.txt"
echo "$log"
logfile="/home/stack/logs/$log"
HBASE_CLASSPATH="${HBASE_HOME}/lib/hbase-server-2.0.0-beta-1.SNAPSHOT-tests.jar"
perf stat ${HBASE_HOME}/bin/hbase --config /home/stack/conf_hbase
org.apache.hadoop.hbase.wal.WALPerformanceEvaluation -path /user/stack/logs/
-verify -threads "${i}" -iterations 1000000 -keySize 50 -valueSize 100
-syncInterval 10 &> "${logfile}"
grep "Summary: " "${logfile}" &> /dev/null
done;
done
{code}
was (Author: stack):
I ran a compare using WALPE. Setup was multi-DNs. asyncfs does better until we
hit 100 threads. Latest branch-2. +1 on this is good enough to enable to by
default. We can work on improving perf in-context.
||Threads||Attribute||FSHLog||AsyncFSWAL||
|1|Duration|144|58|
|1|Ops/s|6956|17032|
|1|Context-Switches|890k|490k|
|1|page-faults|1.7M|1.5M|
|3|Duration|151|74|
|3|Ops/s|19833|40423|
|3|Context-Switches|1.6M|.7M|
|3|page-faults|3.3M|3M|
|3|Duration|285|220|
|3|Ops/s|87663|113233|
|3|Context-Switches|10M|5M|
|3|page-faults|11M|13M|
|100|Duration|736|761|
|100|Ops/s|135865|131304|
|100|Context-Switches|33M|22.5M|
|100|page-faults|36.7M|36.4M|
Here is how I ran test:
{code}
export HBASE_HOME=/home/stack/hbase
for i in 1 3 5 25 100; do
for j in 1; do
log="log-really-async${i}.${j}.txt"
echo "$log"
logfile="/home/stack/logs/$log"
HBASE_CLASSPATH="${HBASE_HOME}/lib/hbase-server-2.0.0-beta-1.SNAPSHOT-tests.jar"
perf stat ${HBASE_HOME}/bin/hbase --config /home/stack/conf_hbase
org.apache.hadoop.hbase.wal.WALPerformanceEvaluation -path /user/stack/logs/
-verify -threads "${i}" -iterations 1000000 -keySize 50 -valueSize 100
-syncInterval 10 &> "${logfile}"
grep "Summary: " "${logfile}" &> /dev/null
done;
done
{code}
> Analyze the performance of AsyncWAL and fix the same
> ----------------------------------------------------
>
> Key: HBASE-16890
> URL: https://issues.apache.org/jira/browse/HBASE-16890
> Project: HBase
> Issue Type: Sub-task
> Components: wal
> Affects Versions: 2.0.0
> Reporter: ramkrishna.s.vasudevan
> Assignee: ramkrishna.s.vasudevan
> Priority: Blocker
> Fix For: 2.0.0-beta-1
>
> Attachments: AsyncWAL_disruptor.patch, AsyncWAL_disruptor_1
> (2).patch, AsyncWAL_disruptor_3.patch, AsyncWAL_disruptor_3.patch,
> AsyncWAL_disruptor_4.patch, AsyncWAL_disruptor_6.patch,
> HBASE-16890-rc-v2.patch, HBASE-16890-rc-v3.patch,
> HBASE-16890-remove-contention-v1.patch, HBASE-16890-remove-contention.patch,
> Screen Shot 2016-10-25 at 7.34.47 PM.png, Screen Shot 2016-10-25 at 7.39.07
> PM.png, Screen Shot 2016-10-25 at 7.39.48 PM.png, Screen Shot 2016-11-04 at
> 5.21.27 PM.png, Screen Shot 2016-11-04 at 5.30.18 PM.png, async.svg,
> classic.svg, contention.png, contention_defaultWAL.png,
> ycsb_FSHlog.vs.Async.png
>
>
> Tests reveal that AsyncWAL under load in single node cluster performs slower
> than the Default WAL. This task is to analyze and see if we could fix it.
> See some discussions in the tail of JIRA HBASE-15536.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
