[
https://issues.apache.org/jira/browse/HBASE-19344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16270727#comment-16270727
]
Duo Zhang commented on HBASE-19344:
-----------------------------------
When debug HBASE-19371 I have also run several WALPE tests.
The command is
{noformat}
./bin/hbase org.apache.hadoop.hbase.wal.WALPerformanceEvaluation -path
/tmp/wals -verify -threads 25 -iterations 1000000 -keySize 50 -valueSize 100
-syncInterval 10
{noformat}
Run 3 times for each
FSHLog
{noformat}
[main] wal.WALPerformanceEvaluation: Summary: threads=25, iterations=1000000,
syncInterval=10 took 183.165s 136488.969ops/s
[main] wal.WALPerformanceEvaluation: Summary: threads=25, iterations=1000000,
syncInterval=10 took 180.782s 138288.109ops/s
[main] wal.WALPerformanceEvaluation: Summary: threads=25, iterations=1000000,
syncInterval=10 took 183.109s 136530.703ops/s
{noformat}
AsyncFSWAL
{noformat}
[main] wal.WALPerformanceEvaluation: Summary: threads=25, iterations=1000000,
syncInterval=10 took 176.312s 141794.094ops/s
[main] wal.WALPerformanceEvaluation: Summary: threads=25, iterations=1000000,
syncInterval=10 took 172.403s 145009.078ops/s
[main] wal.WALPerformanceEvaluation: Summary: threads=25, iterations=1000000,
syncInterval=10 took 171.578s 145706.328ops/s
{noformat}
AsyncFSWAL on branch HBASE-19344
{noformat}
[main] wal.WALPerformanceEvaluation: Summary: threads=25, iterations=1000000,
syncInterval=10 took 176.080s 141980.922ops/s
[main] wal.WALPerformanceEvaluation: Summary: threads=25, iterations=1000000,
syncInterval=10 took 173.030s 144483.609ops/s
[main] wal.WALPerformanceEvaluation: Summary: threads=25, iterations=1000000,
syncInterval=10 took 174.046s 143640.188ops/s
{noformat}
My result is similar with [~zghaobac] where AsyncFSWAL is a bit faster than
FSHLog, but there is almost no difference between the current AsyncFSWAL and
the one on branch HBASE-19344.
This may because we use hdd so our io speed is slower than you Alibaba guys who
use PCI-E ssd.
Thanks.
> improve asyncWAL by using Independent thread for netty #IO in
> FanOutOneBlockAsyncDFSOutput
> -------------------------------------------------------------------------------------------
>
> Key: HBASE-19344
> URL: https://issues.apache.org/jira/browse/HBASE-19344
> Project: HBase
> Issue Type: Sub-task
> Components: wal
> Affects Versions: 2.0.0-beta-1
> Reporter: Chance Li
> Assignee: Chance Li
> Fix For: 2.0.0
>
> Attachments: HBASE-19344-branch2.patch,
> HBASE-19344-branch2.patch.2.POC, wal-1-test-result.png,
> wal-8-test-result.png, ycsb_result_apache20_async_wal.pdf
>
>
> The logic now is that the netty #IO thread and asyncWal's thread are the same
> one.
> Improvement proposal:
> 1, Split into two.
> 2, All multiWal share the netty #IO thread pool.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
