[
https://issues.apache.org/jira/browse/HBASE-16890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16259663#comment-16259663
]
Chance Li commented on HBASE-16890:
-----------------------------------
It's very nice code. :) I will work on this.
And I read the code roughly. I'm sorry I haven't read all the discussions. pls
correct me if I'm wrong.
1. Based on some previous tests, Evenloop execute sync(especially
finishSyncLowerThanTxid) by single thread is not enough in #consume.
2. We think #consume should be as non blocking as possible (except append).
Otherwise it may cause the updates of #highestProcessedAppendTxid not fast
enough (not sure it's related to HBASE-17049)
And Our expectations:
1.SyncFutrue should to be handled ASAP
2.We hope that wal is real fast-failover, such as in 5s, instead of 60s
(FanOutOneBlockAsyncDFSOutput#IdleStateHandler on dfs.client.socket-timeout),
which can effectively improve the P999 (long tail). Sometimes some sync hung
near 60s in our product.
3.Append is built on the new DfsClient pipeline model (FanOut).
In the past, we tried to resolve 1 and 2 base on FSHLog (we didn't think async
is absolutely necessary, low RT is more important) , some ideas were more like
this.
> Analyze the performance of AsyncWAL and fix the same
> ----------------------------------------------------
>
> Key: HBASE-16890
> URL: https://issues.apache.org/jira/browse/HBASE-16890
> Project: HBase
> Issue Type: Sub-task
> Components: wal
> Affects Versions: 2.0.0
> Reporter: ramkrishna.s.vasudevan
> Assignee: ramkrishna.s.vasudevan
> Priority: Blocker
> Fix For: 2.0.0-beta-1
>
> Attachments: AsyncWAL_disruptor.patch, AsyncWAL_disruptor_1
> (2).patch, AsyncWAL_disruptor_3.patch, AsyncWAL_disruptor_3.patch,
> AsyncWAL_disruptor_4.patch, AsyncWAL_disruptor_6.patch,
> HBASE-16890-rc-v2.patch, HBASE-16890-rc-v3.patch,
> HBASE-16890-remove-contention-v1.patch, HBASE-16890-remove-contention.patch,
> Screen Shot 2016-10-25 at 7.34.47 PM.png, Screen Shot 2016-10-25 at 7.39.07
> PM.png, Screen Shot 2016-10-25 at 7.39.48 PM.png, Screen Shot 2016-11-04 at
> 5.21.27 PM.png, Screen Shot 2016-11-04 at 5.30.18 PM.png, async.svg,
> classic.svg, contention.png, contention_defaultWAL.png
>
>
> Tests reveal that AsyncWAL under load in single node cluster performs slower
> than the Default WAL. This task is to analyze and see if we could fix it.
> See some discussions in the tail of JIRA HBASE-15536.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
