[
https://issues.apache.org/jira/browse/HBASE-19344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16265720#comment-16265720
]
Duo Zhang commented on HBASE-19344:
-----------------------------------
See the comments of syncFailed method in AsyncFSWAL
{code}
private void syncFailed(Throwable error) {
LOG.warn("sync failed", error);
// Here we depends on the implementation of FanOutOneBlockAsyncDFSOutput
and netty.
// When error occur, FanOutOneBlockAsyncDFSOutput will fail all pending
flush requests. It
// is execute inside EventLoop. And in DefaultPromise in netty, it will
notifyListener
// directly if it is already in the EventLoop thread. And in the listener
method, it will
// call us. So here we know that all failed flush request will call us
continuously, and
// before the last one finish, no other task can be executed in EventLoop.
So here we are
// safe to use writerBroken as a guard.
// Do not forget to revisit this if we change the implementation of
// FanOutOneBlockAsyncDFSOutput!
consumeLock.lock();
try {
if (writerBroken) {
return;
}
writerBroken = true;
if (waitingRoll) {
readyForRolling = true;
readyForRollingCond.signalAll();
}
} finally {
consumeLock.unlock();
}
for (Iterator<FSWALEntry> iter = unackedAppends.descendingIterator();
iter.hasNext();) {
toWriteAppends.addFirst(iter.next());
}
highestUnsyncedTxid = highestSyncedTxid.get();
// request a roll.
requestLogRoll();
}
{code}
It is not straight-forward to use different event loop for AsyncDFSOutput. But
anyway, I think this is a way to increase performance as we can shard the
computations from one thread to several threads.
> improve asyncWAL by using Independent thread for netty #IO in
> FanOutOneBlockAsyncDFSOutput
> -------------------------------------------------------------------------------------------
>
> Key: HBASE-19344
> URL: https://issues.apache.org/jira/browse/HBASE-19344
> Project: HBase
> Issue Type: Improvement
> Components: wal
> Affects Versions: 2.0.0-beta-1
> Reporter: Chance Li
> Assignee: Chance Li
> Fix For: 2.0.0
>
> Attachments: HBASE-19344-branch2.patch, wal-1-test-result.png,
> wal-8-test-result.png
>
>
> The logic now is that the netty #IO thread and asyncWal's thread are the same
> one.
> Improvement proposal:
> 1, Split into two.
> 2, All multiWal share the netty #IO thread pool.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
