[
https://issues.apache.org/jira/browse/HDFS-7964?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14386638#comment-14386638
]
Yi Liu commented on HDFS-7964:
------------------------------
[~daryn], this is a cool and important improvement.
Besides the benefits you stated in the JIRA description, another benefit is: in
current HDFS, if force sync happens in logEdit for HDFS ops, it is protected by
FSN write lock, even we increase handler threads, HDFS read ops which don't
require logEdit are still blocked (require read lock), this improvement also
resolves this.
{quote}
editsBatchedInSync = txid - synctxid - 1;
{quote}
Should be {{txid - synctxid}}.
Don't need {{synchronized}} for AsyncEdit#logSyncNotify ?
add the configuration to hdfs-default.xml?
> Add support for async edit logging
> ----------------------------------
>
> Key: HDFS-7964
> URL: https://issues.apache.org/jira/browse/HDFS-7964
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: namenode
> Affects Versions: 2.0.2-alpha
> Reporter: Daryn Sharp
> Assignee: Daryn Sharp
> Attachments: HDFS-7964.patch
>
>
> Edit logging is a major source of contention within the NN. LogEdit is
> called within the namespace write log, while logSync is called outside of the
> lock to allow greater concurrency. The handler thread remains busy until
> logSync returns to provide the client with a durability guarantee for the
> response.
> Write heavy RPC load and/or slow IO causes handlers to stall in logSync.
> Although the write lock is not held, readers are limited/starved and the call
> queue fills. Combining an edit log thread with postponed RPC responses from
> HADOOP-10300 will provide the same durability guarantee but immediately free
> up the handlers.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
