[
https://issues.apache.org/jira/browse/HDFS-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16922886#comment-16922886
]
Konstantin Shvachko commented on HDFS-14703:
--------------------------------------------
Hey [~arp] , thanks for reviewing.
??Do atomic rename and snapshots still work as before with these changes???
Yes, the intention is to support atomic renames, snapshots, and all features.
For general case renames, which include moving a large directory ubder another
parent, could require locking multiple partitions. In the worst case it could
be equivalent (in performance) to holding a global lock, because all partitions
will be locked. But more frequent small operations will be faster.
We [did discuss snapshots in this
regard|https://docs.google.com/document/d/1jXM5Ujvf-zhcyw_5kiQVx6g-HeKe-YGnFS_1-qFXomI/edit#].
Doesn't look impossible, needs some thinking around copy-on-write cases.
??Did you measure write throughput improvement with
dfs.namenode.edits.asynclogging???
Yes, I used the default value {{dfs.namenode.edits.asynclogging = true}}
> NameNode Fine-Grained Locking via Metadata Partitioning
> -------------------------------------------------------
>
> Key: HDFS-14703
> URL: https://issues.apache.org/jira/browse/HDFS-14703
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: hdfs, namenode
> Reporter: Konstantin Shvachko
> Priority: Major
> Attachments: 001-partitioned-inodeMap-POC.tar.gz, NameNode
> Fine-Grained Locking.pdf
>
>
> We target to enable fine-grained locking by splitting the in-memory namespace
> into multiple partitions each having a separate lock. Intended to improve
> performance of NameNode write operations.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
