[ https://issues.apache.org/jira/browse/HDFS-17366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17852294#comment-17852294 ]
ZanderXu commented on HDFS-17366: --------------------------------- Thanks [~zhengchenyu] for your watching. {quote}Do we need to acquire the inode file lock for block every time we process a block? If so, will this make the block report slower? {quote} Yes, BR/IBR should acquire the INodeFileLock to process a block. About the performance, BR/IBR may take more time, but if we take the waiting time into account, the total time of BR/IBR may reduced, especially for the high load namenode. {quote}For deleting the block lock, we first obtain the inode file lock, and then obtain the dn lock {quote} Block deleting logic just acquire the INodeFileLock, will not acquire the DN Lock. HDFS-13671 introduced relationship between blocks, it caused that BR/IBR acquire multiple INodeFileLocks when processing one block. So the HDFS-13671 should be reverted, and we can find a new solution to fix this performance problem. > NameNode Fine-Grained Locking via Namespace Tree > ------------------------------------------------ > > Key: HDFS-17366 > URL: https://issues.apache.org/jira/browse/HDFS-17366 > Project: Hadoop HDFS > Issue Type: Improvement > Components: hdfs, namenode > Reporter: ZanderXu > Assignee: ZanderXu > Priority: Major > Attachments: NameNode Fine-Grained Locking Based On Directory Tree.pdf > > > As we all known, the write performance of NameNode is limited by the global > lock. We target to enable fine-grained locking based on the Namespace tree to > improve the performance of NameNode write operations. > There are multiple motivations for creating this ticket: > * We have implemented this fine-grained locking and gained nearly 7x > performance improvements in our prod environment > * Other companies made similar improvements based on their internal branch. > Internal branches are quite different from the community, so few feedback and > discussions in the community. > * The topic of fine-grained locking has been discussed for a very long time, > but still without any results. > > We implemented this fine-gained locking based on the namespace tree to > maximize the number of concurrency for disjoint or independent operations. -- This message was sent by Atlassian Jira (v8.20.10#820010) --------------------------------------------------------------------- To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org