[ https://issues.apache.org/jira/browse/HDFS-4183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13887091#comment-13887091 ]
Kihwal Lee commented on HDFS-4183: ---------------------------------- I think the overhead of block recoveries caused by massive lease expiration will drastically reduce after HDFS-5790. > Throttle block recovery > ----------------------- > > Key: HDFS-4183 > URL: https://issues.apache.org/jira/browse/HDFS-4183 > Project: Hadoop HDFS > Issue Type: Improvement > Components: namenode > Affects Versions: 0.23.4, 2.0.2-alpha > Reporter: Kihwal Lee > Assignee: Kihwal Lee > Priority: Critical > > When a large number of files are abandoned without closing, a storm of lease > expiration follows in about an hour (lease hard limit). For the last block of > each file, block recovery is initiated and when the datanode is done, it > calls commitBlockSynchronization() is called against namenode. A burst of > these calls can slow down namenode considerably. We need to throttle block > recovery and/or speed up the rate at which commitBlockSynchronization() is > served. -- This message was sent by Atlassian JIRA (v6.1.5#6160)