[
https://issues.apache.org/jira/browse/HDFS-16524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17575888#comment-17575888
]
ASF GitHub Bot commented on HDFS-16524:
---------------------------------------
ted12138 commented on code in PR #4139:
URL: https://github.com/apache/hadoop/pull/4139#discussion_r938898456
##########
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java:
##########
@@ -3410,6 +3410,33 @@ FSPermissionChecker getPermissionChecker()
return dir.getPermissionChecker();
}
+ /**
+ * If blockDeletionAsync enables, blocks will be deleted asynchronously.
+ * If not, incrementally remove the blocks from blockManager
+ * Writelock is dropped and reacquired every BLOCK_DELETION_INCREMENT to
+ * ensure that other waiters on the lock can get in. See HDFS-2938
+ *
+ * @param toDeleteList
+ * a list of blocks that need to be removed from blocksMap
+ */
+ void removeBlocks(List<BlockInfo> toDeleteList) {
+ if (this.blockDeletionAsync) {
+ blockManager.addBLocksToMarkedDeleteQueue(toDeleteList);
+ } else {
+ Iterator<BlockInfo> iter = toDeleteList.iterator();
+ while (iter.hasNext()) {
+ writeLock();
+ try {
+ for (int i = 0; i < blockDeletionIncrement && iter.hasNext(); i++) {
Review Comment:
About "blockDeletionIncrement", in my opinion, why we removed it in
HDFS-16554 is that we decided to use delete async to replace the previous
method, so the "blockDeletionIncrement" won't be used any more. But when we add
a switch to control blocks deletion async, we need to save the previous block
deletion method, so we still need it.
> Add configuration to control blocks deletion asynchronous or synchronous
> ------------------------------------------------------------------------
>
> Key: HDFS-16524
> URL: https://issues.apache.org/jira/browse/HDFS-16524
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: namenode
> Reporter: leo sun
> Priority: Major
> Labels: pull-request-available
> Time Spent: 50m
> Remaining Estimate: 0h
>
> When users delete files frequently, deletion is slower than its enqueues. The
> pending deletions blocks will keep increasing.
> Under this circumstance, deletion synchronous is better.
> So I will add configuration to switch deletion asynchronous or synchronous.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
