[
https://issues.apache.org/jira/browse/HDFS-16524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17864542#comment-17864542
]
ASF GitHub Bot commented on HDFS-16524:
---------------------------------------
zeekling commented on code in PR #4139:
URL: https://github.com/apache/hadoop/pull/4139#discussion_r1639098795
##########
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java:
##########
@@ -3403,6 +3407,33 @@ FSPermissionChecker getPermissionChecker()
return dir.getPermissionChecker();
}
+ /**
+ * If blockDeletionAsync enables, blocks will be deleted asynchronously.
Review Comment:
use dfs.namenode.block.deletion.async not blockDeletionAsync
##########
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java:
##########
@@ -3403,6 +3407,33 @@ FSPermissionChecker getPermissionChecker()
return dir.getPermissionChecker();
}
+ /**
+ * If blockDeletionAsync enables, blocks will be deleted asynchronously.
+ * If not, incrementally remove the blocks from blockManager
+ * Writelock is dropped and reacquired every BLOCK_DELETION_INCREMENT to
+ * ensure that other waiters on the lock can get in. See HDFS-2938
+ *
+ * @param toDeleteList
+ * a list of blocks that need to be removed from blocksMap
+ */
+ void removeBlocks(List<BlockInfo> toDeleteList) {
+ if (this.blockDeletionAsync) {
+ blockManager.addBLocksToMarkedDeleteQueue(toDeleteList);
+ } else {
+ Iterator<BlockInfo> iter = toDeleteList.iterator();
+ while (iter.hasNext()) {
+ writeLock();
+ try {
+ for (int i = 0; i < blockDeletionIncrement && iter.hasNext(); i++) {
+ blockManager.removeBlock(iter.next());
Review Comment:
Compared with the previous one, the following code is missing:
```java
toRemoveBlocks.clear();
```
https://github.com/apache/hadoop/pull/3063/files#diff-dac9de4dd225110eff2f29a44000bf32705f02df2b3fcf17b5d89bc236c12f01
> Add configuration to control blocks deletion asynchronous or synchronous
> ------------------------------------------------------------------------
>
> Key: HDFS-16524
> URL: https://issues.apache.org/jira/browse/HDFS-16524
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: namenode
> Reporter: leo sun
> Priority: Major
> Labels: pull-request-available
> Time Spent: 50m
> Remaining Estimate: 0h
>
> When users delete files frequently, deletion is slower than its enqueues. The
> pending deletions blocks will keep increasing.
> Under this circumstance, deletion synchronous is better.
> So I will add configuration to switch deletion asynchronous or synchronous.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
