[
https://issues.apache.org/jira/browse/HDFS-16524?focusedWorklogId=793522&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-793522
]
ASF GitHub Bot logged work on HDFS-16524:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 21/Jul/22 04:19
Start Date: 21/Jul/22 04:19
Worklog Time Spent: 10m
Work Description: cxzl25 commented on code in PR #4139:
URL: https://github.com/apache/hadoop/pull/4139#discussion_r926235580
##########
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java:
##########
@@ -3410,6 +3410,33 @@ FSPermissionChecker getPermissionChecker()
return dir.getPermissionChecker();
}
+ /**
+ * If blockDeletionAsync enables, blocks will be deleted asynchronously.
+ * If not, incrementally remove the blocks from blockManager
+ * Writelock is dropped and reacquired every BLOCK_DELETION_INCREMENT to
+ * ensure that other waiters on the lock can get in. See HDFS-2938
+ *
+ * @param toDeleteList
+ * a list of blocks that need to be removed from blocksMap
+ */
+ void removeBlocks(List<BlockInfo> toDeleteList) {
+ if (this.blockDeletionAsync) {
+ blockManager.addBLocksToMarkedDeleteQueue(toDeleteList);
+ } else {
+ Iterator<BlockInfo> iter = toDeleteList.iterator();
+ while (iter.hasNext()) {
+ writeLock();
+ try {
+ for (int i = 0; i < blockDeletionIncrement && iter.hasNext(); i++) {
Review Comment:
Removed blockDeletionIncrement in HDFS-16554
##########
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java:
##########
@@ -481,6 +481,10 @@ public class DFSConfigKeys extends CommonConfigurationKeys
{
"dfs.namenode.block.deletion.increment";
public static final int DFS_NAMENODE_BLOCK_DELETION_INCREMENT_DEFAULT = 1000;
+ /** Block deletion asynchronous. */
+ public static final String DFS_NAMENODE_BLOCK_DELETION_ASYNC_KEY =
"dfs.namenode.block.deletion.async";
+ public static final boolean DFS_NAMENODE_BLOCK_DELETION_ASYNC_DEFAULT =
false;
Review Comment:
Should it default to true?
Issue Time Tracking
-------------------
Worklog Id: (was: 793522)
Time Spent: 0.5h (was: 20m)
> Add configuration to control blocks deletion asynchronous or synchronous
> ------------------------------------------------------------------------
>
> Key: HDFS-16524
> URL: https://issues.apache.org/jira/browse/HDFS-16524
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: namenode
> Reporter: leo sun
> Priority: Major
> Labels: pull-request-available
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> When users delete files frequently, deletion is slower than its enqueues. The
> pending deletions blocks will keep increasing.
> Under this circumstance, deletion synchronous is better.
> So I will add configuration to switch deletion asynchronous or synchronous.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
