[
https://issues.apache.org/jira/browse/HDFS-16386?focusedWorklogId=698269&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-698269
]
ASF GitHub Bot logged work on HDFS-16386:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 18/Dec/21 15:00
Start Date: 18/Dec/21 15:00
Worklog Time Spent: 10m
Work Description: jianghuazhu commented on a change in pull request #3806:
URL: https://github.com/apache/hadoop/pull/3806#discussion_r771833746
##########
File path: hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml
##########
@@ -2982,6 +2982,16 @@
</description>
</property>
+<property>
+ <name>dfs.datanode.fsdatasetasyncdisk.max.threads.per.volume</name>
+ <value>4</value>
+ <description>
+ The maximum number of threads per volume used to delete blocks on
Review comment:
Thanks @tomscut for your comments and review.
I agree to your suggestion. But now FsDatasetAsyncDiskService mainly aimed
at removing block.
Do you have a better description of this property?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 698269)
Time Spent: 1h 40m (was: 1.5h)
> Reduce DataNode load when FsDatasetAsyncDiskService is working
> --------------------------------------------------------------
>
> Key: HDFS-16386
> URL: https://issues.apache.org/jira/browse/HDFS-16386
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: datanode
> Affects Versions: 2.9.2
> Reporter: JiangHua Zhu
> Assignee: JiangHua Zhu
> Priority: Major
> Labels: pull-request-available
> Attachments: monitor.png
>
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
> Our DataNode node has 36 disks. When FsDatasetAsyncDiskService is working, it
> will cause a high load on the DataNode.
> Here are some monitoring related to memory:
> !monitor.png!
> Since each disk deletes the block asynchronously, and each thread allows 4
> threads to work,
> This will cause some troubles to the DataNode, such as increased cpu and
> increased memory.
> We should appropriately reduce the number of jobs of the total thread so that
> the DataNode can work better.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
