[
https://issues.apache.org/jira/browse/HDFS-16386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
JiangHua Zhu updated HDFS-16386:
--------------------------------
Description:
Our DataNode node has 36 disks. When FsDatasetAsyncDiskService is working, it
will cause a high load on the DataNode.
Here are some monitoring related to memory:
!monitor.png!
Since each disk deletes the block asynchronously, and each thread allows 4
threads to work,
This will cause some troubles to the DataNode, such as increased cpu and
increased memory.
We should appropriately reduce the number of jobs of the total thread so that
the DataNode can work better.
was:
Our DataNode node has 36 disks. When FsDatasetAsyncDiskService is working, it
will cause a high load on the DataNode.
Here are some monitoring related to memory:
Since each disk deletes the block asynchronously, and each thread allows 4
threads to work,
This will cause some troubles to the DataNode, such as increased cpu and
increased memory.
We should appropriately reduce the number of jobs of the total thread so that
the DataNode can work better.
> Reduce DataNode load when FsDatasetAsyncDiskService is working
> --------------------------------------------------------------
>
> Key: HDFS-16386
> URL: https://issues.apache.org/jira/browse/HDFS-16386
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: datanode
> Affects Versions: 2.9.2
> Reporter: JiangHua Zhu
> Assignee: JiangHua Zhu
> Priority: Major
> Attachments: monitor.png
>
>
> Our DataNode node has 36 disks. When FsDatasetAsyncDiskService is working, it
> will cause a high load on the DataNode.
> Here are some monitoring related to memory:
> !monitor.png!
> Since each disk deletes the block asynchronously, and each thread allows 4
> threads to work,
> This will cause some troubles to the DataNode, such as increased cpu and
> increased memory.
> We should appropriately reduce the number of jobs of the total thread so that
> the DataNode can work better.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
