[
https://issues.apache.org/jira/browse/HDFS-16203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Takanobu Asanuma resolved HDFS-16203.
-------------------------------------
Fix Version/s: 3.4.0
Resolution: Fixed
> Discover datanodes with unbalanced block pool usage by the standard deviation
> -----------------------------------------------------------------------------
>
> Key: HDFS-16203
> URL: https://issues.apache.org/jira/browse/HDFS-16203
> Project: Hadoop HDFS
> Issue Type: New Feature
> Reporter: tomscut
> Assignee: tomscut
> Priority: Major
> Labels: pull-request-available
> Fix For: 3.4.0
>
> Attachments: image-2021-09-01-19-16-27-172.png
>
> Time Spent: 6h 10m
> Remaining Estimate: 0h
>
> *Discover datanodes with unbalanced volume usage by the standard deviation.*
> *In some scenarios, we may cause unbalanced datanode disk usage:*
> 1. Repair the damaged disk and make it online again.
> 2. Add disks to some Datanodes.
> 3. Some disks are damaged, resulting in slow data writing.
> 4. Use some custom volume choosing policies.
> In the case of unbalanced disk usage, a sudden increase in datanode write
> traffic may result in busy disk I/O with low volume usage, resulting in
> decreased throughput across datanodes.
> We need to find these nodes in time to do diskBalance, or other processing.
> Based on the volume usage of each datanode, we can calculate the standard
> deviation of the volume usage. The more unbalanced the volume, the higher the
> standard deviation.
> *We can display the result on the Web of namenode, and then sorting directly
> to find the nodes where the volumes usages are unbalanced.*
> *{color:#172b4d}This interface is only used to obtain metrics and does not
> adversely affect namenode performance.{color}*
>
> {color:#172b4d}!image-2021-09-01-19-16-27-172.png|width=581,height=216!{color}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
