[ https://issues.apache.org/jira/browse/HDFS-16203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
ASF GitHub Bot updated HDFS-16203: ---------------------------------- Labels: pull-request-available (was: ) > Discover datanodes with unbalanced block pool usage by the standard deviation > ----------------------------------------------------------------------------- > > Key: HDFS-16203 > URL: https://issues.apache.org/jira/browse/HDFS-16203 > Project: Hadoop HDFS > Issue Type: New Feature > Reporter: tomscut > Assignee: tomscut > Priority: Major > Labels: pull-request-available > Attachments: image-2021-09-01-19-16-27-172.png > > Time Spent: 10m > Remaining Estimate: 0h > > *Discover datanodes with unbalanced volume usage by the standard deviation.* > *In some scenarios, we may cause unbalanced datanode disk usage:* > 1. Repair the damaged disk and make it online again. > 2. Add disks to some Datanodes. > 3. Some disks are damaged, resulting in slow data writing. > 4. Use some custom volume choosing policies. > In the case of unbalanced disk usage, a sudden increase in datanode write > traffic may result in busy disk I/O with low volume usage, resulting in > decreased throughput across datanodes. > We need to find these nodes in time to do diskBalance, or other processing. > Based on the volume usage of each datanode, we can calculate the standard > deviation of the volume usage. The more unbalanced the volume, the higher the > standard deviation. > *We can display the result on the Web of namenode, and then sorting directly > to find the nodes where the volumes usages are unbalanced.* > *{color:#172b4d}This interface is only used to obtain metrics and does not > adversely affect namenode performance.{color}* > > {color:#172b4d}!image-2021-09-01-19-16-27-172.png|width=581,height=216!{color} -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org