0lai0 commented on code in PR #276: URL: https://github.com/apache/ozone-site/pull/276#discussion_r2739937500
########## blog/2026-01-29-disk-balancer-preview.md: ########## @@ -0,0 +1,103 @@ +--- +title: "No More Hotspots: Introducing the Automatic Disk Balancer in Apache Ozone" +authors: ["apache-ozone-community","jojochuang", "0lai0","Gargi-jais11"] +date: 2026-01-29 +tags: [Ozone, Disk Balancer, Ozone 2.2, Datanode] +--- + +Ever replaced a drive on a Datanode only to watch it become an I/O hotspot? +Or seen one disk hit 95% usage while others on the same machine sit idle? +These imbalances create performance bottlenecks and increase failure risk. +Apache Ozone's new intra-node Disk Balancer is designed to fix this—automatically. + +<!-- truncate --> + +Cluster-wide balancing in Ozone already ensures replicas are evenly spread across Datanodes. But inside a single Datanode, disks can still drift out of balance over time — for example after adding new disks, replacing hardware, or performing large deletions. This leads to I/O hotspots and uneven wear. + +Disk Balancer closes that gap. + +## Why Disk Balancer? + +- **Disks fill unevenly** when nodes gain or lose volumes. + +- **Large deletes** can empty some disks disproportionately. + +- **Hot disks degrade performance** and become failure risks. + +Even if the cluster is balanced, the node itself may not be. Disk Balancer fixes this automatically. + +## How it works + +The design ([HDDS-5713](https://issues.apache.org/jira/browse/HDDS-5713)) introduces a simple metric: **Volume Data Density** — how much a disk's utilization deviates from the node's average. If the deviation exceeds a threshold, the node begins balancing. + +Balancing is local and safe: + +- Only **closed containers** are moved. +- Moves happen entirely **within the same Datanode.** +- A scheduler periodically checks for imbalance and dispatches copy-and-import tasks. +- Bandwidth and concurrency are **operator-tunable** to avoid interfering with production I/O. + +This runs independently on each Datanode. To use it, first enable the feature by setting `hdds.datanode.disk.balancer.enabled = true` in `ozone-site.xml` on your Datanodes. Once enabled, clients use `ozone admin datanode diskbalancer` commands to talk directly to Datanodes, with SCM only used to discover IN_SERVICE datanodes when running batch operations with `--in-service-datanodes`. + +### Disk Balancing Policies Review Comment: Sure, that's pretty good. Thanks ! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
