Gargi-jais11 commented on code in PR #276: URL: https://github.com/apache/ozone-site/pull/276#discussion_r2716335713
########## blog/2025-12-09-disk-balancer-preview.md: ########## @@ -0,0 +1,68 @@ +--- +title: "Disk Balancer in Apache Ozone: A Preview" +authors: ["apache-ozone-community"] +date: 2025-12-09 +tags: [Ozone, Disk Balancer, Ozone 2.2, Datanode] +--- + +The Apache Ozone community is near the completion of a long-awaited feature: **Disk Balancer** — a lightweight, automatic way to keep disk usage within each Datanode evenly distributed. + +<!-- truncate --> + +Cluster-wide balancing in Ozone already ensures replicas are evenly spread across Datanodes. But inside a single Datanode, disks can still drift out of balance over time — for example after adding new disks, replacing hardware, or performing large deletions. This leads to I/O hotspots and uneven wear. + +Disk Balancer closes that gap. + +## Why Disk Balancer? + +- **Disks fill unevenly** when nodes gain or lose volumes. + +- **Large deletes** can empty some disks disproportionately. + +- **Hot disks degrade performance** and become failure risks. + +Even if the cluster is balanced, the node itself may not be. Disk Balancer fixes this automatically. + +## How it works + +The design ([HDDS-5713](https://issues.apache.org/jira/browse/HDDS-5713)) introduces a simple metric: **Volume Data Density** — how much a disk's utilization deviates from the node's average. If the deviation exceeds a threshold, the node begins balancing. + +Balancing is local and safe: + +- Only **closed containers** are moved. +- Moves happen entirely **within the same Datanode.** +- A scheduler periodically checks for imbalance and dispatches copy-and-import tasks. +- Bandwidth and concurrency are **operator-tunable** to avoid interfering with production I/O. + +This runs independently on each Datanode; SCM just receives reports. Review Comment: FYI, SCM is now no longer used for diskBalancer reports. All CLI commands are sent directly from clients to DN. ########## blog/2025-12-09-disk-balancer-preview.md: ########## @@ -0,0 +1,68 @@ +--- +title: "Disk Balancer in Apache Ozone: A Preview" +authors: ["apache-ozone-community"] +date: 2025-12-09 +tags: [Ozone, Disk Balancer, Ozone 2.2, Datanode] +--- + +The Apache Ozone community is near the completion of a long-awaited feature: **Disk Balancer** — a lightweight, automatic way to keep disk usage within each Datanode evenly distributed. + +<!-- truncate --> + +Cluster-wide balancing in Ozone already ensures replicas are evenly spread across Datanodes. But inside a single Datanode, disks can still drift out of balance over time — for example after adding new disks, replacing hardware, or performing large deletions. This leads to I/O hotspots and uneven wear. + +Disk Balancer closes that gap. + +## Why Disk Balancer? + +- **Disks fill unevenly** when nodes gain or lose volumes. + +- **Large deletes** can empty some disks disproportionately. + +- **Hot disks degrade performance** and become failure risks. + +Even if the cluster is balanced, the node itself may not be. Disk Balancer fixes this automatically. + +## How it works + +The design ([HDDS-5713](https://issues.apache.org/jira/browse/HDDS-5713)) introduces a simple metric: **Volume Data Density** — how much a disk's utilization deviates from the node's average. If the deviation exceeds a threshold, the node begins balancing. + +Balancing is local and safe: + +- Only **closed containers** are moved. +- Moves happen entirely **within the same Datanode.** +- A scheduler periodically checks for imbalance and dispatches copy-and-import tasks. +- Bandwidth and concurrency are **operator-tunable** to avoid interfering with production I/O. + +This runs independently on each Datanode; SCM just receives reports. + +## Using Disk Balancer + +CLI examples: + +```bash +# Start on all datanodes +ozone admin datanode diskbalancer start -a + +# Check status +ozone admin datanode diskbalancer status +``` + Review Comment: These are outdated commands. Please use the links provided to update the new CLI commands. And add all of them over here. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
