devmadhuu opened a new pull request, #11399: URL: https://github.com/apache/ozone/pull/11399
Co-authored-by: https://github.com/xichen01 ## What changes were proposed in this pull request? On a cluster with mixed storage media, the `ContainerBalancer` can currently move a replica from one storage tier to another. A replica placed on SSD can be relocated to DISK or ARCHIVE purely because the destination datanode has more free space. This silently defeats the storage policy the key was written with, and the balancer has no way to notice: it measures utilization across all of a datanode's volumes, so a node that is full on SSD but empty on ARCHIVE looks half-used. This PR makes the balancer tier-aware end to end, so a move keeps a replica on the storage type it is already on. **Protocol changes** Two optional fields: ``` - SendContainerRequest.storageTypeID (DatanodeClientProtocol.proto) — sent on the first request only, alongside size - ReplicateContainerCommandProto.volumeStorageType (ScmServerDatanodeHeartbeatProtocol.proto) ``` An SCM without storage-type support sends neither, which leaves the target free to choose a volume, so mixed-version clusters behave as they do today. ## What is the link to the Apache JIRA https://issues.apache.org/jira/browse/HDDS-16653 ## How was this patch tested? New and extended unit tests, each verified to fail when its production change is reverted: - `TestReplicateContainerCommand` — new, 9 tests covering the proto round-trip with and without the storage type - `TestDatanodeUsageInfo` — per-tier utilization, `getStorageTypes()` reporting only tiers with capacity, and an absent tier reporting no usage rather than failing - `TestSCMContainerPlacementCapacity` — ranking on the requested tier, and unchanged overall-usage ranking when no tier is given. Both count outcomes over 2000 draws, since chooseNode picks two random indices and returns without comparing when they match - `TestContainerBalancerSelectionCriteria` — exclusion of replicas not on the tier being balanced - `TestMoveManager`, `TestContainerImporter`, `TestSendContainerOutputStream` — extended for the nullable storage type, including `@EnumSource(StorageType.class)` coverage and a null case -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
