jojochuang opened a new pull request, #10693: URL: https://github.com/apache/ozone/pull/10693
## What changes were proposed in this pull request? HDDS-15412. Add per-volume push replication thread pools on DataNode. Please describe your PR in detail: * Adds optional per-volume replication handler thread pools on the DataNode for push-based container replication. When enabled, each data volume gets its own thread pool, so a slow or backed-off disk no longer blocks push replication for containers on other disks. * This targets the EC decommission scenario described in [HDDS-15412](https://issues.apache.org/jira/browse/HDDS-15412): with a single global pool, replication starts fast on all disks, then slows until only one disk stays busy while others idle. * Default is off (hdds.datanode.replication.per.volume.enabled=false), so existing clusters keep current behavior unless the feature is explicitly enabled. When per.volume.enabled=true: * Push replication is routed to a per-volume thread pool (sized by hdds.datanode.replication.per.volume.streams.limit, default 1) * EC reconstruction and reconcile tasks continue using the global pool (streams.limit) * Volume is resolved from the source container via ContainerSet → container.getContainerData().getVolume() * On volume failure, that volume’s pool is shut down and new push tasks for containers on that volume are rejected When per.volume.enabled=false (default): all tasks use the global pool, same as today. Known limitations / follow-ups * per.volume.enabled requires DN restart (not reconfigurable) * Reconfig of per.volume.streams.limit does not re-apply outofservice.limit.factor on decommissioning nodes (node state transitions do) * Rejected tasks are logged but not counted in replication metrics * No per-volume metrics yet (queue depth, active threads) * No integration test in this PR (unit tests only) ## What is the link to the Apache JIRA https://issues.apache.org/jira/browse/HDDS-15412 ## How was this patch tested? Made with Cursor Composer 2.5. Unit tests added in TestReplicationSupervisor: Feature disabled → no per-volume pools; legacy behavior Init logging with volume paths Per-volume pool size respected Runtime resize via setPerVolumePoolSize() Decommission scaling via outofservice.limit.factor Non-push tasks (reconcile) use global pool when feature enabled Cross-volume push isolation (blocking on volume A does not block volume B) Volume failure shuts down pool and rejects subsequent push tasks TestReplicationConfig: per-volume config validation. All TestReplicationConfig + TestReplicationSupervisor tests pass. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
