jojochuang commented on PR #11450: URL: https://github.com/apache/ozone/pull/11450#issuecomment-6093304876
## Plain-language summary (for reviewers) ### The problem When you **decommission a datanode**, Ozone must **copy EC container shards** off that node before it can shut down safely. The usual approach is **one-to-one copy**: read one shard from the decommissioning node and push a copy to another node. That keeps the decommissioning node busy as a **source** for many transfers. If that node is already overloaded, decommission slows down and the node stays hot. ### What this PR adds (opt-in) It adds a **smarter option** for EC decommission, controlled by a config flag (**off by default**): 1. **Watch load** — SCM checks how full each datanode’s replication queue is (decommissioning nodes still use the same higher effective limit as today). 2. **Switch strategy when a source is “too busy”** — If a decommissioning node is over a threshold (default 90% of its limit), SCM can **stop using it as a 1–1 copy source** and instead schedule **EC reconstruction**: read from several **other** healthy shards and rebuild the missing piece elsewhere. 3. **Prefer healthy nodes as readers** — When reconstructing, SCM prefers shards on normal in-service nodes and skips using the overloaded decommissioning node as a reader when enough other shards exist. 4. **Respect global reconstruction limits** — Reconstruction still goes through the **cluster-wide cap** from HDDS-15071. If reconstruction is throttled or not possible, SCM **falls back to the old 1–1 copy** so decommission can continue. ### What does not change - With the feature **disabled** (default), behavior is **unchanged**: same 1–1 decommission copies as before. - HDDS-15071 added config and global reconstruction throttling; this PR is the **“when loaded, try reconstruction instead of copy”** behavior. ### One sentence **When enabled, if a decommissioning datanode is too busy serving copies, SCM tries to rebuild EC shards using other nodes instead of hammering that node—while still falling back to normal copies if limits or missing shards prevent reconstruction.** -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
