sravani-revuri opened a new pull request, #11342: URL: https://github.com/apache/ozone/pull/11342
## What changes were proposed in this pull request? Add ozone admin containerbalancer recommend command The command should: Fetch live datanode usage from SCM (via the existing getDatanodeUsageInfo RPC) Read local Ozone configuration for global defaults (threshold, timeouts, move limits etc.) Analyze the cluster using existing ContainerBalancerClusterAnalyzer / ContainerBalancerClusterSnapshot ([HDDS-16173](https://issues.apache.org/jira/browse/HDDS-16173)) For each profile (SLOW, MEDIUM, FAST), produce: Recommended configuration parameters with a short rationale Time/iteration estimates by reusing ContainerBalancerAdvisor.estimateDryRun() ([HDDS-16179](https://issues.apache.org/jira/browse/HDDS-16179)) Each recommended values should be validated first. No --profile, config overrides, or interactive mode on this command If the user wants custom inputs or refined estimates, they use dry-run instead. But the command should allow the user to provide include/exclude datanodes and threshold values. ``` Example output: RECOMMENDED CONFIGURATION (profile: SLOW) ══════════════════════════════════════════════════════ Parameter Value Rationale ───────────────────────────────────────────────────── --threshold <value> <why this value> --max-datanodes-percentage-to-involve <value> <why this value> --max-size-to-move-per-iteration-in-gb <value> <why this value> --max-size-entering-target-in-gb <value> <why this value> --max-size-leaving-source-in-gb <value> <why this value> --move-timeout-minutes <value> <why this value> --move-replication-timeout-minutes <value> <why this value> --balancing-iteration-interval-minutes <value> <why this value> --iterations <value> <why this value> ─── Estimation ───────────────────────────────────────────── Bytes to move: <cluster-derived> Per iteration (estimate): <estimate> Estimated iterations: <N>(planning estimate: <M>, includes +30% buffer) Estimated duration: upper bound <N> (planning estimate: <M>, includes +30% buffer) (assumes full move timeout + interval each cycle) (profile: MEDIUM) ... (profile: FAST) ... ``` ## What is the link to the Apache JIRA https://issues.apache.org/jira/browse/HDDS-16181 ## How was this patch tested? written test and manual test. ### all profiles recommend + estimate : ``` bash-5.1$ ozone admin containerbalancer recommend RECOMMENDED CONFIGURATION (profile: SLOW) Recommended parameters: Parameter Value Rationale --threshold 10.0% default from configuration --max-datanodes-percentage-to-involve 34% profile 10%, raised to 34% (≥2 nodes) --max-size-to-move-per-iteration-in-gb 10 GB min(500 GB ceiling, max(per-iteration 258.10 MB, entering 10 GB, leaving 10 GB)) --max-size-entering-target-in-gb 10 GB / node profile default (10 GB) --max-size-leaving-source-in-gb 10 GB / node profile default (10 GB) --move-timeout-minutes 65 min configuration default (65 min) --move-replication-timeout-minutes 50 min configuration default (50 min) --balancing-iteration-interval-minutes 0 min configuration default (0 min) --iterations 2 includes +30% buffer (planning estimate: 2) Estimation: Bytes to move: 258.10 MB Per iteration (estimate): ~258.10 MB Estimated iterations: 1 (planning estimate: 2, includes +30% buffer) Estimated duration: upper bound ~1.1 hours (planning estimate: ~2.2 hours, includes +30% buffer) (assumes full move timeout + interval each cycle) RECOMMENDED CONFIGURATION (profile: MEDIUM) Recommended parameters: Parameter Value Rationale --threshold 10.0% default from configuration --max-datanodes-percentage-to-involve 34% profile 20%, raised to 34% (≥2 nodes) --max-size-to-move-per-iteration-in-gb 26 GB min(500 GB ceiling, max(per-iteration 258.10 MB, entering 26 GB, leaving 26 GB)) --max-size-entering-target-in-gb 26 GB / node profile default (26 GB) --max-size-leaving-source-in-gb 26 GB / node profile default (26 GB) --move-timeout-minutes 65 min configuration default (65 min) --move-replication-timeout-minutes 50 min configuration default (50 min) --balancing-iteration-interval-minutes 0 min configuration default (0 min) --iterations 2 includes +30% buffer (planning estimate: 2) Estimation: Bytes to move: 258.10 MB Per iteration (estimate): ~258.10 MB Estimated iterations: 1 (planning estimate: 2, includes +30% buffer) Estimated duration: upper bound ~1.1 hours (planning estimate: ~2.2 hours, includes +30% buffer) (assumes full move timeout + interval each cycle) RECOMMENDED CONFIGURATION (profile: FAST) Recommended parameters: Parameter Value Rationale --threshold 10.0% default from configuration --max-datanodes-percentage-to-involve 40% profile default (40%) --max-size-to-move-per-iteration-in-gb 100 GB min(500 GB ceiling, max(per-iteration 258.10 MB, entering 100 GB, leaving 100 GB)) --max-size-entering-target-in-gb 100 GB / node profile default (100 GB) --max-size-leaving-source-in-gb 100 GB / node profile default (100 GB) --move-timeout-minutes 65 min configuration default (65 min) --move-replication-timeout-minutes 50 min configuration default (50 min) --balancing-iteration-interval-minutes 0 min configuration default (0 min) --iterations 2 includes +30% buffer (planning estimate: 2) Estimation: Bytes to move: 258.10 MB Per iteration (estimate): ~258.10 MB Estimated iterations: 1 (planning estimate: 2, includes +30% buffer) Estimated duration: upper bound ~1.1 hours (planning estimate: ~2.2 hours, includes +30% buffer) (assumes full move timeout + interval each cycle) bash-5.1$ ozone admin containerbalancer estimate --all Profile: SLOW Based on: Threshold: 10.0% Datanode involvement: 34% Max entering target: 10 GB / node Max leaving source: 10 GB / node Max per iteration: 500 GB Move timeout: 65 min Balancing interval: 0 min Bytes to move: 258.10 MB Per iteration (estimate): ~258.10 MB Estimated iterations: 1 (planning estimate: 2, includes +30% buffer) Estimated duration: upper bound ~1.1 hours (planning estimate: ~2.2 hours, includes +30% buffer) (assumes full move timeout + interval each cycle) Profile: MEDIUM Based on: Threshold: 10.0% Datanode involvement: 34% Max entering target: 26 GB / node Max leaving source: 26 GB / node Max per iteration: 500 GB Move timeout: 65 min Balancing interval: 0 min Bytes to move: 258.10 MB Per iteration (estimate): ~258.10 MB Estimated iterations: 1 (planning estimate: 2, includes +30% buffer) Estimated duration: upper bound ~1.1 hours (planning estimate: ~2.2 hours, includes +30% buffer) (assumes full move timeout + interval each cycle) Profile: FAST Based on: Threshold: 10.0% Datanode involvement: 40% Max entering target: 100 GB / node Max leaving source: 100 GB / node Max per iteration: 500 GB Move timeout: 65 min Balancing interval: 0 min Bytes to move: 258.10 MB Per iteration (estimate): ~258.10 MB Estimated iterations: 1 (planning estimate: 2, includes +30% buffer) Estimated duration: upper bound ~1.1 hours (planning estimate: ~2.2 hours, includes +30% buffer) (assumes full move timeout + interval each cycle) ``` ### threshold recommend + estimate ``` bash-5.1$ ozone admin containerbalancer recommend -t 5 RECOMMENDED CONFIGURATION (profile: SLOW) Recommended parameters: Parameter Value Rationale --threshold 5.0% user override --max-datanodes-percentage-to-involve 34% profile 10%, raised to 34% (≥2 nodes) --max-size-to-move-per-iteration-in-gb 10 GB min(500 GB ceiling, max(per-iteration 565.27 MB, entering 10 GB, leaving 10 GB)) --max-size-entering-target-in-gb 10 GB / node profile default (10 GB) --max-size-leaving-source-in-gb 10 GB / node profile default (10 GB) --move-timeout-minutes 65 min configuration default (65 min) --move-replication-timeout-minutes 50 min configuration default (50 min) --balancing-iteration-interval-minutes 0 min configuration default (0 min) --iterations 2 includes +30% buffer (planning estimate: 2) Estimation: Bytes to move: 565.27 MB Per iteration (estimate): ~565.27 MB Estimated iterations: 1 (planning estimate: 2, includes +30% buffer) Estimated duration: upper bound ~1.1 hours (planning estimate: ~2.2 hours, includes +30% buffer) (assumes full move timeout + interval each cycle) RECOMMENDED CONFIGURATION (profile: MEDIUM) Recommended parameters: Parameter Value Rationale --threshold 5.0% user override --max-datanodes-percentage-to-involve 34% profile 20%, raised to 34% (≥2 nodes) --max-size-to-move-per-iteration-in-gb 26 GB min(500 GB ceiling, max(per-iteration 565.27 MB, entering 26 GB, leaving 26 GB)) --max-size-entering-target-in-gb 26 GB / node profile default (26 GB) --max-size-leaving-source-in-gb 26 GB / node profile default (26 GB) --move-timeout-minutes 65 min configuration default (65 min) --move-replication-timeout-minutes 50 min configuration default (50 min) --balancing-iteration-interval-minutes 0 min configuration default (0 min) --iterations 2 includes +30% buffer (planning estimate: 2) Estimation: Bytes to move: 565.27 MB Per iteration (estimate): ~565.27 MB Estimated iterations: 1 (planning estimate: 2, includes +30% buffer) Estimated duration: upper bound ~1.1 hours (planning estimate: ~2.2 hours, includes +30% buffer) (assumes full move timeout + interval each cycle) RECOMMENDED CONFIGURATION (profile: FAST) Recommended parameters: Parameter Value Rationale --threshold 5.0% user override --max-datanodes-percentage-to-involve 40% profile default (40%) --max-size-to-move-per-iteration-in-gb 100 GB min(500 GB ceiling, max(per-iteration 565.27 MB, entering 100 GB, leaving 100 GB)) --max-size-entering-target-in-gb 100 GB / node profile default (100 GB) --max-size-leaving-source-in-gb 100 GB / node profile default (100 GB) --move-timeout-minutes 65 min configuration default (65 min) --move-replication-timeout-minutes 50 min configuration default (50 min) --balancing-iteration-interval-minutes 0 min configuration default (0 min) --iterations 2 includes +30% buffer (planning estimate: 2) Estimation: Bytes to move: 565.27 MB Per iteration (estimate): ~565.27 MB Estimated iterations: 1 (planning estimate: 2, includes +30% buffer) Estimated duration: upper bound ~1.1 hours (planning estimate: ~2.2 hours, includes +30% buffer) (assumes full move timeout + interval each cycle) bash-5.1$ ozone admin containerbalancer estimate --all -t 5 Profile: SLOW Based on: Threshold: 5.0% Datanode involvement: 34% Max entering target: 10 GB / node Max leaving source: 10 GB / node Max per iteration: 500 GB Move timeout: 65 min Balancing interval: 0 min Bytes to move: 565.27 MB Per iteration (estimate): ~565.27 MB Estimated iterations: 1 (planning estimate: 2, includes +30% buffer) Estimated duration: upper bound ~1.1 hours (planning estimate: ~2.2 hours, includes +30% buffer) (assumes full move timeout + interval each cycle) Profile: MEDIUM Based on: Threshold: 5.0% Datanode involvement: 34% Max entering target: 26 GB / node Max leaving source: 26 GB / node Max per iteration: 500 GB Move timeout: 65 min Balancing interval: 0 min Bytes to move: 565.27 MB Per iteration (estimate): ~565.27 MB Estimated iterations: 1 (planning estimate: 2, includes +30% buffer) Estimated duration: upper bound ~1.1 hours (planning estimate: ~2.2 hours, includes +30% buffer) (assumes full move timeout + interval each cycle) Profile: FAST Based on: Threshold: 5.0% Datanode involvement: 40% Max entering target: 100 GB / node Max leaving source: 100 GB / node Max per iteration: 500 GB Move timeout: 65 min Balancing interval: 0 min Bytes to move: 565.27 MB Per iteration (estimate): ~565.27 MB Estimated iterations: 1 (planning estimate: 2, includes +30% buffer) Estimated duration: upper bound ~1.1 hours (planning estimate: ~2.2 hours, includes +30% buffer) (assumes full move timeout + interval each cycle) ``` ### include all matches with estimate --all ``` bash-5.1$ ozone admin containerbalancer recommend \ --include-datanodes ozone-balancer-datanode1-1.ozone-balancer_default,ozone-balancer-datanode2-1.ozone-balancer_default,ozone-balancer-datanode3-1.ozone-balancer_default,ozone-balancer-datanode4-1.ozone-balancer_default,ozone-balancer-datanode5-1.ozone-balancer_default,ozone-balancer-datanode6-1.ozone-balancer_default RECOMMENDED CONFIGURATION (profile: SLOW) Recommended parameters: Parameter Value Rationale --threshold 10.0% default from configuration --max-datanodes-percentage-to-involve 34% profile 10%, raised to 34% (≥2 nodes) --max-size-to-move-per-iteration-in-gb 10 GB min(500 GB ceiling, max(per-iteration 258.10 MB, entering 10 GB, leaving 10 GB)) --max-size-entering-target-in-gb 10 GB / node profile default (10 GB) --max-size-leaving-source-in-gb 10 GB / node profile default (10 GB) --move-timeout-minutes 65 min configuration default (65 min) --move-replication-timeout-minutes 50 min configuration default (50 min) --balancing-iteration-interval-minutes 0 min configuration default (0 min) --iterations 2 includes +30% buffer (planning estimate: 2) Estimation: Bytes to move: 258.10 MB Per iteration (estimate): ~258.10 MB Estimated iterations: 1 (planning estimate: 2, includes +30% buffer) Estimated duration: upper bound ~1.1 hours (planning estimate: ~2.2 hours, includes +30% buffer) (assumes full move timeout + interval each cycle) RECOMMENDED CONFIGURATION (profile: MEDIUM) Recommended parameters: Parameter Value Rationale --threshold 10.0% default from configuration --max-datanodes-percentage-to-involve 34% profile 20%, raised to 34% (≥2 nodes) --max-size-to-move-per-iteration-in-gb 26 GB min(500 GB ceiling, max(per-iteration 258.10 MB, entering 26 GB, leaving 26 GB)) --max-size-entering-target-in-gb 26 GB / node profile default (26 GB) --max-size-leaving-source-in-gb 26 GB / node profile default (26 GB) --move-timeout-minutes 65 min configuration default (65 min) --move-replication-timeout-minutes 50 min configuration default (50 min) --balancing-iteration-interval-minutes 0 min configuration default (0 min) --iterations 2 includes +30% buffer (planning estimate: 2) Estimation: Bytes to move: 258.10 MB Per iteration (estimate): ~258.10 MB Estimated iterations: 1 (planning estimate: 2, includes +30% buffer) Estimated duration: upper bound ~1.1 hours (planning estimate: ~2.2 hours, includes +30% buffer) (assumes full move timeout + interval each cycle) RECOMMENDED CONFIGURATION (profile: FAST) Recommended parameters: Parameter Value Rationale --threshold 10.0% default from configuration --max-datanodes-percentage-to-involve 40% profile default (40%) --max-size-to-move-per-iteration-in-gb 100 GB min(500 GB ceiling, max(per-iteration 258.10 MB, entering 100 GB, leaving 100 GB)) --max-size-entering-target-in-gb 100 GB / node profile default (100 GB) --max-size-leaving-source-in-gb 100 GB / node profile default (100 GB) --move-timeout-minutes 65 min configuration default (65 min) --move-replication-timeout-minutes 50 min configuration default (50 min) --balancing-iteration-interval-minutes 0 min configuration default (0 min) --iterations 2 includes +30% buffer (planning estimate: 2) Estimation: Bytes to move: 258.10 MB Per iteration (estimate): ~258.10 MB Estimated iterations: 1 (planning estimate: 2, includes +30% buffer) Estimated duration: upper bound ~1.1 hours (planning estimate: ~2.2 hours, includes +30% buffer) (assumes full move timeout + interval each cycle) ``` ### exclude recommend + estimate ``` ozone admin containerbalancer recommend --exclude-datanodes ozone-balancer-datanode6-1.ozone-balancer_default RECOMMENDED CONFIGURATION (profile: SLOW) Recommended parameters: Parameter Value Rationale --threshold 10.0% default from configuration --max-datanodes-percentage-to-involve 40% profile 10%, raised to 40% (≥2 nodes) --max-size-to-move-per-iteration-in-gb 10 GB min(500 GB ceiling, max(per-iteration 288.39 MB, entering 10 GB, leaving 10 GB)) --max-size-entering-target-in-gb 10 GB / node profile default (10 GB) --max-size-leaving-source-in-gb 10 GB / node profile default (10 GB) --move-timeout-minutes 65 min configuration default (65 min) --move-replication-timeout-minutes 50 min configuration default (50 min) --balancing-iteration-interval-minutes 0 min configuration default (0 min) --iterations 2 includes +30% buffer (planning estimate: 2) Estimation: Bytes to move: 288.39 MB Per iteration (estimate): ~288.39 MB Estimated iterations: 1 (planning estimate: 2, includes +30% buffer) Estimated duration: upper bound ~1.1 hours (planning estimate: ~2.2 hours, includes +30% buffer) (assumes full move timeout + interval each cycle) RECOMMENDED CONFIGURATION (profile: MEDIUM) Recommended parameters: Parameter Value Rationale --threshold 10.0% default from configuration --max-datanodes-percentage-to-involve 40% profile 20%, raised to 40% (≥2 nodes) --max-size-to-move-per-iteration-in-gb 26 GB min(500 GB ceiling, max(per-iteration 288.39 MB, entering 26 GB, leaving 26 GB)) --max-size-entering-target-in-gb 26 GB / node profile default (26 GB) --max-size-leaving-source-in-gb 26 GB / node profile default (26 GB) --move-timeout-minutes 65 min configuration default (65 min) --move-replication-timeout-minutes 50 min configuration default (50 min) --balancing-iteration-interval-minutes 0 min configuration default (0 min) --iterations 2 includes +30% buffer (planning estimate: 2) Estimation: Bytes to move: 288.39 MB Per iteration (estimate): ~288.39 MB Estimated iterations: 1 (planning estimate: 2, includes +30% buffer) Estimated duration: upper bound ~1.1 hours (planning estimate: ~2.2 hours, includes +30% buffer) (assumes full move timeout + interval each cycle) RECOMMENDED CONFIGURATION (profile: FAST) Recommended parameters: Parameter Value Rationale --threshold 10.0% default from configuration --max-datanodes-percentage-to-involve 40% profile default (40%) --max-size-to-move-per-iteration-in-gb 100 GB min(500 GB ceiling, max(per-iteration 288.39 MB, entering 100 GB, leaving 100 GB)) --max-size-entering-target-in-gb 100 GB / node profile default (100 GB) --max-size-leaving-source-in-gb 100 GB / node profile default (100 GB) --move-timeout-minutes 65 min configuration default (65 min) --move-replication-timeout-minutes 50 min configuration default (50 min) --balancing-iteration-interval-minutes 0 min configuration default (0 min) --iterations 2 includes +30% buffer (planning estimate: 2) Estimation: Bytes to move: 288.39 MB Per iteration (estimate): ~288.39 MB Estimated iterations: 1 (planning estimate: 2, includes +30% buffer) Estimated duration: upper bound ~1.1 hours (planning estimate: ~2.2 hours, includes +30% buffer) (assumes full move timeout + interval each cycle) bash-5.1$ ozone admin containerbalancer estimate --all --exclude-datanodes ozone-balancer-datanode6-1.ozone-balancer_default Profile: SLOW Based on: Threshold: 10.0% Datanode involvement: 40% Max entering target: 10 GB / node Max leaving source: 10 GB / node Max per iteration: 500 GB Move timeout: 65 min Balancing interval: 0 min Bytes to move: 288.39 MB Per iteration (estimate): ~288.39 MB Estimated iterations: 1 (planning estimate: 2, includes +30% buffer) Estimated duration: upper bound ~1.1 hours (planning estimate: ~2.2 hours, includes +30% buffer) (assumes full move timeout + interval each cycle) Profile: MEDIUM Based on: Threshold: 10.0% Datanode involvement: 40% Max entering target: 26 GB / node Max leaving source: 26 GB / node Max per iteration: 500 GB Move timeout: 65 min Balancing interval: 0 min Bytes to move: 288.39 MB Per iteration (estimate): ~288.39 MB Estimated iterations: 1 (planning estimate: 2, includes +30% buffer) Estimated duration: upper bound ~1.1 hours (planning estimate: ~2.2 hours, includes +30% buffer) (assumes full move timeout + interval each cycle) Profile: FAST Based on: Threshold: 10.0% Datanode involvement: 40% Max entering target: 100 GB / node Max leaving source: 100 GB / node Max per iteration: 500 GB Move timeout: 65 min Balancing interval: 0 min Bytes to move: 288.39 MB Per iteration (estimate): ~288.39 MB Estimated iterations: 1 (planning estimate: 2, includes +30% buffer) Estimated duration: upper bound ~1.1 hours (planning estimate: ~2.2 hours, includes +30% buffer) (assumes full move timeout + interval each cycle) ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
