This is an automated email from the ASF dual-hosted git repository.
jark pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/fluss.git
The following commit(s) were added to refs/heads/main by this push:
new 27ac61839 [docs] Add docs for rack-aware rebalance goal (#2540)
27ac61839 is described below
commit 27ac61839cce6fc14e49085467021bb250797236
Author: yunhong <[email protected]>
AuthorDate: Sun Feb 1 22:44:50 2026 +0800
[docs] Add docs for rack-aware rebalance goal (#2540)
---
website/docs/engine-flink/procedures.md | 11 +++--
website/docs/maintenance/operations/rebalance.md | 51 ++++++++++++++++++++++--
2 files changed, 55 insertions(+), 7 deletions(-)
diff --git a/website/docs/engine-flink/procedures.md
b/website/docs/engine-flink/procedures.md
index 50c8377a6..182ef033e 100644
--- a/website/docs/engine-flink/procedures.md
+++ b/website/docs/engine-flink/procedures.md
@@ -371,7 +371,8 @@ CALL [catalog_name.]sys.rebalance(
**Parameters:**
-- `priorityGoals` (required): The rebalance goals to achieve, specified as
goal types. Can be a single goal (e.g., `'REPLICA_DISTRIBUTION'`) or multiple
goals separated by commas (e.g., `'REPLICA_DISTRIBUTION,LEADER_DISTRIBUTION'`).
Valid goal types are:
+- `priorityGoals` (required): The rebalance goals to achieve, specified as
goal types. Can be a single goal (e.g., `'REPLICA_DISTRIBUTION'`) or multiple
goals separated by commas (e.g.,
`'RACK_AWARE,REPLICA_DISTRIBUTION,LEADER_DISTRIBUTION'`). Valid goal types are:
+ - `'RACK_AWARE'`: Ensures replicas of the same bucket are distributed
across different racks. This goal should be placed first when rack awareness is
required to prevent non-rack-balanced assignments.
- `'REPLICA_DISTRIBUTION'`: Generates replica movement tasks to ensure the
number of replicas on each TabletServer is near balanced.
- `'LEADER_DISTRIBUTION'`: Generates leadership movement and leader
replica movement tasks to ensure the number of leader replicas on each
TabletServer is near balanced.
@@ -380,6 +381,7 @@ CALL [catalog_name.]sys.rebalance(
**Important Notes:**
- Multiple goals can be specified in priority order. The system will attempt
to achieve goals in the order specified.
+- When rack awareness is required, place `RACK_AWARE` as the first goal to
ensure subsequent goals respect rack constraints.
- Rebalance operations run asynchronously in the background. Use the returned
rebalance ID to monitor progress.
- The rebalance operation respects server tags set by `add_server_tag`. For
example, servers marked with `PERMANENT_OFFLINE` will have their buckets
migrated away.
@@ -389,11 +391,14 @@ CALL [catalog_name.]sys.rebalance(
-- Use the Fluss catalog (replace 'fluss_catalog' with your catalog name if
different)
USE fluss_catalog;
--- Trigger rebalance with replica distribution goal
+-- Trigger rebalance with rack-aware goal (recommended for multi-rack
deployments)
+CALL sys.rebalance('RACK_AWARE,REPLICA_DISTRIBUTION');
+
+-- Trigger rebalance with replica distribution goal only
CALL sys.rebalance('REPLICA_DISTRIBUTION');
-- Trigger rebalance with multiple goals in priority order
-CALL sys.rebalance('REPLICA_DISTRIBUTION,LEADER_DISTRIBUTION');
+CALL sys.rebalance('RACK_AWARE,REPLICA_DISTRIBUTION,LEADER_DISTRIBUTION');
```
### list_rebalance
diff --git a/website/docs/maintenance/operations/rebalance.md
b/website/docs/maintenance/operations/rebalance.md
index 25e6a2a97..48d89ac40 100644
--- a/website/docs/maintenance/operations/rebalance.md
+++ b/website/docs/maintenance/operations/rebalance.md
@@ -51,13 +51,17 @@ Initiate a rebalance operation with specified optimization
goals:
```java
import org.apache.fluss.cluster.rebalance.GoalType;
-// Trigger rebalance with replica distribution goal
-List<GoalType> goals =
Collections.singletonList(GoalType.REPLICA_DISTRIBUTION);
+// Trigger rebalance with rack-aware and replica distribution goals
(recommended for multi-rack deployments)
+List<GoalType> goals = Arrays.asList(
+ GoalType.RACK_AWARE,
+ GoalType.REPLICA_DISTRIBUTION
+);
String rebalanceId = admin.rebalance(goals).get();
System.out.println("Rebalance started with ID: " + rebalanceId);
// Trigger rebalance with multiple goals in priority order
List<GoalType> multipleGoals = Arrays.asList(
+ GoalType.RACK_AWARE,
GoalType.REPLICA_DISTRIBUTION,
GoalType.LEADER_DISTRIBUTION
);
@@ -65,9 +69,14 @@ String rebalanceId = admin.rebalance(multipleGoals).get();
```
Available rebalance goals:
+- **RACK_AWARE**: Ensures replicas of the same bucket are distributed across
different racks. This goal is essential for high availability in multi-rack
deployments, as it prevents data loss when an entire rack fails.
- **REPLICA_DISTRIBUTION**: Ensures the number of replicas on each
TabletServer is near balanced
- **LEADER_DISTRIBUTION**: Ensures the number of leader replicas on each
TabletServer is near balanced
+:::tip Goal Priority
+Goals are processed in the order specified. When using `RACK_AWARE`, always
place it first to ensure subsequent goals (like `REPLICA_DISTRIBUTION`) respect
rack constraints. If `RACK_AWARE` is not the first goal, replica movements may
violate rack awareness requirements.
+:::
+
### 3. Monitor Progress
Track the rebalance operation using the returned rebalance ID:
@@ -156,6 +165,7 @@ public class RebalanceExample {
// Step 2: Trigger rebalance
System.out.println("Triggering rebalance...");
List<GoalType> goals = Arrays.asList(
+ GoalType.RACK_AWARE,
GoalType.REPLICA_DISTRIBUTION,
GoalType.LEADER_DISTRIBUTION
);
@@ -214,8 +224,8 @@ Example using Flink SQL:
-- Tag a server for permanent offline
CALL sys.add_server_tag('0', 'PERMANENT_OFFLINE');
--- Trigger rebalance
-CALL sys.rebalance('REPLICA_DISTRIBUTION,LEADER_DISTRIBUTION');
+-- Trigger rebalance with rack-aware goal (recommended)
+CALL sys.rebalance('RACK_AWARE,REPLICA_DISTRIBUTION,LEADER_DISTRIBUTION');
-- Monitor progress
CALL sys.list_rebalance();
@@ -226,6 +236,38 @@ CALL sys.cancel_rebalance();
## Best Practices
+### Goal Ordering for Rack Awareness
+
+When your cluster is deployed across multiple racks for high availability,
always place `RACK_AWARE` as the **first** goal in your rebalance request. This
ensures that:
+
+1. The rebalance algorithm first satisfies rack distribution constraints
+2. Subsequent goals (like `REPLICA_DISTRIBUTION` and `LEADER_DISTRIBUTION`)
only consider movements that maintain rack awareness
+3. You won't end up with multiple replicas of the same bucket on the same rack
+
+**Recommended goal order for multi-rack deployments:**
+```java
+List<GoalType> goals = Arrays.asList(
+ GoalType.RACK_AWARE, // First: ensure rack distribution
+ GoalType.REPLICA_DISTRIBUTION, // Second: balance replica count
+ GoalType.LEADER_DISTRIBUTION // Third: balance leader count
+);
+```
+
+**Example using Flink SQL:**
+```sql
+-- Correct: RACK_AWARE first ensures rack-balanced assignments
+CALL sys.rebalance('RACK_AWARE,REPLICA_DISTRIBUTION,LEADER_DISTRIBUTION');
+
+-- Incorrect: May produce non-rack-balanced assignments
+-- CALL sys.rebalance('REPLICA_DISTRIBUTION,RACK_AWARE'); -- Don't do this!
+```
+
+:::warning
+If `RACK_AWARE` is not placed first, replica movements generated by earlier
goals may violate rack awareness constraints. The `RACK_AWARE` goal only
validates and constrains movements; it does not undo movements made by prior
goals.
+:::
+
+### General Best Practices
+
1. **Plan Ahead**: Tag servers appropriately before triggering rebalance to
guide the algorithm
2. **Monitor Progress**: Regularly check rebalance status to ensure smooth
operation
3. **Off-Peak Hours**: Schedule rebalance operations during off-peak hours to
minimize impact
@@ -233,6 +275,7 @@ CALL sys.cancel_rebalance();
5. **Backup First**: For production environments, ensure data is backed up
before major topology changes
6. **Goal Priority**: Order rebalance goals by priority - the system attempts
to achieve them in order
7. **Server Tags**: Use `TEMPORARY_OFFLINE` for maintenance scenarios to allow
buckets to return after maintenance
+8. **Rack Awareness First**: In multi-rack deployments, always place
`RACK_AWARE` as the first goal
## Troubleshooting
