narendly commented on a change in pull request #637: Add new WAGED rebalancer
config item "GLOBAL_REBALANCE_ASYNC_MODE".
URL: https://github.com/apache/helix/pull/637#discussion_r352017413
##########
File path: helix-core/src/main/java/org/apache/helix/model/ClusterConfig.java
##########
@@ -94,7 +99,15 @@
// The preference of the rebalance result.
// EVENNESS - Evenness of the resource utilization, partition, and top
state distribution.
// LESS_MOVEMENT - the tendency of keeping the current assignment instead
of moving the partition for optimal assignment.
- REBALANCE_PREFERENCE
+ REBALANCE_PREFERENCE,
+ // Specify if the WAGED rebalancer should asynchronously perform global
rebalance.
+ // Note that asynchronous calculation will reduce the rebalance delay but
may cause more
+ // partition movements. This is because the partial rebalance will be
preformed with an stale
+ // baseline. The rebalance result would be an intermediate one and could
be changed again when
+ // a new baseline is calculated.
+ //
Review comment:
I think it would be a good idea to add:
1. why synchronous baseline calculation would necessarily be slower (seems
like you could make an argument that it's actually not slow because it will
just compute based on the latest (non-stale) baseline). In an async case, since
you calculate based on a stale baseline, your results may not be up-to-date, so
you have to move more partitions, so overall are you losing out on speed?
2. The actual order of calculation. Something like:
- synchronous: 1) partial rebalance -> 2) global baseline calculation based
on result of 1 -> 3) best possible calculation (all in a linear fashion)
- asynchronous: 1) (a) partial rebalance + (b) async global baseline
calculation based on previous best possible before partial rebalance -> 2) best
possible calculation based on result of (a) and (b).
Please let me know if this understanding is accurate. The reason I'm being
very verbose and detailed is because I want to make sure this doesn't become a
"mystery" piece of logic.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
