It seems we don't have access to the event that triggers rebalance in rebalancer. Maybe we can add another method in HelixRebalancer similar to something: shouldRebalance(ClusterEvent: event): boolean, and if this returns false just don't rebalance.
On Thu, Jun 25, 2015 at 11:07 AM, kishore g <[email protected]> wrote: > Hi Changgeng, > > Yes, prevAssignment will be preserved even if controller moves to another > node. Agree, we can either have flag in resourceAssignment/marker > interface/annotation on the computeAssignment method. > > Yeah using current state of each partition should work. One thing to note > that is pending state is not guaranteed to reach (the transition might > fail). In fact, all the in built rebalancers in Helix work off of current > state. > > Yes, rebalancer is invoked on every change in the cluster. You can check > for the event type and return the same assignment for some event types. > Just basing on LiveNodes may not work if the participants fails to handle > the transition. > > thanks, > Kishore G > > On Thu, Jun 25, 2015 at 9:55 AM, Changgeng Li <[email protected]> > wrote: > >> I'm also trying to work around this for now by using currentState to >> figure out the previous resource assignment, using pending state override >> the current state of each partition. Do you think there is any potential >> issue with this approach before prevResourceAssignment is persisted? >> >> I also have another question about when the rebalancer is invoked. My >> test shows it's called a lot of times when add just one new node. Looks >> like it's also called when the current state is changed. If so, we probably >> need implement a shortcut that if livenodes are constraints are not >> changed, just return the previous resource assignment. >> >> >> On Thu, Jun 25, 2015 at 9:50 AM, Changgeng Li <[email protected]> >> wrote: >> >>> Actually I was going to ask whether prevAssignment is preserved if the >>> controller moves to another node. A marker either for the rebalancer or for >>> the returned resource assignment would work. It would be more flexible if >>> there's a flag in the returned resource assignment. >>> >>> >>> >>> On Wed, Jun 24, 2015 at 11:44 PM, kishore g <[email protected]> wrote: >>> >>>> You are right. I think this is a bug. We need to preserve the previous >>>> assignment and pass it on. Having said that we should not do this for all >>>> types of rebalancers. Doing so will introduce additional latency(additional >>>> writes to Zookeeper) and will impact fail over time. Most rebalancers also >>>> tend to be idempotent, i.e they compute the same output given the same set >>>> of inputs. This property in general is not error prone and makes debugging >>>> easier. >>>> >>>> One idea is to have a marker interface Persistable or something and >>>> only save it when the implementation implements this interface. The fix is >>>> easy but love to hear your feedback. >>>> >>>> Also, the AutoRebalancer that comes with Helix, already provides the >>>> features that you are planning to implement. May be you can clone it and >>>> modify it to suite your needs. >>>> >>>> >>>> https://github.com/apache/helix/blob/master/helix-core/src/main/java/org/apache/helix/controller/rebalancer/FullAutoRebalancer.java >>>> >>>> https://github.com/apache/helix/blob/master/helix-core/src/main/java/org/apache/helix/controller/strategy/AutoRebalanceStrategy.java >>>> >>>> >>>> thanks, >>>> Kishore G >>>> >>>> On Wed, Jun 24, 2015 at 11:13 PM, Changgeng Li <[email protected]> >>>> wrote: >>>> >>>>> yes, it's always null. >>>>> >>>>> actually I checked the code that parameter was never set. >>>>> >>>>> https://github.com/apache/helix/blob/acd902e2433f65e9864ccf49fcd1a04c36b1f206/helix-core/src/main/java/org/apache/helix/controller/stages/BestPossibleStateCalcStage.java#L222 >>>>> >>>>> On Wed, Jun 24, 2015 at 10:41 PM, kishore g <[email protected]> >>>>> wrote: >>>>> >>>>>> I see that you haven't subscribed to mailing list yet. Please do that >>>>>> my sending an email to [email protected]. >>>>>> >>>>>> I am also available on #apachehelix irc if you have additional >>>>>> questions. http://helix.apache.org/IRC.html >>>>>> >>>>>> >>>>>> ---------- Forwarded message ---------- >>>>>> From: kishore g <[email protected]> >>>>>> Date: Wed, Jun 24, 2015 at 10:37 PM >>>>>> Subject: Re: Customized rebalancer >>>>>> To: "[email protected]" <[email protected]> >>>>>> >>>>>> >>>>>> Hi Changgeng, >>>>>> >>>>>> I think the first invocation will always be null, subsequent >>>>>> invocations should provide you the previous resource assignment. Are you >>>>>> saying its always null?. >>>>>> >>>>>> thanks, >>>>>> Kishore G >>>>>> >>>>>> On Wed, Jun 24, 2015 at 3:42 PM, Changgeng Li <[email protected] >>>>>> > wrote: >>>>>> >>>>>>> Hello, >>>>>>> >>>>>>> We have a use case that when adding or removing a new node, we hope >>>>>>> to minimize the shuffle of partitions between nodes. I'm trying to >>>>>>> implement a customized rebalancer calculating the new resource >>>>>>> assignment >>>>>>> based on the previous resource assignment. When a new node is added, >>>>>>> just >>>>>>> move some partitions from existing node to the new node, and when a >>>>>>> node is >>>>>>> down, move the partitions on this node to the other nodes. Partitions >>>>>>> would >>>>>>> not move between two nodes if the both status are not changed. >>>>>>> >>>>>>> The documentation says: >>>>>>> In rebalance(), ... the third is the output of the previous >>>>>>> invocation of this method (if supported) ... >>>>>>> >>>>>>> During my testing I found the previous resource assignment passed in >>>>>>> is always null. my question is what "if supported" means here? Does the >>>>>>> rebalancer need to support something? >>>>>>> >>>>>>> >>>>>>> Any other suggestions are also welcome. >>>>>>> >>>>>>> Thanks, >>>>>>> Changgeng >>>>>>> >>>>>> >>>>>> >>>>>> >>>>> >>>> >>> >> >
