Hi Krishore, I'v opened a pull request https://github.com/apache/helix/pull/37 for the rebalancer.
It is not a complete pull request, but gives some idea what we are trying to do with the rebalancer. Thanks Hang Qi On Mon, Sep 21, 2015 at 12:56 PM, Hang Qi <[email protected]> wrote: > Sure, I will first test the new rebalancer, and after that I can send out > the pull request, and Iet's see whether it is generalized or not. > > Thanks > Hang Qi > > > On Monday, September 21, 2015, kishore g <[email protected]> wrote: > >> Please consider contributing the rebalancer to Helix if it can be >> generalized. >> On Sep 21, 2015 11:57 AM, "Hang Qi" <[email protected]> wrote: >> >>> Hi Kanak & Kishore, >>> >>> Thanks very much for your reply. >>> >>> Yes, I have looked into the HelixCustomCodeRunner by myself, however I >>> ended up to go with option 2, write our own rebalancer. >>> >>> Our use case is that we want to extend the Master/Slave state model by >>> adding one special state for special purpose, like partition splitting. >>> Normally, partitions still follow Master/Slave model to perform the state >>> transition. And we have a admin node, which can mark some partitions on one >>> instance to this special state in IdealState before the operation, and >>> after the operation is done, the admin node can remove the special state >>> for those partitions on the instance in IdealState. >>> >>> We are using SEMI_AUTO mode right now, so we are looking into the CUSTOM >>> mode, later on, we think the our new rebalancer is kind of mixing of >>> SEMI_AUTO and CUSTOM, thus we go with option 2. >>> >>> Thanks >>> Hang Qi >>> >>> On Sat, Sep 19, 2015 at 2:04 PM, kishore g <[email protected]> wrote: >>> >>>> There are two ways to write your own rebalancer >>>> >>>> #1. Using CustomCode Runner. This is regsistered in any of the >>>> participants and can be invoked on any node in the cluster starts/stops. >>>> Kanak's email points to this option. >>>> >>>> #2. Implement the rebalancer interface. This is one you have specified >>>> in your email. This is run in controller and is invoked on each and every >>>> change in the cluster such as node added/removed, start/stop, resource >>>> add/remove etc. >>>> >>>> Let me know which mode you are planning to use, we can provide >>>> additional details. >>>> >>>> >>>> >>>> On Sat, Sep 19, 2015 at 11:19 AM, Kanak Biscuitwala < >>>> [email protected]> wrote: >>>> >>>>> Hi, >>>>> >>>>> >>>>> Yeah, you need to register a custom code runner. Here is an example: >>>>> https://github.com/apache/helix/blob/master/helix-core/src/test/java/org/apache/helix/integration/TestHelixCustomCodeRunner.java >>>>> >>>>> >>>>> >>>>> Kanak >>>>> >>>>> From: [email protected] >>>>> Date: Mon, 14 Sep 2015 23:04:21 -0700 >>>>> Subject: Question regarding to CUSTOMIZED balance mode >>>>> To: [email protected] >>>>> >>>>> Hi, >>>>> >>>>> We are evaluating CUSTOMIZED balance mode in helix 0.6.4 currently, >>>>> and I have one question regarding to it. >>>>> >>>>> How do I handle cluster failure case, e.g. the initial ideal state as >>>>> following: >>>>> "mapFields" : { >>>>> "MyResource_0" : { >>>>> "N1" : "MASTER", >>>>> "N2" : "SLAVE", >>>>> } >>>>> } >>>>> If N1 goes down, N2 will not become MASTER automatically, right? >>>>> >>>>> If so, how do I register the listener for cluster state change to >>>>> update ideal state? The doc says "The application needs to implement a >>>>> callback interface that Helix invokes when the cluster state changes. >>>>> Within this callback, the application can recompute the idealstate.", is >>>>> that interface Rebalancer? >>>>> >>>>> Do we have an example for CUSTOMIZED balance? >>>>> >>>>> Besides, when looking at the code, CustomRebalancer implements >>>>> Rebalancer interface, and just returns currentIdealState >>>>> when computeNewIdealState is called. >>>>> >>>>> Thanks >>>>> Hang Qi >>>>> >>>> >>>> >>> >>> >>> >>> > > -- > Qi hang > -- Qi hang
