Kishore,
I pushed the code to [email protected]:rmckinnon/helix-poc.git, the rebalance
code is copied from the tutorial.
1. The user defined configuration for rebalancing can be found in
STEP5_Rebalance.java
2. The participants should be running before the rebalance happens
After the rebalancing is invoked, I see the following information about the
cluster:
IdealState for data:
{
"id" : "data",
"mapFields" : {
"data_0" : {
},
"data_1" : {
},
"data_2" : {
},
"data_3" : {
},
"data_4" : {
},
"data_5" : {
}
},
"listFields" : {
"data_0" : [ ],
"data_1" : [ ],
"data_2" : [ ],
"data_3" : [ ],
"data_4" : [ ],
"data_5" : [ ]
},
"simpleFields" : {
"IDEAL_STATE_MODE" : "AUTO",
"NUM_PARTITIONS" : "6",
"REBALANCER_CLASS_NAME" : "rob.local.MyRebalancer",
"REBALANCE_MODE" : "USER_DEFINED",
"REBALANCE_STRATEGY" : "DEFAULT",
"REPLICAS" : "1",
"STATE_MODEL_DEF_REF" : "OnlineOffline",
"STATE_MODEL_FACTORY_NAME" : "DEFAULT"
}
}
ExternalView for data:
{
"id" : "data",
"mapFields" : {
},
"listFields" : {
},
"simpleFields" : {
"BUCKET_SIZE" : "0",
"IDEAL_STATE_MODE" : "AUTO",
"NUM_PARTITIONS" : "6",
"REBALANCER_CLASS_NAME" : "rob.local.MyRebalancer",
"REBALANCE_MODE" : "USER_DEFINED",
"REBALANCE_STRATEGY" : "DEFAULT",
"REPLICAS" : "1",
"STATE_MODEL_DEF_REF" : "OnlineOffline",
"STATE_MODEL_FACTORY_NAME" : "DEFAULT"
}
}
I was expecting that the IdealState or the External View mapfields would
have changed after the rebalancing. I did have my own implementation but
defaulted back to the Tutorial example after I saw my code had no affect on
the cluster.
Thanks,
Rob
On Tue, Oct 16, 2018 at 10:17 AM kishore g <[email protected]> wrote:
> Can you share the code? Feel free to IM me on gtalk.
>
> On Tue, Oct 16, 2018 at 4:54 AM Rob McKinnon <[email protected]>
> wrote:
>
>> Kishore or Lei,
>>
>> I am wondering if you can comment on my question? I am trying to figure
>> out if I am understanding the User-Define Rebalancing correctly or is there
>> an issue with the software.
>>
>>
>> Thanks,
>>
>> Rob
>>
>> On Wed, Oct 10, 2018 at 2:48 PM Rob McKinnon <[email protected]>
>> wrote:
>>
>>> I was using the "run-helix-controll.sh" script to handle the controller
>>> and after implementing a custom controller, I am now able to get the
>>> Rebalancer to be invoked.
>>>
>>> I am running into a related issue as the result of my rebalancing
>>> doesn't seem to have any affect on the cluster. I am assuming that the
>>> Rebalance functionality affects the external view? In the tutorial example,
>>> it looks like the currentIdealState preference list is being updated and a
>>> variable called "assignment" is returned. I did similar things in my
>>> implementations but I am not seeing any changes to the cluster. 1. Am I
>>> understanding this correctly? 2. Should I see the changes in the external
>>> view?
>>>
>>> On Tue, Oct 9, 2018 at 1:02 PM kishore g <[email protected]> wrote:
>>>
>>>> Admin.rebalance method won’t invoke user rebalancer
>>>>
>>>> The rebalancer is invoked when ever there is a change in the cluster
>>>> such as nodes added/started/stopped etc.
>>>>
>>>> Also look for error message in the log as Lei suggested. Note the
>>>> controller should have access to the user defined rebalancer class
>>>>
>>>> On Tue, Oct 9, 2018 at 9:46 AM Lei Xia <[email protected]> wrote:
>>>>
>>>>> Hi, Rob
>>>>>
>>>>> It seems your rebalancer class is not correctly linked into Helix
>>>>> lib, i.e, Helix can not find your rebalancer class at runtime. Do you
>>>>> see
>>>>> some error log from Helix like: ""Exception while invoking custom
>>>>> rebalancer class:"?
>>>>>
>>>>> On Mon, Oct 8, 2018 at 5:37 AM Rob McKinnon <[email protected]>
>>>>> wrote:
>>>>>
>>>>>> I am working through the tutorial on the User-Defined Rebalancing and
>>>>>> I am not able to get my Rebalancer to be invoked.
>>>>>>
>>>>>> Sample Code
>>>>>> ======
>>>>>>
>>>>>> public static void main(String[] args) {
>>>>>> ZKHelixAdmin admin = new ZKHelixAdmin(Conf.ZK_ADDRESS);
>>>>>>
>>>>>> IdealState idealState =
>>>>>> admin.getResourceIdealState(Conf.CLUSTER_NAME, Conf.RESOURCE_NAME);
>>>>>> idealState.setRebalanceMode(IdealState.RebalanceMode.USER_DEFINED);
>>>>>> idealState.setRebalancerClassName(MyRebalancer.class.getName());
>>>>>> admin.setResourceIdealState(Conf.CLUSTER_NAME, Conf.RESOURCE_NAME,
>>>>>> idealState);
>>>>>>
>>>>>>
>>>>>> admin.rebalance(Conf.CLUSTER_NAME, RESOURCE_NAME, NUM_REPLICAS);
>>>>>> }
>>>>>>
>>>>>> ======
>>>>>> I was expecting that when calling the "admin.rebalance" method, it
>>>>>> would invoke "MyRebalance" code but when I run it "MyRebalance" code was
>>>>>> not invoked.
>>>>>>
>>>>>>
>>>>>> Thanks,
>>>>>>
>>>>>> Rob
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Lei Xia
>>>>>
>>>>
