Hi JunKai and Jiajun,
Thank you for your help. When I tried to use a customized Rebalancer, I saw
the rebalancer is triggered and the idea state is set by it.
3784 18:38:38.268 [GenericHelixController-event_process] INFO
c.u.k.w.c.CustomizedRebalancer - CustomizedRebalancer.computeNewIdealState
invoked!
3785 18:38:38.268 [GenericHelixController-event_process] INFO
c.u.k.w.c.CustomizedRebalancer - set ############### dca1d:
localhost_12001 dca1d_0 online
3786 18:38:38.268 [GenericHelixController-event_process] INFO
c.u.k.w.c.CustomizedRebalancer - set ############### dca1d:
localhost_12002 dca1d_1 online
3787 18:38:38.269 [GenericHelixController-event_process] WARN
o.a.h.c.s.BestPossibleStateCalcStage - Event b4896f24_DEFAULT : Rebalancer
does not have a mapping calculator, defaulting to SEMI_AUT O, resource:
dca1d
However the Helix UI shows "no date to display". The snapshot is attached.
The EXTERNALVIEW and IDEASTATE are:
>>> get /Watchdog/EXTERNALVIEW/dca1d
{
"id":"dca1d"
,"simpleFields":{
"BUCKET_SIZE":"0"
,"IDEAL_STATE_MODE":"AUTO"
,"NUM_PARTITIONS":"2"
,"REBALANCER_CLASS_NAME":"mypackage.controller.CustomizedRebalancer"
,"REBALANCE_MODE":"USER_DEFINED"
,"REBALANCE_STRATEGY":"DEFAULT"
,"REPLICAS":"1"
,"STATE_MODEL_DEF_REF":"MyStateModel"
,"STATE_MODEL_FACTORY_NAME":"DEFAULT"
}
,"listFields":{
}
,"mapFields":{
"dca1d_0":{
"localhost_12010":"ONLINE"
}
,"dca1d_1":{
"localhost_12000":"ONLINE"
}
}
}
>>> get /Watchdog/IDEALSTATES/dca1d
{
"id" : "dca1d",
"simpleFields" : {
"IDEAL_STATE_MODE" : "AUTO",
"NUM_PARTITIONS" : "2",
"REBALANCER_CLASS_NAME" : "mypackage.controller.CustomizedRebalancer",
"REBALANCE_MODE" : "USER_DEFINED",
"REBALANCE_STRATEGY" : "DEFAULT",
"REPLICAS" : "1",
"STATE_MODEL_DEF_REF" : "MyStateModel",
"STATE_MODEL_FACTORY_NAME" : "DEFAULT"
},
"mapFields" : {
"dca1d_0" : {
},
"dca1d_1" : {
}
},
"listFields" : {
"dca1d_0" : [ ],
"dca1d_1" : [ ]
}
}
Comparing with before, when i use CUSTOMIZED rebalancer and model, The
EXTERNALVIEW and IDEASTATE were working well with Helix UI:
>>> get /Watchdog/IDEALSTATES/dca1d
{
"id":"dca1d"
,"simpleFields":{
"IDEAL_STATE_MODE":"CUSTOMIZED"
,"MAX_PARTITIONS_PER_INSTANCE":"1"
,"NUM_PARTITIONS":"2"
,"REBALANCE_MODE":"CUSTOMIZED"
,"REPLICAS":"1"
,"STATE_MODEL_DEF_REF":"MyStateModel"
,"STATE_MODEL_FACTORY_NAME":"DEFAULT"
}
,"listFields":{
}
,"mapFields":{
"0":{
"localhost_12004":"ONLINE"
}
,"1":{
"localhost_12005":"ONLINE"
}
}
}
>>> get /Watchdog/EXTERNALVIEW/dca1d
{
"id":"dca1d"
,"simpleFields":{
"BUCKET_SIZE":"0"
,"IDEAL_STATE_MODE":"CUSTOMIZED"
,"MAX_PARTITIONS_PER_INSTANCE":"1"
,"NUM_PARTITIONS":"2"
,"REBALANCE_MODE":"CUSTOMIZED"
,"REPLICAS":"1"
,"STATE_MODEL_DEF_REF":"MyStateModel"
,"STATE_MODEL_FACTORY_NAME":"DEFAULT"
}
,"listFields":{
}
,"mapFields":{
"0":{
"localhost_12004":"ONLINE"
}
,"1":{
"localhost_12005":"ONLINE"
}
}
}
I see that the mapFields key changed after using "USER_DEFINED" rebalancer.
Here are the code i used:
public static void addResource(String resource, int num) {
clusterSetup.addResourceToCluster(CLUSTER_NAME, resource, num,
STATE_MODEL_NAME);
clusterSetup.addResourceProperty(CLUSTER_NAME, resource,
IdealState.IdealStateProperty.REBALANCER_CLASS_NAME.toString(),
CustomizedRebalancer.class.getName());
clusterSetup.addResourceProperty(CLUSTER_NAME, resource,
IdealState.IdealStateProperty.REBALANCE_MODE.toString(),
IdealState.RebalanceMode.USER_DEFINED.toString());
clusterSetup.rebalanceStorageCluster(CLUSTER_NAME, resource, 1, "");
What is the right way to set a rebalancer?
The rebalancer method has a single resource interface. What is the right
way to rebalance all resources in the cluster for any event change in the
cluster? My concern is the sequence and thread safety.
@Override
public IdealState computeNewIdealState(String resourceName, IdealState
currentIdealState,
CurrentStateOutput currentStateOutput, ClusterDataCache clusterData) {
Regards,
Leo
On Fri, Oct 30, 2020 at 10:21 PM Wang Jiajun <[email protected]> wrote:
> Hi Leo,
>
> Are you looking for guidance on how to configure the IdealState so the
> resource will be rebalanced by your user_defined rebalancer? Maybe you can
> refer to this test case TestCustomizedIdealStateRebalancer.
>
> Best Regards,
> Jiajun
>
>
> On Fri, Oct 30, 2020 at 10:11 PM Xue Junkai <[email protected]> wrote:
>
>> Hi Leo,
>>
>> What kind of examples are you looking for? Participant side? Or how to
>> build IdealState?
>>
>> Best,
>>
>> Junkai
>>
>> On Fri, Oct 30, 2020 at 9:57 PM Leo, Li <[email protected]> wrote:
>>
>>> Hi,
>>> I am a new user of Helix. I wonder if you could give an example code of
>>> using this user_defined model
>>> https://helix.apache.org/1.0.1-docs/tutorial_user_def_rebalancer.html?
>>>
>>> Thank you,
>>> Leo
>>>
>>
