Ignore what is said it works with idealState.setRebalancerClassName(DelayedAutoRebalancer.class.getName());
On Thu, Mar 8, 2018 at 6:08 PM, Utsav Kanani <[email protected]> wrote: > sorry for the late response > hey guys this does not help either > do you guys want me to send you my code files? > > this is the code change i made in the ZKHelixAdmin class > tried with both > > idealState.setRebalancerClassName(DelayedAutoRebalancer.class.getName()); > > and withoug > > @Override > public void addResource(String clusterName, String resourceName, int > partitions, > String stateModelRef, String rebalancerMode, String rebalanceStrategy, > int bucketSize, > int maxPartitionsPerInstance) { > if (!ZKUtil.isClusterSetup(clusterName, _zkClient)) { > throw new HelixException("cluster " + clusterName + " is not setup yet"); > } > > IdealState idealState = new IdealState(resourceName); > idealState.setNumPartitions(partitions); > idealState.setStateModelDefRef(stateModelRef); > RebalanceMode mode = > idealState.rebalanceModeFromString(rebalancerMode, > RebalanceMode.SEMI_AUTO); > idealState.setRebalanceMode(mode); > idealState.setRebalanceStrategy(rebalanceStrategy); > -------->idealState.setMinActiveReplicas(0); > > idealState.setStateModelFactoryName(HelixConstants.DEFAULT_STATE_MODEL_FACTORY); > idealState.setRebalanceDelay(100000); > idealState.setDelayRebalanceEnabled(true); > //idealState.setRebalancerClassName(DelayedAutoRebalancer.class.getName()); > > if (maxPartitionsPerInstance > 0 && maxPartitionsPerInstance < > Integer.MAX_VALUE) { > idealState.setMaxPartitionsPerInstance(maxPartitionsPerInstance); > } > if (bucketSize > 0) { > idealState.setBucketSize(bucketSize); > } > addResource(clusterName, resourceName, idealState); > } > > > > On Thu, Mar 1, 2018 at 11:20 AM, kishore g <[email protected]> wrote: > >> We should have a recipe for delayed rebalancer >> >> On Thu, Mar 1, 2018 at 9:39 AM, Lei Xia <[email protected]> wrote: >> >>> Hi, Utsav >>> >>> Sorry to get back to you late. There is one more thing to config, >>> >>> idealstate.setMinActiveReplicas(0); >>> >>> This tell Helix the minimal replica it needs to maintain, by default >>> is set to 1, it means Helix needs to maintain at least 1 replica >>> irregardless of delayed rebalancing. For your case, you want to set it to >>> 0. >>> >>> >>> Lei >>> >>> On Mon, Feb 26, 2018 at 11:38 AM, Utsav Kanani <[email protected]> >>> wrote: >>> >>>> Hi Lei, >>>> >>>> That did not work >>>> Seeing the same behavior >>>> Added the following method to ZKHelixAdmin Class >>>> >>>> public void enableClusterDelayMode(String clusterName) { >>>> ConfigAccessor configAccessor = new ConfigAccessor(_zkClient); >>>> ClusterConfig clusterConfig = >>>> configAccessor.getClusterConfig(clusterName); >>>> clusterConfig.setDelayRebalaceEnabled(true); >>>> clusterConfig.setRebalanceDelayTime(100000); >>>> configAccessor.setClusterConfig(clusterName, clusterConfig); >>>> } >>>> >>>> and calling it in the demo class >>>> >>>> HelixAdmin admin = new ZKHelixAdmin(zkAddress); >>>> admin.addCluster(clusterName, true); >>>> ---->((ZKHelixAdmin)admin).enableClusterDelayMode(clusterName); >>>> StateModelConfigGenerator generator = new StateModelConfigGenerator(); >>>> admin.addStateModelDef(clusterName, "OnlineOffline", >>>> new StateModelDefinition(generator.generateConfigForOnlineOffline())); >>>> >>>> admin.addResource(clusterName, lockGroupName, numPartitions, >>>> "OnlineOffline", >>>> RebalanceMode.FULL_AUTO.toString()); >>>> admin.rebalance(clusterName, lockGroupName, 1); >>>> >>>> >>>> >>>> >>>> >>>> STARTING localhost_12000 >>>> STARTING localhost_12001 >>>> STARTING localhost_12002 >>>> STARTED localhost_12000 >>>> STARTED localhost_12002 >>>> STARTED localhost_12001 >>>> localhost_12000 acquired lock:lock-group_0 >>>> localhost_12002 acquired lock:lock-group_3 >>>> localhost_12002 acquired lock:lock-group_9 >>>> localhost_12001 acquired lock:lock-group_2 >>>> localhost_12001 acquired lock:lock-group_5 >>>> localhost_12000 acquired lock:lock-group_11 >>>> localhost_12002 acquired lock:lock-group_6 >>>> localhost_12000 acquired lock:lock-group_7 >>>> localhost_12002 acquired lock:lock-group_10 >>>> localhost_12001 acquired lock:lock-group_8 >>>> localhost_12001 acquired lock:lock-group_1 >>>> localhost_12000 acquired lock:lock-group_4 >>>> lockName acquired By >>>> ====================================== >>>> lock-group_0 localhost_12000 >>>> lock-group_1 localhost_12001 >>>> lock-group_10 localhost_12002 >>>> lock-group_11 localhost_12000 >>>> lock-group_2 localhost_12001 >>>> lock-group_3 localhost_12002 >>>> lock-group_4 localhost_12000 >>>> lock-group_5 localhost_12001 >>>> lock-group_6 localhost_12002 >>>> lock-group_7 localhost_12000 >>>> lock-group_8 localhost_12001 >>>> lock-group_9 localhost_12002 >>>> Stopping localhost_12000 >>>> localhost_12000Interrupted >>>> localhost_12001 acquired lock:lock-group_11 >>>> localhost_12001 acquired lock:lock-group_0 >>>> localhost_12002 acquired lock:lock-group_7 >>>> localhost_12002 acquired lock:lock-group_4 >>>> lockName acquired By >>>> ====================================== >>>> lock-group_0 localhost_12001 >>>> lock-group_1 localhost_12001 >>>> lock-group_10 localhost_12002 >>>> lock-group_11 localhost_12001 >>>> lock-group_2 localhost_12001 >>>> lock-group_3 localhost_12002 >>>> lock-group_4 localhost_12002 >>>> lock-group_5 localhost_12001 >>>> lock-group_6 localhost_12002 >>>> lock-group_7 localhost_12002 >>>> lock-group_8 localhost_12001 >>>> lock-group_9 localhost_12002 >>>> ===Starting localhost_12000 >>>> STARTING localhost_12000 >>>> localhost_12000 acquired lock:lock-group_11 >>>> localhost_12000 acquired lock:lock-group_0 >>>> STARTED localhost_12000 >>>> localhost_12000 acquired lock:lock-group_7 >>>> localhost_12000 acquired lock:lock-group_4 >>>> localhost_12001 releasing lock:lock-group_11 >>>> localhost_12001 releasing lock:lock-group_0 >>>> localhost_12002 releasing lock:lock-group_7 >>>> localhost_12002 releasing lock:lock-group_4 >>>> lockName acquired By >>>> ====================================== >>>> lock-group_0 localhost_12000 >>>> lock-group_1 localhost_12001 >>>> lock-group_10 localhost_12002 >>>> lock-group_11 localhost_12000 >>>> lock-group_2 localhost_12001 >>>> lock-group_3 localhost_12002 >>>> lock-group_4 localhost_12000 >>>> lock-group_5 localhost_12001 >>>> lock-group_6 localhost_12002 >>>> lock-group_7 localhost_12000 >>>> lock-group_8 localhost_12001 >>>> lock-group_9 localhost_12002 >>>> >>>> >>>> On Sat, Feb 24, 2018 at 8:26 PM, Lei Xia <[email protected]> wrote: >>>> >>>>> Hi, Utsav >>>>> >>>>> Delay rebalancer by default is disabled in cluster level (this is to >>>>> keep back-compatible somehow), you need to enable it in the clusterConfig, >>>>> e.g >>>>> >>>>> ConfigAccessor configAccessor = new ConfigAccessor(zkClient); >>>>> ClusterConfig clusterConfig = >>>>> configAccessor.getClusterConfig(clusterName); >>>>> clusterConfig.setDelayRebalaceEnabled(enabled); >>>>> configAccessor.setClusterConfig(clusterName, clusterConfig); >>>>> >>>>> >>>>> Could you please have a try and let me know whether it works or >>>>> not? Thanks >>>>> >>>>> >>>>> Lei >>>>> >>>>> >>>>> On Fri, Feb 23, 2018 at 2:33 PM, Utsav Kanani <[email protected]> >>>>> wrote: >>>>> >>>>>> I am trying to expand the Lockmanager example http://helix.apache.or >>>>>> g/0.6.2-incubating-docs/recipes/lock_manager.html to introduce delay >>>>>> >>>>>> tried doing something like this >>>>>> IdealState state = admin.getResourceIdealState(clusterName, >>>>>> lockGroupName); >>>>>> state.setRebalanceDelay(100000); >>>>>> state.setDelayRebalanceEnabled(true); >>>>>> state.setRebalancerClassName(DelayedAutoRebalancer.class.ge >>>>>> tName()); >>>>>> admin.setResourceIdealState(clusterName, lockGroupName, state); >>>>>> admin.rebalance(clusterName, lockGroupName, 1); >>>>>> >>>>>> On killing a node there is immediate rebalancing which takes place. I >>>>>> was hoping for a delay of 100 seconds before rebalancing but i am not >>>>>> seeing that behavior >>>>>> >>>>>> >>>>>> On Stopping localhost_12000 the locks are acquired immediately by >>>>>> localhost_12001 and localhost_12002 >>>>>> >>>>>> on STARTING localhost_12000 the rebalance is again immediate. >>>>>> >>>>>> localhost_12000 acquired lock:lock-group_11 >>>>>> localhost_12000 acquired lock:lock-group_7 >>>>>> localhost_12000 acquired lock:lock-group_0 >>>>>> localhost_12000 acquired lock:lock-group_4 >>>>>> STARTED localhost_12000 >>>>>> localhost_12001 releasing lock:lock-group_0 >>>>>> localhost_12001 releasing lock:lock-group_11 >>>>>> localhost_12002 releasing lock:lock-group_4 >>>>>> localhost_12002 releasing lock:lock-group_7 >>>>>> >>>>>> >>>>>> Here is the output >>>>>> ========================================= >>>>>> >>>>>> STARTING localhost_12000 >>>>>> STARTING localhost_12001 >>>>>> STARTING localhost_12002 >>>>>> STARTED localhost_12001 >>>>>> STARTED localhost_12002 >>>>>> STARTED localhost_12000 >>>>>> localhost_12000 acquired lock:lock-group_11 >>>>>> localhost_12002 acquired lock:lock-group_10 >>>>>> localhost_12002 acquired lock:lock-group_9 >>>>>> localhost_12002 acquired lock:lock-group_3 >>>>>> localhost_12001 acquired lock:lock-group_2 >>>>>> localhost_12001 acquired lock:lock-group_1 >>>>>> localhost_12001 acquired lock:lock-group_8 >>>>>> localhost_12002 acquired lock:lock-group_6 >>>>>> localhost_12000 acquired lock:lock-group_4 >>>>>> localhost_12000 acquired lock:lock-group_0 >>>>>> localhost_12000 acquired lock:lock-group_7 >>>>>> localhost_12001 acquired lock:lock-group_5 >>>>>> lockName acquired By >>>>>> ====================================== >>>>>> lock-group_0 localhost_12000 >>>>>> lock-group_1 localhost_12001 >>>>>> lock-group_10 localhost_12002 >>>>>> lock-group_11 localhost_12000 >>>>>> lock-group_2 localhost_12001 >>>>>> lock-group_3 localhost_12002 >>>>>> lock-group_4 localhost_12000 >>>>>> lock-group_5 localhost_12001 >>>>>> lock-group_6 localhost_12002 >>>>>> lock-group_7 localhost_12000 >>>>>> lock-group_8 localhost_12001 >>>>>> lock-group_9 localhost_12002 >>>>>> Stopping localhost_12000 >>>>>> localhost_12000Interrupted >>>>>> localhost_12002 acquired lock:lock-group_4 >>>>>> localhost_12001 acquired lock:lock-group_11 >>>>>> localhost_12002 acquired lock:lock-group_7 >>>>>> localhost_12001 acquired lock:lock-group_0 >>>>>> lockName acquired By >>>>>> ====================================== >>>>>> lock-group_0 localhost_12001 >>>>>> lock-group_1 localhost_12001 >>>>>> lock-group_10 localhost_12002 >>>>>> lock-group_11 localhost_12001 >>>>>> lock-group_2 localhost_12001 >>>>>> lock-group_3 localhost_12002 >>>>>> lock-group_4 localhost_12002 >>>>>> lock-group_5 localhost_12001 >>>>>> lock-group_6 localhost_12002 >>>>>> lock-group_7 localhost_12002 >>>>>> lock-group_8 localhost_12001 >>>>>> lock-group_9 localhost_12002 >>>>>> ===Starting localhost_12000 >>>>>> STARTING localhost_12000 >>>>>> localhost_12000 acquired lock:lock-group_11 >>>>>> localhost_12000 acquired lock:lock-group_7 >>>>>> localhost_12000 acquired lock:lock-group_0 >>>>>> localhost_12000 acquired lock:lock-group_4 >>>>>> STARTED localhost_12000 >>>>>> localhost_12001 releasing lock:lock-group_0 >>>>>> localhost_12001 releasing lock:lock-group_11 >>>>>> localhost_12002 releasing lock:lock-group_4 >>>>>> localhost_12002 releasing lock:lock-group_7 >>>>>> lockName acquired By >>>>>> ====================================== >>>>>> lock-group_0 localhost_12000 >>>>>> lock-group_1 localhost_12001 >>>>>> lock-group_10 localhost_12002 >>>>>> lock-group_11 localhost_12000 >>>>>> lock-group_2 localhost_12001 >>>>>> lock-group_3 localhost_12002 >>>>>> lock-group_4 localhost_12000 >>>>>> lock-group_5 localhost_12001 >>>>>> lock-group_6 localhost_12002 >>>>>> lock-group_7 localhost_12000 >>>>>> lock-group_8 localhost_12001 >>>>>> lock-group_9 localhost_12002 >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> Lei Xia >>>>> >>>> >>>> >>>> >>>> On Sat, Feb 24, 2018 at 8:26 PM, Lei Xia <[email protected]> wrote: >>>> >>>>> Hi, Utsav >>>>> >>>>> Delay rebalancer by default is disabled in cluster level (this is to >>>>> keep back-compatible somehow), you need to enable it in the clusterConfig, >>>>> e.g >>>>> >>>>> ConfigAccessor configAccessor = new ConfigAccessor(zkClient); >>>>> ClusterConfig clusterConfig = >>>>> configAccessor.getClusterConfig(clusterName); >>>>> clusterConfig.setDelayRebalaceEnabled(enabled); >>>>> configAccessor.setClusterConfig(clusterName, clusterConfig); >>>>> >>>>> >>>>> Could you please have a try and let me know whether it works or >>>>> not? Thanks >>>>> >>>>> >>>>> Lei >>>>> >>>>> >>>>> On Fri, Feb 23, 2018 at 2:33 PM, Utsav Kanani <[email protected]> >>>>> wrote: >>>>> >>>>>> I am trying to expand the Lockmanager example >>>>>> http://helix.apache.org/0.6.2-incubating-docs/recipes/lock_m >>>>>> anager.html to introduce delay >>>>>> >>>>>> tried doing something like this >>>>>> IdealState state = admin.getResourceIdealState(clusterName, >>>>>> lockGroupName); >>>>>> state.setRebalanceDelay(100000); >>>>>> state.setDelayRebalanceEnabled(true); >>>>>> state.setRebalancerClassName(DelayedAutoRebalancer.class.ge >>>>>> tName()); >>>>>> admin.setResourceIdealState(clusterName, lockGroupName, state); >>>>>> admin.rebalance(clusterName, lockGroupName, 1); >>>>>> >>>>>> On killing a node there is immediate rebalancing which takes place. I >>>>>> was hoping for a delay of 100 seconds before rebalancing but i am not >>>>>> seeing that behavior >>>>>> >>>>>> >>>>>> On Stopping localhost_12000 the locks are acquired immediately by >>>>>> localhost_12001 and localhost_12002 >>>>>> >>>>>> on STARTING localhost_12000 the rebalance is again immediate. >>>>>> >>>>>> localhost_12000 acquired lock:lock-group_11 >>>>>> localhost_12000 acquired lock:lock-group_7 >>>>>> localhost_12000 acquired lock:lock-group_0 >>>>>> localhost_12000 acquired lock:lock-group_4 >>>>>> STARTED localhost_12000 >>>>>> localhost_12001 releasing lock:lock-group_0 >>>>>> localhost_12001 releasing lock:lock-group_11 >>>>>> localhost_12002 releasing lock:lock-group_4 >>>>>> localhost_12002 releasing lock:lock-group_7 >>>>>> >>>>>> >>>>>> Here is the output >>>>>> ========================================= >>>>>> >>>>>> STARTING localhost_12000 >>>>>> STARTING localhost_12001 >>>>>> STARTING localhost_12002 >>>>>> STARTED localhost_12001 >>>>>> STARTED localhost_12002 >>>>>> STARTED localhost_12000 >>>>>> localhost_12000 acquired lock:lock-group_11 >>>>>> localhost_12002 acquired lock:lock-group_10 >>>>>> localhost_12002 acquired lock:lock-group_9 >>>>>> localhost_12002 acquired lock:lock-group_3 >>>>>> localhost_12001 acquired lock:lock-group_2 >>>>>> localhost_12001 acquired lock:lock-group_1 >>>>>> localhost_12001 acquired lock:lock-group_8 >>>>>> localhost_12002 acquired lock:lock-group_6 >>>>>> localhost_12000 acquired lock:lock-group_4 >>>>>> localhost_12000 acquired lock:lock-group_0 >>>>>> localhost_12000 acquired lock:lock-group_7 >>>>>> localhost_12001 acquired lock:lock-group_5 >>>>>> lockName acquired By >>>>>> ====================================== >>>>>> lock-group_0 localhost_12000 >>>>>> lock-group_1 localhost_12001 >>>>>> lock-group_10 localhost_12002 >>>>>> lock-group_11 localhost_12000 >>>>>> lock-group_2 localhost_12001 >>>>>> lock-group_3 localhost_12002 >>>>>> lock-group_4 localhost_12000 >>>>>> lock-group_5 localhost_12001 >>>>>> lock-group_6 localhost_12002 >>>>>> lock-group_7 localhost_12000 >>>>>> lock-group_8 localhost_12001 >>>>>> lock-group_9 localhost_12002 >>>>>> Stopping localhost_12000 >>>>>> localhost_12000Interrupted >>>>>> localhost_12002 acquired lock:lock-group_4 >>>>>> localhost_12001 acquired lock:lock-group_11 >>>>>> localhost_12002 acquired lock:lock-group_7 >>>>>> localhost_12001 acquired lock:lock-group_0 >>>>>> lockName acquired By >>>>>> ====================================== >>>>>> lock-group_0 localhost_12001 >>>>>> lock-group_1 localhost_12001 >>>>>> lock-group_10 localhost_12002 >>>>>> lock-group_11 localhost_12001 >>>>>> lock-group_2 localhost_12001 >>>>>> lock-group_3 localhost_12002 >>>>>> lock-group_4 localhost_12002 >>>>>> lock-group_5 localhost_12001 >>>>>> lock-group_6 localhost_12002 >>>>>> lock-group_7 localhost_12002 >>>>>> lock-group_8 localhost_12001 >>>>>> lock-group_9 localhost_12002 >>>>>> ===Starting localhost_12000 >>>>>> STARTING localhost_12000 >>>>>> localhost_12000 acquired lock:lock-group_11 >>>>>> localhost_12000 acquired lock:lock-group_7 >>>>>> localhost_12000 acquired lock:lock-group_0 >>>>>> localhost_12000 acquired lock:lock-group_4 >>>>>> STARTED localhost_12000 >>>>>> localhost_12001 releasing lock:lock-group_0 >>>>>> localhost_12001 releasing lock:lock-group_11 >>>>>> localhost_12002 releasing lock:lock-group_4 >>>>>> localhost_12002 releasing lock:lock-group_7 >>>>>> lockName acquired By >>>>>> ====================================== >>>>>> lock-group_0 localhost_12000 >>>>>> lock-group_1 localhost_12001 >>>>>> lock-group_10 localhost_12002 >>>>>> lock-group_11 localhost_12000 >>>>>> lock-group_2 localhost_12001 >>>>>> lock-group_3 localhost_12002 >>>>>> lock-group_4 localhost_12000 >>>>>> lock-group_5 localhost_12001 >>>>>> lock-group_6 localhost_12002 >>>>>> lock-group_7 localhost_12000 >>>>>> lock-group_8 localhost_12001 >>>>>> lock-group_9 localhost_12002 >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> Lei Xia >>>>> >>>> >>>> >>> >>> >>> -- >>> Lei Xia >>> >> >> >
