For your issue, 1) how do I specify full auto mode using ./helix-admin.sh command? I don't see mode parameter (may be am I missing something?) You need -mode FULL_AUTO to let the resource be automatically assigned by Helix with out your preference list.
2) Is there a way to specify delay for state transition using ./helix-admin.sh ? Delay state transitions only when you using delay rebalancer. As I replied your last email, firstly you need to set delay time in ClusterConfig. Then you need to change your rebalancer to be DelayedRebalancer class in your ideal state. You need to get your ideal state and change the field and update to ZK. Command line helix admin does not support it. 3) I tried other combination of 6 partitions with 3 replicas. Initially Ideal State and External view client looks same. but after some time node 8086 showing offline state for some partitions. But surprisingly I don't see any logs in my state transitions model class from any transitions to offline. what could be the issue on why call backs are missing? You need to check your controller logs to see whether there is a state transition message for SLAVE -> OFFLINE sent to that 8086 participant. Then try to figure out why the message is sent. If you dont find the message, it means your participant encounter network outage or instance is down or something cause ZK lost connection. Your participant is not blocking call for start. It will be finished as your function finishes. You can follow our MockParticipant code: https://github.com/apache/helix/blob/master/helix-core/src/test/java/org/apache/helix/integration/manager/MockParticipantManager.java Best, Junkai On Fri, Nov 3, 2017 at 7:08 PM, leela maheswararao < leela_mahesh...@yahoo.com.invalid> wrote: > The set up I have used is, > #Define the Resource and Partitioning#Create a Database with 6 Partitions > using the MasterSlave State Model > ./helix-admin.sh --zkSvr ${zkSvr} --addResource ${CLUSTER} ${RESOURCE} 3 > MasterSlave > #Let Helix Assign Partitions to Nodes > ./helix-admin.sh --zkSvr ${zkSvr} --rebalance ${CLUSTER} ${RESOURCE} 3 > The above command is creating resource in Semi auto mode by default. > Open Issues:1) how do I specify full auto mode using ./helix-admin.sh > command? I don't see mode parameter (may be am I missing something?) > 2) Is there a way to specify delay for state transition using > ./helix-admin.sh ? > 3) I tried other combination of 6 partitions with 3 replicas. Initially > Ideal State and External view client looks same. but after some time node > 8086 showing offline state for some partitions. But surprisingly I don't > see any logs in my state transitions model class from any transitions to > offline. what could be the issue on why call backs are missing? > json snippet from listResourceInfo > > lrao-ltm:bin lrao$ ./helix-admin.sh --zkSvr localhost:2181 > --listResourceInfo C1 R1 > > IdealState for R1: > > { > > "id" : "R1", > > "mapFields" : { > > "R1_0" : { > > "localhost_8085" : "SLAVE", > > "localhost_8086" : "MASTER", > > "localhost_8087" : "SLAVE" > > }, > > "R1_1" : { > > "localhost_8085" : "SLAVE", > > "localhost_8086" : "SLAVE", > > "localhost_8087" : "MASTER" > > }, > > "R1_2" : { > > "localhost_8085" : "MASTER", > > "localhost_8086" : "SLAVE", > > "localhost_8087" : "SLAVE" > > }, > > "R1_3" : { > > "localhost_8085" : "SLAVE", > > "localhost_8086" : "SLAVE", > > "localhost_8087" : "MASTER" > > }, > > "R1_4" : { > > "localhost_8085" : "MASTER", > > "localhost_8086" : "SLAVE", > > "localhost_8087" : "SLAVE" > > }, > > "R1_5" : { > > "localhost_8085" : "SLAVE", > > "localhost_8086" : "MASTER", > > "localhost_8087" : "SLAVE" > > } > > }, > > "listFields" : { > > "R1_0" : [ "localhost_8086", "localhost_8085", "localhost_8087" ], > > "R1_1" : [ "localhost_8087", "localhost_8085", "localhost_8086" ], > > "R1_2" : [ "localhost_8085", "localhost_8087", "localhost_8086" ], > > "R1_3" : [ "localhost_8087", "localhost_8086", "localhost_8085" ], > > "R1_4" : [ "localhost_8085", "localhost_8087", "localhost_8086" ], > > "R1_5" : [ "localhost_8086", "localhost_8087", "localhost_8085" ] > > }, > > "simpleFields" : { > > "IDEAL_STATE_MODE" : "AUTO", > > "NUM_PARTITIONS" : "6", > > "REBALANCE_MODE" : "SEMI_AUTO", > > "REBALANCE_STRATEGY" : "DEFAULT", > > "REPLICAS" : "3", > > "STATE_MODEL_DEF_REF" : "MasterSlave", > > "STATE_MODEL_FACTORY_NAME" : "DEFAULT" > > } > > } > > > > > ExternalView for R1: > > { > > "id" : "R1", > > "mapFields" : { > > "R1_0" : { > > "localhost_8085" : "SLAVE", > > "localhost_8086" : "OFFLINE", > > "localhost_8087" : "SLAVE" > > }, > > "R1_1" : { > > "localhost_8085" : "SLAVE", > > "localhost_8086" : "SLAVE", > > "localhost_8087" : "MASTER" > > }, > > "R1_2" : { > > "localhost_8085" : "SLAVE", > > "localhost_8086" : "OFFLINE", > > "localhost_8087" : "SLAVE" > > }, > > "R1_3" : { > > "localhost_8085" : "SLAVE", > > "localhost_8086" : "SLAVE", > > "localhost_8087" : "MASTER" > > }, > > "R1_4" : { > > "localhost_8085" : "SLAVE", > > "localhost_8086" : "OFFLINE", > > "localhost_8087" : "SLAVE" > > }, > > "R1_5" : { > > "localhost_8085" : "SLAVE", > > "localhost_8086" : "SLAVE", > > "localhost_8087" : "MASTER" > > } > > }, > > "listFields" : { > > }, > > "simpleFields" : { > > "BUCKET_SIZE" : "0", > > "IDEAL_STATE_MODE" : "AUTO", > > "NUM_PARTITIONS" : "6", > > "REBALANCE_MODE" : "SEMI_AUTO", > > "REBALANCE_STRATEGY" : "DEFAULT", > > "REPLICAS" : "3", > > "STATE_MODEL_DEF_REF" : "MasterSlave", > > "STATE_MODEL_FACTORY_NAME" : "DEFAULT" > > } > > } > > My participant class code: > > public class HelixParticipant { > > > > > > > > String clusterName; > > String instanceName; > > String zkConnectString; > > SconeVagabondHandler stuff; > > > > > > > public HelixParticipant(String clusterName, String instanceName, String > zkConnectString, SconeVagabondHandler stuff) { > > super(); > > this.clusterName = clusterName; > > this.instanceName = instanceName; > > this.zkConnectString = zkConnectString; > > this.stuff = stuff; > > > > } > > > > > public void start() { > > try { > > HelixManager manager = HelixManagerFactory.getZKHelixManager(clusterName, > instanceName, > > InstanceType.PARTICIPANT, zkConnectString); > > StateMachineEngine stateMach = manager.getStateMachineEngine(); > > RoutingTableProvider routingTableProvider = new RoutingTableProvider(); > > MasterSlaveStateModelFactory stateModelFactory = new > MasterSlaveStateModelFactory(stuff,routingTableProvider); > > stateMach.registerStateModelFactory("MasterSlave", stateModelFactory); > > manager.connect(); > > manager.addExternalViewChangeListener(routingTableProvider); > > > > > > > > } catch (Exception ex) { > > logger.log(Level.SEVERE, "Error during Participant start", ex); > > } > > } > > > > > } > Master slave model: > > public class MasterSlaveStateModelFactory > > extends StateModelFactory<StateModel> { > > > > > > > > SconeVagabondHandler stuff; > > RoutingTableProvider routingTableProvider; > > > > public MasterSlaveStateModelFactory(SconeVagabondHandler stuff, > RoutingTableProvider routingTableProvider) { > > this.stuff = stuff; > > this.routingTableProvider = routingTableProvider; > > } > > > > > @Override > > public StateModel createNewStateModel(String resourceName, String > stateUnitKey) { > > logger.info( > > "MasterSlaveStateModelFactory => " + "resourceName " + resourceName + > " stateUnitKey " + stateUnitKey); > > return new MasterSlaveStateModel(stateUnitKey, stuff, > routingTableProvider); > > } > > > > > @StateModelInfo(initialState = "OFFLINE", states = > "{'OFFLINE','SLAVE','MASTER', 'DROPPED'}") > > public static class MasterSlaveStateModel extends StateModel { > > > > > private final String _stateUnitKey; > > SconeHandler stuff; > > RoutingTableProvider routingTableProvider; > > > > > > > public MasterSlaveStateModel(String stateUnitKey, SconeHandler > stuff,RoutingTableProvider routingTableProvider) { > > _stateUnitKey = stateUnitKey; > > this.stuff = stuff; > > this.routingTableProvider = routingTableProvider; > > } > > > > > @Transition(from = "MASTER", to = "SLAVE") > > public void masterToSlave(Message message, NotificationContext context) { > > logger.info("BootstrapProcess.BootstrapStateModel.masterToSlave()"); > > } > > > > > @Transition(from = "OFFLINE", to = "SLAVE") > > public void offlineToSlave(Message message, NotificationContext context) > { > > logger.info("BootstrapProcess.BootstrapStateModel.offlineToSlave()"); > > > > } > > > > > @Transition(from = "SLAVE", to = "OFFLINE") > > public void slaveToOffline(Message message, NotificationContext context) > { > > logger.info("BootstrapProcess.BootstrapStateModel.slaveToOffline()"); > > } > > > > > @Transition(from = "SLAVE", to = "MASTER") > > public void slaveToMaster(Message message, NotificationContext context) { > > logger.info("BootstrapProcess.BootstrapStateModel.slaveToMaster()"); > > } > > > > > @Transition(from = "OFFLINE", to = "DROPPED") > > public void onBecomeDroppedFromOffline(Message message, > NotificationContext context) { > > logger.info("BootstrapProcess.BootstrapStateModel.offlineToDropped()"); > > } > > > > > } > > } > > Controller class: > > public class HelixController { > > > > > > > String clusterName; > > String instanceName; > > String zkConnectString; > > > > public HelixController(String clusterName, String instanceName, String > zkConnectString) { > > super(); > > this.clusterName = clusterName; > > this.instanceName = instanceName; > > this.zkConnectString = zkConnectString; > > } > > > > public void start(){ > > try{ > > HelixManager manager = HelixManagerFactory. > getZKHelixManager(clusterName, > > instanceName, > > InstanceType.CONTROLLER, > > zkConnectString); > > manager.connect(); > > GenericHelixController controller = new GenericHelixController(); > > manager.addConfigChangeListener(controller); > > manager.addLiveInstanceChangeListener(controller); > > manager.addIdealStateChangeListener(controller); > > //manager..addExternalViewChangeListener(controller); > > manager.addControllerListener(controller); > > }catch(Exception ex){ > > logger.log(Level.SEVERE, "Error during Controller start", ex); > > } > > } > > > > > > } > > do you see any issues with my above classes? > On Saturday, November 4, 2017, 6:13:46 AM GMT+5:30, Xue Junkai < > j...@apache.org> wrote: > > Hi Leela, > > Are your resources all in SemiAuto mode? SemiAuto mode assignment is based > on your preference list in your IdealState list fields. If you want to > spread it out, you have to change your preference list node order for > different partitions. > > If you are using FullAuto mode, you will not have this problem. > > best, > > Junkai > > On Fri, Nov 3, 2017 at 5:35 PM, leela maheswararao < > leela_mahesh...@yahoo.com.invalid> wrote: > > > Team,In my example(SEMI_AUTO), I have configured 3 partitions with 3 > > replicas on 3 nodes. Once all nodes are started, external view shows as > > below. Here all MASTER's are assigned to same node. Is there a way to > > spread MASTER's evenly? is this behavior due to server booting order? > > > > IdealState for R1: > > > > { > > > > "id" : "R1", > > > > "mapFields" : { > > > > "R1_0" : { > > > > "localhost_8085" : "MASTER", > > > > "localhost_8086" : "SLAVE", > > > > "localhost_8087" : "SLAVE" > > > > }, > > > > "R1_1" : { > > > > "localhost_8085" : "SLAVE", > > > > "localhost_8086" : "SLAVE", > > > > "localhost_8087" : "MASTER" > > > > }, > > > > "R1_2" : { > > > > "localhost_8085" : "SLAVE", > > > > "localhost_8086" : "MASTER", > > > > "localhost_8087" : "SLAVE" > > > > } > > > > }, > > > > "listFields" : { > > > > "R1_0" : [ "localhost_8085", "localhost_8086", "localhost_8087" ], > > > > "R1_1" : [ "localhost_8087", "localhost_8085", "localhost_8086" ], > > > > "R1_2" : [ "localhost_8086", "localhost_8085", "localhost_8087" ] > > > > }, > > > > "simpleFields" : { > > > > "IDEAL_STATE_MODE" : "AUTO", > > > > "NUM_PARTITIONS" : "3", > > > > "REBALANCE_MODE" : "SEMI_AUTO", > > > > "REBALANCE_STRATEGY" : "DEFAULT", > > > > "REPLICAS" : "3", > > > > "STATE_MODEL_DEF_REF" : "MasterSlave", > > > > "STATE_MODEL_FACTORY_NAME" : "DEFAULT" > > > > } > > > > } > > > > > > > > > > ExternalView for R1: > > > > { > > > > "id" : "R1", > > > > "mapFields" : { > > > > "R1_0" : { > > > > "localhost_8085" : "SLAVE", > > > > "localhost_8086" : "MASTER", > > > > "localhost_8087" : "SLAVE" > > > > }, > > > > "R1_1" : { > > > > "localhost_8085" : "SLAVE", > > > > "localhost_8086" : "MASTER", > > > > "localhost_8087" : "SLAVE" > > > > }, > > > > "R1_2" : { > > > > "localhost_8085" : "SLAVE", > > > > "localhost_8086" : "MASTER", > > > > "localhost_8087" : "SLAVE" > > > > } > > > > }, > > > > "listFields" : { > > > > }, > > > > "simpleFields" : { > > > > "BUCKET_SIZE" : "0", > > > > "IDEAL_STATE_MODE" : "AUTO", > > > > "NUM_PARTITIONS" : "3", > > > > "REBALANCE_MODE" : "SEMI_AUTO", > > > > "REBALANCE_STRATEGY" : "DEFAULT", > > > > "REPLICAS" : "3", > > > > "STATE_MODEL_DEF_REF" : "MasterSlave", > > > > "STATE_MODEL_FACTORY_NAME" : "DEFAULT" > > > > } > > > > } > > Regards,Mahesh > > >
