Changing/removing entities in ZK is highly undesirable. We still want old-version controllers to manage new-version participants, and vice versa. Making a change like that would require system-wide upgrades and downtime.
What we can do is change how the entities are exposed to the users through an API. That is really the goal here: avoid changing the backing store, but rethink how the user interacts with it. Kanak ---------------------------------------- > Date: Sun, 23 Feb 2014 11:22:55 -0800 > Subject: Re: [DISCUSS] New Helix Api > From: [email protected] > To: [email protected] > > Hey guys, > > I understand we want to quickly move to defining APIs but I am hoping > the entities defined in the first mail are not a closed decision and > is open for discussion > > So here is the list of entities defined in the first mail > > Entities > -- Idealstate > -- ExternalView > -- CurrentState > -- LiveInstance > -- HelixConfig > -- UserConfig > -- StateModelDefinition > -- Alerts > > I feel like there is a overlap of a lot of different constructs here > which are not really entities. > > The basic entities in Helix IMO should be something like > > * Entities > -- Cluster > -- Instance > -- Resource //I imagine replicas are a specialization of a resource so > dont seem to be first class entities > -- Partition > -- User? //not clear how this is modelled but I put a place holder, > probably not needed > > I imagine given that we could have multiple clusters, we may need > something which contains clusters i.e. a HelixSystem entity. > > * Entities have configurations (there are again two perspectives to > configurations) > -- Configuration passed in during creation i.e. which is not yet realized > -- Runtime configuration of the entity i.e. the current values in the > system which are realized > > * Entities may or may not have a role > -- This is where all the instance roles come into play > > * Entities have a state > > Given that we are redefining things anyway what do you guys think > about this perspective? > > Thanks, > > Sandeep > > On Sun, Feb 23, 2014 at 10:20 AM, Kanak Biscuitwala <[email protected]> > wrote: >> Entities are the basic units of information necessary in order for Helix to >> function. It's the "language" Helix speaks. If you look at ZooKeeper, this >> is what you would see. >> >> Scopes are the levels at which things can be configured. That is to say, you >> can configure the entire cluster, a single resource, a participant, etc. >> >> Roles are the way nodes are labeled in the system. Currently Helix nodes can >> have the following roles: Participant, Spectator, Controller, and >> Administrator. A single machine may take on multiple roles. >> >> The purpose of the API is to sufficiently abstract the entities at >> appropriate scopes for each role. >> >> When we started brainstorming APIs, we identified the following use cases: >> configuration, registration, state transitions, and rebalancing. Here is a >> take on what part of configuration and registration could look like: >> >> Administrator admin = new ZKHelixAdministrator(); >> admin.connect(); >> ClusterConfig = new ClusterConfig().setName().setuserproperties(); >> admin.createCluster(clusterconfig); >> StateModelDef def = new MasterSlaveModel(); >> ResourceConfig resourceConfig = new >> ResourceConfig(resourceId).withFSM(def).partitioned(24).replicated(3).withRebalancer(RebalancerConfig); >> admin.addResource(resourceConfig); >> >> //add Instances, we shud probably call this ParticipantConfig >> ParticipantConfig instanceConfig = new ParticipantConfig(); >> >> // We can probably have controllerService that wraps controller -- this can >> help people who want to control which events trigger the pipeline >> >> // not StandaloneController? no -- standalone means its only one controller, >> people think its not fault tolerant >> SingleClusterController controller = new SingleClusterController(); >> controller.start(); >> >> // controls multiple clusters >> MultiClusterController controller = new MultiClusterController(); >> controller.start(); >> >> Participant participant = new Participant(); >> ParticipantService service = new PartiticipantService(participant); >> service.start(); // service has some callbacks init/start getListeners >> callback that will be called appropriately >> >> Specator spectator = new Spectator(); >> SpecatorService service = new SpectatorService().watch(resource(s)); >> service.start(); //similar to participant gives a callback to get the >> listeners to be added >> >> ---------------------------------------- >>> Date: Sun, 23 Feb 2014 09:51:01 -0800 >>> Subject: Re: [DISCUSS] New Helix Api >>> From: [email protected] >>> To: [email protected] >>> >>> Hey guys, >>> >>> For the sake of the newcomer could you please define/describe what you >>> mean by the following >>> >>> (a) Entity >>> (b) Scope >>> >>> I have some thoughts on the entities etc but wanted to understand what >>> their definitions are before I suggest something. >>> >>> Thanks, >>> >>> Sandeep >>> >>> On Sun, Feb 23, 2014 at 7:54 AM, kishore g <[email protected]> wrote: >>>> Hi, >>>> >>>> Here are the entities, scopes and roles defined in Helix. >>>> Entities >>>> -- Idealstate >>>> -- ExternalView >>>> -- CurrentState >>>> -- LiveInstance >>>> -- HelixConfig >>>> -- UserConfig >>>> -- StateModelDefinition >>>> -- Alerts >>>> >>>> >>>> Scopes >>>> -- Cluster >>>> -- Participant/Group >>>> -- Resource/Group >>>> -- Partition >>>> -- Transition >>>> >>>> Misc >>>> -- One time read v/s cache v/s cache and update automatically on >>>> notification >>>> >>>> ==================== >>>> HelixConfig >>>> -- Cluster >>>> -- Participant >>>> -- Resource >>>> -- Partition >>>> >>>> UserConfig >>>> -- Cluster >>>> -- Participant >>>> -- Resource >>>> -- Partition >>>> >>>> StateModelDefinition >>>> -- States >>>> -- Transitions >>>> -- Constraints (Partition only) >>>> >>>> >>>> Constraints >>>> -- Cluster >>>> -- Node >>>> -- Resource >>>> -- Partition >>>> -- Resource/ResourceGroup >>>> >>>> Kanak and I did one pass on this and made some changes. Kanak, do you have >>>> send across the changed version. >>>> >>>> We need to come up with the user facing api's that support simple CRUD on >>>> these entities. >>>> >>>> thanks, >>>> Kishore G
