Thanks Vanson for the detailed note. It narrowed down the problem. +1 for using group level deployment policy to provide the min/max of group instances only. Cartridge instances min/max can be specified by the respective cartridge in cartridge deployment policy according whether it is R1/R2/R3. Hope that it will clear out your confusion.
Yah..In order to support these patterns, we will need to satisfy R1, R2 and R3 requirements as you mentioned by our deployment policy definition and by the implementation. Thanks, Reka On Wed, Apr 15, 2015 at 8:04 AM, Vanson Lim <[email protected]> wrote: > On 4/9/15, 8:19 AM, Reka Thirunavukkarasu wrote: > > Hi All, > > We have discussed about the handling deployment policy in group level and > caridge level in multiple other mails. I have started to look into > https://issues.apache.org/jira/browse/STRATOS-1297 and thought of > discussing my concerns before implementing/fixing the flow. Sorry for the > long mail as i couldn't explain the concept in a short mail. > > The current Design on Deployment Policy > ------------------------------------------------------- > > We can have deployment policy in all the leaf level as in cartridge > level. Since defining cartrdigeMin and cartiridgeMax along with defining > partitionMax in the deployment policy complicates the user, we are going to > go with defining partitionMin and partitionMax in the deployment policy for > the cartridge as below. > > { > "id": "deployment-policy-1", > "networkPartitions": [ > { > "id": "network-partition-1", > "partitionAlgo": "one-after-another", > "partitions": [ > { > "id": "partition-1", > "partitionMin" : 2, > "partitionMax": 20 > } > ] > } > ] > } > > When we have the above deployment policy in each cartridge level of an > application, each cluster instance can refer to that and adhere to the > min/max members as defined in the policy. > > Since we have Group scaling/high availability for the groups by > supporting group instances concept, when you deploy an application, an > application instance will get created. Then based on the definition of > application, Group instance for a group and cluster instance for a > cartridge will get created where group instance. When Group scaling is > enabled or there is a chance to create multiple group instances, then we > can define the group instance deployment pattern as below: > > Eg: G1 has C1 and C2 where one GroupInstnace of G1 has two C1 members > and two C2 members. > > > > *Pattern-1* > > This is the high availability pattern for the group where one group > instance will be reside in one partition. In order to deploy in pattern-1, > we will need a deployment policy in the Group level. If we define a > Deployment Policy in the group level, then underlying cartridges should use > the algorithm and the partition selected by the parent in order to create > new members. Then only the members of the same group instance can be > deployed into the same partition. The sample Group level policy would be as > below: > > { > "id": "deployment-policy-G1", > "networkPartitions": [ > { > "id": "network-partition-1", > "partitionAlgo": "round-robin", > "partitions": [ > { > "id": "P1", > "partitionMin" : 1, > "partitionMax": 2 > }, > { > "id": "P2", > "partitionMin" : 1, > "partitionMax": 2 > } > ] > } > ] > } > > With this policy, two group instances(G11 and G12) gets created in P1 > and P2 respectively. Since we don't define another deployment policy in > cartridge level, cartridges need to use the same min/max defined in the > group level deployment policy. This will be a limitation as cartridges are > unable to enforce their own min/max.* How can the underlying cartridges > created their member without a min/max in the cartridge level?* *Would > that make sense to read the Group level deployment policy for the > underlying cartridges as well?* > > Pattern-2 > > In order to deploy the group instances across partitions, we don't need > a deployment policy in the group level. The partitions will be decided by > the children. But we will need to specify GroupMinInstances and > GroupMaxInstances in the group level. Then only we can control the number > group instances to be created. Since we are moving from defining > GroupMinInstances and GroupMaxInstances in the application, how can we > handle this as we don't need a policy for this case? *Can we keep the > GroupMinInstances and GroupMaxInstances in the application?* > > > Please provide your feedback on migrating from > cartridgeMin/cartridgeMax to deployment policy's partitionMin/partitionMax > as above two patterns need to be fixed properly. If anyone aware of more > real use cases, please let me know. It would help improving this model > further to fit with more real use cases. > > Thanks, > Reka > > > > > > Reka, > > For the use cases I've seen I think it's valuable (essential) to be able > to control the min/max instances of a cartridge at the cartridge level as > shown in deployment pattern 2. When C1 and C2 are unlike cartridges, we > need to be able to individually control the min and max instances of each > cartridge independently. For example, if C1=web server and C2=cassandra > databases, I can't imagine ever having the same number web servers as > there are database backends. > > I can also see value with pattern 1 for some uses cases where you want the > instances of a group to be on the same partition and achieve high > availability by having multiple instances of groups spread across > partitions. We don't currently plan to use pattern 1 for our uses cases > but I also don't want to be too restrictive on the implementation such that > it is inflexible for others who may think of other uses cases. > > > A few points I find confusing: > 1) When we have a group deployment policy, why doesn't the > PartitionMin/Max value apply to the individual cartridge instances instead > of the instances of the group? > > 2) It seems to me that the cartridge deployment policy should override any > group level definitions of min/max instances. Defining of partitionMin/Max > at the group level and applying it to the number of cartridge instances > instead of group instances just seems wrong. > > > We seem to have the following requirements if we want to support both > pattern 1 and 2. > > R1) if we want control VM placement at the group policy level (pattern 1) > > a) define deployment number of group instances at the policy with > partitions. Partition min max at in the group policy defines the number > of group instances per partition. > b) support only defining min/max instance per cartridge, not partition. > > R2) if we want to control VM placement at the cartridge level (pattern 2) > > a) group level deployment policy should only specify the min/max instances > of the group, not the partition the group runs in. > b) each cartridge needs a cartridge level deployment policy > > R3) if we want to control the VM placement of a group instance and > different number of cartridges depending on partition (another variant of > pattern 1, also covers R1 above) > > a) define deployment number of group instances at the policy with > partitions. Partition min max at in the group policy defines the number > of group instances per partition. > b) define deployment policy at the cartridge level containing a partition > definition for each partition defined in the group. When a group instance > occupies a partitions P1, then only a cartridges P1 partition definitions > applies. For example, if we have a group deployment policy containing a > P1 and P2, and a cartridge C1 has a deployment policy with partition P2 > and P3, when the group is scheduled in P1, there will be zero instances > of C1 because there is no corresponding partition defined for that > cartridge. > > > Supporting requirements R1, R2 and R3 boils down to the following 3 > Features: > > F1) deployment policy needs to support defining either a list of > partitions or just a min/max (meaning the min/max whichever partition this > cartridge is deployed in) > F2) the min/max in a deploy policy should apply at the level at which it's > defined. If it's a group policy then it should be instances of a group. > And if defined at a cartridge level applies to instances of that cartridge. > F3) if we want to support placement at the group level and enforce > placement of cartridges based on group placement, there has to be the same > partitions defined at the cartridge level as at the group level. If there > is no like partition defined then it implied that the cartridges has P1 > defined with partitionMin/Max=0 (or is it better to imply 1 instead of 0) > for that partition. > > > -Vanson > > > > > > > > -- > Reka Thirunavukkarasu > Senior Software Engineer, > WSO2, Inc.:http://wso2.com, > Mobile: +94776442007 > > > > -- Reka Thirunavukkarasu Senior Software Engineer, WSO2, Inc.:http://wso2.com, Mobile: +94776442007
