Hi all,
I thought of updating a bit about Rules we use in Autoscaler.
Currently we have three main rules,
1. Minimum rule
This rule is there to make sure spawning minimum number of instances in
each cloud partition of the policy select when user subscribe. This rule
will be called immediately when a new subscription is received.
But when the autoscaler is started, this rule will be run against all the
clusters available in the topology after full topology is received.
2. Scale up rule
3. Scale down rule
Above two rules decide whether to scale up, scale down, or do nothing
according to load details that are received through Event Processing
Engine. These rules are run periodically and will use latest information
available to the Autoscaler.
Thanks.
On Tue, Nov 12, 2013 at 10:56 PM, Nirmal Fernando <[email protected]>wrote:
>
>
>
> On Tue, Nov 12, 2013 at 10:49 PM, Lahiru Sandaruwan <[email protected]>wrote:
>
>>
>>
>>
>> On Tue, Nov 12, 2013 at 10:32 PM, Lahiru Sandaruwan <[email protected]>wrote:
>>
>>>
>>>
>>>
>>> On Tue, Nov 12, 2013 at 9:38 AM, Nirmal Fernando <[email protected]
>>> > wrote:
>>>
>>>> Also, IMO /s/PartitionMax/max and same for min.
>>>>
>>>
>>>
>>> +1. 'partition' part is redundant.
>>>
>>>
>>>> Also why are these starts with upper-case letters? I think we should
>>>> lower the case of the starting letter (eg: haPolicy).
>>>>
>>>
>>> Any reason?
>>>
>>
>> Ahh.. seems this is a good practice. It improves the clarity as well.. So
>> +1.
>>
>
> It's convenient to type too.
>
>>
>>>>
>>>>
>>>>
>>>> On Tue, Nov 12, 2013 at 9:32 AM, Nirmal Fernando <
>>>> [email protected]> wrote:
>>>>
>>>>> Hi Lahiru,
>>>>>
>>>>> How does partitions and Cartridges relates?
>>>>>
>>>>>
>>>>> On Thu, Nov 7, 2013 at 11:42 PM, Lahiru Sandaruwan
>>>>> <[email protected]>wrote:
>>>>>
>>>>>> Hi all,
>>>>>>
>>>>>> Here is a update/change about how we are going to handle Scale up/
>>>>>> Scale down algorithms.
>>>>>>
>>>>>> These algorithms will apply on cloud partitions. Cloud partition can
>>>>>> be any scope of a cloud.
>>>>>>
>>>>>> E.g. A cloud partition can be belong to EC2 Iaas, ap-southeast
>>>>>> Region, 1b Zone etc.
>>>>>> It is also enough to specify upto IaaS level and further fine grained
>>>>>> levels are optional.
>>>>>>
>>>>>> Partitions are grouped and it should specify an algorithm per group.
>>>>>> There are two algorithm that we are going to ship with Stratos 4.0.
>>>>>>
>>>>>> 1. One after another
>>>>>>
>>>>>> This means it will start scaling up in the next partition after
>>>>>> reaching maximum number specified in the current partition.
>>>>>>
>>>>>> 2. Round robin
>>>>>>
>>>>>> This option will spawn instances among all the partitions Round robin
>>>>>> manner until maximum is reached.
>>>>>>
>>>>>> See a sample HA policy at[1]
>>>>>>
>>>>>> Thanks.
>>>>>>
>>>>>> [1]
>>>>>>
>>>>>>
>>>>>> <HAPolicy>
>>>>>>
>>>>>> <PartitionGroups>
>>>>>>
>>>>>> <PartitionGroup>
>>>>>>
>>>>>> <PartitionAlgo>OneAfterAnother</PartitionAlgo>
>>>>>>
>>>>>> <Partition id="partition1" iaas="EC2" zone="A">
>>>>>>
>>>>>> <PartitionMax>3</PartitionMax>
>>>>>>
>>>>>> <PartitionMin>1</PartitionMin>
>>>>>>
>>>>>> </Partition>
>>>>>>
>>>>>> <Partition id="partition2" iaas="EC2" zone="B">
>>>>>>
>>>>>> <PartitionMax>5</PartitionMax>
>>>>>>
>>>>>> <PartitionMin>1</PartitionMin>
>>>>>>
>>>>>> </Partition>
>>>>>>
>>>>>> <Partition id="partition3" iaas="EC2" zone="B">
>>>>>>
>>>>>> <PartitionMax>20</PartitionMax>
>>>>>>
>>>>>> <PartitionMin>1</PartitionMin>
>>>>>>
>>>>>> </Partition>
>>>>>>
>>>>>> </PartitionGroup>
>>>>>>
>>>>>> <PartitionGroup>
>>>>>>
>>>>>> <PartitionAlgo>RoundRobin</PartitionAlgo>
>>>>>>
>>>>>> <Partition id="partition4" iaas="Openstack" zone="A">
>>>>>>
>>>>>> <PartitionMax>3</PartitionMax>
>>>>>>
>>>>>> <PartitionMin>1</PartitionMin>
>>>>>>
>>>>>> </Partition>
>>>>>>
>>>>>> <Partition id="partition5" iaas="Openstack" zone="B">
>>>>>>
>>>>>> <PartitionMax>5</PartitionMax>
>>>>>>
>>>>>> <PartitionMin>1</PartitionMin>
>>>>>>
>>>>>> </Partition>
>>>>>>
>>>>>> <Partition id="partition6" iaas="Openstack" zone="B">
>>>>>>
>>>>>> <PartitionMax>20</PartitionMax>
>>>>>>
>>>>>> <PartitionMin>1</PartitionMin>
>>>>>>
>>>>>> </Partition>
>>>>>>
>>>>>> </PartitionGroup>
>>>>>>
>>>>>> </PartitionGroups>
>>>>>>
>>>>>> </HAPolicy>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Thu, Oct 10, 2013 at 3:17 PM, Lahiru Sandaruwan
>>>>>> <[email protected]>wrote:
>>>>>>
>>>>>>> Hi all,
>>>>>>>
>>>>>>> Here is the plan for autoscaling component implementation. As per
>>>>>>> the proposed 4.0 architecture, autoscaling decisions making will be
>>>>>>> separated from cloud controller and load balancer and established
>>>>>>> separately.
>>>>>>>
>>>>>>> Autoscaler will get its input from two topics. One topic is
>>>>>>> responsible for passing summarized statistics like cartridge health and
>>>>>>> LB
>>>>>>> statistics sent from Event Processing Engine. Other topic is
>>>>>>> for receiving Topology information.
>>>>>>>
>>>>>>> Here is the architecture diagram,
>>>>>>>
>>>>>>>
>>>>>>> POJOs
>>>>>>>
>>>>>>> Maintain a Java object model(POJOs) for keeping updated topology
>>>>>>> state and autoscaling policies. Clusters and its instances will get
>>>>>>> updated
>>>>>>> with every event received. Deployed policies are also kept as POJOs.
>>>>>>> Serialize POJOs if required.
>>>>>>> Autoscaling Policy
>>>>>>>
>>>>>>> Have a deployable xml model to keep High Availability policy and
>>>>>>> load thresholds. Possibly those will be deployed by Dev-ops at start or
>>>>>>> may
>>>>>>> be hot deploy.
>>>>>>>
>>>>>>> <AutoscalePolicy>
>>>>>>>
>>>>>>> <LoadThresholds>
>>>>>>>
>>>>>>> <RequestsInFlight>
>>>>>>>
>>>>>>> <UpperLimit value="6000"/>
>>>>>>>
>>>>>>> <LowerLimit value="1000"/>
>>>>>>>
>>>>>>> <IdealGraidient value="0"/>
>>>>>>>
>>>>>>> </RequestsInFlight>
>>>>>>>
>>>>>>> <MemoryConsumption>
>>>>>>>
>>>>>>> <UpperLimit value="6000"/>
>>>>>>>
>>>>>>> <LowerLimit value="1000"/>
>>>>>>>
>>>>>>> <IdealGraidient value="0"/>
>>>>>>>
>>>>>>> </MemoryConsumption>
>>>>>>>
>>>>>>> <LoadAverage>
>>>>>>>
>>>>>>> <UpperLimit value="6000"/>
>>>>>>>
>>>>>>> <LowerLimit value="1000"/>
>>>>>>>
>>>>>>> <IdealGraidient value="0"/>
>>>>>>>
>>>>>>> </LoadAverage>
>>>>>>>
>>>>>>> </LoadThresholds>
>>>>>>>
>>>>>>> <HAPolicy>
>>>>>>>
>>>>>>> <IaasAlgo>OneAfterAnother</IaasAlgo>
>>>>>>>
>>>>>>> <Iaas name=Openstack/>
>>>>>>>
>>>>>>> <ZoneAlgo>RoundRobin</ZoneAlgo>
>>>>>>>
>>>>>>> <IaaSMax>10</IaaSMax>
>>>>>>>
>>>>>>> <IaaSMin>1</IaaSMin>
>>>>>>>
>>>>>>> <Zone name=X max=5 min=1/>
>>>>>>>
>>>>>>> <Zone name=Y max=5 min=0/>
>>>>>>>
>>>>>>> </Iaas>
>>>>>>>
>>>>>>> <Iaas name=EC2/>
>>>>>>>
>>>>>>> <ZoneAlgo>OneAfterAnother</ZoneAlgo>
>>>>>>>
>>>>>>> <IaaSMax>7</IaaSMax>
>>>>>>>
>>>>>>> <IaaSMin>0</IaaSMin>
>>>>>>>
>>>>>>> <Zone name=A max=5 min=0/>
>>>>>>>
>>>>>>> <Zone name=B max=2 min=0/>
>>>>>>>
>>>>>>> </Iaas>
>>>>>>>
>>>>>>> </HAPolicy>
>>>>>>>
>>>>>>> </AutoscalePolicy>
>>>>>>> Rules Engine
>>>>>>>
>>>>>>> Input the POJOs periodically to rules engine and get the autoscaling
>>>>>>> decisions.
>>>>>>>
>>>>>>> Please send your feedback on this,
>>>>>>>
>>>>>>> Thanks.
>>>>>>> --
>>>>>>> --
>>>>>>> Lahiru Sandaruwan
>>>>>>> Software Engineer,
>>>>>>> Platform Technologies,
>>>>>>> WSO2 Inc., http://wso2.com
>>>>>>> lean.enterprise.middleware
>>>>>>>
>>>>>>> email: [email protected] cell: (+94) 773 325 954
>>>>>>> blog: http://lahiruwrites.blogspot.com/
>>>>>>> twitter: http://twitter.com/lahirus
>>>>>>> linked-in: http://lk.linkedin.com/pub/lahiru-sandaruwan/16/153/146
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> --
>>>>>> Lahiru Sandaruwan
>>>>>> Software Engineer,
>>>>>> Platform Technologies,
>>>>>> WSO2 Inc., http://wso2.com
>>>>>> lean.enterprise.middleware
>>>>>>
>>>>>> email: [email protected] cell: (+94) 773 325 954
>>>>>> blog: http://lahiruwrites.blogspot.com/
>>>>>> twitter: http://twitter.com/lahirus
>>>>>> linked-in: http://lk.linkedin.com/pub/lahiru-sandaruwan/16/153/146
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Best Regards,
>>>>> Nirmal
>>>>>
>>>>> Nirmal Fernando.
>>>>> PPMC Member & Committer of Apache Stratos,
>>>>> Senior Software Engineer, WSO2 Inc.
>>>>>
>>>>> Blog: http://nirmalfdo.blogspot.com/
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Best Regards,
>>>> Nirmal
>>>>
>>>> Nirmal Fernando.
>>>> PPMC Member & Committer of Apache Stratos,
>>>> Senior Software Engineer, WSO2 Inc.
>>>>
>>>> Blog: http://nirmalfdo.blogspot.com/
>>>>
>>>
>>>
>>>
>>> --
>>> --
>>> Lahiru Sandaruwan
>>> Software Engineer,
>>> Platform Technologies,
>>> WSO2 Inc., http://wso2.com
>>> lean.enterprise.middleware
>>>
>>> email: [email protected] cell: (+94) 773 325 954
>>> blog: http://lahiruwrites.blogspot.com/
>>> twitter: http://twitter.com/lahirus
>>> linked-in: http://lk.linkedin.com/pub/lahiru-sandaruwan/16/153/146
>>>
>>>
>>
>>
>> --
>> --
>> Lahiru Sandaruwan
>> Software Engineer,
>> Platform Technologies,
>> WSO2 Inc., http://wso2.com
>> lean.enterprise.middleware
>>
>> email: [email protected] cell: (+94) 773 325 954
>> blog: http://lahiruwrites.blogspot.com/
>> twitter: http://twitter.com/lahirus
>> linked-in: http://lk.linkedin.com/pub/lahiru-sandaruwan/16/153/146
>>
>>
>
>
> --
> Best Regards,
> Nirmal
>
> Nirmal Fernando.
> PPMC Member & Committer of Apache Stratos,
> Senior Software Engineer, WSO2 Inc.
>
> Blog: http://nirmalfdo.blogspot.com/
>
--
--
Lahiru Sandaruwan
Software Engineer,
Platform Technologies,
WSO2 Inc., http://wso2.com
lean.enterprise.middleware
email: [email protected] cell: (+94) 773 325 954
blog: http://lahiruwrites.blogspot.com/
twitter: http://twitter.com/lahirus
linked-in: http://lk.linkedin.com/pub/lahiru-sandaruwan/16/153/146
