Hi devs,
I thought of writing a small description with a diagram regarding $subject,
at [1], as i understood. As the code base is getting complex, this might
help people to debug, improve the code. Copied content below, feedback is
appreciated.
Autoscaler Member Lifecycle
Autoscaling periodic task
Autoscaler can decide to add a member to cluster for several reasons.
-
Fulfil the minimum member count of the cluster
Minimum check rule is responsible for this task. Logic resides in
mincheck.drl file
-
Scaling up due to high values of statistics such as memory consumption,
requests in flight etc.
scaling.drl file will be the one if anyone want to check logic related to
this.
-
Scaling up a dependency which affects the current cluster
This is triggered when we mention the relationship of scaling dependencies
in the application.
E.g.
"scalingDependants": [
"cartridge.my-php, cartridge.my-tomcat"
]
Pending status
This status is the first status a member will become in Autoscaler. When
the member is created in IaaS, we will keep the member in this status until
we receive the MemberActivatedEvent. We also have a timeout for pending
members which can be configured using “autoscale.xml”. If the member does
not become active in that period of time, Autoscaler will move the member
to Obsolete list.
We have a “ObsoletedMemberWatcher” class which runs periodically and check
if the members are expired.
Active Status
This is the state of the member which will be, after the
MemberActivatedEvent is received to Autoscaler from Cloud Controller.
Termination Pending
If the Autoscaler decided to remove a member of a cluster due to one of the
following reasons, it will move the member to this status.
-
Scaling down due to low values of statistics such as memory consumption,
requests in flight etc.
scaling.drl file will be the one if anyone want to check logic related to
this.
-
Scaling down a dependency which affects the current cluster
This is triggered when we mention the relationship of scaling dependencies
in the application.
E.g.
"scalingDependants": [
"cartridge.my-php, cartridge.my-tomcat"
]
Autoscaler will also send the instance cleanup event for termination
pending members. “obsoletecheck.drl” has the logic to send that event.
When the MemberReadyToShutdownEvent received for that particular member,
Autoscaler will move that member to Obsolete status. Also if the
MemberReadyToShutdownEvent is not received for the
“terminationPendingMemberExpiryTime” particular member, which is
configurable using “autoscale.xml”, it will move the member to Obsolete
status.
Obsolete Status
These members are terminated using cloud controller API directly, by
running the “ObsoletedMemberWatcher” task. If the member is failed to
terminated for a timeout defined in “autoscaler.xml”, it will remove the
member from obsolete list and give up on the member termination.
Terminated
Autoscaler will remove the member from Obsolete list and when the
MemberTerminatedEvent is received. Then Autoscaler will have no record
about that member.
Thanks.
[1]
http://lahiruwrites.blogspot.com/2015/03/stratos-410-deveper-guide-autoscaler.html
--
--
Lahiru Sandaruwan
Committer and PMC member, Apache Stratos,
Senior Software Engineer,
WSO2 Inc., http://wso2.com
lean.enterprise.middleware
phone: +94773325954
email: [email protected] blog: http://lahiruwrites.blogspot.com/
linked-in: http://lk.linkedin.com/pub/lahiru-sandaruwan/16/153/146
