Thanks HU yanyan, I still has some question about your answers, and I respond
inline.
2015-11-20 16:06 GMT+08:00 xu...@cmss.chinamobile.com
<xu...@cmss.chinamobile.com>:
Thanks yanyan!
Xu Jun is a contributor from CMCC. He asked a very interesting question about
cluster scaling support in Senlin. To make the discussion more thorough, I just
post the question and my answer here.
The question from Jun is as following:
For an action, senlin will check all according polices, like if a cluster
attach two scaling-in polices,
the two scaling-in polices will be checked when doing a scaling-action on this
cluster. This is not same as OS::Heat::ScalingPolicy in heat？
How should I use senlin for following cases？
1. 15% < cpu_util < 30%, scaling_in 1 instance
2. cpu_util < 15%, scaling_in 2 instances
This is a very interesting question and you're right about the difference
between Senlin ScalingPolicy and OS::Heat::ScalingPolicy. In Heat,
OS::Heat::ScalingPolicy is actually not just a policy. It is a combination of a
webhook and a rule about how ASG respond to the webhook triggering(resource
signal). So you can define two different OS::Heat::ScalingPolicy instances to
make them deal with two cases you described respectively.
But in Senlin, ScalingPolicy is a REAL policy, it only describes how a Senlin
cluster react to an action triggered by Senlin webhook which is defined
separately. The problem is when an cluster action e.g. CLUSTER_SCALE_IN is
triggered, all policies attached to it will be checked in sequence based on
policies priority definition. So if you create two Senlin ScalingPolicy and
attach them to the same cluster, only one of them will take effect actually.
# 1. But in policy_check function, all the policies will be checked in
priority-based order for a CLUSTER_SCALING_IN action if the cluster attached
with SCALING multiple policies.
is this a bug? or what is the significance of prority).
Sorry, I didn't describe it clearly. I mean although both scaling
policies will be checked before CLUSTER_SCALING_IN action is executed, count
result from one ScalingPolicy will actually be overridden by the result from
another ScalingPolicy which has higher priority.
After debug it , I found thart former result is not overridden by another
policy.
http://git.openstack.org/cgit/openstack/senlin/tree/senlin/engine/actions/base.py#n441
2. if a cluster attached a scaling policy with event = CLUSTER_SCALE_IN,
when a CLUSTER_SCALING_OUT action is triggered, the policy also will be
checked, is this reasonable?
When a ScalingPolicy is defined, you can use 'event' property to
specify the action type you want the policy to take effect on, like:
http://git.openstack.org/cgit/openstack/senlin/tree/examples/policies/scaling_policy.yaml#n5
Although a ScalingPolicy will be checked for both CLUSTER_SCALE_IN and
CLUSTER_SCALE_OUT actions, the check routine will return immediately if the
action type is not what it is expecting.
http://git.openstack.org/cgit/openstack/senlin/tree/senlin/policies/scaling_policy.py#n133
Yes in pre_op， it‘s not checked, but all ScalingPolicies still will check
whether in cooldown.
http://git.openstack.org/cgit/openstack/senlin/tree/senlin/engine/actions/base.py#n431
Currently, you can use the following way to support your use case in Senlin:
1. Define two Senlin webhooks which target on the CLUSTER_SCALE_OUT action of
the cluster and specify the 'param' as {'count': 1} for webhook1 and {'count':
2 } for webhook2;
1. Define two ceilometer/aodh alarms with the first one matching case1 and
second one matching case2. Then define webhook1 url as alarm1's alarm-action
and webhook2 url as alarm2's alarm-action.
#
Your suggestion has a problem when I want different cooldown for each
ceilometer/aodh alarms, for following cases, how should I do?
1. 15% < cpu_util < 30%, scaling_in 1 instance with 300s cooldown time
2. cpu_util < 15%, scaling_in 2 instances with 600s cooldown time
You can define the cooldown by specifying it when creating the policy or
attaching it to a cluster. The cooldown check logic will prevent a policy
taking effect if cooldown is still in progress.
http://git.openstack.org/cgit/openstack/senlin/tree/senlin/engine/actions/base.py#n431
Yes we can define cooldown for each policy, but I want each cluster_scaling_in
action is only checked by one scaling_policy like OS::Heat::ScalingPolicy?
In heat, we could define two scaling_in actions(via define two
OS::Heat::ScalingPolicy polices ), each scaling_in action is checked by one
OS::Heat::ScalingPolicy, so each scaling_in action's cooldown is only checked
in one OS::Heat::ScalingPolicy.
but in senlin, each scaling_in action will be checked by all attached
scaling_policies, so all scaling_polices' cooldown will be checked.
For a senlin webhook, could we assign a policy which will be checked ?
User is not allowed to specify the policy when defining a webhook. The
webhook target is decided by target object(cluster or node) and target action
type.
Then each time alarm1 is triggered, cluster will be scaled out with count 1
which means one new node will be created and added to cluster. When alarm2 is
triggered, cluster will be scaled out with count 2 that two new nodes will be
created and added to cluster.
The question you asked is really interesting and we did consider to support
this kind of requirement using a 'complex' ScalingPolicy which defined both
trigger(alarm), webhook and some rules for scaling. But after some discussion,
we felt that maybe we should let some high level service/enduser to define this
kind of 'POLICY' since it's more like a workflow definition rather than a
description of the rule cluster scaling. So currently, we only provide atomic
operation(e.g. webhook, 'simple' ScalingPolicy) in Senlin while leaving the
work of combining these operations to support a use case to enduser/high-level
service.
Thanks a lot for throwing this interesting question and I do agree that we
should make more discussion about it to think whether we need to adjust our
policy design to support this kind of scenario more smoothly.
--
Best regards,
Yanyan
--
Best regards,
Yanyan
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
