Re: [openstack-dev] [heat] One more lifecycle plug point - in scaling groups

2014-07-04 Thread Chris Friesen

On 07/03/2014 10:13 PM, Mike Spreitzer wrote:


I do think the issue these address --- the need to get application logic
involved in, e.g., shutdown --- is most of what an application needs;
involvement in selection of which member(s) to delete is much less
important (provided that clean shutdown mechanism prevents concurrent
shutdowns).


I assume this is more of the whole cattle model, where an instance 
could disappear at any time so applications should design for that?


As an alternate viewpoint, if a particular instance in a group is 
working on something expensive (long-running, difficult to checkpoint, 
etc.), maybe it would make sense to allow the application to help make 
the decision on which instance to shut down (or possibly even veto/delay 
the scale down operation).  If it takes a minute to finish the 
operation, and 15 minutes to redo it on another instance...


Chris

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [heat] One more lifecycle plug point - in scaling groups

2014-07-03 Thread Mike Spreitzer
Steven Hardy sha...@redhat.com wrote on 07/02/2014 06:16:21 AM:

 On Wed, Jul 02, 2014 at 02:41:19AM +, Adrian Otto wrote:
 Zane,
 If you happen to have a link to this blueprint, could you replywith 
it? ...

 I believe Zane was referring to:
 
 https://blueprints.launchpad.net/heat/+spec/update-hooks
 
 This is also related to the action aware software config spec:
 
 https://review.openstack.org/#/c/98742/
 
 So in future, you might autoscale nested stacks containing action-aware
 software config resources, then you could define specific actions which
 happen e.g on scale-down (on action DELETE).

Thanks, those are great pointers.  The second pretty much covers the 
first, right?

I do think the issue these address --- the need to get application logic 
involved in, e.g., shutdown --- is most of what an application needs; 
involvement in selection of which member(s) to delete is much less 
important (provided that clean shutdown mechanism prevents concurrent 
shutdowns).  So that provides a pretty good decoupling between the 
application's concerns and a holistic scheduler's concerns.

Thanks,
Mike

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [heat] One more lifecycle plug point - in scaling groups

2014-07-02 Thread Steven Hardy
On Wed, Jul 02, 2014 at 02:41:19AM +, Adrian Otto wrote:
Zane,
If you happen to have a link to this blueprint, could you reply with it? I
took a look, but did not find it.
I'd like to suggest that the implementation allow apps to call
unauthenticated (signed) webhook URLs in order to trigger a scale up/down
event within a scaling group. This is about the simplest possible API to
allow any application to control it's own elasticity.

This is already possible, you just need to get the scaling policy alarm URL
into the instance where the application is running.

I believe Zane was referring to:

https://blueprints.launchpad.net/heat/+spec/update-hooks

This is also related to the action aware software config spec:

https://review.openstack.org/#/c/98742/

So in future, you might autoscale nested stacks containing action-aware
software config resources, then you could define specific actions which
happen e.g on scale-down (on action DELETE).

Steve

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [heat] One more lifecycle plug point - in scaling groups

2014-07-02 Thread Zane Bitter

On 01/07/14 21:09, Mike Spreitzer wrote:

Zane Bitter zbit...@redhat.com wrote on 07/01/2014 07:05:15 PM:

  On 01/07/14 16:30, Mike Spreitzer wrote:
   Thinking about my favorite use case for lifecycle plug points for cloud
   providers (i.e., giving something a chance to make a holistic placement
   decision), it occurs to me that one more is needed: a scale-down plug
   point.  A plugin for this point has a distinctive job: to decide which
   group member(s) to remove from a scaling group (i.e.,
   OS::Heat::AutoScalingGroup or OS::Heat::InstanceGroup or
   OS::Heat::ResourceGroup or AWS::AutoScaling::AutoScalingGroup).  The
   plugin's signature could be something like this: given a list of group
   members and a number to remove, return the list of members to remove
   (or, equivalently, return the list of members to keep).  What do
you think?
 
  I think you're not thinking big enough ;)

I agree, I was taking only a small bite in hopes of a quick success.

  There exist a whole class of applications that would benefit from
  autoscaling but which are not quite stateless. (For example, a PaaS.) So
  it's not enough to have plugins that place the choice of which node to
  scale down under operator control; in fact it needs to be under
  _application_ control.

Exactly.  There are two different roles that want such control; in
general, neither is happy if only the other gets it.  Now the question
becomes, how do we get them to play nice together?  In the case of
TripleO there may be an exceptionally easy out: the owner of an
application deployed on the undercloud may well be the same as the
provider of the undercloud (i.e., the operator whose end goal is to
provide the overcloud(s) ).


Let's assume that this feature takes that form of some additional data 
in the alarm trigger (i.e. the input to the scaling policy) that 
specifies which server(s) to delete first. The application would handle 
this by receiving the trigger from Ceilometer (or wherever) itself, and 
then inserting the additional data before passing it to Heat/autoscaling.


That gives us three options for e.g. a holistic scheduler to insert 
hints as to which servers to delete:


(a) Insert them into the outgoing triggers from Ceilometer. The 
application has the choice to override.


(b) Let the user explicitly configure the flow of notifications. So it 
could be any of:

Ceilometer - Heat
Ceilometer - Scheduler - Heat
Ceilometer - Application - Heat
Ceilometer - Scheduler - Application - Heat

(c) Insert them into incoming triggers in Heat whenever the application 
has not specified them. This is basically your original proposal.


I'm guessing that, of those, (c) is probably the winner. But we'd need 
to have that debate.


Another possible implementation is to do it with a notification and 
reply, rather than including it in the existing datapath.



  This is on the roadmap, and TripleO really needs it, so hopefully it
  will happen in Juno.

I assume you mean giving this control to the application, which I
presume amounts to giving it to the template author.  Is this written up
somewhere?


I had a quick look, but it doesn't appear we have a blueprint for it 
yet, unless you count the notifications blueprint that Steve mentioned 
(but I don't think that addresses this case specifically).


cheers,
Zane.


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [heat] One more lifecycle plug point - in scaling groups

2014-07-01 Thread Zane Bitter

On 01/07/14 16:30, Mike Spreitzer wrote:

Thinking about my favorite use case for lifecycle plug points for cloud
providers (i.e., giving something a chance to make a holistic placement
decision), it occurs to me that one more is needed: a scale-down plug
point.  A plugin for this point has a distinctive job: to decide which
group member(s) to remove from a scaling group (i.e.,
OS::Heat::AutoScalingGroup or OS::Heat::InstanceGroup or
OS::Heat::ResourceGroup or AWS::AutoScaling::AutoScalingGroup).  The
plugin's signature could be something like this: given a list of group
members and a number to remove, return the list of members to remove
(or, equivalently, return the list of members to keep).  What do you think?


I think you're not thinking big enough ;)

There exist a whole class of applications that would benefit from 
autoscaling but which are not quite stateless. (For example, a PaaS.) So 
it's not enough to have plugins that place the choice of which node to 
scale down under operator control; in fact it needs to be under 
_application_ control.


This is on the roadmap, and TripleO really needs it, so hopefully it 
will happen in Juno.


cheers,
Zane.

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [heat] One more lifecycle plug point - in scaling groups

2014-07-01 Thread Mike Spreitzer
Zane Bitter zbit...@redhat.com wrote on 07/01/2014 07:05:15 PM:

 On 01/07/14 16:30, Mike Spreitzer wrote:
  Thinking about my favorite use case for lifecycle plug points for 
cloud
  providers (i.e., giving something a chance to make a holistic 
placement
  decision), it occurs to me that one more is needed: a scale-down plug
  point.  A plugin for this point has a distinctive job: to decide which
  group member(s) to remove from a scaling group (i.e.,
  OS::Heat::AutoScalingGroup or OS::Heat::InstanceGroup or
  OS::Heat::ResourceGroup or AWS::AutoScaling::AutoScalingGroup).  The
  plugin's signature could be something like this: given a list of group
  members and a number to remove, return the list of members to remove
  (or, equivalently, return the list of members to keep).  What do you 
think?
 
 I think you're not thinking big enough ;)

I agree, I was taking only a small bite in hopes of a quick success.

 There exist a whole class of applications that would benefit from 
 autoscaling but which are not quite stateless. (For example, a PaaS.) So 

 it's not enough to have plugins that place the choice of which node to 
 scale down under operator control; in fact it needs to be under 
 _application_ control.

Exactly.  There are two different roles that want such control; in 
general, neither is happy if only the other gets it.  Now the question 
becomes, how do we get them to play nice together?  In the case of TripleO 
there may be an exceptionally easy out: the owner of an application 
deployed on the undercloud may well be the same as the provider of the 
undercloud (i.e., the operator whose end goal is to provide the 
overcloud(s) ).

 This is on the roadmap, and TripleO really needs it, so hopefully it 
 will happen in Juno.

I assume you mean giving this control to the application, which I presume 
amounts to giving it to the template author.  Is this written up 
somewhere?

Thanks,
Mike
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [heat] One more lifecycle plug point - in scaling groups

2014-07-01 Thread Adrian Otto
Zane,

If you happen to have a link to this blueprint, could you reply with it? I took 
a look, but did not find it.

I’d like to suggest that the implementation allow apps to call unauthenticated 
(signed) webhook URLs in order to trigger a scale up/down event within a 
scaling group. This is about the simplest possible API to allow any application 
to control it’s own elasticity.

Thanks,

Adrian

On Jul 1, 2014, at 6:09 PM, Mike Spreitzer 
mspre...@us.ibm.commailto:mspre...@us.ibm.com wrote:

Zane Bitter zbit...@redhat.commailto:zbit...@redhat.com wrote on 07/01/2014 
07:05:15 PM:

 On 01/07/14 16:30, Mike Spreitzer wrote:
  Thinking about my favorite use case for lifecycle plug points for cloud
  providers (i.e., giving something a chance to make a holistic placement
  decision), it occurs to me that one more is needed: a scale-down plug
  point.  A plugin for this point has a distinctive job: to decide which
  group member(s) to remove from a scaling group (i.e.,
  OS::Heat::AutoScalingGroup or OS::Heat::InstanceGroup or
  OS::Heat::ResourceGroup or AWS::AutoScaling::AutoScalingGroup).  The
  plugin's signature could be something like this: given a list of group
  members and a number to remove, return the list of members to remove
  (or, equivalently, return the list of members to keep).  What do you think?

 I think you're not thinking big enough ;)

I agree, I was taking only a small bite in hopes of a quick success.

 There exist a whole class of applications that would benefit from
 autoscaling but which are not quite stateless. (For example, a PaaS.) So
 it's not enough to have plugins that place the choice of which node to
 scale down under operator control; in fact it needs to be under
 _application_ control.

Exactly.  There are two different roles that want such control; in general, 
neither is happy if only the other gets it.  Now the question becomes, how do 
we get them to play nice together?  In the case of TripleO there may be an 
exceptionally easy out: the owner of an application deployed on the undercloud 
may well be the same as the provider of the undercloud (i.e., the operator 
whose end goal is to provide the overcloud(s) ).

 This is on the roadmap, and TripleO really needs it, so hopefully it
 will happen in Juno.

I assume you mean giving this control to the application, which I presume 
amounts to giving it to the template author.  Is this written up somewhere?

Thanks,
Mike
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.orgmailto:OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev