Re: [openstack-dev] [Heat] About LaunchConfiguration and Autoscaling
> On 30/01/14 12:20, Randall Burt wrote: > > On Jan 30, 2014, at 12:09 PM, Clint Byrum > > wrote: > > >> >I would hope we would solve that at a deeper level, rather than making > >> >resources for the things we think will need re-use. I think nested stacks > >> >allow this level of re-use already anyway. Software config just allows > >> >sub-resource composition. > > Agreed. Codifying re-use inside specific resource types is a game of > > catch-up I don't think we can win in the end. > > Thomas started this discussion talking about resources, but IMHO it's > mostly really about the scaling API. It just happens that we're > implementing the resources first and they need to match what we > eventually decide the API should look like. Not talking about the API was completely intentional. I believe that we should offer the best resource interface possible, regardless of the underlying API. > So we're creating a new API. That doesn't happen very often. It's > absolutely appropriate to consider whether we want two classes of > resources in that API to have a 1:1 or 1:Many relationship, regardless > of how many fancy transclusion things we have in the templates. For a > start because one of the main reasons to create an API is so people can > access it without templates. Fair enough. To give my opinion, I think we're overstating how useful it'd be to reuse launch configurations. The real reusability is at software config level, not so much about what flavor you want to use when starting an instance. Also, we already have provider templates to be able to embed resources and their properties. Finally, if we go back to the API, it's one less thing to manage and simplify the model. The group becomes the bag containing scaling properties and resource definition. -- Thomas ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Heat] About LaunchConfiguration and Autoscaling
> I favor the second option for the same reasons as Zane described, but also > don't think we need a LaunchConfiguration resource. How about just adding a > attribute to the resources such that the engine knows is not meant to be > handled in the usual way, and instead it is really a "template" (sorry for > the overloaded term) used in a scaling group. For example: > > group: > type: OS::Heat::ScalingGroup > properties: > scaled_resource: server_for_scaling > > server_for_scaling: > use_for_scaling: true ( the name of this attribute is clearly up for > discussion ;-) ) > type: OS::Nova::Server > properties: > image: my_image > flavor: m1.large > > When the engine sees the "use_for_scaling" set to true, then it does not call > things like "handle_create". Anyway, that's the general idea. I'm sure there > are many other ways to achieve a similar effect. I'm strongly opposed to that. It reduces the readability of the template a lot for no obvious benefit. Any time you radically change behavior by putting a flag somewhere, somethings's wrong. -- Thomas ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Heat] About LaunchConfiguration and Autoscaling
On 30/01/14 12:20, Randall Burt wrote: On Jan 30, 2014, at 12:09 PM, Clint Byrum wrote: >I would hope we would solve that at a deeper level, rather than making >resources for the things we think will need re-use. I think nested stacks >allow this level of re-use already anyway. Software config just allows >sub-resource composition. Agreed. Codifying re-use inside specific resource types is a game of catch-up I don't think we can win in the end. Thomas started this discussion talking about resources, but IMHO it's mostly really about the scaling API. It just happens that we're implementing the resources first and they need to match what we eventually decide the API should look like. So we're creating a new API. That doesn't happen very often. It's absolutely appropriate to consider whether we want two classes of resources in that API to have a 1:1 or 1:Many relationship, regardless of how many fancy transclusion things we have in the templates. For a start because one of the main reasons to create an API is so people can access it without templates. cheers, Zane. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Heat] About LaunchConfiguration and Autoscaling
The reuse case can be handled via using a nested Stack. The scaled_resource type property would allow that to happen in the first arrangement. I don't think you can specify a resource type/nested stack with a LaunchConfig which makes it much less preferable I think. So its less flexible and more verbose. Thanks, Kevin From: Clint Byrum [cl...@fewbar.com] Sent: Thursday, January 30, 2014 9:09 AM To: openstack-dev Subject: Re: [openstack-dev] [Heat] About LaunchConfiguration and Autoscaling Excerpts from Zane Bitter's message of 2014-01-30 07:38:38 -0800: > On 30/01/14 06:01, Thomas Herve wrote: > > Hi all, > > > > While talking to Zane yesterday, he raised an interesting question about > > whether or not we want to keep a LaunchConfiguration object for the native > > autoscaling resources. > > > > The LaunchConfiguration object basically holds properties to be able to > > fire new servers in a scaling group. In the new design, we will be able to > > start arbitrary resources, so we can't keep a strict LaunchConfiguration > > object as it exists, as we can have arbitrary properties. > > > > It may be still be interesting to store it separately to be able to reuse > > it between groups. > > > > So either we do this: > > > > group: > >type: OS::Heat::ScalingGroup > >properties: > > scaled_resource: OS::Nova::Server > > resource_properties: > >image: my_image > >flavor: m1.large > > The main advantages of this that I see are: > > * It's one less resource. > * We can verify properties against the scaled_resource at the place the > LaunchConfig is defined. (Note: in _both_ models these would be verified > at the same place the _ScalingGroup_ is defined.) > > > Or: > > > > group: > >type: OS::Heat::ScalingGroup > >properties: > > scaled_resource: OS::Nova::Server > > launch_configuration: server_config > > server_config: > >type: OS::Heat::LaunchConfiguration > >properties: > > image: my_image > > flavor: m1.large > > > I favour this one for a few reasons: > > * A single LaunchConfiguration can be re-used by multiple scaling > groups. Reuse is good, and is one of the things we have been driving > toward with e.g. software deployments. I agree with the desire for re-use. In fact I am somewhat desperate to have it as we try to write templates which allow assembling different topologies of OpenStack deployment. I would hope we would solve that at a deeper level, rather than making resources for the things we think will need re-use. I think nested stacks allow this level of re-use already anyway. Software config just allows sub-resource composition. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Heat] About LaunchConfiguration and Autoscaling
On Jan 30, 2014, at 12:09 PM, Clint Byrum wrote: > Excerpts from Zane Bitter's message of 2014-01-30 07:38:38 -0800: >> On 30/01/14 06:01, Thomas Herve wrote: >>> Hi all, >>> >>> While talking to Zane yesterday, he raised an interesting question about >>> whether or not we want to keep a LaunchConfiguration object for the native >>> autoscaling resources. >>> >>> The LaunchConfiguration object basically holds properties to be able to >>> fire new servers in a scaling group. In the new design, we will be able to >>> start arbitrary resources, so we can't keep a strict LaunchConfiguration >>> object as it exists, as we can have arbitrary properties. >>> >>> It may be still be interesting to store it separately to be able to reuse >>> it between groups. >>> >>> So either we do this: >>> >>> group: >>> type: OS::Heat::ScalingGroup >>> properties: >>> scaled_resource: OS::Nova::Server >>> resource_properties: >>> image: my_image >>> flavor: m1.large >> >> The main advantages of this that I see are: >> >> * It's one less resource. >> * We can verify properties against the scaled_resource at the place the >> LaunchConfig is defined. (Note: in _both_ models these would be verified >> at the same place the _ScalingGroup_ is defined.) This looks a lot like OS::Heat::ResourceGroup, which I believe already addresses some of Zane's concerns around "dynamic" property validation. >> >>> Or: >>> >>> group: >>> type: OS::Heat::ScalingGroup >>> properties: >>> scaled_resource: OS::Nova::Server >>> launch_configuration: server_config >>> server_config: >>> type: OS::Heat::LaunchConfiguration >>> properties: >>> image: my_image >>> flavor: m1.large >> >> >> I favour this one for a few reasons: >> >> * A single LaunchConfiguration can be re-used by multiple scaling >> groups. Reuse is good, and is one of the things we have been driving >> toward with e.g. software deployments. > > I agree with the desire for re-use. In fact I am somewhat desperate to > have it as we try to write templates which allow assembling different > topologies of OpenStack deployment. > > I would hope we would solve that at a deeper level, rather than making > resources for the things we think will need re-use. I think nested stacks > allow this level of re-use already anyway. Software config just allows > sub-resource composition. Agreed. Codifying re-use inside specific resource types is a game of catch-up I don't think we can win in the end. > > ___ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Heat] About LaunchConfiguration and Autoscaling
Excerpts from Zane Bitter's message of 2014-01-30 07:38:38 -0800: > On 30/01/14 06:01, Thomas Herve wrote: > > Hi all, > > > > While talking to Zane yesterday, he raised an interesting question about > > whether or not we want to keep a LaunchConfiguration object for the native > > autoscaling resources. > > > > The LaunchConfiguration object basically holds properties to be able to > > fire new servers in a scaling group. In the new design, we will be able to > > start arbitrary resources, so we can't keep a strict LaunchConfiguration > > object as it exists, as we can have arbitrary properties. > > > > It may be still be interesting to store it separately to be able to reuse > > it between groups. > > > > So either we do this: > > > > group: > >type: OS::Heat::ScalingGroup > >properties: > > scaled_resource: OS::Nova::Server > > resource_properties: > >image: my_image > >flavor: m1.large > > The main advantages of this that I see are: > > * It's one less resource. > * We can verify properties against the scaled_resource at the place the > LaunchConfig is defined. (Note: in _both_ models these would be verified > at the same place the _ScalingGroup_ is defined.) > > > Or: > > > > group: > >type: OS::Heat::ScalingGroup > >properties: > > scaled_resource: OS::Nova::Server > > launch_configuration: server_config > > server_config: > >type: OS::Heat::LaunchConfiguration > >properties: > > image: my_image > > flavor: m1.large > > > I favour this one for a few reasons: > > * A single LaunchConfiguration can be re-used by multiple scaling > groups. Reuse is good, and is one of the things we have been driving > toward with e.g. software deployments. I agree with the desire for re-use. In fact I am somewhat desperate to have it as we try to write templates which allow assembling different topologies of OpenStack deployment. I would hope we would solve that at a deeper level, rather than making resources for the things we think will need re-use. I think nested stacks allow this level of re-use already anyway. Software config just allows sub-resource composition. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Heat] About LaunchConfiguration and Autoscaling
I favor the second option for the same reasons as Zane described, but also don't think we need a LaunchConfiguration resource. How about just adding a attribute to the resources such that the engine knows is not meant to be handled in the usual way, and instead it is really a "template" (sorry for the overloaded term) used in a scaling group. For example: group: type: OS::Heat::ScalingGroup properties: scaled_resource: server_for_scaling server_for_scaling: use_for_scaling: true ( the name of this attribute is clearly up for discussion ;-) ) type: OS::Nova::Server properties: image: my_image flavor: m1.large When the engine sees the "use_for_scaling" set to true, then it does not call things like "handle_create". Anyway, that's the general idea. I'm sure there are many other ways to achieve a similar effect. Edmund Troche From: Zane Bitter To: openstack-dev@lists.openstack.org, Date: 01/30/2014 09:43 AM Subject:Re: [openstack-dev] [Heat] About LaunchConfiguration and Autoscaling On 30/01/14 06:01, Thomas Herve wrote: > Hi all, > > While talking to Zane yesterday, he raised an interesting question about whether or not we want to keep a LaunchConfiguration object for the native autoscaling resources. > > The LaunchConfiguration object basically holds properties to be able to fire new servers in a scaling group. In the new design, we will be able to start arbitrary resources, so we can't keep a strict LaunchConfiguration object as it exists, as we can have arbitrary properties. > > It may be still be interesting to store it separately to be able to reuse it between groups. > > So either we do this: > > group: >type: OS::Heat::ScalingGroup >properties: > scaled_resource: OS::Nova::Server > resource_properties: >image: my_image >flavor: m1.large The main advantages of this that I see are: * It's one less resource. * We can verify properties against the scaled_resource at the place the LaunchConfig is defined. (Note: in _both_ models these would be verified at the same place the _ScalingGroup_ is defined.) > Or: > > group: >type: OS::Heat::ScalingGroup >properties: > scaled_resource: OS::Nova::Server > launch_configuration: server_config > server_config: >type: OS::Heat::LaunchConfiguration >properties: > image: my_image > flavor: m1.large I favour this one for a few reasons: * A single LaunchConfiguration can be re-used by multiple scaling groups. Reuse is good, and is one of the things we have been driving toward with e.g. software deployments. * Assuming the Autoscaling API and Resources use the same model (as they should), in this model the Launch Configuration can be defined in a separate template to the scaling group, if the user so chooses. Or it can even be defined outside Heat and passed in as a parameter. * We can do the same with the LaunchConfiguration for the existing AWS-compatibility resources. That will allows us to fix the current broken implementation that goes magically fishing in the local stack for launch configs[1]. If we pick a model that is strictly less powerful than stuff we already know we have to support, we will likely be stuck with broken hacks forever :( > (Not sure we can actually define dynamic properties, in which case it'd be behind a top property.) (This part is just a question of how the resource would look in Heat, and the answer would not really effect the API.) I think this would be possible, but it would require working around the usual code we have for managing/validating properties. Probably not a show-stopper, but it is more work. If we can do this there are a couple more benefits to this way: * Extremely deeply nested structures are unwieldy to deal with, both for us as developers[2] and for users writing templates; shallower hierarchies are better. * You would be able to change an OS::Nova::Server resource into a LaunchConfiguration, in most cases, just by changing the resource type. (This also opens up the possibility of switching between them using the environment, although I don't know how useful that would be.) cheers, Zane. [1] https://etherpad.openstack.org/p/icehouse-summit-heat-exorcism [2] https://github.com/openstack/heat/blob/master/contrib/rackspace/heat/engine/plugins/auto_scale.py ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev <>___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Heat] About LaunchConfiguration and Autoscaling
On 30/01/14 06:01, Thomas Herve wrote: Hi all, While talking to Zane yesterday, he raised an interesting question about whether or not we want to keep a LaunchConfiguration object for the native autoscaling resources. The LaunchConfiguration object basically holds properties to be able to fire new servers in a scaling group. In the new design, we will be able to start arbitrary resources, so we can't keep a strict LaunchConfiguration object as it exists, as we can have arbitrary properties. It may be still be interesting to store it separately to be able to reuse it between groups. So either we do this: group: type: OS::Heat::ScalingGroup properties: scaled_resource: OS::Nova::Server resource_properties: image: my_image flavor: m1.large The main advantages of this that I see are: * It's one less resource. * We can verify properties against the scaled_resource at the place the LaunchConfig is defined. (Note: in _both_ models these would be verified at the same place the _ScalingGroup_ is defined.) Or: group: type: OS::Heat::ScalingGroup properties: scaled_resource: OS::Nova::Server launch_configuration: server_config server_config: type: OS::Heat::LaunchConfiguration properties: image: my_image flavor: m1.large I favour this one for a few reasons: * A single LaunchConfiguration can be re-used by multiple scaling groups. Reuse is good, and is one of the things we have been driving toward with e.g. software deployments. * Assuming the Autoscaling API and Resources use the same model (as they should), in this model the Launch Configuration can be defined in a separate template to the scaling group, if the user so chooses. Or it can even be defined outside Heat and passed in as a parameter. * We can do the same with the LaunchConfiguration for the existing AWS-compatibility resources. That will allows us to fix the current broken implementation that goes magically fishing in the local stack for launch configs[1]. If we pick a model that is strictly less powerful than stuff we already know we have to support, we will likely be stuck with broken hacks forever :( (Not sure we can actually define dynamic properties, in which case it'd be behind a top property.) (This part is just a question of how the resource would look in Heat, and the answer would not really effect the API.) I think this would be possible, but it would require working around the usual code we have for managing/validating properties. Probably not a show-stopper, but it is more work. If we can do this there are a couple more benefits to this way: * Extremely deeply nested structures are unwieldy to deal with, both for us as developers[2] and for users writing templates; shallower hierarchies are better. * You would be able to change an OS::Nova::Server resource into a LaunchConfiguration, in most cases, just by changing the resource type. (This also opens up the possibility of switching between them using the environment, although I don't know how useful that would be.) cheers, Zane. [1] https://etherpad.openstack.org/p/icehouse-summit-heat-exorcism [2] https://github.com/openstack/heat/blob/master/contrib/rackspace/heat/engine/plugins/auto_scale.py ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Heat] About LaunchConfiguration and Autoscaling
Excerpts from Thomas Herve's message of 2014-01-30 03:01:38 -0800: > Hi all, > > While talking to Zane yesterday, he raised an interesting question about > whether or not we want to keep a LaunchConfiguration object for the native > autoscaling resources. > > The LaunchConfiguration object basically holds properties to be able to fire > new servers in a scaling group. In the new design, we will be able to start > arbitrary resources, so we can't keep a strict LaunchConfiguration object as > it exists, as we can have arbitrary properties. > IIRC, LaunchConfiguration is just part of the API for AWS's separate auto scaling service. Since we're auto scaling in Heat we have template intelligence in our auto scaler and thus we shouldn't need two resources. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Heat] About LaunchConfiguration and Autoscaling
Hi Thomas, I haven't looked at the details of the autoscaling design for a while, but the first option looks more intuitive to me. It seems to cover the same content as LaunchConfiguration, but is it generic and therefore would provide for one common approach for all kinds of resources. Regards, Thomas Thomas Herve wrote on 30/01/2014 12:01:38: > From: Thomas Herve > To: "OpenStack Development Mailing List (not for usage questions)" > > Date: 30/01/2014 12:06 > Subject: [openstack-dev] [Heat] About LaunchConfiguration and Autoscaling > > Hi all, > > While talking to Zane yesterday, he raised an interesting question > about whether or not we want to keep a LaunchConfiguration object > for the native autoscaling resources. > > The LaunchConfiguration object basically holds properties to be able > to fire new servers in a scaling group. In the new design, we will > be able to start arbitrary resources, so we can't keep a strict > LaunchConfiguration object as it exists, as we can have arbitrary properties. > > It may be still be interesting to store it separately to be able to > reuse it between groups. > > So either we do this: > > group: > type: OS::Heat::ScalingGroup > properties: > scaled_resource: OS::Nova::Server > resource_properties: > image: my_image > flavor: m1.large > > Or: > > group: > type: OS::Heat::ScalingGroup > properties: > scaled_resource: OS::Nova::Server > launch_configuration: server_config > server_config: > type: OS::Heat::LaunchConfiguration > properties: > image: my_image > flavor: m1.large > > (Not sure we can actually define dynamic properties, in which case > it'd be behind a top property.) > > Thoughts? > > -- > Thomas > > ___ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [Heat] About LaunchConfiguration and Autoscaling
Hi all, While talking to Zane yesterday, he raised an interesting question about whether or not we want to keep a LaunchConfiguration object for the native autoscaling resources. The LaunchConfiguration object basically holds properties to be able to fire new servers in a scaling group. In the new design, we will be able to start arbitrary resources, so we can't keep a strict LaunchConfiguration object as it exists, as we can have arbitrary properties. It may be still be interesting to store it separately to be able to reuse it between groups. So either we do this: group: type: OS::Heat::ScalingGroup properties: scaled_resource: OS::Nova::Server resource_properties: image: my_image flavor: m1.large Or: group: type: OS::Heat::ScalingGroup properties: scaled_resource: OS::Nova::Server launch_configuration: server_config server_config: type: OS::Heat::LaunchConfiguration properties: image: my_image flavor: m1.large (Not sure we can actually define dynamic properties, in which case it'd be behind a top property.) Thoughts? -- Thomas ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
