Re: [openstack-dev] [Heat] About LaunchConfiguration and Autoscaling
I favor the second option for the same reasons as Zane described, but also don't think we need a LaunchConfiguration resource. How about just adding a attribute to the resources such that the engine knows is not meant to be handled in the usual way, and instead it is really a "template" (sorry for the overloaded term) used in a scaling group. For example: group: type: OS::Heat::ScalingGroup properties: scaled_resource: server_for_scaling server_for_scaling: use_for_scaling: true ( the name of this attribute is clearly up for discussion ;-) ) type: OS::Nova::Server properties: image: my_image flavor: m1.large When the engine sees the "use_for_scaling" set to true, then it does not call things like "handle_create". Anyway, that's the general idea. I'm sure there are many other ways to achieve a similar effect. Edmund Troche From: Zane Bitter To: openstack-dev@lists.openstack.org, Date: 01/30/2014 09:43 AM Subject:Re: [openstack-dev] [Heat] About LaunchConfiguration and Autoscaling On 30/01/14 06:01, Thomas Herve wrote: > Hi all, > > While talking to Zane yesterday, he raised an interesting question about whether or not we want to keep a LaunchConfiguration object for the native autoscaling resources. > > The LaunchConfiguration object basically holds properties to be able to fire new servers in a scaling group. In the new design, we will be able to start arbitrary resources, so we can't keep a strict LaunchConfiguration object as it exists, as we can have arbitrary properties. > > It may be still be interesting to store it separately to be able to reuse it between groups. > > So either we do this: > > group: >type: OS::Heat::ScalingGroup >properties: > scaled_resource: OS::Nova::Server > resource_properties: >image: my_image >flavor: m1.large The main advantages of this that I see are: * It's one less resource. * We can verify properties against the scaled_resource at the place the LaunchConfig is defined. (Note: in _both_ models these would be verified at the same place the _ScalingGroup_ is defined.) > Or: > > group: >type: OS::Heat::ScalingGroup >properties: > scaled_resource: OS::Nova::Server > launch_configuration: server_config > server_config: >type: OS::Heat::LaunchConfiguration >properties: > image: my_image > flavor: m1.large I favour this one for a few reasons: * A single LaunchConfiguration can be re-used by multiple scaling groups. Reuse is good, and is one of the things we have been driving toward with e.g. software deployments. * Assuming the Autoscaling API and Resources use the same model (as they should), in this model the Launch Configuration can be defined in a separate template to the scaling group, if the user so chooses. Or it can even be defined outside Heat and passed in as a parameter. * We can do the same with the LaunchConfiguration for the existing AWS-compatibility resources. That will allows us to fix the current broken implementation that goes magically fishing in the local stack for launch configs[1]. If we pick a model that is strictly less powerful than stuff we already know we have to support, we will likely be stuck with broken hacks forever :( > (Not sure we can actually define dynamic properties, in which case it'd be behind a top property.) (This part is just a question of how the resource would look in Heat, and the answer would not really effect the API.) I think this would be possible, but it would require working around the usual code we have for managing/validating properties. Probably not a show-stopper, but it is more work. If we can do this there are a couple more benefits to this way: * Extremely deeply nested structures are unwieldy to deal with, both for us as developers[2] and for users writing templates; shallower hierarchies are better. * You would be able to change an OS::Nova::Server resource into a LaunchConfiguration, in most cases, just by changing the resource type. (This also opens up the possibility of switching between them using the environment, although I don't know how useful that would be.) cheers, Zane. [1] https://etherpad.openstack.org/p/icehouse-summit-heat-exorcism [2] https://github.com/openstack/heat/blob/master/contrib/rackspace/heat/engine/plugins/auto_scale.py ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev <>___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [heat] [glance] Heater Proposal
I thought about that, i.e. first step in implementation just adding templates, but like you said, you might end up duplicating 5 of the 7 tables in the Glance database, for every new "asset type" (image, template, etc). Then you would do a similar thing for the endpoints. So, I'm not sure what's a better way to approach this. For all I know, doing a "s/image/asset/g" for *.py,, adding attribute images.type, and a little more refactoring, might get us 80% of the asset management functionality that we would need initially ;-) Not knowing the Glance code base I'm only going by the surface footprint, so I'll leave it to the experts to comment on what would be a good approach to take Glance to the next level. Edmund Troche From: Randall Burt To: "OpenStack Development Mailing List (not for usage questions)" , Date: 12/06/2013 04:47 PM Subject:Re: [openstack-dev] [heat] [glance] Heater Proposal I too have warmed to this idea but wonder about the actual implementation around it. While I like where Edmund is going with this, I wonder if it wouldn't be valuable in the short-to-mid-term (I/J) to just add /templates to Glance (/assemblies, /applications, etc) along side /images. Initially, we could have separate endpoints and data structures for these different asset types, refactoring the easy bits along the way and leveraging the existing data storage and caching bits, but leaving more disruptive changes alone. That can get the functionality going, prove some concepts, and allow all of the interested parties to better plan a more general v3 api. On Dec 6, 2013, at 4:23 PM, Edmund Troche wrote: I agree with what seems to also be the general consensus, that Glance can "become" Heater+Glance (the service that manages images in OS today). Clearly, if someone looks at the Glance DB schema, APIs and service type (as returned by keystone service-list), all of the terminology is about images, so we would need to more formally define what are the characteristics or "image", "template", maybe "assembly", "components" etc and find what is a good generalization. When looking at the attributes for "image" (image table), I can see where there are a few that would be generic enough to apply to "image", "template" etc, so those could be taken to be the base set of attributes, and then based on the "type" (image, template, etc) we could then have attributes that are type-specific (maybe by leveraging what is today "image_properties"). As I read through the discussion, the one thing that came to mind is "asset management". I can see where if someone bothers to create an image, or a template, then it is for a good reason, and that perhaps you'd like to maintain it as an IT asset. Along those lines, it occurred to me that maybe what we need is to make Glance some sort of asset management service that can be leveraged by Service Catalogs, Nova, etc. Instead of storing "images" and "templates" we store assets of one kind or another, with artifacts (like files, image content, etc), and associated metadata. There is some work we could borrow from, conceptually at least, from OSLC's Asset Management specification: http://open-services.net/wiki/asset-management/OSLC-Asset-Management-2.0-Specification/ . Looking at this spec, it probably has more than we need, but there's plenty we could borrow from it. Edmund Troche Georgy Okrokvertskhov ---12/06/2013 01:34:13 PM---As a Murano team we will be happy to contribute to Glance. Our Murano metadata repository is a stand From: Georgy Okrokvertskhov To: "OpenStack Development Mailing List (not for usage questions)" < openstack-dev@lists.openstack.org>, Date: 12/06/2013 01:34 PM Subject: Re: [openstack-dev] [heat] [glance] Heater Proposal As a Murano team we will be happy to contribute to Glance. Our Murano metadata repository is a standalone component (with its own git repository)which is not tightly coupled with Murano itself. We can easily add our functionality to Glance as a new component\subproject. Thanks Georgy On Fri, Dec 6, 2013 at 11:11 AM, Vishvananda Ishaya < vishvana...@gmail.com> wrote: On Dec 6, 2013, at 10:38 AM, Clint Byrum wrote: > Excerpts from Jay Pipes's message of 2013-12-05 21:32:54 -0800: >> On 12/05/2013 04:25 PM, Clint Byrum wrote: >>> Excerpts from Andrew Plunk's message of 2013-12-05 12:42:49 -0800: >>&
Re: [openstack-dev] [heat] [glance] Heater Proposal
I agree with what seems to also be the general consensus, that Glance can "become" Heater+Glance (the service that manages images in OS today). Clearly, if someone looks at the Glance DB schema, APIs and service type (as returned by keystone service-list), all of the terminology is about images, so we would need to more formally define what are the characteristics or "image", "template", maybe "assembly", "components" etc and find what is a good generalization. When looking at the attributes for "image" (image table), I can see where there are a few that would be generic enough to apply to "image", "template" etc, so those could be taken to be the base set of attributes, and then based on the "type" (image, template, etc) we could then have attributes that are type-specific (maybe by leveraging what is today "image_properties"). As I read through the discussion, the one thing that came to mind is "asset management". I can see where if someone bothers to create an image, or a template, then it is for a good reason, and that perhaps you'd like to maintain it as an IT asset. Along those lines, it occurred to me that maybe what we need is to make Glance some sort of asset management service that can be leveraged by Service Catalogs, Nova, etc. Instead of storing "images" and "templates" we store assets of one kind or another, with artifacts (like files, image content, etc), and associated metadata. There is some work we could borrow from, conceptually at least, from OSLC's Asset Management specification: http://open-services.net/wiki/asset-management/OSLC-Asset-Management-2.0-Specification/. Looking at this spec, it probably has more than we need, but there's plenty we could borrow from it. Edmund Troche From: Georgy Okrokvertskhov To: "OpenStack Development Mailing List (not for usage questions)" , Date: 12/06/2013 01:34 PM Subject:Re: [openstack-dev] [heat] [glance] Heater Proposal As a Murano team we will be happy to contribute to Glance. Our Murano metadata repository is a standalone component (with its own git repository)which is not tightly coupled with Murano itself. We can easily add our functionality to Glance as a new component\subproject. Thanks Georgy On Fri, Dec 6, 2013 at 11:11 AM, Vishvananda Ishaya wrote: On Dec 6, 2013, at 10:38 AM, Clint Byrum wrote: > Excerpts from Jay Pipes's message of 2013-12-05 21:32:54 -0800: >> On 12/05/2013 04:25 PM, Clint Byrum wrote: >>> Excerpts from Andrew Plunk's message of 2013-12-05 12:42:49 -0800: >>>>> Excerpts from Randall Burt's message of 2013-12-05 09:05:44 -0800: >>>>>> On Dec 5, 2013, at 10:10 AM, Clint Byrum >>>>>> wrote: >>>>>> >>>>>>> Excerpts from Monty Taylor's message of 2013-12-04 17:54:45 -0800: >>>>>>>> Why not just use glance? >>>>>>>> >>>>>>> >>>>>>> I've asked that question a few times, and I think I can collate the >>>>>>> responses I've received below. I think enhancing glance to do these >>>>>>> things is on the table: >>>>>>> >>>>>>> 1. Glance is for big blobs of data not tiny templates. >>>>>>> 2. Versioning of a single resource is desired. >>>>>>> 3. Tagging/classifying/listing/sorting >>>>>>> 4. Glance is designed to expose the uploaded blobs to nova, not users >>>>>>> >>>>>>> My responses: >>>>>>> >>>>>>> 1: Irrelevant. Smaller things will fit in it just fine. >>>>>> >>>>>> Fitting is one thing, optimizations around particular assumptions about the size of data and the frequency of reads/writes might be an issue, but I admit to ignorance about those details in Glance. >>>>>> >>>>> >>>>> Optimizations can be improved for various use cases. The design, however, >>>>> has no assumptions that I know about that would invalidate storing blobs >>>>> of yaml/json vs. blobs of kernel/qcow2/raw image. >>>> >>>> I think we are getting out into the weeds a little bit here. It is important to think about these apis in terms of what they actually do, before the decision of combining them or not can be made. >>>> >>>> I think of HeatR as a template storage service, it provides extra data and operations on templates. HeatR should not care a
Re: [openstack-dev] [heat][horizon]Heat UI related requirements & roadmap
You bring up a good point Thomas. I think some of the discussions are mixing template and stack perspectives, they are not the same thing, stack == instance of a template. There likely is room for "tagging" stacks, all under the control of the user and meant for user consumption, vs the long going discussion on template-level metadata. This may be yet another use case ;-) Edmund Troche Senior Software Engineer IBM Software Group | 11501 Burnet Rd. | Austin, TX 78758 ( +1.512.286.8977 ) T/L 363.8977 * edmund.tro...@us.ibm.com 7 +1.512.286.8977 From: Thomas Spatzier To: "OpenStack Development Mailing List \(not for usage questions \)" , Date: 11/27/2013 11:00 AM Subject:Re: [openstack-dev] [heat][horizon]Heat UI related requirements & roadmap Thanks, that clarified the use case a bit. Bot looking at the use case now, isn't this stack tagging instead of template tagging? I.e. assume that for each stack a user creates, he/she can assign one or more tags so you can do better queries to find stacks later? Regards, Thomas Tim Schnell wrote on 27.11.2013 16:24:18: > From: Tim Schnell > To: "OpenStack Development Mailing List (not for usage questions)" > , > Date: 27.11.2013 16:28 > Subject: Re: [openstack-dev] [heat][horizon]Heat UI related > requirements & roadmap > > Ok, I just re-read my example and that was a terrible example. I'll try > and create the user story first and hopefully answer Clint's and Thomas's > concerns. > > > If the only use case for adding keywords to the template is to help > organize the template catalog then I would agree the keywords would go > outside of heat. The second purpose for keywords is why I think they > belong in the template so I'll cover that. > > Let's assume that an end-user of Heat has spun up 20 stacks and has now > requested help from a Support Operator of heat. In this case, the end-user > did not have a solid naming convention for naming his stacks, they are all > named "tim1", "tim2", etcŠ And also his request to the Support Operator > was really vague, like "My Wordpress stack is broken." > > The first thing that the Support Operator would do, would be to pull up > end-user's stacks in either Horizon or via the heat client api. In both > cases, at the moment, he would then have to either stack-show on each > stack to look at the description of the stack or ask the end-user for a > stack-id/stack-name. This currently gets the job done but a better > experience would be for stack-list to already display some keywords about > each stack so the Support Operator would have to do less digging. > > In this case the end-user only has one Wordpress stack so he would have > been annoyed if the Support Operator requested more information from him. > (Or maybe he has more than one wordpress stack, but only one currently in > CREATE_FAILED state). > > As a team, we have already encountered this exact situation just doing > team testing so I imagine that others would find value in a consistent way > to determine at least a general purpose of a stack, from the stack-list > page. Putting the stack-description in the stack-list table would take up > too much room from a design standpoint. > > Once keywords has been added to the template then part of the blueprint > would be to return it with the stack-list information. > > The previous example I attempted to explain is really more of an edge > case, so let's ignore it for now. > > Thanks, > Tim > > On 11/27/13 3:19 AM, "Thomas Spatzier" wrote: > > >Excerpts from Tim Schnell's message on 27.11.2013 00:44:04: > >> From: Tim Schnell > >> To: "OpenStack Development Mailing List (not for usage questions)" > >> , > >> Date: 27.11.2013 00:47 > >> Subject: Re: [openstack-dev] [heat][horizon]Heat UI related > >> requirements & roadmap > >> > >> > > > > > > > >> That is not the use case that I'm attempting to make, let
