Re: [openstack-dev] [TripleO][Heat] Tuskar v. Heat responsibilities
On Mon, Jun 29, 2015 at 04:42:00PM -0400, Jay Dobies wrote: I had originally been thinking of it like slagle describes, from the child up to the parent as well. What I like about that approach is that it achieves a more pluggable model when you think about extensions that aren't accepted or applicable in TripleO upstream. If someone comes along and adds a new ControllerConfig to your above example, they have to edit whatever environment you're talking about that defines the constraints (I'm call it overcloud-something.yaml for now). This becomes a problem from a packaging point of view, especially when you factor in non-TripleO integrators (without revealing too much inside baseball, think partner integrations). How do I add in an extra package (RPM, DEB, whatever) that provides that ControllerConfig and have it picked up as a valid option? We don't want to be editing the overcloud-something.yaml because it's owned by another package and there's the potential for conflicts if multiple extra implementations start stepping on each other. An interface/discovery sort of mechanism, which I agree is more complex, would be easier to work with in those cases. I'm effectively replying to my own e-mail here, but I've expressed these thoughts on the spec and it'd probably be better to continue this train of thought there: https://review.openstack.org/#/c/196656/ Thanks for all the great feedback! So, based on your (and others) feedback, I've revised it to more of a discovery based model, where we add an optional new annotation to HOT: resource_type_mapping: resource_type: OS::TripleO::Controller This is analogous to the subsitution_mappings section defined in TOSCA, but renamed to make it a bit more HOT-ish: This would then be discoverable via heatclient, e.g: heat resource-find-template OS::TripleO::Controller which might return: puppet/controller.yaml docker/controller.yaml ... And heat would enforce type-mapping when validating the resource_registry. We could make heatclient support discovering via local filesystem, URL and Swift bucket (it already supports getting templates via all these). I've also taken a first-pass at the recursive validation spec discussed in my other thread: https://review.openstack.org/#/c/197199/ Thanks for the help defining this so far, any further feedback or ideas much appreciated! :) Steve __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [TripleO][Heat] Tuskar v. Heat responsibilities
FWIW, I liked what you were proposing in the other thread. In thinking about the deployment flow in the Tuskar-UI, I think it would enable exposing and setting the nested stack parameters easily (you choose various resources as displayed in a widget, click a reload/refresh button, and new parameters are exposed). I agree, I was thinking something similar too. There's a step to pick the larger decisions (implementations of resource types) and then a refresh that will ask Heat to recalculate the full set of parameters. What might also be neat is if something like heatclient then had support to automatically generate stub yaml environment files based on the output of the template-validate. So it could spit out a yaml file that had a parameter_defaults: section with all the expected parameters and their default values, that way the user could then just edit that stub to complete the required inputs. This is similar to what Tuskar API was looking to do. I think it'd be awesome to see Heat support it natively. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [TripleO][Heat] Tuskar v. Heat responsibilities
We could do likewise in the environment: resource_registry: OS::TripleO::ControllerConfig: puppet/controller-config.yaml ... constraints: OS::TripleO::ControllerConfig: - allowed_values: - puppet/controller-config.yaml, - foo/other-config.yaml] These constraints would be enforced at stack validation time such that the environment would be rejected if the optional constraints were not met. I like this approach. Originally, I was thinking it might be cleaner to encode the relationship in the opposite direction. Something like this in puppet/controller-config.yaml: implements: OS::TripleO::ControllerConfig But then, you leave it up to the external tools (a UI, etc) to know how to discover these implementing templates. If they're explicitly listed in a list as in your example, that helps UI's / API's more easily present these choices. Maybe it could work both ways. Yeah the strict interface definition is basically the TOSCA approach referenced by Thomas in my validation thread, and while I'm not opposed to that, it just feels like overkill for this particular problem. I don't see any mutually exclusive logic here, we could probably consider adding resource_registry constraints and still add interfaces later if it becomes apparent we really need them - atm I'm just slightly wary of adding more complexity to already complex templates, and also on relying on deep introspection to match up interfaces (when we've got no deep validation capabilities at all in heat atm) vs some simple rules in the environment. Sounds like we've got enough consensus on this idea to be worth raising a spec, I'll do that next week. I had originally been thinking of it like slagle describes, from the child up to the parent as well. What I like about that approach is that it achieves a more pluggable model when you think about extensions that aren't accepted or applicable in TripleO upstream. If someone comes along and adds a new ControllerConfig to your above example, they have to edit whatever environment you're talking about that defines the constraints (I'm call it overcloud-something.yaml for now). This becomes a problem from a packaging point of view, especially when you factor in non-TripleO integrators (without revealing too much inside baseball, think partner integrations). How do I add in an extra package (RPM, DEB, whatever) that provides that ControllerConfig and have it picked up as a valid option? We don't want to be editing the overcloud-something.yaml because it's owned by another package and there's the potential for conflicts if multiple extra implementations start stepping on each other. An interface/discovery sort of mechanism, which I agree is more complex, would be easier to work with in those cases. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [TripleO][Heat] Tuskar v. Heat responsibilities
I had originally been thinking of it like slagle describes, from the child up to the parent as well. What I like about that approach is that it achieves a more pluggable model when you think about extensions that aren't accepted or applicable in TripleO upstream. If someone comes along and adds a new ControllerConfig to your above example, they have to edit whatever environment you're talking about that defines the constraints (I'm call it overcloud-something.yaml for now). This becomes a problem from a packaging point of view, especially when you factor in non-TripleO integrators (without revealing too much inside baseball, think partner integrations). How do I add in an extra package (RPM, DEB, whatever) that provides that ControllerConfig and have it picked up as a valid option? We don't want to be editing the overcloud-something.yaml because it's owned by another package and there's the potential for conflicts if multiple extra implementations start stepping on each other. An interface/discovery sort of mechanism, which I agree is more complex, would be easier to work with in those cases. I'm effectively replying to my own e-mail here, but I've expressed these thoughts on the spec and it'd probably be better to continue this train of thought there: https://review.openstack.org/#/c/196656/ __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [TripleO][Heat] Tuskar v. Heat responsibilities
On Fri, Jun 26, 2015 at 03:05:31PM -0400, James Slagle wrote: On Thu, Jun 25, 2015 at 5:40 PM, Steven Hardy sha...@redhat.com wrote: On Tue, Jun 23, 2015 at 04:05:08PM -0400, Jay Dobies wrote: On top of that, only certain templates can be used to fulfill certain resource types. For instance, you can't point CinderBackend to rhel-registration.yaml. That information isn't explicitly captured by Heat templates. I suppose you could inspect usages of a resource type in overcloud to determine the api of that type and then compare that to possible implementation templates' parameter lists to figure out what is compatible, but that seems like a heavy-weight approach. I mention that because part of the user experience would be knowing which resource types can have a template substitution made and what possible templates can fulfill it. This is an interesting observation - assuming we wanted to solve this in Heat (and I'm not saying we necessarily should), one can imagine a resource_registry feature which works like constraints do for parameters, e.g: paramters: my_param: type: string constraints: - allowed_values: [dog, cat] We could do likewise in the environment: resource_registry: OS::TripleO::ControllerConfig: puppet/controller-config.yaml ... constraints: OS::TripleO::ControllerConfig: - allowed_values: - puppet/controller-config.yaml, - foo/other-config.yaml] These constraints would be enforced at stack validation time such that the environment would be rejected if the optional constraints were not met. I like this approach. Originally, I was thinking it might be cleaner to encode the relationship in the opposite direction. Something like this in puppet/controller-config.yaml: implements: OS::TripleO::ControllerConfig But then, you leave it up to the external tools (a UI, etc) to know how to discover these implementing templates. If they're explicitly listed in a list as in your example, that helps UI's / API's more easily present these choices. Maybe it could work both ways. Yeah the strict interface definition is basically the TOSCA approach referenced by Thomas in my validation thread, and while I'm not opposed to that, it just feels like overkill for this particular problem. I don't see any mutually exclusive logic here, we could probably consider adding resource_registry constraints and still add interfaces later if it becomes apparent we really need them - atm I'm just slightly wary of adding more complexity to already complex templates, and also on relying on deep introspection to match up interfaces (when we've got no deep validation capabilities at all in heat atm) vs some simple rules in the environment. Sounds like we've got enough consensus on this idea to be worth raising a spec, I'll do that next week. Regardless, I think this does need to be solved in Heat. We've got these features in Heat now that are enabling all this flexibility, but what we're finding is that when you try to use things like the resource registry at scale, it becomes difficult to manage. So, we end up writing a new tool, API, or whatever to do that management. If Heat doesn't solve it, it's likely that Tuskar could evolve into your Resource registry index service. I'm not saying that's what Heat needs to do now, but whatever it can enable such that consumers don't feel that they need to write an API that has to encode a bunch of additional logic, would be a good thing IMO. Historically, TripleO has leaned on new features in Heat extensively, and where things aren't available, new tooling is written to address those needs. Then Heat matures, and ends up solving just a slightly different problem in just a slightly different way, and we're having this same conversation about how we need to move forward with the TripleO tooling built on top of Heat (merge.py, now tuskar, etc). +1, I'm *really* trying to break the cycle of papering over feature gaps in heat elsewhere :) I understand why that's happened in the past, but I think it's better for everyone long term if we manage to just mature heat faster rather than maintaining workarounds elsewhere (particularly TripleO specific ones, as time has taught us TripleO requirements are normally just requirements nobody else has discovered yet..) That leaves the problem of encoding this is the resource which selects the CinderBackends, which I currently think can be adequately expressed via resource namespacing/naming? I don't think I'm following how the constraints example wouldn't solve this part as well... I guess I just meant we'll have to rely on naming e.g OS::TripleO::CinderBackend or something to express that the thing being included/configured is a CinderBackend, as opposed to some other chunk of logic. And you probably want to configure this setting after you've, say, chosen the implementation for your
Re: [openstack-dev] [TripleO][Heat] Tuskar v. Heat responsibilities
On Thu, Jun 25, 2015 at 5:40 PM, Steven Hardy sha...@redhat.com wrote: On Tue, Jun 23, 2015 at 04:05:08PM -0400, Jay Dobies wrote: On top of that, only certain templates can be used to fulfill certain resource types. For instance, you can't point CinderBackend to rhel-registration.yaml. That information isn't explicitly captured by Heat templates. I suppose you could inspect usages of a resource type in overcloud to determine the api of that type and then compare that to possible implementation templates' parameter lists to figure out what is compatible, but that seems like a heavy-weight approach. I mention that because part of the user experience would be knowing which resource types can have a template substitution made and what possible templates can fulfill it. This is an interesting observation - assuming we wanted to solve this in Heat (and I'm not saying we necessarily should), one can imagine a resource_registry feature which works like constraints do for parameters, e.g: paramters: my_param: type: string constraints: - allowed_values: [dog, cat] We could do likewise in the environment: resource_registry: OS::TripleO::ControllerConfig: puppet/controller-config.yaml ... constraints: OS::TripleO::ControllerConfig: - allowed_values: - puppet/controller-config.yaml, - foo/other-config.yaml] These constraints would be enforced at stack validation time such that the environment would be rejected if the optional constraints were not met. I like this approach. Originally, I was thinking it might be cleaner to encode the relationship in the opposite direction. Something like this in puppet/controller-config.yaml: implements: OS::TripleO::ControllerConfig But then, you leave it up to the external tools (a UI, etc) to know how to discover these implementing templates. If they're explicitly listed in a list as in your example, that helps UI's / API's more easily present these choices. Maybe it could work both ways. Regardless, I think this does need to be solved in Heat. We've got these features in Heat now that are enabling all this flexibility, but what we're finding is that when you try to use things like the resource registry at scale, it becomes difficult to manage. So, we end up writing a new tool, API, or whatever to do that management. If Heat doesn't solve it, it's likely that Tuskar could evolve into your Resource registry index service. I'm not saying that's what Heat needs to do now, but whatever it can enable such that consumers don't feel that they need to write an API that has to encode a bunch of additional logic, would be a good thing IMO. Historically, TripleO has leaned on new features in Heat extensively, and where things aren't available, new tooling is written to address those needs. Then Heat matures, and ends up solving just a slightly different problem in just a slightly different way, and we're having this same conversation about how we need to move forward with the TripleO tooling built on top of Heat (merge.py, now tuskar, etc). That leaves the problem of encoding this is the resource which selects the CinderBackends, which I currently think can be adequately expressed via resource namespacing/naming? I don't think I'm following how the constraints example wouldn't solve this part as well... == Responsibility == Where should that be implemented? That's a good question. The idea of resolving resource type uses against candidate template parameter lists could fall under the model Steve Hardy is proposing of having Heat do it (he suggested the validate call, but this may be leading us more towards template inspection sorts of APIs supported by Heat. It is also possibly an addition to HOT, to somehow convey an interface so that we can more easily programatically look at a series of templates and understand how they play together. We used to be able to use the resource_registry to understand those relationships, but that's not going to work if we're trying to find substitutions into the registry. Alternatively, if Heat/HOT has no interest in any of this, this is something that Tuskar (or a Tuskar-like substitute) will need to solve going forward. I think this problem of template composition is a general one, and I'm keen to at least partially solve it via some (hopefully relatively simple and incremental) additions to heat. Clearly there's still plenty of scope for application builder type API's on top of any new iterfaces, and I'm not trying to solve that problem, but by exposing some richer interfaces around both template inspection/validation and composition, hopefully we make such tasks easier. Yea, I think this is line with my thinking as well. Heat doesn't have to do it *all*, there's always going to be rich modeling and UX flows that are going to need separate API's. But +1 to Heat making that as easy as possible, especially when it's just enabling it's own
Re: [openstack-dev] [TripleO][Heat] Tuskar v. Heat responsibilities
On Tue, Jun 23, 2015 at 04:05:08PM -0400, Jay Dobies wrote: I didn't want to hijack Steve Hardy's thread about the recursive validation, but I wanted to summarize the needs that Tuskar and the UI have been trying to answer and some of the problems we ran into. I think it's fairly common knowledge now that Tuskar and the THT templates diverged over the past few months, so I won't rehash it. If you need a summary of what happened, look here: https://jdob.fedorapeople.org/tuskar-heat.jpg Haha, I think that's a fair analysis, although it's also true that functionality in heat and our understanding of requirements has changed a fair bit since Tuskar started out :) Below are some of the needs that the Tuskar UI in general has when working with the TripleO Heat Templates. I'm hoping we can come up with a decent list and use that to help drive what belongs in Heat v. what belongs elsewhere, and ultimately what that elsewhere actually is. Definitely, and also I've increasingly come to the realization that when TripleO needs something, it's normally because it's a generally needed thing, e.g having a deployment specific composition API is probably a mistake long-term and we do need to figure out a more general way to solve these problems. = Choosing Component Implementations = == Background == I'm already off to a bad start, since the word component isn't actually a term in this context. What I'm referring to is the fact that we are starting to see what is almost a plugin model in the THT templates. Previously, we had assumed that all of the overcloud configuration would be done through parameters. This is no longer the case as the resource_registry is used to add certain functionality. For example, in overcloud-resource-registry-puppet.yaml, we see: # set to controller-config-pacemaker.yaml to enable pacemaker OS::TripleO::ControllerConfig: puppet/controller-config.yaml That's a major overcloud configuration setting, but that choice isn't made through a parameter. It's in a different location and a different mechanism entirely. Similarly, enabling a Netapp backend for Cinder is done by setting a resource_registry entry to change the CinderBackend template [1]. This is a slightly different case conceptually than HA since the original template being overridden is a noop [2], but the mechanics of how to set it are the same. There are also a number of pre and post hooks that exist in the overcloud template that we are seeing more and more implementations of. RHEL registration is implemented as such a hook [3]. I'm drawing a difference here between fundamental configuration changes (HA v. non-HA) and optional additions (RHEL registration). Again, mechanically they are implemented as resource_registry substitutions, though from a UI standpoint we'd likely want to treat them differently. Whether or not that difference is actually captured by the templates themselves or is purely in the UI is open to debate. == Usage in TripleO == All of the examples I mentioned above have landed upstream and the Heat features necessary to facilitate them all exist. What doesn't exist is a way to manipulate the resource_registry. Tuskar doesn't have APIs for that level of changes; it assumed all configuration changes would be through parameters and hasn't yet had time to add in support for dorking with the registry in this fashion. While, technically, all of the resource_registry entries can be overridden, there are only a few that would make sense for a user to want to configure (I'm not talking about advanced users writing their own templates). On top of that, only certain templates can be used to fulfill certain resource types. For instance, you can't point CinderBackend to rhel-registration.yaml. That information isn't explicitly captured by Heat templates. I suppose you could inspect usages of a resource type in overcloud to determine the api of that type and then compare that to possible implementation templates' parameter lists to figure out what is compatible, but that seems like a heavy-weight approach. I mention that because part of the user experience would be knowing which resource types can have a template substitution made and what possible templates can fulfill it. This is an interesting observation - assuming we wanted to solve this in Heat (and I'm not saying we necessarily should), one can imagine a resource_registry feature which works like constraints do for parameters, e.g: paramters: my_param: type: string constraints: - allowed_values: [dog, cat] We could do likewise in the environment: resource_registry: OS::TripleO::ControllerConfig: puppet/controller-config.yaml ... constraints: OS::TripleO::ControllerConfig: - allowed_values: - puppet/controller-config.yaml, - foo/other-config.yaml] These constraints would be enforced at stack validation time such that the environment
[openstack-dev] [TripleO][Heat] Tuskar v. Heat responsibilities
I didn't want to hijack Steve Hardy's thread about the recursive validation, but I wanted to summarize the needs that Tuskar and the UI have been trying to answer and some of the problems we ran into. I think it's fairly common knowledge now that Tuskar and the THT templates diverged over the past few months, so I won't rehash it. If you need a summary of what happened, look here: https://jdob.fedorapeople.org/tuskar-heat.jpg Below are some of the needs that the Tuskar UI in general has when working with the TripleO Heat Templates. I'm hoping we can come up with a decent list and use that to help drive what belongs in Heat v. what belongs elsewhere, and ultimately what that elsewhere actually is. = Choosing Component Implementations = == Background == I'm already off to a bad start, since the word component isn't actually a term in this context. What I'm referring to is the fact that we are starting to see what is almost a plugin model in the THT templates. Previously, we had assumed that all of the overcloud configuration would be done through parameters. This is no longer the case as the resource_registry is used to add certain functionality. For example, in overcloud-resource-registry-puppet.yaml, we see: # set to controller-config-pacemaker.yaml to enable pacemaker OS::TripleO::ControllerConfig: puppet/controller-config.yaml That's a major overcloud configuration setting, but that choice isn't made through a parameter. It's in a different location and a different mechanism entirely. Similarly, enabling a Netapp backend for Cinder is done by setting a resource_registry entry to change the CinderBackend template [1]. This is a slightly different case conceptually than HA since the original template being overridden is a noop [2], but the mechanics of how to set it are the same. There are also a number of pre and post hooks that exist in the overcloud template that we are seeing more and more implementations of. RHEL registration is implemented as such a hook [3]. I'm drawing a difference here between fundamental configuration changes (HA v. non-HA) and optional additions (RHEL registration). Again, mechanically they are implemented as resource_registry substitutions, though from a UI standpoint we'd likely want to treat them differently. Whether or not that difference is actually captured by the templates themselves or is purely in the UI is open to debate. == Usage in TripleO == All of the examples I mentioned above have landed upstream and the Heat features necessary to facilitate them all exist. What doesn't exist is a way to manipulate the resource_registry. Tuskar doesn't have APIs for that level of changes; it assumed all configuration changes would be through parameters and hasn't yet had time to add in support for dorking with the registry in this fashion. While, technically, all of the resource_registry entries can be overridden, there are only a few that would make sense for a user to want to configure (I'm not talking about advanced users writing their own templates). On top of that, only certain templates can be used to fulfill certain resource types. For instance, you can't point CinderBackend to rhel-registration.yaml. That information isn't explicitly captured by Heat templates. I suppose you could inspect usages of a resource type in overcloud to determine the api of that type and then compare that to possible implementation templates' parameter lists to figure out what is compatible, but that seems like a heavy-weight approach. I mention that because part of the user experience would be knowing which resource types can have a template substitution made and what possible templates can fulfill it. == Responsibility == Where should that be implemented? That's a good question. The idea of resolving resource type uses against candidate template parameter lists could fall under the model Steve Hardy is proposing of having Heat do it (he suggested the validate call, but this may be leading us more towards template inspection sorts of APIs supported by Heat. It is also possibly an addition to HOT, to somehow convey an interface so that we can more easily programatically look at a series of templates and understand how they play together. We used to be able to use the resource_registry to understand those relationships, but that's not going to work if we're trying to find substitutions into the registry. Alternatively, if Heat/HOT has no interest in any of this, this is something that Tuskar (or a Tuskar-like substitute) will need to solve going forward. = Consolidated Parameter List = == Background == This is what Steve was getting at in his e-mail. I'll rehash the issue briefly. We used to be able to look at the parameters list in the overcloud template and know all of the parameters that need to be specified to configure the overcloud. The parameter passing is pretty strict, so if overcloud
