Re: [openstack-dev] [heat][hot]
Thank you very much
On Tue, Jan 27, 2015 at 1:06 PM, Angus Salkeld asalk...@mirantis.com
wrote:
On Tue, Jan 27, 2015 at 7:00 PM, Dmitry mey...@gmail.com wrote:
I have another question, is it possible to get the stack name in the hot
script?
E.g.
params:
$stack_name: {get_global_variable: $stack.name}
See:
http://docs.openstack.org/developer/heat/template_guide/hot_spec.html#pseudo-parameters
Regards
Angus
On Tue, Jan 27, 2015 at 3:53 AM, Qiming Teng teng...@linux.vnet.ibm.com
wrote:
On Mon, Jan 26, 2015 at 07:44:25PM +0200, Dmitry wrote:
thanks, exactly what I was looking for:
curl http://169.254.169.254/1.0/meta-data/instance-id
or, /var/lib/cloud/data/instance-id, if cloud-init is there.
Regards,
Qiming
On Mon, Jan 26, 2015 at 7:31 PM, Zane Bitter zbit...@redhat.com
wrote:
On 25/01/15 10:41, Dmitry wrote:
Hello,
I need to receive instance id as part of the instance installation
script.
Something like:
params:
$current_id: {get_param: $this.id http://this.id}
I have no idea what this is supposed to mean, sorry.
Is it possible?
The get_resource function will return the server UUID for a server
resource, but you can't use it from within that resource itself (it
would
be a circular reference).
The UUID of a server is provided to the server through the Nova
metadata;
you should retrieve it from there in your user_data script.
cheers,
Zane.
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe:
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe:
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe:
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe:
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [heat][hot]
I have another question, is it possible to get the stack name in the hot
script?
E.g.
params:
$stack_name: {get_global_variable: $stack.name}
On Tue, Jan 27, 2015 at 3:53 AM, Qiming Teng teng...@linux.vnet.ibm.com
wrote:
On Mon, Jan 26, 2015 at 07:44:25PM +0200, Dmitry wrote:
thanks, exactly what I was looking for:
curl http://169.254.169.254/1.0/meta-data/instance-id
or, /var/lib/cloud/data/instance-id, if cloud-init is there.
Regards,
Qiming
On Mon, Jan 26, 2015 at 7:31 PM, Zane Bitter zbit...@redhat.com wrote:
On 25/01/15 10:41, Dmitry wrote:
Hello,
I need to receive instance id as part of the instance installation
script.
Something like:
params:
$current_id: {get_param: $this.id http://this.id}
I have no idea what this is supposed to mean, sorry.
Is it possible?
The get_resource function will return the server UUID for a server
resource, but you can't use it from within that resource itself (it
would
be a circular reference).
The UUID of a server is provided to the server through the Nova
metadata;
you should retrieve it from there in your user_data script.
cheers,
Zane.
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe:
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe:
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [heat][hot]
On Tue, Jan 27, 2015 at 7:00 PM, Dmitry mey...@gmail.com wrote:
I have another question, is it possible to get the stack name in the hot
script?
E.g.
params:
$stack_name: {get_global_variable: $stack.name}
See:
http://docs.openstack.org/developer/heat/template_guide/hot_spec.html#pseudo-parameters
Regards
Angus
On Tue, Jan 27, 2015 at 3:53 AM, Qiming Teng teng...@linux.vnet.ibm.com
wrote:
On Mon, Jan 26, 2015 at 07:44:25PM +0200, Dmitry wrote:
thanks, exactly what I was looking for:
curl http://169.254.169.254/1.0/meta-data/instance-id
or, /var/lib/cloud/data/instance-id, if cloud-init is there.
Regards,
Qiming
On Mon, Jan 26, 2015 at 7:31 PM, Zane Bitter zbit...@redhat.com
wrote:
On 25/01/15 10:41, Dmitry wrote:
Hello,
I need to receive instance id as part of the instance installation
script.
Something like:
params:
$current_id: {get_param: $this.id http://this.id}
I have no idea what this is supposed to mean, sorry.
Is it possible?
The get_resource function will return the server UUID for a server
resource, but you can't use it from within that resource itself (it
would
be a circular reference).
The UUID of a server is provided to the server through the Nova
metadata;
you should retrieve it from there in your user_data script.
cheers,
Zane.
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe:
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe:
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe:
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [heat][hot]
I'm using OS::Nova::Server, the versions is 2013-05-23
On Mon, Jan 26, 2015 at 5:10 PM, Qiming Teng teng...@linux.vnet.ibm.com
wrote:
On Sun, Jan 25, 2015 at 05:41:33PM +0200, Dmitry wrote:
Hello,
I need to receive instance id as part of the instance installation
script.
Something like:
params:
$current_id: {get_param: $this.id}
Please be specific about the 'installation script', i.e. which resource
type and which template version are you using?
Regards,
Qiming
Is it possible?
Thanks,
Dmitry
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe:
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [heat][hot]
On Sun, Jan 25, 2015 at 05:41:33PM +0200, Dmitry wrote:
Hello,
I need to receive instance id as part of the instance installation script.
Something like:
params:
$current_id: {get_param: $this.id}
Please be specific about the 'installation script', i.e. which resource
type and which template version are you using?
Regards,
Qiming
Is it possible?
Thanks,
Dmitry
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [heat][hot]
On 25/01/15 10:41, Dmitry wrote:
Hello,
I need to receive instance id as part of the instance installation script.
Something like:
params:
$current_id: {get_param: $this.id http://this.id}
I have no idea what this is supposed to mean, sorry.
Is it possible?
The get_resource function will return the server UUID for a server
resource, but you can't use it from within that resource itself (it
would be a circular reference).
The UUID of a server is provided to the server through the Nova
metadata; you should retrieve it from there in your user_data script.
cheers,
Zane.
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [heat][hot]
On Mon, Jan 26, 2015 at 07:44:25PM +0200, Dmitry wrote:
thanks, exactly what I was looking for:
curl http://169.254.169.254/1.0/meta-data/instance-id
or, /var/lib/cloud/data/instance-id, if cloud-init is there.
Regards,
Qiming
On Mon, Jan 26, 2015 at 7:31 PM, Zane Bitter zbit...@redhat.com wrote:
On 25/01/15 10:41, Dmitry wrote:
Hello,
I need to receive instance id as part of the instance installation script.
Something like:
params:
$current_id: {get_param: $this.id http://this.id}
I have no idea what this is supposed to mean, sorry.
Is it possible?
The get_resource function will return the server UUID for a server
resource, but you can't use it from within that resource itself (it would
be a circular reference).
The UUID of a server is provided to the server through the Nova metadata;
you should retrieve it from there in your user_data script.
cheers,
Zane.
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [heat][hot]
Hello,
I need to receive instance id as part of the instance installation script.
Something like:
params:
$current_id: {get_param: $this.id}
Is it possible?
Thanks,
Dmitry
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Heat] HOT software configuration refined after design summit discussions
On 11/21/2013 08:48 PM, Thomas Spatzier wrote: Excerpts from Steve Baker's message on 21.11.2013 00:00:47: From: Steve Baker sba...@redhat.com To: openstack-dev@lists.openstack.org, Date: 21.11.2013 00:04 Subject: Re: [openstack-dev] [Heat] HOT software configuration refined after design summit discussions On 11/21/2013 11:41 AM, Clint Byrum wrote: Excerpts from Mike Spreitzer's message of 2013-11-20 13:46:25 -0800: Clint Byrum cl...@fewbar.com wrote on 11/19/2013 04:28:31 PM: snip I am worried about the explosion of possibilities that comes from trying to deal with all of the diff's possible inside an instance. If there is an actual REST interface for a thing, then yes, let's use that. For instance, if we are using docker, there is in fact a very straight forward way to say remove entity X. If we are using packages we have the same thing. However, if we are just trying to write chef configurations, we have to write reverse chef configurations. What I meant to convey is let's give this piece of the interface a lot of thought. Not this is wrong to even have. Given a couple of days now, I think we do need apply and remove. We should also provide really solid example templates for this concept. You're right, I'm already starting to see issues with my current approach. This smells like a new blueprint. I'll remove it from the scope of the current software config work and raise a blueprint to track remove-config. So I read thru those recent discussions and in parallel also started to update the design wiki. BTW, nanjj renamed the wiki to [1] (but also made a redirect from the previous ...-WIP page) and linked it as spec to BP [2]. I'll leave out the remove-config thing for now. While thinking about the overall picture, I came up with some other comments: I thought about the name SoftwareApplier some more and while it is clear what it does (it applies a software config to a server), the naming is not really consistent with all the other resources in Heat. Every other resource type is called after the thing that you get when the template gets instantiated (a Server, a FloatingIP, a VolumeAttachment etc). In case of SoftwareApplier what you actually get from a user perspective is a deployed instance of the piece of software described be a SoftwareConfig. Therefore, I was calling it SoftwareDeployment orignally, because you get a software deployment (according to a config). Any comments on that name? SoftwareDeployment is a better name, apart from those 3 extra letters. I'll rename my POC. Sorry nannj, you'll need to rename them back ;) If we think this thru with respect to remove-config (even though this needs more thought), a SoftwareApplier (that thing itself) would not really go to state DELETE_IN_PROGRESS during an update. It is always there on the VM but the software it deploys gets deleted and then reapplied or whatever ... Now thinking more about update scenarios (which we can leave for an iteration after the initial deployment is working), in my mental model it would be more consistent to have information for handle_create, handle_delete, handle_update kinds of events all defined in the SoftwareConfig resource. SoftwareConfig for represent configuration information for one specific piece of software, e.g. a web server. So it could provide all the information you need to install it, to uninstall it, or to update its config. By updating the SoftwareApplier's (or SoftwareDeployment's - my preferred name) state at runtime, the in-instance tools would grab the respective script of whatever an run it. So SoftwareConfig could look like: resources: my_webserver_config: type: OS::Heat::SoftwareConfig properties: http_port: type: number # some more config props config_create: http://www.example.com/my_scripts/webserver/install.sh config_delete: http://www.example.com/my_scripts/webserver/uninstall.sh config_update: http://www.example.com/my_scripts/webserver/applyupdate.sh At runtime, when a SoftwareApplier gets created, it looks for the 'config_create' hook and triggers that automation. When it gets deleted, it looks for the 'config_delete' hook and so on. Only config_create is mandatory. I think that would also give us nice extensibility for future use cases. For example, Heat today does not support something like stop-stack or start-stack which would be pretty useful though. If we have it one day, we would just add a 'config_start' hook to the SoftwareConfig. [1] https://wiki.openstack.org/wiki/Heat/Blueprints/hot-software-config-spec [2] https://blueprints.launchpad.net/heat/+spec/hot-software-config With the caveat that what we're discussing here is a future enhancement... The problem I see with config_create/config_update/config_delete in a single SoftwareConfig is that we probably can't assume these 3 scripts consume the same inputs and produce the same outputs
Re: [openstack-dev] [Heat] HOT software configuration refined after design summit discussions
Thomas Spatzier thomas.spatz...@de.ibm.com wrote on 11/21/2013 02:48:14 AM: ... Now thinking more about update scenarios (which we can leave for an iteration after the initial deployment is working), I recommend thinking about UPDATE from the start. We should have an implementation in which CREATE and UPDATE share as much mechanism as is reasonable, which requires thinking about UPDATE while designing CREATE. in my mental model it would be more consistent to have information for handle_create, handle_delete, handle_update kinds of events all defined in the SoftwareConfig resource. +1 for putting these on the definition instead of the use; I also noted this earlier. -1 for having an update method. The orientation to idempotent forward-progress operations means that we need only one, which handles both CREATE and UPDATE. Regards, Mike ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Heat] HOT software configuration refined after design summit discussions
Steve Baker sba...@redhat.com wrote on 21.11.2013 21:19:07: From: Steve Baker sba...@redhat.com To: openstack-dev@lists.openstack.org, Date: 21.11.2013 21:25 Subject: Re: [openstack-dev] [Heat] HOT software configuration refined after design summit discussions On 11/21/2013 08:48 PM, Thomas Spatzier wrote: Excerpts from Steve Baker's message on 21.11.2013 00:00:47: From: Steve Baker sba...@redhat.com To: openstack-dev@lists.openstack.org, Date: 21.11.2013 00:04 Subject: Re: [openstack-dev] [Heat] HOT software configuration refined after design summit discussions snip I thought about the name SoftwareApplier some more and while it is clear what it does (it applies a software config to a server), the naming is not really consistent with all the other resources in Heat. Every other resource type is called after the thing that you get when the template gets instantiated (a Server, a FloatingIP, a VolumeAttachment etc). In case of SoftwareApplier what you actually get from a user perspective is a deployed instance of the piece of software described be a SoftwareConfig. Therefore, I was calling it SoftwareDeployment orignally, because you get a software deployment (according to a config). Any comments on that name? SoftwareDeployment is a better name, apart from those 3 extra letters. I'll rename my POC. Sorry nannj, you'll need to rename them back ;) Ok, I'll change the name back in the wiki :-) If we think this thru with respect to remove-config (even though this needs more thought), a SoftwareApplier (that thing itself) would not really go to state DELETE_IN_PROGRESS during an update. It is always there on the VM but the software it deploys gets deleted and then reapplied or whatever ... Now thinking more about update scenarios (which we can leave for an iteration after the initial deployment is working), in my mental model it would be more consistent to have information for handle_create, handle_delete, handle_update kinds of events all defined in the SoftwareConfig resource. SoftwareConfig for represent configuration information for one specific piece of software, e.g. a web server. So it could provide all the information you need to install it, to uninstall it, or to update its config. By updating the SoftwareApplier's (or SoftwareDeployment's - my preferred name) state at runtime, the in-instance tools would grab the respective script of whatever an run it. So SoftwareConfig could look like: resources: my_webserver_config: type: OS::Heat::SoftwareConfig properties: http_port: type: number # some more config props config_create: http://www.example.com/my_scripts/webserver/install.sh config_delete: http://www.example.com/my_scripts/webserver/uninstall.sh config_update: http://www.example.com/my_scripts/webserver/applyupdate.sh At runtime, when a SoftwareApplier gets created, it looks for the 'config_create' hook and triggers that automation. When it gets deleted, it looks for the 'config_delete' hook and so on. Only config_create is mandatory. I think that would also give us nice extensibility for future use cases. For example, Heat today does not support something like stop-stack or start-stack which would be pretty useful though. If we have it one day, we would just add a 'config_start' hook to the SoftwareConfig. [1] https://wiki.openstack.org/wiki/Heat/Blueprints/hot-software-config-spec [2] https://blueprints.launchpad.net/heat/+spec/hot-software-config With the caveat that what we're discussing here is a future enhancement... The problem I see with config_create/config_update/config_delete in a single SoftwareConfig is that we probably can't assume these 3 scripts consume the same inputs and produce the same outputs. We could make it a convention that creators of software configs have to use the same signature for the automation of create, delete etc. Or at least input param names must be the same, while some pieces might take a subset only. E.g. delete will probably take less inputs. This way we could have a self-contained config. As you said above, implementation-wise this is probably a future enhancement, so once we have he config_create handling in place we could just do a PoC patch on-top and try it out. Another option might be to have a separate confg/deployment pair for delete workloads, and a property on the deployment resource which states which phase the workload is executed in (create or delete). Yes, this would be an option, but IMO a bit confusing for users. Especially when I inspect a deployed stack, I would be wondering why there are many SoftwareDeployment resources hanging around for the same piece of software installed on a server. I'd like to think that special treatment for config_update won't be needed at all, since CM tools are supposed to be good at converging to whatever you
Re: [openstack-dev] [Heat] HOT software configuration refined after design summit discussions
Excerpts from Thomas Spatzier's message of 2013-11-19 23:35:40 -0800: Excerpts from Steve Baker's message on 19.11.2013 21:40:54: From: Steve Baker sba...@redhat.com To: openstack-dev@lists.openstack.org, Date: 19.11.2013 21:43 Subject: Re: [openstack-dev] [Heat] HOT software configuration refined after design summit discussions snip I think there needs to a CM tool specific agent delivered to the server which os-collect-config invokes. This agent will transform the config data (input values, CM script, CM specific specialness) to a CM tool invocation. How to define and deliver this agent is the challenge. Some options are: 1) install it as part of the image customization/bootstrapping (golden images or cloud-init) 2) define a (mustache?) template in the SoftwareConfig which os-collect-config transforms into the agent script, which os-collect-config then executes 3) a CM tool specific implementation of SoftwareApplier builds and delivers a complete agent to os-collect-config which executes it I may be leaning towards 3) at the moment. Hopefully any agent can be generated with a sufficiently sophisticated base SoftwareApplier type, plus maybe some richer intrinsic functions. This is good summary of options; about the same we had in mind. And we were also leaning towards 3. Probably the approach we would take is to get a SoftwareApplier running for one CM tool (e.g. Chef), then look at another tool (base shell scripts), and then see what the generic parts art that can be factored into a base class. The POC I'm working on is actually backed by a REST API which does dumb (but structured) storage of SoftwareConfig and SoftwareApplier entities. This has some interesting implications for managing SoftwareConfig resources outside the context of the stack which uses them, but lets not worry too much about that *yet*. Sounds good. We are also defining some blueprints to break down the overall software config topic. We plan to share them later this week, and then we can consolidate with your plans and see how we can best join forces. At this point it would be very helpful to spec out how specific CM tools are invoked with given inputs, script, and CM tool specific options. That's our plan; and we would probably start with scripts and chef. Maybe if you start with shell scripts, cfn-init and chef then we can all contribute other CM tools like os-config-applier, puppet, ansible, saltstack. Hopefully by then my POC will at least be able to create resources, if not deliver some data to servers. We've been thinking about getting metadata to the in-instance parts on the server and whether the resources you are building can serve the purpose. I.e. pass and endpoint to the SoftwareConfig resources to the instance and let the instance query the metadata from the resource. Sounds like this is what you had in mind, so that would be a good point for integrating the work. In the meantime, we can think of some shortcuts. Note that os-collect-config is intended to be a light-weight generic in-instance agent to do exactly this. Watch for Metadata changes, and feed them to an underlying tool in a predictable interface. I'd hope that any of the appliers would mostly just configure os-collect-config to run a wrapper that speaks os-collect-config's interface. The interface is defined in the README: https://pypi.python.org/pypi/os-collect-config It is inevitable that we will extend os-collect-config to be able to collect config data from whatever API these config applier resources make available. I would suggest then that we not all go off and reinvent os-collect-config for each applier, but rather enhance os-collect-config as needed and write wrappers for the other config tools which implement its interface. os-apply-config already understands this interface for obvious reasons. Bash scripts can use os-apply-config to extract individual values, as you might see in some of the os-refresh-config scripts that are run as part of tripleo. I don't think anything further is really needed there. For chef, some kind of ohai plugin to read os-collect-config's collected data would make sense. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Heat] HOT software configuration refined after design summit discussions
On 11/20/2013 09:29 PM, Clint Byrum wrote: Excerpts from Thomas Spatzier's message of 2013-11-19 23:35:40 -0800: Excerpts from Steve Baker's message on 19.11.2013 21:40:54: From: Steve Baker sba...@redhat.com To: openstack-dev@lists.openstack.org, Date: 19.11.2013 21:43 Subject: Re: [openstack-dev] [Heat] HOT software configuration refined after design summit discussions snip I think there needs to a CM tool specific agent delivered to the server which os-collect-config invokes. This agent will transform the config data (input values, CM script, CM specific specialness) to a CM tool invocation. How to define and deliver this agent is the challenge. Some options are: 1) install it as part of the image customization/bootstrapping (golden images or cloud-init) 2) define a (mustache?) template in the SoftwareConfig which os-collect-config transforms into the agent script, which os-collect-config then executes 3) a CM tool specific implementation of SoftwareApplier builds and delivers a complete agent to os-collect-config which executes it I may be leaning towards 3) at the moment. Hopefully any agent can be generated with a sufficiently sophisticated base SoftwareApplier type, plus maybe some richer intrinsic functions. This is good summary of options; about the same we had in mind. And we were also leaning towards 3. Probably the approach we would take is to get a SoftwareApplier running for one CM tool (e.g. Chef), then look at another tool (base shell scripts), and then see what the generic parts art that can be factored into a base class. The POC I'm working on is actually backed by a REST API which does dumb (but structured) storage of SoftwareConfig and SoftwareApplier entities. This has some interesting implications for managing SoftwareConfig resources outside the context of the stack which uses them, but lets not worry too much about that *yet*. Sounds good. We are also defining some blueprints to break down the overall software config topic. We plan to share them later this week, and then we can consolidate with your plans and see how we can best join forces. At this point it would be very helpful to spec out how specific CM tools are invoked with given inputs, script, and CM tool specific options. That's our plan; and we would probably start with scripts and chef. Maybe if you start with shell scripts, cfn-init and chef then we can all contribute other CM tools like os-config-applier, puppet, ansible, saltstack. Hopefully by then my POC will at least be able to create resources, if not deliver some data to servers. We've been thinking about getting metadata to the in-instance parts on the server and whether the resources you are building can serve the purpose. I.e. pass and endpoint to the SoftwareConfig resources to the instance and let the instance query the metadata from the resource. Sounds like this is what you had in mind, so that would be a good point for integrating the work. In the meantime, we can think of some shortcuts. Note that os-collect-config is intended to be a light-weight generic in-instance agent to do exactly this. Watch for Metadata changes, and feed them to an underlying tool in a predictable interface. I'd hope that any of the appliers would mostly just configure os-collect-config to run a wrapper that speaks os-collect-config's interface. The interface is defined in the README: https://pypi.python.org/pypi/os-collect-config It is inevitable that we will extend os-collect-config to be able to collect config data from whatever API these config applier resources make available. I would suggest then that we not all go off and reinvent os-collect-config for each applier, but rather enhance os-collect-config as needed and write wrappers for the other config tools which implement its interface. os-apply-config already understands this interface for obvious reasons. Bash scripts can use os-apply-config to extract individual values, as you might see in some of the os-refresh-config scripts that are run as part of tripleo. I don't think anything further is really needed there. For chef, some kind of ohai plugin to read os-collect-config's collected data would make sense. I'd definitely start with occ as Clint outlines. It would be nice if occ only had to be configured to poll metadata for the OS::Nova::Server to fetch the aggregated data for the currently available SoftwareAppliers. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Heat] HOT software configuration refined after design summit discussions
Steve Baker sba...@redhat.com wrote on 20.11.2013 09:51:34: From: Steve Baker sba...@redhat.com To: openstack-dev@lists.openstack.org, Date: 20.11.2013 09:55 Subject: Re: [openstack-dev] [Heat] HOT software configuration refined after design summit discussions On 11/20/2013 09:29 PM, Clint Byrum wrote: Excerpts from Thomas Spatzier's message of 2013-11-19 23:35:40 -0800: Excerpts from Steve Baker's message on 19.11.2013 21:40:54: From: Steve Baker sba...@redhat.com To: openstack-dev@lists.openstack.org, Date: 19.11.2013 21:43 Subject: Re: [openstack-dev] [Heat] HOT software configuration refined after design summit discussions snip I think there needs to a CM tool specific agent delivered to the server which os-collect-config invokes. This agent will transform the config data (input values, CM script, CM specific specialness) to a CM tool invocation. How to define and deliver this agent is the challenge. Some options are: 1) install it as part of the image customization/bootstrapping (golden images or cloud-init) 2) define a (mustache?) template in the SoftwareConfig which os-collect-config transforms into the agent script, which os-collect-config then executes 3) a CM tool specific implementation of SoftwareApplier builds and delivers a complete agent to os-collect-config which executes it I may be leaning towards 3) at the moment. Hopefully any agent can be generated with a sufficiently sophisticated base SoftwareApplier type, plus maybe some richer intrinsic functions. This is good summary of options; about the same we had in mind. And we were also leaning towards 3. Probably the approach we would take is to get a SoftwareApplier running for one CM tool (e.g. Chef), then look at another tool (base shell scripts), and then see what the generic parts art that can be factored into a base class. The POC I'm working on is actually backed by a REST API which does dumb (but structured) storage of SoftwareConfig and SoftwareApplier entities. This has some interesting implications for managing SoftwareConfig resources outside the context of the stack which uses them, but lets not worry too much about that *yet*. Sounds good. We are also defining some blueprints to break down the overall software config topic. We plan to share them later this week, and then we can consolidate with your plans and see how we can best join forces. At this point it would be very helpful to spec out how specific CM tools are invoked with given inputs, script, and CM tool specific options. That's our plan; and we would probably start with scripts and chef. Maybe if you start with shell scripts, cfn-init and chef then we can all contribute other CM tools like os-config-applier, puppet, ansible, saltstack. Hopefully by then my POC will at least be able to create resources, if not deliver some data to servers. We've been thinking about getting metadata to the in-instance parts on the server and whether the resources you are building can serve the purpose. I.e. pass and endpoint to the SoftwareConfig resources to the instance and let the instance query the metadata from the resource. Sounds like this is what you had in mind, so that would be a good point for integrating the work. In the meantime, we can think of some shortcuts. Note that os-collect-config is intended to be a light-weight generic in-instance agent to do exactly this. Watch for Metadata changes, and feed them to an underlying tool in a predictable interface. I'd hope that any of the appliers would mostly just configure os-collect-config to run a wrapper that speaks os-collect-config's interface. The interface is defined in the README: https://pypi.python.org/pypi/os-collect-config It is inevitable that we will extend os-collect-config to be able to collect config data from whatever API these config applier resources make available. I would suggest then that we not all go off and reinvent os-collect-config for each applier, but rather enhance os-collect-config as needed and write wrappers for the other config tools which implement its interface. os-apply-config already understands this interface for obvious reasons. Bash scripts can use os-apply-config to extract individual values, as you might see in some of the os-refresh-config scripts that are run as part of tripleo. I don't think anything further is really needed there. For chef, some kind of ohai plugin to read os-collect-config's collected data would make sense. Thanks for all that information, Clint. Fully agree that we should leverage what is already there instead of re-inventing the wheel. I'd definitely start with occ as Clint outlines. It would be nice if occ only had to be configured to poll metadata for the OS::Nova::Server to fetch the aggregated data for the currently available SoftwareAppliers. Yep, sounds like a plan
Re: [openstack-dev] [Heat] HOT software configuration refined after design summit discussions
Steve Baker sba...@redhat.com wrote on 11/19/2013 03:40:54 PM: ... How to define and deliver this agent is the challenge. Some options are: 1) install it as part of the image customization/bootstrapping (golden images or cloud-init) 2) define a (mustache?) template in the SoftwareConfig which os-collect-config transforms into the agent script, which os-collect-config then executes I do not follow what you mean here. Can you please elaborate a little? 3) a CM tool specific implementation of SoftwareApplier builds and delivers a complete agent to os-collect-config which executes it Could instead the agent be a package that the applier templates asks to be installed? Thanks, Mike___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Heat] HOT software configuration refined after design summit discussions
Clint Byrum cl...@fewbar.com wrote on 11/19/2013 04:28:31 PM: From: Clint Byrum cl...@fewbar.com To: openstack-dev openstack-dev@lists.openstack.org, Date: 11/19/2013 04:30 PM Subject: Re: [openstack-dev] [Heat] HOT software configuration refined after design summit discussions Excerpts from Steve Baker's message of 2013-11-19 13:06:21 -0800: On 11/20/2013 09:50 AM, Clint Byrum wrote: Excerpts from Steve Baker's message of 2013-11-18 12:52:04 -0800: Regarding apply_config/remove_config, if a SoftwareApplier resource is deleted it should trigger any remove_config and wait for the server to acknowledge when that is complete. This allows for any evacuation/deregistering workloads to be executed. I'm a little worried about the road that leads us down. Most configuration software defines forward progress only. Meaning, if you want something not there, you don't remove it from your assertions, you assert that it is not there. I am worried too. But I do not entirely follow your reasoning. When I UPDATE a stack with a new template, am I supposed to write in that template not just what I want the stack to be but also how that differs from what it currently is? That is not REST. Not that I am a total REST zealot, but I am a fan of managing in terms of desired state. But I agree there is a conflict between defining a 'remove' operation and the forward progress only mindset of most config tooling. ... A specific use-case I'm trying to address here is tripleo doing an update-replace on a nova compute node. The remove_config contains the workload to evacuate VMs and signal heat when the node is ready to be shut down. This is more involved than just uninstall the things. Could you outline in some more detail how you think this could be done? So for that we would not remove the software configuration for the nova-compute, we would assert that the machine needs vms evacuated. We want evacuation to be something we explicitly do, not a side effect of deleting things. Really? You want to force the user to explicitly say evacuate the VMs in all the various ways a host deletion can happen? E.g., when an autoscaling group of hosts shrinks? Perhaps having delete hooks for starting delete work-flows is right, but it set off a red flag for me so I want to make sure we think it through. Also IIRC, evacuation is not necessarily an in-instance thing. It looks more like the weird thing we've been talking about lately which is how do we orchestrate tenant API's: https://etherpad.openstack.org/p/orchestrate-tenant-apis This looks promising to me. Regards, Mike___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Heat] HOT software configuration refined after design summit discussions
On 11/21/2013 10:46 AM, Mike Spreitzer wrote: Clint Byrum cl...@fewbar.com wrote on 11/19/2013 04:28:31 PM: From: Clint Byrum cl...@fewbar.com To: openstack-dev openstack-dev@lists.openstack.org, Date: 11/19/2013 04:30 PM Subject: Re: [openstack-dev] [Heat] HOT software configuration refined after design summit discussions Excerpts from Steve Baker's message of 2013-11-19 13:06:21 -0800: On 11/20/2013 09:50 AM, Clint Byrum wrote: Excerpts from Steve Baker's message of 2013-11-18 12:52:04 -0800: Regarding apply_config/remove_config, if a SoftwareApplier resource is deleted it should trigger any remove_config and wait for the server to acknowledge when that is complete. This allows for any evacuation/deregistering workloads to be executed. I'm a little worried about the road that leads us down. Most configuration software defines forward progress only. Meaning, if you want something not there, you don't remove it from your assertions, you assert that it is not there. I am worried too. But I do not entirely follow your reasoning. When I UPDATE a stack with a new template, am I supposed to write in that template not just what I want the stack to be but also how that differs from what it currently is? That is not REST. Not that I am a total REST zealot, but I am a fan of managing in terms of desired state. But I agree there is a conflict between defining a 'remove' operation and the forward progress only mindset of most config tooling. As I'm currently proposing, here are some stack update scenarios: * update results in modified software config, apply_config will be executed again on the affected server * update results in a server that requires replacement. This results in: * execute the remove_config workload on that server. The SoftwareApplier resource remains in DELETE_IN_PROGRESS until signalled that remove_config is complete * delete the server * create the replacement server * execute apply_config on that server... ... A specific use-case I'm trying to address here is tripleo doing an update-replace on a nova compute node. The remove_config contains the workload to evacuate VMs and signal heat when the node is ready to be shut down. This is more involved than just uninstall the things. Could you outline in some more detail how you think this could be done? So for that we would not remove the software configuration for the nova-compute, we would assert that the machine needs vms evacuated. We want evacuation to be something we explicitly do, not a side effect of deleting things. Really? You want to force the user to explicitly say evacuate the VMs in all the various ways a host deletion can happen? E.g., when an autoscaling group of hosts shrinks? Nobody is being forced. remove_config is entirely optional and only exists for the more complex scenarios requiring evacuation/deregistering. If remove_config is not specified, the SoftwareApplier should probably go straight to DELETE_COMPLETE without waiting for any signal. Perhaps having delete hooks for starting delete work-flows is right, but it set off a red flag for me so I want to make sure we think it through. Also IIRC, evacuation is not necessarily an in-instance thing. It looks more like the weird thing we've been talking about lately which is how do we orchestrate tenant API's: https://etherpad.openstack.org/p/orchestrate-tenant-apis This looks promising to me. It looks like these might be represented as SoftwareConfigs ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Heat] HOT software configuration refined after design summit discussions
Excerpts from Mike Spreitzer's message of 2013-11-20 13:46:25 -0800: Clint Byrum cl...@fewbar.com wrote on 11/19/2013 04:28:31 PM: From: Clint Byrum cl...@fewbar.com To: openstack-dev openstack-dev@lists.openstack.org, Date: 11/19/2013 04:30 PM Subject: Re: [openstack-dev] [Heat] HOT software configuration refined after design summit discussions Excerpts from Steve Baker's message of 2013-11-19 13:06:21 -0800: On 11/20/2013 09:50 AM, Clint Byrum wrote: Excerpts from Steve Baker's message of 2013-11-18 12:52:04 -0800: Regarding apply_config/remove_config, if a SoftwareApplier resource is deleted it should trigger any remove_config and wait for the server to acknowledge when that is complete. This allows for any evacuation/deregistering workloads to be executed. I'm a little worried about the road that leads us down. Most configuration software defines forward progress only. Meaning, if you want something not there, you don't remove it from your assertions, you assert that it is not there. I am worried too. But I do not entirely follow your reasoning. When I UPDATE a stack with a new template, am I supposed to write in that template not just what I want the stack to be but also how that differs from what it currently is? That is not REST. Not that I am a total REST zealot, but I am a fan of managing in terms of desired state. But I agree there is a conflict between defining a 'remove' operation and the forward progress only mindset of most config tooling. I am worried about the explosion of possibilities that comes from trying to deal with all of the diff's possible inside an instance. If there is an actual REST interface for a thing, then yes, let's use that. For instance, if we are using docker, there is in fact a very straight forward way to say remove entity X. If we are using packages we have the same thing. However, if we are just trying to write chef configurations, we have to write reverse chef configurations. What I meant to convey is let's give this piece of the interface a lot of thought. Not this is wrong to even have. Given a couple of days now, I think we do need apply and remove. We should also provide really solid example templates for this concept. ... A specific use-case I'm trying to address here is tripleo doing an update-replace on a nova compute node. The remove_config contains the workload to evacuate VMs and signal heat when the node is ready to be shut down. This is more involved than just uninstall the things. Could you outline in some more detail how you think this could be done? So for that we would not remove the software configuration for the nova-compute, we would assert that the machine needs vms evacuated. We want evacuation to be something we explicitly do, not a side effect of deleting things. Really? You want to force the user to explicitly say evacuate the VMs in all the various ways a host deletion can happen? E.g., when an autoscaling group of hosts shrinks? Autoscaling doesn't really fly with stateful services IMO. Also for TripleO's use case, auto-scaling is not really a high priority. Hardware isn't nearly as easily allocatable as VM's. Anyway, there is a really complicated work-flow for decomissioning any stateful service, and it differs wildly between them. I do want to have a place to define that work-flow and reliably trigger it when it needs to be triggered. I do not want it to _only_ be available in the delete this resource case, and I also do not want it to _always_ be run in that case, as I may legitimately be destroying the data too. I need a way to express that intention, and in my mind, the way to do that is to first complete an evacuation and then delete the thing. Better ideas are _most_ welcome. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Heat] HOT software configuration refined after design summit discussions
On 11/21/2013 11:41 AM, Clint Byrum wrote: Excerpts from Mike Spreitzer's message of 2013-11-20 13:46:25 -0800: Clint Byrum cl...@fewbar.com wrote on 11/19/2013 04:28:31 PM: From: Clint Byrum cl...@fewbar.com To: openstack-dev openstack-dev@lists.openstack.org, Date: 11/19/2013 04:30 PM Subject: Re: [openstack-dev] [Heat] HOT software configuration refined after design summit discussions Excerpts from Steve Baker's message of 2013-11-19 13:06:21 -0800: On 11/20/2013 09:50 AM, Clint Byrum wrote: Excerpts from Steve Baker's message of 2013-11-18 12:52:04 -0800: Regarding apply_config/remove_config, if a SoftwareApplier resource is deleted it should trigger any remove_config and wait for the server to acknowledge when that is complete. This allows for any evacuation/deregistering workloads to be executed. I'm a little worried about the road that leads us down. Most configuration software defines forward progress only. Meaning, if you want something not there, you don't remove it from your assertions, you assert that it is not there. I am worried too. But I do not entirely follow your reasoning. When I UPDATE a stack with a new template, am I supposed to write in that template not just what I want the stack to be but also how that differs from what it currently is? That is not REST. Not that I am a total REST zealot, but I am a fan of managing in terms of desired state. But I agree there is a conflict between defining a 'remove' operation and the forward progress only mindset of most config tooling. I am worried about the explosion of possibilities that comes from trying to deal with all of the diff's possible inside an instance. If there is an actual REST interface for a thing, then yes, let's use that. For instance, if we are using docker, there is in fact a very straight forward way to say remove entity X. If we are using packages we have the same thing. However, if we are just trying to write chef configurations, we have to write reverse chef configurations. What I meant to convey is let's give this piece of the interface a lot of thought. Not this is wrong to even have. Given a couple of days now, I think we do need apply and remove. We should also provide really solid example templates for this concept. You're right, I'm already starting to see issues with my current approach. This smells like a new blueprint. I'll remove it from the scope of the current software config work and raise a blueprint to track remove-config. ... A specific use-case I'm trying to address here is tripleo doing an update-replace on a nova compute node. The remove_config contains the workload to evacuate VMs and signal heat when the node is ready to be shut down. This is more involved than just uninstall the things. Could you outline in some more detail how you think this could be done? So for that we would not remove the software configuration for the nova-compute, we would assert that the machine needs vms evacuated. We want evacuation to be something we explicitly do, not a side effect of deleting things. Really? You want to force the user to explicitly say evacuate the VMs in all the various ways a host deletion can happen? E.g., when an autoscaling group of hosts shrinks? Autoscaling doesn't really fly with stateful services IMO. Also for TripleO's use case, auto-scaling is not really a high priority. Hardware isn't nearly as easily allocatable as VM's. Anyway, there is a really complicated work-flow for decomissioning any stateful service, and it differs wildly between them. I do want to have a place to define that work-flow and reliably trigger it when it needs to be triggered. I do not want it to _only_ be available in the delete this resource case, and I also do not want it to _always_ be run in that case, as I may legitimately be destroying the data too. I need a way to express that intention, and in my mind, the way to do that is to first complete an evacuation and then delete the thing. Better ideas are _most_ welcome. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Heat] HOT software configuration refined after design summit discussions
Regarding my previous email: Steve Baker sba...@redhat.com wrote on 11/19/2013 03:40:54 PM: ... How to define and deliver this agent is the challenge. Some options are: 1) install it as part of the image customization/bootstrapping (golden images or cloud-init) 2) define a (mustache?) template in the SoftwareConfig which os-collect-config transforms into the agent script, which os-collect-config then executes I do not follow what you mean here. Can you please elaborate a little? 3) a CM tool specific implementation of SoftwareApplier builds and delivers a complete agent to os-collect-config which executes it Could instead the agent be a package that the applier templates asks to be installed? We may not need to active agent to be CM tool specific. I think the remarks about os-collect-config show this. Maybe all we need is a CM tool specific package to be installed to supply the (I use the singular because of our earlier conversation that affirmed the expectation of only one CM tool per VM) hook that os-collect-config calls. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Heat] HOT software configuration refined after design summit discussions
Excerpts from Mike Spreitzer's message of 2013-11-20 15:16:45 -0800: Clint Byrum cl...@fewbar.com wrote on 11/20/2013 05:41:16 PM: Autoscaling doesn't really fly with stateful services IMO. I presume you're concerned about the auto part, not the scaling. Even a stateful group is something you may want to scale; it just takes a more complex set of operations to accomplish that. If we can make a Heat autoscaling group invoke the right set of operations, why not? It is most definitely possible and necessary. We _must_ do this. It does not fly with today's limited auto-scaling. If we can get the link between orchestration and work-flow working well, the end result should be harmonious automation and then automatic scaling will indeed be possible for stateful services. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Heat] HOT software configuration refined after design summit discussions
Excerpts from Steve Baker's message on 21.11.2013 00:00:47: From: Steve Baker sba...@redhat.com To: openstack-dev@lists.openstack.org, Date: 21.11.2013 00:04 Subject: Re: [openstack-dev] [Heat] HOT software configuration refined after design summit discussions On 11/21/2013 11:41 AM, Clint Byrum wrote: Excerpts from Mike Spreitzer's message of 2013-11-20 13:46:25 -0800: Clint Byrum cl...@fewbar.com wrote on 11/19/2013 04:28:31 PM: snip I am worried about the explosion of possibilities that comes from trying to deal with all of the diff's possible inside an instance. If there is an actual REST interface for a thing, then yes, let's use that. For instance, if we are using docker, there is in fact a very straight forward way to say remove entity X. If we are using packages we have the same thing. However, if we are just trying to write chef configurations, we have to write reverse chef configurations. What I meant to convey is let's give this piece of the interface a lot of thought. Not this is wrong to even have. Given a couple of days now, I think we do need apply and remove. We should also provide really solid example templates for this concept. You're right, I'm already starting to see issues with my current approach. This smells like a new blueprint. I'll remove it from the scope of the current software config work and raise a blueprint to track remove-config. So I read thru those recent discussions and in parallel also started to update the design wiki. BTW, nanjj renamed the wiki to [1] (but also made a redirect from the previous ...-WIP page) and linked it as spec to BP [2]. I'll leave out the remove-config thing for now. While thinking about the overall picture, I came up with some other comments: I thought about the name SoftwareApplier some more and while it is clear what it does (it applies a software config to a server), the naming is not really consistent with all the other resources in Heat. Every other resource type is called after the thing that you get when the template gets instantiated (a Server, a FloatingIP, a VolumeAttachment etc). In case of SoftwareApplier what you actually get from a user perspective is a deployed instance of the piece of software described be a SoftwareConfig. Therefore, I was calling it SoftwareDeployment orignally, because you get a software deployment (according to a config). Any comments on that name? If we think this thru with respect to remove-config (even though this needs more thought), a SoftwareApplier (that thing itself) would not really go to state DELETE_IN_PROGRESS during an update. It is always there on the VM but the software it deploys gets deleted and then reapplied or whatever ... Now thinking more about update scenarios (which we can leave for an iteration after the initial deployment is working), in my mental model it would be more consistent to have information for handle_create, handle_delete, handle_update kinds of events all defined in the SoftwareConfig resource. SoftwareConfig for represent configuration information for one specific piece of software, e.g. a web server. So it could provide all the information you need to install it, to uninstall it, or to update its config. By updating the SoftwareApplier's (or SoftwareDeployment's - my preferred name) state at runtime, the in-instance tools would grab the respective script of whatever an run it. So SoftwareConfig could look like: resources: my_webserver_config: type: OS::Heat::SoftwareConfig properties: http_port: type: number # some more config props config_create: http://www.example.com/my_scripts/webserver/install.sh config_delete: http://www.example.com/my_scripts/webserver/uninstall.sh config_update: http://www.example.com/my_scripts/webserver/applyupdate.sh At runtime, when a SoftwareApplier gets created, it looks for the 'config_create' hook and triggers that automation. When it gets deleted, it looks for the 'config_delete' hook and so on. Only config_create is mandatory. I think that would also give us nice extensibility for future use cases. For example, Heat today does not support something like stop-stack or start-stack which would be pretty useful though. If we have it one day, we would just add a 'config_start' hook to the SoftwareConfig. [1] https://wiki.openstack.org/wiki/Heat/Blueprints/hot-software-config-spec [2] https://blueprints.launchpad.net/heat/+spec/hot-software-config ... A specific use-case I'm trying to address here is tripleo doing an update-replace on a nova compute node. The remove_config contains the workload to evacuate VMs and signal heat when the node is ready to be shut down. This is more involved than just uninstall the things. Could you outline in some more detail how you think this could be done? So for that we would not remove the software configuration for the nova-compute, we would assert that the machine needs vms
Re: [openstack-dev] [Heat] HOT software configuration refined after design summit discussions
On 11/19/2013 08:37 PM, Thomas Spatzier wrote: Steve Baker sba...@redhat.com wrote on 18.11.2013 21:52:04: From: Steve Baker sba...@redhat.com To: openstack-dev@lists.openstack.org, Date: 18.11.2013 21:54 Subject: Re: [openstack-dev] [Heat] HOT software configuration refined after design summit discussions On 11/19/2013 02:22 AM, Thomas Spatzier wrote: Hi all, I have reworked the wiki page [1] I created last week to reflect discussions we had on the mail list and in IRC. From ML discussions last week it looked like we were all basically on the same page (with some details to be worked out), and I hope the new draft eliminates some confusion that the original draft had. [1] https://wiki.openstack.org/wiki/Heat/Blueprints/hot-software-config-WIP Thanks Thomas, this looks really good. I've actually started on a POC which maps to this model. Good to hear that, Steve :-) Now that we are converging, should we consolidate the various wiki pages and just have one? E.g. copy the complete contents of hot-software-config-WIP to your original hot-software-config, or deprecate all others and make hot-software-config-WIP the master? Lets just bless hot-software-config-WIP and add to it as we flesh out the implementation. I've used different semantics which you may actually prefer some of, please comment below. Resource types: SoftwareConfig - SoftwareConfig (yay!) SoftwareDeployment - SoftwareApplier - less typing, less mouth-fatigue I'm ok with SoftwareApplier. If we don't hear objections, I can change it in the wiki. SoftwareConfig properties: parameters - inputs - just because parameters is overloaded already. Makes sense. Although if the CM tool has their own semantics for inputs then that should be used in that SoftwareConfig resource implementation instead. outputs - outputs SoftwareApplier properties: software_config - apply_config - because there will sometimes be a corresponding remove_config Makes sense, and the remove_config thought is a very good point! server - server parameters - input_values - to match the 'inputs' schema property in SoftwareConfig Agree on input_values. Other comments on hot-software-config-WIP: Regarding apply_config/remove_config, if a SoftwareApplier resource is deleted it should trigger any remove_config and wait for the server to acknowledge when that is complete. This allows for any evacuation/deregistering workloads to be executed. I'm unclear yet what the SoftwareConfig 'role' is for, unless the role specifies the contract for a given inputs and outputs schema? How would this be documented or enforced? I'm inclined to leave it out for now. So about 'role', as I stated in the wiki, my thinking was that there will be different SoftwareConfig and SoftwareApplier implementations per CM tool (more on that below), since all CM tools will probably have their specific metadata and runtime implementation. So in my example I was using Chef, and 'role' is just a Chef concept, i.e. you take a cookbook and configure a specific Chef role on a server. OK, its Chef specific; I'm fine with that. It should be possible to write a SoftwareConfig type for a new CM tool as a provider template. This has some nice implications for deployers and users. I think provider templates are a good thing to have clean componentization for re-use. However, I think it still would be good to allow users to define their SoftwareConfigs inline in a template for simple use cases. I heard that requirement in several posts on the ML last week. The question is whether we can live with a single implementation of SoftwareConfig and SoftwareApplier then (see also below). Yes, a provider template would encapsulate some base SoftwareConfig resource type, but users would be free to use this type inline in their template too. My hope is that there will not need to be a different SoftwareApplier type written for each CM tool. But maybe there will be one for each delivery mechanism. The first implementation will use metadata polling and signals, another might use Marconi. Bootstrapping an image to consume a given CM tool and applied configuration data is something that we need to do, but we can make it beyond the scope of this particular proposal. I was thinking about a single implementation, too. However, I cannot really imagine how one single implementation could handle both the different metadata of different CM tools, and different runtime implementation. I think we would want to support at least a handful of the most favorite tools, but cannot see at the moment how to cover them all in one implementation. My thought was that there could be a super-class for common behavior, and then plugins with specific behavior for each tool. Anyway, all of that needs to be verified, so working on PoC patches is definitely the right thing to do. For example, if we work on implementation for two CM tools (e.g. Chef and simple scripts), we can
Re: [openstack-dev] [Heat] HOT software configuration refined after design summit discussions
On 11/20/2013 09:50 AM, Clint Byrum wrote: Excerpts from Steve Baker's message of 2013-11-18 12:52:04 -0800: Regarding apply_config/remove_config, if a SoftwareApplier resource is deleted it should trigger any remove_config and wait for the server to acknowledge when that is complete. This allows for any evacuation/deregistering workloads to be executed. I'm a little worried about the road that leads us down. Most configuration software defines forward progress only. Meaning, if you want something not there, you don't remove it from your assertions, you assert that it is not there. The reason this is different than the way we operate with resources is that resources are all under Heat's direct control via well defined APIs. In-instance things, however, will be indirectly controlled. So I feel like focusing on a diff mechanism for user-deployed tools may be unnecessary and might confuse. I'd much rather have a converge mechanism for the users to focus on. A specific use-case I'm trying to address here is tripleo doing an update-replace on a nova compute node. The remove_config contains the workload to evacuate VMs and signal heat when the node is ready to be shut down. This is more involved than just uninstall the things. Could you outline in some more detail how you think this could be done? ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Heat] HOT software configuration refined after design summit discussions
Excerpts from Steve Baker's message of 2013-11-19 13:06:21 -0800: On 11/20/2013 09:50 AM, Clint Byrum wrote: Excerpts from Steve Baker's message of 2013-11-18 12:52:04 -0800: Regarding apply_config/remove_config, if a SoftwareApplier resource is deleted it should trigger any remove_config and wait for the server to acknowledge when that is complete. This allows for any evacuation/deregistering workloads to be executed. I'm a little worried about the road that leads us down. Most configuration software defines forward progress only. Meaning, if you want something not there, you don't remove it from your assertions, you assert that it is not there. The reason this is different than the way we operate with resources is that resources are all under Heat's direct control via well defined APIs. In-instance things, however, will be indirectly controlled. So I feel like focusing on a diff mechanism for user-deployed tools may be unnecessary and might confuse. I'd much rather have a converge mechanism for the users to focus on. A specific use-case I'm trying to address here is tripleo doing an update-replace on a nova compute node. The remove_config contains the workload to evacuate VMs and signal heat when the node is ready to be shut down. This is more involved than just uninstall the things. Could you outline in some more detail how you think this could be done? So for that we would not remove the software configuration for the nova-compute, we would assert that the machine needs vms evacuated. We want evacuation to be something we explicitly do, not a side effect of deleting things. Perhaps having delete hooks for starting delete work-flows is right, but it set off a red flag for me so I want to make sure we think it through. Also IIRC, evacuation is not necessarily an in-instance thing. It looks more like the weird thing we've been talking about lately which is how do we orchestrate tenant API's: https://etherpad.openstack.org/p/orchestrate-tenant-apis ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Heat] HOT software configuration refined after design summit discussions
Hi all, I have reworked the wiki page [1] I created last week to reflect discussions we had on the mail list and in IRC. From ML discussions last week it looked like we were all basically on the same page (with some details to be worked out), and I hope the new draft eliminates some confusion that the original draft had. [1] https://wiki.openstack.org/wiki/Heat/Blueprints/hot-software-config-WIP Regards, Thomas ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Heat] HOT software configuration refined after design summit discussions
Steve Baker sba...@redhat.com wrote on 18.11.2013 21:52:04: From: Steve Baker sba...@redhat.com To: openstack-dev@lists.openstack.org, Date: 18.11.2013 21:54 Subject: Re: [openstack-dev] [Heat] HOT software configuration refined after design summit discussions On 11/19/2013 02:22 AM, Thomas Spatzier wrote: Hi all, I have reworked the wiki page [1] I created last week to reflect discussions we had on the mail list and in IRC. From ML discussions last week it looked like we were all basically on the same page (with some details to be worked out), and I hope the new draft eliminates some confusion that the original draft had. [1] https://wiki.openstack.org/wiki/Heat/Blueprints/hot-software-config-WIP Thanks Thomas, this looks really good. I've actually started on a POC which maps to this model. Good to hear that, Steve :-) Now that we are converging, should we consolidate the various wiki pages and just have one? E.g. copy the complete contents of hot-software-config-WIP to your original hot-software-config, or deprecate all others and make hot-software-config-WIP the master? I've used different semantics which you may actually prefer some of, please comment below. Resource types: SoftwareConfig - SoftwareConfig (yay!) SoftwareDeployment - SoftwareApplier - less typing, less mouth-fatigue I'm ok with SoftwareApplier. If we don't hear objections, I can change it in the wiki. SoftwareConfig properties: parameters - inputs - just because parameters is overloaded already. Makes sense. Although if the CM tool has their own semantics for inputs then that should be used in that SoftwareConfig resource implementation instead. outputs - outputs SoftwareApplier properties: software_config - apply_config - because there will sometimes be a corresponding remove_config Makes sense, and the remove_config thought is a very good point! server - server parameters - input_values - to match the 'inputs' schema property in SoftwareConfig Agree on input_values. Other comments on hot-software-config-WIP: Regarding apply_config/remove_config, if a SoftwareApplier resource is deleted it should trigger any remove_config and wait for the server to acknowledge when that is complete. This allows for any evacuation/deregistering workloads to be executed. I'm unclear yet what the SoftwareConfig 'role' is for, unless the role specifies the contract for a given inputs and outputs schema? How would this be documented or enforced? I'm inclined to leave it out for now. So about 'role', as I stated in the wiki, my thinking was that there will be different SoftwareConfig and SoftwareApplier implementations per CM tool (more on that below), since all CM tools will probably have their specific metadata and runtime implementation. So in my example I was using Chef, and 'role' is just a Chef concept, i.e. you take a cookbook and configure a specific Chef role on a server. It should be possible to write a SoftwareConfig type for a new CM tool as a provider template. This has some nice implications for deployers and users. I think provider templates are a good thing to have clean componentization for re-use. However, I think it still would be good to allow users to define their SoftwareConfigs inline in a template for simple use cases. I heard that requirement in several posts on the ML last week. The question is whether we can live with a single implementation of SoftwareConfig and SoftwareApplier then (see also below). My hope is that there will not need to be a different SoftwareApplier type written for each CM tool. But maybe there will be one for each delivery mechanism. The first implementation will use metadata polling and signals, another might use Marconi. Bootstrapping an image to consume a given CM tool and applied configuration data is something that we need to do, but we can make it beyond the scope of this particular proposal. I was thinking about a single implementation, too. However, I cannot really imagine how one single implementation could handle both the different metadata of different CM tools, and different runtime implementation. I think we would want to support at least a handful of the most favorite tools, but cannot see at the moment how to cover them all in one implementation. My thought was that there could be a super-class for common behavior, and then plugins with specific behavior for each tool. Anyway, all of that needs to be verified, so working on PoC patches is definitely the right thing to do. For example, if we work on implementation for two CM tools (e.g. Chef and simple scripts), we can probably see if one common implementation is possible or not. Someone from our team is going to write a provider for Chef to try things out. I think that can be aligned nicely with your work. The POC I'm working on is actually backed by a REST API which does dumb (but structured) storage of SoftwareConfig and SoftwareApplier entities
Re: [openstack-dev] [Heat] HOT software configuration refined after design summit discussions
It seems to me we have been discussing a proposal whose write-up intertwines two ideas: (1) making software components look like resources, and (2) using nested stacks and environments to achieve the pattern of definitions and uses. The ideas are separable, and I think the discussion has sort of revealed that. There are issues with both. Regarding the first, I think Clint has been pursuing the critical issue. Clint Byrum cl...@fewbar.com wrote on 11/12/2013 07:34:34 PM: ... I have implementation questions about both of these approaches though, as it appears they'd have to reach backward in the graph to insert their configuration, or have a generic bucket for all configuration Yeah, it does depend on the implementation. If we use Mistral the agent will need to ask Mistral for the tasks that apply to the server. $ mistral task-consume \ --tags=instance_id=$(my_instance_id);stack_id=$(stack_id) Actually that makes perfect sense. Thanks. If we have a hidden work-flow handle for any resources that have need of it passed in much the same way we pass in the cfn-metadata-server now, that would allow us to write our work-flow afterward. The image can decide when it wants to start doing work-flows and can decide to just spin until the work-flow exists and is enabled. For the tiny-work-flow case of one task it works the same as the complicated work-flow, so I think this sounds like a workable plan.. assuming a work-flow service. :) If I understand correctly, Clint is dismayed at the idea of a heat engine plugin reaching backward to find the software configs needed to apply to a server, but is OK with a task in a taskflow doing that. It seems like a double standard to me. And it makes the software config stuff depend on an essentially independent big change, which is dismaying to those of us eager to make progress on software config. How about allowing a Heat plugin to query for dependents in the graph? Regarding issue (2), look at where we are going. If I write a template that I want to share with you, and that template exercises the definition/use distinction, I have to give you: (i) the templates that comprise my definitions, (ii) my template that has my uses, and (iii) a *fragment* of environment that binds the templates in (i) to the names used for them in (ii). This is a pretty ragged assembly of stuff. Regards, Mike___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Heat] HOT software configuration refined after design summit discussions
Excerpts form Clint Byrum's message on 12.11.2013 19:32:50:
From: Clint Byrum cl...@fewbar.com
To: openstack-dev openstack-dev@lists.openstack.org,
Date: 12.11.2013 19:35
Subject: Re: [openstack-dev] [Heat] HOT software configuration
refined after design summit discussions
Excerpts from Thomas Spatzier's message of 2013-11-11 08:57:58 -0800:
Hi all,
I have just posted the following wiki page to reflect a refined
proposal
snip
Hi Thomas, thanks for spelling this out clearly.
I am still -1 on anything that specifies the place a configuration is
hosted inside the configuration definition itself. Because configurations
are encapsulated by servers, it makes more sense to me that the servers
(or server groups) would specify their configurations. If changing to a
IMO the current proposal does _not_ the concrete hosting inside component
definition. The component definition is in this external template file and
all we do is give it a pointer to the server at deploy time so that the
implementation can perform whatever is needed to perform at that time.
The resource in the actual template file is like the intermediate
association resource you are suggesting below (similar to what
VolumeAttachment does), so this is the place where you say which component
gets deployed where. This represents a concrete use of a software
component. Again, all we do is pass in a pointer to the server where _this
use_ of the software component shall be installed.
more logical model is just too hard for TOSCA to adapt to, then I suggest
this be an area that TOSCA differs from Heat. We don't need two models
The current proposal was done completely unrelated to TOSCA, but really
just a try to have a pragmatic approach for solving the use cases we talked
about. I don't really care in which directions the relations point. Both
ways can be easily mapped to TOSCA. I just think the current proposal is
intuitive, at least to me. And you could see it as kind of a short notation
that avoids another association class.
for communicating configurations to servers, and I'd prefer Heat stay
focused on making HOT template authors' and users' lives better.
I have seen an alternative approach which separates a configuration
definition from a configuration deployer. This at least makes it clear
that the configuration is a part of a server. In pseudo-HOT:
resources:
WebConfig:
type: OS::Heat::ChefCookbook
properties:
cookbook_url: https://some.test/foo
parameters:
endpoint_host:
type: string
WebServer:
type: OS::Nova::Server
properties:
image: webserver
flavor: 100
DeployWebConfig:
type: OS::Heat::ConfigDeployer
properties:
configuration: {get_resource: WebConfig}
on_server: {get_resource: WebServer}
parameters:
endpoint_host: {get_attribute: [ WebServer, first_ip]}
The DeployWebConfig association class actually is the 'mysql' resource in
the template on the wiki page. See the Design alternative section I put it.
That would be fine with me as well.
snip
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Heat] HOT software configuration refined after design summit discussions
Georgy Okrokvertskhov gokrokvertsk...@mirantis.com wrote on 12.11.2013
21:27:13:
From: Georgy Okrokvertskhov gokrokvertsk...@mirantis.com
To: OpenStack Development Mailing List (not for usage questions)
openstack-dev@lists.openstack.org,
Date: 12.11.2013 21:29
Subject: Re: [openstack-dev] [Heat] HOT software configuration
refined after design summit discussions
Hi,
I agree with Clint that component placement specified inside
component configuration is not a right thing. I remember that mostly
everyone agreed that hosted_on should not be in HOT templates.
When one specify placement explicitly inside  a component definition
it prevents the following:
1. Reusability - you can't reuse component without creating its
definition copy with another placement parameter.
See my reply to Clint's mail. The deployment location in form of the
server reference is _not_ hardcoded in the component definition. All we
do is to provide a pointer to the server where a software shall be deployed
at deploy time. You can use a component definition in many place, and in
each place where you use it you provide it a pointer to the target server.
2. Composability - it will be no clear way to express composable
configurations. There was a clear way in a template showed during
design session where server had a list of components to be placed.
I think we have full composability with the deployment resources that
mark uses of software component definitions.
3. Deployment order - some components should be placed in strict
order and it will be much easier just make an ordered list of
components then express artificial dependencies between them just
for ordering.
With the deployment resources and Heat's normal way of handling
dependencies between any resources, we should be able have proper ordering.
I agree that strict ordering is probably the most easy way of doing it, but
we have implementations that do deployment in a more flexible manner
without any problems.
Thanks
Georgy
On Tue, Nov 12, 2013 at 10:32 AM, Clint Byrum cl...@fewbar.com wrote:
Excerpts from Thomas Spatzier's message of 2013-11-11 08:57:58 -0800:
Hi all,
I have just posted the following wiki page to reflect a refined
proposal
for HOT software configuration based on discussions at the design
summit
last week. Angus also put a sample up in an etherpad last week, but we
did
not have enough time to go thru it in the design session. My write-up
is
based on Angus' sample, actually a refinement, and on discussions we
had in
breaks, plus it is trying to reflect all the good input from ML
discussions
and Steve Baker's initial proposal.
https://wiki.openstack.org/wiki/Heat/Blueprints/hot-software-config-WIP
Please review and provide feedback.
Hi Thomas, thanks for spelling this out clearly.
I am still -1 on anything that specifies the place a configuration is
hosted inside the configuration definition itself. Because configurations
are encapsulated by servers, it makes more sense to me that the servers
(or server groups) would specify their configurations. If changing to a
more logical model is just too hard for TOSCA to adapt to, then I suggest
this be an area that TOSCA differs from Heat. We don't need two models
for communicating configurations to servers, and I'd prefer Heat stay
focused on making HOT template authors' and users' lives better.
I have seen an alternative approach which separates a configuration
definition from a configuration deployer. This at least makes it clear
that the configuration is a part of a server. In pseudo-HOT:
resources:
 WebConfig:
  type: OS::Heat::ChefCookbook
  properties:
   cookbook_url: https://some.test/foo
   parameters:
    endpoint_host:
     type: string
 WebServer:
  type: OS::Nova::Server
  properties:
   image: webserver
   flavor: 100
 DeployWebConfig:
  type: OS::Heat::ConfigDeployer
  properties:
   configuration: {get_resource: WebConfig}
   on_server: {get_resource: WebServer}
   parameters:
    endpoint_host: {get_attribute: [ WebServer, first_ip]}
I have implementation questions about both of these approaches though,
as it appears they'd have to reach backward in the graph to insert
their configuration, or have a generic bucket for all configuration
to be inserted. IMO that would look a lot like the method I proposed,
which was to just have a list of components attached directly to the
server like this:
components:
 WebConfig:
  type: Chef::Cookbook
  properties:
   cookbook_url: https://some.test/foo
   parameters:
    endpoing_host:
     type: string
resources:
 WebServer:
  type: OS::Nova::Server
  properties:
   image: webserver
   flavor: 100
  components:
   - webconfig:
    component: {get_component: WebConfig}
    parameters:
     endpoint_host: {get_attribute: [ WebServer, first_ip ]}
Of course
Re: [openstack-dev] [Heat] HOT software configuration refined after design summit discussions
Excerpts from Thomas Spatzier's message of 2013-11-13 00:28:59 -0800:
Angus Salkeld asalk...@redhat.com wrote on 13.11.2013 00:22:44:
From: Angus Salkeld asalk...@redhat.com
To: openstack-dev@lists.openstack.org,
Date: 13.11.2013 00:25
Subject: Re: [openstack-dev] [Heat] HOT software configuration
refined after design summit discussions
On 12/11/13 10:32 -0800, Clint Byrum wrote:
Excerpts from Thomas Spatzier's message of 2013-11-11 08:57:58 -0800:
Hi all,
I have just posted the following wiki page to reflect a refined
proposal
for HOT software configuration based on discussions at the design
summit
last week. Angus also put a sample up in an etherpad last week, but we
did
not have enough time to go thru it in the design session. My write-up
is
based on Angus' sample, actually a refinement, and on discussionswe
had in
breaks, plus it is trying to reflect all the good input from ML
discussions
and Steve Baker's initial proposal.
https://wiki.openstack.org/wiki/Heat/Blueprints/hot-software-config-WIP
Please review and provide feedback.
Hi Thomas, thanks for spelling this out clearly.
I am still -1 on anything that specifies the place a configuration is
hosted inside the configuration definition itself. Because
configurations
are encapsulated by servers, it makes more sense to me that the servers
(or server groups) would specify their configurations. If changing to a
more logical model is just too hard for TOSCA to adapt to, then I
suggest
this be an area that TOSCA differs from Heat. We don't need two models
for communicating configurations to servers, and I'd prefer Heat stay
focused on making HOT template authors' and users' lives better.
I have seen an alternative approach which separates a configuration
definition from a configuration deployer. This at least makes it clear
that the configuration is a part of a server. In pseudo-HOT:
resources:
WebConfig:
type: OS::Heat::ChefCookbook
properties:
cookbook_url: https://some.test/foo
parameters:
endpoint_host:
type: string
WebServer:
type: OS::Nova::Server
properties:
image: webserver
flavor: 100
DeployWebConfig:
type: OS::Heat::ConfigDeployer
properties:
configuration: {get_resource: WebConfig}
on_server: {get_resource: WebServer}
parameters:
endpoint_host: {get_attribute: [ WebServer, first_ip]}
This is what Thomas defined, with one optimisation.
- The webconfig is a yaml template.
As you say the component is static - if so why even put it inline in
the template (well that was my thinking, it seems like a template not
really a resource).
Yes, exactly. Our idea was to put it in its own file since it is really
static and having it in its own file makes it much more reusable.
With 'WebConfig' defined inline in the template as in the snippet above,
you will have to update many template files where you use the component,
whereas you will only have to touch one place when it is in its own file.
Ok, the example above looks simple, but in reality we will see more complex
sets of parameters etc.
Maybe for very simple use cases, we can allow a shortcut of inlining it in
the template (I mentioned this in the wiki) and avoid the need for a
separate file.
I think I understand now, and we're all basically on the same page. As
usual, I was confused by the subtleties.
I think the in-line capability is critical to have in the near-term
plan, but would +2 an implementation that left it out at the beginning.
Before we ratify this and people run off and write code, I'd like to
present my problems in TripleO and try to see if I can express them
using the spec you've laid out. Will try and do that in the next couple
of days.
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Heat] HOT software configuration refined after design summit discussions
Zane Bitter zbit...@redhat.com wrote on 13.11.2013 18:11:18:
From: Zane Bitter zbit...@redhat.com
To: openstack-dev@lists.openstack.org,
Date: 13.11.2013 18:14
Subject: Re: [openstack-dev] [Heat] HOT software configuration
refined after design summit discussions
On 11/11/13 17:57, Thomas Spatzier wrote:
snip
https://wiki.openstack.org/wiki/Heat/Blueprints/hot-software-config-WIP
Please review and provide feedback.
I believe there's an error in the Explicit dependency section, where it
says that depends_on is a property. In cfn DependsOn actually exists at
the same level as Type, Properties, c.
resources:
client:
type: My::Software::SomeClient
properties:
server: { get_resource: my_server }
params:
# params ...
depends_on:
- get_resource: server_process1
- get_resource: server_process2
Good point. I think the reason was tied too much to the provider template
concept where all properties get passed automatically to the provider
template and in there you can basically do anything that is necessary,
including hanlding dependencies. But I was missing the fact that this is a
generic concept for all resources.
I'll fix it in the wiki.
And conceptually this seems correct, because it applies to any kind of
resource, whereas properties are defined per-resource-type.
Don't be fooled by our implementation:
https://review.openstack.org/#/c/44733/
It also doesn't support a list, but I think we can and should fix that
in HOT.
Doesn't DependsOn already support lists? I quickly checked the code and it
seems it does:
https://github.com/openstack/heat/blob/master/heat/engine/resource.py#L288
cheers,
Zane.
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Heat] HOT software configuration refined after design summit discussions
On 13/11/13 18:29, Thomas Spatzier wrote: It also doesn't support a list, but I think we can and should fix that in HOT. Doesn't DependsOn already support lists? I quickly checked the code and it seems it does: https://github.com/openstack/heat/blob/master/heat/engine/resource.py#L288 Oh, cool. Looks like Angus added that last month. Thanks, I missed that one :) - ZB ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Heat] HOT software configuration refined after design summit discussions
On 11/14/2013 06:11 AM, Zane Bitter wrote:
On 11/11/13 17:57, Thomas Spatzier wrote:
Hi all,
I have just posted the following wiki page to reflect a refined proposal
for HOT software configuration based on discussions at the design summit
last week. Angus also put a sample up in an etherpad last week, but
we did
not have enough time to go thru it in the design session. My write-up is
based on Angus' sample, actually a refinement, and on discussions we
had in
breaks, plus it is trying to reflect all the good input from ML
discussions
and Steve Baker's initial proposal.
https://wiki.openstack.org/wiki/Heat/Blueprints/hot-software-config-WIP
Please review and provide feedback.
I believe there's an error in the Explicit dependency section, where
it says that depends_on is a property. In cfn DependsOn actually
exists at the same level as Type, Properties, c.
resources:
client:
type: My::Software::SomeClient
properties:
server: { get_resource: my_server }
params:
# params ...
depends_on:
- get_resource: server_process1
- get_resource: server_process2
And conceptually this seems correct, because it applies to any kind of
resource, whereas properties are defined per-resource-type.
Don't be fooled by our implementation:
https://review.openstack.org/#/c/44733/
It also doesn't support a list, but I think we can and should fix that
in HOT.
This has already been fixed
https://review.openstack.org/#/c/51507/1
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Heat] HOT software configuration refined after design summit discussions
On 11/14/2013 06:02 AM, Zane Bitter wrote: On 13/11/13 01:34, Clint Byrum wrote: Excerpts from Angus Salkeld's message of 2013-11-12 15:22:44 -0800: IMO is should just be a template/formatted file. I'd prefer that we have the ability to pull in a chunk of in-line template as well. Perhaps that is the template resource, I have not thought that through. It is not o-k, IMO, to push things off entirely to external files/urls/providers, etc. That is just cumbersome and unnecessary for a common case which is to deploy two things using the same base config with parameters having different values. Of course, for my use case of having different topologies reusing bits of config, it is perfect to have the reusable bits split into different files. So, if I understand Angus's get_file suggestion correctly, it parses out to the equivalent of inlining the file's contents. So if you implement the resource as accepting inline data and add in get_file, then you get: a) Composability, OR b) Everything in one file but not both. I think that is probably sufficient, but I would be interested in your opinion: is it essential that you be able to compose software components defined in the same file? Note that the implementation of get_file would also involve python-heatclient automagically detecting it and making sure the relevant file is uploaded in the files section. So this shouldn't create a lot of mental overhead for the user. (BTW I think I like this plan.) Yes, and get_file is a HOT function which is only evaluated where other functions are evaluated, which is probably a good thing. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Heat] HOT software configuration refined after design summit discussions
On 12/11/13 14:59, Alex Heneveld wrote:
One minor suggestion is to consider using a special character (eg $)
rather than reserved keywords. As I understand it the keywords are only
interpreted when they exactly match the value of a key in a map, so it
is already unlikely to be problematic. However I think it would be more
familiar and clear if we instead used the rule that any item (key or
value) which _starts_ with a $ is interpreted specially. What those
rules are is TBD but you could for instance write functions -- as either
`$get_param('xxx')` or `$get_param: xxx` -- as well as allow accessing a
parameter directly `$xxx `.
This sounds like a nice idea on the surface. AWS accomplished the same
thing by namespacing functions with the Fn:: prefix (except for 'Ref',
bizarrely), and it works fine because the chances are if you randomly
(maybe in a Metadata section) have a dict key that happens to start with
Fn:: then you can probably just choose a different name. However, if
for any reason you have a dict key starting with $ and we interpret
that specially, then you are basically hosed since you almost certainly
_needed_ it to actually start with $ for a reason. So -1.
cheers,
Zane.
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Heat] HOT software configuration refined after design summit discussions
Excerpts from Thomas Spatzier's message of 2013-11-11 08:57:58 -0800:
Hi all,
I have just posted the following wiki page to reflect a refined proposal
for HOT software configuration based on discussions at the design summit
last week. Angus also put a sample up in an etherpad last week, but we did
not have enough time to go thru it in the design session. My write-up is
based on Angus' sample, actually a refinement, and on discussions we had in
breaks, plus it is trying to reflect all the good input from ML discussions
and Steve Baker's initial proposal.
https://wiki.openstack.org/wiki/Heat/Blueprints/hot-software-config-WIP
Please review and provide feedback.
Hi Thomas, thanks for spelling this out clearly.
I am still -1 on anything that specifies the place a configuration is
hosted inside the configuration definition itself. Because configurations
are encapsulated by servers, it makes more sense to me that the servers
(or server groups) would specify their configurations. If changing to a
more logical model is just too hard for TOSCA to adapt to, then I suggest
this be an area that TOSCA differs from Heat. We don't need two models
for communicating configurations to servers, and I'd prefer Heat stay
focused on making HOT template authors' and users' lives better.
I have seen an alternative approach which separates a configuration
definition from a configuration deployer. This at least makes it clear
that the configuration is a part of a server. In pseudo-HOT:
resources:
WebConfig:
type: OS::Heat::ChefCookbook
properties:
cookbook_url: https://some.test/foo
parameters:
endpoint_host:
type: string
WebServer:
type: OS::Nova::Server
properties:
image: webserver
flavor: 100
DeployWebConfig:
type: OS::Heat::ConfigDeployer
properties:
configuration: {get_resource: WebConfig}
on_server: {get_resource: WebServer}
parameters:
endpoint_host: {get_attribute: [ WebServer, first_ip]}
I have implementation questions about both of these approaches though,
as it appears they'd have to reach backward in the graph to insert
their configuration, or have a generic bucket for all configuration
to be inserted. IMO that would look a lot like the method I proposed,
which was to just have a list of components attached directly to the
server like this:
components:
WebConfig:
type: Chef::Cookbook
properties:
cookbook_url: https://some.test/foo
parameters:
endpoing_host:
type: string
resources:
WebServer:
type: OS::Nova::Server
properties:
image: webserver
flavor: 100
components:
- webconfig:
component: {get_component: WebConfig}
parameters:
endpoint_host: {get_attribute: [ WebServer, first_ip ]}
Of course, the keen eye will see the circular dependency there with the
WebServer trying to know its own IP. We've identified quite a few use
cases for self-referencing attributes, so that is a separate problem we
should solve independent of the template composition problem.
Anyway, I prefer the idea that parse-time things are called components
and run-time things are resources. I don't need a database entry for
WebConfig above. It is in the template and entirely static, just
sitting there as a reusable chunk for servers to pull in as-needed.
Anyway, I don't feel that we resolved any of these issues in the session
about configuration at the summit. If we did, we did not record them
in the etherpad or the blueprint. We barely got through the prepared
list of requirements and only were able to spell out problems, not
any solutions. So forgive me if I missed something and want to keep on
discussing this.
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Heat] HOT software configuration refined after design summit discussions
On 12/11/13 10:32 -0800, Clint Byrum wrote:
Excerpts from Thomas Spatzier's message of 2013-11-11 08:57:58 -0800:
Hi all,
I have just posted the following wiki page to reflect a refined proposal
for HOT software configuration based on discussions at the design summit
last week. Angus also put a sample up in an etherpad last week, but we did
not have enough time to go thru it in the design session. My write-up is
based on Angus' sample, actually a refinement, and on discussions we had in
breaks, plus it is trying to reflect all the good input from ML discussions
and Steve Baker's initial proposal.
https://wiki.openstack.org/wiki/Heat/Blueprints/hot-software-config-WIP
Please review and provide feedback.
Hi Thomas, thanks for spelling this out clearly.
I am still -1 on anything that specifies the place a configuration is
hosted inside the configuration definition itself. Because configurations
are encapsulated by servers, it makes more sense to me that the servers
(or server groups) would specify their configurations. If changing to a
more logical model is just too hard for TOSCA to adapt to, then I suggest
this be an area that TOSCA differs from Heat. We don't need two models
for communicating configurations to servers, and I'd prefer Heat stay
focused on making HOT template authors' and users' lives better.
I have seen an alternative approach which separates a configuration
definition from a configuration deployer. This at least makes it clear
that the configuration is a part of a server. In pseudo-HOT:
resources:
WebConfig:
type: OS::Heat::ChefCookbook
properties:
cookbook_url: https://some.test/foo
parameters:
endpoint_host:
type: string
WebServer:
type: OS::Nova::Server
properties:
image: webserver
flavor: 100
DeployWebConfig:
type: OS::Heat::ConfigDeployer
properties:
configuration: {get_resource: WebConfig}
on_server: {get_resource: WebServer}
parameters:
endpoint_host: {get_attribute: [ WebServer, first_ip]}
This is what Thomas defined, with one optimisation.
- The webconfig is a yaml template.
As you say the component is static - if so why even put it inline in
the template (well that was my thinking, it seems like a template not
really a resource).
I have implementation questions about both of these approaches though,
as it appears they'd have to reach backward in the graph to insert
their configuration, or have a generic bucket for all configuration
Yeah, it does depend on the implementation. If we use Mistral the
agent will need to ask Mistral for the tasks that apply to the server.
$ mistral task-consume \
--tags=instance_id=$(my_instance_id);stack_id=$(stack_id)
to be inserted. IMO that would look a lot like the method I proposed,
which was to just have a list of components attached directly to the
server like this:
components:
WebConfig:
type: Chef::Cookbook
properties:
cookbook_url: https://some.test/foo
parameters:
endpoing_host:
type: string
resources:
WebServer:
type: OS::Nova::Server
properties:
image: webserver
flavor: 100
components:
- webconfig:
component: {get_component: WebConfig}
parameters:
endpoint_host: {get_attribute: [ WebServer, first_ip ]}
I'd change this to:
components:
- webconfig:
component: {get_file: ./my_configs/webconfig.yaml}
parameters:
endpoint_host: {get_attribute: [ WebServer, first_ip ]}
This *could* be a short hand notation like the volumes property on
aws instances.
Of course, the keen eye will see the circular dependency there with the
WebServer trying to know its own IP. We've identified quite a few use
cases for self-referencing attributes, so that is a separate problem we
should solve independent of the template composition problem.
(aside) I don't like the idea of self ref as it breaks the idea that
references are resolved top down. Basically we have to put in
a nasty hack to produce broken behaviour (first resolution is
bogus and only following resoultions are possibly correct).
In this case just use the deployer to break your circular dep?
Anyway, I prefer the idea that parse-time things are called components
and run-time things are resources. I don't need a database entry for
WebConfig above. It is in the template and entirely static, just
sitting there as a reusable chunk for servers to pull in as-needed.
IMO is should just be a template/formatted file.
Anyway, I don't feel that we resolved any of these issues in the session
about configuration at the summit. If we did, we did not record them
in the etherpad or the blueprint. We barely got through the prepared
list of requirements and only were able to spell out problems, not
any solutions. So forgive me if I missed something and want to keep on
discussing this.
___
OpenStack-dev mailing list
[openstack-dev] [Heat] HOT software configuration refined after design summit discussions
Hi all, I have just posted the following wiki page to reflect a refined proposal for HOT software configuration based on discussions at the design summit last week. Angus also put a sample up in an etherpad last week, but we did not have enough time to go thru it in the design session. My write-up is based on Angus' sample, actually a refinement, and on discussions we had in breaks, plus it is trying to reflect all the good input from ML discussions and Steve Baker's initial proposal. https://wiki.openstack.org/wiki/Heat/Blueprints/hot-software-config-WIP Please review and provide feedback. Regards, Thomas ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Heat] HOT Software configuration proposal
On 10/26/2013 05:25 AM, Clint Byrum wrote:
Excerpts from Angus Salkeld's message of 2013-10-24 18:48:16 -0700:
On 24/10/13 11:54 +0200, Patrick Petit wrote:
Hi Clint,
Thank you! I have few replies/questions in-line.
Cheers,
Patrick
On 10/23/13 8:36 PM, Clint Byrum wrote:
I think this fits into something that I want for optimizing
os-collect-config as well (our in-instance Heat-aware agent). That is
a way for us to wait for notification of changes to Metadata without
polling.
Interesting... If I understand correctly that's kinda replacement of
cfn-hup... Do you have a blueprint pointer or something more
specific? While I see the benefits of it, in-instance notifications
is not really what we are looking for. We are looking for a
notification service that exposes an API whereby listeners can
register for Heat notifications. AWS Alarming / CloudFormation has
that. Why not Ceilometer / Heat? That would be extremely valuable for
those who build PaaS-like solutions above Heat. To say it bluntly,
I'd like to suggest we explore ways to integrate Heat with Marconi.
Yeah, I am trying to do a PoC of this now. I'll let you know how
it goes.
I am trying to implement the following:
heat_template_version: 2013-05-23
parameters:
key_name:
type: String
flavor:
type: String
default: m1.small
image:
type: String
default: fedora-19-i386-heat-cfntools
resources:
config_server:
type: OS::Marconi::QueueServer
properties:
image: {get_param: image}
flavor: {get_param: flavor}
key_name: {get_param: key_name}
configA:
type: OS::Heat::OrderedConfig
properties:
marconi_server: {get_attr: [config_server, url]}
hosted_on: {get_resource: serv1}
script: |
#!/bin/bash
logger 1. hello from marconi
configB:
type: OS::Heat::OrderedConfig
properties:
marconi_server: {get_attr: [config_server, url]}
hosted_on: {get_resource: serv1}
depends_on: {get_resource: configA}
script: |
#!/bin/bash
logger 2. hello from marconi
serv1:
type: OS::Nova::Server
properties:
image: {get_param: image}
flavor: {get_param: flavor}
key_name: {get_param: key_name}
user_data: |
#!/bin/sh
# poll marconi url/v1/queues/{hostname}/messages
# apply config
# post a response message with any outputs
# delete request message
If I may diverge this a bit, I'd like to consider the impact of
hosted_on on reusability in templates. hosted_on feels like an
anti-pattern, and I've never seen anything quite like it. It feels wrong
for a well contained component to then reach out and push itself onto
something else which has no mention of it.
I'll rewrite your template as I envision it working:
resources:
config_server:
type: OS::Marconi::QueueServer
properties:
image: {get_param: image}
flavor: {get_param: flavor}
key_name: {get_param: key_name}
configA:
type: OS::Heat::OrderedConfig
properties:
marconi_server: {get_attr: [config_server, url]}
script: |
#!/bin/bash
logger 1. hello from marconi
configB:
type: OS::Heat::OrderedConfig
properties:
marconi_server: {get_attr: [config_server, url]}
depends_on: {get_resource: configA}
script: |
#!/bin/bash
logger 2. hello from marconi
serv1:
type: OS::Nova::Server
properties:
image: {get_param: image}
flavor: {get_param: flavor}
key_name: {get_param: key_name}
components:
- configA
- configB
user_data: |
#!/bin/sh
# poll marconi url/v1/queues/{hostname}/messages
# apply config
# post a response message with any outputs
# delete request message
This only becomes obvious why it is important when you want to do this:
configC:
type: OS::Heat::OrderedConfig
properties:
script: |
#!/bin/bash
logger ?. I can race with A, no dependency needed
serv2:
type: OS::Nova::Server
properties:
...
components:
- configA
- configC
This is proper composition, where the caller defines the components, not
the callee. Now you can re-use configA with a different component in the
same template. As we get smarter we can have these configs separate from
the template and reusable across templates.
Anyway, I'd like to see us stop talking about hosted_on, and if it has
been implemented, that it be deprecated and eventually removed, as it is
just plain confusing.
My components proposals had no hosted_on, but I've been thinking about
the implications of implementing software configuration as resources,
and one of the natural
Re: [openstack-dev] [Heat] HOT Software configuration proposal
In-line comments.
On 10/28/13 5:43 PM, Steve Baker sba...@redhat.com wrote:
On 10/26/2013 05:25 AM, Clint Byrum wrote:
Excerpts from Angus Salkeld's message of 2013-10-24 18:48:16 -0700:
On 24/10/13 11:54 +0200, Patrick Petit wrote:
Hi Clint,
Thank you! I have few replies/questions in-line.
Cheers,
Patrick
On 10/23/13 8:36 PM, Clint Byrum wrote:
I think this fits into something that I want for optimizing
os-collect-config as well (our in-instance Heat-aware agent). That is
a way for us to wait for notification of changes to Metadata without
polling.
Interesting... If I understand correctly that's kinda replacement of
cfn-hup... Do you have a blueprint pointer or something more
specific? While I see the benefits of it, in-instance notifications
is not really what we are looking for. We are looking for a
notification service that exposes an API whereby listeners can
register for Heat notifications. AWS Alarming / CloudFormation has
that. Why not Ceilometer / Heat? That would be extremely valuable for
those who build PaaS-like solutions above Heat. To say it bluntly,
I'd like to suggest we explore ways to integrate Heat with Marconi.
Yeah, I am trying to do a PoC of this now. I'll let you know how
it goes.
I am trying to implement the following:
heat_template_version: 2013-05-23
parameters:
key_name:
type: String
flavor:
type: String
default: m1.small
image:
type: String
default: fedora-19-i386-heat-cfntools
resources:
config_server:
type: OS::Marconi::QueueServer
properties:
image: {get_param: image}
flavor: {get_param: flavor}
key_name: {get_param: key_name}
configA:
type: OS::Heat::OrderedConfig
properties:
marconi_server: {get_attr: [config_server, url]}
hosted_on: {get_resource: serv1}
script: |
#!/bin/bash
logger 1. hello from marconi
configB:
type: OS::Heat::OrderedConfig
properties:
marconi_server: {get_attr: [config_server, url]}
hosted_on: {get_resource: serv1}
depends_on: {get_resource: configA}
script: |
#!/bin/bash
logger 2. hello from marconi
serv1:
type: OS::Nova::Server
properties:
image: {get_param: image}
flavor: {get_param: flavor}
key_name: {get_param: key_name}
user_data: |
#!/bin/sh
# poll marconi url/v1/queues/{hostname}/messages
# apply config
# post a response message with any outputs
# delete request message
If I may diverge this a bit, I'd like to consider the impact of
hosted_on on reusability in templates. hosted_on feels like an
anti-pattern, and I've never seen anything quite like it. It feels wrong
for a well contained component to then reach out and push itself onto
something else which has no mention of it.
I'll rewrite your template as I envision it working:
resources:
config_server:
type: OS::Marconi::QueueServer
properties:
image: {get_param: image}
flavor: {get_param: flavor}
key_name: {get_param: key_name}
configA:
type: OS::Heat::OrderedConfig
properties:
marconi_server: {get_attr: [config_server, url]}
script: |
#!/bin/bash
logger 1. hello from marconi
configB:
type: OS::Heat::OrderedConfig
properties:
marconi_server: {get_attr: [config_server, url]}
depends_on: {get_resource: configA}
script: |
#!/bin/bash
logger 2. hello from marconi
serv1:
type: OS::Nova::Server
properties:
image: {get_param: image}
flavor: {get_param: flavor}
key_name: {get_param: key_name}
components:
- configA
- configB
user_data: |
#!/bin/sh
# poll marconi url/v1/queues/{hostname}/messages
# apply config
# post a response message with any outputs
# delete request message
This only becomes obvious why it is important when you want to do this:
configC:
type: OS::Heat::OrderedConfig
properties:
script: |
#!/bin/bash
logger ?. I can race with A, no dependency needed
serv2:
type: OS::Nova::Server
properties:
...
components:
- configA
- configC
This is proper composition, where the caller defines the components, not
the callee. Now you can re-use configA with a different component in the
same template. As we get smarter we can have these configs separate from
the template and reusable across templates.
Anyway, I'd like to see us stop talking about hosted_on, and if it has
been implemented, that it be deprecated and eventually removed, as it is
just plain confusing.
My components proposals had no hosted_on, but I've been thinking about
the implications of implementing
Re: [openstack-dev] [Heat] HOT Software configuration proposal
Excerpts from Steve Baker's message of 2013-10-28 15:43:02 -0700:
On 10/26/2013 05:25 AM, Clint Byrum wrote:
Excerpts from Angus Salkeld's message of 2013-10-24 18:48:16 -0700:
On 24/10/13 11:54 +0200, Patrick Petit wrote:
Hi Clint,
Thank you! I have few replies/questions in-line.
Cheers,
Patrick
On 10/23/13 8:36 PM, Clint Byrum wrote:
I think this fits into something that I want for optimizing
os-collect-config as well (our in-instance Heat-aware agent). That is
a way for us to wait for notification of changes to Metadata without
polling.
Interesting... If I understand correctly that's kinda replacement of
cfn-hup... Do you have a blueprint pointer or something more
specific? While I see the benefits of it, in-instance notifications
is not really what we are looking for. We are looking for a
notification service that exposes an API whereby listeners can
register for Heat notifications. AWS Alarming / CloudFormation has
that. Why not Ceilometer / Heat? That would be extremely valuable for
those who build PaaS-like solutions above Heat. To say it bluntly,
I'd like to suggest we explore ways to integrate Heat with Marconi.
Yeah, I am trying to do a PoC of this now. I'll let you know how
it goes.
I am trying to implement the following:
heat_template_version: 2013-05-23
parameters:
key_name:
type: String
flavor:
type: String
default: m1.small
image:
type: String
default: fedora-19-i386-heat-cfntools
resources:
config_server:
type: OS::Marconi::QueueServer
properties:
image: {get_param: image}
flavor: {get_param: flavor}
key_name: {get_param: key_name}
configA:
type: OS::Heat::OrderedConfig
properties:
marconi_server: {get_attr: [config_server, url]}
hosted_on: {get_resource: serv1}
script: |
#!/bin/bash
logger 1. hello from marconi
configB:
type: OS::Heat::OrderedConfig
properties:
marconi_server: {get_attr: [config_server, url]}
hosted_on: {get_resource: serv1}
depends_on: {get_resource: configA}
script: |
#!/bin/bash
logger 2. hello from marconi
serv1:
type: OS::Nova::Server
properties:
image: {get_param: image}
flavor: {get_param: flavor}
key_name: {get_param: key_name}
user_data: |
#!/bin/sh
# poll marconi url/v1/queues/{hostname}/messages
# apply config
# post a response message with any outputs
# delete request message
If I may diverge this a bit, I'd like to consider the impact of
hosted_on on reusability in templates. hosted_on feels like an
anti-pattern, and I've never seen anything quite like it. It feels wrong
for a well contained component to then reach out and push itself onto
something else which has no mention of it.
I'll rewrite your template as I envision it working:
resources:
config_server:
type: OS::Marconi::QueueServer
properties:
image: {get_param: image}
flavor: {get_param: flavor}
key_name: {get_param: key_name}
configA:
type: OS::Heat::OrderedConfig
properties:
marconi_server: {get_attr: [config_server, url]}
script: |
#!/bin/bash
logger 1. hello from marconi
configB:
type: OS::Heat::OrderedConfig
properties:
marconi_server: {get_attr: [config_server, url]}
depends_on: {get_resource: configA}
script: |
#!/bin/bash
logger 2. hello from marconi
serv1:
type: OS::Nova::Server
properties:
image: {get_param: image}
flavor: {get_param: flavor}
key_name: {get_param: key_name}
components:
- configA
- configB
user_data: |
#!/bin/sh
# poll marconi url/v1/queues/{hostname}/messages
# apply config
# post a response message with any outputs
# delete request message
This only becomes obvious why it is important when you want to do this:
configC:
type: OS::Heat::OrderedConfig
properties:
script: |
#!/bin/bash
logger ?. I can race with A, no dependency needed
serv2:
type: OS::Nova::Server
properties:
...
components:
- configA
- configC
This is proper composition, where the caller defines the components, not
the callee. Now you can re-use configA with a different component in the
same template. As we get smarter we can have these configs separate from
the template and reusable across templates.
Anyway, I'd like to see us stop talking about hosted_on, and if it has
been implemented, that it be deprecated and eventually
Re: [openstack-dev] [Heat] HOT Software configuration proposal
On 28/10/13 16:26 -0700, Clint Byrum wrote:
[snip]
My components proposals had no hosted_on, but I've been thinking about
the implications of implementing software configuration as resources,
and one of the natural consequences might be that hosted_on is the best
way of establishing the relationship with compute and its
configurations. Let me elaborate.
Lets say that Heat has resource types for software configuration, with
the following behaviours:
* like other resources, a config resource goes into CREATE IN_PROGRESS
as soon as its dependencies are satisfied (these dependencies may be
values signalled from other config resources)
* a config resource goes to state CREATE COMPLETE when it receives a
signal that configuration on the compute resource is complete (by some
mechanism; wait condition, marconi message, whatevs)
* the relationship between config resources and compute resources are
achieved with existing intrinsic functions (get_resource, get_attr)
This lifecycle behaviour means that a configuration resource can only
run on a single compute resource, and that relationship needs to be
established somehow. Config resources will have a quirk in that they
need to provide 2 sources of configuration data at different times:
1) cloud-init config-data (or other boot-only mechanism), which must be
available when the compute resource goes into CREATE IN_PROGRESS
2) the actual configuration data (oac metadata, puppet manifest, chef
recipe) which the compute resource needs to be able to fetch and execute
whenever it becomes available.
The data in 1) implies that the compute needs to depend on the config,
but then all concurrency is lost (this won't matter for a
cloud-init-only config resource). Either way, the data for 1) will need
to be available when the config resource is still in state INIT
COMPLETE, which may impose limitations on how that is defined (ie
get_resource, get_attr not allowed).
So, 2 concrete examples for handling config/compute dependencies:
hosted_on
-
resources:
configA
type: Heat::ScriptConfig
properties:
hosted_on:
get_resource: the_server
script: |
#!/bin/bash
logger 1. hello config A
configB
type: Heat::ScriptConfig
properties:
hosted_on:
get_resource: the_server
script: |
#!/bin/bash
logger 1. hello config B
the_server:
type: OS::Nova::Server
Here, configA and configB go into CREATE IN_PROGRESS as soon as
the_server is CREATE COMPLETE. configA and configB go into CREATE
COMPLETE when heat-engine receives a signal that they are complete. This
signal may include values that other resources depend on to start their
creation.
My biggest objection to hosted_on is that if I want to reuse configA.. I
can't without an additional level of abstraction. This feels awkward,
and will be an awkward thing for users. In addition, it is very unclear
what what will actually be on the_server when looking at the_server.
Nobody has answered my question where else this sort of pattern is used.
OS::Nova::Server config_resources
-
resources:
configA
type: Heat::ScriptConfig
properties:
script: |
#!/bin/bash
logger 1. hello config A
configB
type: Heat::ScriptConfig
properties:
script: |
#!/bin/bash
logger 1. hello config B
the_server:
type: OS::Nova::Server
properties:
config_resources:
- {get_resource: configA}
- {get_resource: configB}
Here there would need to be some bypassing of dependency calculation to
allow configA and configB to be created after the_server, maybe by:
* special treatment of config_resources to prevent dependencies being
created
* a get_resource variant which doesn't create a hard dependency
(get_resource_deferred?)
IMO no, these are not best expressed as resources in the same way that
python classes are not expressed as objects.
The template above reads more like code. If used as a component, configA
remains reusable throughout the rest of the template as it grows and
changes, and it is very obvious what is expected on the_server. Each
server using configA gets its very own completion waitcondition for that
instantiation of the component.
This is why I liked the components concept, rather than have them as
resources. It seems clear to me that components are parse-time objects,
so if I refer to the same component 4 times, I really want 4 copies of
it. Resources, on the other hand, are real things that are instantiated.
Neither the hosted_on nor the config_resources behaviours are ideal, but
I'm leaning towards hosted_on at the moment since it doesn't require any
new soft-dependency mechanism.
IMO hosted_on is a new dependency mechanism, so avoiding the more clear
paradigm for the reason of not introducing dependency mechanism seems a
bit backwards too.
As for composability, what *actually* needs to be
Re: [openstack-dev] [Heat] HOT Software configuration proposal
Hi Keith, thanks for sharing your opinion. That seems to make sense, and I know Adrian was heavily involved in discussion at the Portland summit,. so seems like the right contacts are hooked up. Looking forward to the discussions at the summit. Regards, Thomas Keith Bray keith.b...@rackspace.com wrote on 25.10.2013 02:23:55: From: Keith Bray keith.b...@rackspace.com To: OpenStack Development Mailing List openstack-dev@lists.openstack.org, Date: 25.10.2013 02:31 Subject: Re: [openstack-dev] [Heat] HOT Software configuration proposal Hi Thomas, here's my opinion: Heat and Solum contributors will work closely together to figure out where specific feature implementations belong... But, in general, Solum is working at a level above Heat. To write a Heat template, you have to know about infrastructure setup and configuration settings of infrastructure and API services. I believe Solum intends to provide the ability to tweak and configure the amount of complexity that gets exposed or hidden so that it becomes easier for cloud consumers to just deal with their application and not have to necessarily know or care about the underlying infrastructure and API services, but that level of detail can be exposed to them if necessary. Solum will know what infrastructure and services to set up to run applications, and it will leverage Heat and Heat templates for this. The Solum project has been very vocal about leveraging Heat under the hood for the functionality and vision of orchestration that it intends to provide. It seems, based on this thread (and +1 from me), enough people are interested in having Heat provide some level of software orchestration, even if it's just bootstrapping other CM tools and coordinating the when are you done, and I haven't heard any Solum folks object to Heat implementing software orchestration capabilities... So, I'm looking forward to great discussions on this topic for Heat at the summit. If you recall, Adrian Otto (who announced project Solum) was also the one who was vocal at the Portland summit about the need for HOT syntax. I think both projects are on a good path with a lot of fun collaboration time ahead. Kind regards, -Keith On 10/24/13 7:56 AM, Thomas Spatzier thomas.spatz...@de.ibm.com wrote: Hi all, maybe a bit off track with respect to latest concrete discussions, but I noticed the announcement of project Solum on openstack-dev. Maybe this is playing on a different level, but I still see some relation to all the software orchestration we are having. What are your opinions on this? BTW, I just posted a similar short question in reply to the Solum announcement mail, but some of us have mail filters an might read [Heat] mail with higher prio, and I was interested in the Heat view. Cheers, Thomas Patrick Petit patrick.pe...@bull.net wrote on 24.10.2013 12:15:13: From: Patrick Petit patrick.pe...@bull.net To: OpenStack Development Mailing List openstack-dev@lists.openstack.org, Date: 24.10.2013 12:18 Subject: Re: [openstack-dev] [Heat] HOT Software configuration proposal Sorry, I clicked the 'send' button too quickly. On 10/24/13 11:54 AM, Patrick Petit wrote: Hi Clint, Thank you! I have few replies/questions in-line. Cheers, Patrick On 10/23/13 8:36 PM, Clint Byrum wrote: Excerpts from Patrick Petit's message of 2013-10-23 10:58:22 -0700: Dear Steve and All, If I may add up on this already busy thread to share our experience with using Heat in large and complex software deployments. Thanks for sharing Patrick, I have a few replies in-line. I work on a project which precisely provides additional value at the articulation point between resource orchestration automation and configuration management. We rely on Heat and chef-solo respectively for these base management functions. On top of this, we have developed an event-driven workflow to manage the life-cycles of complex software stacks which primary purpose is to support middleware components as opposed to end-user apps. Our use cases are peculiar in the sense that software setup (install, config, contextualization) is not a one-time operation issue but a continuous thing that can happen any time in life-span of a stack. Users can deploy (and undeploy) apps long time after the stack is created. Auto-scaling may also result in an asynchronous apps deployment. More about this latter. The framework we have designed works well for us. It clearly refers to a PaaS-like environment which I understand is not the topic of the HOT software configuration proposal(s) and that's absolutely fine with us. However, the question for us is whether the separation of software config from resources would make our life easier or not. I think the answer is definitely yes but at the condition that the DSL extension preserves almost everything from the expressiveness
Re: [openstack-dev] [Heat] HOT Software configuration proposal
Excerpts from Angus Salkeld's message of 2013-10-24 18:48:16 -0700:
On 24/10/13 11:54 +0200, Patrick Petit wrote:
Hi Clint,
Thank you! I have few replies/questions in-line.
Cheers,
Patrick
On 10/23/13 8:36 PM, Clint Byrum wrote:
I think this fits into something that I want for optimizing
os-collect-config as well (our in-instance Heat-aware agent). That is
a way for us to wait for notification of changes to Metadata without
polling.
Interesting... If I understand correctly that's kinda replacement of
cfn-hup... Do you have a blueprint pointer or something more
specific? While I see the benefits of it, in-instance notifications
is not really what we are looking for. We are looking for a
notification service that exposes an API whereby listeners can
register for Heat notifications. AWS Alarming / CloudFormation has
that. Why not Ceilometer / Heat? That would be extremely valuable for
those who build PaaS-like solutions above Heat. To say it bluntly,
I'd like to suggest we explore ways to integrate Heat with Marconi.
Yeah, I am trying to do a PoC of this now. I'll let you know how
it goes.
I am trying to implement the following:
heat_template_version: 2013-05-23
parameters:
key_name:
type: String
flavor:
type: String
default: m1.small
image:
type: String
default: fedora-19-i386-heat-cfntools
resources:
config_server:
type: OS::Marconi::QueueServer
properties:
image: {get_param: image}
flavor: {get_param: flavor}
key_name: {get_param: key_name}
configA:
type: OS::Heat::OrderedConfig
properties:
marconi_server: {get_attr: [config_server, url]}
hosted_on: {get_resource: serv1}
script: |
#!/bin/bash
logger 1. hello from marconi
configB:
type: OS::Heat::OrderedConfig
properties:
marconi_server: {get_attr: [config_server, url]}
hosted_on: {get_resource: serv1}
depends_on: {get_resource: configA}
script: |
#!/bin/bash
logger 2. hello from marconi
serv1:
type: OS::Nova::Server
properties:
image: {get_param: image}
flavor: {get_param: flavor}
key_name: {get_param: key_name}
user_data: |
#!/bin/sh
# poll marconi url/v1/queues/{hostname}/messages
# apply config
# post a response message with any outputs
# delete request message
If I may diverge this a bit, I'd like to consider the impact of
hosted_on on reusability in templates. hosted_on feels like an
anti-pattern, and I've never seen anything quite like it. It feels wrong
for a well contained component to then reach out and push itself onto
something else which has no mention of it.
I'll rewrite your template as I envision it working:
resources:
config_server:
type: OS::Marconi::QueueServer
properties:
image: {get_param: image}
flavor: {get_param: flavor}
key_name: {get_param: key_name}
configA:
type: OS::Heat::OrderedConfig
properties:
marconi_server: {get_attr: [config_server, url]}
script: |
#!/bin/bash
logger 1. hello from marconi
configB:
type: OS::Heat::OrderedConfig
properties:
marconi_server: {get_attr: [config_server, url]}
depends_on: {get_resource: configA}
script: |
#!/bin/bash
logger 2. hello from marconi
serv1:
type: OS::Nova::Server
properties:
image: {get_param: image}
flavor: {get_param: flavor}
key_name: {get_param: key_name}
components:
- configA
- configB
user_data: |
#!/bin/sh
# poll marconi url/v1/queues/{hostname}/messages
# apply config
# post a response message with any outputs
# delete request message
This only becomes obvious why it is important when you want to do this:
configC:
type: OS::Heat::OrderedConfig
properties:
script: |
#!/bin/bash
logger ?. I can race with A, no dependency needed
serv2:
type: OS::Nova::Server
properties:
...
components:
- configA
- configC
This is proper composition, where the caller defines the components, not
the callee. Now you can re-use configA with a different component in the
same template. As we get smarter we can have these configs separate from
the template and reusable across templates.
Anyway, I'd like to see us stop talking about hosted_on, and if it has
been implemented, that it be deprecated and eventually removed, as it is
just plain confusing.
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Heat] HOT Software configuration proposal
On 25/10/13 09:25 -0700, Clint Byrum wrote:
Excerpts from Angus Salkeld's message of 2013-10-24 18:48:16 -0700:
On 24/10/13 11:54 +0200, Patrick Petit wrote:
Hi Clint,
Thank you! I have few replies/questions in-line.
Cheers
Patrick
On 10/23/13 8:36 PM, Clint Byrum wrote:
I think this fits into something that I want for optimizing
os-collect-config as well (our in-instance Heat-aware agent). That is
a way for us to wait for notification of changes to Metadata without
polling.
Interesting... If I understand correctly that's kinda replacement of
cfn-hup... Do you have a blueprint pointer or something more
specific? While I see the benefits of it, in-instance notifications
is not really what we are looking for. We are looking for a
notification service that exposes an API whereby listeners can
register for Heat notifications. AWS Alarming / CloudFormation has
that. Why not Ceilometer / Heat? That would be extremely valuable for
those who build PaaS-like solutions above Heat. To say it bluntly,
I'd like to suggest we explore ways to integrate Heat with Marconi.
Yeah, I am trying to do a PoC of this now. I'll let you know how
it goes.
I am trying to implement the following:
heat_template_version: 2013-05-23
parameters:
key_name:
type: String
flavor:
type: String
default: m1.small
image:
type: String
default: fedora-19-i386-heat-cfntools
resources:
config_server:
type: OS::Marconi::QueueServer
properties:
image: {get_param: image}
flavor: {get_param: flavor}
key_name: {get_param: key_name}
configA:
type: OS::Heat::OrderedConfig
properties:
marconi_server: {get_attr: [config_server, url]}
hosted_on: {get_resource: serv1}
script: |
#!/bin/bash
logger 1. hello from marconi
configB:
type: OS::Heat::OrderedConfig
properties:
marconi_server: {get_attr: [config_server, url]}
hosted_on: {get_resource: serv1}
depends_on: {get_resource: configA}
script: |
#!/bin/bash
logger 2. hello from marconi
serv1:
type: OS::Nova::Server
properties:
image: {get_param: image}
flavor: {get_param: flavor}
key_name: {get_param: key_name}
user_data: |
#!/bin/sh
# poll marconi url/v1/queues/{hostname}/messages
# apply config
# post a response message with any outputs
# delete request message
If I may diverge this a bit, I'd like to consider the impact of
hosted_on on reusability in templates. hosted_on feels like an
anti-pattern, and I've never seen anything quite like it. It feels wrong
for a well contained component to then reach out and push itself onto
something else which has no mention of it.
Maybe I shouldn't have used hosted_on, it could be role_name/config_queue.
I'll rewrite your template as I envision it working:
resources:
config_server:
type: OS::Marconi::QueueServer
properties:
image: {get_param: image}
flavor: {get_param: flavor}
key_name: {get_param: key_name}
configA:
type: OS::Heat::OrderedConfig
properties:
marconi_server: {get_attr: [config_server, url]}
script: |
#!/bin/bash
logger 1. hello from marconi
configB:
type: OS::Heat::OrderedConfig
properties:
marconi_server: {get_attr: [config_server, url]}
depends_on: {get_resource: configA}
script: |
#!/bin/bash
logger 2. hello from marconi
serv1:
type: OS::Nova::Server
properties:
image: {get_param: image}
flavor: {get_param: flavor}
key_name: {get_param: key_name}
components:
- configA
- configB
user_data: |
#!/bin/sh
# poll marconi url/v1/queues/{hostname}/messages
# apply config
# post a response message with any outputs
# delete request message
This only becomes obvious why it is important when you want to do this:
configC:
type: OS::Heat::OrderedConfig
properties:
script: |
#!/bin/bash
logger ?. I can race with A, no dependency needed
Well if you put no dependency, it's like any other heat resource
they will be run in parallel (or at least either may go first)
there are lots of configs where this may not be important.
serv2:
type: OS::Nova::Server
properties:
...
components:
- configA
- configC
This is proper composition, where the caller defines the components, not
the callee. Now you can re-use configA with a different component in the
same template. As we get smarter we can have these configs separate from
the template and reusable across templates.
Anyway, I'd like to see us stop talking about hosted_on, and if it has
been implemented, that it be deprecated and eventually removed, as it is
just plain confusing.
There are pros and cons to both, I don't however think it is helpful
to shutdown
Re: [openstack-dev] [Heat] HOT Software configuration proposal
Hi Clint, Thank you! I have few replies/questions in-line. Cheers, Patrick On 10/23/13 8:36 PM, Clint Byrum wrote: Excerpts from Patrick Petit's message of 2013-10-23 10:58:22 -0700: Dear Steve and All, If I may add up on this already busy thread to share our experience with using Heat in large and complex software deployments. Thanks for sharing Patrick, I have a few replies in-line. I work on a project which precisely provides additional value at the articulation point between resource orchestration automation and configuration management. We rely on Heat and chef-solo respectively for these base management functions. On top of this, we have developed an event-driven workflow to manage the life-cycles of complex software stacks which primary purpose is to support middleware components as opposed to end-user apps. Our use cases are peculiar in the sense that software setup (install, config, contextualization) is not a one-time operation issue but a continuous thing that can happen any time in life-span of a stack. Users can deploy (and undeploy) apps long time after the stack is created. Auto-scaling may also result in an asynchronous apps deployment. More about this latter. The framework we have designed works well for us. It clearly refers to a PaaS-like environment which I understand is not the topic of the HOT software configuration proposal(s) and that's absolutely fine with us. However, the question for us is whether the separation of software config from resources would make our life easier or not. I think the answer is definitely yes but at the condition that the DSL extension preserves almost everything from the expressiveness of the resource element. In practice, I think that a strict separation between resource and component will be hard to achieve because we'll always need a little bit of application's specific in the resources. Take for example the case of the SecurityGroups. The ports open in a SecurityGroup are application specific. Components can only be made up of the things that are common to all users of said component. Also components would, if I understand the concept correctly, just be for things that are at the sub-resource level. That isn't cop Security groups and open ports would be across multiple resources, and thus would be separately specified from your app's component (though it might be useful to allow components to export static values so that the port list can be referred to along with the app component). Then, designing a Chef or Puppet component type may be harder than it looks at first glance. Speaking of our use cases we still need a little bit of scripting in the instance's user-data block to setup a working chef-solo environment. For example, we run librarian-chef prior to starting chef-solo to resolve the cookbook dependencies. A cookbook can present itself as a downloadable tarball but it's not always the case. A chef component type would have to support getting a cookbook from a public or private git repo (maybe subversion), handle situations where there is one cookbook per repo or multiple cookbooks per repo, let the user choose a particular branch or label, provide ssh keys if it's a private repo, and so forth. We support all of this scenarios and so we can provide more detailed requirements if needed. Correct me if I'm wrong though, all of those scenarios are just variations on standard inputs into chef. So the chef component really just has to allow a way to feed data to chef. I am not sure adding component relations like the 'depends-on' would really help us since it is the job of config management to handle software dependencies. Also, it doesn't address the issue of circular dependencies. Circular dependencies occur in complex software stack deployments. Example. When we setup a Slum virtual cluster, both the head node and compute nodes depend on one another to complete their configuration and so they would wait for each other indefinitely if we were to rely on the 'depends-on'. In addition, I think it's critical to distinguish between configuration parameters which are known ahead of time, like a db name or user name and password, versus contextualization parameters which are known after the fact generally when the instance is created. Typically those contextualization parameters are IP addresses but not only. The fact packages x,y,z have been properly installed and services a,b,c successfully started is contextualization information (a.k.a facts) which may be indicative that other components can move on to the next setup stage. The form of contextualization you mention above can be handled by a slightly more capable wait condition mechanism than we have now. I've been suggesting that this is the interface that workflow systems should use. The case of complex deployments with or without circular dependencies is typically resolved by making the system converge toward the desirable end-state through running idempotent recipes. This
Re: [openstack-dev] [Heat] HOT Software configuration proposal
Sorry, I clicked the 'send' button too quickly. On 10/24/13 11:54 AM, Patrick Petit wrote: Hi Clint, Thank you! I have few replies/questions in-line. Cheers, Patrick On 10/23/13 8:36 PM, Clint Byrum wrote: Excerpts from Patrick Petit's message of 2013-10-23 10:58:22 -0700: Dear Steve and All, If I may add up on this already busy thread to share our experience with using Heat in large and complex software deployments. Thanks for sharing Patrick, I have a few replies in-line. I work on a project which precisely provides additional value at the articulation point between resource orchestration automation and configuration management. We rely on Heat and chef-solo respectively for these base management functions. On top of this, we have developed an event-driven workflow to manage the life-cycles of complex software stacks which primary purpose is to support middleware components as opposed to end-user apps. Our use cases are peculiar in the sense that software setup (install, config, contextualization) is not a one-time operation issue but a continuous thing that can happen any time in life-span of a stack. Users can deploy (and undeploy) apps long time after the stack is created. Auto-scaling may also result in an asynchronous apps deployment. More about this latter. The framework we have designed works well for us. It clearly refers to a PaaS-like environment which I understand is not the topic of the HOT software configuration proposal(s) and that's absolutely fine with us. However, the question for us is whether the separation of software config from resources would make our life easier or not. I think the answer is definitely yes but at the condition that the DSL extension preserves almost everything from the expressiveness of the resource element. In practice, I think that a strict separation between resource and component will be hard to achieve because we'll always need a little bit of application's specific in the resources. Take for example the case of the SecurityGroups. The ports open in a SecurityGroup are application specific. Components can only be made up of the things that are common to all users of said component. Also components would, if I understand the concept correctly, just be for things that are at the sub-resource level. Security groups and open ports would be across multiple resources, and thus would be separately specified from your app's component (though it might be useful to allow components to export static values so that the port list can be referred to along with the app component). Okay got it. If that's the case then that would work Then, designing a Chef or Puppet component type may be harder than it looks at first glance. Speaking of our use cases we still need a little bit of scripting in the instance's user-data block to setup a working chef-solo environment. For example, we run librarian-chef prior to starting chef-solo to resolve the cookbook dependencies. A cookbook can present itself as a downloadable tarball but it's not always the case. A chef component type would have to support getting a cookbook from a public or private git repo (maybe subversion), handle situations where there is one cookbook per repo or multiple cookbooks per repo, let the user choose a particular branch or label, provide ssh keys if it's a private repo, and so forth. We support all of this scenarios and so we can provide more detailed requirements if needed. Correct me if I'm wrong though, all of those scenarios are just variations on standard inputs into chef. So the chef component really just has to allow a way to feed data to chef. That's correct. Boils down to specifying correctly all the constraints that apply to deploying a cookbook in an instance from it's component description. I am not sure adding component relations like the 'depends-on' would really help us since it is the job of config management to handle software dependencies. Also, it doesn't address the issue of circular dependencies. Circular dependencies occur in complex software stack deployments. Example. When we setup a Slum virtual cluster, both the head node and compute nodes depend on one another to complete their configuration and so they would wait for each other indefinitely if we were to rely on the 'depends-on'. In addition, I think it's critical to distinguish between configuration parameters which are known ahead of time, like a db name or user name and password, versus contextualization parameters which are known after the fact generally when the instance is created. Typically those contextualization parameters are IP addresses but not only. The fact packages x,y,z have been properly installed and services a,b,c successfully started is contextualization information (a.k.a facts) which may be indicative that other components can move on to the next setup stage. The form of contextualization you mention above can be handled by a slightly more capable wait condition
Re: [openstack-dev] [Heat] HOT Software configuration proposal
Hi all, maybe a bit off track with respect to latest concrete discussions, but I noticed the announcement of project Solum on openstack-dev. Maybe this is playing on a different level, but I still see some relation to all the software orchestration we are having. What are your opinions on this? BTW, I just posted a similar short question in reply to the Solum announcement mail, but some of us have mail filters an might read [Heat] mail with higher prio, and I was interested in the Heat view. Cheers, Thomas Patrick Petit patrick.pe...@bull.net wrote on 24.10.2013 12:15:13: From: Patrick Petit patrick.pe...@bull.net To: OpenStack Development Mailing List openstack-dev@lists.openstack.org, Date: 24.10.2013 12:18 Subject: Re: [openstack-dev] [Heat] HOT Software configuration proposal Sorry, I clicked the 'send' button too quickly. On 10/24/13 11:54 AM, Patrick Petit wrote: Hi Clint, Thank you! I have few replies/questions in-line. Cheers, Patrick On 10/23/13 8:36 PM, Clint Byrum wrote: Excerpts from Patrick Petit's message of 2013-10-23 10:58:22 -0700: Dear Steve and All, If I may add up on this already busy thread to share our experience with using Heat in large and complex software deployments. Thanks for sharing Patrick, I have a few replies in-line. I work on a project which precisely provides additional value at the articulation point between resource orchestration automation and configuration management. We rely on Heat and chef-solo respectively for these base management functions. On top of this, we have developed an event-driven workflow to manage the life-cycles of complex software stacks which primary purpose is to support middleware components as opposed to end-user apps. Our use cases are peculiar in the sense that software setup (install, config, contextualization) is not a one-time operation issue but a continuous thing that can happen any time in life-span of a stack. Users can deploy (and undeploy) apps long time after the stack is created. Auto-scaling may also result in an asynchronous apps deployment. More about this latter. The framework we have designed works well for us. It clearly refers to a PaaS-like environment which I understand is not the topic of the HOT software configuration proposal(s) and that's absolutely fine with us. However, the question for us is whether the separation of software config from resources would make our life easier or not. I think the answer is definitely yes but at the condition that the DSL extension preserves almost everything from the expressiveness of the resource element. In practice, I think that a strict separation between resource and component will be hard to achieve because we'll always need a little bit of application's specific in the resources. Take for example the case of the SecurityGroups. The ports open in a SecurityGroup are application specific. Components can only be made up of the things that are common to all users of said component. Also components would, if I understand the concept correctly, just be for things that are at the sub-resource level. Security groups and open ports would be across multiple resources, and thus would be separately specified from your app's component (though it might be useful to allow components to export static values so that the port list can be referred to along with the app component). Okay got it. If that's the case then that would work Then, designing a Chef or Puppet component type may be harder than it looks at first glance. Speaking of our use cases we still need a little bit of scripting in the instance's user-data block to setup a working chef-solo environment. For example, we run librarian-chef prior to starting chef-solo to resolve the cookbook dependencies. A cookbook can present itself as a downloadable tarball but it's not always the case. A chef component type would have to support getting a cookbook from a public or private git repo (maybe subversion), handle situations where there is one cookbook per repo or multiple cookbooks per repo, let the user choose a particular branch or label, provide ssh keys if it's a private repo, and so forth. We support all of this scenarios and so we can provide more detailed requirements if needed. Correct me if I'm wrong though, all of those scenarios are just variations on standard inputs into chef. So the chef component really just has to allow a way to feed data to chef. That's correct. Boils down to specifying correctly all the constraints that apply to deploying a cookbook in an instance from it's component description. I am not sure adding component relations like the 'depends-on' would really help us since it is the job of config management to handle software dependencies. Also, it doesn't address the issue of circular dependencies. Circular dependencies occur in complex software
Re: [openstack-dev] [Heat] HOT Software configuration proposal
Hi Thomas, here's my opinion: Heat and Solum contributors will work closely together to figure out where specific feature implementations belong... But, in general, Solum is working at a level above Heat. To write a Heat template, you have to know about infrastructure setup and configuration settings of infrastructure and API services. I believe Solum intends to provide the ability to tweak and configure the amount of complexity that gets exposed or hidden so that it becomes easier for cloud consumers to just deal with their application and not have to necessarily know or care about the underlying infrastructure and API services, but that level of detail can be exposed to them if necessary. Solum will know what infrastructure and services to set up to run applications, and it will leverage Heat and Heat templates for this. The Solum project has been very vocal about leveraging Heat under the hood for the functionality and vision of orchestration that it intends to provide. It seems, based on this thread (and +1 from me), enough people are interested in having Heat provide some level of software orchestration, even if it's just bootstrapping other CM tools and coordinating the when are you done, and I haven't heard any Solum folks object to Heat implementing software orchestration capabilities... So, I'm looking forward to great discussions on this topic for Heat at the summit. If you recall, Adrian Otto (who announced project Solum) was also the one who was vocal at the Portland summit about the need for HOT syntax. I think both projects are on a good path with a lot of fun collaboration time ahead. Kind regards, -Keith On 10/24/13 7:56 AM, Thomas Spatzier thomas.spatz...@de.ibm.com wrote: Hi all, maybe a bit off track with respect to latest concrete discussions, but I noticed the announcement of project Solum on openstack-dev. Maybe this is playing on a different level, but I still see some relation to all the software orchestration we are having. What are your opinions on this? BTW, I just posted a similar short question in reply to the Solum announcement mail, but some of us have mail filters an might read [Heat] mail with higher prio, and I was interested in the Heat view. Cheers, Thomas Patrick Petit patrick.pe...@bull.net wrote on 24.10.2013 12:15:13: From: Patrick Petit patrick.pe...@bull.net To: OpenStack Development Mailing List openstack-dev@lists.openstack.org, Date: 24.10.2013 12:18 Subject: Re: [openstack-dev] [Heat] HOT Software configuration proposal Sorry, I clicked the 'send' button too quickly. On 10/24/13 11:54 AM, Patrick Petit wrote: Hi Clint, Thank you! I have few replies/questions in-line. Cheers, Patrick On 10/23/13 8:36 PM, Clint Byrum wrote: Excerpts from Patrick Petit's message of 2013-10-23 10:58:22 -0700: Dear Steve and All, If I may add up on this already busy thread to share our experience with using Heat in large and complex software deployments. Thanks for sharing Patrick, I have a few replies in-line. I work on a project which precisely provides additional value at the articulation point between resource orchestration automation and configuration management. We rely on Heat and chef-solo respectively for these base management functions. On top of this, we have developed an event-driven workflow to manage the life-cycles of complex software stacks which primary purpose is to support middleware components as opposed to end-user apps. Our use cases are peculiar in the sense that software setup (install, config, contextualization) is not a one-time operation issue but a continuous thing that can happen any time in life-span of a stack. Users can deploy (and undeploy) apps long time after the stack is created. Auto-scaling may also result in an asynchronous apps deployment. More about this latter. The framework we have designed works well for us. It clearly refers to a PaaS-like environment which I understand is not the topic of the HOT software configuration proposal(s) and that's absolutely fine with us. However, the question for us is whether the separation of software config from resources would make our life easier or not. I think the answer is definitely yes but at the condition that the DSL extension preserves almost everything from the expressiveness of the resource element. In practice, I think that a strict separation between resource and component will be hard to achieve because we'll always need a little bit of application's specific in the resources. Take for example the case of the SecurityGroups. The ports open in a SecurityGroup are application specific. Components can only be made up of the things that are common to all users of said component. Also components would, if I understand the concept correctly, just be for things that are at the sub-resource level. Security groups and open ports would be across multiple resources, and thus would
Re: [openstack-dev] [Heat] HOT Software configuration proposal
On 24/10/13 11:54 +0200, Patrick Petit wrote: Hi Clint, Thank you! I have few replies/questions in-line. Cheers, Patrick On 10/23/13 8:36 PM, Clint Byrum wrote: Excerpts from Patrick Petit's message of 2013-10-23 10:58:22 -0700: Dear Steve and All, If I may add up on this already busy thread to share our experience with using Heat in large and complex software deployments. Thanks for sharing Patrick, I have a few replies in-line. I work on a project which precisely provides additional value at the articulation point between resource orchestration automation and configuration management. We rely on Heat and chef-solo respectively for these base management functions. On top of this, we have developed an event-driven workflow to manage the life-cycles of complex software stacks which primary purpose is to support middleware components as opposed to end-user apps. Our use cases are peculiar in the sense that software setup (install, config, contextualization) is not a one-time operation issue but a continuous thing that can happen any time in life-span of a stack. Users can deploy (and undeploy) apps long time after the stack is created. Auto-scaling may also result in an asynchronous apps deployment. More about this latter. The framework we have designed works well for us. It clearly refers to a PaaS-like environment which I understand is not the topic of the HOT software configuration proposal(s) and that's absolutely fine with us. However, the question for us is whether the separation of software config from resources would make our life easier or not. I think the answer is definitely yes but at the condition that the DSL extension preserves almost everything from the expressiveness of the resource element. In practice, I think that a strict separation between resource and component will be hard to achieve because we'll always need a little bit of application's specific in the resources. Take for example the case of the SecurityGroups. The ports open in a SecurityGroup are application specific. Components can only be made up of the things that are common to all users of said component. Also components would, if I understand the concept correctly, just be for things that are at the sub-resource level. That isn't cop Security groups and open ports would be across multiple resources, and thus would be separately specified from your app's component (though it might be useful to allow components to export static values so that the port list can be referred to along with the app component). Then, designing a Chef or Puppet component type may be harder than it looks at first glance. Speaking of our use cases we still need a little bit of scripting in the instance's user-data block to setup a working chef-solo environment. For example, we run librarian-chef prior to starting chef-solo to resolve the cookbook dependencies. A cookbook can present itself as a downloadable tarball but it's not always the case. A chef component type would have to support getting a cookbook from a public or private git repo (maybe subversion), handle situations where there is one cookbook per repo or multiple cookbooks per repo, let the user choose a particular branch or label, provide ssh keys if it's a private repo, and so forth. We support all of this scenarios and so we can provide more detailed requirements if needed. Correct me if I'm wrong though, all of those scenarios are just variations on standard inputs into chef. So the chef component really just has to allow a way to feed data to chef. I am not sure adding component relations like the 'depends-on' would really help us since it is the job of config management to handle software dependencies. Also, it doesn't address the issue of circular dependencies. Circular dependencies occur in complex software stack deployments. Example. When we setup a Slum virtual cluster, both the head node and compute nodes depend on one another to complete their configuration and so they would wait for each other indefinitely if we were to rely on the 'depends-on'. In addition, I think it's critical to distinguish between configuration parameters which are known ahead of time, like a db name or user name and password, versus contextualization parameters which are known after the fact generally when the instance is created. Typically those contextualization parameters are IP addresses but not only. The fact packages x,y,z have been properly installed and services a,b,c successfully started is contextualization information (a.k.a facts) which may be indicative that other components can move on to the next setup stage. The form of contextualization you mention above can be handled by a slightly more capable wait condition mechanism than we have now. I've been suggesting that this is the interface that workflow systems should use. The case of complex deployments with or without circular dependencies is typically resolved by making the system converge toward the desirable
Re: [openstack-dev] [Heat] HOT Software configuration proposal
Clint Byrum cl...@fewbar.com wrote on 23.10.2013 00:28:17:
From: Clint Byrum cl...@fewbar.com
To: openstack-dev openstack-dev@lists.openstack.org,
Date: 23.10.2013 00:30
Subject: Re: [openstack-dev] [Heat] HOT Software configuration proposal
Excerpts from Georgy Okrokvertskhov's message of 2013-10-22 13:32:40
-0700:
Hi Thomas,
I agree with you on semantics part. At the same time I see a potential
question which might appear - if semantics is limited by few states
visible
for Heat engine, then who actually does software orchestration?
Will it be reasonable then to have software orchestration as separate
subproject for Heat as a part of Orchestration OpenStack program?
Heat
engine will then do dependency tracking and will use components as a
reference for software orchestration engine which will perform actual
deployment and high level software components coordination.
This separated software orchestration engine may address all specific
requirements proposed by different teams in this thread without
affecting
existing Heat engine.
I'm not sure I know what software orchestration is, but I will take a
stab at a succinct definition:
Coordination of software configuration across multiple hosts.
If that is what you mean, then I believe what you actually want is
workflow. And for that, we have the Mistral project which was recently
announced [1].
My view of software orchestration in a sense of what Heat should be able
to do is being able to bring up software installations (e.g. web server, a
DBMS, a custom application) on-top of a bare compute resource by invoking a
software config tool (e.g. Chef, Puppet ...) at the right point in time and
let that tool do the actual work.
Invoke does not necessarily mean to call an API of such a tool, but rather
make sure it is bootstrapped and maybe gets a go signal to start.
software orchestration could then further mean to give CM tools across
hosts the go signal when the config on one host has completed. This be
enabled by the signaling enhancements Steve Baker mentioned in one of his
recent mails.
For such kind of stuff, I think we could live without workflows but do it
purely declaratively. Of course, a workflow could be the underlying
mechanism, but I would not want to express this in a template. If users
have very complex problems to solve and cannot live with the simple
software orchestration I outlined, then still a workflow could be used for
everying on-top of the OS.
Anyway, that was just my view and others most probably have different views
again, so seems like we really have to sort out terminology :-)
Use that and you will simply need to define your desired workflow and
feed it into Mistral using a Mistral Heat resource. We can create a
nice bootstrapping resource for Heat instances that shims the mistral
workflow execution agent into machines (or lets us use one already there
via custom images).
I can imagine it working something like this:
resources:
mistral_workflow_handle:
type: OS::Mistral::WorkflowHandle
web_server:
type: OS::Nova::Server
components:
mistral_agent:
component_type: mistral
params:
workflow_: {ref: mistral_workflow_handle}
mysql_server:
type: OS::Nova::Server
components:
mistral_agent:
component_type: mistral
params:
workflow_handle: {ref: mistral_workflow_handle}
mistral_workflow:
type: OS::Mistral::Workflow
properties:
handle: {ref: mistral_workflow_handle}
workflow_reference: mysql_webapp_workflow
params:
mysql_server: {ref: mysql_server}
webserver: {ref: web_server}
While I can imagine that this works, I think for a big percentage of use
cases I would be nice to avoid this inter-weaving of workflow constructs
with a HOT template. I think we could do a purely declarative approach (if
we scope software orchestration in context of Heat right), and not define
such handles and references.
We are trying to shield this from the users in other cases in HOT
(WaitConditionHandle and references), so why introduce it here ...
And then the workflow is just defined outside of the Heat template (ok
I'm sure somebody will want to embed it, but I prefer stronger
separation). Something like this gets uploaded as
mysql_webapp_workflow:
[ 'step1': 'install_stuff',
'step2': 'wait(step1)',
'step3': 'allocate_sql_user(server=%mysql_server%)'
'step4': 'credentials=wait_and_read(step3)'
'step5': 'write_config_file(server=%webserver%)' ]
Or maybe it is declared as a graph, or whatever, but it is not Heat's
problem how to do workflows, it just feeds the necessary data from
orchestration into the workflow engine. This also means you can use a
non OpenStack workflow engine without any problems.
I think after having talked about this, we should have workflow live in
its own program.. we can always combine them if we want
Re: [openstack-dev] [Heat] HOT Software configuration proposal
Dear Steve and All, If I may add up on this already busy thread to share our experience with using Heat in large and complex software deployments. I work on a project which precisely provides additional value at the articulation point between resource orchestration automation and configuration management. We rely on Heat and chef-solo respectively for these base management functions. On top of this, we have developed an event-driven workflow to manage the life-cycles of complex software stacks which primary purpose is to support middleware components as opposed to end-user apps. Our use cases are peculiar in the sense that software setup (install, config, contextualization) is not a one-time operation issue but a continuous thing that can happen any time in life-span of a stack. Users can deploy (and undeploy) apps long time after the stack is created. Auto-scaling may also result in an asynchronous apps deployment. More about this latter. The framework we have designed works well for us. It clearly refers to a PaaS-like environment which I understand is not the topic of the HOT software configuration proposal(s) and that's absolutely fine with us. However, the question for us is whether the separation of software config from resources would make our life easier or not. I think the answer is definitely yes but at the condition that the DSL extension preserves almost everything from the expressiveness of the resource element. In practice, I think that a strict separation between resource and component will be hard to achieve because we'll always need a little bit of application's specific in the resources. Take for example the case of the SecurityGroups. The ports open in a SecurityGroup are application specific. Then, designing a Chef or Puppet component type may be harder than it looks at first glance. Speaking of our use cases we still need a little bit of scripting in the instance's user-data block to setup a working chef-solo environment. For example, we run librarian-chef prior to starting chef-solo to resolve the cookbook dependencies. A cookbook can present itself as a downloadable tarball but it's not always the case. A chef component type would have to support getting a cookbook from a public or private git repo (maybe subversion), handle situations where there is one cookbook per repo or multiple cookbooks per repo, let the user choose a particular branch or label, provide ssh keys if it's a private repo, and so forth. We support all of this scenarios and so we can provide more detailed requirements if needed. I am not sure adding component relations like the 'depends-on' would really help us since it is the job of config management to handle software dependencies. Also, it doesn't address the issue of circular dependencies. Circular dependencies occur in complex software stack deployments. Example. When we setup a Slum virtual cluster, both the head node and compute nodes depend on one another to complete their configuration and so they would wait for each other indefinitely if we were to rely on the 'depends-on'. In addition, I think it's critical to distinguish between configuration parameters which are known ahead of time, like a db name or user name and password, versus contextualization parameters which are known after the fact generally when the instance is created. Typically those contextualization parameters are IP addresses but not only. The fact packages x,y,z have been properly installed and services a,b,c successfully started is contextualization information (a.k.a facts) which may be indicative that other components can move on to the next setup stage. The case of complex deployments with or without circular dependencies is typically resolved by making the system converge toward the desirable end-state through running idempotent recipes. This is our approach. The first configuration phase handles parametrization which in general brings an instance to CREATE_COMPLETE state. A second phase follows to handle contextualization at the stack level. As a matter of fact, a new contextualization should be triggered every time an instance enters or leave the CREATE_COMPLETE state which may happen any time with auto-scaling. In that phase, circular dependencies can be resolved because all contextualization data can be compiled globally. Notice that Heat doesn't provide a purpose built resource or service like Chef's data-bag for the storage and retrieval of metadata. This a gap which IMO should be addressed in the proposal. Currently, we use a kludge that is to create a fake AWS::AutoScaling::LaunchConfiguration resource to store contextualization data in the metadata section of that resource. Aside from the HOT software configuration proposal(s). There are two critical enhancements in Heat that would make software life-cycles management much easier. In fact, they are actual blockers for us. The first one would be
Re: [openstack-dev] [Heat] HOT Software configuration proposal
Excerpts from Patrick Petit's message of 2013-10-23 10:58:22 -0700: Dear Steve and All, If I may add up on this already busy thread to share our experience with using Heat in large and complex software deployments. Thanks for sharing Patrick, I have a few replies in-line. I work on a project which precisely provides additional value at the articulation point between resource orchestration automation and configuration management. We rely on Heat and chef-solo respectively for these base management functions. On top of this, we have developed an event-driven workflow to manage the life-cycles of complex software stacks which primary purpose is to support middleware components as opposed to end-user apps. Our use cases are peculiar in the sense that software setup (install, config, contextualization) is not a one-time operation issue but a continuous thing that can happen any time in life-span of a stack. Users can deploy (and undeploy) apps long time after the stack is created. Auto-scaling may also result in an asynchronous apps deployment. More about this latter. The framework we have designed works well for us. It clearly refers to a PaaS-like environment which I understand is not the topic of the HOT software configuration proposal(s) and that's absolutely fine with us. However, the question for us is whether the separation of software config from resources would make our life easier or not. I think the answer is definitely yes but at the condition that the DSL extension preserves almost everything from the expressiveness of the resource element. In practice, I think that a strict separation between resource and component will be hard to achieve because we'll always need a little bit of application's specific in the resources. Take for example the case of the SecurityGroups. The ports open in a SecurityGroup are application specific. Components can only be made up of the things that are common to all users of said component. Also components would, if I understand the concept correctly, just be for things that are at the sub-resource level. Security groups and open ports would be across multiple resources, and thus would be separately specified from your app's component (though it might be useful to allow components to export static values so that the port list can be referred to along with the app component). Then, designing a Chef or Puppet component type may be harder than it looks at first glance. Speaking of our use cases we still need a little bit of scripting in the instance's user-data block to setup a working chef-solo environment. For example, we run librarian-chef prior to starting chef-solo to resolve the cookbook dependencies. A cookbook can present itself as a downloadable tarball but it's not always the case. A chef component type would have to support getting a cookbook from a public or private git repo (maybe subversion), handle situations where there is one cookbook per repo or multiple cookbooks per repo, let the user choose a particular branch or label, provide ssh keys if it's a private repo, and so forth. We support all of this scenarios and so we can provide more detailed requirements if needed. Correct me if I'm wrong though, all of those scenarios are just variations on standard inputs into chef. So the chef component really just has to allow a way to feed data to chef. I am not sure adding component relations like the 'depends-on' would really help us since it is the job of config management to handle software dependencies. Also, it doesn't address the issue of circular dependencies. Circular dependencies occur in complex software stack deployments. Example. When we setup a Slum virtual cluster, both the head node and compute nodes depend on one another to complete their configuration and so they would wait for each other indefinitely if we were to rely on the 'depends-on'. In addition, I think it's critical to distinguish between configuration parameters which are known ahead of time, like a db name or user name and password, versus contextualization parameters which are known after the fact generally when the instance is created. Typically those contextualization parameters are IP addresses but not only. The fact packages x,y,z have been properly installed and services a,b,c successfully started is contextualization information (a.k.a facts) which may be indicative that other components can move on to the next setup stage. The form of contextualization you mention above can be handled by a slightly more capable wait condition mechanism than we have now. I've been suggesting that this is the interface that workflow systems should use. The case of complex deployments with or without circular dependencies is typically resolved by making the system converge toward the desirable end-state through running idempotent recipes. This is our approach. The first
Re: [openstack-dev] [Heat] HOT Software configuration proposal
Hi Patric, Thank you for such great post! This is very close to the vision I've tried to propose earlier on software orchestration thread and I'm glad other people concern about the same issues. However the problem the problem with PaaS-like approached it that they currently on a little bit higher abstraction layer than Heat is intended to be. Typical Heat users are more of DevOps people rather than those who enjoy PaaS-related solutions. Going that direction would require some major paradigm shift for the Heat which I think is unnecessary. I believe there is a place in OpenStack software-orchestration ecosystem for layers that would sin on top of Heat and provide more high-level services for software composition, dependency management. Heat is not aimed to be software-everything. I would suggest you to take a look at Murano project as it is very very close to what you want to achieve and as every open-source project it needs community contributions. And I believe that it is the place in OpenStack ecosystem where your expirience would be most valuable and appreciated as well as your contributions On Wed, Oct 23, 2013 at 9:58 PM, Patrick Petit patrick.pe...@bull.netwrote: Dear Steve and All, If I may add up on this already busy thread to share our experience with using Heat in large and complex software deployments. I work on a project which precisely provides additional value at the articulation point between resource orchestration automation and configuration management. We rely on Heat and chef-solo respectively for these base management functions. On top of this, we have developed an event-driven workflow to manage the life-cycles of complex software stacks which primary purpose is to support middleware components as opposed to end-user apps. Our use cases are peculiar in the sense that software setup (install, config, contextualization) is not a one-time operation issue but a continuous thing that can happen any time in life-span of a stack. Users can deploy (and undeploy) apps long time after the stack is created. Auto-scaling may also result in an asynchronous apps deployment. More about this latter. The framework we have designed works well for us. It clearly refers to a PaaS-like environment which I understand is not the topic of the HOT software configuration proposal(s) and that's absolutely fine with us. However, the question for us is whether the separation of software config from resources would make our life easier or not. I think the answer is definitely yes but at the condition that the DSL extension preserves almost everything from the expressiveness of the resource element. In practice, I think that a strict separation between resource and component will be hard to achieve because we'll always need a little bit of application's specific in the resources. Take for example the case of the SecurityGroups. The ports open in a SecurityGroup are application specific. Then, designing a Chef or Puppet component type may be harder than it looks at first glance. Speaking of our use cases we still need a little bit of scripting in the instance's user-data block to setup a working chef-solo environment. For example, we run librarian-chef prior to starting chef-solo to resolve the cookbook dependencies. A cookbook can present itself as a downloadable tarball but it's not always the case. A chef component type would have to support getting a cookbook from a public or private git repo (maybe subversion), handle situations where there is one cookbook per repo or multiple cookbooks per repo, let the user choose a particular branch or label, provide ssh keys if it's a private repo, and so forth. We support all of this scenarios and so we can provide more detailed requirements if needed. I am not sure adding component relations like the 'depends-on' would really help us since it is the job of config management to handle software dependencies. Also, it doesn't address the issue of circular dependencies. Circular dependencies occur in complex software stack deployments. Example. When we setup a Slum virtual cluster, both the head node and compute nodes depend on one another to complete their configuration and so they would wait for each other indefinitely if we were to rely on the 'depends-on'. In addition, I think it's critical to distinguish between configuration parameters which are known ahead of time, like a db name or user name and password, versus contextualization parameters which are known after the fact generally when the instance is created. Typically those contextualization parameters are IP addresses but not only. The fact packages x,y,z have been properly installed and services a,b,c successfully started is contextualization information (a.k.a facts) which may be indicative that other components can move on to the next setup stage. The case of complex deployments with or without circular dependencies is typically resolved by
Re: [openstack-dev] [Heat] HOT Software configuration proposal
Hi, Looking through the thread I mentioned couple definitions of software orchestration. I would like to summarize definitions before we go to deep technical discussion about actual implementation. There are two major areas and approaches covered by software orchestration: *Software component installation* - aimed to install specific software component to VM and configure it. This is a typical task for Heat. HOT component is a best way to easily describe what component should be installed and what are configuration parameter. Heat engine will figure out by itself how to do this, probably with some hints from a user in terms of dependencies and placement rules. *Software service installation and life cycle management* - aimed to provision a complex multi component software service over multiple VMs. Also defines actions on specific events and manages software over the entire environment life. This approach is closer to PaaS like solution and relies on specific workflows sequences defined for different events and situations. Instead of defining what should be installed, this approach defines how to react on specific situation and what to do if some event has been triggered. This workflow approach is what is covered by Mistral project from the engine viewpoint. Mistral is going to orchestrate task execution in distributed fashion on both VM and OpenStack levels and it has events and schedule semantics. Actual workflow implementation for OpenStack may be found in Murano project which already defines different workflows for software installation and configuration depending on situation. As I see, both approaches have their own user, and both approaches can coexist in OpenStack ecosystem being complementary to each other. For example workflow can generate HOT template to do some task which fits best to Heat engine and in the same time HOT template can reference external workflow to do the task which fits best to workflow approach. Thanks Georgy On Wed, Oct 23, 2013 at 11:36 AM, Clint Byrum cl...@fewbar.com wrote: Excerpts from Patrick Petit's message of 2013-10-23 10:58:22 -0700: Dear Steve and All, If I may add up on this already busy thread to share our experience with using Heat in large and complex software deployments. Thanks for sharing Patrick, I have a few replies in-line. I work on a project which precisely provides additional value at the articulation point between resource orchestration automation and configuration management. We rely on Heat and chef-solo respectively for these base management functions. On top of this, we have developed an event-driven workflow to manage the life-cycles of complex software stacks which primary purpose is to support middleware components as opposed to end-user apps. Our use cases are peculiar in the sense that software setup (install, config, contextualization) is not a one-time operation issue but a continuous thing that can happen any time in life-span of a stack. Users can deploy (and undeploy) apps long time after the stack is created. Auto-scaling may also result in an asynchronous apps deployment. More about this latter. The framework we have designed works well for us. It clearly refers to a PaaS-like environment which I understand is not the topic of the HOT software configuration proposal(s) and that's absolutely fine with us. However, the question for us is whether the separation of software config from resources would make our life easier or not. I think the answer is definitely yes but at the condition that the DSL extension preserves almost everything from the expressiveness of the resource element. In practice, I think that a strict separation between resource and component will be hard to achieve because we'll always need a little bit of application's specific in the resources. Take for example the case of the SecurityGroups. The ports open in a SecurityGroup are application specific. Components can only be made up of the things that are common to all users of said component. Also components would, if I understand the concept correctly, just be for things that are at the sub-resource level. Security groups and open ports would be across multiple resources, and thus would be separately specified from your app's component (though it might be useful to allow components to export static values so that the port list can be referred to along with the app component). Then, designing a Chef or Puppet component type may be harder than it looks at first glance. Speaking of our use cases we still need a little bit of scripting in the instance's user-data block to setup a working chef-solo environment. For example, we run librarian-chef prior to starting chef-solo to resolve the cookbook dependencies. A cookbook can present itself as a downloadable tarball but it's not always the case. A chef component type would have to support getting a cookbook from a public
Re: [openstack-dev] [Heat] HOT Software configuration proposal
Steve Baker sba...@redhat.com wrote on 21.10.2013 23:02:47:
From: Steve Baker sba...@redhat.com
To: openstack-dev@lists.openstack.org,
Date: 21.10.2013 23:06
Subject: Re: [openstack-dev] [Heat] HOT Software configuration proposal
On 10/22/2013 08:45 AM, Mike Spreitzer wrote:
Steve Baker sba...@redhat.com wrote on 10/15/2013 06:48:53 PM:
I've just written some proposals to address Heat's HOT software
configuration needs, and I'd like to use this thread to get some
feedback:
https://wiki.openstack.org/wiki/Heat/Blueprints/hot-software-config
https://wiki.openstack.org/wiki/Heat/Blueprints/native-tools-
bootstrap-config
Please read the proposals and reply to the list with any comments or
suggestions.
Can you confirm whether I have got the big picture right? I think
some of my earlier remarks were mistaken.
You propose to introduce the concept of component and recognize
software configuration as a matter of invoking components --- with a
DAG of data dependencies among the component invocations. While
this is similar to what today's heat engine does for resources, you
do NOT propose that the heat engine will get in the business of
invoking components. Rather: each VM will run a series of component
invocations, and in-VM mechanisms will handle the cross-component
synchronization and data communication.
This is basically correct, except that in-VM mechanisms won't know
much about cross-component synchronization and data communication.
They will just execute whatever components are available to be
executed, and report back values to heat-engine by signalling to
waitconditions.
You propose to add a bit of sugaring for the wait conditionhandle
mechanism, and the heat engine will do the de-sugaring.
Yes, I think improvements can be made on what I proposed, such as
every component signalling when it is complete, and optionally
including a return value in that signal.
Being able to handle completion of single components inside a VM and being
able to pass outputs of those components seems important to me. I think
that should mostly address the requirements for declaring data dependencies
between components that have been discussed before in this thread. If
wait-conditions are the underlying mechanism, fine, as longs as we can hid
it from the template syntax.
For example, something like this should be possible:
components:
comp_a:
type: OS::Heat::SomeCMProvider
properties:
prop1: { get_param: param1 }
prop2: ...
comp_b:
type: OS::Heat::SomeCMProvider
properties:
propA: { get_attr: [ comp_a, some_attr ] }
resources:
serverA:
type: OS::Nova::Server
# ...
components:
- comp_a
serverB:
type: OS::Nova::Server
# ...
components:
- comp_b
I.e. there are two components comp_a and comp_b on two different servers.
comp_a has a data dependency on an attribute of comp_b. If we treat
'properties' as input to components and 'attributes' as output (the way it
is currently done for resources), that should be doable.
BTW, the convention of properties being input and attributes being output,
i.e. that subtle distinction between properties and attributes is not
really intuitive, at least not to me as non-native speaker, because I used
to use both words as synonyms.
Anyway it seems like the current proposal is a starting point with
enhancements on the roadmap, right?
Each component is written in one of a few supported configuration
management (CM) frameworks, and essentially all component
invocations on a given VM invoke components of the same CM framework
(with possible exceptions for one or two really basic ones).
Rather than being limited to a few supported CM tools, I like the
idea of some kind of provider mechanism so that users or heat admins
can add support for new CM tools. This implies that it needs to be
possible to add a component type without requiring custom python
that runs on heat engine.
The heat engine gains the additional responsibility of making sure
that the appropriate CM framework(s) is(are) bootstrapped in each VM.
Maybe. Or it might be up to the user to invoke images that already
have the CM tools installed, or the user can provide a custom
component provider which installs the tool in the way that they want.
As for the cross-component synchronization and data communication
question, at this stage I'm not comfortable with bringing something
like zookeeper into the mix for a general solution for inter-
component communication. If heat engine handles resource
dependencies and zookeeper handles software configuration
dependencies this would result in the state of the stack being split
between two different co-ordination mechanisms.
I think that zookeeper could be an _optional_ backend for this, but using
the current mechanisms in Heat should probably be the primary or default
way of doing this.
We've put quite some effort into heat engine to co-ordinate
Re: [openstack-dev] [Heat] HOT Software configuration proposal
On 22/10/13 09:15, Thomas Spatzier wrote: BTW, the convention of properties being input and attributes being output, i.e. that subtle distinction between properties and attributes is not really intuitive, at least not to me as non-native speaker, because I used to use both words as synonyms. As a native speaker, I can confidently state that it's not intuitive to anyone ;) We unfortunately inherited these names from the Properties section and the Fn::GetAtt function in cfn templates. It's even worse than that, because there's a whole category of... uh... things (DependsOn, DeletionPolicy, c.) that don't even have a name - I always have to resist the urge to call them 'attributes' too. - ZB ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Heat] HOT Software configuration proposal
Zane Bitter zbit...@redhat.com wrote on 10/22/2013 09:24:28 AM: On 22/10/13 09:15, Thomas Spatzier wrote: BTW, the convention of properties being input and attributes being output, i.e. that subtle distinction between properties and attributes is not really intuitive, at least not to me as non-native speaker, because I used to use both words as synonyms. As a native speaker, I can confidently state that it's not intuitive to anyone ;) We unfortunately inherited these names from the Properties section and the Fn::GetAtt function in cfn templates. It's even worse than that, because there's a whole category of... uh... things (DependsOn, DeletionPolicy, c.) that don't even have a name - I always have to resist the urge to call them 'attributes' too. At least for the components construct being proposed (by Steve Baker), shall we adopt a more explicit convention and require component definitions to explicitly name their inputs and outputs? Thanks, LN___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Heat] HOT Software configuration proposal
Zane Bitter zbit...@redhat.com wrote on 22.10.2013 15:24:28:
From: Zane Bitter zbit...@redhat.com
To: openstack-dev@lists.openstack.org,
Date: 22.10.2013 15:27
Subject: Re: [openstack-dev] [Heat] HOT Software configuration proposal
On 22/10/13 09:15, Thomas Spatzier wrote:
BTW, the convention of properties being input and attributes being
output,
i.e. that subtle distinction between properties and attributes is not
really intuitive, at least not to me as non-native speaker, because I
used
to use both words as synonyms.
As a native speaker, I can confidently state that it's not intuitive to
anyone ;)
Phew, good to read that ;-)
We unfortunately inherited these names from the Properties section and
the Fn::GetAtt function in cfn templates. It's even worse than that,
because there's a whole category of... uh... things (DependsOn,
DeletionPolicy, c.) that don't even have a name - I always have to
resist the urge to call them 'attributes' too.
So is this something we should try to get straight in HOT while we still
have the flexibility?
Regarding properties/attributes for example, to me I would call both just
properties of a resource or component, and then I can write them or read
them like:
components:
my_component:
type: ...
properties:
my_prop: { get_property: [ other_component, other_component_prop ] }
other_component:
# ...
I.e. you write property 'my_prop' of 'my_component' in its properties
section, and you read property 'other_component_prop' of 'other_component'
using the get_property function.
... we can also call them attributes, but use one name, not two different
names for the same thing.
- ZB
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Heat] HOT Software configuration proposal
On 22/10/13 16:35, Thomas Spatzier wrote:
Zane Bitter zbit...@redhat.com wrote on 22.10.2013 15:24:28:
From: Zane Bitter zbit...@redhat.com
To: openstack-dev@lists.openstack.org,
Date: 22.10.2013 15:27
Subject: Re: [openstack-dev] [Heat] HOT Software configuration proposal
On 22/10/13 09:15, Thomas Spatzier wrote:
BTW, the convention of properties being input and attributes being
output,
i.e. that subtle distinction between properties and attributes is not
really intuitive, at least not to me as non-native speaker, because I
used
to use both words as synonyms.
As a native speaker, I can confidently state that it's not intuitive to
anyone ;)
Phew, good to read that ;-)
We unfortunately inherited these names from the Properties section and
the Fn::GetAtt function in cfn templates. It's even worse than that,
because there's a whole category of... uh... things (DependsOn,
DeletionPolicy, c.) that don't even have a name - I always have to
resist the urge to call them 'attributes' too.
So is this something we should try to get straight in HOT while we still
have the flexibility?
Y-yes. Provided that we can do it without making things *more*
confusing, +1. That's hard though, because there are a number of places
we have to refer to them, all with different audiences:
- HOT users
- cfn users
- Existing developers
- New developers
- Plugin developers
and using different names for the same thing can cause problems. My test
for this is: if you were helping a user on IRC debug an issue, is there
a high chance you would spend 15 minutes talking past each other because
they misunderstand the terminology?
Regarding properties/attributes for example, to me I would call both just
properties of a resource or component, and then I can write them or read
them like:
components:
my_component:
type: ...
properties:
my_prop: { get_property: [ other_component, other_component_prop ] }
other_component:
# ...
I.e. you write property 'my_prop' of 'my_component' in its properties
section, and you read property 'other_component_prop' of 'other_component'
using the get_property function.
... we can also call them attributes, but use one name, not two different
names for the same thing.
IMO inputs (Properties) and outputs (Fn::GetAtt) are different things
(and they exist in different namespaces), so -1 for giving them the same
name.
In an ideal world I'd like HOT to use something like get_output_data (or
maybe just get_data), but OTOH we have e.g. FnGetAtt() and
attributes_schema baked in to the plugin API that we can't really
change, so it seems likely to lead to developers and users adopting
different terminology, or making things very difficult for new
developers, or both :(
cheers,
Zane.
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Heat] HOT Software configuration proposal
Zane Bitter zbit...@redhat.com wrote on 22.10.2013 17:23:52:
From: Zane Bitter zbit...@redhat.com
To: openstack-dev@lists.openstack.org,
Date: 22.10.2013 17:26
Subject: Re: [openstack-dev] [Heat] HOT Software configuration proposal
On 22/10/13 16:35, Thomas Spatzier wrote:
Zane Bitter zbit...@redhat.com wrote on 22.10.2013 15:24:28:
From: Zane Bitter zbit...@redhat.com
To: openstack-dev@lists.openstack.org,
Date: 22.10.2013 15:27
Subject: Re: [openstack-dev] [Heat] HOT Software configuration
proposal
On 22/10/13 09:15, Thomas Spatzier wrote:
BTW, the convention of properties being input and attributes being
output,
i.e. that subtle distinction between properties and attributes is not
really intuitive, at least not to me as non-native speaker, because I
used
to use both words as synonyms.
As a native speaker, I can confidently state that it's not intuitive
to
anyone ;)
Phew, good to read that ;-)
We unfortunately inherited these names from the Properties section and
the Fn::GetAtt function in cfn templates. It's even worse than that,
because there's a whole category of... uh... things (DependsOn,
DeletionPolicy, c.) that don't even have a name - I always have to
resist the urge to call them 'attributes' too.
So is this something we should try to get straight in HOT while we
still
have the flexibility?
Y-yes. Provided that we can do it without making things *more*
confusing, +1. That's hard though, because there are a number of places
we have to refer to them, all with different audiences:
- HOT users
- cfn users
- Existing developers
- New developers
- Plugin developers
and using different names for the same thing can cause problems. My test
for this is: if you were helping a user on IRC debug an issue, is there
a high chance you would spend 15 minutes talking past each other because
they misunderstand the terminology?
Hm, good point. Seems like it would really cause more confusion than it
helps. So back away from the general idea of renaming things that exist
both in cfn and HOT.
What we should try of course is to give new concepts that will only exist
in HOT intuitive names.
Regarding properties/attributes for example, to me I would call both
just
properties of a resource or component, and then I can write them or
read
them like:
components:
my_component:
type: ...
properties:
my_prop: { get_property: [ other_component,
other_component_prop ] }
other_component:
# ...
I.e. you write property 'my_prop' of 'my_component' in its properties
section, and you read property 'other_component_prop' of
'other_component'
using the get_property function.
... we can also call them attributes, but use one name, not two
different
names for the same thing.
IMO inputs (Properties) and outputs (Fn::GetAtt) are different things
(and they exist in different namespaces), so -1 for giving them the same
name.
In an ideal world I'd like HOT to use something like get_output_data (or
maybe just get_data), but OTOH we have e.g. FnGetAtt() and
attributes_schema baked in to the plugin API that we can't really
change, so it seems likely to lead to developers and users adopting
different terminology, or making things very difficult for new
developers, or both :(
cheers,
Zane.
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Heat] HOT Software configuration proposal
Hello,
I've been reading through the thread and the wiki pages and I'm still
confused by the terms. Is there a clear definition of what do we understand
by component from user's and developer's point of view. If I write
component, type:MySQL what is behind that definition? I mean how does the
system know what exactly MySQL is and how to install it? What MySQL version
is it gonna be? Will it be x86 or x64? How does the system understand that
I need MySQL for Windows on Windows VM rather then Linux MySQL? What do I
as a developer need to do so that it would be possible to have type:
MyCoolComponentType?
On Tue, Oct 22, 2013 at 8:35 PM, Thomas Spatzier thomas.spatz...@de.ibm.com
wrote:
Zane Bitter zbit...@redhat.com wrote on 22.10.2013 17:23:52:
From: Zane Bitter zbit...@redhat.com
To: openstack-dev@lists.openstack.org,
Date: 22.10.2013 17:26
Subject: Re: [openstack-dev] [Heat] HOT Software configuration proposal
On 22/10/13 16:35, Thomas Spatzier wrote:
Zane Bitter zbit...@redhat.com wrote on 22.10.2013 15:24:28:
From: Zane Bitter zbit...@redhat.com
To: openstack-dev@lists.openstack.org,
Date: 22.10.2013 15:27
Subject: Re: [openstack-dev] [Heat] HOT Software configuration
proposal
On 22/10/13 09:15, Thomas Spatzier wrote:
BTW, the convention of properties being input and attributes being
output,
i.e. that subtle distinction between properties and attributes is not
really intuitive, at least not to me as non-native speaker, because I
used
to use both words as synonyms.
As a native speaker, I can confidently state that it's not intuitive
to
anyone ;)
Phew, good to read that ;-)
We unfortunately inherited these names from the Properties section and
the Fn::GetAtt function in cfn templates. It's even worse than that,
because there's a whole category of... uh... things (DependsOn,
DeletionPolicy, c.) that don't even have a name - I always have to
resist the urge to call them 'attributes' too.
So is this something we should try to get straight in HOT while we
still
have the flexibility?
Y-yes. Provided that we can do it without making things *more*
confusing, +1. That's hard though, because there are a number of places
we have to refer to them, all with different audiences:
- HOT users
- cfn users
- Existing developers
- New developers
- Plugin developers
and using different names for the same thing can cause problems. My test
for this is: if you were helping a user on IRC debug an issue, is there
a high chance you would spend 15 minutes talking past each other because
they misunderstand the terminology?
Hm, good point. Seems like it would really cause more confusion than it
helps. So back away from the general idea of renaming things that exist
both in cfn and HOT.
What we should try of course is to give new concepts that will only exist
in HOT intuitive names.
Regarding properties/attributes for example, to me I would call both
just
properties of a resource or component, and then I can write them or
read
them like:
components:
my_component:
type: ...
properties:
my_prop: { get_property: [ other_component,
other_component_prop ] }
other_component:
# ...
I.e. you write property 'my_prop' of 'my_component' in its properties
section, and you read property 'other_component_prop' of
'other_component'
using the get_property function.
... we can also call them attributes, but use one name, not two
different
names for the same thing.
IMO inputs (Properties) and outputs (Fn::GetAtt) are different things
(and they exist in different namespaces), so -1 for giving them the same
name.
In an ideal world I'd like HOT to use something like get_output_data (or
maybe just get_data), but OTOH we have e.g. FnGetAtt() and
attributes_schema baked in to the plugin API that we can't really
change, so it seems likely to lead to developers and users adopting
different terminology, or making things very difficult for new
developers, or both :(
cheers,
Zane.
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
--
Sincerely yours
Stanislav (Stan) Lagun
Senior Developer
Mirantis
35b/3, Vorontsovskaya St.
Moscow, Russia
Skype: stanlagun
www.mirantis.com
sla...@mirantis.com
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Heat] HOT Software configuration proposal
Hi,
I would agree with Stan that we need to discuss definitions before going
deeply to the implementation.
The first example on the
https://wiki.openstack.org/wiki/Heat/Blueprints/hot-software-config shows
components like install_mysql and install_wordpress.
I would say that this is a bit confusing because I expected to see
component definitions instead of software deployment definition process. I
think this is a quite dangerous path here because this example shows us
that we can use components as installation steps definition instead of real
component definition.
If one continue to do this approach and defines more and more granular
steps as a components they will come to workflow definition composed in
terms of components. This approach does not add either simplicity or
clarity in the HOT template.
Thanks
Georgy
On Tue, Oct 22, 2013 at 10:02 AM, Stan Lagun sla...@mirantis.com wrote:
Hello,
I've been reading through the thread and the wiki pages and I'm still
confused by the terms. Is there a clear definition of what do we understand
by component from user's and developer's point of view. If I write
component, type:MySQL what is behind that definition? I mean how does the
system know what exactly MySQL is and how to install it? What MySQL version
is it gonna be? Will it be x86 or x64? How does the system understand that
I need MySQL for Windows on Windows VM rather then Linux MySQL? What do I
as a developer need to do so that it would be possible to have type:
MyCoolComponentType?
On Tue, Oct 22, 2013 at 8:35 PM, Thomas Spatzier
thomas.spatz...@de.ibm.com wrote:
Zane Bitter zbit...@redhat.com wrote on 22.10.2013 17:23:52:
From: Zane Bitter zbit...@redhat.com
To: openstack-dev@lists.openstack.org,
Date: 22.10.2013 17:26
Subject: Re: [openstack-dev] [Heat] HOT Software configuration proposal
On 22/10/13 16:35, Thomas Spatzier wrote:
Zane Bitter zbit...@redhat.com wrote on 22.10.2013 15:24:28:
From: Zane Bitter zbit...@redhat.com
To: openstack-dev@lists.openstack.org,
Date: 22.10.2013 15:27
Subject: Re: [openstack-dev] [Heat] HOT Software configuration
proposal
On 22/10/13 09:15, Thomas Spatzier wrote:
BTW, the convention of properties being input and attributes being
output,
i.e. that subtle distinction between properties and attributes is
not
really intuitive, at least not to me as non-native speaker, because
I
used
to use both words as synonyms.
As a native speaker, I can confidently state that it's not intuitive
to
anyone ;)
Phew, good to read that ;-)
We unfortunately inherited these names from the Properties section
and
the Fn::GetAtt function in cfn templates. It's even worse than that,
because there's a whole category of... uh... things (DependsOn,
DeletionPolicy, c.) that don't even have a name - I always have to
resist the urge to call them 'attributes' too.
So is this something we should try to get straight in HOT while we
still
have the flexibility?
Y-yes. Provided that we can do it without making things *more*
confusing, +1. That's hard though, because there are a number of places
we have to refer to them, all with different audiences:
- HOT users
- cfn users
- Existing developers
- New developers
- Plugin developers
and using different names for the same thing can cause problems. My test
for this is: if you were helping a user on IRC debug an issue, is there
a high chance you would spend 15 minutes talking past each other because
they misunderstand the terminology?
Hm, good point. Seems like it would really cause more confusion than it
helps. So back away from the general idea of renaming things that exist
both in cfn and HOT.
What we should try of course is to give new concepts that will only exist
in HOT intuitive names.
Regarding properties/attributes for example, to me I would call both
just
properties of a resource or component, and then I can write them or
read
them like:
components:
my_component:
type: ...
properties:
my_prop: { get_property: [ other_component,
other_component_prop ] }
other_component:
# ...
I.e. you write property 'my_prop' of 'my_component' in its properties
section, and you read property 'other_component_prop' of
'other_component'
using the get_property function.
... we can also call them attributes, but use one name, not two
different
names for the same thing.
IMO inputs (Properties) and outputs (Fn::GetAtt) are different things
(and they exist in different namespaces), so -1 for giving them the same
name.
In an ideal world I'd like HOT to use something like get_output_data (or
maybe just get_data), but OTOH we have e.g. FnGetAtt() and
attributes_schema baked in to the plugin API that we can't really
change, so it seems likely to lead to developers and users adopting
different terminology
Re: [openstack-dev] [Heat] HOT Software configuration proposal
Stan Lagun sla...@mirantis.com wrote on 22.10.2013 19:02:38:
From: Stan Lagun sla...@mirantis.com
To: OpenStack Development Mailing List
openstack-dev@lists.openstack.org,
Date: 22.10.2013 19:06
Subject: Re: [openstack-dev] [Heat] HOT Software configuration proposal
Hello,
I've been reading through the thread and the wiki pages and I'm
still confused by the terms. Is there a clear definition of what do
we understand by component from user's and developer's point of
view. If I write component, type:MySQL what is behind that
definition? I mean how does the system know what exactly MySQL is
I think the current proposal is not that Heat would support very specific
component types (like MySQL, Apache, Tomcat etc.) but component is more of
a generic construct to represent a piece of software. The type attribute of
a component then just calls out the config management tool (e.g. Chef) to
install and configure that piece of software. By pointing a component to,
say, a Chef cookbook for setting up MySQL the runtime type would be MySQL.
That is at least my view on this.
I agree, however, that it needs to be straightened out how the term
component is really used.
and how to install it? What MySQL version is it gonna be? Will it be
x86 or x64? How does the system understand that I need MySQL for
Windows on Windows VM rather then Linux MySQL? What do I as a
developer need to do so that it would be possible to have type:
MyCoolComponentType?
On Tue, Oct 22, 2013 at 8:35 PM, Thomas Spatzier
thomas.spatz...@de.ibm.com
wrote:
Zane Bitter zbit...@redhat.com wrote on 22.10.2013 17:23:52:
From: Zane Bitter zbit...@redhat.com
To: openstack-dev@lists.openstack.org,
Date: 22.10.2013 17:26
Subject: Re: [openstack-dev] [Heat] HOT Software configuration proposal
On 22/10/13 16:35, Thomas Spatzier wrote:
Zane Bitter zbit...@redhat.com wrote on 22.10.2013 15:24:28:
From: Zane Bitter zbit...@redhat.com
To: openstack-dev@lists.openstack.org,
Date: 22.10.2013 15:27
Subject: Re: [openstack-dev] [Heat] HOT Software configuration
proposal
On 22/10/13 09:15, Thomas Spatzier wrote:
BTW, the convention of properties being input and attributes being
output,
i.e. that subtle distinction between properties and attributes is
not
really intuitive, at least not to me as non-native speaker, because
I
used
to use both words as synonyms.
As a native speaker, I can confidently state that it's not intuitive
to
anyone ;)
Phew, good to read that ;-)
We unfortunately inherited these names from the Properties section
and
the Fn::GetAtt function in cfn templates. It's even worse than that,
because there's a whole category of... uh... things (DependsOn,
DeletionPolicy, c.) that don't even have a name - I always have to
resist the urge to call them 'attributes' too.
So is this something we should try to get straight in HOT while we
still
have the flexibility?
Y-yes. Provided that we can do it without making things *more*
confusing, +1. That's hard though, because there are a number of places
we have to refer to them, all with different audiences:
 - HOT users
 - cfn users
 - Existing developers
 - New developers
 - Plugin developers
and using different names for the same thing can cause problems. My
test
for this is: if you were helping a user on IRC debug an issue, is there
a high chance you would spend 15 minutes talking past each other
because
they misunderstand the terminology?
Hm, good point. Seems like it would really cause more confusion than it
helps. So back away from the general idea of renaming things that exist
both in cfn and HOT.
What we should try of course is to give new concepts that will only exist
in HOT intuitive names.
Regarding properties/attributes for example, to me I would call both
just
properties of a resource or component, and then I can write them or
read
them like:
components:
  my_component:
   type: ...
   properties:
    my_prop: { get_property: [ other_component,
other_component_prop ] }
  other_component:
   # ...
I.e. you write property 'my_prop' of 'my_component' in its properties
section, and you read property 'other_component_prop' of
'other_component'
using the get_property function.
... we can also call them attributes, but use one name, not two
different
names for the same thing.
IMO inputs (Properties) and outputs (Fn::GetAtt) are different things
(and they exist in different namespaces), so -1 for giving them the
same
name.
In an ideal world I'd like HOT to use something like get_output_data
(or
maybe just get_data), but OTOH we have e.g. FnGetAtt() and
attributes_schema baked in to the plugin API that we can't really
change, so it seems likely to lead to developers and users adopting
different terminology, or making things very difficult for new
developers
Re: [openstack-dev] [Heat] HOT Software configuration proposal
Hi Thomas, I agree with you on semantics part. At the same time I see a potential question which might appear - if semantics is limited by few states visible for Heat engine, then who actually does software orchestration? Will it be reasonable then to have software orchestration as separate subproject for Heat as a part of Orchestration OpenStack program? Heat engine will then do dependency tracking and will use components as a reference for software orchestration engine which will perform actual deployment and high level software components coordination. This separated software orchestration engine may address all specific requirements proposed by different teams in this thread without affecting existing Heat engine. Thanks Georgy On Tue, Oct 22, 2013 at 12:06 PM, Thomas Spatzier thomas.spatz...@de.ibm.com wrote: Georgy Okrokvertskhov gokrokvertsk...@mirantis.com wrote on 22.10.2013 20:01:19: From: Georgy Okrokvertskhov gokrokvertsk...@mirantis.com To: OpenStack Development Mailing List openstack-dev@lists.openstack.org, Date: 22.10.2013 20:05 Subject: Re: [openstack-dev] [Heat] HOT Software configuration proposal Hi, I would agree with Stan that we need to discuss definitions before going deeply to the implementation. The first example on the https://wiki.openstack.org/wiki/Heat/ Blueprints/hot-software-config shows components like install_mysql and install_wordpress. Good point. I also think that at least the examples currently used are more of an automation step than a component. IMO component should represent some kind of software installation (e.g. a web server, a DBMS, etc.), where automation is used under the covers to install and configure that piece of software. From an orchestration point of view, a reasonable semantics would be that when a component is in state CREATE_COMPLETE it is ready to use, e.g. a web server is ready to serve applications. With respect to the automation that was used to bring up the component, it would return successful (and this would be signaled to Heat) when the component setup is done. For example, the following could represent an Apache web server component, installed using Chef: components: apache: type: OS::Heat::SoftwareConfig_Chef cookbook: http://www.example.com/my_apache_cookbook.zip properties: http_port: 8080 'apache' is just a name here that indicates of course what you get. The type indicates that a component provide able to invoke Chef automation is used. The cookbook attribute points to the cookbook to use, which will install and configure apache. By setting the http_port property to 8080, you provide input to the Chef cookbook. The SoftwareConfig_Chef component provider will implement the logic to pass properties to the Chef invocation in the right syntax. I would say that this is a bit confusing because I expected to see component definitions instead of software deployment definition process. I think this is a quite dangerous path here because this example shows us that we can use components as installation steps definition instead of real component definition. If one continue to do this approach and defines more and more granular steps as a components they will come to workflow definition composed in terms of components. This approach does not add either simplicity or clarity in the HOT template. Thanks Georgy On Tue, Oct 22, 2013 at 10:02 AM, Stan Lagun sla...@mirantis.com wrote: Hello, I've been reading through the thread and the wiki pages and I'm still confused by the terms. Is there a clear definition of what do we understand by component from user's and developer's point of view. If I write component, type:MySQL what is behind that definition? I mean how does the system know what exactly MySQL is and how to install it? What MySQL version is it gonna be? Will it be x86 or x64? How does the system understand that I need MySQL for Windows on Windows VM rather then Linux MySQL? What do I as a developer need to do so that it would be possible to have type: MyCoolComponentType? On Tue, Oct 22, 2013 at 8:35 PM, Thomas Spatzier thomas.spatz...@de.ibm.com wrote: Zane Bitter zbit...@redhat.com wrote on 22.10.2013 17:23:52: From: Zane Bitter zbit...@redhat.com To: openstack-dev@lists.openstack.org, Date: 22.10.2013 17:26 Subject: Re: [openstack-dev] [Heat] HOT Software configuration proposal On 22/10/13 16:35, Thomas Spatzier wrote: Zane Bitter zbit...@redhat.com wrote on 22.10.2013 15:24:28: From: Zane Bitter zbit...@redhat.com To: openstack-dev@lists.openstack.org, Date: 22.10.2013 15:27 Subject: Re: [openstack-dev] [Heat] HOT Software configuration proposal On 22/10/13 09:15, Thomas Spatzier wrote: BTW, the convention of properties being input and attributes being output, i.e. that subtle distinction between properties and attributes
Re: [openstack-dev] [Heat] HOT Software configuration proposal
Excerpts from Georgy Okrokvertskhov's message of 2013-10-22 13:32:40 -0700:
Hi Thomas,
I agree with you on semantics part. At the same time I see a potential
question which might appear - if semantics is limited by few states visible
for Heat engine, then who actually does software orchestration?
Will it be reasonable then to have software orchestration as separate
subproject for Heat as a part of Orchestration OpenStack program? Heat
engine will then do dependency tracking and will use components as a
reference for software orchestration engine which will perform actual
deployment and high level software components coordination.
This separated software orchestration engine may address all specific
requirements proposed by different teams in this thread without affecting
existing Heat engine.
I'm not sure I know what software orchestration is, but I will take a
stab at a succinct definition:
Coordination of software configuration across multiple hosts.
If that is what you mean, then I believe what you actually want is
workflow. And for that, we have the Mistral project which was recently
announced [1].
Use that and you will simply need to define your desired workflow and
feed it into Mistral using a Mistral Heat resource. We can create a
nice bootstrapping resource for Heat instances that shims the mistral
workflow execution agent into machines (or lets us use one already there
via custom images).
I can imagine it working something like this:
resources:
mistral_workflow_handle:
type: OS::Mistral::WorkflowHandle
web_server:
type: OS::Nova::Server
components:
mistral_agent:
component_type: mistral
params:
workflow_: {ref: mistral_workflow_handle}
mysql_server:
type: OS::Nova::Server
components:
mistral_agent:
component_type: mistral
params:
workflow_handle: {ref: mistral_workflow_handle}
mistral_workflow:
type: OS::Mistral::Workflow
properties:
handle: {ref: mistral_workflow_handle}
workflow_reference: mysql_webapp_workflow
params:
mysql_server: {ref: mysql_server}
webserver: {ref: web_server}
And then the workflow is just defined outside of the Heat template (ok
I'm sure somebody will want to embed it, but I prefer stronger
separation). Something like this gets uploaded as
mysql_webapp_workflow:
[ 'step1': 'install_stuff',
'step2': 'wait(step1)',
'step3': 'allocate_sql_user(server=%mysql_server%)'
'step4': 'credentials=wait_and_read(step3)'
'step5': 'write_config_file(server=%webserver%)' ]
Or maybe it is declared as a graph, or whatever, but it is not Heat's
problem how to do workflows, it just feeds the necessary data from
orchestration into the workflow engine. This also means you can use a
non OpenStack workflow engine without any problems.
I think after having talked about this, we should have workflow live in
its own program.. we can always combine them if we want to, but having a
clear line would mean keeping the interfaces clean.
[1] http://lists.openstack.org/pipermail/openstack-dev/2013-October/016605.html
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Heat] HOT Software configuration proposal
Sounds like taskflow could be that program (+1 from me, ha)?
Mistral to me is a nice authenticated REST api + other goodies ontop of
something that reliably executes workflows.
But then what I described is also the majority of what openstack does
(authenticated REST api + other goodies ontop of VM/volume/network/...
workflows).
On 10/22/13 3:28 PM, Clint Byrum cl...@fewbar.com wrote:
Excerpts from Georgy Okrokvertskhov's message of 2013-10-22 13:32:40
-0700:
Hi Thomas,
I agree with you on semantics part. At the same time I see a potential
question which might appear - if semantics is limited by few states
visible
for Heat engine, then who actually does software orchestration?
Will it be reasonable then to have software orchestration as separate
subproject for Heat as a part of Orchestration OpenStack program? Heat
engine will then do dependency tracking and will use components as a
reference for software orchestration engine which will perform actual
deployment and high level software components coordination.
This separated software orchestration engine may address all specific
requirements proposed by different teams in this thread without
affecting
existing Heat engine.
I'm not sure I know what software orchestration is, but I will take a
stab at a succinct definition:
Coordination of software configuration across multiple hosts.
If that is what you mean, then I believe what you actually want is
workflow. And for that, we have the Mistral project which was recently
announced [1].
Use that and you will simply need to define your desired workflow and
feed it into Mistral using a Mistral Heat resource. We can create a
nice bootstrapping resource for Heat instances that shims the mistral
workflow execution agent into machines (or lets us use one already there
via custom images).
I can imagine it working something like this:
resources:
mistral_workflow_handle:
type: OS::Mistral::WorkflowHandle
web_server:
type: OS::Nova::Server
components:
mistral_agent:
component_type: mistral
params:
workflow_: {ref: mistral_workflow_handle}
mysql_server:
type: OS::Nova::Server
components:
mistral_agent:
component_type: mistral
params:
workflow_handle: {ref: mistral_workflow_handle}
mistral_workflow:
type: OS::Mistral::Workflow
properties:
handle: {ref: mistral_workflow_handle}
workflow_reference: mysql_webapp_workflow
params:
mysql_server: {ref: mysql_server}
webserver: {ref: web_server}
And then the workflow is just defined outside of the Heat template (ok
I'm sure somebody will want to embed it, but I prefer stronger
separation). Something like this gets uploaded as
mysql_webapp_workflow:
[ 'step1': 'install_stuff',
'step2': 'wait(step1)',
'step3': 'allocate_sql_user(server=%mysql_server%)'
'step4': 'credentials=wait_and_read(step3)'
'step5': 'write_config_file(server=%webserver%)' ]
Or maybe it is declared as a graph, or whatever, but it is not Heat's
problem how to do workflows, it just feeds the necessary data from
orchestration into the workflow engine. This also means you can use a
non OpenStack workflow engine without any problems.
I think after having talked about this, we should have workflow live in
its own program.. we can always combine them if we want to, but having a
clear line would mean keeping the interfaces clean.
[1]
http://lists.openstack.org/pipermail/openstack-dev/2013-October/016605.htm
l
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Heat] HOT Software configuration proposal
Hi Clint,
Thank you for the detailed analysis.
I'm not sure I know what software orchestration is, but I will take a
stab at a succinct definition:
Coordination of software configuration across multiple hosts.
Having this definition of software orchestration what will Heat
software orchestration component BP cover? I just trying to clarify for
myself what is Heat position and view on software orchestration based on
components and Heat view on workflows.
Right now it is not clear where is the separation line between component
and workflow. I think this blurred line introduced a lot of confusion in
this thread as some guys had a workflows based approach in mind and some
had component based view.
Thanks
Georgy
On Tue, Oct 22, 2013 at 3:28 PM, Clint Byrum cl...@fewbar.com wrote:
Excerpts from Georgy Okrokvertskhov's message of 2013-10-22 13:32:40 -0700:
Hi Thomas,
I agree with you on semantics part. At the same time I see a potential
question which might appear - if semantics is limited by few states
visible
for Heat engine, then who actually does software orchestration?
Will it be reasonable then to have software orchestration as separate
subproject for Heat as a part of Orchestration OpenStack program? Heat
engine will then do dependency tracking and will use components as a
reference for software orchestration engine which will perform actual
deployment and high level software components coordination.
This separated software orchestration engine may address all specific
requirements proposed by different teams in this thread without affecting
existing Heat engine.
I'm not sure I know what software orchestration is, but I will take a
stab at a succinct definition:
Coordination of software configuration across multiple hosts.
If that is what you mean, then I believe what you actually want is
workflow. And for that, we have the Mistral project which was recently
announced [1].
Use that and you will simply need to define your desired workflow and
feed it into Mistral using a Mistral Heat resource. We can create a
nice bootstrapping resource for Heat instances that shims the mistral
workflow execution agent into machines (or lets us use one already there
via custom images).
I can imagine it working something like this:
resources:
mistral_workflow_handle:
type: OS::Mistral::WorkflowHandle
web_server:
type: OS::Nova::Server
components:
mistral_agent:
component_type: mistral
params:
workflow_: {ref: mistral_workflow_handle}
mysql_server:
type: OS::Nova::Server
components:
mistral_agent:
component_type: mistral
params:
workflow_handle: {ref: mistral_workflow_handle}
mistral_workflow:
type: OS::Mistral::Workflow
properties:
handle: {ref: mistral_workflow_handle}
workflow_reference: mysql_webapp_workflow
params:
mysql_server: {ref: mysql_server}
webserver: {ref: web_server}
And then the workflow is just defined outside of the Heat template (ok
I'm sure somebody will want to embed it, but I prefer stronger
separation). Something like this gets uploaded as
mysql_webapp_workflow:
[ 'step1': 'install_stuff',
'step2': 'wait(step1)',
'step3': 'allocate_sql_user(server=%mysql_server%)'
'step4': 'credentials=wait_and_read(step3)'
'step5': 'write_config_file(server=%webserver%)' ]
Or maybe it is declared as a graph, or whatever, but it is not Heat's
problem how to do workflows, it just feeds the necessary data from
orchestration into the workflow engine. This also means you can use a
non OpenStack workflow engine without any problems.
I think after having talked about this, we should have workflow live in
its own program.. we can always combine them if we want to, but having a
clear line would mean keeping the interfaces clean.
[1]
http://lists.openstack.org/pipermail/openstack-dev/2013-October/016605.html
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
--
Georgy Okrokvertskhov
Technical Program Manager,
Cloud and Infrastructure Services,
Mirantis
http://www.mirantis.com
Tel. +1 650 963 9828
Mob. +1 650 996 3284
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Heat] HOT Software configuration proposal
Hi Joshua,
Sounds like taskflow could be that program (+1 from me, ha)?
Mistral to me is a nice authenticated REST api + other goodies ontop of
something that reliably executes workflows.
I would say that Mistral is a way to do this. My arguments are the
following:
1. Mistral decouples code. Heat can use API calls to invoke a workflows
execution instead of linking with taskflow library in the code. This is
standard SOA approach which OpenStack uses a lot.
2. Mistral will expose DSL to define tasks while taskflow will require
python code for task definition.
Mistral itself uses taskflow library to execute workflows but Mistral in
addition does parsing and translation from DSL task definition to actual
python code.
Heat can use taskflow for other purposes but workflows execution is not a
good reason for that. Just because of nature of workflows for deployment,
there is no knowledge about workflow until end user uploads it, so you can
not use taskflow itself and code this workflow in python without
preliminary knowledge about workflow.
If Heat uses just taskflow it should do all the work on workflow parsing
and translation to a code that Heat team wants to avoid. At least this is
my understanding.
Thanks
Georgy
On Tue, Oct 22, 2013 at 4:34 PM, Joshua Harlow harlo...@yahoo-inc.comwrote:
Sounds like taskflow could be that program (+1 from me, ha)?
Mistral to me is a nice authenticated REST api + other goodies ontop of
something that reliably executes workflows.
But then what I described is also the majority of what openstack does
(authenticated REST api + other goodies ontop of VM/volume/network/...
workflows).
On 10/22/13 3:28 PM, Clint Byrum cl...@fewbar.com wrote:
Excerpts from Georgy Okrokvertskhov's message of 2013-10-22 13:32:40
-0700:
Hi Thomas,
I agree with you on semantics part. At the same time I see a potential
question which might appear - if semantics is limited by few states
visible
for Heat engine, then who actually does software orchestration?
Will it be reasonable then to have software orchestration as separate
subproject for Heat as a part of Orchestration OpenStack program? Heat
engine will then do dependency tracking and will use components as a
reference for software orchestration engine which will perform actual
deployment and high level software components coordination.
This separated software orchestration engine may address all specific
requirements proposed by different teams in this thread without
affecting
existing Heat engine.
I'm not sure I know what software orchestration is, but I will take a
stab at a succinct definition:
Coordination of software configuration across multiple hosts.
If that is what you mean, then I believe what you actually want is
workflow. And for that, we have the Mistral project which was recently
announced [1].
Use that and you will simply need to define your desired workflow and
feed it into Mistral using a Mistral Heat resource. We can create a
nice bootstrapping resource for Heat instances that shims the mistral
workflow execution agent into machines (or lets us use one already there
via custom images).
I can imagine it working something like this:
resources:
mistral_workflow_handle:
type: OS::Mistral::WorkflowHandle
web_server:
type: OS::Nova::Server
components:
mistral_agent:
component_type: mistral
params:
workflow_: {ref: mistral_workflow_handle}
mysql_server:
type: OS::Nova::Server
components:
mistral_agent:
component_type: mistral
params:
workflow_handle: {ref: mistral_workflow_handle}
mistral_workflow:
type: OS::Mistral::Workflow
properties:
handle: {ref: mistral_workflow_handle}
workflow_reference: mysql_webapp_workflow
params:
mysql_server: {ref: mysql_server}
webserver: {ref: web_server}
And then the workflow is just defined outside of the Heat template (ok
I'm sure somebody will want to embed it, but I prefer stronger
separation). Something like this gets uploaded as
mysql_webapp_workflow:
[ 'step1': 'install_stuff',
'step2': 'wait(step1)',
'step3': 'allocate_sql_user(server=%mysql_server%)'
'step4': 'credentials=wait_and_read(step3)'
'step5': 'write_config_file(server=%webserver%)' ]
Or maybe it is declared as a graph, or whatever, but it is not Heat's
problem how to do workflows, it just feeds the necessary data from
orchestration into the workflow engine. This also means you can use a
non OpenStack workflow engine without any problems.
I think after having talked about this, we should have workflow live in
its own program.. we can always combine them if we want to, but having a
clear line would mean keeping the interfaces clean.
[1]
http://lists.openstack.org/pipermail/openstack-dev/2013-October/016605.htm
l
___
Re: [openstack-dev] [Heat] HOT Software configuration proposal
Ah,
Seems like a reasonable approach then :-)
I guess then heat is mainly doing top-level orchestration, and then mistral
does the workflow middle-level, and taskflow is (hopefully) at the
lowest-level??
Thanks Georgy!
From: Georgy Okrokvertskhov
gokrokvertsk...@mirantis.commailto:gokrokvertsk...@mirantis.com
Reply-To: OpenStack Development Mailing List
openstack-dev@lists.openstack.orgmailto:openstack-dev@lists.openstack.org
Date: Tuesday, October 22, 2013 4:53 PM
To: OpenStack Development Mailing List
openstack-dev@lists.openstack.orgmailto:openstack-dev@lists.openstack.org
Subject: Re: [openstack-dev] [Heat] HOT Software configuration proposal
Hi Joshua,
Sounds like taskflow could be that program (+1 from me, ha)?
Mistral to me is a nice authenticated REST api + other goodies ontop of
something that reliably executes workflows.
I would say that Mistral is a way to do this. My arguments are the following:
1. Mistral decouples code. Heat can use API calls to invoke a workflows
execution instead of linking with taskflow library in the code. This is
standard SOA approach which OpenStack uses a lot.
2. Mistral will expose DSL to define tasks while taskflow will require python
code for task definition.
Mistral itself uses taskflow library to execute workflows but Mistral in
addition does parsing and translation from DSL task definition to actual python
code.
Heat can use taskflow for other purposes but workflows execution is not a good
reason for that. Just because of nature of workflows for deployment, there is
no knowledge about workflow until end user uploads it, so you can not use
taskflow itself and code this workflow in python without preliminary knowledge
about workflow.
If Heat uses just taskflow it should do all the work on workflow parsing and
translation to a code that Heat team wants to avoid. At least this is my
understanding.
Thanks
Georgy
On Tue, Oct 22, 2013 at 4:34 PM, Joshua Harlow
harlo...@yahoo-inc.commailto:harlo...@yahoo-inc.com wrote:
Sounds like taskflow could be that program (+1 from me, ha)?
Mistral to me is a nice authenticated REST api + other goodies ontop of
something that reliably executes workflows.
But then what I described is also the majority of what openstack does
(authenticated REST api + other goodies ontop of VM/volume/network/...
workflows).
On 10/22/13 3:28 PM, Clint Byrum cl...@fewbar.commailto:cl...@fewbar.com
wrote:
Excerpts from Georgy Okrokvertskhov's message of 2013-10-22 13:32:40
-0700:
Hi Thomas,
I agree with you on semantics part. At the same time I see a potential
question which might appear - if semantics is limited by few states
visible
for Heat engine, then who actually does software orchestration?
Will it be reasonable then to have software orchestration as separate
subproject for Heat as a part of Orchestration OpenStack program? Heat
engine will then do dependency tracking and will use components as a
reference for software orchestration engine which will perform actual
deployment and high level software components coordination.
This separated software orchestration engine may address all specific
requirements proposed by different teams in this thread without
affecting
existing Heat engine.
I'm not sure I know what software orchestration is, but I will take a
stab at a succinct definition:
Coordination of software configuration across multiple hosts.
If that is what you mean, then I believe what you actually want is
workflow. And for that, we have the Mistral project which was recently
announced [1].
Use that and you will simply need to define your desired workflow and
feed it into Mistral using a Mistral Heat resource. We can create a
nice bootstrapping resource for Heat instances that shims the mistral
workflow execution agent into machines (or lets us use one already there
via custom images).
I can imagine it working something like this:
resources:
mistral_workflow_handle:
type: OS::Mistral::WorkflowHandle
web_server:
type: OS::Nova::Server
components:
mistral_agent:
component_type: mistral
params:
workflow_: {ref: mistral_workflow_handle}
mysql_server:
type: OS::Nova::Server
components:
mistral_agent:
component_type: mistral
params:
workflow_handle: {ref: mistral_workflow_handle}
mistral_workflow:
type: OS::Mistral::Workflow
properties:
handle: {ref: mistral_workflow_handle}
workflow_reference: mysql_webapp_workflow
params:
mysql_server: {ref: mysql_server}
webserver: {ref: web_server}
And then the workflow is just defined outside of the Heat template (ok
I'm sure somebody will want to embed it, but I prefer stronger
separation). Something like this gets uploaded as
mysql_webapp_workflow:
[ 'step1': 'install_stuff',
'step2': 'wait(step1)',
'step3': 'allocate_sql_user(server=%mysql_server%)'
'step4': 'credentials=wait_and_read(step3)'
'step5
Re: [openstack-dev] [Heat] HOT Software configuration proposal
Hi,
I guess then heat is mainly doing top-level orchestration, and then
mistral does the workflow middle-level, and taskflow is (hopefully) at the
lowest-level??
You drove the right picture. I can not say who is top-level and who is
low-level orchestration. This is all gear wheels which should work all
together well to achieve the result while Het is probably the driving wheel
among them who makes sure that everything is working.
Thanks
Georgy
On Tue, Oct 22, 2013 at 5:14 PM, Joshua Harlow harlo...@yahoo-inc.comwrote:
Ah,
Seems like a reasonable approach then :-)
I guess then heat is mainly doing top-level orchestration, and then
mistral does the workflow middle-level, and taskflow is (hopefully) at the
lowest-level??
Thanks Georgy!
From: Georgy Okrokvertskhov gokrokvertsk...@mirantis.com
Reply-To: OpenStack Development Mailing List
openstack-dev@lists.openstack.org
Date: Tuesday, October 22, 2013 4:53 PM
To: OpenStack Development Mailing List openstack-dev@lists.openstack.org
Subject: Re: [openstack-dev] [Heat] HOT Software configuration proposal
Hi Joshua,
Sounds like taskflow could be that program (+1 from me, ha)?
Mistral to me is a nice authenticated REST api + other goodies ontop of
something that reliably executes workflows.
I would say that Mistral is a way to do this. My arguments are the
following:
1. Mistral decouples code. Heat can use API calls to invoke a workflows
execution instead of linking with taskflow library in the code. This is
standard SOA approach which OpenStack uses a lot.
2. Mistral will expose DSL to define tasks while taskflow will require
python code for task definition.
Mistral itself uses taskflow library to execute workflows but Mistral in
addition does parsing and translation from DSL task definition to actual
python code.
Heat can use taskflow for other purposes but workflows execution is not
a good reason for that. Just because of nature of workflows for deployment,
there is no knowledge about workflow until end user uploads it, so you can
not use taskflow itself and code this workflow in python without
preliminary knowledge about workflow.
If Heat uses just taskflow it should do all the work on workflow parsing
and translation to a code that Heat team wants to avoid. At least this is
my understanding.
Thanks
Georgy
On Tue, Oct 22, 2013 at 4:34 PM, Joshua Harlow harlo...@yahoo-inc.comwrote:
Sounds like taskflow could be that program (+1 from me, ha)?
Mistral to me is a nice authenticated REST api + other goodies ontop of
something that reliably executes workflows.
But then what I described is also the majority of what openstack does
(authenticated REST api + other goodies ontop of VM/volume/network/...
workflows).
On 10/22/13 3:28 PM, Clint Byrum cl...@fewbar.com wrote:
Excerpts from Georgy Okrokvertskhov's message of 2013-10-22 13:32:40
-0700:
Hi Thomas,
I agree with you on semantics part. At the same time I see a potential
question which might appear - if semantics is limited by few states
visible
for Heat engine, then who actually does software orchestration?
Will it be reasonable then to have software orchestration as separate
subproject for Heat as a part of Orchestration OpenStack program?
Heat
engine will then do dependency tracking and will use components as a
reference for software orchestration engine which will perform actual
deployment and high level software components coordination.
This separated software orchestration engine may address all specific
requirements proposed by different teams in this thread without
affecting
existing Heat engine.
I'm not sure I know what software orchestration is, but I will take a
stab at a succinct definition:
Coordination of software configuration across multiple hosts.
If that is what you mean, then I believe what you actually want is
workflow. And for that, we have the Mistral project which was recently
announced [1].
Use that and you will simply need to define your desired workflow and
feed it into Mistral using a Mistral Heat resource. We can create a
nice bootstrapping resource for Heat instances that shims the mistral
workflow execution agent into machines (or lets us use one already there
via custom images).
I can imagine it working something like this:
resources:
mistral_workflow_handle:
type: OS::Mistral::WorkflowHandle
web_server:
type: OS::Nova::Server
components:
mistral_agent:
component_type: mistral
params:
workflow_: {ref: mistral_workflow_handle}
mysql_server:
type: OS::Nova::Server
components:
mistral_agent:
component_type: mistral
params:
workflow_handle: {ref: mistral_workflow_handle}
mistral_workflow:
type: OS::Mistral::Workflow
properties:
handle: {ref: mistral_workflow_handle}
workflow_reference: mysql_webapp_workflow
Re: [openstack-dev] [Heat] HOT Software orchestration proposal for workflows
On 18/10/13 20:24, John Davidge -X (jodavidg - AAP3 INC at Cisco) wrote: It looks like this discussion involves many of the issues faced when developing the Curvature Donabe frameworks, which were presented at the Portland Summit - slides and video here: http://www.openstack.org/summit/portland-2013/session-videos/presentation/i nteractive-visual-orchestration-with-curvature-and-donabe Much of the work on the Donabe side revolved around defining a simple JSON-based API for describing the sorts of virtual application templates being discussed. All of the code for both Curvature and Donabe has recently been made open source and is available here: http://ciscosystems.github.io/curvature/ http://ciscosystems.github.io/donabe/ Hey John, Congrats on getting this stuff Open-Sourced BTW (I know it's been out for a while now). Can you be more specific about the parts that are relevant to this discussion? I'd be interested to know how Donabe handles configuring the software on Nova servers for a start. It looks like some of the ground covered by these projects can be helpful to this discussion. Yep, it would be great to get input from any folks in the community who have experience with this problem. cheers, Zane. John Davidge jodav...@cisco.com -- Forwarded message -- From: Thomas Spatzier thomas.spatz...@de.ibm.com Date: Wed, Oct 9, 2013 at 12:40 AM Subject: Re: [openstack-dev] [Heat] HOT Software orchestration proposal for workflows To: OpenStack Development Mailing List openstack-dev@lists.openstack.org Excerpts from Clint Byrum's message From: Clint Byrum cl...@fewbar.com To: openstack-dev openstack-dev@lists.openstack.org, Date: 09.10.2013 03:54 Subject: Re: [openstack-dev] [Heat] HOT Software orchestration proposal for workflows Excerpts from Stan Lagun's message of 2013-10-08 13:53:45 -0700: Hello, That is why it is necessary to have some central coordination service which would handle deployment workflow and perform specific actions (create VMs and other OpenStack resources, do something on that VM) on each stage according to that workflow. We think that Heat is the best place for such service. I'm not so sure. Heat is part of the Orchestration program, not workflow. I agree. HOT so far was thought to be a format for describing templates in a structural, declaritive way. Adding workflows would stretch it quite a bit. Maybe we should see what aspects make sense to be added to HOT, and then how to do workflow like orchestration in a layer on top. Our idea is to extend HOT DSL by adding workflow definition capabilities as an explicit list of resources, components¹ states and actions. States may depend on each other so that you can reach state X only after you¹ve reached states Y and Z that the X depends on. The goal is from initial state to reach some final state ³Deployed². We also would like to add some mechanisms to HOT for declaratively doing software component orchestration in Heat, e.g. saying that one component depends on another one, or needs input from another one once it has been deployed etc. (I BTW started to write a wiki page, which is admittedly far from complete, but I would be happy to work on it with interested folks - https://wiki.openstack.org/wiki/Heat/Software-Configuration-Provider). However, we must be careful not to make such features too complicated so nobody will be able to use it any more. That said, I believe we could make HOT cover some levels of complexity, but not all. And then maybe workflow based orchestration on top is needed. Orchestration is not workflow, and HOT is an orchestration templating language, not a workflow language. Extending it would just complect two very different (though certainly related) tasks. I think the appropriate thing to do is actually to join up with the TaskFlow project and consider building it into a workflow service or tools (it is just a library right now). There is such state graph for each of our deployment entities (service, VMs, other things). There is also an action that must be performed on each state. Heat does its own translation of the orchestration template into a workflow right now, but we have already discussed using TaskFlow to break up the orchestration graph into distributable jobs. As we get more sophisticated on updates (rolling/canary for instance) we'll need to be able to reason about the process without having to glue all the pieces together. We propose to extend HOT DSL with workflow definition capabilities where you can describe step by step instruction to install service and properly handle errors on each step. We already have an experience in implementation of the DSL, workflow description and processing mechanism for complex deployments and believe we¹ll all benefit by re-using this experience and existing code, having properly discussed and agreed on abstraction layers and distribution of responsibilities between OS
Re: [openstack-dev] [Heat] HOT Software configuration proposal
Steve Baker sba...@redhat.com wrote on 10/15/2013 06:48:53 PM: I've just written some proposals to address Heat's HOT software configuration needs, and I'd like to use this thread to get some feedback: https://wiki.openstack.org/wiki/Heat/Blueprints/hot-software-config https://wiki.openstack.org/wiki/Heat/Blueprints/native-tools-bootstrap-config Please read the proposals and reply to the list with any comments or suggestions. Can you confirm whether I have got the big picture right? I think some of my earlier remarks were mistaken. You propose to introduce the concept of component and recognize software configuration as a matter of invoking components --- with a DAG of data dependencies among the component invocations. While this is similar to what today's heat engine does for resources, you do NOT propose that the heat engine will get in the business of invoking components. Rather: each VM will run a series of component invocations, and in-VM mechanisms will handle the cross-component synchronization and data communication. You propose to add a bit of sugaring for the wait conditionhandle mechanism, and the heat engine will do the de-sugaring. Each component is written in one of a few supported configuration management (CM) frameworks, and essentially all component invocations on a given VM invoke components of the same CM framework (with possible exceptions for one or two really basic ones). The heat engine gains the additional responsibility of making sure that the appropriate CM framework(s) is(are) bootstrapped in each VM. The heat engine gains no additional responsibilities. Have I got that right? Thanks, Mike___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Heat] HOT Software configuration proposal
On 21/10/13 15:45 -0400, Mike Spreitzer wrote: Steve Baker sba...@redhat.com wrote on 10/15/2013 06:48:53 PM: I've just written some proposals to address Heat's HOT software configuration needs, and I'd like to use this thread to get some feedback: https://wiki.openstack.org/wiki/Heat/Blueprints/hot-software-config https://wiki.openstack.org/wiki/Heat/Blueprints/native-tools-bootstrap-config Please read the proposals and reply to the list with any comments or suggestions. Can you confirm whether I have got the big picture right? I think some of my earlier remarks were mistaken. You propose to introduce the concept of component and recognize software configuration as a matter of invoking components --- with a DAG of data dependencies among the component invocations. While this is similar to what today's heat engine does for resources, you do NOT propose that the heat engine will get in the business of invoking components. Rather: each VM will run a series of component invocations, and in-VM mechanisms will handle the cross-component synchronization and data communication. You propose to add a bit of sugaring for the wait conditionhandle mechanism, and the heat engine will do the de-sugaring. Each component is written in one of a few supported configuration management (CM) frameworks, and essentially all component invocations on a given VM invoke components of the same CM framework (with possible exceptions for one or two really basic ones). The heat engine gains the additional responsibility of making sure that the appropriate CM framework(s) is(are) bootstrapped in each VM. The heat engine gains no additional responsibilities. Have I got that right? I hope so, I don't want Heat to get into the business of two different dependency systems. -Angus Thanks, Mike ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Heat] HOT Software configuration proposal
On 10/22/2013 08:45 AM, Mike Spreitzer wrote: Steve Baker sba...@redhat.com wrote on 10/15/2013 06:48:53 PM: I've just written some proposals to address Heat's HOT software configuration needs, and I'd like to use this thread to get some feedback: https://wiki.openstack.org/wiki/Heat/Blueprints/hot-software-config https://wiki.openstack.org/wiki/Heat/Blueprints/native-tools-bootstrap-config Please read the proposals and reply to the list with any comments or suggestions. Can you confirm whether I have got the big picture right? I think some of my earlier remarks were mistaken. You propose to introduce the concept of component and recognize software configuration as a matter of invoking components --- with a DAG of data dependencies among the component invocations. While this is similar to what today's heat engine does for resources, you do NOT propose that the heat engine will get in the business of invoking components. Rather: each VM will run a series of component invocations, and in-VM mechanisms will handle the cross-component synchronization and data communication. This is basically correct, except that in-VM mechanisms won't know much about cross-component synchronization and data communication. They will just execute whatever components are available to be executed, and report back values to heat-engine by signalling to waitconditions. You propose to add a bit of sugaring for the wait conditionhandle mechanism, and the heat engine will do the de-sugaring. Yes, I think improvements can be made on what I proposed, such as every component signalling when it is complete, and optionally including a return value in that signal. Each component is written in one of a few supported configuration management (CM) frameworks, and essentially all component invocations on a given VM invoke components of the same CM framework (with possible exceptions for one or two really basic ones). Rather than being limited to a few supported CM tools, I like the idea of some kind of provider mechanism so that users or heat admins can add support for new CM tools. This implies that it needs to be possible to add a component type without requiring custom python that runs on heat engine. The heat engine gains the additional responsibility of making sure that the appropriate CM framework(s) is(are) bootstrapped in each VM. Maybe. Or it might be up to the user to invoke images that already have the CM tools installed, or the user can provide a custom component provider which installs the tool in the way that they want. As for the cross-component synchronization and data communication question, at this stage I'm not comfortable with bringing something like zookeeper into the mix for a general solution for inter-component communication. If heat engine handles resource dependencies and zookeeper handles software configuration dependencies this would result in the state of the stack being split between two different co-ordination mechanisms. We've put quite some effort into heat engine to co-ordinate resource dependencies. Wait conditions are currently cumbersome to use, but by exposing software configuration state in terms of resource dependencies they do enable heat engine to be central source of state for the entire stack, including progress of software config. If wait conditions can become palatable to use (or completely transparent) then to me that addresses the main concerns about using them in the short term. Longer term I'd consider something like Marconi to replace metadata polling and wait condition signalling but it is too early to be having that conversation. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Heat] HOT Software configuration proposal
(I really do not understand how the archive is ordered. In the by-thread view, in which all messages with this subject are equally indented, the last message listed is not the chronologically last.) I see that components have parameters. In some uses (invocations) of components, parameters are given actual values. I find it surprising that in the component definitions there are no declarations of parameters. Wouldn't that be helpful or needed? Thanks, Mike___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Heat] HOT Software orchestration proposal for workflows
It looks like this discussion involves many of the issues faced when developing the Curvature Donabe frameworks, which were presented at the Portland Summit - slides and video here: http://www.openstack.org/summit/portland-2013/session-videos/presentation/i nteractive-visual-orchestration-with-curvature-and-donabe Much of the work on the Donabe side revolved around defining a simple JSON-based API for describing the sorts of virtual application templates being discussed. All of the code for both Curvature and Donabe has recently been made open source and is available here: http://ciscosystems.github.io/curvature/ http://ciscosystems.github.io/donabe/ It looks like some of the ground covered by these projects can be helpful to this discussion. John Davidge jodav...@cisco.com -- Forwarded message -- From: Thomas Spatzier thomas.spatz...@de.ibm.com Date: Wed, Oct 9, 2013 at 12:40 AM Subject: Re: [openstack-dev] [Heat] HOT Software orchestration proposal for workflows To: OpenStack Development Mailing List openstack-dev@lists.openstack.org Excerpts from Clint Byrum's message From: Clint Byrum cl...@fewbar.com To: openstack-dev openstack-dev@lists.openstack.org, Date: 09.10.2013 03:54 Subject: Re: [openstack-dev] [Heat] HOT Software orchestration proposal for workflows Excerpts from Stan Lagun's message of 2013-10-08 13:53:45 -0700: Hello, That is why it is necessary to have some central coordination service which would handle deployment workflow and perform specific actions (create VMs and other OpenStack resources, do something on that VM) on each stage according to that workflow. We think that Heat is the best place for such service. I'm not so sure. Heat is part of the Orchestration program, not workflow. I agree. HOT so far was thought to be a format for describing templates in a structural, declaritive way. Adding workflows would stretch it quite a bit. Maybe we should see what aspects make sense to be added to HOT, and then how to do workflow like orchestration in a layer on top. Our idea is to extend HOT DSL by adding workflow definition capabilities as an explicit list of resources, components¹ states and actions. States may depend on each other so that you can reach state X only after you¹ve reached states Y and Z that the X depends on. The goal is from initial state to reach some final state ³Deployed². We also would like to add some mechanisms to HOT for declaratively doing software component orchestration in Heat, e.g. saying that one component depends on another one, or needs input from another one once it has been deployed etc. (I BTW started to write a wiki page, which is admittedly far from complete, but I would be happy to work on it with interested folks - https://wiki.openstack.org/wiki/Heat/Software-Configuration-Provider). However, we must be careful not to make such features too complicated so nobody will be able to use it any more. That said, I believe we could make HOT cover some levels of complexity, but not all. And then maybe workflow based orchestration on top is needed. Orchestration is not workflow, and HOT is an orchestration templating language, not a workflow language. Extending it would just complect two very different (though certainly related) tasks. I think the appropriate thing to do is actually to join up with the TaskFlow project and consider building it into a workflow service or tools (it is just a library right now). There is such state graph for each of our deployment entities (service, VMs, other things). There is also an action that must be performed on each state. Heat does its own translation of the orchestration template into a workflow right now, but we have already discussed using TaskFlow to break up the orchestration graph into distributable jobs. As we get more sophisticated on updates (rolling/canary for instance) we'll need to be able to reason about the process without having to glue all the pieces together. We propose to extend HOT DSL with workflow definition capabilities where you can describe step by step instruction to install service and properly handle errors on each step. We already have an experience in implementation of the DSL, workflow description and processing mechanism for complex deployments and believe we¹ll all benefit by re-using this experience and existing code, having properly discussed and agreed on abstraction layers and distribution of responsibilities between OS components. There is an idea of implementing part of workflow processing mechanism as a part of Convection proposal, which would allow other OS projects to benefit by using this. We would like to discuss if such design could become a part of future Heat version as well as other possible contributions from Murano team. Thanks really for thinking this through. Windows servers are not unique in this regard. Puppet and chef are pretty decent
Re: [openstack-dev] [Heat] HOT Software configuration proposal
On 16/10/13 21:02, Clint Byrum wrote: Excerpts from Zane Bitter's message of 2013-10-16 06:16:33 -0700: I'd love to be able to put this control in the user's hands by just using provider templates - i.e. you designate PuppetServer.yaml as the provider for an OS::Nova::Server in your template and it knows how to configure Puppet and handle the various components. We could make available a library of such provider templates, but users wouldn't be limited to only using those. This I don't think I understand well enough to pass judgement on. My understanding of providers is that they are meant to make templates more portable between clouds that have different capabilities. That's certainly one use, and the main one we've been concentrating on. One of the cool things about providers IMHO is that it's a completely generic feature, not tied to one particular use case, so it's potentially very powerful, possibly in ways that we haven't even thought of yet. So the idea here would be that you have a [Puppet|Chef|Salt|Ansible|CFEngine]Server provider template that contains an OS::Nova::Server resource with the UserData filled in to configure the CM system to start and e.g. grab the component configs from the metadata. You then use this template as the provider for one or more of your servers, link the components somehow and off you go. Mapping that onto different CM systems feels like a stretch and presents a few questions for me. How do I have two OS::Nova::Server's using different CM systems when I decide to deploy something that uses salt instead of chef? You can specify a provider for an individual resource, you don't have to map a whole resource type to it in the environment (although you can). Even if you did, you could just make up your own resource types (say, OS::Custom::ChefServer and OS::Custom::SaltServer). As long as components can be composed of other components, then it seems to me that you would just want the CM system to be a component. If you find yourself including the same set of components constantly.. just make a bigger component out of them. This is a good point, and may well be the way to go. I was thinking that the CM system had to be effectively one step higher in the chain than the components that run under it, but maybe that's really only true of cloud-init. (BTW I agree with Steve B here, addressing the problem is much more important to me than any particular solution.) cheers, Zane. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Heat] HOT Software configuration proposal
On Tue, Oct 15, 2013 at 09:21:12PM -0400, Mike Spreitzer wrote: Steve Baker sba...@redhat.com wrote on 10/15/2013 06:48:53 PM: From: Steve Baker sba...@redhat.com To: openstack-dev@lists.openstack.org, Date: 10/15/2013 06:51 PM Subject: [openstack-dev] [Heat] HOT Software configuration proposal I've just written some proposals to address Heat's HOT software configuration needs, and I'd like to use this thread to get some feedback: https://wiki.openstack.org/wiki/Heat/Blueprints/hot-software-config In that proposal, each component can use a different configuration management tool. https://wiki.openstack.org/wiki/Heat/Blueprints/native-tools-bootstrap-config In this proposal, I get the idea that it is intended that each Compute instance run only one configuration management tool. At least, most of the text discusses the support (e.g., the idea that each CM tool supplies userdata to bootstrap itself) in terms appropriate for a single CM tool per instance; also, there is no discussion of combining userdata from several CM tools. IMO it makes no sense to use more than one CM tool on a particular instance, apart from the case already mentioned by stevebaker where cloud-init is used to bootstrap some other tool. From my discussions with folks so far, they want: - Something simple and declarative at the template interface - A way to reuse data and knowledge from existing CM tools (Puppet/Chef/...) in a clean an non-complex way I agree with the separation of concerns issues that have been raised. I think all this software config stuff can be handled by a pre-processor that takes an extended template in and outputs a plain template that can be consumed by today's heat engine (no extension to the heat engine necessary). I think this is the exact opposite of the direction we should be headed. IMO we should be abstracting the software configuration complexity behind a Heat resource interface, not pushing it up to a pre-processor (which implies some horribly complex interfaces at the heat template level) Steve ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Heat] HOT Software configuration proposal
On 16/10/13 06:56, Mike Spreitzer wrote: What is the difference between what today's heat engine does and a workflow? I am interested to hear what you experts think, I hope it will be clarifying. I presume the answers will touch on things like error handling, state tracking, and updates. (Disclaimer: I'm not an expert, I just work on this stuff ;) First off, to be clear, it was my understanding from this thread that the original proposal to add workflow syntax to HOT is effectively dead. (If I'm mistaken, add a giant -1 from me.) Mirantis have since announced, I assume not coincidentally, that they will start implementing a workflow service (Mistral, based on the original Convection proposal from Keith Bray at the Havana summit) for OpenStack, backed by the taskflow library. So bringing workflows back in to this discussion is confusing the issue. (FWIW I think that having a workflow service will be a great thing for other reasons, but I also hope that all of Stan's original example will be possible in Heat *without* resorting to requiring users to define an explicit workflow.) It must be acknowledged that the Heat engine does run a workflow. The workflow part can in principle, and probably should, be delegated to the taskflow library, and I would be surprised if we did not eventually end up doing this (though I'm not looking forward to actually implementing it). To answer your question, the key thing that Heat does is take in two declarative models and generate a workflow to transform one into the other. (The general case of this is a stack update, where the two models are defined in the previous and new templates. Stack create and delete are special cases where one or the other of the models is empty.) Workflows don't belong in HOT because they are a one-off thing. You need a different one for every situation, and this is exactly why Heat exists - to infer the correct workflow to reify a model in any given situation. cheers, Zane. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Heat] HOT Software configuration proposal
On 16/10/13 00:48, Steve Baker wrote: I've just written some proposals to address Heat's HOT software configuration needs, and I'd like to use this thread to get some feedback: https://wiki.openstack.org/wiki/Heat/Blueprints/hot-software-config https://wiki.openstack.org/wiki/Heat/Blueprints/native-tools-bootstrap-config Wow, nice job, thanks for writing all of this up :) Please read the proposals and reply to the list with any comments or suggestions. For me the crucial question is, how do we define the interface for synchronising and passing data from and to arbitrary applications running under an arbitrary configuration management system? Compared to this, defining the actual format in which software applications are specified in HOT seems like a Simple Matter of Bikeshedding ;) (BTW +1 for not having the relationships, hosted_on always reminded me uncomfortably of INTERCAL[1]. We already have DependsOn for resources though, and might well need it here too.) I'm not a big fan of having Heat::Puppet, Heat::CloudInit, Heat::Ansible c. component types insofar as they require your cloud provider to support your preferred configuration management system before you can use it. (In contrast, it's much easier to teach your configuration management system about Heat because you control it yourself, and configuration management systems are already designed for plugging in arbitrary applications.) I'd love to be able to put this control in the user's hands by just using provider templates - i.e. you designate PuppetServer.yaml as the provider for an OS::Nova::Server in your template and it knows how to configure Puppet and handle the various components. We could make available a library of such provider templates, but users wouldn't be limited to only using those. cheers, Zane. [1] https://en.wikipedia.org/wiki/COMEFROM ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Heat] HOT Software configuration proposal
Steven Hardy sha...@redhat.com wrote on 10/16/2013 04:11:40 AM: ... IMO we should be abstracting the software configuration complexity behind a Heat resource interface, not pushing it up to a pre-processor (which implies some horribly complex interfaces at the heat template level) I am not sure I follow. Can you please elaborate on the horrible implication? Thanks, Mike___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Heat] HOT Software configuration proposal
: 16.10.2013 00:51 Subject: [openstack-dev] [Heat] HOT Software configuration proposal I've just written some proposals to address Heat's HOT software configuration needs, and I'd like to use this thread to get some feedback: https://wiki.openstack.org/wiki/Heat/Blueprints/hot-software-config https://wiki.openstack.org/wiki/Heat/Blueprints/native-tools-bootstrap-config Please read the proposals and reply to the list with any comments or suggestions. We can spend some time discussing software configuration at tomorrow's Heat meeting, but I fully expect we'll still be in the discussion phase at Hong Kong. cheers___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Heat] HOT Software configuration proposal
On 16/10/13 15:58, Mike Spreitzer wrote: Zane Bitter zbit...@redhat.com wrote on 10/16/2013 08:25:38 AM: To answer your question, the key thing that Heat does is take in two declarative models and generate a workflow to transform one into the other. (The general case of this is a stack update, where the two models are defined in the previous and new templates. Stack create and delete are special cases where one or the other of the models is empty.) Workflows don't belong in HOT because they are a one-off thing. You need a different one for every situation, and this is exactly why Heat exists - to infer the correct workflow to reify a model in any given situation. Thanks for a great short sharp answer. In that light, I see a concern. Once a workflow has been generated, the system has lost the ability to adapt to changes in either model. In a highly concurrent and dynamic environment, that could be problematic. I think you're referring to the fact if reality diverges from the model we have no way to bring it back in line (and even when doing an update, things can and usually will go wrong if Heat's idea of the existing template does not reflect reality any more). If so, then I agree that we are weak in this area. You're obviously aware of http://summit.openstack.org/cfp/details/95 so it is definitely on the radar. cheers, Zane. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Heat] HOT Software configuration proposal
Zane Bitter zbit...@redhat.com wrote on 10/16/2013 10:30:44 AM: On 16/10/13 15:58, Mike Spreitzer wrote: ... Thanks for a great short sharp answer. In that light, I see a concern. Once a workflow has been generated, the system has lost the ability to adapt to changes in either model. In a highly concurrent and dynamic environment, that could be problematic. I think you're referring to the fact if reality diverges from the model we have no way to bring it back in line (and even when doing an update, things can and usually will go wrong if Heat's idea of the existing template does not reflect reality any more). If so, then I agree that we are weak in this area. You're obviously aware of http://summit.openstack.org/cfp/details/95 so it is definitely on the radar. Actually, I am thinking of both of the two models you mentioned. We are only in the midst of implementing an even newer design (heat based), but for my group's old code we have a revised design in which the infrastructure orchestrator can react to being overtaken by later updates to the model we call target state (origin source is client) as well as concurrent updates to the model we call observed state (origin source is hardware/hypervisor). I haven't yet decided what to recommend to the heat community, so I'm just mentioning the issue as a possible concern. Thanks, Mike___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Heat] HOT Software configuration proposal
Excerpts from Zane Bitter's message of 2013-10-16 06:16:33 -0700: On 16/10/13 00:48, Steve Baker wrote: I've just written some proposals to address Heat's HOT software configuration needs, and I'd like to use this thread to get some feedback: https://wiki.openstack.org/wiki/Heat/Blueprints/hot-software-config https://wiki.openstack.org/wiki/Heat/Blueprints/native-tools-bootstrap-config Wow, nice job, thanks for writing all of this up :) Please read the proposals and reply to the list with any comments or suggestions. For me the crucial question is, how do we define the interface for synchronising and passing data from and to arbitrary applications running under an arbitrary configuration management system? Compared to this, defining the actual format in which software applications are specified in HOT seems like a Simple Matter of Bikeshedding ;) Agreed. This is one area where juju excels (making cross-node message passing simple). So perhaps we should take a look at what works from the juju model and copy it. (BTW +1 for not having the relationships, hosted_on always reminded me uncomfortably of INTERCAL[1]. We already have DependsOn for resources though, and might well need it here too.) Also agreed. The way the new proposal has it, it feels more like composing a workflow. I'm not a big fan of having Heat::Puppet, Heat::CloudInit, Heat::Ansible c. component types insofar as they require your cloud provider to support your preferred configuration management system before you can use it. (In contrast, it's much easier to teach your configuration management system about Heat because you control it yourself, and configuration management systems are already designed for plugging in arbitrary applications.) Also agree. Having Heat know too much about anything that has many implementations is a bit of a layer violation. Heat has an interface, and we should make it really easy for the popular tools to consume said interface. I could see us having a separate set of shims, like heat-puppet, and heat-salt, that are helpers for those systems to consume data from Heat more smoothly. I'd love to be able to put this control in the user's hands by just using provider templates - i.e. you designate PuppetServer.yaml as the provider for an OS::Nova::Server in your template and it knows how to configure Puppet and handle the various components. We could make available a library of such provider templates, but users wouldn't be limited to only using those. This I don't think I understand well enough to pass judgement on. My understanding of providers is that they are meant to make templates more portable between clouds that have different capabilities. Mapping that onto different CM systems feels like a stretch and presents a few questions for me. How do I have two OS::Nova::Server's using different CM systems when I decide to deploy something that uses salt instead of chef? As long as components can be composed of other components, then it seems to me that you would just want the CM system to be a component. If you find yourself including the same set of components constantly.. just make a bigger component out of them. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Heat] HOT Software configuration proposal
Clint Byrum cl...@fewbar.com wrote on 10/16/2013 03:02:13 PM: Excerpts from Zane Bitter's message of 2013-10-16 06:16:33 -0700: For me the crucial question is, how do we define the interface for synchronising and passing data from and to arbitrary applications running under an arbitrary configuration management system? Compared to this, defining the actual format in which software applications are specified in HOT seems like a Simple Matter of Bikeshedding ;) Agreed. This is one area where juju excels (making cross-node message passing simple). So perhaps we should take a look at what works from the juju model and copy it. Actually, this exactly the point how do we define the interface for synchronising and passing data from and to arbitrary applications running under an arbitrary configuration management system? I was addressing in my message/proposal a couple of days back on the mailing list :-) Glad to see that echoed again. I am proposing that Heat should have a higher (than current wait-conditions/signals) level abstraction for synchronization and data exchange. I do not mind it being message passing as in JuJu. Based on our experience I am proposing a zookeeper style global data space with blocking-reads, and non-blocking writes. Thanks, LN ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Heat] HOT Software configuration proposal
On 10/17/2013 02:16 AM, Zane Bitter wrote: On 16/10/13 00:48, Steve Baker wrote: I've just written some proposals to address Heat's HOT software configuration needs, and I'd like to use this thread to get some feedback: https://wiki.openstack.org/wiki/Heat/Blueprints/hot-software-config https://wiki.openstack.org/wiki/Heat/Blueprints/native-tools-bootstrap-config Wow, nice job, thanks for writing all of this up :) Please read the proposals and reply to the list with any comments or suggestions. For me the crucial question is, how do we define the interface for synchronising and passing data from and to arbitrary applications running under an arbitrary configuration management system? Agreed, but I wanted to remove that from the scope of these blueprints, with the hope that what is done here will be an enabler for a new sync/messaging mechanism - or at least not make it harder. I'm not a big fan of having Heat::Puppet, Heat::CloudInit, Heat::Ansible c. component types insofar as they require your cloud provider to support your preferred configuration management system before you can use it. (In contrast, it's much easier to teach your configuration management system about Heat because you control it yourself, and configuration management systems are already designed for plugging in arbitrary applications.) I'd love to be able to put this control in the user's hands by just using provider templates - i.e. you designate PuppetServer.yaml as the provider for an OS::Nova::Server in your template and it knows how to configure Puppet and handle the various components. We could make available a library of such provider templates, but users wouldn't be limited to only using those. I agree you've identified a problem worth solving. The only thing a component type does in heat-engine is build cloud-init chunks. I'll have a think about how this can be implemented as something resembling component providers. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Heat] HOT Software configuration proposal
the content here: depends_on is represented by the order of the components specified in the components property of the OS::Nova::Server. Practically speaking, one component does not depend on another at all. Instead one component might have some precondition which could be satisfied by another specific component, or by a different component which performs that precondition in a different way, or that precondition might already be satisfied by the server image. Modeling the satisfaction of these preconditions is a core purpose of many configuration management tools. Doing this in HOT components would be reinventing something which is more completely implemented by the very tools that some components will be using. Therefore this proposal takes the position that component dependencies should not be modeled at all, and template authors should ensure themselves that preconditions of a component are satisfied either through specifying component order, or by building custom images. Regarding component_execution: async Is this necessary? I think in any case, it should be possible to create infrastructure resources (servers) in parallel. Then only the component startup should be synchronized once the servers are up, and this should be the default behavior. I think this actually related to the dependency topic above. BTW, even component startup inside servers can be done in parallel unless components have dependencies on each other, so doing a component startup in the strict order as given in a list in the template is probably not necessary. There is an argument for not having a component_execution property and async being the default as soon as it is implemented. However some users may prefer having slower stack launch times just so they don't need to think of concurrency concerns ;) As for allowing a server to configure components in parallel, if your configuration is so slow that you're tempted to parallelise then you're possibly doing it wrong. The slowest task would be package installation - package managers take exclusive locks so that won't parallelise. In any case we would encourage custom images with packages preloaded. That leaves writing configuration files and restarting services - this should be fast enough to run in whatever order is specified in OS::Nova::Server. Regarding the wait condition example: I get the idea and it surely would work, but I think it is still un-intuitive and we should about a more abstract declarative way for expressing such use cases. Maybe, but there should still be an easier way of consuming waitconditions - not necessarily resembling that example. Regarding the native tool bootstrap config proposal: I agree with other comments already made on this thread that the sheer number of different config components seems to much. I guess for users it will be hard to understand, which one to use when, what combination makes sense, in which order they have to be combined etc. Especially, when things are getting combined, my gut feeling is that the likelyhood of templates breaking whenever some of the underlying implementation changes will increase. The only reason to support multiple tools is that potential users have often already invested effort into the scripts for their favorite tool, and would likely balk at trying heat if they had to rewrite all of their software configuration in a different syntax. There will be documentation guiding the user on what are the most appropriate component types for a given situation. Steve Baker sba...@redhat.com wrote on 16.10.2013 00:48:53: From: Steve Baker sba...@redhat.com To: openstack-dev@lists.openstack.org, Date: 16.10.2013 00:51 Subject: [openstack-dev] [Heat] HOT Software configuration proposal I've just written some proposals to address Heat's HOT software configuration needs, and I'd like to use this thread to get some feedback: https://wiki.openstack.org/wiki/Heat/Blueprints/hot-software-config https://wiki.openstack.org/wiki/Heat/Blueprints/native-tools-bootstrap-config ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [Heat] HOT Software configuration proposal
I've just written some proposals to address Heat's HOT software configuration needs, and I'd like to use this thread to get some feedback: https://wiki.openstack.org/wiki/Heat/Blueprints/hot-software-config https://wiki.openstack.org/wiki/Heat/Blueprints/native-tools-bootstrap-config Please read the proposals and reply to the list with any comments or suggestions. We can spend some time discussing software configuration at tomorrow's Heat meeting, but I fully expect we'll still be in the discussion phase at Hong Kong. cheers ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Heat] HOT Software configuration proposal
Steve Baker sba...@redhat.com wrote on 10/15/2013 06:48:53 PM: From: Steve Baker sba...@redhat.com To: openstack-dev@lists.openstack.org, Date: 10/15/2013 06:51 PM Subject: [openstack-dev] [Heat] HOT Software configuration proposal I've just written some proposals to address Heat's HOT software configuration needs, and I'd like to use this thread to get some feedback: https://wiki.openstack.org/wiki/Heat/Blueprints/hot-software-config In that proposal, each component can use a different configuration management tool. https://wiki.openstack.org/wiki/Heat/Blueprints/native-tools-bootstrap-config In this proposal, I get the idea that it is intended that each Compute instance run only one configuration management tool. At least, most of the text discusses the support (e.g., the idea that each CM tool supplies userdata to bootstrap itself) in terms appropriate for a single CM tool per instance; also, there is no discussion of combining userdata from several CM tools. I agree with the separation of concerns issues that have been raised. I think all this software config stuff can be handled by a pre-processor that takes an extended template in and outputs a plain template that can be consumed by today's heat engine (no extension to the heat engine necessary). Regards, Mike ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Heat] HOT Software configuration proposal
On 10/16/2013 02:21 PM, Mike Spreitzer wrote: Steve Baker sba...@redhat.com wrote on 10/15/2013 06:48:53 PM: From: Steve Baker sba...@redhat.com To: openstack-dev@lists.openstack.org, Date: 10/15/2013 06:51 PM Subject: [openstack-dev] [Heat] HOT Software configuration proposal I've just written some proposals to address Heat's HOT software configuration needs, and I'd like to use this thread to get some feedback: https://wiki.openstack.org/wiki/Heat/Blueprints/hot-software-config In that proposal, each component can use a different configuration management tool. https://wiki.openstack.org/wiki/Heat/Blueprints/native-tools-bootstrap-config In this proposal, I get the idea that it is intended that each Compute instance run only one configuration management tool. At least, most of the text discusses the support (e.g., the idea that each CM tool supplies userdata to bootstrap itself) in terms appropriate for a single CM tool per instance; also, there is no discussion of combining userdata from several CM tools. I think users should be told that it is possible but foolhardy to mix CM tools in a single server. The exception to this *might* be allowing one Heat::CloudInit (or Heat::SoftwareConfig) component to install the other CM tool on a pristine image before running the other components that use that tool. Initially I thought that each component type should be able to ensure that its CM tool is installed before invoking that tool. The realities of doing this the right way all the time are difficult though[1] so I've backed away from this for now. It can always be added as an enhancement later. In the meantime users are free to build images that have all required prerequisites, or install them with a Heat::CloudInit (or Heat::SoftwareConfig) component on boot. I agree with the separation of concerns issues that have been raised. I think all this software config stuff can be handled by a pre-processor that takes an extended template in and outputs a plain template that can be consumed by today's heat engine (no extension to the heat engine necessary). A stated goal of the heat template format is that it be fit-for-use by humans. Pre-processing is a perfectly valid technique for other services that use heat and we encourage that. However if the pre-processing is to work around usability issues in the template format I would rather focus on fixing those issues. [1] Currently the most reliable way of installing heat-cfntools on any arbitrary pristine image is in a venv ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Heat] HOT Software configuration proposal
The threading in the archive includes this discussion under the HOT Software orchestration proposal for workflows heading, and the overall ordering in the archive looks very mixed up to me. I am going to reply here, hoping that the new subject line will be subject to less strange ordering in the archive; this is really a continuation of the overall discussion, not just Steve Baker's proposal. What is the difference between what today's heat engine does and a workflow? I am interested to hear what you experts think, I hope it will be clarifying. I presume the answers will touch on things like error handling, state tracking, and updates. I see the essence of Steve Baker's proposal to be that of doing the minimal mods necessary to enable the heat engine to orchestrate software components. The observation is that not much has to change, since the heat engine is already in the business of calling out to things and passing values around. I see a little bit of a difference, maybe because I am too new to already know why it is not an issue. In today's heat engine, the calls are made to fixed services to do CRUD operations on virtual resources in the cloud, using credentials managed implicitly; the services have fixed endpoints, even as the virtual resources come and go. Software components have no fixed service endpoints; the service endpoints come and go as the host Compute instances come and go; I did not notice a story about authorization for the software component calls. Interestingly, Steve Baker's proposal reminds me a lot of Chef. If you just rename Steve's component to recipe, the alignment gets real obvious; I am sure it is no accident. I am not saying it is isomorphic --- clearly Steve Baker's proposal has more going on, with its cross-VM data dependencies and synchronization. But let me emphasize that we can start to see a different way of thinking here. Rather than focusing on a centrally-run workflow, think of each VM as independently running its own series of recipes --- with the recipes invocations now able to communicate and synchronize between VMs as well as within VMs. Steve Baker's proposal uses two forms of communication and synchronization between VMs: (1) get_attr and (2) wait conditions and handles (sugar coated or not). The implementation of (1) is part of the way the heat engine invokes components, the implementation of (2) is independent of the heat engine. Using the heat engine for orchestration is limited to the kinds of logic that the heat engine can run. This may be one reason people are suggesting using a general workflow engine. However, the recipes (components) running in the VMs can do general computation; if we allow general cross-VM communication and synchronization as part of those general computations, we clearly have a more expressive system than the heat engine. Of course, a general distributed computation can get itself into trouble (e.g., deadlock, livelock). If we structure that computation as a set of components (recipe invocations) with a DAG of dependencies then we avoid those troubles. And the kind of orchestration that the heat engine does is sufficient to invoke such components. Structuring software orchestration as a DAG of components also gives us a leg up on UPDATE. Rather than asking the user to write a workflow for each different update, or a general meta-workflow that does introspection to decide what work needs to be done, we ask the thing that invokes the components to run through the components in the way that today's heat engine runs through resources for an UPDATE. Lakshmi has been working on a software orchestration technique that is also centered on the idea of a DAG of components. It was created before we got real interested in Heat. It is implemented as a pre-processor that runs upstream of where today's heat engine goes, emitting fairly minimal userdata needed for bootstrapping. The dependencies between recipe invocations are handled very smoothly in the recipes, which are written in Chef. No hackery is needed in the recipe text at all (thanks to Ruby metaprogramming); what is needed is only an additional declaration of what are the cross-VM inputs and outputs of each recipe. The propagation of data and synchronization between VMs is handled, under the covers, via simple usage of ZooKeeper (other implementations are reasonable too). But the idea of heat-independent propagation of data and synchronization among a DAG of components is not limited to chef-based components, and can appear fairly smooth in any recipe language. A value of making software orchestration independent of today's heat engine is that it enables the four-stage pipeline that I have sketched at https://docs.google.com/drawings/d/1Y_yyIpql5_cdC8116XrBHzn6GfP_g0NHTTG_W4o0R9U and whose ordering of functionality has been experimentally vetted with some non-trivial examples. The first big one
Re: [openstack-dev] [Heat] HOT Software orchestration proposal for workflows
Steven Dake sd...@redhat.com wrote on 11.10.2013 21:02:38: From: Steven Dake sd...@redhat.com To: OpenStack Development Mailing List openstack-dev@lists.openstack.org, Date: 11.10.2013 21:04 Subject: Re: [openstack-dev] [Heat] HOT Software orchestration proposal for workflows On 10/11/2013 11:55 AM, Lakshminaraya Renganarayana wrote: Clint Byrum cl...@fewbar.com wrote on 10/11/2013 12:40:19 PM: From: Clint Byrum cl...@fewbar.com To: openstack-dev openstack-dev@lists.openstack.org Date: 10/11/2013 12:43 PM Subject: Re: [openstack-dev] [Heat] HOT Software orchestration proposal for workflows 3. Ability to return arbitrary (JSON-compatible) data structure from config application and use attributes of that structure as an input for other configs Note that I'd like to see more use cases specified for this ability. The random string generator that Steve Baker has put up should handle most cases where you just need passwords. Generated key sharing might best be deferred to something like Barbican which does a lot more than Heat to try and keep your secrets safe. I had seen a deployment scenario that needed more than random string generator. It was during the deployment of a system that has clustered application servers, i.e., a cluster of application server nodes + a cluster manager node. The deployment progresses by all the VMs (cluster-manager and cluster-nodes) starting concurrently. Then the cluster-nodes wait for the cluster-manager to send them data (xml) to configure themselves. The cluster-manager after reading its own config file, generates config-data for each cluster-node and sends it to them. Is the config data per cluster node unique to each node? If not: I think Lakshmi's example (IBM WebSphere, right?) talks about a case where the per cluster member info is unique per member, so the one fits all approach does not work. In addition, I think there is a constraint that members must join one by one and cannot join concurrently. Change deployment to following model: 1. deploy cluster-manager as a resource with a waitcondition - passing the data using the cfn-signal -d to send the xml blob 2. have cluster nodes wait on wait condition in #1, using data from the cfn-signal If so, join the config data sent in cfn-signal and break it apart by the various cluster nodes in #2 Thanks, LN ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Heat] HOT Software orchestration proposal for workflows
Hi Angus, Thanks for detailed reply. I have a few comments that I have written below in the context. Angus Salkeld asalk...@redhat.com wrote on 10/13/2013 06:40:01 PM: - INPUTS: all the attributes that are consumed/used/read by that resource (currently, we have Ref, GetAttrs that can give this implicitly) - OUTPUTS: all the attributes that are produced/written by that resource (I do not know if this write-set is currently well-defined for a resource. I think some of them are implicitly defined by Heat on particular resource types.) - Global name-space and data-space : all the values produced and consumed (INPUTS/OUTPUTS) are described using a names that are fully qualified (XXX.stack_name.resource_name.property_name). The data values associated with these names are stored in a global data-space. Reads are blocking, i.e., reading a value will block the execution resource/thread until the value is available. Writes are non-blocking, i.e., any thread can write a value and the write will succeed immediately. I don't believe this would give us any new behaviour. I believe that in today's Heat, wait-conditions and signals are the only mechanism for synchronization during software configuration. The proposed mechanism would provide a higher level synchronization based on blocking-reads. For example, if one is using Chef for software configuration, then the recipes can use the proposed mechanism to wait for the all the node[][] attributes they require before starting the recipe execution. And, Heat can actually analyze and reason about deadlock properties of such a synchronization. On the other hand, if the recipe were using wait-conditions how would Heat reason about deadlock properties of it? The ability to define resources at arbitrary levels of granularity together with the explicit specification of INPUTS/OUTPUTS allows us to reap the benefits G1 and G2 outlined above. Note that the ability to reason about the inputs/outputs of each resource and the induced dependencies will also allow Heat to detect dead-locks via dependence cycles (benefit G3). This is already done today in Heat for Refs, GetAttr on base-resources, but the proposal is to extend the same to arbitrary attributes for any resource. How are TemplateResources and NestedStacks any different? To my knowledge this is aleady the case. The blocking-read and non-blocking writes further structures the specification to avoid deadlocks and race conditions (benefit G3). Have you experienced deadlocks with heat? I have never seen this... Heat as it is today does not tackle the problem of synchronization during software configuration and hence the problems I see cannot be attributed to Heat and can only be attributed to the scripts / recipes that do the software configuration. However, if we envision Heat to provide some support for software configuration I can easily imagine cases where it is impossible for Heat to analyze/reason with wait-conditions and hence leading to deadlocks. Wait-conditions and signals are equal to Timed-Semaphores in their power and expressivity and these are known for their problems with deadlocks. To me what is missing to better support complex software configuration is : - better integrating with existing configuration tools (puppet, chef, salt, ansible, etc). (resource types) One question is, whether in this integration the synchronization is completely left to the configuration tools or Heat would be involved in it. If it is left to configuration tools, say chef, then the question is how does the iterative convergence style execution of chef interfere with the schedule order that Heat determines for a template. On the other hand, if Heat provides the mechanism for synchronization, then the question is whether wait-conditions and signals are the right abstractions for them. What are your thoughts on this? Thanks, LN ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Heat] HOT Software orchestration proposal for workflows
write a Blueprint / proposal with more details and use-cases. We already have this functionality. Whenever you use a Ref or Attr from another resource we become dependant on it. This feels like replacing an existing system with a completely different one with very little benefit. To me what is missing to better support complex software configuration is : - better integrating with existing configuration tools (puppet, chef, salt, ansible, etc). (resource types) - integration with a ssh key store (Barbican) so the tools above can do non-userdata related configuration -Angus Thanks, LN Clint Byrum cl...@fewbar.com wrote on 10/11/2013 12:40:19 PM: From: Clint Byrum cl...@fewbar.com To: openstack-dev openstack-dev@lists.openstack.org Date: 10/11/2013 12:43 PM Subject: Re: [openstack-dev] [Heat] HOT Software orchestration proposal for workflows Excerpts from Stan Lagun's message of 2013-10-11 07:22:37 -0700: Hello, Thanks Angus, Clint, I've got your design. It seems that Murano can built on top of that. With service metadata knowledge Murano can generate HOT templates with set of interdependent configs. Here is what will be needed: 1. Ability to implement support for custom software configuration tool (type: OS::SoftwareConfig::MuranoAgent) These are really syntactic sugar, but you can implement them as resource plugins for users who want Murano resources. In the absence of this, just putting things in the free form metadata area of the resource that the MuranoAgent can interpret would suffice. 2. Ability to provide arbitrary input values for the config We already have that, there's a free-form json document called metadata attached to every resource. Or maybe I missed what you mean here. The new capability that is in the works that will make that better is to have multiple reusable metadata blocks referenced on one instance. 3. Ability to return arbitrary (JSON-compatible) data structure from config application and use attributes of that structure as an input for other configs Note that I'd like to see more use cases specified for this ability. The random string generator that Steve Baker has put up should handle most cases where you just need passwords. Generated key sharing might best be deferred to something like Barbican which does a lot more than Heat to try and keep your secrets safe. 4. Ability to provide config body that is an input to Murano Agent of arbitrary size Isn't this the same as 2? 5. Work well with large graph of configs with a lot of dependencies. Independent configs on different VMs should be applied in parallel. Yes this does look good. For dependent configs, the exsiting wait condition can be used. Does it confirm to your plans? I think it confirms that we're heading toward consensus on where to draw the software config vs. infrastructure orchestration line. That is very exciting. :) ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Heat] HOT Software orchestration proposal for workflows
Hello,
Thanks Angus, Clint, I've got your design.
It seems that Murano can built on top of that. With service metadata
knowledge Murano can generate HOT templates with set of interdependent
configs.
Here is what will be needed:
1. Ability to implement support for custom software configuration tool
(type: OS::SoftwareConfig::MuranoAgent)
2. Ability to provide arbitrary input values for the config
3. Ability to return arbitrary (JSON-compatible) data structure from config
application and use attributes of that structure as an input for other
configs
4. Ability to provide config body that is an input to Murano Agent of
arbitrary size
5. Work well with large graph of configs with a lot of dependencies.
Independent configs on different VMs should be applied in parallel.
Does it confirm to your plans?
On Fri, Oct 11, 2013 at 3:47 AM, Clint Byrum cl...@fewbar.com wrote:
Excerpts from Angus Salkeld's message of 2013-10-10 15:27:48 -0700:
On 10/10/13 11:59 +0400, Stan Lagun wrote:
This rises number of questions:
1. What about conditional dependencies? Like config3 depends on config1
AND
config2 OR config3.
We have the AND, but not an OR. To depend on two resources you just
have 2 references to the 2 resources.
AND is concrete. OR is not. I don't actually think it is useful for what
Heat is intended to do. This is not not packaging, this is deploying.
For deploying, Heat needs to know _what to do_, not what is possible.
2. How do I pass values between configs? For example config1 requires
value
from user input and config2 needs an output value obtained from applying
config1
{Fn::GetAtt: [config2, the_name_of_the_attribute]}
This is a little bit misleading. Heat does not have any good ways to
get a value obtained from applying config1. The data attribute of
the WaitCondition is the only way I know, and it is really unwieldy,
as it can basically only dump a json string of all of the things each
signaler has fed back in.
That said, I wonder how many of the value obtained from applying config1
would be satisfied by the recently proposed random string generation
resource. Most of the time what people want to communicate back is just
auth details. If we push auth details in from Heat to both sides, that
alleviates all of my current use cases for this type of feature.
3. How would you do error handling? For example config3 on server3
requires
config1 to be applied on server1 and config2 on server2. Suppose that
there
was an error while applying config2 (and config1 succeeded). How do I
specify reaction for that? Maybe I need then to try to apply config4 to
server2 and continue or maybe just roll everything back
We currently have no on_error but it is not out of scope. The
current action is either to rollback the stack or leave it in the
failed state (depending on what you choose).
Right, I can definitely see more actions being added as we identify the
commonly desired options.
4. How these config dependencies play with nested stacks and resources
like
LoadBalancer that create such stacks? How do I specify that myConfig
depends on HA proxy being configured if that config was declared in
nested
stack that is generated by resource's Python code and is not declared
in my
HOT template?
It is normally based on the actual data/variable that you are
dependant on.
loadbalancer: depends on autoscaling instance_list
(actually in the loadbalancer config would be a GetAtt: [scalegroup,
InstanceList])
Then if you want to depend on that config you could depend on an
attribute of that resource that changes on reconfigure.
config1:
type: OS::SoftwareConfig::Ssh
properties:
script: {GetAtt: [scalegroup, InstanceList]}
hosted_on: loadbalancer
...
config2:
type: OS::SoftwareConfig::Ssh
properties:
script: {GetAtt: [config1, ConfigAppliedCount]}
hosted_on: somewhere_else
...
I am sure we could come up with some better syntax for this. But
the logic seems easily possible to me.
As far as nested stacks go: you just need an output to be useable
externally - basically design your API.
5. The solution is not generic. For example I want to write HOT template
for my custom load-balancer and a scalable web-servers group. Load
balancer
config depends on all configs of web-servers. But web-servers are
created
dynamically (autoscaling). That means dependency graph needs to be also
dynamically modified. But if you explicitly list config names instead of
something like depends on all configs of web-farm X you have no way to
describe such rule. In other words we need generic dependency, not just
dependency on particular config
Why won't just depending on the scaling group be enough? if it needs
to be updated it will update all within the group before progressing
to the dependants.
In the example, loadbalancer doesn't
Re: [openstack-dev] [Heat] HOT Software orchestration proposal for workflows
Excerpts from Stan Lagun's message of 2013-10-11 07:22:37 -0700: Hello, Thanks Angus, Clint, I've got your design. It seems that Murano can built on top of that. With service metadata knowledge Murano can generate HOT templates with set of interdependent configs. Here is what will be needed: 1. Ability to implement support for custom software configuration tool (type: OS::SoftwareConfig::MuranoAgent) These are really syntactic sugar, but you can implement them as resource plugins for users who want Murano resources. In the absence of this, just putting things in the free form metadata area of the resource that the MuranoAgent can interpret would suffice. 2. Ability to provide arbitrary input values for the config We already have that, there's a free-form json document called metadata attached to every resource. Or maybe I missed what you mean here. The new capability that is in the works that will make that better is to have multiple reusable metadata blocks referenced on one instance. 3. Ability to return arbitrary (JSON-compatible) data structure from config application and use attributes of that structure as an input for other configs Note that I'd like to see more use cases specified for this ability. The random string generator that Steve Baker has put up should handle most cases where you just need passwords. Generated key sharing might best be deferred to something like Barbican which does a lot more than Heat to try and keep your secrets safe. 4. Ability to provide config body that is an input to Murano Agent of arbitrary size Isn't this the same as 2? 5. Work well with large graph of configs with a lot of dependencies. Independent configs on different VMs should be applied in parallel. Yes this does look good. For dependent configs, the exsiting wait condition can be used. Does it confirm to your plans? I think it confirms that we're heading toward consensus on where to draw the software config vs. infrastructure orchestration line. That is very exciting. :) ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Heat] HOT Software orchestration proposal for workflows
On 10/9/2013 12:55 PM, Joshua Harlow wrote: Your example sounds a lot like what taskflow is build for doing. https://github.com/stackforge/taskflow/blob/master/taskflow/examples/calculate_in_parallel.py is a decent example. In that one, tasks are created and input/output dependencies are specified (provides, rebind, and the execute function arguments itself). This is combined into the taskflow concept of a flow, one of those flows types is a dependency graph. Using a parallel engine (similar in concept to a heat engine) we can run all non-dependent tasks in parallel. An example that I just created that shows this (and shows it running) that closer matches your example. Program (this will work against the current taskflow codebase): http://paste.openstack.org/show/48156/ I think that there is a major difference between building a set of virtual servers (what Heat does) and performing a specific task on a set of servers (what taskflow is designed for). Taskflow is currently planned to have a more complex and robust state machine that what Heat plans. This is natural given that simplicity has a relatively higher value for deployment, while efficiency of execution matters more for a task that will be performed repeatedly. However, if a simpler state model is needed, keep in mind that a simpler interface can always be translated into the more complete interface with a shim layer. You cannot build a more flexible solution on top of a simple solution. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
