On 09/10/13 19:31 +0100, Steven Hardy wrote:
On Wed, Oct 09, 2013 at 06:59:22PM +0200, Alex Rudenko wrote:
Hi everyone,
I've read this thread and I'd like to share some thoughts. In my opinion,
workflows (which run on VMs) can be integrated with heat templates as
follows:
1. workflow definitions should be defined separately and processed by
stand-alone workflow engines (chef, puppet etc).
I agree, and I think this is the direction we're headed with the
software-config blueprints - essentially we should end up with some new
Heat *resources* which encapsulate software configuration.
Exactly.
I think we need a software-configuration-aas sub-project that knows
how to take puppet/chef/salt/... config and deploy it. Then Heat just
has Resources for these (OS::SoftwareConfig::Puppet).
We should even move our WaitConditions and Metadata over to that
yet-to-be-made service so that Heat is totally clean of software config.
How would this solve ordering issues:
resources:
config1:
type: OS::SoftwareConfig::Puppet
hosted_on: server1
...
config2:
type: OS::SoftwareConfig::Puppet
hosted_on: server1
depends_on: config3
...
config3:
type: OS::SoftwareConfig::Puppet
hosted_on: server2
depends_on: config1
...
server1:
type: OS::Nova::Server
...
server2:
type: OS::Nova::Server
...
Heat knows all about ordering:
It starts the resources:
server1, server2
config1
config3
config2
There is the normal contract in the client:
we post the config to software-config-service
and we wait for the state == ACTIVE (when the config is applied)
before progressing to a resource that is dependant on it.
-Angus
IMO there is some confusion around the scope of HOT, we should not be
adding functionality to it which already exists in established config
management tools IMO, instead we should focus on better integration with
exisitng tools at the resource level, and identifying template interfaces
which require more flexibility (for example serialization primitives)
2. the HOT resources should reference workflows which they require,
specifying a type of workflow and the way to access a workflow definition.
The workflow definition might be provided along with HOT.
So again, I think this acatually has very little to do with HOT. The
*Heat* resources may define software configuration, or possibly some sort
of workflow, which is acted upon by $thing which is not Heat.
So in the example provided by the OP, maybe you'd have a Murano resource,
which knows how to define the input to the Murano API, which might trigger
workflow type actions to happen in the Murano service.
3. Heat should treat the orchestration templates as transactions (i.e.
Heat should be able to rollback in two cases: 1) if something goes wrong
during processing of an orchestration workflow 2) when a stand-alone
workflow engine reports an error during processing of a workflow associated
with a resource)
So we already have the capability for resources to recieve signals, which
would allow (2) in the asynchronous case. But it seems to me that this is
still a serialization problem, ie a synchronous case, therefore (2) is just
part of (1).
E.g
- Heat stack create starts
- Murano resource created (CREATE IN_PROGRESS state)
- Murano workdlow stuff happens, signals Heat with success/failure
- Murano resource transitions to either COMPLETE or FAILED state
- If a FAILED state happened, e.g on update, we can roll back to the
previous stack definition (this is already possible in Heat)
4. Heat should expose an API which enables basic communication between
running workflows. Additionally, Heat should provide an API to workflows
that allows workflows to specify whether they completed successfully or
not. The reference to these APIs should be passed to the workflow engine
that is responsible for executing workflows on VMs.
I personally don't think this is in scope for Heat. We already have an API
which exposes the status of stacks and resources. Exposing some different
API which describes a workflow implemented by a specific subset of resource
types makes no sense to me.
Pros of each point:
1 & 2 - keeps Heat simple and gives a possibility to choose the best
workflows and engines among available ones.
3 - adds some kind of all-or-nothing semantics improving the control and
awareness of what's going on inside VMs.
4 - allows workflow synchronization and communication through Heat API.
Provides the error reporting mechanism for workflows. If a workflow does
not need this functionality, it can ignore it.
IMHO (4) is very much a step too far, and is not well aligned with the
current interfaces provided by Heat.
I'm really keen to further discuss the use-cases here, but if possible, it
would be helpful if folks can describe their requirements in less abstract
terms, and with reference to our existing interfaces and template model.
These thoughts might show some gaps in my understanding of how Heat works,
but I would like to share them anyway.
You've raised some good points, thanks.
I'm really keen to further discuss the use-cases here, but if possible, it
would be helpful if folks can describe their requirements in less abstract
terms, and with reference to our existing interfaces and template model.
That way we can start defining what is actually missing to support specific
use-cases.
So far I see the following emerging as definite requirements:
- Better/more flexible native serialization interfaces (possbly HOT
additions)
- Better/more flexible *resources* which encapsulate software configuration
on instances, probably with the flexibility of applying more than one
config to an instance (not necessarily related to any HOT changes at all)
Thanks,
Steve
_______________________________________________
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
_______________________________________________
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
