---------- Forwarded message ----------
From: Thomas Spatzier <[email protected]>
Date: Wed, Oct 9, 2013 at 12:40 AM
Subject: Re: [openstack-dev] [Heat] HOT Software orchestration
proposal for workflows
To: OpenStack Development Mailing List <[email protected]>
Excerpts from Clint Byrum's message
From: Clint Byrum <[email protected]>
To: openstack-dev <[email protected]>,
Date: 09.10.2013 03:54
Subject: Re: [openstack-dev] [Heat] HOT Software orchestration
proposal for workflows
Excerpts from Stan Lagun's message of 2013-10-08 13:53:45 -0700:
Hello,
That is why it is necessary to have some central coordination service
which
would handle deployment workflow and perform specific actions (create
VMs
and other OpenStack resources, do something on that VM) on each stage
according to that workflow. We think that Heat is the best place for
such
service.
I'm not so sure. Heat is part of the Orchestration program, not
workflow.
I agree. HOT so far was thought to be a format for describing templates in
a structural, declaritive way. Adding workflows would stretch it quite a
bit. Maybe we should see what aspects make sense to be added to HOT, and
then how to do workflow like orchestration in a layer on top.
Our idea is to extend HOT DSL by adding workflow definition
capabilities
as an explicit list of resources, components¹ states and actions.
States
may depend on each other so that you can reach state X only after
you¹ve
reached states Y and Z that the X depends on. The goal is from initial
state to reach some final state ³Deployed².
We also would like to add some mechanisms to HOT for declaratively doing
software component orchestration in Heat, e.g. saying that one component
depends on another one, or needs input from another one once it has been
deployed etc. (I BTW started to write a wiki page, which is admittedly far
from complete, but I would be happy to work on it with interested folks -
https://wiki.openstack.org/wiki/Heat/Software-Configuration-Provider).
However, we must be careful not to make such features too complicated so
nobody will be able to use it any more. That said, I believe we could make
HOT cover some levels of complexity, but not all. And then maybe workflow
based orchestration on top is needed.
Orchestration is not workflow, and HOT is an orchestration templating
language, not a workflow language. Extending it would just complect two
very different (though certainly related) tasks.
I think the appropriate thing to do is actually to join up with the
TaskFlow project and consider building it into a workflow service or
tools
(it is just a library right now).
There is such state graph for each of our deployment entities
(service,
VMs, other things). There is also an action that must be performed on
each
state.
Heat does its own translation of the orchestration template into a
workflow right now, but we have already discussed using TaskFlow to
break up the orchestration graph into distributable jobs. As we get more
sophisticated on updates (rolling/canary for instance) we'll need to
be able to reason about the process without having to glue all the
pieces together.
We propose to extend HOT DSL with workflow definition capabilities
where
you can describe step by step instruction to install service and
properly
handle errors on each step.
We already have an experience in implementation of the DSL, workflow
description and processing mechanism for complex deployments and
believe
we¹ll all benefit by re-using this experience and existing code,
having
properly discussed and agreed on abstraction layers and distribution
of
responsibilities between OS components. There is an idea of
implementing
part of workflow processing mechanism as a part of Convection
proposal,
which would allow other OS projects to benefit by using this.
We would like to discuss if such design could become a part of future
Heat
version as well as other possible contributions from Murano team.
Thanks really for thinking this through. Windows servers are not unique
in
this regard. Puppet and chef are pretty decent at expressing single-node
workflows but they are awkward for deferring control and resuming on
other
nodes, so I do think there is a need for a general purpose distributed
workflow definition tool.
I'm not, however, convinced that extending HOT would yield a better
experience for users. I'd prefer to see HOT have a well defined
interface
for where to defer to external workflows. Wait conditions are actually
decent at that, and I'm sure with a little more thought we can make them
more comfortable to use.
Good discussion to have, i.e. what extensions we would need in HOT for
making HOT alone more capable, and what we would need to hook it up with
other orchestration like workflows.
_______________________________________________
OpenStack-dev mailing list
[email protected]
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
_______________________________________________
OpenStack-dev mailing list
[email protected]
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
