Excerpts from Clint Byrum's message > From: Clint Byrum <cl...@fewbar.com> > To: openstack-dev <openstack-dev@lists.openstack.org>, > Date: 09.10.2013 03:54 > Subject: Re: [openstack-dev] [Heat] HOT Software orchestration > proposal for workflows > > Excerpts from Stan Lagun's message of 2013-10-08 13:53:45 -0700: > > Hello, > > > > > > That is why it is necessary to have some central coordination service which > > would handle deployment workflow and perform specific actions (create VMs > > and other OpenStack resources, do something on that VM) on each stage > > according to that workflow. We think that Heat is the best place for such > > service. > > > > I'm not so sure. Heat is part of the Orchestration program, not workflow. >
I agree. HOT so far was thought to be a format for describing templates in a structural, declaritive way. Adding workflows would stretch it quite a bit. Maybe we should see what aspects make sense to be added to HOT, and then how to do workflow like orchestration in a layer on top. > > Our idea is to extend HOT DSL by adding workflow definition capabilities > > as an explicit list of resources, components’ states and actions. States > > may depend on each other so that you can reach state X only after you’ve > > reached states Y and Z that the X depends on. The goal is from initial > > state to reach some final state “Deployed”. > > We also would like to add some mechanisms to HOT for declaratively doing software component orchestration in Heat, e.g. saying that one component depends on another one, or needs input from another one once it has been deployed etc. (I BTW started to write a wiki page, which is admittedly far from complete, but I would be happy to work on it with interested folks - https://wiki.openstack.org/wiki/Heat/Software-Configuration-Provider). However, we must be careful not to make such features too complicated so nobody will be able to use it any more. That said, I believe we could make HOT cover some levels of complexity, but not all. And then maybe workflow based orchestration on top is needed. > > Orchestration is not workflow, and HOT is an orchestration templating > language, not a workflow language. Extending it would just complect two > very different (though certainly related) tasks. > > I think the appropriate thing to do is actually to join up with the > TaskFlow project and consider building it into a workflow service or tools > (it is just a library right now). > > > There is such state graph for each of our deployment entities (service, > > VMs, other things). There is also an action that must be performed on each > > state. > > Heat does its own translation of the orchestration template into a > workflow right now, but we have already discussed using TaskFlow to > break up the orchestration graph into distributable jobs. As we get more > sophisticated on updates (rolling/canary for instance) we'll need to > be able to reason about the process without having to glue all the > pieces together. > > > We propose to extend HOT DSL with workflow definition capabilities where > > you can describe step by step instruction to install service and properly > > handle errors on each step. > > > > We already have an experience in implementation of the DSL, workflow > > description and processing mechanism for complex deployments and believe > > we’ll all benefit by re-using this experience and existing code, having > > properly discussed and agreed on abstraction layers and distribution of > > responsibilities between OS components. There is an idea of implementing > > part of workflow processing mechanism as a part of Convection proposal, > > which would allow other OS projects to benefit by using this. > > > > We would like to discuss if such design could become a part of future Heat > > version as well as other possible contributions from Murano team. > > > > Thanks really for thinking this through. Windows servers are not unique in > this regard. Puppet and chef are pretty decent at expressing single-node > workflows but they are awkward for deferring control and resuming on other > nodes, so I do think there is a need for a general purpose distributed > workflow definition tool. > > I'm not, however, convinced that extending HOT would yield a better > experience for users. I'd prefer to see HOT have a well defined interface > for where to defer to external workflows. Wait conditions are actually > decent at that, and I'm sure with a little more thought we can make them > more comfortable to use. Good discussion to have, i.e. what extensions we would need in HOT for making HOT alone more capable, and what we would need to hook it up with other orchestration like workflows. > > _______________________________________________ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev _______________________________________________ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
