Hi Mistral folks,
Congrats on getting the 0.0.2 release out. I had a look at Renat's screencast and the examples, and I wanted to share some feedback based on my experience with Heat. Y'all will have to judge for yourselves to what extent this experience is applicable to Mistral. (Assume that everything I know about it was covered in the screencast and you won't be far wrong.)

The first thing that struck me looking at https://github.com/stackforge/mistral-extra/tree/master/examples/create_vm is that I have to teach Mistral how to talk to Nova. I can't overstate how surprising this is as a user, because Mistral is supposed to want to become a part of OpenStack. It should know how to talk to Nova! There is actually an existing DSL for interacting with OpenStack[1], and here's what the equivalent operation looks like:

os server create $server_name --image $image_id --flavor $flavor_id --nic net-id=$network_id

Note that this is approximately exactly 96.875% shorter (or 3200% shorter, if you're in advertising).

This approach reminds me a bit of TOSCA, in the way that it requires you to define every node type before you use it. (Even TOSCA is moving away from this by developing a Simple Profile that includes the most common ones in the box - an approach I assume/hope you're considering also.) The stated reason for this is that they want TOSCA templates to run on any cloud regardless of its underlying features (rather than take a lowest-common-denominator approach, as other attempts at hybrid clouds have done). Contrast that with Heat, which is unapologetically an orchestration system *for OpenStack*.

I note from the screencast that Mistral's stated mission is to:

  Provide a mechanism to define and execute
  tasks and workflows *in OpenStack clouds*

(My emphasis.) IMO the design doesn't reflect the mission. You need to decide whether you are trying to build the OpenStack workflow DSL or the workflow DSL to end all workflow DSLs.

That problem could be solved by including built-in definitions for core OpenStack service in a similar way to std.* (i.e. take the TOSCA Simple Profile approach), but I'm actually not sure that goes far enough. The lesson of Heat is that we do best when we orchestrate *only* OpenStack APIs.

For example, when we started working on Heat, there was no autoscaling in OpenStack so we implemented it ourselves inside Heat. Two years later, there's still no autoscaling in OpenStack other than what we implemented, and we've been struggling for a year to try to split Heat's implementation out into a separate API so that everyone can use it.

Looking at things like std.email, I feel a similar way about them. OpenStack is missing something equivalent to SNS, where a message on a queue can trigger an email or another type of notification, and a lot of projects are going to eventually need something like that. It would be really unfortunate if all of them went out and invented it independently. It's much better to implement such things as their own building blocks that can be combined together in complex ways rather than adding that complexity to a bunch of services.

Such a notification service could even be extended to do std.http-like ReST calls, although personally the whole idea of OpenStack services calling out to arbitrary HTTP APIs makes me extremely uncomfortable. Much better IMO to just post messages to queues and let the receiver (long) poll for it.

So I would favour a DSL that is *much* simpler, and replaces all of std.* with functions that call OpenStack APIs, and only OpenStack APIs, including the API for posting messages to Marconi queues, which would be the method of communication to the outside world. (If the latter part sounds a bit like SWF, it's for a good reason, but the fact that it would allow access directly to all of the OpenStack APIs before resorting to an SDK makes it much more powerful, as well as providing a solid justification for why this should be part of OpenStack.)

The ideal way to get support for all of the possible OpenStack APIs would be to do it by introspection on python-openstackclient. That means you'd only have to do the work once and it will stay up to date. This would avoid the problem we have in Heat, where we have to implement each resource type separately. (This is the source of a great deal of Heat's value to users - the existence of tested resource plugins - but also the thing that stops us from iterating the code quicker.)

I'm also unsure that it's a good idea for things like timers to be set up inside the DSL. I would prefer that the DSL just define workflows and export entry points to them. Then have various ways to trigger them: from the API manually, from a message to a Marconi queue, from a timer, &c. The latter two you'd set up through the Mistral API. If a user wanted a single document that set up one or more workflows and their triggers, a Heat template would do that job.

I can see that your goal is to make a system that works with any existing application without changes. I think this is not so important as you think; the lesson of AWS is that developers will happily write their applications to use your service if you make it simple enough for them to understand. In a year's time will anybody think twice about spinning up a container to poll a message queue and proxy it into ReST calls, if that's what they need to do to interface to some legacy/outside code?

As I said at the beginning, you are the experts and you'll have to decide for yourselves how much of this feedback is relevant to Mistral. You certainly know a bunch of things that I don't about workflow-as-a-service. (I am, of course, interested in being re-educated!) But I hope that some of our experience on the Heat project might be helpful to you.


[1] http://docs.openstack.org/developer/python-openstackclient/

OpenStack-dev mailing list

Reply via email to