On 12/11/13 13:04 +0100, Thomas Spatzier wrote:
Hi Angus,
that is an interesting idea. Since you mentioned the software config
proposal in the beginning as a related item, I guess you are trying to
solve some software config related issues with Mistral. So a few questions,
looking at this purely from a software config perspect:
Are you thinking about doing the infrastructure orchestration (VMs,
volumes, network etc) with Heat's current capabilities and then let the
complete software orchestration be handled by Mistral tasks? I.e. bootstrap
the workers on each VM and have the definition of when which agent does
something defined in a flow?
Well we either add an api to heat to do "install_config" or we use
a service that is designed to do tasks. Clint convienced me quite
easily that "install/apply_config" is just a task.
If yes, is there a way for passing data around - e.g. output produced by
one software config step is input for another software config step?
Again, if my above assumption is true, couldn't there be problems when we
having two ways of doing orchestration, when the software layer thing would
take the Heat engine out of some processing and take away some control? Or
are you thinking about using Mistral as a general mechanism for task
execution in Heat, which would then probably resolve the conflict?
At this point we really do not need a "flow", just a task concept
from Mistral. Prehaps ways of grouping them and targeting them
for a particular server.
I'd see the config_deployer resource posting a task to Mistral
and we have an agent in the server that can consume tasks and
pass them to sub-agents that understand the particular format.
If we do this then Heat is in charge of the orchestration and
there are not two workflows fighting for control. I do agree
that there should just be one.
I think once Mistral is more mature we can decide whether to pass
full workflow control over to it, but for now the task functionality
is all we need. (and a time based one would be neat too btw).
-Angus
Regards,
Thomas
Angus Salkeld <[email protected]> wrote on 12.11.2013 02:15:15:
From: Angus Salkeld <[email protected]>
To: [email protected],
Date: 12.11.2013 02:18
Subject: [openstack-dev] [Mistral] really simple workflow for Heat
configuration tasks
Hi all
I think some of you were at the Software Config session at summit,
but I'll link the ideas that were discussed:
https://wiki.openstack.org/wiki/Heat/Blueprints/hot-software-config-WIP
https://wiki.openstack.org/wiki/Heat/Blueprints/hot-software-config
To me the basics of it are:
1. we need an entity/resource to place the configuration (in Heat)
2. we need a resource to install the configuration
(basically a task in Mistral)
A big issue to me is the conflict between heat's taskflow and the new
external one. What I mean by conflict is that it will become tricky
to manage two parallel taskflow instances in one stack.
This could be solved by:
1: totally using mistral (only use mistral workflow)
2: use a very simple model of just asking mistral to run tasks (no
workflow) this allows us to use heat's workflow
but mistral's task runner.
Given that mistral has no real implementation yet, "2" would seem
reasonable to me. (I think Heat developers are open to "1" when
Mistral is more mature.)
How could we use Mistral for config installation?
-------------------------------------------------
1. We have a resource type in Heat that creates tasks in a Mistral
workflow (manual workflow).
2. Heat pre-configures the server to have a Mistral worker
installed.
3. the Mistral worker pulls tasks from the workflow and passes them
to an agent that can run it. (the normal security issues jump up
here - giving access to the taskflow from a guest).
To do this we need an api that can add tasks to a workflow dynamically.
like this:
- create a simple workflow
- create and run task A [run on server X]
- create and run task B [run on server Y]
- create and run task C [run on server X]
(note: the task is run and completes before the next is added if there
is a dependancy, if tasks can be run in parallel then we add
multiple
tasks)
The api could be something like:
CRUD <mistral>/workflows/
CRUD <mistral>/workflows/<wf>/tasks
One thing that I am not sure of is how a server(worker) would know if a
task
was for it or not.
- perhaps we have a capability property of the task that we can use
(capablitiy[server] = <server-id>) or actually specify the worker we
want.
I think this would be a good starting point for Mistral as it is a
very simple but concrete starting point. Also if this is not done in
Mistral we will have to add this in Heat (lets rather have it where
it should be). This will also give us a chance to have confidence
with Mistral before trying to do more complex workflows.
If you (Heat and Mistral developers) are open to this we can discuss
what needs to be done. I am willing to help with implementation.
Thanks
-Angus
_______________________________________________
OpenStack-dev mailing list
[email protected]
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
_______________________________________________
OpenStack-dev mailing list
[email protected]
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
_______________________________________________
OpenStack-dev mailing list
[email protected]
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
