loud-self-healing/
[3] Integration points (aka packs) https://exchange.stackstorm.org/
On Sat, May 6, 2017 at 12:25 PM, Dmitri Zimine <dzim...@brocade.com> wrote:
>
>
> From: Lingxian Kong <anlin.k...@gmail.com>
> Reply-To: "OpenStack Development Mailing List (not fo
StackStorm is game for PGT: Winson will likely be there for Mistral F2F:
from what we see now it’s quite a few things where F2F get-together can
help & accelerate.
So if Renat you’re in, we have 3 companies.
Cheers, Dmitri.
Tony,
You can also connect Mistral with Ansible via StackStorm and Ansible
integration pack:
1) StackStorm http://docs.stackstorm.com
2) https://github.com/StackStorm/st2contrib/tree/master/packs/ansible
StackStorm is open-source Apache 2.0 “wrapper” around Mistral workflow service
that
that renaming the task is a source of too many frustrating
errors :)
What other think?
DZ.
On Sep 3, 2015, at 4:23 AM, Renat Akhmerov <rakhme...@mirantis.com> wrote:
>
>> On 02 Sep 2015, at 21:01, Dmitri Zimine <dzim...@stackstorm.com> wrote:
>>
>> Agree,
>>
Agree,
with one detail: make it explicit - task(task_name).
res - we often see folks confused by result of what (action, task, workflow)
although we cleaned up our lingo: action-output, task-result, workflow-output….
but still worth being explicit.
And full result is being thought as the
Goirand z...@debian.org wrote:
On 06/30/2015 07:09 AM, Dmitri Zimine wrote:
Thanks Stan,
This release is few more months. How soon? Are you planning to support
0.2 in the meantime?
We are really pressed on this transition to 1.0.
For instance. Number 1 user error while dealing with YAQL
Awesome! Really.
Thank you folks for doing this.
I am so much looking forward to moving it to 1.0 with more built-in functions
and more power to extend it...
Note that Mistral has a few extensions, like `str`, `len`, which are not in the
scope of evaluator.
DZ
On Aug 2, 2015, at 12:44
Hi Noy,
The short answer is No, BPMN is currently not supported, and No, we didn’t hear
requests to support it yet. It is architecturally feasible but is a substantial
effort, and not on the current roadmap.
The longer answer is: Both Mistral DSL and BPMN come from the common root of
This is great news Alex, was looking forward to it, will be happy to migrate
Mistral.
Some heads-up on what syntactically changed would be much appreciated to pass
on to our users;
we likely will catch much of them with Mistral tests, but some may bubble up.
DZ.
On Jul 27, 2015, at 2:04
) for
that
Sincerely yours,
Stan Lagun
Principal Software Engineer @ Mirantis
On Tue, Aug 4, 2015 at 9:57 PM, Dmitri Zimine dzim...@stackstorm.com wrote:
This is great news Alex, was looking forward to it, will be happy to migrate
Mistral.
Some heads-up on what syntactically changed
, it’s a pretty important thing that we’d like to clarify as soon as
possible.
Any input from Murano team?
Renat Akhmerov
@ Mirantis Inc.
On 26 Jun 2015, at 06:01, Dmitri Zimine dzim...@stackstorm.com wrote:
Folks,
it’s been some time,what’s the news:
* Is Murano moving YAQL
Folks,
it’s been some time,what’s the news:
* Is Murano moving YAQL 1.0 in this cycle?
* What’s your recommendation for this cycle - stay on 0.2.6 or move on?
* Any progress on documentation?
Thanks, DZ
__
OpenStack
+1 great write-up Winson,
I propose we move the discussion to an etherpad, and flash out details there so
it won’t get lost in a long thread.
Winson would you care to create one and post here?
Re: ‘error state’: I think it’s not absolutely necessary: pause/resume can be
done without enabling
Folks, pls remind me where we ended up with ‘concurrency’?
In the current implementation concurrency is a task policy (and not sure how
much we tested it, not with unit testing/automated testing).
Also I recall discussing/ going back and forth re if concurrency is a task
policy or it belongs
may be not quite - please advice how it works in these cases
1) if break-on expression contains the reference to task result, like
break-on: % $.my_task.foo.bar = true %
but action returns ERROR and task payload is None (desired behavior: don’t
puke, evaluate to false and don’t break)
2) if
My 2c:
Yes Mistral moved to YAQL 1.0 based on Murano team recommendations :)
some questions/comments before we decide how to proceed:
1) Let’s clarify the impact: this problem doesn’t affect Murano directly; but
it impacts Murano-Congress-Mistral initiative, correct?
Is this a voting gate?
Renat,
following up on the IRC meeting: I recalled one more item, and registered a
blueprint - https://blueprints.launchpad.net/mistral/+spec/yaql-v1-0I think
it’s good timing to do it in Kilo if YAQL 1.0 is announced; so I propose it for
Kilo-rc1;
One question is when YAQL 1.0 is going to
Hi folks,
I’d like to propose Winson Chan (m4dcoder) to become Mistral core team member.
Winson brings unique mix of field experience implementing Mistral workflows in
user environments, and solid development skills.
He has been contributing to Mistral since Mar 2014. Winson did a 23
Team, thanks for your input so far, I flashed out one more section, please take
a look and comment
https://docs.google.com/a/stackstorm.com/document/d/1Gy6V9YBt8W4llyErO_itHetkF1oNYv4ka-_5LdFKA18/edit#heading=h.n1jc8i9qhikt
DZ.
On Mar 25, 2015, at 9:41 AM, Dmitri Zimine dzim...@stackstorm.com
the proposal at your earliest convenience,
happy to answer questions and give more information here, or IRC on the
#openstack-mistral.
Thanks,
Dmitri Zimine (irc dzimine)
on behalf of Mistral contributors.
[1] http://governance.openstack.org/reference/new-projects-requirements.html
Thanks Winson for the summary.
@Lingxian Kong
The context for a task is used
internally, I know the aim for this feature is to make it very easy
and convinient for users to see the details for the workflow exection,
but what users can do next with the context? Do you have a plan to
change
Folks,
we are discussing DSL improvements, based on Mistral field use and lessons
learned.
Please join: comment on the document welcome, extra ideas, preferably based on
your experience writing Mistral workflows.
The summary is in the doc, it contains two sections:
1) certain improvements,
Mistral team got an exciting contribution:
our good friend and industry veteran GP De Ciantis had built a sublime plugin
for Mistral DSL - workbooks, workflows, etc.
Check it out, and enjoy writing Mistral workflows!
https://github.com/giampierod/mistral-sublime
Thanks GP for an excellent
16:06, Dmitri Zimine wrote:
2) Use functions, like Heat HOT or TOSCA:
HOT templates and TOSCA doesn’t seem to have a concept of typed
variables to borrow from (please correct me if I missed it). But they
have functions: function: { function_name: {foo: [parameter1, parameter
2], bar:xxx
2015, at 03:06, Dmitri Zimine dzim...@stackstorm.com wrote:
SUMMARY:
We are changing the syntax for inlining YAQL expressions in Mistral YAML
from {1+$.my.var} (or “{1+$.my.var}”) to % 1+$.my.var %
Below I explain the rationale and the criteria for the choice. Comments
SUMMARY:
We are changing the syntax for inlining YAQL expressions in Mistral YAML from
{1+$.my.var} (or “{1+$.my.var}”) to % 1+$.my.var %
Below I explain the rationale and the criteria for the choice. Comments and
suggestions welcome.
DETAILS:
-
We faced a
and adopted OpenStack development process and
tools but the work is still in progress. Any help from Mistral team and/or
other YAQL adopters is appreciated.
Sincerely yours,
Stan Lagun
Principal Software Engineer @ Mirantis
On Thu, Jan 15, 2015 at 10:54 PM, Dmitri Zimine dzim
Folks,
We use YAQL in Mistral for referencing variables, expressing conditions, etc.
Murano is using it extensively, I saw Heat folks thought of using it, at least
once :) May be others...
We are learning that YAQL incredibly powerful comparing to alternatives like
Jinja2 templates used in
The problem:
Refer to workflow / action output without explicitly re-publishing the output
values. Why we want it: to reduce repetition, and to make modifications in the
place values are used, not where they are obtained (and not in multiple
places). E.g., as an editor of a workflow, when I
Thanks Angus.
One obvious thing is we either make it somewhat consistent, or name it
differently.
These looks similar, at least on the surface. I wonder if the feedback we’ve
got so far (for-each is confusing because it brings wrong expectations) is
applicable to Heat, too.
Another
Anastasia, any start is a good start.
1 api 1 engine 1 executor, list-workbooks
what exactly doest it mean: 1) is mistral deployed on 3 boxes with component
per box, or all three are processes on the same box? 2) is list-workbooks test
running while workflow executions going on? How many?
Based on the feedback so far, I updated the document and added some more
details from the comments and discussions.
We still think for-each as a keyword confuses people by setting up some
behavior expectations (e.g., it will run sequentially, you can work with data
inside the loop, you can
The problem with existing syntax is it is not defined: there is no docs on
inlining complex variables [*], and we haven’t tested it for anything more than
the simplest cases:
https://github.com/stackforge/mistral/blob/master/mistral/tests/unit/workbook/v2/test_dsl_specs_v2.py#L114.
I will be
I had a short user feedback sessions with Patrick and James, the short summary
is:
1) simplify the syntax to optimize for the most common use case
2) 'concurrency' is the best word - but bring it out of for-each to task level,
or task/policies
3) all-permutation - relatively rare case, either
Winson, Lakshmi, Renat:
It looked good and I began to write down the summary:
https://etherpad.openstack.org/p/mistral-global-context
But than realized that it’s not safe to assume from the action, that the global
context will be supplied as part of API call.
Check it out in the etherpad.
Hi Raanan,
1) yes, the workflow execution can be triggered by the REST API call.
2) yes the Mistral is the closest in OpenStack for the higher-level automation
and integration across operations and with external systems.
Are you coming to OpenStack summit by any chance? Your comments expose
Folks, I have moved the DSL and API docs to the project (up on review). Now it
contains enough examples to get going, also see doc/README.md for some useful
hints.
Renat, Nikolay, if you find any omissions please add; and once we are happy
let’s remove the DSL and API wiki to avoid confusion.
Thanks Renat for running and capturing.
One addition - allocate time for bug fixes, we’ll have quite a few :)
DZ.
On Oct 2, 2014, at 9:56 PM, Renat Akhmerov rakhme...@mirantis.com wrote:
Hi,
Mistral team has selected blueprints for Mistral 0.2 which is currently
scheduled for 10/31/2014
Use case:
The client software fires the workflow execution and needs to be know when the
workflow is complete. There is no good pool strategy as workflow can take
arbitrary time from ms to days. Callback notification is needed.
Solution is a webhook
Option 1: pass callback URL as part of
Once we got dependencies on keystone-python client, Mistral doesn’t build for
me on Mac.
Before, I installed a new openssl (1.0.1h) - keystone authentication didn’t
work with out it, remember?
enykeev suggested to return to the old stock openssl, it worked.
but this sort of sucks to switch
Hi Stackers,
we are discussing the API, v2.
The core team captured the initial thoughts
here (most recent):
https://etherpad.openstack.org/p/mistral-API-v2-discussion
and here:
https://docs.google.com/a/stackstorm.com/document/d/12j66DZiyJahdnV8zXM_fCRJb0qxpsVS_eyhZvdHlIVU/edit
Have a
Renat, team:
The current plan of record on 0.1 here,
https://launchpad.net/mistral/+milestone/0.1
Some suggested adjustments:
https://docs.google.com/a/stackstorm.com/document/d/1Ukd6SDk9tMpn8kpti-kXaOzVhl8Jm1YWTh3Kw1A6lgQ/edit#
Pls use the doc to comment/disccuss and let’s decide by next
Got some questions while fixing a bug https://review.openstack.org/#/c/102391/:
* We must convey the action ERROR details back to the engine, and to the end
user. Log is not sufficient. How exactly? Via context? Via extra parameters to
convey_execution_results? Need a field in the model.
https://review.openstack.org/#/c/100390/
Angus asked:
Why do we even need start: start-task?
Can't we parse the workbook and figure out what to run? Tasks at the bottom of
the tree have no on-{fail,success} and then
follow upwards until you find a task that is not referenced by anything.
Yes
Hi,
This is a reminder about the community meeting in IRC June 02 at 16.00 UTC at
#openstack-meeting.
Agenda:
Review action items
Current status (quickly by team members)
Further plans
Open discussion
You can also find it at https://wiki.openstack.org/wiki/Meetings/MistralAgenda
as well as
We shared and discussed the directions on Mistral development session,
and work through the list of smaller post POC steps, placing them on the
roadmap.
The notes are here: https://etherpad.openstack.org/p/juno-summit-mistral.
Renat and I developed shared understanding on most of them.
Next
+1 for breaking down the configuration by modules.
Not sure about names for configuration group. Do you mean just using the same
group name? or more?
IMO groups are project specific; it doesn’t always make sense to use the same
group name in the context of different projects. Our requirement
Hi folks,
Last summit we proposed Search project for OpenStack[1]. We didn't get much
traction and other things took priorities, so the project didn't take off
during Havana cycle.
Now - before and during the summit - more people expressed keen interest. Some
of us met on the summit and
Renat, Joshua, Dmitri and Timur discussed the integration path for Mistral and
TaskFlow.
Quick summary - guys please correct any mistakes or omissions:
Taskflow - Mistral integration plan
1) Mistral: define the set of workflow primitives, define correspondent DSL
- Driven by user
The problem: Keystone authentication on MAC works from command line, but fails
when run from PyCharm. And I want to run PyCharm to debug!
The root cause: Keystone uses openssl cms which is only available in new
openssl. I installed openssl via brew, and got it in /usr/local/bin. Terminal
Ivan,
any update/progress on this?
Thanks, DZ.
On Apr 15, 2014, at 10:21 AM, Joshua Harlow harlo...@yahoo-inc.com wrote:
Well Ivan afaik is thinking through it, but being a community project its not
exactly easy to put estimations or timelines on things (I don't control Ivan,
or
We prototyped Mistral / TaskFlow integration and have a follow-up discussions.
SUMMARY: Mistral (Workflow Service) can embed TaskFlow as a workflow library,
with some required modifications to function resliently as a service, and for
smooth integration. However, the TaskFlow flow controls are
IRC to discuss http://tinyurl.com/k3s2gmy
Joshua, 2000 UTC doesn't quite work for Renat and Kirill (3 am their time).
The overlap is:
PST (UTC-7) UTC NOVT (UTC+7)
04pm (16:00)11pm (23:00)6am (06:00)
10pm (22:00)05am (05:00)12pm (12:00)
Kirill's
Two more workflows drafted - cloud cron, and lifecycle, version 1.
The mindset questions are:
1) is DSL syntax expressive, and capable and convenient to handle real use
cases?
2) most importantly: what are the implied workflow capabilities which make it
all work?
* Take a look here:
Even more responses inline :)
On Mar 31, 2014, at 8:44 PM, Joshua Harlow harlo...@yahoo-inc.com wrote:
More responses inline :)
From: Dmitri Zimine d...@stackstorm.com
Date: Monday, March 31, 2014 at 5:59 PM
To: Joshua Harlow harlo...@yahoo-inc.com
Cc: OpenStack Development Mailing List
On Apr 1, 2014, at 3:43 AM, Renat Akhmerov rakhme...@mirantis.com wrote:
On 25 Mar 2014, at 01:51, Joshua Harlow harlo...@yahoo-inc.com wrote:
The first execution model I would call the local execution model, this model
involves forming tasks and flows and then executing them inside an
I am in the full agreement with the proposed approach (risked to copy below,
let's see if email handles your diagrams):
* Single Engine handles multiple executions asynchronously, in non-blocking way.
* Persistence access only from API and/or Engine, Engine writes, API reads.
* Action runes
/TaskFlow#Retries (new feature!)
This is nice. I would call it a 'repeater': running a sub flow several times
with various data for various reasons is reacher then 'retry'.
What about the 'retry policy' on individual task?
From: Dmitri Zimine d...@stackstorm.com
Reply-To: OpenStack Development
on top of TaskFlow prototype - summary, next steps
Open discussion
Looking forward to see you there.
Dmitri Zimine
@ StackStorm
PS. Hope Renat is coming back this week. ___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http
Following up on http://tinyurl.com/l8gtmsw and http://tinyurl.com/n3v9lt8:
this explains how Mistral handles long running delegate tasks. Note that a
'passive' workflow engine can handle both normal tasks and delegates the same
way. I'll also put that on ActionDesign wiki, after discussion.
other options to schedule tasks in TaskFlow, if we have
too many of these workflows with long running tasks, seems like it'll
become a memory issue for mistral...
On Thu, Mar 20, 2014 at 3:07 PM, Dmitri Zimine d...@stackstorm.com
wrote:
For the 'asynchronous manner' discussion see
http
My understanding is: it's the engine which finalizes the task results, based on
the status returned by the task via convey_task_result call.
https://github.com/stackforge/mistral/blob/master/mistral/engine/abstract_engine.py#L82-L84
Hi folks, thanks for taking part in Mistral meeting on #openstack-meeting
Here is the minutes and the full log.
Minutes
http://eavesdrop.openstack.org/meetings/mistral/2014/mistral.2014-03-24-16.03.html
Log:
For the 'asynchronous manner' discussion see http://tinyurl.com/n3v9lt8; I'm
still not sure why u would want to make is_sync/is_async a primitive concept
in a workflow system, shouldn't this be only up to the entity running the
workflow to decide? Why is a task allowed to be sync/async,
(design,
reviews, code).
Dmitri Zimine (i-dz at launchpad). Dmitri joined the project about 2 months
ago. Since then he’s made a series of important high-quality commits, a lot
of valuable reviews and, IMO most importantly, he has a solid vision of the
project in general (requirements, use
Thanks Renat for a clear design summary,
thanks Joshua for the questions,
+1 to let's move TaskFlow vs Mistral discussion to separate thread,
and my questions/comments on
https://wiki.openstack.org/wiki/Mistral/Blueprints/ActionsDesign below:
- Async actions: how do results of async action
We have access to all configuration parameters in the context of api.py. May be
you don't pass it but just instantiate it where you need it? Or I may
misunderstand what you're trying to do...
DZ
PS: can you generate and update mistral.config.example to include new oslo
messaging options? I
of that
real-world workflow look like? Then we can ask ourselves, does the DSL
make this better or worse? Would we need to expose things like email
handlers, or leave that to the general python libraries?
$0.02
-S
On 03/05/2014 10:50 PM, Dmitri Zimine wrote:
Folks,
I took a crack
I just moved the sample to Git; let's leverage git review for specific comments
on the syntax.
https://github.com/dzimine/mistral-workflows/commit/d8c4a8c845e9ca49f6ea94362cef60489f2a46a3
DZ
On Mar 6, 2014, at 10:36 PM, Dmitri Zimine d...@stackstorm.com wrote:
Folks, thanks for the input
Folks,
I took a crack at using our DSL to build a real-world workflow.
Just to see how it feels to write it. And how it compares with alternative
tools.
This one automates a page from OpenStack operation guide:
Agree on both.
On Feb 26, 2014, at 8:51 PM, Renat Akhmerov rakhme...@mirantis.com wrote:
Hi team,
When I tell peopleI about Mistral I always have a hard time explaining why we
use term “event” for declaring ways to start workflows. For example, take a
look at the following snippet:
My understanding, correct me if I'm missing the intention:
Action are defined as code, aka REST_API or SEND_EMAIL.
These are base actions, they are analogous to function definitions.
Tasks is a set of parameters for the action.
Actions can be also defined declaratively, under Services, based
+1
Should we use VERB_NOUN pattern? Or relax it for some obvious? I can't figure a
good VERB_NOUN for HTTP. REQUEST_HTTP is dull.
BTW it's interesting that we assume that a service can contain only actions of
the same type.
DZ
On Feb 26, 2014, at 1:10 AM, Nikolay Makhotkin
We do use the term DSL, I invite you guys to clarify, how exactly.
Based on the terminology from [1], it's not part of the model, but the language
that describes the model in the file. And theoretically this may be not the
only language to express the workflow. Once the file is parsed, we
As a learning exercise, I was trying to extract the intend behind the DSL and
current code. Giving that many things are still moving, I filled up the blanks
with imagination. Here is my understanding on where you are going:
I have created a blueprint to capture the intention to simplify calling
standard actions:
https://blueprints.launchpad.net/mistral/+spec/mistral-shorthand-action-in-dsl
DZ
On Feb 11, 2014, at 7:40 AM, Dmitri Zimine d...@stackstorm.com wrote:
Yes it makes sense, let's draft how it may look
I agree with Winson's points. Inline.
On Feb 24, 2014, at 8:31 PM, Renat Akhmerov rakhme...@mirantis.com wrote:
On 25 Feb 2014, at 07:12, W Chan m4d.co...@gmail.com wrote:
As I understand, the local engine runs the task immediately whereas the
scalable engine sends it over the message
Winson,
While you're looking into this and working on the design, may be also think
through other executor/engine communications.
We talked about executor communicating to engine over 3 channels (DB, REST,
RabbitMQ) which I wasn't happy about ;) and put it off for some time. May be it
can be
We touched this on review https://review.openstack.org/#/c/73205/, and fixed a
bit, bringing it up here to further discuss at slightly higher level.
Let's go over a tiny bit of YAML definition, clarifying terminology on the way.
Current DSL snippet:
actions:
my-action
parameters:
: vm_id
parameters:
foo: bar
On Fri, Feb 14, 2014 at 12:36 PM, Renat Akhmerov rakhme...@mirantis.com
wrote:
On 14 Feb 2014, at 15:02, Dmitri Zimine d...@stackstorm.com wrote:
Current DSL snippet:
actions:
my-action
parameters:
foo
contains the
info about how the input data will be taken for corresponding task from
initial context. So, it will be a kind of expression (e.g. YAQL).
Renat, am I right?
On Fri, Feb 14, 2014 at 9:51 PM, Dmitri Zimine d...@stackstorm.com wrote:
I like output, too. But it should go
Do we have (or think about) a shorthand to calling REST_API action, without
defining a service?
FULL DSL:
Services:
TimeService:
type: REST_API
parameters:
baseUrl:http://api.timezonedb.com
key:my_api_key
actions:
get-time:
to an action
type rather than the action itself”. Does that make sense?
Renat Akhmerov
@ Mirantis Inc.
On 11 Feb 2014, at 16:23, Dmitri Zimine d...@stackstorm.com wrote:
Do we have (or think about) a shorthand to calling REST_API action, without
defining a service?
FULL DSL:
Services
Ops! missed [openstack-dev], fixing…
Following up from yesterday's community meeting
I am still catching up with the project, still miss a lot of context. Put my
questions and comments on DSL definition:
https://etherpad.openstack.org/p/mistral-dsl-discussion
Let's review, and thanks for
Following up from yesterday's community meeting
I am still catching up with the project, still miss a lot of context. Put my
questions and comments on DSL definition:
https://etherpad.openstack.org/p/mistral-dsl-discussion
Let's review, and thanks for helping us get up to speed.
DZ
85 matches
Mail list logo
