loud-self-healing/
[3] Integration points (aka packs) https://exchange.stackstorm.org/
On Sat, May 6, 2017 at 12:25 PM, Dmitri Zimine wrote:
>
>
> From: Lingxian Kong
> Reply-To: "OpenStack Development Mailing List (not for usage questions)" <
> openstack-dev@lists.
StackStorm is game for PGT: Winson will likely be there for Mistral F2F:
from what we see now it’s quite a few things where F2F get-together can
help & accelerate.
So if Renat you’re in, we have 3 companies.
Cheers, Dmitri.
Tony,
You can also connect Mistral with Ansible via StackStorm and Ansible
integration pack:
1) StackStorm http://docs.stackstorm.com
2) https://github.com/StackStorm/st2contrib/tree/master/packs/ansible
StackStorm is open-source Apache 2.0 “wrapper” around Mistral workflow service
that integ
renaming the task is a source of too many frustrating
errors :)
What other think?
DZ.
On Sep 3, 2015, at 4:23 AM, Renat Akhmerov wrote:
>
>> On 02 Sep 2015, at 21:01, Dmitri Zimine wrote:
>>
>> Agree,
>>
>> with one detail: make it explicit - task(task_name)
Agree,
with one detail: make it explicit - task(task_name).
res - we often see folks confused by result of what (action, task, workflow)
although we cleaned up our lingo: action-output, task-result, workflow-output….
but still worth being explicit.
And full result is being thought as the ro
mas Goirand wrote:
> On 06/30/2015 07:09 AM, Dmitri Zimine wrote:
>> Thanks Stan,
>>
>> This release is few more months. How soon? Are you planning to support
>> 0.2 in the meantime?
>>
>> We are really pressed on this transition to 1.0.
>> For instance
or name, replaced with better
> alternative or just accept additional (optional) parameters
> 6. $collection[$ > 0] doesn't work anymore. Use $collection.select($ > 0) for
> that
>
>
>
> Sincerely yours,
> Stan Lagun
> Principal Software Engineer @ Miranti
Hi Noy,
The short answer is No, BPMN is currently not supported, and No, we didn’t hear
requests to support it yet. It is architecturally feasible but is a substantial
effort, and not on the current roadmap.
The longer answer is: Both Mistral DSL and BPMN come from the common root of
workflo
This is great news Alex, was looking forward to it, will be happy to migrate
Mistral.
Some heads-up on what syntactically changed would be much appreciated to pass
on to our users;
we likely will catch much of them with Mistral tests, but some may bubble up.
DZ.
On Jul 27, 2015, at 2:04 AM
Awesome! Really.
Thank you folks for doing this.
I am so much looking forward to moving it to 1.0 with more built-in functions
and more power to extend it...
Note that Mistral has a few extensions, like `str`, `len`, which are not in the
scope of evaluator.
DZ>
On Aug 2, 2015, at 12:44 PM
es, it’s a pretty important thing that we’d like to clarify as soon as
> possible.
>
> Any input from Murano team?
>
> Renat Akhmerov
> @ Mirantis Inc.
>
>
>
> > On 26 Jun 2015, at 06:01, Dmitri Zimine wrote:
> >
> > Folks,
> >
> &
Folks,
it’s been some time,what’s the news:
* Is Murano moving YAQL 1.0 in this cycle?
* What’s your recommendation for this cycle - stay on 0.2.6 or move on?
* Any progress on documentation?
Thanks, DZ>
__
OpenStack D
+1 great write-up Winson,
I propose we move the discussion to an etherpad, and flash out details there so
it won’t get lost in a long thread.
Winson would you care to create one and post here?
Re: ‘error state’: I think it’s not absolutely necessary: pause/resume can be
done without enabling
Folks, pls remind me where we ended up with ‘concurrency’?
In the current implementation concurrency is a task policy (and not sure how
much we tested it, not with unit testing/automated testing).
Also I recall discussing/ going back and forth re if concurrency is a task
policy or it belongs t
may be not quite - please advice how it works in these cases
1) if break-on expression contains the reference to task result, like
break-on: <% $.my_task.foo.bar = true %>
but action returns ERROR and task payload is None (desired behavior: don’t
puke, evaluate to false and don’t break)
2) if
My 2c:
Yes Mistral moved to YAQL 1.0 based on Murano team recommendations :)
some questions/comments before we decide how to proceed:
1) Let’s clarify the impact: this problem doesn’t affect Murano directly; but
it impacts Murano-Congress-Mistral initiative, correct?
Is this a voting gate? W
Hi folks,
I’d like to propose Winson Chan (m4dcoder) to become Mistral core team member.
Winson brings unique mix of field experience implementing Mistral workflows in
user environments, and solid development skills.
He has been contributing to Mistral since Mar 2014. Winson did a 23 commits
Renat,
following up on the IRC meeting: I recalled one more item, and registered a
blueprint - https://blueprints.launchpad.net/mistral/+spec/yaql-v1-0I think
it’s good timing to do it in Kilo if YAQL 1.0 is announced; so I propose it for
Kilo-rc1;
One question is when YAQL 1.0 is going to mo
Team, thanks for your input so far, I flashed out one more section, please take
a look and comment
https://docs.google.com/a/stackstorm.com/document/d/1Gy6V9YBt8W4llyErO_itHetkF1oNYv4ka-_5LdFKA18/edit#heading=h.n1jc8i9qhikt
DZ.
On Mar 25, 2015, at 9:41 AM, Dmitri Zimine wrote:
> Fo
proposal at your earliest convenience,
happy to answer questions and give more information here, or IRC on the
#openstack-mistral.
Thanks,
Dmitri Zimine (irc dzimine)
on behalf of Mistral contributors.
[1] http://governance.openstack.org/reference/new-projects-requirements.html
Thanks Winson for the summary.
@Lingxian Kong
> The context for a task is used
> internally, I know the aim for this feature is to make it very easy
> and convinient for users to see the details for the workflow exection,
> but what users can do next with the context? Do you have a plan to
> cha
Folks,
we are discussing DSL improvements, based on Mistral field use and lessons
learned.
Please join: comment on the document welcome, extra ideas, preferably based on
your experience writing Mistral workflows.
The summary is in the doc, it contains two sections:
1) certain improvements,
Mistral team got an exciting contribution:
our good friend and industry veteran GP De Ciantis had built a sublime plugin
for Mistral DSL - workbooks, workflows, etc.
Check it out, and enjoy writing Mistral workflows!
https://github.com/giampierod/mistral-sublime
Thanks GP for an excellent c
example, to fully implement ‘with-items’ (although that’s a little
>>>>> bit different story).
>>>>>
>>>>> Just a general note about all changes happening now: Once we release kilo
>>>>> stable release our API, DSL of version 2 must
.
On Feb 18, 2015, at 7:22 AM, Zane Bitter wrote:
> On 16/02/15 16:06, Dmitri Zimine wrote:
>> 2) Use functions, like Heat HOT or TOSCA:
>>
>> HOT templates and TOSCA doesn’t seem to have a concept of typed
>> variables to borrow from (please correct me if I missed
SUMMARY:
We are changing the syntax for inlining YAQL expressions in Mistral YAML from
{1+$.my.var} (or “{1+$.my.var}”) to <% 1+$.my.var %>
Below I explain the rationale and the criteria for the choice. Comments and
suggestions welcome.
DETAILS:
-
We faced a num
adopted OpenStack development process and
> tools but the work is still in progress. Any help from Mistral team and/or
> other YAQL adopters is appreciated.
>
>
>
> Sincerely yours,
> Stan Lagun
> Principal Software Engineer @ Mirantis
>
>
> On Thu, Jan 15,
Folks,
We use YAQL in Mistral for referencing variables, expressing conditions, etc.
Murano is using it extensively, I saw Heat folks thought of using it, at least
once :) May be others...
We are learning that YAQL incredibly powerful comparing to alternatives like
Jinja2 templates used in s
The problem:
Refer to workflow / action output without explicitly re-publishing the output
values. Why we want it: to reduce repetition, and to make modifications in the
place values are used, not where they are obtained (and not in multiple
places). E.g., as an editor of a workflow, when I jus
Anastasia, any start is a good start.
> 1 api 1 engine 1 executor, list-workbooks
what exactly doest it mean: 1) is mistral deployed on 3 boxes with component
per box, or all three are processes on the same box? 2) is list-workbooks test
running while workflow executions going on? How many? wh
ideas there, too.
https://docs.google.com/document/d/1iw0OgQcU0LV_i3Lnbax9NqAJ397zSYA3PMvl6F_uqm0/edit#heading=h.5hjdjqxsgfle
DZ.
On Dec 19, 2014, at 1:01 PM, Dmitri Zimine wrote:
> Thanks Angus.
>
> One obvious thing is we either make it somewhat consistent, or name it
>
Thanks Angus.
One obvious thing is we either make it somewhat consistent, or name it
differently.
These looks similar, at least on the surface. I wonder if the feedback we’ve
got so far (for-each is confusing because it brings wrong expectations) is
applicable to Heat, too.
Another observat
Based on the feedback so far, I updated the document and added some more
details from the comments and discussions.
We still think for-each as a keyword confuses people by setting up some
behavior expectations (e.g., it will run sequentially, you can work with data
inside the loop, you can ‘ne
The problem with existing syntax is it is not defined: there is no docs on
inlining complex variables [*], and we haven’t tested it for anything more than
the simplest cases:
https://github.com/stackforge/mistral/blob/master/mistral/tests/unit/workbook/v2/test_dsl_specs_v2.py#L114.
I will be su
I had a short user feedback sessions with Patrick and James, the short summary
is:
1) simplify the syntax to optimize for the most common use case
2) 'concurrency' is the best word - but bring it out of for-each to task level,
or task/policies
3) all-permutation - relatively rare case, either i
Winson, Lakshmi, Renat:
It looked good and I began to write down the summary:
https://etherpad.openstack.org/p/mistral-global-context
But than realized that it’s not safe to assume from the action, that the global
context will be supplied as part of API call.
Check it out in the etherpad.
Wh
Winson,
thanks for filing the blueprint:
https://blueprints.launchpad.net/mistral/+spec/mistral-global-context,
some clarification questions:
1) how exactly would the user describe these global variables syntactically? In
DSL? What can we use as syntax? In the initial workflow input?
2) what
Hi Raanan,
1) yes, the workflow execution can be triggered by the REST API call.
2) yes the Mistral is the closest in OpenStack for the higher-level automation
and integration across operations and with external systems.
Are you coming to OpenStack summit by any chance? Your comments expose y
Folks, I have moved the DSL and API docs to the project (up on review). Now it
contains enough examples to get going, also see doc/README.md for some useful
hints.
Renat, Nikolay, if you find any omissions please add; and once we are happy
let’s remove the DSL and API wiki to avoid confusion.
Thanks Renat for running and capturing.
One addition - allocate time for bug fixes, we’ll have quite a few :)
DZ.
On Oct 2, 2014, at 9:56 PM, Renat Akhmerov wrote:
> Hi,
>
> Mistral team has selected blueprints for Mistral 0.2 which is currently
> scheduled for 10/31/2014 (may slightly chang
Use case:
The client software fires the workflow execution and needs to be know when the
workflow is complete. There is no good pool strategy as workflow can take
arbitrary time from ms to days. Callback notification is needed.
Solution is a webhook
Option 1: pass callback URL as part of star
Once we got dependencies on keystone-python client, Mistral doesn’t build for
me on Mac.
Before, I installed a new openssl (1.0.1h) - keystone authentication didn’t
work with out it, remember?
enykeev suggested to return to the old stock openssl, it worked.
but this sort of sucks to switch
Hi Stackers,
we are discussing the API, v2.
The core team captured the initial thoughts
here (most recent):
https://etherpad.openstack.org/p/mistral-API-v2-discussion
and here:
https://docs.google.com/a/stackstorm.com/document/d/12j66DZiyJahdnV8zXM_fCRJb0qxpsVS_eyhZvdHlIVU/edit
Have a loo
Renat, team:
The current plan of record on 0.1 here,
https://launchpad.net/mistral/+milestone/0.1
Some suggested adjustments:
https://docs.google.com/a/stackstorm.com/document/d/1Ukd6SDk9tMpn8kpti-kXaOzVhl8Jm1YWTh3Kw1A6lgQ/edit#
Pls use the doc to comment/disccuss and let’s decide by next mis
Got some questions while fixing a bug https://review.openstack.org/#/c/102391/:
* We must convey the action ERROR details back to the engine, and to the end
user. Log is not sufficient. How exactly? Via context? Via extra parameters to
convey_execution_results? Need a field in the model.
https:
https://review.openstack.org/#/c/100390/
Angus asked:
>Why do we even need "start: "?
>Can't we parse the workbook and figure out what to run? Tasks at the bottom of
>the tree have no "on-{fail,success}" and then
>follow upwards until you find a task that is not referenced by anything.
Yes we
Hi,
This is a reminder about the community meeting in IRC June 02 at 16.00 UTC at
#openstack-meeting.
Agenda:
Review action items
Current status (quickly by team members)
Further plans
Open discussion
You can also find it at https://wiki.openstack.org/wiki/Meetings/MistralAgenda
as well as lin
We shared and discussed the directions on Mistral development session,
and work through the list of smaller post POC steps, placing them on the
roadmap.
The notes are here: https://etherpad.openstack.org/p/juno-summit-mistral.
Renat and I developed shared understanding on most of them.
Next ste
Hi folks,
Last summit we proposed Search project for OpenStack[1]. We didn't get much
traction and other things took priorities, so the project didn't take off
during Havana cycle.
Now - before and during the summit - more people expressed keen interest. Some
of us met on the summit and dec
+1 for breaking down the configuration by modules.
Not sure about names for configuration group. Do you mean just using the same
group name? or more?
IMO groups are project specific; it doesn’t always make sense to use the same
group name in the context of different projects. Our requirement
Renat, Joshua, Dmitri and Timur discussed the integration path for Mistral and
TaskFlow.
Quick summary - guys please correct any mistakes or omissions:
Taskflow - Mistral integration plan
1) Mistral: define the set of workflow primitives, define correspondent DSL
- Driven by user require
The problem: Keystone authentication on MAC works from command line, but fails
when run from PyCharm. And I want to run PyCharm to debug!
The root cause: Keystone uses openssl cms which is only available in new
openssl. I installed openssl via brew, and got it in /usr/local/bin. Terminal
work
Ivan,
any update/progress on this?
Thanks, DZ.
On Apr 15, 2014, at 10:21 AM, Joshua Harlow wrote:
> Well Ivan afaik is thinking through it, but being a community project its not
> exactly easy to put estimations or timelines on things (I don't control Ivan,
> or others).
>
> Likely faste
We prototyped Mistral / TaskFlow integration and have a follow-up discussions.
SUMMARY: Mistral (Workflow Service) can embed TaskFlow as a workflow library,
with some required modifications to function resliently as a service, and for
smooth integration. However, the TaskFlow flow controls are
ight (afaik 8am local my
> time). We can just hold off till then?
>
> Or u guys can just drop in #openstack-state-management anytime u are free and
> usually someone is around (usually).
>
> Thoughts?
>
> -Josh
>
> From: Dmitri Zimine
> Date: Thursday, April 3,
IRC to discuss http://tinyurl.com/k3s2gmy
Joshua, 2000 UTC doesn't quite work for Renat and Kirill (3 am their time).
The overlap is:
PST (UTC-7) UTC NOVT (UTC+7)
04pm (16:00)11pm (23:00)6am (06:00)
10pm (22:00)05am (05:00)12pm (12:00)
Kirill's
Two more workflows drafted - cloud cron, and lifecycle, version 1.
The mindset questions are:
1) is DSL syntax expressive, and capable and convenient to handle real use
cases?
2) most importantly: what are the implied workflow capabilities which make it
all work?
* Take a look here:
https:
We agreed to review this and if/when OK, fix the murano-cli. Can you guys
please take a look?
https://review.openstack.org/#/c/81941/
DZ>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listi
On Apr 1, 2014, at 3:43 AM, Renat Akhmerov wrote:
> On 25 Mar 2014, at 01:51, Joshua Harlow wrote:
>
>> The first execution model I would call the local execution model, this model
>> involves forming tasks and flows and then executing them inside an
>> application, that application is runnin
Even more responses inline :)
On Mar 31, 2014, at 8:44 PM, Joshua Harlow wrote:
> More responses inline :)
>
> From: Dmitri Zimine
> Date: Monday, March 31, 2014 at 5:59 PM
> To: Joshua Harlow
> Cc: "OpenStack Development Mailing List (not for usage questions)"
ty on a task. So Mistral didn't have to have it. For
the proposal above, a separate treatment is necessary for _delayed_ tasks.
>
> [1] https://wiki.openstack.org/wiki/TaskFlow#Retries (new feature!)
This is nice. I would call it a 'repeater': running a sub flow several times
with variou
I am in the full agreement with the proposed approach (risked to copy below,
let's see if email handles your diagrams):
* Single Engine handles multiple executions asynchronously, in non-blocking way.
* Persistence access only from API and/or Engine, Engine writes, API reads.
* Action runes don
top of TaskFlow prototype - summary, next steps
Open discussion
Looking forward to see you there.
Dmitri Zimine
@ StackStorm
PS. Hope Renat is coming back this week. ___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http
Following up on http://tinyurl.com/l8gtmsw and http://tinyurl.com/n3v9lt8:
this explains how Mistral handles long running delegate tasks. Note that a
'passive' workflow engine can handle both normal tasks and delegates the same
way. I'll also put that on ActionDesign wiki, after discussion.
Di
My understanding is: it's the engine which finalizes the task results, based on
the status returned by the task via convey_task_result call.
https://github.com/stackforge/mistral/blob/master/mistral/engine/abstract_engine.py#L82-L84
https://github.com/stackforge/mistral/blob/master/mistral/engin
wrote:
>>>>> Can the long running task be handled by putting the target task in the
>>>>> workflow in a persisted state until either an event triggers it or
>>>>> timeout occurs? An event (human approval or trigger from an external
>>>>&g
Hi folks, thanks for taking part in Mistral meeting on #openstack-meeting
Here is the minutes and the full log.
Minutes
http://eavesdrop.openstack.org/meetings/mistral/2014/mistral.2014-03-24-16.03.html
Log:
http://eavesdrop.openstack.org/meetings/mistral/2014/mistral.2014-03-24-16.03.log.html
> For the 'asynchronous manner' discussion see http://tinyurl.com/n3v9lt8; I'm
> still not sure why u would want to make is_sync/is_async a primitive concept
> in a workflow system, shouldn't this be only up to the entity running the
> workflow to decide? Why is a task allowed to be sync/async,
nd made significant contribution (design,
> reviews, code).
> Dmitri Zimine (i-dz at launchpad). Dmitri joined the project about 2 months
> ago. Since then he’s made a series of important high-quality commits, a lot
> of valuable reviews and, IMO most importantly, he has a solid vision o
We have access to all configuration parameters in the context of api.py. May be
you don't pass it but just instantiate it where you need it? Or I may
misunderstand what you're trying to do...
DZ>
PS: can you generate and update mistral.config.example to include new oslo
messaging options? I f
Thanks Renat for a clear design summary,
thanks Joshua for the questions,
+1 to "let's move TaskFlow vs Mistral" discussion to separate thread,
and my questions/comments on
https://wiki.openstack.org/wiki/Mistral/Blueprints/ActionsDesign below:
- Async actions: how do results of async action
I just moved the sample to Git; let's leverage git review for specific comments
on the syntax.
https://github.com/dzimine/mistral-workflows/commit/d8c4a8c845e9ca49f6ea94362cef60489f2a46a3
DZ>
On Mar 6, 2014, at 10:36 PM, Dmitri Zimine wrote:
> Folks, thanks for the input!
t;>>> constructs. As compared to a Ruby-based DSL (for example) where you
>>>> still have Ruby under the hood for the basic stuff and extensions to the
>>>> language for the domain-specific stuff.
>>>>
>>>> Honestly, I'd like to see a
Folks,
I took a crack at using our DSL to build a real-world workflow.
Just to see how it feels to write it. And how it compares with alternative
tools.
This one automates a page from OpenStack operation guide:
http://docs.openstack.org/trunk/openstack-ops/content/maintenance.html#planned_ma
Agree on both.
On Feb 26, 2014, at 8:51 PM, Renat Akhmerov wrote:
> Hi team,
>
> When I tell peopleI about Mistral I always have a hard time explaining why we
> use term “event” for declaring ways to start workflows. For example, take a
> look at the following snippet:
>
> Workflow:
> ..
As a learning exercise, I was trying to extract the intend behind the DSL and
current code. Giving that many things are still moving, I filled up the blanks
with imagination. Here is my understanding on where you are going:
https://docs.google.com/a/stackstorm.com/drawings/d/1qBxrmQ8F7YFlz930zE
We do use the term DSL, I invite you guys to clarify, how exactly.
Based on the terminology from [1], it's not part of the model, but the language
that describes the model in the file. And theoretically this may be not the
only language to express the workflow. Once the file is parsed, we opera
+1
Should we use VERB_NOUN pattern? Or relax it for some obvious? I can't figure a
good VERB_NOUN for HTTP. REQUEST_HTTP is dull.
BTW it's interesting that we assume that a service can contain only actions of
the same type.
DZ>
On Feb 26, 2014, at 1:10 AM, Nikolay Makhotkin wrote:
> Agre
My understanding, correct me if I'm missing the intention:
Action are defined as code, aka REST_API or SEND_EMAIL.
These are base actions, they are analogous to function definitions.
Tasks is a set of parameters for the action.
Actions can be also defined declaratively, under Services, based
I have created a blueprint to capture the intention to simplify calling
standard actions:
https://blueprints.launchpad.net/mistral/+spec/mistral-shorthand-action-in-dsl
DZ>
On Feb 11, 2014, at 7:40 AM, Dmitri Zimine wrote:
> Yes it makes sense, let's draft how it may look;
&g
Winson,
While you're looking into this and working on the design, may be also think
through other executor/engine communications.
We talked about executor communicating to engine over 3 channels (DB, REST,
RabbitMQ) which I wasn't happy about ;) and put it off for some time. May be it
can be
I agree with Winson's points. Inline.
On Feb 24, 2014, at 8:31 PM, Renat Akhmerov wrote:
>
> On 25 Feb 2014, at 07:12, W Chan wrote:
>
>> As I understand, the local engine runs the task immediately whereas the
>> scalable engine sends it over the message queue to one or more executors.
>
ains the
> info about how the input data will be taken for corresponding task from
> initial context. So, it will be a kind of expression (e.g. YAQL).
>
> Renat, am I right?
>
>
> On Fri, Feb 14, 2014 at 9:51 PM, Dmitri Zimine wrote:
> I like output, too. But it should go
ll be taken and stored.
>>
>> However, I agree that this block should be at action-property level:
>>
>> actions:
>>my-action
>> result:
>> select: '$.server_id'
>> store_as: vm_id
>> parameters:
>> fo
We touched this on review https://review.openstack.org/#/c/73205/, and fixed a
bit, bringing it up here to further discuss at slightly higher level.
Let's go over a tiny bit of YAML definition, clarifying terminology on the way.
Current DSL snippet:
actions:
my-action
parameters:
n definition and task property “action” refers to an action
> type rather than the action itself”. Does that make sense?
>
> Renat Akhmerov
> @ Mirantis Inc.
>
> On 11 Feb 2014, at 16:23, Dmitri Zimine wrote:
>
>> Do we have (or think about) a shorthand to calli
Do we have (or think about) a shorthand to calling REST_API action, without
defining a service?
FULL DSL:
Services:
TimeService:
type: REST_API
parameters:
baseUrl:http://api.timezonedb.com
key:
actions:
get-time:
task-paramet
Following up from yesterday's community meeting
I am still catching up with the project, still miss a lot of context. Put my
questions and comments on DSL definition:
https://etherpad.openstack.org/p/mistral-dsl-discussion
Let's review, and thanks for helping us get up to speed.
DZ>
Ops! missed [openstack-dev], fixing…
Following up from yesterday's community meeting
I am still catching up with the project, still miss a lot of context. Put my
questions and comments on DSL definition:
https://etherpad.openstack.org/p/mistral-dsl-discussion
Let's review, and thanks for help
89 matches
Mail list logo
