[openstack-dev] [mistral] [murano] [yaql] An online YAQL evaluator [was RE: [mistral] [murano] An online YAQL evaluator]

[ELISHA, Moshe (Moshe)]

Hi all,

Thank you for the kind words.
Just wanted to let you know that yaqluator[1] was updated and now supports YAQL 
1.0.2.
There is also a checkbox there to work in "Legacy Mode".

Hope you will find it useful.

[1] http://yaqluator.com/


From: Renat Akhmerov [mailto:rakhme...@mirantis.com]
Sent: Friday, August 14, 2015 11:51 AM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [mistral] [murano] An online YAQL evaluator

I just read this thread so decided to add my 2 cents into the collection of 
opinions.

Guys, I tried it out a couple of weeks ago (was told about it by one of my 
colleagues). This is really incredible! Especially given that you completed it 
in 24 hours :) I think as YAQL attracts more and more users it will be very 
handy tool. I am actually for improving it further.

Thanks a lot! Looking forward to switch to yaql 1.0!

Renat Akhmerov
@ Mirantis Inc.



On 05 Aug 2015, at 04:09, Stan Lagun 
> wrote:

Dmitry,

yaql 1.0 has both str() and len() and much much more so there is no need to 
support them explicitly since Mistral is going to switch to yaql 1.0 and 
yaqluator.com is going to do the same

Sincerely yours,
Stan Lagun
Principal Software Engineer @ Mirantis



On Tue, Aug 4, 2015 at 8:55 PM, Dmitri Zimine 
> wrote:
Awesome! Really.

Thank you folks for doing this.

I am so much looking forward to moving it to 1.0 with more built-in functions 
and more power to extend it...

Note that Mistral has a few extensions, like `str`, `len`, which are not in the 
scope of evaluator.

DZ>


On Aug 2, 2015, at 12:44 PM, Stan Lagun 
> wrote:


Guys, this is awesome!!!

Happy to see yaql gets attention. One more initiative that you may find 
interesting is https://review.openstack.org/#/c/159905/
This is an attempt to port yaql 1.0 from Python to JS so that the same can be 
done in browser

Sincerely yours,
Stan Lagun
Principal Software Engineer @ Mirantis



On Sun, Aug 2, 2015 at 5:30 PM, Nikolay Makhotkin 
> wrote:
Hi guys!

That's awesome! It is very useful for all us!

--
Best Regards,
Nikolay

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: 
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: 
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: 
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: 
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [mistral] "Mistral HA and multi-regional support" meeting minutes

[ELISHA, Moshe (Moshe)]

Hi all,

Renat and I had an action item to think about "Mistral HA and multi-regional 
support".
No big surprises. These are the meeting minutes:

Mistral Multi-Region:
* A blueprint already exists [1]
* Most topics were already discussed in Mitaka OpenStack summit and are 
described in the blueprint.

Mistral HA
* Add a gate that runs Mistral in HA mode (Ask akuznetsova for 
more info as she looked into this once).
* Add more functional tests that are focused on HA tests
* Put together a list of known HA issues that are currently not handled (For 
example, if an executor dies immediately after dequeuing a task) and think of 
solutions.
* Expose Mistral load metrics to allow some external system to decide if it 
needs to scale Mistral components in / out.

[1] https://blueprints.launchpad.net/mistral/+spec/mistral-multi-region-support

Thanks.
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [mistral] Improving Mistral pep8 rules files to match Mistral guidelines

[ELISHA, Moshe (Moshe)]

Thanks, Anastasia!

Who can take start documenting the rules? I remember only a few rules and I 
don’t know all the nuances.
For example, if the return statement is the only statement of a function – do 
you still need a blank line before it?

Once the rules doc will be available I can work on adding these rules to our 
pep8.


From: Anastasia Kuznetsova [mailto:akuznets...@mirantis.com]
Sent: Wednesday, December 09, 2015 1:13 PM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [mistral] Improving Mistral pep8 rules files to 
match Mistral guidelines

Hi Moshe,

Great idea!

It is possible to prepare some additional code checks, for example you can take 
a look how it was done in Rally project [1].
Before starting such work in Mistral, I guess that we can describe our addition 
code style rules in our official docs (somewhere in "Developer Guide" section 
[2]).

[1] https://github.com/openstack/rally/tree/master/tests/hacking
[2] http://docs.openstack.org/developer/mistral/#developer-guide

On Wed, Dec 9, 2015 at 11:21 AM, ELISHA, Moshe (Moshe) 
<moshe.eli...@alcatel-lucent.com<mailto:moshe.eli...@alcatel-lucent.com>> wrote:
Hi all,

Is it possible to add all / some of the special guidelines of Mistral (like 
blank line before return, period at end of comment, …) to our pep8 rules file?

This can save a lot of time for both committers and reviewers.

Thanks!

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: 
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe<http://openstack-dev-requ...@lists.openstack.org?subject:unsubscribe>
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



--
Best regards,
Anastasia Kuznetsova
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [mistral] Improving Mistral pep8 rules files to match Mistral guidelines

[ELISHA, Moshe (Moshe)]

Hi all,

Is it possible to add all / some of the special guidelines of Mistral (like 
blank line before return, period at end of comment, ...) to our pep8 rules file?

This can save a lot of time for both committers and reviewers.

Thanks!
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [oslo.db][sqlalchemy][mistral] Configuring default transaction isolation level

[ELISHA, Moshe (Moshe)]

Thank you, Gordon for your reply.

In your patch I see that you are setting the isolation_level to 
REPEATABLE_READ. We are trying to change the isolation level to READ_COMMITTED.
I like to refer you to this performance article[1] that determines that 
READ_COMMITTED is usually better than REPEATABLE_READ.

In addition, maybe an additional reason for your slowness is that you do it 
when you get the connection - so you do an extra DB command that sets the 
isolation level.
My patch is changing the default isolation level for all connections upfront.

[1] 
https://www.percona.com/blog/2015/01/14/mysql-performance-implications-of-innodb-isolation-modes/



From: gord chung [mailto:g...@live.ca]
Sent: Tuesday, December 08, 2015 2:50 PM
To: openstack-dev@lists.openstack.org
Subject: Re: [openstack-dev] [oslo.db][sqlalchemy][mistral] Configuring default 
transaction isolation level

we had this in ceilometer events. you can see it here: 
https://github.com/openstack/ceilometer/commit/898cd3d036c4358aa16f7b3e2028365dc9829213

note, that patch is removing it because it slowed everything way down because 
of locking. if you can avoid it, avoid it.
On 08/12/2015 7:28 AM, Renat Akhmerov wrote:
Hi,

Moshe, thanks a lot for bringing this up. I remember I tried to find a way to 
change isolation level per connection but also was unable to do that.

An advice from oslo team would be very helpful.

Renat Akhmerov
@ Mirantis Inc.



On 08 Dec 2015, at 13:41, ELISHA, Moshe (Moshe) 
<moshe.eli...@alcatel-lucent.com<mailto:moshe.eli...@alcatel-lucent.com>> wrote:

Hi,

We at Mistral want to move from the default transaction isolation level of 
REPEATABLE READ to READ COMMITTED as part of a bugfix[1].

I did not find a way to pass the isolation level to sqlachemy using oslo.db and 
the current solution is to use monkey-patching[2] that adds the 
"isolation_level" property.

Is there currently a better way to set the default isolation level?
If not - I will create a BP for it.

Thanks.

[1] https://review.openstack.org/#/c/253819
[2] https://review.openstack.org/#/c/253819/11/mistral/db/sqlalchemy/base.py

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: 
openstack-dev-requ...@lists.openstack.org<mailto:openstack-dev-requ...@lists.openstack.org>?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev





__

OpenStack Development Mailing List (not for usage questions)

Unsubscribe: 
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe<mailto:openstack-dev-requ...@lists.openstack.org?subject:unsubscribe>

http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



--

gord
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [oslo.db][sqlalchemy][mistral] Configuring default transaction isolation level

[ELISHA, Moshe (Moshe)]

Hi,

We at Mistral want to move from the default transaction isolation level of 
REPEATABLE READ to READ COMMITTED as part of a bugfix[1].

I did not find a way to pass the isolation level to sqlachemy using oslo.db and 
the current solution is to use monkey-patching[2] that adds the 
"isolation_level" property.

Is there currently a better way to set the default isolation level?
If not - I will create a BP for it.

Thanks.

[1] https://review.openstack.org/#/c/253819
[2] https://review.openstack.org/#/c/253819/11/mistral/db/sqlalchemy/base.py

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [mistral] bugfix for "Fix concurrency issues by using READ_COMMITTED" unveils / creates a different bug

[ELISHA, Moshe (Moshe)]

Hi all,

The current bugfix I am working on[1] have unveiled / created a bug.
Test "WorkflowResumeTest.test_resume_different_task_states" sometimes fails 
because "task4" is executed twice instead of once (See unit test output and 
workflow below).

This happens because task2 on-complete is running task4 as expected but also 
task3 executes task4 by mistake.

It is not consistent but it happens quite often - This happens if the unit test 
resumes the WF and updates action execution of task2 and finishes task2 before 
task3 is finished.
Scenario:


1.   Task2 in method on_action_complete - changes task2 state to RUNNING.

2.   Task3 in method on_action_complete - changes task2 state to RUNNING 
(before task2 calls _on_task_state_change).

3.   Task3 in "_on_task_state_change" > "continue_workflow" > 
"DirectWorkflowController ._find_next_commands" - it finds task2 because task2 
is in SUCCESS and processed = False and "_find_next_commands_for_task(task2)" 
returns task4.

4.   Task3 executes command to RunTask task4.

5.   Task2 in "_on_task_state_change" > "continue_workflow" > 
"DirectWorkflowController ._find_next_commands" - it finds task2 because task2 
is in SUCCESS and processed = False and "_find_next_commands_for_task(task2)" 
returns task4.

6.   Task2 executes command to RunTask task4.


[1] - https://review.openstack.org/#/c/253819/


If I am not mistaken - this issue also exists in the current code and my bugfix 
only made it much more often. Can you confirm?
I don't have enough knowledge on how to fix this issue...
For now - I have modified the test_resume_different_task_states unit test to 
wait for task3 to be processed before updating the action execution of task2.
If you agree this bug exist today as well - we can proceed with my bugfix and 
open a different bug for that issue.

Thanks.



[stack@melisha-devstack mistral(keystone_admin)]$ tox -e py27 -- 
WorkflowResumeTest.test_resume_different_task_states
...
==
FAIL: 
mistral.tests.unit.engine.test_workflow_resume.WorkflowResumeTest.test_resume_different_task_states
tags: worker-0
--
pythonlogging:'': {{{WARNING [oslo_db.sqlalchemy.utils] Id not in sort_keys; is 
sort_keys unique?}}}
stderr: {{{
/opt/stack/mistral/.tox/py27/local/lib/python2.7/site-packages/novaclient/v2/client.py:109:
 UserWarning: 'novaclient.v2.client.Client' is not designed to be initialized 
directly. It is inner class of novaclient. Please, use 
'novaclient.client.Client' instead. Related lp bug-report: 1493576
  _LW("'novaclient.v2.client.Client' is not designed to be "
}}}

stdout: {{{
Engine test case exception occurred: 4 != 5
Exception type: 

Printing workflow executions...

wb.wf1 [state=SUCCESS, output={u'__execution': {u'params': {}, u'id': 
u'2807dd99-ca6f-49d7-886d-7d3b79e1c49e', u'spec': {u'type': u'direct', u'name': 
u'wf1', u'tasks': {u'task4': {u'type': u'direct', u'name': u'task4', 
u'version': u'2.0', u'action': u'std.echo output="Task 4"'}, u'task2': 
{u'type': u'direct', u'name': u'task2', u'on-complete': [u'task4'], u'version': 
u'2.0', u'action': u'std.mistral_http url="http://google.com;'}, u'task3': 
{u'type': u'direct', u'name': u'task3', u'version': u'2.0', u'action': 
u'std.echo output="Task 3"'}, u'task1': {u'type': u'direct', u'name': u'task1', 
u'on-complete': [u'task3', u'pause'], u'version': u'2.0', u'action': u'std.echo 
output="Hi!"'}}, u'version': u'2.0'}, u'input': {}}, u'task4': u'Task 4', 
u'task3': u'Task 3', u'__tasks': {u'848c6e92-b1b1-4d54-b11d-c93cfb4fc88f': 
u'task2', u'00a546e7-8da9-4603-b6be-54d58b14c625': u'task1'}}]
 task2 [id=848c6e92-b1b1-4d54-b11d-c93cfb4fc88f, state=SUCCESS, 
published={}]
 task1 [id=00a546e7-8da9-4603-b6be-54d58b14c625, state=SUCCESS, 
published={}]
 task3 [id=8ce20324-7fba-4424-bcd2-1e0c9b27fd4a, state=SUCCESS, 
published={}]
 task4 [id=3758de43-9bc3-4ac9-b3f3-29eb543b16ef, state=SUCCESS, 
published={}]
 task4 [id=f12ee464-0ba5-48c7-8423-9f425a00e675, state=SUCCESS, 
published={}]
}}}

Traceback (most recent call last):
  File "mistral/tests/unit/engine/test_workflow_resume.py", line 321, in 
test_resume_different_task_states
self.assertEqual(4, len(wf_ex.task_executions))
  File 
"/opt/stack/mistral/.tox/py27/local/lib/python2.7/site-packages/testtools/testcase.py",
 line 350, in assertEqual
self.assertThat(observed, matcher, message)
  File 
"/opt/stack/mistral/.tox/py27/local/lib/python2.7/site-packages/testtools/testcase.py",
 line 435, in assertThat
raise mismatch_error
testtools.matchers._impl.MismatchError: 4 != 5
Ran 1 tests in 15.517s (+2.591s)
FAILED (id=301, failures=1 (+1))
error: testr failed (1)
ERROR: InvocationError: '/opt/stack/mistral/.tox/py27/bin/python setup.py testr 
--slowest --testr-args=WorkflowResumeTest.test_resume_different_task_states'

Re: [openstack-dev] [mistral] Planning and prioritizing session for Mitaka

[ELISHA, Moshe (Moshe)]

Great. That works for ALU folks.

-Original Message-
From: Renat Akhmerov [mailto:rakhme...@mirantis.com] 
Sent: Thursday, November 05, 2015 9:23 AM
To: OpenStack Development Mailing List (not for usage questions)
Subject: [openstack-dev] [mistral] Planning and prioritizing session for Mitaka

Team,

We’ve done a great job at the summit discussing our most hot topics within the 
project and a lot of important decisions were made. I would like to have though 
one more session in IRC to wrap up this by going over all the BPs/bugs we 
created in order to scope and prioritize them.

I’m proposing next Monday 9 Nov at 7.00 UTC. If you have other time options 
let’s communicate.

Thanks

Renat Akhmerov
@ Mirantis Inc.




__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [mistral] [heat] Mistral Workflow resource type - resource signal handling

[ELISHA, Moshe (Moshe)]

Hi,

I would like to propose a change in the behavior of the OS::Mistral::Workflow 
resource signal.

CURRENT:
The OS::Mistral::Workflow resource type is expecting the following request body 
on resource signal request:

{
  "input": {
...
  },
  "params": {
...
  }
}

The input section is optional and if exists it will be passed to the workflow 
execution as inputs
The params section is also optional and if exists it will be passed to the 
workflow execution as parameters.

The problem this approach creates is that external systems many times send a 
predefined body that you cannot control and it is obviously not in the format 
the resource is expecting.
So you basically have no way to pass the information from the request body to 
the workflow execution.


SUGGESTION:
OS::Mistral::Workflow will treat the root of the JSON request body as input 
parameters.
That way you will be able to use external systems by making sure your WF inputs 
are aligned with what the external system sends.

For example, if you try to put the WF alarm_url as a Ceilometer alarm action - 
Ceilometer will send a request similar to:

{
 "severity": "low",
 "alarm_name": "my-alarm",
 "current": "insufficient data",
 "alarm_id": "895fe8c8-3a6e-48bf-b557-eede3e7f4bbd",
 "reason": "1 datapoints are unknown",
 "reason_data": {
   "count": 1,
   "most_recent": null,
   "type": "threshold",
   "disposition": "unknown"
 },
 "previous": "ok"
}

The WF could get this info as input if it will be defined like so:

  my_workflow:
type: OS::Mistral::Workflow
properties:
  input:
current: !!null
alarm_id: !!null
reason: !!null
previous: !!null
severity: !!null
alarm_name: !!null
reason_data: !!null


The (least used) "params" section can be passed in an custom HTTP header and 
the OS::Mistral::Workflow will read those from the header and pass it to the WF 
execution.
Remember, we are trying to solve the problem where you can't influence the 
request format - so in any case the signal will not get the params in the 
request body.
If the WF of the user must receive params, the user will always be able to 
create a wrapper WF with only inputs that starts the orig WF with inputs and 
params.

In order to make this non-backward compatible change, I suggest to add a 
property "params_alarm_http_header_name" to the OS::Mistral::Workflow. If null 
the params are expected to be in the body as today.
If not null - the request should have a header with that name and the value 
will be a string representing a JSON dict.

I would really like to hear your opinion and comments.

Thanks.
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [mistral][yaql] Addressing task result using YAQL function

[ELISHA, Moshe (Moshe)]

I agree with the task object as well.
It should be an object you can get info like the start time, end time, error 
code, ...

-Original Message-
From: Renat Akhmerov [mailto:rakhme...@mirantis.com] 
Sent: יום ב 07 ספטמבר 2015 19:59
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [mistral][yaql] Addressing task result using YAQL 
function


> On 07 Sep 2015, at 19:18, Stan Lagun  wrote:
> 
> I believe this is a good change. $.task_name requires you that $ be 
> pointing to a tasks dictionary. But in the middle of the query like 
> [1.2.3].select($ + 1)  "$" will change its value. With a function approach 
> you can write [1, 2, 3].select($ + task(taskName)). However the name "task" 
> looks confusing as to my understanding tasks may have attributes other than 
> result. It may make sense to use task(taskName).result instead.

Yes, I like the idea that task() should be more than just a result. Good point!

Renat Akhmerov
@ Mirantis Inc.


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [mistral] Adding screenshot when making UI changes

[ELISHA, Moshe (Moshe)]

Hey,

Following our discussion in the IRC, when pushing UI changes please upload a 
screenshot of the result and add a link in the commit message.
This will allow us to better understand the change and will also allow non-UI 
developers to comment on the change.
Having the screenshot link in the commit message will allow the developer to 
update the screenshot if there are visible changes as a result of the reviews.

If the UI change is very minor or it is only infra and there are no visible 
changes - use Screenshot: N/A.

You can see an example at the recent Task details overview screen push [1]

[1] https://review.openstack.org/#/c/212489/
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [mistral] [murano] An online YAQL evaluator

[ELISHA, Moshe (Moshe)]

Hey,

A couple of weeks ago we had a Python-oriented Hackathon at Alcatel-Lucent 
CloudBand.
We divided into groups and each group had to think of an original / innovative 
/ creative idea to be completed in 24 hours.

My group decided to create an online YAQL evaluator. Although we did not win 
first place, we got really good feedback and decided to publish it [1].

Some features:
* Provide a YAML and a YAQL and view the evaluation result.
* A catalog of common OpenStack API responses.
* YAQL auto complete (very basic for now)
* Currently using yaql-0.2.7 - we will upgrade it to yaql-1.0 once Mistral and 
Murano will upgrade as well.

All the source code of the project is available on GitHub [2].

Hope it will come as useful for you as it is for us.

Best regards.

[1] - http://yaqluator.comhttp://yaqluator.com/
[2] - https://github.com/ALU-CloudBand/yaqluator

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [mistral] A new BP - Support large datasets being passed

[ELISHA, Moshe (Moshe)]

I agree. We will try to allocate someone soon for this but it seems that all 
are very busy.

With the use cases I had encountered – the columns listed in the BP were OK. Do 
you have any suggestion for other columns?

Another important thing – when I changed the columns just for testing purposes 
– I did not get any “Data too long” anymore but I did get a segmentation fault:

2015-07-14 18:13:30.750 24152 INFO mistral.api.controllers.v2.execution [-] 
Fetch executions
/bin/bash: line 19: 24152 Segmentation fault  (core dumped) 
/usr/local/bin/mistral-server --config-file /etc/mistral/mistral.conf
mistral failed to start

There is probably an out of memory error or something. This should be addressed 
in the BP.


From: Renat Akhmerov [mailto:rakhme...@mirantis.com]
Sent: יום ד 15 יולי 2015 09:06
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [mistral] A new BP - Support large datasets being 
passed

Moshe, thanks for addressing this.

I think we need to implement this BP in this cycle (actually asap) as I think 
it can potentially create problems with real usage now. So I fully support it. 
One thing we need to do is to see what other columns (besides listed out in BP) 
should also be transformed.


Renat Akhmerov
@ Mirantis Inc.



On 14 Jul 2015, at 23:42, ELISHA, Moshe (Moshe) 
moshe.eli...@alcatel-lucent.commailto:moshe.eli...@alcatel-lucent.com wrote:

Hey,

I have created a new BP “Support large datasets being passed” [1].
You comments / suggestion / approval are most welcome.

[1] - https://blueprints.launchpad.net/mistral/+spec/support-large-datasets
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: 
openstack-dev-requ...@lists.openstack.orgmailto:openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [mistral] A new BP - Support large datasets being passed

[ELISHA, Moshe (Moshe)]

Hey,

I have created a new BP Support large datasets being passed [1].
You comments / suggestion / approval are most welcome.

[1] - https://blueprints.launchpad.net/mistral/+spec/support-large-datasets
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [mistral] Best practices for the DB maintanence in production

[ELISHA, Moshe (Moshe)]

Hey,

We are planning to use Mistral in production in the next few months.

We noticed that having even a simple workflow with a cron-trigger (For example 
monitor and heal workflow) can create large amounts of data in the DB (MariaDB).

Does Mistral have a mechanism / configuration of automatic deletion of old 
executions?
What is the best practice to handle this type of challenge?

Thanks.
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [mistral] Best practices for the DB maintanence in production

[ELISHA, Moshe (Moshe)]

Thanks, Renat.

I also believe the right place to do it is in Mistral.
I will create a blueprint and we will discuss the details in the spec.

Thanks.


From: Renat Akhmerov [mailto:rakhme...@mirantis.com]
Sent: יום ה 02 יולי 2015 12:34
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [mistral] Best practices for the DB maintanence in 
production

Hi Elisha,

Currently Mistral doesn’t support any expiration policies for 
workflow/task/action runtime objects. It keeps them forever until someone 
deletes them manually.

I see the following ways of addressing your need:

  *   Implement some cleanup component within Mistral (how to call it?) using 
its Scheduler component to periodically query and delete objects based on a 
criteria provided in a config.
  *   Just implement something on top of Mistral API to do the same. The cons 
of this approach is that Mistral now doesn’t provide any flexible mechanism to 
do criteria-based search of its objects. There’s an adjacent BP for that [1]. 
Generally, there’s a number of things in Mistral API we are not satisfied with 
and we’ve been planning to design and suggest API v3 for Mistral to support 
those features (don’t confuse with DSL v3, there’s no plan for now to implement 
a new backwards incompatible DSL). So this option may not be effective from 
performance perspective.

I think it deserves its own blueprint so that we can discuss nuances.

[1] https://blueprints.launchpad.net/mistral/+spec/mistral-items-filtering

Renat Akhmerov
@ Mirantis Inc.



On 02 Jul 2015, at 13:37, ELISHA, Moshe (Moshe) 
moshe.eli...@alcatel-lucent.commailto:moshe.eli...@alcatel-lucent.com wrote:

Hey,

We are planning to use Mistral in production in the next few months.

We noticed that having even a simple workflow with a cron-trigger (For example 
monitor and heal workflow) can create large amounts of data in the DB (MariaDB).

Does Mistral have a mechanism / configuration of automatic deletion of old 
executions?
What is the best practice to handle this type of challenge?

Thanks.
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: 
openstack-dev-requ...@lists.openstack.orgmailto:openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova][heat] Autoscaling parameters blueprint

[ELISHA, Moshe (Moshe)]

I am familiar of the removal policies. Thanks!

Our use case for parameters on scale out is as follows:

Every server has a unique index that identifies it.
The first server has an index of 1, the second has an index of 2, etc.
The index of each server must exist prior to the configuration phase of the 
server.

This use case is an outcome of a virtualization process for an NFV application.
In the past this application was scaled manually by adding physical cards into 
slots - the index is the slot number.
In order to allow a smooth and fast transition of the app into the cloud - the 
requirement is to stay with the same architecture.


The current suggested solution is as follows:

The HOT will be created with an AutoScalingGroup and two ScalingPolicies for 
scale out and scale in.
Like many other NFV applications, this application also has a Life Cycle 
Manager of its own that monitors and decides when to scale.
When scale is needed, the LCM will invoke the alarm_url exposed by these 
ScalingPolicies while providing the server index for the newly created server.

The index is just one example of a parameter needed at scale out - there can be 
others.
Much more design is needed when the desired_capacity  1 or  the 
scaling_adjustment  1 or in percentage but let's first agree that the use case 
is OK.


-Original Message-
From: Steven Hardy [mailto:sha...@redhat.com] 
Sent: Wednesday, March 11, 2015 12:39 PM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [nova][heat] Autoscaling parameters blueprint

On Wed, Mar 11, 2015 at 09:01:04AM +, ELISHA, Moshe (Moshe) wrote:
Hey,
 
 
 
Can someone please share the current status of the Autoscaling signals to
allow parameter passing for UserData blueprint -
 https://blueprints.launchpad.net/heat/+spec/autoscaling-parameters.

This is quite old, and subsequent discussions have happened which indicate a 
slightly different approach, e.g this thread here where I discuss approaches to 
signalling an AutoScalingGroup to remove a specific group member.  As Angus has 
noted, ResourceGroup already allows this via a different interface.

http://lists.openstack.org/pipermail/openstack-dev/2014-December/053447.html

We have a very concrete use case that require passing parameters on scale
out.
 
What is the best way to revive this blueprint?

Probably the first thing is to provide a more detailed description of your 
use-case.

I'll try to revive the AutoScalingGroup signal patch mentioned in the thread 
above this week, it's been around for a while and is probably needed for any 
interface where we pass data in to influence AutoScalingGroup adjustment 
behaviour asynchronously (e.g not via the template definition).

https://review.openstack.org/#/c/143496/

Steve

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova][heat] Autoscaling parameters blueprint

[ELISHA, Moshe (Moshe)]

This is a very nice idea but I’m afraid that the index is not the only 
parameter that is different between the instances.
I will write down the full use case so we could continue the discussion. Thanks!


From: Pavlo Shchelokovskyy [mailto:pshchelokovs...@mirantis.com]
Sent: יום ד 11 מרץ 2015 14:59
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [nova][heat] Autoscaling parameters blueprint

Hi,

as you have a separate monitoring solution (not Ceilometer), it seems you can 
use ResourceGroup instead of AutoscalingGroup and issue heatclient/rest calls 
to do a stack-update with desired size of the group when needed. The group 
members will be numbered, and as already said you can also control the removal. 
One possible downside is that AFAIR the numbers would not be reused on 
subsequent scale-down/ups.

Best regards,

Pavlo Shchelokovskyy
Software Engineer
Mirantis Inc
www.mirantis.comhttp://www.mirantis.com

On Wed, Mar 11, 2015 at 2:46 PM, ELISHA, Moshe (Moshe) 
moshe.eli...@alcatel-lucent.commailto:moshe.eli...@alcatel-lucent.com wrote:
I am familiar of the removal policies. Thanks!

Our use case for parameters on scale out is as follows:

Every server has a unique index that identifies it.
The first server has an index of 1, the second has an index of 2, etc.
The index of each server must exist prior to the configuration phase of the 
server.

This use case is an outcome of a virtualization process for an NFV application.
In the past this application was scaled manually by adding physical cards into 
slots - the index is the slot number.
In order to allow a smooth and fast transition of the app into the cloud - the 
requirement is to stay with the same architecture.


The current suggested solution is as follows:

The HOT will be created with an AutoScalingGroup and two ScalingPolicies for 
scale out and scale in.
Like many other NFV applications, this application also has a Life Cycle 
Manager of its own that monitors and decides when to scale.
When scale is needed, the LCM will invoke the alarm_url exposed by these 
ScalingPolicies while providing the server index for the newly created server.

The index is just one example of a parameter needed at scale out - there can be 
others.
Much more design is needed when the desired_capacity  1 or  the 
scaling_adjustment  1 or in percentage but let's first agree that the use case 
is OK.


-Original Message-
From: Steven Hardy [mailto:sha...@redhat.commailto:sha...@redhat.com]
Sent: Wednesday, March 11, 2015 12:39 PM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [nova][heat] Autoscaling parameters blueprint

On Wed, Mar 11, 2015 at 09:01:04AM +, ELISHA, Moshe (Moshe) wrote:
Hey,



Can someone please share the current status of the Autoscaling signals to
allow parameter passing for UserData blueprint -
 https://blueprints.launchpad.net/heat/+spec/autoscaling-parameters.

This is quite old, and subsequent discussions have happened which indicate a 
slightly different approach, e.g this thread here where I discuss approaches to 
signalling an AutoScalingGroup to remove a specific group member.  As Angus has 
noted, ResourceGroup already allows this via a different interface.

http://lists.openstack.org/pipermail/openstack-dev/2014-December/053447.html

We have a very concrete use case that require passing parameters on scale
out.

What is the best way to revive this blueprint?

Probably the first thing is to provide a more detailed description of your 
use-case.

I'll try to revive the AutoScalingGroup signal patch mentioned in the thread 
above this week, it's been around for a while and is probably needed for any 
interface where we pass data in to influence AutoScalingGroup adjustment 
behaviour asynchronously (e.g not via the template definition).

https://review.openstack.org/#/c/143496/

Steve

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: 
openstack-dev-requ...@lists.openstack.org?subject:unsubscribehttp://openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: 
openstack-dev-requ...@lists.openstack.org?subject:unsubscribehttp://openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [nova][heat] Autoscaling parameters blueprint

[ELISHA, Moshe (Moshe)]

Hey,

Can someone please share the current status of the Autoscaling signals to 
allow parameter passing for UserData blueprint -  
https://blueprints.launchpad.net/heat/+spec/autoscaling-parameters.

We have a very concrete use case that require passing parameters on scale out.
What is the best way to revive this blueprint?

Thanks.

Moshe Elisha.
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [heat] Can heat automatically create a flavor as part of stack creation?

[ELISHA, Moshe (Moshe)]

Hello,

I am wondering if instead of being obligated to use an existing flavor, I could 
declare a flavor (with its properties) inside Heat template and let Heat create 
the flavor automatically?
Similar to the ability to create networks as part of the template.

Thanks.
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [heat] Heat API v2 - Removal of template_url?

[ELISHA, Moshe (Moshe)]

Hey,

I really liked the v2 Heat API (as proposed in Create a new v2 Heat 
APIhttps://blueprints.launchpad.net/heat/+spec/v2api) and I think it makes a 
lot of sense.

One of the proposed changes is to Remove template_url from the request POST, 
so the template will be passed using the template parameter in the request 
body.

Could someone please elaborate how exactly Heat Orchestration Templates written 
in YAML will be embedded in the body?

As I understand the YAML template should be inserted as string otherwise JSON 
parsers will not be able to parse the JSON body.
If indeed the template is inserted as string, as far as I know, JSON does not 
support multiline strings and the available workarounds are not so pretty and 
require escaping.
The escaping issue gets more complicated when UserData is used in the YAML.

Will the template_url be removed and if so how will the template contain 
the YAML template?

Thanks.
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev