Re: [openstack-dev] [murano] Where can we find an image with Murano Agent

2017-11-27 Thread Stan Lagun
Hi Jerry,

Previously these images were available at apps.openstack.org. But since
this site is no longer available, the only way to get the images that I
know is to build them by yourself.
Here is how you can build most basic ubuntu+murano-agent image:
https://github.com/openstack/murano-agent/tree/master/contrib/elements
Some of the applications in the murano-apps repo require an image with
additional elements. They all come with instructions on how to build the
image.

But, if you are on a recent murano version, you can try to go with a bare
ubuntu cloud image since murano should be capable to install agent through
the cloud-init without backing it into the image.


Sincerely yours,
Stan Lagun
Principal Software Engineer @ Mirantis

<sla...@mirantis.com>

On Fri, Nov 24, 2017 at 11:05 AM, Sun, Yicheng (Jerry) <
jerry@windriver.com> wrote:

> We are trying to test our Murano deployment.
>
> We are working with the Pike version of Murano.
>
>
>
> Do you know where we can get an image with Murano Agent?
>
> The community application catalog was taken down.
>
> We tried following the instructions here:
>
> https://murano.readthedocs.io/en/latest/image_builders/linux.html
>
> https://pypi.python.org/pypi/murano-agent/3.3.0
>
>
>
> We were not able to build ubuntu with Murano Agent.
>
>
>
> Thanks in advance,
>
> Jerry
>
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [murano] MuranoPL types question

2017-03-30 Thread Stan Lagun
Hi Paul,

> Here both key and value appear to be a string (note, I can't confirm this
as the typeinfo function doesn't appear to be available in Instance.yaml
for some reason... perhaps I'm using it wrong)

They are not strings. Reporter converts everything that is passed to it
into string by doing unicode(obj). What you see is the Python string
representation of lists and dicts, where every unicode string is prefixed
with "u".
Typinfo function works on objects (instances of MuranoPL classes), not on
primitive types like strings, dicts and lists.

> 1) Why is the content of this 'get_attr' coming from heat being stored in
the stack outputs as a string,
it is stored exactly as it comes from Heat. In theory conversion to string
could happen on Heat side, but most likely it just the output of reporter
made it look like this

> 2) Is there a way I can cast this to a list of dicts or similar structure
so it can be iterated as expected?
It's hard to answer without seeing your code and how you got the results
that you provided.

>  when I access this from a sample app, I end up with a list of strings,
shown by $reporter as: ...

Curly braces in the output indicate that this is not a list of strings but
rather the single dict converted to a string by reporter. So what you see
is the value of that 'vol-7c8ee61f-a444-46a1-ad70-fc6fce6a7b56-attachments'
output, which sounds like what you're wanted it to be. In MuranoPL you can
have very detailed contracts: not just [] (any list) but something like
Contract:
  - key1: string().notNull()
key2: int().notNull()

which is a list of dicts with contract on each dict entry. If you don't get
contract violation exception, you can be sure that the list contains list
of dicts with appropriate keys/values rather than list of strings or
anything else


Sincerely yours,
Stan Lagun
Principal Software Engineer @ Mirantis

<sla...@mirantis.com>

On Thu, Mar 30, 2017 at 10:17 AM, Paul Bourke <paul.bou...@oracle.com>
wrote:

> Hi Stan,
>
> I had a quick(hopefully) question about MuranoPL that I hope you might be
> able to help with, Felipe had mentioned you are very knowledgeable in this
> area. If you don't have time please disregard!
>
> I'm working on a patch for the Murano core library to make volume
> attachment info available, which is available at
> https://review.openstack.org/451909. It's mostly working, but I'm having
> some issues getting the types correct in MuranoPL to make this info
> consumable by end users.
>
> The attachment info from a sample run in the stack $outputs looks like
> this (taken from the dashboard using $reporter)
>
> u'vol-7c8ee61f-a444-46a1-ad70-fc6fce6a7b56-attachments':
> u"[{u'server_id': u'2891067b-e7bb-4ab9-a759-9e81096c0682',
> u'attachment_id': u'7456a4b0-3abd-48a0-a0cb-f3fa0f2706bb',
> u'attached_at': u'2017-03-30T16:51:57.00', u'host_name': None,
> u'volume_id': u'373e4a5a-46b6-4091-82d6-b3dba62d552b', u'device':
> u'/dev/vda', u'id': u'373e4a5a-46b6-4091-82d6-b3dba62d552b'}]"
>
> Here both key and value appear to be a string (note, I can't confirm this
> as the typeinfo function doesn't appear to be available in Instance.yaml
> for some reason... perhaps I'm using it wrong)
>
> My goal is to then set this into a property of CinderVolume.yaml, with a
> Contract of '[]'. However, when I access this from a sample app, I end up
> with a list of strings, shown by $reporter as:
>
> (u"[{u'server_id': u'2891067b-e7bb-4ab9-a759-9e81096c0682',
> u'attachment_id': u'f5bd50ab-4040-4e2b-8b50-790781d9bc37',
> u'attached_at': u'2017-03-30T16:51:59.00', u'host_name': None,
> u'volume_id': u'ed7eb535-9e81-4c97-b063-86936d4ee306', u'device':
> u'/dev/vdb', u'id': u'ed7eb535-9e81-4c97-b063-86936d4ee306'}]",)
>
> So my question is:
>
> 1) Why is the content of this 'get_attr' coming from heat being stored in
> the stack outputs as a string, and
>
> 2) Is there a way I can cast this to a list of dicts or similar structure
> so it can be iterated as expected?
>
> Any tips much appreciated.
>
> Thanks,
> -Paul
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Murano] Newton version of Murano fails trying to auto-import a Liberty version of io.murano.applications.zip

2017-03-08 Thread Stan Lagun
It is probably that the package io.murano.applications is not present in
the app catalog. io.murano and io.murano.applications are the packages that
are bundled with Murano and usually should be preinstalled. You can obtain
them here:
https://github.com/openstack/murano/tree/stable/newton/meta

Regards,
Stan

On March 7, 2017 at 11:24:48 AM, Waines, Greg (greg.wai...@windriver.com)
wrote:

I am running a NEWTON version of Murano.



When I try to import a package that is using io.murano.applications classes
(e.g. SingleServerApplication),

it seems to try to auto-import the liberty version of
io.murano.applications.zip … and fails.



Is this a bug ?   Is it suppose to be using NEWTON version of
io.murano.applciations.zip ?  Actually can’t find NEWTON version, is the
URL wrong ?



[wrsroot@controller-0 newDemo(keystone_tenant1)]$ murano package-import
com.wrs.titanium.murano.examples.afwDemo

Error Got non-ok status(404) while connecting to
http://apps.openstack.org/api/v1/murano_repo/liberty/apps/io.murano.applications.zip
occurred while parsing package io.murano.applications, required by
com.wrs.titanium.murano.examples.afwDemo package

Inspecting required images

Importing package com.wrs.titanium.murano.examples.afwDemo

+--+-+--+-++---+-+-+

| ID   | Name| FQN
| Author  | Active | Is
Public | Type| Version |

+--+-+--+-++---+-+-+

| e7a08d26300e4812b9a5d0f91b2ad430 | APP FW Titanium Murano Demo App |
com.wrs.titanium.murano.examples.afwDemo | Greg Waines, Wind River | True   |
  | Application | |

+--+-+--+-++---+-+-+

[wrsroot@controller-0 newDemo(keystone_tenant1)]$



let me know,

Greg.






__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Murano] ERROR murano.common.engine [-] yaql.language.exceptions.NoMatchingMethodException: No method "values" for receiver None matches supplied arguments

2017-03-07 Thread Stan Lagun
HOWEVER … for future reference …

how do I go about seeing the internal HEAT STACK that Murano builds based
on the Murano Application / Packaging ?




You can do it be ether

1) Obtaining from the Heat since the stack is accessible and there is an
API and UI to get the template

2) Grep murano-engine logs for "Pushing" followed by ay a stack template
JSON. Note, that Murano usually does series of stack updates rather than
creates it at once. Therefore there going to be several such log records
for each deployment


Regards,

Stan
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Murano] ERROR murano.common.engine [-] yaql.language.exceptions.NoMatchingMethodException: No method "values" for receiver None matches supplied arguments

2017-03-06 Thread Stan Lagun
Greg,

based on the stack trace provided I see that it happened because the Heat
stack output had wrong value.

If you look at
https://github.com/openstack/murano/blob/stable/newton/meta/io.murano/Classes/resources/Instance.yaml#L178-L179

  - $netIdToIpsMap: $outputs.get(format('{0}-assigned-ips', $this.name))
  - $.ipAddresses: $netIdToIpsMap.values().flatten().distinct()  # <---
exception is thrown here

"No method "values" for receiver None..." means that $netIdToIpsMap is null
(None), which, in turn, means that
ether there in no output value with the name "$instanceName-assigned-ips"
or its value is null. In both cases it is not
something that can happen under normal conditions. Unfortunately I cannot
tell why it happened in your case.
I'd start with debugging of the Heat stack in attempt to find out why Heat
didn't set the output values

Regards,
Stan


On March 6, 2017 at 12:02:59 PM, Waines, Greg (greg.wai...@windriver.com)
wrote:

We are attempting to integrate a NEWTON-version of MURANO integrated into
our OpenStack solution.



With a very simple murano package/app, that basically just does a “
$.instance.deploy() “ in its deploy method of its main class,

we are getting the following traceback error (see end of email).



Any thoughts / guidance on potential issues ?



( NOTE … we had been trying to run with the secondary rabbit server, but
now (temporarily) just pointing murano at the core rabbit server to get
past some issues with not having the secondary rabbit server running. )



thanks in advance,

Greg.





2017-03-06 19:28:08.904 184898 ERROR murano.common.engine [-]

  yaql.language.exceptions.NoMatchingMethodException: No method "values"
for receiver None matches supplied arguments

  Traceback (most recent call last):

File
"/tmp/murano-packages-cache/io.murano/0.0.0/37e6dfe1415c4caa9150a53fc18506ef/Classes/Environment.yaml",
line 120:9 in method deploy of type io.murano.Environment

$.applications.pselect($.deploy())

File
"/tmp/murano-packages-cache/wrs.titanium.murano.examples.cgcs_guest/0.0.0/b7554a5f915748629da6e425005e1933/Classes/cgcs_guest.yaml",
line 41:13 in method deploy of type wrs.titanium.murano.examples.cgcs_guest

$.instance.deploy()

File
"/tmp/murano-packages-cache/io.murano/0.0.0/37e6dfe1415c4caa9150a53fc18506ef/Classes/resources/Instance.yaml",
line 194:9 in method deploy of type io.murano.resources.Instance

$this.endDeploy()

File
"/tmp/murano-packages-cache/io.murano/0.0.0/37e6dfe1415c4caa9150a53fc18506ef/Classes/resources/Instance.yaml",
line 179:24 in method endDeploy of type io.murano.resources.Instance

$netIdToIpsMap.values().flatten().distinct()

File "/usr/lib/python2.7/site-packages/murano/dsl/helpers.py", line 53
in method evaluate

return value(context)

File "/usr/lib/python2.7/site-packages/murano/dsl/yaql_expression.py",
line 85 in method __call__

return self._parsed_expression.evaluate(context=context)

File "/usr/lib/python2.7/site-packages/yaql/language/expressions.py",
line 165 in method evaluate

return self(utils.NO_VALUE, context, self.engine)

File "/usr/lib/python2.7/site-packages/yaql/language/expressions.py",
line 156 in method __call__

return super(Statement, self).__call__(receiver, context, engine)

File "/usr/lib/python2.7/site-packages/yaql/language/expressions.py",
line 37 in method __call__

return context(self.name, engine, receiver, context)(*self.args)

File "/usr/lib/python2.7/site-packages/yaql/language/contexts.py", line
65 in method 

data_context, use_convention, function_filter)

File "/usr/lib/python2.7/site-packages/yaql/language/runner.py", line
49 in method call

name, all_overloads, engine, receiver, data_context, args, kwargs)

File "/usr/lib/python2.7/site-packages/yaql/language/runner.py", line
117 in method choose_overload

args = tuple(arg_evaluator(i, arg) for i, arg in enumerate(args))

File "/usr/lib/python2.7/site-packages/yaql/language/runner.py", line
117 in method 

args = tuple(arg_evaluator(i, arg) for i, arg in enumerate(args))

File "/usr/lib/python2.7/site-packages/yaql/language/runner.py", line
113 in method 

and not isinstance(arg, expressions.Constant))

File "/usr/lib/python2.7/site-packages/yaql/language/expressions.py",
line 37 in method __call__

return context(self.name, engine, receiver, context)(*self.args)

File "/usr/lib/python2.7/site-packages/yaql/language/contexts.py", line
65 in method 

data_context, use_convention, function_filter)

File "/usr/lib/python2.7/site-packages/yaql/language/runner.py", line
49 in method call

name, all_overloads, engine, receiver, data_context, args, kwargs)



File "/usr/lib/python2.7/site-packages/yaql/language/runner.py", line
117 in method choose_overload

args = tuple(arg_evaluator(i, arg) for i, arg in enumerate(args))


Re: [openstack-dev] [Murano] Errors on $.instance.deploy() ... WHEN RUNNING WITHOUT MURANO-AGENT RABBIT

2017-03-06 Thread Stan Lagun
py",
line 37 in method __call__

return context(self.name, engine, receiver, context)(*self.args)

File "/usr/lib/python2.7/site-packages/yaql/language/contexts.py", line
65 in method 

data_context, use_convention, function_filter)

File "/usr/lib/python2.7/site-packages/yaql/language/runner.py", line
51 in method call

result = delegate()

File "/usr/lib/python2.7/site-packages/yaql/language/runner.py", line
142 in method 

return lambda: delegate()

File "/usr/lib/python2.7/site-packages/yaql/language/specs.py", line
341 in method func

six.iteritems(keyword_args)))

File "/usr/lib/python2.7/site-packages/murano/dsl/yaql_functions.py",
line 183 in method op_dot

return operator(ctx2, receiver, expr)

File "/usr/lib/python2.7/site-packages/yaql/language/yaqltypes.py",
line 323 in method func

new_name, engine, new_receiver, new_context)(*args, **kwargs)

File "/usr/lib/python2.7/site-packages/yaql/language/contexts.py", line
65 in method 

data_context, use_convention, function_filter)

File "/usr/lib/python2.7/site-packages/yaql/language/runner.py", line
51 in method call

result = delegate()

File "/usr/lib/python2.7/site-packages/yaql/language/runner.py", line
142 in method 

return lambda: delegate()

File "/usr/lib/python2.7/site-packages/yaql/language/specs.py", line
341 in method func

six.iteritems(keyword_args)))

File "/usr/lib/python2.7/site-packages/murano/dsl/yaql_functions.py",
line 183 in method op_dot

return operator(ctx2, receiver, expr)

File "/usr/lib/python2.7/site-packages/yaql/language/yaqltypes.py",
line 323 in method func

new_name, engine, new_receiver, new_context)(*args, **kwargs)

File "/usr/lib/python2.7/site-packages/yaql/language/contexts.py", line
65 in method 

data_context, use_convention, function_filter)

File "/usr/lib/python2.7/site-packages/yaql/language/runner.py", line
51 in method call

result = delegate()

File "/usr/lib/python2.7/site-packages/yaql/language/runner.py", line
142 in method 

return lambda: delegate()

File "/usr/lib/python2.7/site-packages/yaql/language/specs.py", line
341 in method func

six.iteritems(keyword_args)))

File
"/usr/lib/python2.7/site-packages/yaql/standard_library/system.py", line 34
in method op_dot

return expr(receiver)

File "/usr/lib/python2.7/site-packages/yaql/language/yaqltypes.py",
line 269 in method func

engine, args, kwargs)

File "/usr/lib/python2.7/site-packages/yaql/language/yaqltypes.py",
line 236 in method _call

result = value(receiver, context, engine)

File "/usr/lib/python2.7/site-packages/yaql/language/expressions.py",
line 37 in method __call__

return context(self.name, engine, receiver, context)(*self.args)

File "/usr/lib/python2.7/site-packages/yaql/language/contexts.py", line
65 in method 

data_context, use_convention, function_filter)

File "/usr/lib/python2.7/site-packages/yaql/language/runner.py", line
51 in method call

result = delegate()

File "/usr/lib/python2.7/site-packages/yaql/language/runner.py", line
142 in method 

return lambda: delegate()

File "/usr/lib/python2.7/site-packages/yaql/language/specs.py", line
341 in method func

six.iteritems(keyword_args)))

File "/usr/lib/python2.7/site-packages/murano/dsl/yaql_integration.py",
line 246 in method payload

__receiver, args, kwargs, __context, True), __context)

File "/usr/lib/python2.7/site-packages/murano/dsl/murano_method.py",
line 230 in method invoke

self, this, context, args, kwargs, skip_stub)

File "/usr/lib/python2.7/site-packages/murano/dsl/executor.py", line
163 in method invoke_method

result = call()

File "/usr/lib/python2.7/site-packages/murano/dsl/executor.py", line
153 in method call

yaql_engine, context, native_this)(*args, **kwargs)

File "/usr/lib/python2.7/site-packages/yaql/language/specs.py", line 60
in method func

return self.get_delegate(receiver, engine, context, args, kwargs)()

File "/usr/lib/python2.7/site-packages/yaql/language/specs.py", line
341 in method func

six.iteritems(keyword_args)))

File "/usr/lib/python2.7/site-packages/murano/dsl/yaql_integration.py",
line 192 in method payload

return body(__self.extension, *args, **kwargs)

File "/usr/lib/python2.7/site-packages/murano/engine/system/agent.py",
line 71 in method queue_name

return self._queue











*From: *Stan Lagun <sla...@mirantis.com>
*Reply-T

Re: [openstack-dev] [Murano] Errors on $.instance.deploy() ... WHEN RUNNING WITHOUT MURANO-AGENT RABBIT

2017-03-03 Thread Stan Lagun
Greg,

you're right in everything you said.

This was fixed in master in https://review.openstack.org/#/c/387993/ with
intent to fix tests and unblock the gate. But since those tests were
introduced in Ocata the fix was not backported to Newton.
So I've just did a partial packport of the root cause fix. Here it is:
https://review.openstack.org/#/c/441477/


Sincerely yours,
Stan Lagun
Principal Software Engineer @ Mirantis

<sla...@mirantis.com>

On Fri, Mar 3, 2017 at 11:49 AM, Waines, Greg <greg.wai...@windriver.com>
wrote:

> Looking for guidance on fixing the following error (end of email) that we
> are seeing when ‘deploying’ a very simple murano package/app on a Murano
> deployment WITHOUT the second rabbit server for murano-agent.
>
>
>
> So we have a NEWTON-version of MURANO integrated into our OpenStack
> solution.
>
>
>
> We have modified murano.conf:
>
>  …
>
>  # Disallow the use of murano-agent (boolean value)
>
>  disable_murano_agent = true
>
>  …
>
>
>
> We are NOT even running the second rabbit server for communication with
> murano-agent.
>
>
>
> With a very simple murano package/app, that basically just does a “
> $.instance.deploy() “ in its deploy method of its main class.
>
> *( NOTE:  I have tested this murano package/app on other Murano setups and
> it works … although those Murano setups had murano-agent and second rabbit
> enabled. )*
>
>
>
> And we get the following traceback error (see end of email).
>
>
>
> Questions:
>
> - I believe this is a valid configuration
>
> - but does anyone actually run Murano in this type of setup ?   I am
> guessing not many … so is this an upstream murano bug that I am seeing ? …
> that nobody else sees because basically no one runs in this mode ?
>
>
>
> - if this should work, and works in other setups,
>
>   … any guidance on how to resolve / debug what is going on here ?
>
>
>
>
>
> Thanks in advance,
>
> Greg.
>
>
>
>
>
>
>
> 2017-03-03 17:46:35.175 12568 ERROR murano.common.engine [-]
>
>   AttributeError: 'AgentListener' object has no attribute '_results_queue'
>
>   Traceback (most recent call last):
>
> File "/tmp/murano-packages-cache/io.murano/0.0.0/
> 3317e706ecd1417bb748361a6a3385d2/Classes/Environment.yaml", line 120:9 in
> method deploy of type io.murano.Environment
>
> $.applications.pselect($.deploy())
>
> File "/tmp/murano-packages-cache/wrs.titanium.murano.examples.
> VmFip_NoAppDeploy/0.0.0/829a861c408a4516b0589d04cce232
> 48/Classes/VmFip_NoAppDeploy.yaml", line 41:13 in method deploy of type
> wrs.titanium.murano.examples.VmFip_NoAppDeploy
>
> $.instance.deploy()
>
> File "/tmp/murano-packages-cache/io.murano/0.0.0/
> 3317e706ecd1417bb748361a6a3385d2/Classes/resources/Instance.yaml", line
> 193:9 in method deploy of type io.murano.resources.Instance
>
> $this.beginDeploy()
>
> File "/tmp/murano-packages-cache/io.murano/0.0.0/
> 3317e706ecd1417bb748361a6a3385d2/Classes/resources/Instance.yaml", line
> 131:28 in method beginDeploy of type io.murano.resources.Instance
>
> $.prepareUserData()
>
> File "/tmp/murano-packages-cache/io.murano/0.0.0/
> 3317e706ecd1417bb748361a6a3385d2/Classes/resources/LinuxMuranoInstance.yaml",
> line 14:19 in method prepareUserData of type io.murano.resources.
> LinuxMuranoInstance
>
> $.generateUserData()
>
> File "/tmp/murano-packages-cache/io.murano/0.0.0/
> 3317e706ecd1417bb748361a6a3385d2/Classes/resources/LinuxMuranoInstance.yaml",
> line 81:31 in method generateUserData of type io.murano.resources.
> LinuxMuranoInstance
>
> $region.agentListener.queueName()
>
> File "/usr/lib/python2.7/site-packages/murano/dsl/helpers.py", line
> 58 in method evaluate
>
> for d_key, d_value in six.iteritems(value))
>
> File "/usr/lib/python2.7/site-packages/yaql/language/utils.py", line
> 122 in method __init__
>
> self._d = dict(*args, **kwargs)
>
> File "/usr/lib/python2.7/site-packages/murano/dsl/helpers.py", line
> 58 in method 
>
> for d_key, d_value in six.iteritems(value))
>
> File "/usr/lib/python2.7/site-packages/murano/dsl/helpers.py", line
> 53 in method evaluate
>
> return value(context)
>
> File "/usr/lib/python2.7/site-packages/murano/dsl/yaql_expression.py",
> line 85 in method __call__
>
> return self._parsed_expression.evaluate(context=context)
>
> File "/usr/lib/python2.7/site-packag

Re: [openstack-dev] [heat][yaql] Deep merge map of lists?

2016-08-30 Thread Stan Lagun
There is a bug in how yaql interprets its 3rd argument (aggregator). It was
supposed to be a function to aggregate values within each group but instead
was applied to the outcome of groupBy. I submitted a fix for this:

https://review.openstack.org/363191. Though I'm not sure we can release new
yaql version that late in the Newton cycle. Meanwhile it is better to use
one of alternative solutions above so that it won't break after this patch
get merged.

Sincerely yours,
Stan Lagun
Principal Software Engineer @ Mirantis

<sla...@mirantis.com>

On Tue, Aug 30, 2016 at 12:46 PM, Thomas Herve <the...@redhat.com> wrote:

> On Tue, Aug 30, 2016 at 6:02 PM, Steven Hardy <sha...@redhat.com> wrote:
> > On Tue, Aug 30, 2016 at 04:10:47PM +0200, Jiří Stránský wrote:
> >>
> >> On 30.8.2016 10:17, Steven Hardy wrote:
> >>
> >> 
> >>
> >> > Yeah, that gets us closer, but we do need to handle more than one
> value
> >> > (list entry) per key, e.g:
> >> >
> >> >  data:
> >> >l:
> >> >  - "gnocchi_metricd_node_names": ["a0", "a1", "a2"]
> >> >"tripleo_packages_node_names": ["a0", "a1", "a2"]
> >> >  - "nova_compute_node_names": ["b0"]
> >> >"tripleo_packages_node_names": ["b0"]
> >> >
> >> > Output needs to be like:
> >> >
> >> >  "gnocchi_metricd_node_names": ["a0", "a1", "a2"]
> >> >  "tripleo_packages_node_names": ["a0", "a1", "a2", "b0"]
> >> >  "nova_compute_node_names": ["b0"]
> >> >
> >>
> >> Hoping this could do it:
> >>
> >> [stack@instack ~]$ cat yaq.yaml
> >> heat_template_version: 2016-10-14
> >>
> >> outputs:
> >>   debug:
> >> value:
> >>   yaql:
> >> expression: $.data.l.reduce($1.mergeWith($2))
> >> data:
> >>   l:
> >> - "gnocchi_metricd_node_names": ["a0", "a1", "a2"]
> >>   "tripleo_packages_node_names": ["a0", "a1", "a2"]
> >> - "nova_compute_node_names": ["b0"]
> >>   "tripleo_packages_node_names": ["b0"]
> >
> > Thanks for this!
> >
> > Unfortunately I dont think it works with more than two list items:
> >
> >   debug_tripleo2:
> > value:
> >   yaql:
> > expression: $.data.l.reduce($1.mergeWith($2))
> > data:
> >   l:
> > - "gnocchi_metricd_node_names": ["overcloud-controller-0",
> >   "overcloud-controller-1", "overcloud-controller-2"]
> >   "tripleo_packages_node_names": ["overcloud-controller-0",
> "overcloud-controller-1", "overcloud-controller-2"]
> > - "nova_compute_node_names": ["overcloud-compute-0"]
> >   "tripleo_packages_node_names": ["overcloud-compute-0"]
> >   "tripleo_packages_node_names2": ["overcloud-compute-0"]
> > - "ceph_osd_node_names": ["overcloud-cephstorage-0"]
> >   "tripleo_packages_node_names": ["overcloud-cephstorage-0"]
> >   "tripleo_packages_node_names2":
> ["overcloud-cephstorage-0"]
> >
> > $ heat output-show foo5 debug_tripleo2
> > stack output show" instead
> > Output error: can only concatenate tuple (not "list") to tuple
> >
> > I've not dug too deeply yet, but assuming that's a yaql error vs a heat
> bug
> > it looks like it won't work.
>
> I'd say it's a yaql bug (commented on the bug opened in Heat), that
> can work around using the list_merger argument:
>
> $.data.l.reduce($1.mergeWith($2, listMerger => $1.toList() + $2.toList()))
>
> Still slightly more elegant than the one I came up with.
>
> --
> Thomas
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Murano] Developer's productivity with MuranoPL

2016-08-24 Thread Stan Lagun
Hi Alex,

I'm two hands up for making developers life easier. However we should
remember that MuranoPL is not a scripting language and not a Python
replacement. There's very little point in writing Hello Word on MuranoPL.
And once you go for even the simplest app possible you will still have to
have OpenStack installed, manifest file written, class created etc. So the
only use case that I see is to type some simple statement that doesn't
involve any deployment or other I/O
and see how it works. In 80% of cases the same could be done in yaql CLI
tool. That's why I don't expect it to be that much of a productivity boost.
IMO debugger will do much better. And we could have an expression
evaluation functionality within debugger.

Btw, most of the steps you mentioned can be solved by an application stub
generator and configuration setting that will make murano take packages
from the local folder

Sincerely yours,
Stan Lagun
Principal Software Engineer @ Mirantis

<sla...@mirantis.com>

On Tue, Aug 16, 2016 at 7:31 AM, Alexander Tivelkov <ativel...@mirantis.com>
wrote:

> Hi folks,
>
> The developer's productivity and ease of use is critical for the language
> adoption, and I believe we have issues with these in MuranoPL. The things
> are slowly getting better (thanks to the guys who are bulding th MuranoPL
> syntax checker [1] which will be a very useful tool), but that's not
> enough: we need more tools to simplify dev's life.
>
> Let me provide a quick example: what does it take to run a "Hello World"
> in MuranoPL?
> To create such an introductory MuranoPL program the developer has to:
> 1) Install Murano with the rest of openstack (i.e. spin up devstack with
> Murano and Glare plugins)
> 2) Write the actual HelloWorld "program" (mind the absence of simple
> "print" function and the non-so-obvious need to extend
> io.murano.Application class for the example to be runnable as murano's
> entry point)
> 3) Write the UI.yaml file to provide the input object graph (even if there
> is no user-supplied paramters - we still have to have this file)
> 4) Write a manifest.yaml
> 5) zip the program, the ui and the manifest into a package
> 6) Upload the package to Glare
> 7) Create an environment in Murano dashboard
> 8) Add the demo app to the environment
> 9) Deploy the environment
>
> I hope I din't miss any step. Even excluding the time to spin up devstack
> (and the high probability that a newcomer will not do that properly from
> the first attempt) this is going to take at least 30 minutes. Even when the
> environment is set up, the whole "make code changes - recreate the package
> - reupload the package - recreate the environment - redepoy" cycle is
> cumbersome.
>
> What do we need is a simple and easy way to run MuranoPL code without the
> need to set up all the server-sides components, generate object model and
> interact with all the production-grade machinery murano has under the hood.
> To do something like:
>
>$ cat "- print('Hello, World')" > ./hello.yaml
>$ murano-pl ./hello.yaml
>
> Ideally there should be an interactive REPL-like shell, with smart
> indentation and code completion similar to the Python's (we have one for
> yaql, and so we should for MuranoPL).
>
> That is not a very hard thing to do, and it will simplify the developers'
> onboarding dramatically.
>
> So, I propose to start with a simple thing: to separate the MuranoPL
> interpreter (mostly the code located in murano.dsl package, plus some other
> stuff) from the rest of Murano. Put it into a standalone reporitory, so it
> may be packaged and distributed independently. The developers will just
> need to 'pip install murano-pl' to have the local interpreter without all
> the dependencies on murano api, engine, database etc. This new package may
> include the murano-pl test-runner (this tool is currently part of the main
> murano repo and is hard to use since it requires to have a valid config
> file which is not a proper option for a cli tool). Then we may include some
> other devloper-side tools, such as a murano-pl debugger (when we finally
> have one: this is a separate topic).
> Finally, we will need to remove the core library (murano.engine.system)
> package from the main murano repo and also make it a standalone
> repo/package with its own lifecycle (it would be very helpfull if we could
> release/update the core library more frequently). So the main murano repo
> (and appropriate package) will contain only the server-side murano
> components: the REST API, the engine, DB api, model and migrations etc.
>
> When this is done we may begin adding other developer's productivity
> tools: starting with debugger and then to various kinds of IDE
> integrations.

Re: [openstack-dev] NoMethodRegisteredException when deploying Murano environment

2016-08-24 Thread Stan Lagun
Hi,

most likely it is caused by using wrong version of the core library
package. Some parts of it are written in Python rather than in MuranoPL and
located inside murano source codes. So if you use newer version of core
library with older version of Murano you get this error. In particular it
cannot find
https://github.com/openstack/murano/blob/stable/mitaka/murano/engine/system/net_explorer.py#L195
method which is available since mitaka.

Sincerely yours,
Stan Lagun
Principal Software Engineer @ Mirantis

<sla...@mirantis.com>

On Wed, Aug 24, 2016 at 4:37 AM, Kirill Zaitsev <kzait...@mirantis.com>
wrote:

> Hi, it’s impossible to say without knowing what application you’re trying
> to deploy. Also this mailing list is not the best place for usage questions
> =) Please come to our IRC channel #murano — we would try to help you find
> what went wrong.
>
> --
> Kirill Zaitsev
> Murano Project Tech Lead
> Software Engineer at
> Mirantis, Inc
>
> On 24 août 2016 at 05:57:32, wuhan (wuhan9...@163.com) wrote:
>
> listNeutronExtensions
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [mistral] [murano] [yaql] yaqluator bug

2016-07-04 Thread Stan Lagun
Hi!

The issue with join is just a yaql bug that is already fixed. The problem
with yaqluator is that it doesn't use the latest yaql library.

Another problem is that it does't sets options correctly. As a result it is
possible to bring the site down with a query that produces endless
collection

Sincerely yours,
Stan Lagun
Principal Software Engineer @ Mirantis

<sla...@mirantis.com>

On Tue, Jun 28, 2016 at 9:46 AM, Elisha, Moshe (Nokia - IL) <
moshe.eli...@nokia.com> wrote:

> Hi,
>
> Thank you for the kind words, Alexey.
>
> I was able to reproduce your bug and I have also found the issue.
>
> The problem is that we did not create the parser with the engine_options
> used in the yaql library by default when using the CLI.
> Specifically, the "yaql.limitIterators" was missing… I am not sure that
> this settings should have this affect but maybe the Yaql guys can comment
> on that.
>
> If we will change yaqluator to use this setting it will mean that
> yaqluator will not be consistent with Mistral because Mistral is using YAQL
> without this engine option (If I use your example in a workflow, Mistral
> returns exactly like the yaqluator returns)
>
>
> Workflow:
>
> ---
> version: '2.0'
>
> test_yaql:
>   tasks:
> test_yaql:
>   action: std.noop
>   publish:
> output_expr: <% [1,2].join([3], true, [$1, $2]) %>
>
>
> Workflow result:
>
>
> [root@s53-19 ~(keystone_admin)]# mistral task-get-published
> 01d2bce3-20d0-47b2-84f2-7bd1cb2bf9f7
> {
> "output_expr": [
> [
> 1,
> 3
> ]
> ]
> }
>
>
> As Matthews pointed out, the yaqluator is indeed OpenSource and
> contributions are welcomed.
>
> [1]
> https://github.com/ALU-CloudBand/yaqluator/commit/e523dacdde716d200b5ed1015543d4c4680c98c2
>
>
>
> From: Dougal Matthews <dou...@redhat.com>
> Reply-To: "OpenStack Development Mailing List (not for usage questions)" <
> openstack-dev@lists.openstack.org>
> Date: Monday, 27 June 2016 at 16:44
> To: "OpenStack Development Mailing List (not for usage questions)" <
> openstack-dev@lists.openstack.org>
> Subject: Re: [openstack-dev] [mistral] [murano] [yaql] yaqluator bug
>
> On 27 June 2016 at 14:30, Alexey Khivin <akhi...@gmail.com> wrote:
>
>> Hello, Moshe
>>
>> Tomorrow I discovered yaqluator.com for myself! Thanks for the useful
>> tool!
>>
>> But suddenly I was said that the expression
>> [1,2].join([3], true, [$1, $2])
>> evaluated to [[1,3]] on the yaqluator
>>
>> A the same time this expression evaluated right when I using raw yaql
>> interpreter.
>>
>> Could we fix this issue?
>>
>> By the way, don't you want to make yaqluator opensource? If you would
>> transfer yaqluator to Openstack Foundation, then  community will be able to
>> fix such kind of bugs
>>
>
> It looks like it is open source, there is a link in the footer:
> https://github.com/ALU-CloudBand/yaqluator
>
>
>>
>> Thank you!
>> Best regards, Alexey Khivin
>>
>>
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Murano] Does Murano support version management?

2016-06-01 Thread Stan Lagun
Hi,

Murano does have partial support for version management. Murano
(deployment) engine can work with several versions of the same package. You
can even have them in single environment. I order to store several such
packages you must use Glare backend rather than old API backend and must
use recent Murano version. But there is no support in UI yet. So if you
plan to use CLI or API for automation then it is doable. However if you
want to choose package version in the dashboard then we are not there yet.

Also note that the package version is for the Murano application, not for
the software that this app installs. So if you want to let user choose what
version of the software to install this can be either one of application's
inputs or there can be several independent Murano apps (one per version +
library package to put common code there)

See [1] and [2] for more information.

[1]:
http://docs.openstack.org/developer/murano/draft/appdev-guide/murano_pl.html#versioning
[2]:
http://murano-specs.readthedocs.io/en/latest/specs/liberty/murano-versioning.html

Sincerely yours,
Stan Lagun
Principal Software Engineer @ Mirantis

<sla...@mirantis.com>

On Tue, May 31, 2016 at 10:20 PM, Jay Lau <jay.lau@gmail.com> wrote:

> Hi,
>
> I have a question for Murano: Suppose I want to manage two different
> version Spark packages, does Murano can enable me create one Application in
> application catalog but can enable me select different version spark
> packages to install?
>
> --
> Thanks,
>
> Jay Lau (Guangya Liu)
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [murano] Nominate Victor Ryzhenkin to Murano Core

2016-01-26 Thread Stan Lagun
+1!
Well deserved!
Sincerely yours,
Stan Lagun
Principal Software Engineer @ Mirantis



On Tue, Jan 26, 2016 at 5:20 AM, Yang, Lin A <lin.a.y...@intel.com> wrote:
> Glad to see this happened. :)
> Well deserved, Victor.
>
> Lin Yang
> @Intel
>
>> On Jan 23, 2016, at 01:33, Serg Melikyan <smelik...@mirantis.com> wrote:
>>
>> I would like to nominate Victor Ryzhenkin (freerunner on IRC) join to the 
>> Murano
>> core-reviewers team. He has been an active member of the Murano team
>> for some time now. You can check his review activity report here:
>> http://stackalytics.com/report/contribution/murano-group/90
>>
>> Team please vote for this change in reply to this message.
>> --
>> Serg Melikyan, Senior Software Engineer at Mirantis, Inc.
>> http://mirantis.com | smelik...@mirantis.com
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


[openstack-dev] [Murano] Murano actions authentication and accessability

2016-01-17 Thread Stan Lagun
Hello everyone!

As many of you know, Murano actions is the Murano way to do something
to applications that are already deployed using simple ReST API. For
example, health monitoring system may trigger an action on high CPU
load so that application could scale itself up.

However there are several obstacles that makes it hard to use actions
for such scenarios:

* There is no way to obtain a URI for the action from within MuranoPL
code. If application wants that health monitoring system to call
Murano back it need to provide it with a URI it could use to signal
the event (i.e. URI of the action).
However applications are not aware of Murano API server endpoint and
other required details which makes it hard to construct such a URI
from MuranoPL. This effectively puts a constraint that the action
caller must be murano-aware which is a rare case yet.

* Even if application manages somehow to provide external system with
an action URI, an attempt to call it will require a valid Keystone
token. It is because action URIs are served by Murano API which
requires it and especially because all changes made to applications
must be authorized. This may be a problem, especially when that
external system is not OpenStack-aware and cannot authenticate to
Keystone.


Recently I submitted a specification for review that addresses those
issues. It can be fond here: [1]. It is not perfect and it adds some
pieces of code to Murano that I wish could be handled by Keystone
alone. However it should solve most of use cases we have today.

I'd like to ask anyone who faced those issues or have an experience in
the field to read and share your feedback on the spec. We need to make
sure that it doesn't have serious flaws and does cover your use cases
before we proceed with implementation.
Thanks for your help!

[1]: https://review.openstack.org/#/c/251870/


Sincerely yours,
Stan Lagun
Principal Software Engineer @ Mirantis

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [mistral] [murano] [yaql] An online YAQL evaluator [was RE: [mistral] [murano] An online YAQL evaluator]

2016-01-16 Thread Stan Lagun
Very cool, indeed! Even myself (developer of yaql) often use yaqluator
instead of yaql CLI :)

Finally took a time to see source codes.
Few tips that you may find useful:
* The most expensive operation in yaql is YaqlFactory.create(). This
method builds a parser and it takes significant time (orders of
magnitude more then expression parsing). I see that you create
factory/parser
  on each evaluation. However parser was designed to be immutable and
thread safe so it may be reasonable to create it only once (+1 for
legacy mode) and reuse it across queries. This will make a big
difference
  if the site become popular
* You don't use engine options. There are number of useful settings
that allow to limit iterator length or memory usage. For example looks
like I killed the site by executing [1, 2].cycle() which returns
indefinite collection.
   Sorry for that. Another useful option is the one that converts sets
to lists on output. Without it set function cannot be used in
yaqluator. Also see [1]

[1]: https://review.openstack.org/#/c/268673/
Sincerely yours,
Stan Lagun
Principal Software Engineer @ Mirantis



On Wed, Jan 13, 2016 at 9:17 AM, Renat Akhmerov <rakhme...@mirantis.com> wrote:
> Very cool! It now even supports code assistance :)
>
> Moshe, can’t stop saying good words :) Amazing!
>
> Renat Akhmerov
> @ Mirantis Inc.
>
>
>
> On 22 Dec 2015, at 23:06, ELISHA, Moshe (Moshe)
> <moshe.eli...@alcatel-lucent.com> wrote:
>
> Hi all,
>
> Thank you for the kind words.
> Just wanted to let you know that yaqluator[1] was updated and now supports
> YAQL 1.0.2.
> There is also a checkbox there to work in “Legacy Mode”.
>
> Hope you will find it useful.
>
> [1] http://yaqluator.com/
>
>
> From: Renat Akhmerov [mailto:rakhme...@mirantis.com]
> Sent: Friday, August 14, 2015 11:51 AM
> To: OpenStack Development Mailing List (not for usage questions)
> Subject: Re: [openstack-dev] [mistral] [murano] An online YAQL evaluator
>
> I just read this thread so decided to add my 2 cents into the collection of
> opinions.
>
> Guys, I tried it out a couple of weeks ago (was told about it by one of my
> colleagues). This is really incredible! Especially given that you completed
> it in 24 hours :) I think as YAQL attracts more and more users it will be
> very handy tool. I am actually for improving it further.
>
> Thanks a lot! Looking forward to switch to yaql 1.0!
>
> Renat Akhmerov
> @ Mirantis Inc.
>
>
>
>
> On 05 Aug 2015, at 04:09, Stan Lagun <sla...@mirantis.com> wrote:
>
> Dmitry,
>
> yaql 1.0 has both str() and len() and much much more so there is no need to
> support them explicitly since Mistral is going to switch to yaql 1.0 and
> yaqluator.com is going to do the same
>
> Sincerely yours,
> Stan Lagun
> Principal Software Engineer @ Mirantis
>
>
>
> On Tue, Aug 4, 2015 at 8:55 PM, Dmitri Zimine <dzim...@stackstorm.com>
> wrote:
> Awesome! Really.
>
> Thank you folks for doing this.
>
> I am so much looking forward to moving it to 1.0 with more built-in
> functions and more power to extend it...
>
> Note that Mistral has a few extensions, like `str`, `len`, which are not in
> the scope of evaluator.
>
> DZ>
>
>
> On Aug 2, 2015, at 12:44 PM, Stan Lagun <sla...@mirantis.com> wrote:
>
>
> Guys, this is awesome!!!
>
> Happy to see yaql gets attention. One more initiative that you may find
> interesting is https://review.openstack.org/#/c/159905/
> This is an attempt to port yaql 1.0 from Python to JS so that the same can
> be done in browser
>
> Sincerely yours,
> Stan Lagun
> Principal Software Engineer @ Mirantis
>
>
>
> On Sun, Aug 2, 2015 at 5:30 PM, Nikolay Makhotkin <nmakhot...@mirantis.com>
> wrote:
> Hi guys!
>
> That's awesome! It is very useful for all us!
>
> --
> Best Regards,
> Nikolay
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.

Re: [openstack-dev] [Murano] How to debug a failed deployment?

2015-12-11 Thread Stan Lagun
Vahid,

The bug you mentioned was fixed recently:
https://review.openstack.org/#/c/255154/

Sincerely yours,
Stan Lagun
Principal Software Engineer @ Mirantis

<sla...@mirantis.com>

On Fri, Dec 11, 2015 at 10:46 PM, Vahid S Hashemian <
vahidhashem...@us.ibm.com> wrote:

> Hi Nikolay,
>
> Is there further information I could provide about the bug?
>
> Thanks.
> --Vahid
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [murano] Is there blueprint to add argument collection UI for action

2015-12-09 Thread Stan Lagun
Tony,

I forgot to mention that it possible to call actions with parameters using
API. It is just a UI that is missing.

Also I'd like to mention that any involvement of yours will be appreciated
- ideas, suggestions, commits or even if you take a lead over this feature
- everything will help to make it be available sooner than later.
Please let me know if I can help.

Sincerely yours,
Stan Lagun
Principal Software Engineer @ Mirantis

<sla...@mirantis.com>

On Mon, Dec 7, 2015 at 10:25 AM, WANG, Ming Hao (Tony T) <
tony.a.w...@alcatel-lucent.com> wrote:

> Stan,
>
>
>
> Thanks for your information very much.
>
> I’m looking forward for this feature! J
>
>
>
> Thanks,
>
> Tony
>
>
>
> *From:* Stan Lagun [mailto:sla...@mirantis.com]
> *Sent:* Saturday, December 05, 2015 4:52 AM
> *To:* WANG, Ming Hao (Tony T)
> *Cc:* OpenStack Development Mailing List (not for usage questions)
> *Subject:* Re: [murano] Is there blueprint to add argument collection UI
> for action
>
>
>
> Tony,
>
>
>
> Here is the blueprint:
> https://blueprints.launchpad.net/murano/+spec/action-ui
>
>
>
> The problem is much bigger then it seems to. Action parameters are very
> similar to regular application properties and thus there should be a
> dedicated dynamic UI form (think ui.yaml) for each action describing its
> inputs.
>
> We discussed it many times and each time the resolution was that it is
> better to do major refactoring of dynamic UI forms before introducing
> additional forms. The intention was to either simplify or completely get
> rid of ui.yaml because it is yet another DSL to learn. Most of the
> information from ui.yaml could be obtained by examining MuranoPL class
> properties. And what is missing could be added as additional metadata right
> to the classes. However a lot of work requires to do it properly (starting
> from new API that would be MuranoPL-aware). That's why we still have no
> proper UI for the actions.
>
>
>
> Maybe we should reconsider and to have many ui forms in single package
> until we have a better solution.
>
>
>
>
>
>
> Sincerely yours,
> Stan Lagun
> Principal Software Engineer @ Mirantis
>
>
>
> On Fri, Dec 4, 2015 at 5:42 AM, WANG, Ming Hao (Tony T) <
> tony.a.w...@alcatel-lucent.com> wrote:
>
> Dear Stan and Murano developers,
>
>
>
> The current murano-dashboards can add the action button for the actions
> defined in Murano class definition automatically, which is great.
>
> Is there any blueprint to add argument collection UI for the action?
>
> Currently, the murano-dashboards only uses environment_id + action_id to
> run the actions, and user has no way to provide action arguments from
> UI.
>
>
>
> Thanks in advance,
>
> Tony
>
>
>
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Murano] How to debug a failed deployment?

2015-12-04 Thread Stan Lagun
Vahid,

The main log file that is missing is murano-engine log. It is murano-engine
who does the deployment. There are also no agent logs because agents are on
VMs and none of them were created.

>From what I see from you screenshots deployment task doesn't even reaches
murano-engine.

There can be 2 possible reasons for this:

1) murano-engine is not running
2) incorrect configuration of oslo.messaging.

Murano API sends tasks to murano-engine through RabbitMQ using
oslo.messaging library. Settings in your config file differ from those in
mine. For example in my config file rabbit_host is in [DEFAULT] section
while in yours it is in [oslo_messaging_rabbit]. Maybe it is okay and my
config is outdated. Maybe you use newer version of oslo.messaging that has
different config or it can be configured in several ways. I'm not sure.

Sincerely yours,
Stan Lagun
Principal Software Engineer @ Mirantis

<sla...@mirantis.com>

On Fri, Dec 4, 2015 at 8:54 AM, Vahid S Hashemian <vahidhashem...@us.ibm.com
> wrote:

> Dmytro, Kate, Stan,
>
> Thank you for your responses. I'll try to provide more information about
> my environment and the deployment in this note.
> Hopefully, this will give you some clue about what the issue is with my
> setup.
>
>1. The murano config file I am using is at
>http://paste.openstack.org/show/480855/
>2. The HOT package I am trying to deploy is attached
>3. A sample of log entries I see in murano log is at
>http://paste.openstack.org/show/480854/
>4. No heat stack is created at all. As I mentioned in my previous
>message, it's as if murano cannot talk to heat.
>5. Heat engine logs are at http://paste.openstack.org/show/480856/
>6. Heat API logs are at http://paste.openstack.org/show/480857/
>7. Some screen shots are also attached that show the status of
>deployment more than 30 minutes after start. Existing heat stacks show that
>the template by itself was deployed to heat without an issue.
>8. I couldn't find murano agent logs. Please let me know where I can
>find them and I'll provide those too.
>
>
>
> Regards,
> --Vahid
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [murano] Is there blueprint to add argument collection UI for action

2015-12-04 Thread Stan Lagun
Tony,

Here is the blueprint:
https://blueprints.launchpad.net/murano/+spec/action-ui

The problem is much bigger then it seems to. Action parameters are very
similar to regular application properties and thus there should be a
dedicated dynamic UI form (think ui.yaml) for each action describing its
inputs.
We discussed it many times and each time the resolution was that it is
better to do major refactoring of dynamic UI forms before introducing
additional forms. The intention was to either simplify or completely get
rid of ui.yaml because it is yet another DSL to learn. Most of the
information from ui.yaml could be obtained by examining MuranoPL class
properties. And what is missing could be added as additional metadata right
to the classes. However a lot of work requires to do it properly (starting
from new API that would be MuranoPL-aware). That's why we still have no
proper UI for the actions.

Maybe we should reconsider and to have many ui forms in single package
until we have a better solution.



Sincerely yours,
Stan Lagun
Principal Software Engineer @ Mirantis

<sla...@mirantis.com>

On Fri, Dec 4, 2015 at 5:42 AM, WANG, Ming Hao (Tony T) <
tony.a.w...@alcatel-lucent.com> wrote:

> Dear Stan and Murano developers,
>
>
>
> The current murano-dashboards can add the action button for the actions
> defined in Murano class definition automatically, which is great.
>
> Is there any blueprint to add argument collection UI for the action?
>
> Currently, the murano-dashboards only uses environment_id + action_id to
> run the actions, and user has no way to provide action arguments from
> UI.
>
>
>
> Thanks in advance,
>
> Tony
>
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Murano] How to debug a failed deployment?

2015-12-03 Thread Stan Lagun
Vahid,

The only reason HOT deployment could last forever is that Heat stack never
enters Ready of Failed state. Please check what is the Heat stack status. I
guess it is not Murano issue
(except for the fact that we don't provide timeout for HOT deployment :) )

Sincerely yours,
Stan Lagun
Principal Software Engineer @ Mirantis

<sla...@mirantis.com>

On Thu, Dec 3, 2015 at 11:54 AM, Ekaterina Chernova <efedor...@mirantis.com>
wrote:

> Hi!
>
> Murano agent doesn't involved in the deployment of hot packages, that's
> why this setting doesn't help.
>
> Please attach agent logs and heat logs also (check if heat stack is
> created or not).
>
> Regards, Kate.
>
>
> Regards,
> Kate.
>
>
> On Thu, Dec 3, 2015 at 11:44 AM, Dmytro Dovbii <ddov...@mirantis.com>
> wrote:
>
>> Hi Vahid,
>>
>> Could you provide your logs anyway? Please publish them to
>> http://paste.openstack.org/ for example
>>
>> 2015-12-02 23:08 GMT+02:00 Vahid S Hashemian <vahidhashem...@us.ibm.com>:
>>
>>> Hello,
>>>
>>> I am trying to deploy a hello world HOT package in murano but the
>>> deployment runs forever and does not stop (does not succeed or fail).
>>> Stan advised me to modify the config setting below to set a timeout:
>>>
>>> # Time for waiting for a response from murano agent during the
>>> # deployment (integer value)
>>> agent_timeout = 300
>>>
>>> As you see I set the timeout to 300 seconds and restarted murano API,
>>> but even after 15 minutes my deployment is still going.
>>> I think my environment is not configured correctly for this to happen.
>>>
>>> I have devstack running on a VM, and my murano development is done on a
>>> separate VM.
>>> I bring up murano API and UI on my dev VM that successfully connect to
>>> my devstack VM.
>>>
>>> I am able to deploy the HOT yaml using heat without an issue.
>>> From what I can see it seems that murano is not able to talk to heat,
>>> because I don't see any stack being created as a result of my deployment.
>>>
>>> I have also enabled murano logging in the config file as below:
>>>
>>> # (Optional) Name of log file to output to. If no default is set,
>>> # logging will go to stdout. (string value)
>>> # Deprecated group/name - [DEFAULT]/logfile
>>> log_file = /opt/stack/logs/murano.log
>>>
>>> But no error  is reported while the deployment is in progress.
>>>
>>> Any tips on how to force the timeout, and also on how to resolve the
>>> deployment problem is appreciated.
>>>
>>> Regards,
>>> --Vahid
>>>
>>>
>>> __
>>> OpenStack Development Mailing List (not for usage questions)
>>> Unsubscribe:
>>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>>
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [murano]How to use Murano to transmit files to Mistral and execute scripts on Mistral

2015-12-01 Thread Stan Lagun
On Mon, Nov 30, 2015 at 5:07 AM, WANG, Ming Hao (Tony T) <
tony.a.w...@alcatel-lucent.com> wrote:

> For the object storage support, does Murano have any plan to support the
> auto-uploading function?


Murano has plans to support everything applications may need. It is just a
matter of priorities/schedule/time etc. It will happen much sooner with
your help/contributions.
Currently AFAIK there is no blueprint for this feature so nobody developing
it

Sincerely yours,
Stan Lagun
Principal Software Engineer @ Mirantis

<sla...@mirantis.com>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [murano]How to use Murano to transmit files to Mistral and execute scripts on Mistral

2015-11-28 Thread Stan Lagun
Hi Tony,

Technically your solution is possible. Murano agent can be installed
manually. All queue names are in agent's config file. Murano can talk to
that manually installed agent and put files to that server.

However I strongly advice not to do that. First of all it is not secure.
Murano agent was designed as a remote execution mechanism rather than a
tool to transfer files. Also it was design to be installed on VM
rather that on a shared server. Currently there is no authentication for
agents. Although it is not truly secure it still provides significant level
of security because to make use of the agent you need to know its queue name
which is a long random string that only murano engine and agent know and
AMQP doesn't provide an API to query list of queue names. But if you
install agent manually there will be a single queue. And all applications
that
need to access that agent will also need to know queue name. Thus it can be
easily be compromised and attackers will gain ability to execute arbitrary
code on the nova server.

It is also doesn't seem like a right solution. There should not be such
specific requirements like this to perform such trivial thing. Especially
if it involves using tool that was design for something else.

So what is the right solution?

I can see 2 possible ways to address this issue:

1. Improve Mistral API to support file attachments to its workbooks (if it
doesn't support it yet). Improve murano-mistral integration to support this
feature.
2. Implement integration of Murano with object storage. Thus it will be
possible from Murano workflow to upload file and provide Mistral with a URI
to it.

Solution #2 seem to be better. At least because it opens new possibilities
for all applications and not just those that use Mistral.

There is also a workaround that I can think of: make Murano application
create temporary VM with FTP server or something like that, put file into
it and give Mistral a URI it could use to access the file.

Sincerely yours,
Stan Lagun
Principal Software Engineer @ Mirantis

<sla...@mirantis.com>

On Wed, Nov 18, 2015 at 1:06 PM, WANG, Ming Hao (Tony T) <
tony.a.w...@alcatel-lucent.com> wrote:

> Dear Alexander and Murano developers and testers,
>
>
>
> Any suggestion for this? J
>
>
>
> Thanks,
>
> Tony
>
>
>
> *From:* WANG, Ming Hao (Tony T)
> *Sent:* Tuesday, November 17, 2015 6:12 PM
> *To:* OpenStack Development Mailing List (not for usage questions)
> *Subject:* RE: [openstack-dev] [murano]How to use Murano to transmit
> files to Mistral and execute scripts on Mistral
>
>
>
> Alexander,
>
>
>
> Thanks for your response.
>
> During the workflow running stage, it may need to access some other
> artifacts.
>
> In my case, I use Mistral workflow to call Ansible playbook, and I need
> Murano to put Ansible playbook into right place on Mistral server so that
> Mistral workflow can find it.
>
>
>
> Thanks,
>
> Tony
>
>
>
> *From:* Alexander Tivelkov [mailto:ativel...@mirantis.com
> <ativel...@mirantis.com>]
> *Sent:* Tuesday, November 17, 2015 4:19 PM
> *To:* OpenStack Development Mailing List (not for usage questions)
> *Subject:* Re: [openstack-dev] [murano]How to use Murano to transmit
> files to Mistral and execute scripts on Mistral
>
>
>
> Hi Tony,
>
>
>
> Probably I am missing something, but why do you need Murano Agent to
> interact with Mistral? You can call Mistral APIs right from MuranoPL code
> being executed by Murano Engine. Murano's core library contains the 
> io.murano.system.MistralClient
> class ([1]) which may be used to upload and run mistral workflows.
>
>
>
> Please let me know if you need more details on this
>
>
>
> [1]
> https://github.com/openstack/murano/blob/master/murano/engine/system/mistralclient.py
>
>
>
> On Tue, Nov 17, 2015 at 5:07 AM WANG, Ming Hao (Tony T) <
> tony.a.w...@alcatel-lucent.com> wrote:
>
> Dear Murano developers and testers,
>
>
>
> I want to put some files that Mistral workflow needs into Murano package,
> and hope Murano can transmit them to Mistral before it calls Mistral
> workflow.
>
> The flow should be as following:
>
> 1.   User uploads one Murano package which includes both
> Murano artifacts and Mistral artifacts to Murano;
>
> 2.   Murano transmits the Mistral artifacts to Mistral,
> and Mistral does its work.
>
>
>
> After study, I thought muranoagent may be helpful and plan to install a
> muranoagent on the Mistral server since it can put files into nova server,
> and can run scripts on the nova server.
>
> After further study, I found muranoagent solution may be not feasible:
>
> 1.   muranoagent and m

Re: [openstack-dev] [Murano] 'NoMatchingFunctionException: No function "#operator_." matches supplied arguments' error when adding an application to an environment

2015-11-27 Thread Stan Lagun
Here is the full story:

real YAML that is generated is stored in murano database in packages table.
Murano Dashboard obtains form definitions in YAML from API. But to improve
performance it also caches them locally. And when it does it stores them in
Python pickle format [1] rather then original YAML, but doesn't change the
extension (actually this is a bug). That's why when you take yamls from
dashboard cache they doesn't look like a valid YAML though they contain the
same information and can be decrypted back (that's what I did to find the
error you had). But is is much easier to take real YAML from database though

[1]: https://docs.python.org/2/library/pickle.html

Sincerely yours,
Stan Lagun
Principal Software Engineer @ Mirantis

<sla...@mirantis.com>

On Fri, Nov 27, 2015 at 1:25 AM, Vahid S Hashemian <
vahidhashem...@us.ibm.com> wrote:

> Thanks Stan for the pointer.
>
> I removed the line that referred to the 'name' property and now my
> application is added to the environment without any errors.
> However, what I see in ui.yaml still doesn't look like YAML.
>
> I'm attaching samples again.
>
>
> Even for HOT packages the content is not YAML.
>
> Regards,
> --Vahid
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Murano] 'NoMatchingFunctionException: No function "#operator_." matches supplied arguments' error when adding an application to an environment

2015-11-26 Thread Stan Lagun
Vahid,

I see what is the problem.

You are generating UI form that looks like this:

  Forms:
-  group0:
fields: []
  Application:
name: $.group0.name
?:
  type: "io.murano.apps.generated.CsarHelloWorld"

  Version: 2.2


The problem in in "name: $.group0.name". Previously you removed the "name"
field from the form because it is provided automatically by the 2.2 form
format. However you did not removed the expression that was referring to it.

So just drop this expression from the form (and form generation code).
If you will need name during deployment time you always can obtain it using
name($this) YAQL expression as here:
https://github.com/openstack/murano/blob/master/contrib/plugins/cloudify_plugin/cloudify_applications_library/Classes/CloudifyApplication.yaml#L46

Hope this will help

Sincerely yours,
Stan Lagun
Principal Software Engineer @ Mirantis

<sla...@mirantis.com>

On Thu, Nov 26, 2015 at 3:23 AM, Vahid S Hashemian <
vahidhashem...@us.ibm.com> wrote:

> Hi Stan,
>
> Thanks for looking into those files.
>
> They are what were generated on my machine.
>
> I tried one more time, and this time used a HOT package (something that
> comes ootb) and noticed the ui.yaml file generated was still in the same
> format as the previous ones.
> I am attaching the full folder structure created for this HOT package
> under /tmp/muranodashboard-cache/apps/ along with the hot package itself
> that I imported.
>
>
>
> I'd appreciate it if you could take a look and let me know if you see
> something wrong. Thanks.
>
> Regards,
> --Vahid
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Murano] 'NoMatchingFunctionException: No function "#operator_." matches supplied arguments' error when adding an application to an environment

2015-11-25 Thread Stan Lagun
Vahid,

The files you attached doesn't look like a YAML at all

Sincerely yours,
Stan Lagun
Principal Software Engineer @ Mirantis

<sla...@mirantis.com>

On Wed, Nov 25, 2015 at 10:21 PM, Vahid S Hashemian <
vahidhashem...@us.ibm.com> wrote:

> Kate, Serg,
>
> Thank you very much for your quick responses.
>
> I am attaching the requested yaml file.
>
> For Hot2 plugin, the only difference with hot_package.py is that I changed
> the UI translation version to 2.2 and removed predefined_fields in
> _translate_ui_parameters method.
> The ui.yaml file is attached.
>
>
>
> The CSAR plugin is a work in progress. For example, the workflow method is
> not yet implemented.
> The ui.yaml file is attached.
>
>
> I'd like to make sure the issue is not on my side before I open a bug, as
> Kate suggested.
>
> If necessary, I can also send you the plugin python code.
>
> Thanks.
>
> Regards,
> --Vahid
>
>
>
>
> From:Serg Melikyan <smelik...@mirantis.com>
> To:"OpenStack Development Mailing List (not for usage questions)"
> <openstack-dev@lists.openstack.org>
> Date:11/25/2015 03:29 AM
> Subject:Re: [openstack-dev] [Murano]
> 'NoMatchingFunctionException: No function "#operator_." matches supplied
> arguments' error when adding an application to an environment
> --
>
>
>
> Hi Vahid,
>
> you can find generated UI definitions for the package here:
> /tmp/muranodashboard-cache/apps/.../ui/ui.yaml
>
> On Wed, Nov 25, 2015 at 12:54 PM, Ekaterina Chernova
> <efedor...@mirantis.com> wrote:
> >
> > Hi Vahid,
> >
> > Forms are rended after app is added to the environment in accordance
> with the UI definition.
> > UI definition contains yaql expressions and one of your expression is
> incorrect.
> > Please, share UI yaml so we can find out what's the problem.
> > You can also create a bug that it's not obvious which expression is
> failed.
> >
> >
> > Also, make sure that you have valid YAQL version installed (should
> correspond to the version in requirements.txt)
> > And you can ask for help in #murano channel.
> >
> > Regards,
> > Kate.
> >
> >
> > On Wed, Nov 25, 2015 at 4:39 AM, Vahid S Hashemian <
> vahidhashem...@us.ibm.com> wrote:
> >>
> >> Hi,
> >>
> >> I am working on the TOSCA CSAR plugin for murano and so far am able to
> successfully import an application definition archive of my CSAR example to
> murano.
> >> However, when I try to add the imported application to an environment I
> get this error from Murano Dashboard:
> >>
> >> DEBUG:muranodashboard.common.cache:Using cached value from
> /tmp/muranodashboard-cache/apps/91/4c2bfd5d504419a94a9affb7af809a/package_fqn.
> >> DEBUG:muranodashboard.catalog.views:Clearing forms data for application
> io.murano.apps.generated.CsarHelloWorld.
> >> DEBUG:muranodashboard.catalog.views:Clearing any leftover wizard step
> data.
> >> DEBUG:muranodashboard.common.cache:Using cached value from
> /tmp/muranodashboard-cache/apps/91/4c2bfd5d504419a94a9affb7af809a/ui/ui.yaml.
> >> DEBUG:muranodashboard.common.cache:Using cached value from
> /tmp/muranodashboard-cache/apps/91/4c2bfd5d504419a94a9affb7af809a/package_fqn.
> >> DEBUG:muranodashboard.dynamic_ui.services:Using data {} for app
> io.murano.apps.generated.CsarHelloWorld
> >> DEBUG:muranodashboard.dynamic_ui.services:Using in-memory forms for app
> io.murano.apps.generated.CsarHelloWorld
> >> DEBUG:muranodashboard.common.cache:Using cached value from
> /tmp/muranodashboard-cache/apps/91/4c2bfd5d504419a94a9affb7af809a/ui/ui.yaml.
> >> DEBUG:muranodashboard.common.cache:Using cached value from
> /tmp/muranodashboard-cache/apps/91/4c2bfd5d504419a94a9affb7af809a/package_fqn.
> >> DEBUG:muranodashboard.dynamic_ui.services:Using data {} for app
> io.murano.apps.generated.CsarHelloWorld
> >> DEBUG:muranodashboard.dynamic_ui.services:Using in-memory forms for app
> io.murano.apps.generated.CsarHelloWorld
> >> INFO:muranodashboard.dynamic_ui.forms:Creating form workflowManagement
> >> INFO:muranodashboard.dynamic_ui.forms:Creating form group0
> >> DEBUG:muranodashboard.api:Murano::Client http://localhost:8082/>
> >> DEBUG:muranoclient.common.http:curl -i -X GET -H 'X-Auth-Token:
> 324759651d234c4eaf08f6093dfd7000' -H 'Content-Type: application/json' -H
> 'User-Agent: python-muranoclient'
> http://localhost:8082//v1/catalog/packages/914c2bfd5d504419a94a9affb7af809a
> >> DEBUG:muranoclient.common.http:
> >> HT

Re: [openstack-dev] [murano] Proposing Nikolai Starodubtsev for core

2015-09-08 Thread Stan Lagun
+1

Sincerely yours,
Stan Lagun
Principal Software Engineer @ Mirantis

<sla...@mirantis.com>

On Tue, Sep 1, 2015 at 3:03 PM, Alexander Tivelkov <ativel...@mirantis.com>
wrote:

> +1. Well deserved.
>
> --
> Regards,
> Alexander Tivelkov
>
> On Tue, Sep 1, 2015 at 2:47 PM, Victor Ryzhenkin <vryzhen...@mirantis.com>
> wrote:
>
>> +1 from me ;)
>>
>> --
>> Victor Ryzhenkin
>> Junior QA Engeneer
>> freerunner on #freenode
>>
>> Включено 1 сентября 2015 г. в 12:18:19, Ekaterina Chernova (
>> efedor...@mirantis.com) написал:
>>
>> +1
>>
>> On Tue, Sep 1, 2015 at 10:03 AM, Dmitro Dovbii <ddov...@mirantis.com>
>> wrote:
>>
>>> +1
>>>
>>> 2015-09-01 2:24 GMT+03:00 Serg Melikyan <smelik...@mirantis.com>:
>>>
>>>> +1
>>>>
>>>> On Mon, Aug 31, 2015 at 3:45 PM, Kirill Zaitsev <kzait...@mirantis.com>
>>>> wrote:
>>>>
>>>>> I’m pleased to nominate Nikolai for Murano core.
>>>>>
>>>>> He’s been actively participating in development of murano during
>>>>> liberty and is among top5 contributors during last 90 days. He’s also
>>>>> leading the CloudFoundry integration initiative.
>>>>>
>>>>> Here are some useful links:
>>>>>
>>>>> Overall contribution: http://stackalytics.com/?user_id=starodubcevna
>>>>> List of reviews:
>>>>> https://review.openstack.org/#/q/reviewer:%22Nikolay+Starodubtsev%22,n,z
>>>>> Murano contribution during latest 90 days
>>>>> http://stackalytics.com/report/contribution/murano/90
>>>>>
>>>>> Please vote with +1/-1 for approval/objections
>>>>>
>>>>> --
>>>>> Kirill Zaitsev
>>>>> Murano team
>>>>> Software Engineer
>>>>> Mirantis, Inc
>>>>>
>>>>>
>>>>> __
>>>>> OpenStack Development Mailing List (not for usage questions)
>>>>> Unsubscribe:
>>>>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> Serg Melikyan, Senior Software Engineer at Mirantis, Inc.
>>>> http://mirantis.com | smelik...@mirantis.com
>>>>
>>>> +7 (495) 640-4904, 0261
>>>> +7 (903) 156-0836
>>>>
>>>>
>>>> __
>>>> OpenStack Development Mailing List (not for usage questions)
>>>> Unsubscribe:
>>>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>
>>>>
>>>
>>>
>>> __
>>> OpenStack Development Mailing List (not for usage questions)
>>> Unsubscribe:
>>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>>
>> __
>>
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [mistral][yaql] Addressing task result using YAQL function

2015-09-07 Thread Stan Lagun
I believe this is a good change. $.task_name requires you that $ be
pointing to a tasks dictionary. But in the middle of the query like
[1.2.3].select($ + 1)  "$" will change its value. With a function approach
you can write [1, 2, 3].select($ + task(taskName)). However the name "task"
looks confusing as to my understanding tasks may have attributes other than
result. It may make sense to use task(taskName).result instead.

Sincerely yours,
Stan Lagun
Principal Software Engineer @ Mirantis

<sla...@mirantis.com>

On Sun, Sep 6, 2015 at 12:14 AM, Dmitri Zimine <dzim...@stackstorm.com>
wrote:

> Yes meant to ask for consistency of referencing to task results. So it’s
> task(task_name) regardless of where.
>
> One use case in favor of this is tooling: I refactor workflow with an
> automated tool which wants to automatically rename the task name
> EVERYWHERE. You guys know well by now that renaming the task is a source of
> too many frustrating errors :)
>
> What other think?
>
> DZ.
>
> On Sep 3, 2015, at 4:23 AM, Renat Akhmerov <rakhme...@mirantis.com> wrote:
>
>
> On 02 Sep 2015, at 21:01, Dmitri Zimine <dzim...@stackstorm.com> wrote:
>
> Agree,
>
> with one detail: make it explicit -  task(task_name).
>
>
> So do you suggest we just replace res() with task() and it looks like
>
> task() - get task result when we are in “publish”
> task(task_name) - get task result from anywhere
>
> ?
>
> Is that correct you mean we must always specify a task name? The reason
> I’d like to have a simplified form (w/o task name) is that I see a lot of
> workflows that we have to repeat task name in publish so that it just look
> too verbose to me. Especially in case of very long task name.
>
> Consider something like this:
>
> tasks:
>   *get_volumes_by_names*:
> with-items: name in <% $.vol_names %>
> workflow: get_volume_by_name name=<% $.name %>
> publish:
>   volumes: <% $.*get_volumes_by_names* %>
>
> So in publish we have to repeat a task name, there’s no other way now. I’d
> like to soften this requirement, but if you still want to use task names
> you’ll be able to.
>
>
> res - we often see folks confused by result of what (action, task,
> workflow) although we cleaned up our lingo: action-output, task-result,
> workflow-output…. but still worth being explicit.
>
> And full result is being thought as the root context $.
>
> Publishing to global context may be ok for now, IMO.
>
>
> Not sure what you meant by "Publishing to global context”. Can you clarify
> please?
>
>
> Renat Akhmerov
> @ Mirantis Inc.
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [murano] Questions on creating and importing HOT packages

2015-08-11 Thread Stan Lagun
1. This is for simplicity as file name doesn't really matter. HOT
processing code needs to know where to look for template and it is easier
to have fixed file name/location rather than force users to explicitly
specify location in package manifest. But if if you have valid case where
this becomes an issue please share it so that we could improve the code.
BTW python-muranoclient may/should rename template file and what inside
archive package is just an implementation detail

2. HOT syntax is not validated during import time as HOT validation is
Heat's job and this can be done only during deployment time. However
because Murano generates UI form and MuranoPL class syntax errors in HOT
file may prevent it doing so. So this can be considered as some sort of
validation but it is very very basic. Heat does much more in this area. We
are doing this on server-side because this code is needed during deployment
time (to generate MuranoPL on the fly) and also so that any changes made to
HOT processor will immediately affect all HOT packages as this generation
done on each deployment. This would be impossible if python-muranoclient
would do the code generation (and this is the only thing that validates HOT
in Murano). Also it is good to have clients thin to simplify development of
clients for languages other then Python

Sincerely yours,
Stan Lagun
Principal Software Engineer @ Mirantis

sla...@mirantis.com

On Sat, Aug 8, 2015 at 2:40 AM, Vahid S Hashemian vahidhashem...@us.ibm.com
 wrote:

 Hello,

 These two subjects have probably been discussed already, but I couldn't
 find any info on them, so very much appreciate if someone could clarify.

1. Why is the HOT template that is fed to package-create command
renamed to template.yaml? Any issue with keeping the original name?
2. Why HOT syntax validation is deferred until package import time?
Why not do it when creating the package?


 Thanks.

 Regards,

 - 
 *Vahid
 Hashemian, Ph.D.*
 Advisory Software Engineer, IBM Cloud Labs


 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [murano] [mistral] [yaql] Prepare to Yaql 1.0 release

2015-08-04 Thread Stan Lagun
Dmitry, this depends on how you're going to use yaql. yaql 1.0 has so
called legacy mode that is a layer on top of yaql that brings nearly full
backward compatibility including some of the things that were wrong in yaql
0.2. Use this mode when backward compatibility is a must. Without it the
following changes may affect queries written for 0.2:

1. There are no more tuples. foo(a = b) will mean now f(a='b') instead of
foo(tuple('a', 'b')) as it was in yaql 0.2. However dict(key1 = value1,
key2 = value2)  and several other functions with the same syntax work
exactly how they used to work in 0.2
2. Contradictory to the first point there are no more lists. Yaql works on
immutable structures (tuple, frozenset and yaql1 own FrozenDict). All
operations that seem to modify data in fact return modified copies. All
input data will be automatically converted to their immutable versions and
converted back upon completion. This has 2 side effects: a) if you have
custom yaql functions they should follow the same pattern b) even if your
input data were build from tuples etc output will still use lists (e.g.
JSON format)
3. $dict.key will raise KeyError for missing keys instead of returning
None. Also key must be a valid keyword and cannot start with 2 underscores.
There are many other ways to retrieve dictionary value: $dict.get(value),
$dict.get(value, default), $dict[value], $dict[value, default]
4. In yaql 0.2 every function could be called as a method. So $x.foo() was
the same as foo($x). In yaql 1.0 it is up to function to decide if it wants
to be a function (foo($x), method ($x.foo()) or extension method (both
syntax).Considering that yaql handles different function overloads
$x.foo() may be a valid expression and execute something completely
different from foo($x).
 Most of type-specific functions are declared as a methods. So you no
longer can say toUpper(str) but only str.toUpper()
5. yaql 0.2 had very few functions. yaql 1.0 has huge (~260
functions/operators etc) standard library. However several of the 0.2
functions now have different signature or name, replaced with better
alternative or just accept additional (optional) parameters
6. $collection[$  0] doesn't work anymore. Use $collection.select($  0)
for that



Sincerely yours,
Stan Lagun
Principal Software Engineer @ Mirantis

sla...@mirantis.com

On Tue, Aug 4, 2015 at 9:57 PM, Dmitri Zimine dzim...@stackstorm.com
wrote:

 This is great news Alex, was looking forward to it, will be happy to
 migrate Mistral.

 Some heads-up on what syntactically changed would be much appreciated to
 pass on to our users;
 we likely will catch much of them with Mistral tests, but some may bubble
 up.

 DZ.

 On Jul 27, 2015, at 2:04 AM, Alexander Tivelkov ativel...@mirantis.com
 wrote:

  Hi folks,
 
  We are finally ready to release the 1.0.0 version of YAQL. It is a
  huge milestone: the language finally looks the way we initially wanted
  it to look. The engine got completely rewritten, tons of new
  capabilities have been added. Here is a brief (and incomplete) list of
  new features and improvements:
 
  * Support for kwargs and keyword-only args (Py3)
  * Optional function arguments
  * Smart algorithm to find matching function overload without side effects
  * Ability to organize functions into layers
  * Configurable list of operators (left/right associative binary,
  prefix/suffix unary with precedence)
  * No global variables. There can be  more than one parser with
  different set of operators simultaneously
  * List literals ([a, b])
  * Dictionary literals ({ a = b})
  * Handling of escape characters in string literals
  * Verbatim strings (`...`) and double-quotes (...)
  * =~ and !~ operators in default configuration (similar to Perl)
  * - operator to pass context
  * Alternate operator names (for example '*equal' instead of
 '#operator_=')
   so that it will be possible to have different symbol for particular
 operator
   without breaking standard library that expects operator to have well
  known names
  * Set operations
  * Support for lists and dictionaries as a dictionary keys and set
 elements
  * New framework to decorate functions
  * Ability to distinguish between functions and methods
  * Switchable naming conventions
  * Unicode support
  * Execution options available to all invoked functions
  * Iterators limitation
  * Ability to limit memory consumption
  * Can work with custom context classes
  * It is possible to extend both parser and set of expression classes
  on user-side
  * It is possible to create user-defined types (also can be used for
  dependency injection)
  * Legacy yaql 0.2.x backward compatibility mode
  * Comprehensive standard library of functions
  * High unit test coverage
  * Delegate and lambda support, including higher order lambdas
 
  etc, etc.
 
  So, this is a big change.
 
  And as it always happens when moving from 0.something to 1.x the
  breaking changes are inevitable. We have included the backwards

Re: [openstack-dev] [mistral] [murano] An online YAQL evaluator

2015-08-04 Thread Stan Lagun
Dmitry,

yaql 1.0 has both str() and len() and much much more so there is no need to
support them explicitly since Mistral is going to switch to yaql 1.0 and
yaqluator.com is going to do the same

Sincerely yours,
Stan Lagun
Principal Software Engineer @ Mirantis

sla...@mirantis.com

On Tue, Aug 4, 2015 at 8:55 PM, Dmitri Zimine dzim...@stackstorm.com
wrote:

 Awesome! Really.

 Thank you folks for doing this.

 I am so much looking forward to moving it to 1.0 with more built-in
 functions and more power to extend it...

 Note that Mistral has a few extensions, like `str`, `len`, which are not
 in the scope of evaluator.

 DZ


 On Aug 2, 2015, at 12:44 PM, Stan Lagun sla...@mirantis.com wrote:

 Guys, this is awesome!!!

 Happy to see yaql gets attention. One more initiative that you may find
 interesting is https://review.openstack.org/#/c/159905/
 This is an attempt to port yaql 1.0 from Python to JS so that the same can
 be done in browser

 Sincerely yours,
 Stan Lagun
 Principal Software Engineer @ Mirantis

 sla...@mirantis.com

 On Sun, Aug 2, 2015 at 5:30 PM, Nikolay Makhotkin nmakhot...@mirantis.com
  wrote:

 Hi guys!

 That's awesome! It is very useful for all us!

 --
 Best Regards,
 Nikolay

 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe:
 openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://openstack-dev-requ...@lists.openstack.org/?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [mistral] [murano] An online YAQL evaluator

2015-08-02 Thread Stan Lagun
Guys, this is awesome!!!

Happy to see yaql gets attention. One more initiative that you may find
interesting is https://review.openstack.org/#/c/159905/
This is an attempt to port yaql 1.0 from Python to JS so that the same can
be done in browser

Sincerely yours,
Stan Lagun
Principal Software Engineer @ Mirantis

sla...@mirantis.com

On Sun, Aug 2, 2015 at 5:30 PM, Nikolay Makhotkin nmakhot...@mirantis.com
wrote:

 Hi guys!

 That's awesome! It is very useful for all us!

 --
 Best Regards,
 Nikolay

 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Mistral][Murano] What's the latest/greatest on YAQL?

2015-06-26 Thread Stan Lagun
The plan is to move to yaql 1.0 this release.  Please do not merge yaql 1.0
into Mistral yet. Migration is going to happen really soon

Sincerely yours,
Stan Lagun
Principal Software Engineer @ Mirantis

sla...@mirantis.com

On Fri, Jun 26, 2015 at 8:17 AM, Renat Akhmerov rakhme...@mirantis.com
wrote:

 Yes, it’s a pretty important thing that we’d like to clarify as soon as
 possible.

 Any input from Murano team?

 Renat Akhmerov
 @ Mirantis Inc.



  On 26 Jun 2015, at 06:01, Dmitri Zimine dzim...@stackstorm.com wrote:
 
  Folks,
 
  it’s been some time,what’s the news:
 
  * Is Murano moving YAQL 1.0 in this cycle?
  * What’s your recommendation for this cycle - stay on 0.2.6 or move on?
  * Any progress on documentation?
 
  Thanks, DZ
 
 
 
 __
  OpenStack Development Mailing List (not for usage questions)
  Unsubscribe:
 openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
  http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Murano] Does Murano support nested HOT templates?

2015-06-12 Thread Stan Lagun
Currently Murano doesn't support nested template unless they are specified
by URL. However this is going to be fixed in current release cycle as there
are many requests for this feature.

Sincerely yours,
Stan Lagun
Principal Software Engineer @ Mirantis

sla...@mirantis.com

On Thu, Jun 11, 2015 at 1:55 AM, Vahid S Hashemian 
vahidhashem...@us.ibm.com wrote:

 Just wondering if Murano supports nested templates.
 For example, the autoscaling template at
 https://github.com/openstack/heat-templates/blob/master/hot/autoscaling.yaml
 references the template at
 https://github.com/openstack/heat-templates/blob/master/hot/lb_server.yaml.
 When I create the package, the referenced template does not seem to be
 included in the application archive. Even though the package is imported
 successfully, upon deployment an error is reported saying:

 ERROR: Failed to validate : : resources.asg: : Could not fetch remote
 template lb_server.yaml: Invalid URL scheme

 I'm not sure whether such templates should be imported differently or they
 are not supported yet.

 Regards,

 -
 Vahid Hashemian, Ph.D.
 Advisory Software Engineer, IBM Cloud Labs


 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [murano] python versions

2015-06-12 Thread Stan Lagun
I'd rather go with Heat approach (job first) because it makes easier to
track what is left to port to Py34 and track progress in this area

Sincerely yours,
Stan Lagun
Principal Software Engineer @ Mirantis

sla...@mirantis.com

On Mon, Jun 8, 2015 at 2:46 PM, Kirill Zaitsev kzait...@mirantis.com
wrote:

 I’ve looked into several OS projects, and they first seen to implement py3
 support and create a job later. (Except for heat. They already have a
 non-voting py34, which seem to fail every time =))

 I suggest we do the same: first make murano work on py34, then make a py34
 job. I’ll file a blueprint shortly.

 --
 Kirill Zaitsev
 Murano team
 Software Engineer
 Mirantis, Inc

 On 2 Jun 2015 at 15:58:17, Serg Melikyan (smelik...@mirantis.com) wrote:

 Hi Kirill,

 I agree with Alexander that we should not remove support for python
 2.6 in python-muranoclient.

 Regarding adding python-3 jobs - great idea! But we need to migrate
 python-muranoclient to yaql 1.0 first and then add python-3 jobs,
 because previous versions of yaql are not compatible with python-3.

 On Tue, Jun 2, 2015 at 3:33 PM, Alexander Tivelkov
 ativel...@mirantis.com wrote:
  Hi Kirill,
 
  Client libraries usually have wider range of python requirements, as they
  may be run on various kinds of legacy environments, including the ones
 with
  python 2.6. only.
  Murano is definitely not the only project in Openstack which still
 maintains
  py26 compatibility for its client: nova, glance, cinder and other
 integrated
  ones do this.
 
  So, I would not drop py26 support for client code without any good
 reasons
  to. Are there any significant reasons to do it?
  Regarding py3.4 - this is definitely a good idea.
 
 
  --
  Regards,
  Alexander Tivelkov
 
  On Tue, Jun 2, 2015 at 3:04 PM, Kirill Zaitsev kzait...@mirantis.com
  wrote:
 
  It seems that python-muranoclient is the last project from
 murano-official
  group, that still supports python2.6. Other projects do not have a 2.6
  testing job (correct me if I’m wrong).
 
  Personally I think it’s time to drop support for 2.6 completely, and add
  (at least non-voting) jobs with python3.4 support tests.
  It seems to fit the whole process of moving OS projects towards python
 3:
  https://etherpad.openstack.org/p/liberty-cross-project-python3
 
  What do you think? Does anyone have any objections?
 
  --
  Kirill Zaitsev
  Murano team
  Software Engineer
  Mirantis, Inc
 
 
 __
  OpenStack Development Mailing List (not for usage questions)
  Unsubscribe:
 openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
  http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
 
 
 
 
 __
  OpenStack Development Mailing List (not for usage questions)
  Unsubscribe:
 openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
  http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
 



 --
 Serg Melikyan, Senior Software Engineer at Mirantis, Inc.
 http://mirantis.com | smelik...@mirantis.com

 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Murano] Nominating Kirill Zaitsev for murano-core

2015-06-02 Thread Stan Lagun
+1 without any doubt

Sincerely yours,
Stan Lagun
Principal Software Engineer @ Mirantis

sla...@mirantis.com

On Tue, Jun 2, 2015 at 10:43 AM, Ekaterina Chernova efedor...@mirantis.com
wrote:

 +1

 Regards,
 Kate.

 On Tue, Jun 2, 2015 at 9:32 AM, Serg Melikyan smelik...@mirantis.com
 wrote:

 I'd like to propose Kirill Zaitsev to core members of Murano team.

 Kirill Zaitsev is active member of our community, he implemented
 https://launchpad.net/murano/+milestone/2015.1.0 several blueprint in
 Kilo and fixed number of bugs, he maintains a really good score as
 contributor:
 http://stackalytics.com/report/users/kzaitsev

 Existing Murano cores, please vote +1/-1 for the addition
 of Kirill to the murano-core.
 --
 Serg Melikyan, Senior Software Engineer at Mirantis, Inc.
 http://mirantis.com | smelik...@mirantis.com

 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe:
 openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Murano] Nominating Filip Blaha for murano-core

2015-06-02 Thread Stan Lagun
+1

Sincerely yours,
Stan Lagun
Principal Software Engineer @ Mirantis

sla...@mirantis.com

On Tue, Jun 2, 2015 at 9:25 AM, Serg Melikyan smelik...@mirantis.com
wrote:

 Folks, I'd like to propose Filip Blaha to core members of Murano team.

 Filip is active member of our community and he maintains a good score
 as contributor:
 http://stackalytics.com/report/users/filip-blaha

 Existing Murano cores, please vote +1/-1 for the addition of Filip to
 the murano-core.
 --
 Serg Melikyan, Senior Software Engineer at Mirantis, Inc.
 http://mirantis.com | smelik...@mirantis.com

 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Murano] [Mistral] SSH workflow action

2015-05-13 Thread Stan Lagun
On Wed, May 13, 2015 at 5:42 PM, Zane Bitter zbit...@redhat.com wrote:

 There is more to multi-tenancy than just authentication/authorisation.
 It's also things like making sure one tenant's use of resources doesn't
 affect another tenant's (e.g. creating a denial of service by maxing out
 capacity); being able to measure and bill for usage; quotas to prevent
 accidental overuse c.


This is a indeed a strong point that makes sense for shared MQ broker.
However I'm not convinced that Zaqar approach is an optimal one:
1. There are a several dozens of message-oriented middleware solutions
around (QPID, ActiveMQ, Camel, Kafka, HortnetQ and many more). Many of them
are well know, well tested, have significant user base, published books
etc. Does no one of them could be used instead with additional wrapping or
special configs or additional plugins/extensions? So that is would left
just to implement the integration layer instead of build MQ solution from
scratch?

2. Even RabbitMQ has a lot of things pluggable and just maybe all those
requirements can be met by a set of RabbitMQ plugins. It is also possible
to contribute something to RabbitMQ or any other OSS MQ.

3. I'm not sure that noisy neighbor problem and other QoS related issues
are a big problem here. The question is who is the consumer of MQ? If
consumers are user-land applications running on OpenStack instance VMs a
better solution could be just to set dedicated RabbitMQ/somethingMQ on a VM
for those apps instead of a single shared server. I do understand that a
common shared resource is easier to mange and administrate. On the other
hand it is less secure and subject to QoS issues. There are so many
software configuration management tools around (docker containers and
container managers, prebuilt images, puppet manifests and so on) that it is
not a real challenge any more. And if consumers are other OpenStack
services QoS is unlikely to be a problem because in each OpenStack
distribution there are fixed number of tested services with known shape of
MQ workload.

4. Another option could be just to bring new MQ broker per tenant. For
example to run docker RabbitMQ container per tenant + something small to
manage them (to sync containers with tenant list). Docker has resource
quotas that can be used.

5. Stable API is good. However open and standardized API like AMQP is even
better.

Maybe all of those possible solutions are bad and I'm sure there are
downsides to each approach. It is just not obvious that development of yet
another MQ solution worth it.

Sincerely yours,
Stan Lagun
Principal Software Engineer @ Mirantis

sla...@mirantis.com
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Murano] [Mistral] SSH workflow action

2015-05-13 Thread Stan Lagun
On Wed, May 13, 2015 at 2:46 AM, Fox, Kevin M kevin@pnnl.gov wrote:

 Awesome. When is it/where do I go to look up that info?


http://sched.co/3Clo

Sincerely yours,
Stan Lagun
Principal Software Engineer @ Mirantis

sla...@mirantis.com
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Murano] [Mistral] SSH workflow action

2015-05-12 Thread Stan Lagun
We are going to have a design session at the summit entirely dedicated to
Murano guest agent security and isolation. Everyone is welcomed to attend
and discuss security requirements, concerns and possible solutions.

Sincerely yours,
Stan Lagun
Principal Software Engineer @ Mirantis

sla...@mirantis.com

On Tue, May 12, 2015 at 8:32 PM, Fox, Kevin M kevin@pnnl.gov wrote:

  Barbican has the same issue. If Barbican is for storing secrets, how do
 you get a secret to the VM so it can get its secrets from Barbican?
 Aaaggg! :)

 I've been working on a solution here:
 https://blueprints.launchpad.net/barbican/+spec/vm-integration

 We're planning on talking more about that spec at the summit though.

 I've heard through the grape vine that Amazon lets you have a machine
 account that is associated with the vm. I'm not sure that's true or not,
 but some kind of keystone account integration into nova might help...

 Thanks,
 Kevin
  --
 *From:* Georgy Okrokvertskhov [gokrokvertsk...@mirantis.com]
 *Sent:* Tuesday, May 12, 2015 10:06 AM
 *To:* OpenStack Development Mailing List (not for usage questions)

 *Subject:* Re: [openstack-dev] [Murano] [Mistral] SSH workflow action

   There is one thing which still bothers me. It is authentication. Right
 now with separate RabbitMQ instance we keep VMs authentication isolated
 from OpenStack infra.
 This is still a problem if you want to use webhooks (Heat autoscaling,
 Murano actions) via our own authentication models. If we plan to use Zaqar
 it will be interesting to know how Zaqar solves this issue. Frankly, I
 don't think that this is a good idea to use Keystone credentials or tokens
 for MQ clients on VMs. This topic, probably, deserves its own e-mail thread.

  It will be interesting to discuss this with Keystone team. What is it is
 possible to have a token which is restricted to be authenticated to
 specific API URL like  GET /v1/queues/queue-id/


  Thanks
 Gosha



 On Tue, May 12, 2015 at 8:58 AM, Fox, Kevin M kevin@pnnl.gov wrote:

 +1
 
 From: Zane Bitter [zbit...@redhat.com]
 Sent: Monday, May 11, 2015 6:15 PM
 To: openstack-dev@lists.openstack.org
 Subject: Re: [openstack-dev] [Murano] [Mistral] SSH workflow action

  Hello!

 This looks like a perfect soapbox from which to talk about my favourite
 issue ;)

 You're right about the ssh idea, for the reasons discussed related to
 networking and a few more that weren't (e.g. users shouldn't have to and
 generally don't want to give their private SSH keys to cloud services).
 I didn't know, or had forgotten, about the message queue implementation
 in Murano and while I think that's the correct shape for the solution,
 as long as the service in question is not multi-tenant capable it's a
 non-starter for a public clouds at least and probably many private
 clouds as well (after all, if you don't need multi-tenancy then why are
 you using OpenStack?).

 There's been a tendency within the application-facing OpenStack projects
 to hack together whatever local solutions to problems that we can in
 order to make progress without being held up by other projects. Let's
 take a moment to acknowledge that Heat is both the earliest and the
 biggest offender here, and that I am as culpable as anyone in the
 current state of affairs. There are multiple reasons for how things have
 gone - part of it is that it turned out we developed services in the
 wrong order, starting at too high a level. Part of it, frankly, is due
 to that element of the community that maintains a hostile position
 toward application-facing services and have used their influence in the
 community to maintain a disincentive against integrating projects
 together.[1] (If deployment of your project is discouraged that's one
 thing, but if it depends on another project whose deployment is also
 being discouraged then the hurdle you have to jump over is twice the
 height.)

 That said, I think we're at the point where we are hurting ourselves
 more than anyone else is by failing to come up with coherent,
 cross-project solutions.

 The problem articulated in this thread is not an isolated one. It's part
 of a more general pattern that affects a lot of projects: we need a way
 for the cloud to communicate to applications running in it. Angus
 started a recent discussion of this already on the list.[2] The
 requirements, IMHO, are roughly:

   * Reliability - we must be able to guarantee delivery to applications
   * Asynchrony - the cloud cannot block on user-controlled processes
   * Multitenancy - this is table stakes for OpenStack
   * Access control - even within tenants, we need to trust guest VMs
 minimally

 IMNSHO Zaqar messages are the obvious choice for the transport here. (Or
 something very similar in shape to Zaqar - but it'd be much better to
 join forces with the Zaqar team to improve it where necessary than to
 start a new project.) I really believe that if we

Re: [openstack-dev] [Murano] [Mistral] SSH workflow action

2015-05-12 Thread Stan Lagun
+1 for making Murano Engine - Murano Agent communication plugable so that
one can switch to Zaqar or anything else. However watching RabbitMQ
development for years I know hard can it be to build efficient and reliable
system and I'm just not sure Zaqar can compete with such battle-proven
thing like RabbitMQ yet. The only advantage I see is multi-tenancy. But I
do believe it can be relatively easy be implemented with RabbitMQ. At lease
in Murano. Don't want to go off topic here. The main idea is to use
https://github.com/rabbitmq/rabbitmq-auth-backend-amqp and dynamically
grant agent permissions only to his dedicated input queue so that it cannot
access anything else. Not just other tenants queues but also queues of
other VMs in the same tenant. In case of Murano we will not need to
maintain additional secrets or databases. Neither it will be needed to
create RabbitMQ users/vhosts as all of this becomes virtual. And agent will
not be holding any RabbitMQ passwords at all



Sincerely yours,
Stan Lagun
Principal Software Engineer @ Mirantis

sla...@mirantis.com

On Tue, May 12, 2015 at 10:52 AM, Renat Akhmerov rakhme...@mirantis.com
wrote:

 Zane,

 Fully agree with you vision here.

 On 12 May 2015, at 07:15, Zane Bitter zbit...@redhat.com wrote:

 * Add an action in Mistral for sending a message to a Zaqar queue. This is
 easy and there's no reason you couldn't do it right now.


 Any volunteers?

 * Add a way to trigger a Mistral workflow with a Zaqar message. This is
 one piece in the puzzle to build user-configurable messaging flows between
 OpenStack services.[3]


 I added an agenda item for the summit in
 https://etherpad.openstack.org/p/vancouver-2015-design-summit-mistral to
 discuss this. Everyone is welcome.

 Imagine if there were one place where we implemented reliable queuing
 semantics at cloud scale, and when we added e.g. long-polling or WebSockets
 everyone could benefit immediately.[4] Imagine if there were one place for
 notifications, at cloud scale, for operators to secure. (How many webhook
 implementations are there in OpenStack right now? How many of them are
 actually secure against malicious users?) One format for messages between
 services so that users can connect up their own custom pipelines. We're not
 that far away! All of this is within reach if we work together.


 Cool picture of a wonderful future :)

 Thanks for reading. Please grab me at summit if you want to know more; I
 am always happy to bend the ear of anyone who will listen at length on this
 topic. As usual, I'll be the tall dude with the weird accent ;)


 With the great pleasure.

 (P.S. your accent is cool!)

 Renat Akhmerov
 @ Mirantis Inc.


 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Murano] [Mistral] SSH workflow action

2015-05-11 Thread Stan Lagun
Filip,

 Currently there is no support in mistral how to execute scripts on VM via
murano agent

Mistral can call Murano application action that will do the job via agent.
Actions are intended to be called by 3rd party systems with single HTTP
request

Sincerely yours,
Stan Lagun
Principal Software Engineer @ Mirantis

sla...@mirantis.com

On Mon, May 11, 2015 at 11:27 AM, Filip Blaha filip.bl...@hp.com wrote:

  Hi

 there is VPN mechanism in neutron we could consider for future how to get
 around these networking obstacles if we would like to use direct SSH.

 1) every private created by murano would create VPN gateway on public
 interface of the router [1]

 neutron vpn-service-create --name myvpn --description My vpn service
 router1 mysubnet

 2) any service like mistral which needs directly access VM via SSH (or
 other protocols) would connect to that VPN and then it could directly
 access VM on its fixed IP

 This mechanism would probably resolve network obstacles. But it requires
 more effort to analyse it.

 [1] https://wiki.openstack.org/wiki/Neutron/VPNaaS/HowToInstall

 Filip


 On 05/08/2015 10:22 AM, Renat Akhmerov wrote:

 Generally yes, std.ssh action works as long as network infrastructure allows 
 access to a host using specified IP, it doesn’t provide anything on top of 
 that.



  On 06 May 2015, at 22:26, Fox, Kevin M kevin@pnnl.gov 
 kevin@pnnl.gov wrote:

 This would also probably be a good use case for Zaqar I think. Have a generic 
 run shell commands from Zaqar queue agent, that pulls commands from a Zaqar 
 queue, and executes it.
 The vm's don't have to be directly reachable from the network then. You just 
 have to push messages into Zaqar.

  Yes, in Mistral it would be another action that puts a command into Zaqar 
 queue. This type of action doesn’t exist yet but it can be plugged in easily.


  Should Mistral abstract away how to execute the action, leaving it up to 
 Mistral how to get the action to the vm?

  Like I mentioned previously it should be just a different type of action: 
 “zaqar.something” instead of “std.ssh”. Mistral engine itself works with all 
 actions equally, they are just basically functions that we can plug in and 
 use in Mistral workflow language. From this standpoint Mistral is already 
 abstract enough.


  If that's the case, then ssh vs queue/agent is just a Mistral implementation 
 detail?

  More precisely: implementation detail of Mistral action which may not be 
 even hardcoded part of Mistral, we can rather plug them in (using stevedore 
 underneath).


 Renat Akhmerov
 @ Mirantis Inc.


 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: 
 openstack-dev-requ...@lists.openstack.org?subject:unsubscribehttp://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Murano] [Mistral] SSH workflow action

2015-05-10 Thread Stan Lagun
Kevin,

I do agree that lack of RabbitMQ multi-tenancy is a problem. However as
this is developers mailing list I would suggest to contribute and make
Murano guest agent secure for the benefit of all users and all existing
applications instead of spending comparable amount of time developing
intricate solutions that will make everything way more complex. If Murano
application developers need to write additional Mistral workflow for each
deployment step that would make application development be extremely harder
and Murano be mostly useless.

There are several approaches that can be taken to solve agent isolation
problem and all of them are relatively easy to implement.
This task is one of out top priorities for the Liberty and will be solved
very soon anyway.

Another approach that IMO also better than SSH is to use HOT software
config that uses HTTP polling and doesn't suffer from lack of tenant
isolation.

I do want to see better Mistral integration in Murano as well as many other
tools like puppet etc. And there are some good use cases for Mistral. But
when it comes to the most basic things that Murano was designed to do from
the foundation I want to make sure that Murano can do them the best way
possible without requiring users to learn additional DSLs/tools or go extra
step and involve additional services where not necessary. If something
important is missing in Murano that makes usage of Mistral for deployment
more attractive I'd rather focus on improving Murano and bringing those
features to Murano. We can even use Mistral under the hood as long as we
not make users to write both MuranoPL and Mistral DSL code for trivial
things like service restart.

Sincerely yours,
Stan Lagun
Principal Software Engineer @ Mirantis

sla...@mirantis.com

On Sun, May 10, 2015 at 8:44 PM, Fox, Kevin M kevin@pnnl.gov wrote:

  Im planning on deploying murano but wont be supporting the murano guest
 agent. The lack of multi tenant security is a big problem I think.

 Thanks,
 Kevin

 --
 *From:* Stan Lagun
 *Sent:* Saturday, May 09, 2015 7:21:17 PM
 *To:* OpenStack Development Mailing List (not for usage questions)
 *Subject:* Re: [openstack-dev] [Murano] [Mistral] SSH workflow action

  Filip,

  If I got you right the plan is to have Murano application execute
 Mistral workflow that SSH to VM and executes particular command? And
 alternative is Murano-Mistral-Zaquar-Zaquar agent?
 Why can't you just send this command directly from Murano (to Murano agent
 on VM)? This is the most common use case that is found in nearly all Murano
 applications and it is battle-proven. If you need SSH you can contribute
 SSH plugin to Murano (Mistral will require similar plugin anyway). The more
 moving parts you involve the more chances you have for everything to fail


  Sincerely yours,
 Stan Lagun
 Principal Software Engineer @ Mirantis

  sla...@mirantis.com

 On Fri, May 8, 2015 at 11:22 AM, Renat Akhmerov rakhme...@mirantis.com
 wrote:

 Generally yes, std.ssh action works as long as network infrastructure
 allows access to a host using specified IP, it doesn’t provide anything on
 top of that.


  On 06 May 2015, at 22:26, Fox, Kevin M kevin@pnnl.gov wrote:
 
  This would also probably be a good use case for Zaqar I think. Have a
 generic run shell commands from Zaqar queue agent, that pulls commands
 from a Zaqar queue, and executes it.
  The vm's don't have to be directly reachable from the network then. You
 just have to push messages into Zaqar.

 Yes, in Mistral it would be another action that puts a command into Zaqar
 queue. This type of action doesn’t exist yet but it can be plugged in
 easily.

  Should Mistral abstract away how to execute the action, leaving it up
 to Mistral how to get the action to the vm?

 Like I mentioned previously it should be just a different type of action:
 “zaqar.something” instead of “std.ssh”. Mistral engine itself works with
 all actions equally, they are just basically functions that we can plug in
 and use in Mistral workflow language. From this standpoint Mistral is
 already abstract enough.

  If that's the case, then ssh vs queue/agent is just a Mistral
 implementation detail?

 More precisely: implementation detail of Mistral action which may not be
 even hardcoded part of Mistral, we can rather plug them in (using stevedore
 underneath).


 Renat Akhmerov
 @ Mirantis Inc.


 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe:
 openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Murano] [Mistral] SSH workflow action

2015-05-09 Thread Stan Lagun
Filip,

If I got you right the plan is to have Murano application execute Mistral
workflow that SSH to VM and executes particular command? And alternative is
Murano-Mistral-Zaquar-Zaquar agent?
Why can't you just send this command directly from Murano (to Murano agent
on VM)? This is the most common use case that is found in nearly all Murano
applications and it is battle-proven. If you need SSH you can contribute
SSH plugin to Murano (Mistral will require similar plugin anyway). The more
moving parts you involve the more chances you have for everything to fail


Sincerely yours,
Stan Lagun
Principal Software Engineer @ Mirantis

sla...@mirantis.com

On Fri, May 8, 2015 at 11:22 AM, Renat Akhmerov rakhme...@mirantis.com
wrote:

 Generally yes, std.ssh action works as long as network infrastructure
 allows access to a host using specified IP, it doesn’t provide anything on
 top of that.


  On 06 May 2015, at 22:26, Fox, Kevin M kevin@pnnl.gov wrote:
 
  This would also probably be a good use case for Zaqar I think. Have a
 generic run shell commands from Zaqar queue agent, that pulls commands
 from a Zaqar queue, and executes it.
  The vm's don't have to be directly reachable from the network then. You
 just have to push messages into Zaqar.

 Yes, in Mistral it would be another action that puts a command into Zaqar
 queue. This type of action doesn’t exist yet but it can be plugged in
 easily.

  Should Mistral abstract away how to execute the action, leaving it up to
 Mistral how to get the action to the vm?

 Like I mentioned previously it should be just a different type of action:
 “zaqar.something” instead of “std.ssh”. Mistral engine itself works with
 all actions equally, they are just basically functions that we can plug in
 and use in Mistral workflow language. From this standpoint Mistral is
 already abstract enough.

  If that's the case, then ssh vs queue/agent is just a Mistral
 implementation detail?

 More precisely: implementation detail of Mistral action which may not be
 even hardcoded part of Mistral, we can rather plug them in (using stevedore
 underneath).


 Renat Akhmerov
 @ Mirantis Inc.


 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Murano] python-openstackclient support

2015-04-22 Thread Stan Lagun
+1 for the idea though not sure on priority of this since we have so many
way more important things to implement in Kilo. I'd say that would be a
great contribution if we find someone willing to contribute it :)

Sincerely yours,
Stan Lagun
Principal Software Engineer @ Mirantis

sla...@mirantis.com

On Thu, Apr 23, 2015 at 1:55 AM, Kirill Zaitsev kzait...@mirantis.com
wrote:

 Since python-openstackclient is now a part of openstack — I guess it would
 be a good idea to support it in murano. It has setuptools-based plugin
 system, and it should be fairly easy to add murano commands as plugins to
 it.
 BTW, It’s based on cliff and has a terrific completion support (which is
 basically why I started looking into the issue in the first place =))

 What do you think, is it a good idea?

 --
 Kirill Zaitsev
 Sent with Airmail

 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Mistral] [Murano] Mistral devstack installation is failing in murano gate job

2015-04-13 Thread Stan Lagun
1) yaql 1.0 is not a drop-in replacement for yaql 0.2 but only one version
can be installed on any given system. Unless we use virtualenv, Docker or
anything else to isolate applications. So if Murano and Mistral uses
different yaql versions they will unable to live together on the same host

2) Currently we observe such impact on devstack tests but in general that
will mean Murano and Mistral cannot be installed on the same DevStack or be
together in some OpenStack distribution like Mirantis OpenStack at least
the way they are deployed currently

3) yaql 1.0 is in beta status and is ready exactly to that degree. We don't
expect any breaking changes anymore but it may still contain some bugs

4) Murano will move to yaql 1.0. We just didn't managed to do that in time
before FF and it is too late to do that in Kilo

5) Generally we should have documentation for both versions. But the fact
is that at the moment we don't even have documentation for 1.0 which is of
a higher priority for us. So once again I suggest to contribute rather than
wait for somebody else

Sincerely yours,
Stan Lagun
Principal Software Engineer @ Mirantis

sla...@mirantis.com

On Mon, Apr 13, 2015 at 9:19 PM, Dmitri Zimine dzim...@stackstorm.com
wrote:

 My 2c:

 Yes Mistral moved to YAQL 1.0 based on Murano team recommendations :)

 some questions/comments before we decide how to proceed:

 1) Let’s clarify the impact: this problem doesn’t affect Murano directly;
 but it impacts Murano-Congress-Mistral initiative, correct?
 Is this a voting gate? What exactly is impacted? Are there any simpler
 workarounds?

 2) on YAQL readiness:
 Mistral moved to YAQL it because 1) power 2) upcoming docs and 3)
 compatibility.

 We target to claim Mistral DSL “complete” in Kilo. YAQL is a big part of
 DSL from the user standpoint.
 Changing YAQL makes users migrate their workflows.
 Thus we want to stick to a version of YAQL which will be documented and
 used long term.

 If YAQL 1.0 is not ready in Kilo we should revert no questions.
 If it is ready, and comes with documentation - would it be good for Murano
 users if Murano moves to it?

 3) given that YAQL 0.2 is supported for another cycle (.5 year) and users
 of both Mistral and Murano are using it,
 are there any plans to add documentation to it? It is the lack of docs on
 0.2 is the biggest reason to push forward.
 (Does this sound like an invitation to cheat and offer no docs for 1.0 in
 kilo to convince Mistral to stay on 0.2?)

 DZ

 On Apr 13, 2015, at 6:13 AM, Serg Melikyan smelik...@mirantis.com wrote:

  Hi Nikolay  Filip,
 
  indeed, root cause of the issue is that Murano  Mistral use different
  version of yaql library. Murano installs yaql 0.2.4 and overrides
  1.0.0b2 already installed and expected by Mistral.
 
  We decided that we are not going to switch to the yaql 1.0.0 in Kilo
  since we already finished Kilo development and working on bug-fixes
  and releasing RC. This gate may be fixed if only Mistral will revert
  1.0.0 support in Kilo :'(
 
  Nikolay, what do you think about migrating to YAQL 1.0.0 in the next
  release? I know that it was me who proposed Mistral team to adopt yaql
  1.0.0, and I am sorry, I didn't realize all consequences of moving
  Mistral to yaql 1.0.0 and Murano team living with yaql 0.2.4.
 
  We need to work on packaging and supporting yaql in Ubuntu/CentOS in
  order to add this library to the global-requirements and to avoid this
  kind of issues in the future.
 
  On Mon, Apr 13, 2015 at 3:58 PM, Nikolay Makhotkin
  nmakhot...@mirantis.com wrote:
 
  We are facing an issue with Mistral devstack installation in our gate
 job testing murano-congress-mistral integration (policy enforcement) [1] .
 Mistral devstack scripts are failing with following import error [2]
 
 
  Hi, Filip!
 
  Recently Mistral has moved to new YAQL, and it seems this dependency is
 missed (yaql 1.0, currently yaql 1.0.0b2)
 
  I think the root of problem is that Murano and Mistral have different
 yaql versions installed.
 
  --
  Best Regards,
  Nikolay
 
 
 __
  OpenStack Development Mailing List (not for usage questions)
  Unsubscribe:
 openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
  http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
 
 
 
 
  --
  Serg Melikyan, Senior Software Engineer at Mirantis, Inc.
  http://mirantis.com | smelik...@mirantis.com
 
 
 __
  OpenStack Development Mailing List (not for usage questions)
  Unsubscribe:
 openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
  http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http

Re: [openstack-dev] [Mistral] Upgrading to YAQL 1.0

2015-04-06 Thread Stan Lagun
The plan was to integrate YAQL 1.0 into Murano so that we could say it is
battle-proven, remove beta from version and release on PyPI. And then
probably release v1.0.1 that is YAQL with documentation :)

Sincerely yours,
Stan Lagun
Principal Software Engineer @ Mirantis

sla...@mirantis.com

On Mon, Apr 6, 2015 at 9:44 PM, Dmitri Zimine dzim...@stackstorm.com
wrote:

 Renat,

 following up on the IRC meeting: I recalled one more item, and registered
 a blueprint - https://blueprints.launchpad.net/mistral/+spec/yaql-v1-0I
 think it’s good timing to do it in Kilo if YAQL 1.0 is announced; so I
 propose it for Kilo-rc1;

 One question is when YAQL 1.0 is going to move to Pypi? we can get it from
 github link for some time but ideally should use pypi. Alex, Stan, what’s
 the plan?

 DZ.



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Murano] Murano agent development

2015-03-24 Thread Stan Lagun
One trick that can be done to simplify agent development is to

a) configure it to listen to some specific queue and run it locally or on
VM (you can even run it from IDE under debugger) so that you can edit its
code in place
b) using code from agent.py write sample application that will send
execution plan to you agent

Sincerely yours,
Stan Lagun
Principal Software Engineer @ Mirantis

sla...@mirantis.com

On Tue, Mar 24, 2015 at 1:03 PM, Serg Melikyan smelik...@mirantis.com
wrote:

 Hi Filip,

 Sure, murano-agent is just a python application, you can easily run
 this application locally (even in IDE like Pycharm with attached
 debugger). To send execution plan to the agent you need to publish
 execution plan to specified queue, results also will be published to
 the queue.

 murano-agent require configuration file for executing, you can build
 sample configuration file using following instruction: tox -e
 genconfig

 Please, take a look at example murano-agent configuration file that
 can be used to run it locally:

 [DEFAULT]
 # set location where obtained execution plans will be stored
 storage = /tmp/murano/plans
 debug = true
 verbose = true

 # credentials for rabbitmq
 [rabbitmq]
 host = localhost
 port = 5672
 login = guest
 password = guest
 virtual_host = /

 # with this routing key execution plans results will be published
 result_routing_key = None
 # to this exchange execution plans results will be published
 result_exchange = None
 # from this queue agent will take exection plans
 input_queue = None

 On Tue, Mar 24, 2015 at 11:59 AM, Filip Blaha filip.bl...@hp.com wrote:
  Hello
 
  I would like to test and debug some new features of murano agent like
 chef
  recipes.
 
  I would like to know how to develop murano agent? How to test and debug
 new
  changes in agent code? Creating new image with every change and and test
  deployment on devstack is time-consuming and not very comfortable. Is
 there
  any shortcut for this development cycle?
 
  Thanks
  Filip
 
 
 __
  OpenStack Development Mailing List (not for usage questions)
  Unsubscribe:
 openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
  http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



 --
 Serg Melikyan, Senior Software Engineer at Mirantis, Inc.
 http://mirantis.com | smelik...@mirantis.com

 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Murano][Heat][Mistral] Use and adoption of YAQL

2015-03-10 Thread Stan Lagun
I would suggest to do the migration but not to merge it till official yaql
1.0 release which is going to happen soon.
As for the docs it is still very hard to write them since yaql 1.0 has got
tons on new features and hundreds of functions. Any help is appreciated.
But 99% of yaql 1.0 features and functions are covered by unit tests and
there are ~250 of them so this is the best source of information at the
moment.

Sincerely yours,
Stan Lagun
Principal Software Engineer @ Mirantis

sla...@mirantis.com

On Sat, Feb 7, 2015 at 12:46 AM, Dmitri Zimine dzim...@stackstorm.com
wrote:

 Stan, Alex, Renat:

 Should we migrate to YAQL 1.0 now? and stop using the initial one? What’s
 the delta?

 Still short on the docs :) but I understand they’re coming up.
 https://github.com/stackforge/yaql/tree/master/doc/source

 Cheers, Dmitri.

 On Jan 16, 2015, at 6:46 AM, Stan Lagun sla...@mirantis.com wrote:

 Dmitri,

 we are working hard towards stable YAQL 1.0 which is expected to be
 released during Kilo cycle. It is going to have proper documentation and
 high unit test coverage which can also serve as a documentation source.
 YAQL has already migrated to StackForge and adopted OpenStack development
 process and tools but the work is still in progress. Any help from Mistral
 team and/or other YAQL adopters is appreciated.



 Sincerely yours,
 Stan Lagun
 Principal Software Engineer @ Mirantis

 sla...@mirantis.com

 On Thu, Jan 15, 2015 at 10:54 PM, Dmitri Zimine dzim...@stackstorm.com
 wrote:

 Folks,

 We use YAQL in Mistral for referencing variables, expressing conditions,
 etc. Murano is using it extensively, I saw Heat folks thought of using it,
 at least once :) May be others...

 We are learning that YAQL incredibly powerful comparing to alternatives
 like Jinja2 templates used  in salt / ansible.

 But with lack of documentation, it becomes one of adoption blockers to
 Mistral (we got very vocal user feedback on this).

 This is pretty much all the docs I can offer our users on YAQL so far.
 Not much.
 http://yaql.readthedocs.org/en/latest/
 https://github.com/ativelkov/yaql/blob/master/README.rst
 https://murano.readthedocs.org/en/latest/murano_pl/murano_pl.html#yaql

 Are there any plans to fix it?

 Are there interest from other projects to use YAQL?

 Cheers,
 DZ.

 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe:
 openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://openstack-dev-requ...@lists.openstack.org/?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Murano][Heat][Mistral] Use and adoption of YAQL

2015-01-16 Thread Stan Lagun
Dmitri,

we are working hard towards stable YAQL 1.0 which is expected to be
released during Kilo cycle. It is going to have proper documentation and
high unit test coverage which can also serve as a documentation source.
YAQL has already migrated to StackForge and adopted OpenStack development
process and tools but the work is still in progress. Any help from Mistral
team and/or other YAQL adopters is appreciated.



Sincerely yours,
Stan Lagun
Principal Software Engineer @ Mirantis

sla...@mirantis.com

On Thu, Jan 15, 2015 at 10:54 PM, Dmitri Zimine dzim...@stackstorm.com
wrote:

 Folks,

 We use YAQL in Mistral for referencing variables, expressing conditions,
 etc. Murano is using it extensively, I saw Heat folks thought of using it,
 at least once :) May be others...

 We are learning that YAQL incredibly powerful comparing to alternatives
 like Jinja2 templates used  in salt / ansible.

 But with lack of documentation, it becomes one of adoption blockers to
 Mistral (we got very vocal user feedback on this).

 This is pretty much all the docs I can offer our users on YAQL so far. Not
 much.
 http://yaql.readthedocs.org/en/latest/
 https://github.com/ativelkov/yaql/blob/master/README.rst
 https://murano.readthedocs.org/en/latest/murano_pl/murano_pl.html#yaql

 Are there any plans to fix it?

 Are there interest from other projects to use YAQL?

 Cheers,
 DZ.

 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Murano] Nominating Kate Chernova for murano-core

2014-12-25 Thread Stan Lagun
+2

Sincerely yours,
Stan Lagun
Principal Software Engineer @ Mirantis

sla...@mirantis.com

On Thu, Dec 25, 2014 at 12:42 PM, Ruslan Kamaldinov 
rkamaldi...@mirantis.com wrote:

 Great addition to core team!

 +2



 On Thu, Dec 25, 2014 at 11:02 AM, Timur Sufiev tsuf...@mirantis.com
 wrote:
  +1 from me.
 
  On Thu, Dec 25, 2014 at 10:28 AM, Serg Melikyan smelik...@mirantis.com
  wrote:
 
  I'd like to propose that we add Kate Chernova to the murano-core.
 
  Kate is active member of our community for more than a year, she is
  regular participant in our IRC meeting and maintains a good score as
  contributor:
 
  http://stackalytics.com/report/users/efedorova
 
  Please vote by replying to this thread. As a reminder of your options,
 +1
  votes from 5 cores is sufficient; a -1 is a veto.
  --
  Serg Melikyan, Senior Software Engineer at Mirantis, Inc.
  http://mirantis.com | smelik...@mirantis.com
 
  ___
  OpenStack-dev mailing list
  OpenStack-dev@lists.openstack.org
  http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
 
 
 
 
  --
  Timur Sufiev
 
  ___
  OpenStack-dev mailing list
  OpenStack-dev@lists.openstack.org
  http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
 

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Murano] Murano Agent

2014-12-15 Thread Stan Lagun
Murano agent is required as long as you deploy applications that use it.
You can take (write) application that uses Heat Software Configuration
instead of Murano agent and use image without agent

Sincerely yours,
Stan Lagun
Principal Software Engineer @ Mirantis

sla...@mirantis.com

On Mon, Dec 15, 2014 at 7:54 AM, Ruslan Kamaldinov rkamaldi...@mirantis.com
 wrote:

 On Mon, Dec 15, 2014 at 7:10 AM,  raghavendra@accenture.com wrote:
  Please let me kow why Murano-agent is required and the components that
 needs
  to be installed in it.

 You can find more details about murano agent at:
 https://wiki.openstack.org/wiki/Murano/UnifiedAgent

 It can be installed with diskimage-builder:
 http://git.openstack.org/cgit/stackforge/murano-agent/tree/README.rst#n34

 - Ruslan

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Murano] - puka dependency

2014-11-21 Thread Stan Lagun
Hi,

It seems that you are using very old version of Murano because puka
dependency was dropped several releases ago.
Please see https://launchpad.net/murano for links to an up to date code and
documentation

Sincerely yours,
Stan Lagun
Principal Software Engineer @ Mirantis

sla...@mirantis.com

On Fri, Nov 21, 2014 at 9:38 AM, raghavendra@accenture.com wrote:

  Hi All,



 I am new to Murano trying to implement on Openstack Juno with Ubuntu 14.04
 LTS version. However It needs puka 0.7 as dependency.

 I would appreciate any help with the build guides. I also find Github
 prompts for username and password while I try installing packages.







 Warm Regards,

 *Raghavendra Lad*



 --

 This message is for the designated recipient only and may contain
 privileged, proprietary, or otherwise confidential information. If you have
 received it in error, please notify the sender immediately and delete the
 original. Any other use of the e-mail by you is prohibited. Where allowed
 by local law, electronic communications with Accenture and its affiliates,
 including e-mail and instant messaging (including content), may be scanned
 by our systems for the purposes of information security and assessment of
 internal compliance with Accenture policy.

 __

 www.accenture.com

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Heat] Conditionals, was: New function: first_nonnull

2014-11-12 Thread Stan Lagun
On Wed, Nov 12, 2014 at 3:50 PM, Zane Bitter zbit...@redhat.com wrote:

 It's actually potentially horrible, because you introduce potential
 quoting issues when you embed mistral workbooks in Heat templates or pass
 Heat templates to Murano.


Could you please give an example of template with such issue?
Also note that
a. YAQL uses only single quote marks (') for string while JSON uses double
quotes ()
b. If Mistral worbook is embedded into HOT template it should not be parsed
or interpreted anyway. Otherwise consider what would happen if that
workbook would in turn have HOT snippets embedded


Sincerely yours,
Stan Lagun
Principal Software Engineer @ Mirantis

sla...@mirantis.com
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Heat] Conditionals, was: New function: first_nonnull

2014-11-12 Thread Stan Lagun
My point is that quoting problem do exists probably but it exists even
without YAQL being used anywhere.
For example consider Mistral workbook containing value: { get_attr:
[my_instance, first_address] }. get_attr in Mistral may have nothing to to
with Heat's get_attr and even if it is it may be just a HOT snippet that
cannot be evaluated at the moment. For example it may be part of a HOT
template embedded into Mistral workbook and Mistral workflow is going to
create another Heat stack using that template. Now if you embed such
workbook into HOT template (and thus get HOT embedded into Mistral embedded
into HOT :) ) you need to make sure that Heat's parser is not going to try
to parse that workbook but treats it like a black box. This is regardless
of YAQL being even exist. But if you treat workbooks like
black-boxes/arbitrary JSON you should not care if it contains YAQL or not.

As for Murano this may cause some problems in cases when someone just took
YAML HOT and copy-paste it into MuranoPL class without double-checking. But
those errors can be easily fixed and such issues do exist even with current
HOT syntax without YAQL being used at all.

What I'm really confused by is why we have
 a new weird ugly language like YAQL (sorry, it, like JQ, is hideous),


YAQL syntax is almost identical to Python's or JS if you narrow them to
one-line expressions. The only noticeable difference is that all
variable/parameter names are prefixed with '$' as in PHP/Perl which are
both popular languages. I can hardly imagine why such negligible difference
from JS or Python (languages you do like) makes language weird ugly

Sincerely yours,
Stan Lagun
Principal Software Engineer @ Mirantis

sla...@mirantis.com

On Wed, Nov 12, 2014 at 6:10 PM, Clint Byrum cl...@fewbar.com wrote:

 Excerpts from Zane Bitter's message of 2014-11-11 13:06:17 -0800:
  On 11/11/14 13:34, Ryan Brown wrote:
   I am strongly against allowing arbitrary Javascript functions for
   complexity reasons. It's already difficult enough to get meaningful
   errors when you  up your YAML syntax.
 
  Agreed, and FWIW literally everyone that Clint has pitched the JS idea
  to thought it was crazy ;)
 

 So far nobody has stepped up to defend me, so I'll accept that maybe
 people do think it is crazy. What I'm really confused by is why we have
 a new weird ugly language like YAQL (sorry, it, like JQ, is hideous),
 and that would somehow be less crazy than a well known mature language
 that has always been meant for embedding such as javascript.

 Anyway, I'd prefer YAQL over trying to get the intrinsic functions in
 HOT just right. Users will want to do things we don't expect. I say, let
 them, or large sections of the users will simply move on to something
 else.

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Heat] Conditionals, was: New function: first_nonnull

2014-11-11 Thread Stan Lagun
For those who don't know 100% of guys behind YAQL are also active OpenStack
contributors. During (early) Kilo cycle we plan to release YAQL version 1.0
on stackforge. This release is going to fix some flaws in early versions,
add some more flexibility and have very high UT coverage. There are at
least 2 projects in OpenStack that already make heavy use of YAQL: Mistral
and Murano. If Heat team decides to use YAQL as well we will be glad to
help integrate it smoothly with HOT as well as make sure that YAQL 1.0 is
aligned with Heat use cases.

Sincerely yours,
Stan Lagun
Principal Software Engineer @ Mirantis

sla...@mirantis.com

On Wed, Nov 12, 2014 at 12:32 AM, Angus Salkeld asalk...@mirantis.com
wrote:



 On Wed, Nov 12, 2014 at 1:35 AM, Alexis Lee alex...@hp.com wrote:

 Alexis Lee said on Mon, Nov 10, 2014 at 05:34:13PM +:
  How about we support YAQL expressions?
 https://github.com/ativelkov/yaql
  Plus some HOFs (higher-order functions) like cond, map, filter, foldleft
  etc?

 We could also use YAQL to provide the HOFs.

  Here's first_nonnull:
 
config:
  Fn::Select
- 0
filter:
  - yaql: $.0 != null
  - item1
  - itemN

   config:
 yaql: $[$ != null][0]
 - item1
 - itemN

 This approach requires less change to Heat, at the price of learning
 more YAQL.


 +1 to YAQL


 -Angus




 Alexis
 --
 Nova Engineer, HP Cloud.  AKA lealexis, lxsli.

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] Time to Samba! :-)

2014-08-17 Thread Stan Lagun
This can be addressed by Murano only if its deployed to the cloud (on VM
belonging to some tenant). Having it on OpenStack service layer integrated
with major OpenStack services sounds very promising. The problem I see is
significant overlap with Keystone, especially in Kerberos and LDAP parts

Sincerely yours,
Stan Lagun
Principal Software Engineer @ Mirantis

 sla...@mirantis.com


On Sun, Aug 17, 2014 at 4:56 AM, Martinx - ジェームズ thiagocmarti...@gmail.com
wrote:

 I know!   :-P


 On 16 August 2014 21:17, Adam Lawson alaw...@aqorn.com wrote:

 Also, don't forget that AD != LDAP. ;)
 On Aug 16, 2014 5:16 PM, Adam Lawson alaw...@aqorn.com wrote:

 Doesn't Murano address this already?
 On Aug 16, 2014 2:35 PM, Martinx - ジェームズ thiagocmarti...@gmail.com
 wrote:

 I think that it would be great too! OpenLDAP-as-a-Service... With
 multi-domain support! :-)

 Nevertheless, last time I used Samba, was back in 2001... It is
 impressive these days! It worth take a look... I'm using it for about two
 months now, it is great!

 Cheers!


 On 16 August 2014 18:01, Clint Byrum cl...@fewbar.com wrote:

 Excerpts from Martinx - ジェームズ's message of 2014-08-16 12:03:20 -0700:
  Hey Stackers,
 
   I'm wondering here... Samba4 is pretty solid (up coming 4.2 rocks),
 I'm
  using it on a daily basis as an AD DC controller, for both Windows
 and
  Linux Instances! With replication, file system ACLs - cifs, built-in
 LDAP,
  dynamic DNS with Bind9 as a backend (no netbios) and etc... Pretty
 cool!
 
   In OpenStack ecosystem, there are awesome solutions like Trove,
 Solum,
  Designate and etc... Amazing times BTW! So, why not try to integrate
  Samba4, working as an AD DC, within OpenStack itself?!
 

 But, if we did that, what would be left for us to reinvent in our own
 slightly different way?

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [all] The future of the integrated release

2014-08-17 Thread Stan Lagun
On Fri, Aug 15, 2014 at 7:17 PM, Sandy Walsh sandy.wa...@rackspace.com
wrote:

 I recently suggested that the Ceilometer API (and integration tests) be
 separated from the implementation (two repos) so others might plug in a
 different implementation while maintaining compatibility, but that wasn't
 well received.

 Personally, I'd like to see that model extended for all OpenStack
 projects. Keep compatible at the API level and welcome competing
 implementations.


Brilliant idea I'd vote for


Sincerely yours,
Stan Lagun
Principal Software Engineer @ Mirantis

 sla...@mirantis.com
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Murano]

2014-07-22 Thread Stan Lagun
Hi Steve,

1. There are no objections whatsoever if you know how to do it without
breaking the entire concept
2. I thing that deployment workflow need to be broken to more fine-grained
steps. Maybe instead of single deploy methdos have prepareDeploy (which
doesn't push the changes to Heat), deploy and finishDeploy. Maybe
more/other methods need to be defined. This will make the whole process
more customizible
3. If you want to have single-instance applications based on a fixed
prebuild image then maybe what you need is to have your apps inhertir both
Application and Instance classes and then override Instance's deploy method
and add HOT snippet before VM instantiation. This may also require ability
for child class to bind fixed values to parent class properties (narrowing
class public contract, hiding those properties from user). This is not yet
supported in MuranoPL but can be done in UI form as a temporary workaround
4. Didn't get why you mentioned object model. Object model is mostly user
input. Do you suggest passing HOT snippets as part of user input? If so
that would be something I oppose to
5. I guess image tagging would be better solution to image-based deployment
6. Personally I believe that problem can be eficently solved by Murano
today or in the nearest future without resorting to pure HOT packages. This
is not against Murano design and perfectly alligned with it


Sincerely yours,
Stan Lagun
Principal Software Engineer @ Mirantis

 sla...@mirantis.com


On Tue, Jul 22, 2014 at 8:05 PM, McLellan, Steven steve.mclel...@hp.com
wrote:

  Hi,



 This is a little rambling, so I’ll put this summary here and some
 discussion below. I would like to be able to add heat template fragments
 (primarily softwareconfig) to a template before an instance is created by
 Heat. This could be possible by updating but not pushing the heat template
 before instance.deploy, except that instance.deploy does a stack.push to
 configure networking before it adds information about the nova instance.
 This seems like the wrong place for the networking parts of the stack to be
 configured (maybe in the Environment before it tries to deploy
 applications). Thoughts?



 --



 The long version:



 I’ve been looking at using disk-image-builder (a project that came out of
 triple-o) to build images for consumption through Murano. Disk images are
 built from a base OS plus a set of ‘elements’ which can include packages to
 install when building the image, templatized config file etc, and allows
 for substitutions based on heat metadata at deploy time. This uses a lot of
 the existing heat software config agents taking configuration from
 StructuredConfig and StructuredDeployment heat elements.



 I’m typically finding for our use cases that instances will tend to be
 single purpose (that is, the image will be created specifically to run a
 piece of software that requires some configuration). Currently Murano
 provisions the instance, and then adds software configuration as a separate
 stack-update step. This is quite inefficient since os-refresh-config ends
 up having to re-run, and so I’m wondering if there’s strong opposition to
 allowing the object model to support injection of software configuration
 heat elements before the instance is deployed.



 Alternatively maybe this is something that is best supported by pure HOT
 packages, but I think there’s value having murano’s composition ability
 even if just to be able to combine heat fragments (perhaps in the drag 
 drop manner that was briefly discussed in Atlanta).



 Thanks,



 Steve



 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Murano] Image tagging

2014-07-22 Thread Stan Lagun
How do you like alternate design: uses can chose any image he wants (say
any Linux) but the JSON that is in image tag has enough information on what
applications are installed on that image. And not just installed or not but
the exact state installation was frozen (say binaries are deployed but
config files are need to be modified). The deployment workflow can peak
that state from image tag and continue right from the place it was stopped
last time. So if user has chosen image with MySQL preinstalled the workflow
will just post-configure it while if the user chosen clean Linux image it
will do the whole deployment from scratch. Thus it will become only a
matter of optimization and user will still be able to to share instance for
several applications (good example is Firewall app) or deploy his app even
if there is no image where it was built in.

Those are only my thoughts and this need a proper design. For now I agree
that we need to improve tagging to support yours use case. But this need to
be done in a way that would allow both user and machine to work with. UI at
least needs to distinguish between Linux and Windows while for user a
free-form tagging may be appropriate. Both can be stored in a single JSON
tag.

So lets create blueprint/etherpad for this and both think on exact format
that can be implemented right now

Sincerely yours,
Stan Lagun
Principal Software Engineer @ Mirantis

 sla...@mirantis.com


On Tue, Jul 22, 2014 at 10:08 PM, McLellan, Steven steve.mclel...@hp.com
wrote:

  Thanks for the response.



 Primarily I’m thinking about a situation where I have an image that has a
 specific piece of software installed (let’s say MySQL for the sake of
 argument). My application (which configures mysql) requires a glance image
 that has MySQL pre-installed, and doesn’t particularly care what OS (though
 again for the sake of argument assume it’s linux of some kind, so that
 configuration files are expected to be in the same place regardless of OS).



 Currently we have a list of three hardcoded values in the UI, and none of
 them apply properly. I’m suggesting instead of that list, we allow
 free-form text; if you’re tagging glance images, you are expected to know
 what applications will be looking for. This still leaves a problem in that
 I can upload a package but I don’t necessarily have the ability to mark any
 images as valid for it, but I think that can be a later evolution; for now,
 I’m focusing on the situation where an admin is both uploading glance
 images and murano packages.



 As a slight side note, we do have the ability to filter image sizes based
 on glance properties (RAM, cpus), but this is in the UI code, not enforced
 at the contractual level. I agree reengineering some of this to be at the
 contract level is a good goal, but it seems like that would involve major
 reengineering of the dashboard to make it much dumber and go through the
 murano API for everything (which ultimately is probably a good thing).



 *From:* Stan Lagun [mailto:sla...@mirantis.com]
 *Sent:* Sunday, July 20, 2014 5:42 AM
 *To:* OpenStack Development Mailing List (not for usage questions)
 *Subject:* Re: [openstack-dev] [Murano] Image tagging



 Hi!



 I think it would be usefull to share the original vision on tagging that
 we had back in 0.4 era when it was introduced.

 Tagging was supposed to be JSON image metadata with extendable scheme.
 Workflow should be able to both utilize that metadata and impose some
 constraints on it. That feature was never really designed so I cannot tell
 exactly how this JSON should work or look like. As far as I see it it can
 contain:



 1. Operating system information. For example os: { family: Linux,
 name: Ubuntu, version: 12.04, arch: x86_x64 } (this also may be
 encoded as a single string)

 Workflows (MuranoPL contracts) need to be able to express requirements
 based on those attributes. For example



 image:

   Contract($.class(Image).check($.family = Linux and $.arch = x86)



In UI only those images that matches such contract should be displayed.



 2. Human readable image title Ubuntu Linux 12.04 x86



 3. Information about built-in software for image-based deployment. Not
 sure exactly what information is needed. Meybe even portion of Object Model
 so that if such image is used Murano environment will automatically
 recocnize and incorporate that application like it was added by user to be
 installed on clean instance. This will allow using of pre-build images with
 preinstalled software (e.g. speed up deployment) but will make it
 transparent for the user so that this software could be managed as well as
 applications that user choses to install



 4. Minimal hardware requirement for the image. Murano could use that
 information to guarantee that user will not select flavor that is too small
 for that operating system.



 5. General-purposed tags



 We need to think how this concept fits into our roadmap and new Glance

Re: [openstack-dev] [Murano] Image tagging

2014-07-20 Thread Stan Lagun
Hi!

I think it would be usefull to share the original vision on tagging that we
had back in 0.4 era when it was introduced.
Tagging was supposed to be JSON image metadata with extendable scheme.
Workflow should be able to both utilize that metadata and impose some
constraints on it. That feature was never really designed so I cannot tell
exactly how this JSON should work or look like. As far as I see it it can
contain:

1. Operating system information. For example os: { family: Linux,
name: Ubuntu, version: 12.04, arch: x86_x64 } (this also may be
encoded as a single string)
Workflows (MuranoPL contracts) need to be able to express requirements
based on those attributes. For example

image:
  Contract($.class(Image).check($.family = Linux and $.arch = x86)

   In UI only those images that matches such contract should be displayed.

2. Human readable image title Ubuntu Linux 12.04 x86

3. Information about built-in software for image-based deployment. Not sure
exactly what information is needed. Meybe even portion of Object Model so
that if such image is used Murano environment will automatically recocnize
and incorporate that application like it was added by user to be installed
on clean instance. This will allow using of pre-build images with
preinstalled software (e.g. speed up deployment) but will make it
transparent for the user so that this software could be managed as well as
applications that user choses to install

4. Minimal hardware requirement for the image. Murano could use that
information to guarantee that user will not select flavor that is too small
for that operating system.

5. General-purposed tags

We need to think how this concept fits into our roadmap and new Glance
design (probably there are other projects that can benefit from extended
image metadata) before chosing one of your approaches



Sincerely yours,
Stan Lagun
Principal Software Engineer @ Mirantis

 sla...@mirantis.com


On Fri, Jul 18, 2014 at 6:46 PM, McLellan, Steven steve.mclel...@hp.com
wrote:

  Hi,



 Images that can be used for package deployment have to be tagged in glance
 in order to enable the UI to filter the list of images to present to a user
 (and potentially preselect). Currently the tags are defined in the
 dashboard code (images/forms.py) which makes things very inflexible; if I
 can upload an image and a package that consumes that image, I don’t want to
 have to make a code change to use it.



 Anyone who can upload images should also be able to specify tags for them.
 There is also the question of whether a user should be allowed to tag
 images that don’t belong to them (e.g. a shared image used by a private
 package), but I think that can be punted down the road somewhat.



 I think this needs to be more dynamic, and if that’s agreed upon, there
 are a couple of approaches:

 1)  Store allowed tags in the database, and allow administrators to
 add to that list. Ordinary users would likely not be able to create tags,
 though they could use pre-defined ones for images they owned.

 2)  Have some public tags, but also allow user-specified tags for
 private packages. I think this leads to all sorts of tricky edge cases

 3)  Allow freeform tags (i.e. don’t provide any hints). Since there’s
 no formal link between the tag that a package looks for and the tags
 currently defined in code, this wouldn’t make anything more susceptible to
 inaccuracies



 It would also be worth considering if there’s value allowing multiple tags
 per image (I’m on the fence).



 Personally, I think that 1) would be an improvement over the current
 situation that’s reasonably easy to implement; that would allow the bare
 minimum of flexibility without requiring much extra design. The other
 options would perhaps be a longer term goal.



 Thoughts? If there’s general agreement I will turn this into a blueprint.



 Steve

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Murano] Field 'name' is removed from Apps dynamic UI markup, should 'Version' be changed?

2014-07-06 Thread Stan Lagun
I we increment version to say 2.1 we could add code to dashboard to check
for markup version and if it encounters version 2.0 to print verbose error
telling how to migrate markup to 2.1.
I don't see how both version can be supported simulteniously but at lease
Version attribute must be checked and forms older version must fail with
descriptive message rather than causing unpredictable behavior.

Sincerely yours,
Stan Lagun
Principal Software Engineer @ Mirantis

 sla...@mirantis.com


On Fri, Jul 4, 2014 at 8:24 PM, Timur Sufiev tsuf...@mirantis.com wrote:

 Hi, folks!

 Recently we had decided to change a bit how Murano's dynamic UI works,
 namely do not explicitly specify 'name' field in first 'Add
 Application' form, but add it here automatically, since every
 component in Murano has a name. To avoid confusion with the 'name'
 field added by hand to the first form's markup, 'name' field on the
 first step will be forbidden and processing of an old UI markup which
 has such field will cause an exception. All these changes are
 described in the blueprint [1] in a greater detail.

 What is not entirely clear to me is whether should we increase
 'Version' attribute of UI markup or not? On one hand, the format of UI
 markup is definitely changing - and old UI definitions won't work with
 the UI processor after [1] is implemented. It is quite reasonable to
 bump a format's version to reflect that fact. On the other hand, we
 will hardly support both format versions, instead we'll rewrite UI
 markup in all existing Murano Apps (there are not so many of them yet)
 and eventually forget that once upon a time the user needed to specify
 'name' field explicitly.

 What do you think?

 [1]
 https://blueprints.launchpad.net/murano/+spec/dynamic-ui-specify-no-explicit-name-field

 --
 Timur Sufiev

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Murano] Nominations to Murano core

2014-06-26 Thread Stan Lagun
+1 for both (or should I say +2?)

Sincerely yours,
Stan Lagun
Principal Software Engineer @ Mirantis

 sla...@mirantis.com


On Thu, Jun 26, 2014 at 1:49 PM, Alexander Tivelkov ativel...@mirantis.com
wrote:

 +1 on both Serge and Steve

 --
 Regards,
 Alexander Tivelkov


 On Thu, Jun 26, 2014 at 1:37 PM, Ruslan Kamaldinov 
 rkamaldi...@mirantis.com wrote:

 I would like to nominate Serg Melikyan and Steve McLellan to Murano core.

 Serge has been a significant reviewer in the Icehouse and Juno release
 cycles.

 Steve has been providing consistent quality reviews and they continue
 to get more frequent and better over time.


 Thanks,
 Ruslan

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Murano] environment deletion

2014-06-25 Thread Stan Lagun
Hi Steve,

The initial implementation of environment deletion (before 'destroy' method
was introduced in MuranoPL) was to deploy environment with empty
application list. So the code that deletes Heat stack was in 'deploy'. And
it is still there. Environment's 'destroy' will be executed only in case
Object Model is empty (doesn't contain Environment object), not in the case
it is present, but application list is empty. Now if you say that stack
doesn't get deleted it is clearly a bug that need to be filled in
launchpad. I'm not sure why it can happen. Maybe API behavior was changed
recently to send empty Object Model or it sends incorrect action name. This
just need to be debugged.



Sincerely yours,
Stan Lagun
Principal Software Engineer @ Mirantis

 sla...@mirantis.com


On Wed, Jun 25, 2014 at 3:57 AM, McLellan, Steven steve.mclel...@hp.com
wrote:

  Is there any reason the system Environment class doesn’t implement
 destroy? Without it, the pieces of the heat stack not owned by other
 resources get left lying around. It looks like it was once implemented as
 part of deploy, but that no longer seems to execute.

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Murano] Murano API improvements

2014-06-02 Thread Stan Lagun
I think API need to be redesigned at some point. There is a blueprint for
this: https://blueprints.launchpad.net/murano/+spec/api-vnext
It seems reasonable to implement new API on new framework at once

Sincerely yours,
Stan Lagun
Principal Software Engineer @ Mirantis

 sla...@mirantis.com


On Mon, Jun 2, 2014 at 12:21 PM, Ruslan Kamaldinov rkamaldi...@mirantis.com
 wrote:

 Let's follow the standard procedure. Both blueprints lack specification of
 implementation details. There also has to be someone willing to implement
 these
 blueprints in near feature.

 I'm not opposed to these ideas and I'd really like to see Pecan added
 during
 Juno, but we still need to follow the procedure. I cannot approve an idea,
 it
 should be a specification. Let's work together on the new API specification
 first, then we'll need to find a volunteer to implement it on top of Pecan.


 --
 Ruslan

 On Mon, Jun 2, 2014 at 8:35 AM, Timur Nurlygayanov
 tnurlygaya...@mirantis.com wrote:
  Hi all,
 
  We need to rewrite Murano API on new API framework and we have the
 commit:
  https://review.openstack.org/#/c/60787
  (Sergey, sorry, but -1 from me, need to fix small isses)
 
  Also, today I created blueprint:
  https://blueprints.launchpad.net/murano/+spec/murano-api-workers
  this feature allows to run many API threads on one host and this allows
 to
  scale Murano API processes.
 
  I suggest to update and merge this commit with migration to Pecan
 framework
  and after that we can easily implement this blueprint and add many other
  improvements to Murano API and Murano python agent.
 
  Ruslan, could you please approve these blueprints and target them to some
  milestone?
 
 
  Thank you!
 
  --
 
  Timur,
  QA Engineer
  OpenStack Projects
  Mirantis Inc
 
  ___
  OpenStack-dev mailing list
  OpenStack-dev@lists.openstack.org
  http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
 

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Murano] Application Actions

2014-05-30 Thread Stan Lagun
Agree. Lets for now assume nothing about events and have separate list of
blueprints for actions. As soon as we get back to event design we will
decide how they connected with actions and maybe then mark some of event
blueprints as obsolete/superseded

Sincerely yours,
Stan Lagun
Principal Software Engineer @ Mirantis

 sla...@mirantis.com


On Fri, May 30, 2014 at 9:56 AM, Serg Melikyan smelik...@mirantis.com
wrote:

 Hi, Alexander

 Thank you for formalizing specifications and requirements! I will be glad
 to work on implementation of app actions, once we finish discussing them. I
 have created new blueprint
 https://blueprints.launchpad.net/murano/+spec/application-actions and
 referenced etherpad as specification.
 Actually your specification in etherpad looks very comprehensive and quite
 enough to implement first version of the feature. I think I can start
 drafting implementation while we also working on specification at the same
 time.

 I think Events should be based on Actions, so we can return to Events in
 the next milestone and revisit them.


 On Thu, May 29, 2014 at 8:26 PM, Alexander Tivelkov 
 ativel...@mirantis.com wrote:

 Hi folks!

 During the Atlanta Summit there was quite a lot of talks about the
 Application Lifecycle management and Murano's role in this process. There
 were several cross-project sessions between Murano, Heat and Solum teams
 ([1]) at which it was decided that Murano has its own place in the
 application-management ecosystem and should be able to define custom
 actions or workflows for its applications, while using Heat and its ALM
 capabilities as the underlying service.

 At the same time I had some conversation with potential customers and
 contributors, who have expressed strong interest in having actions in
 Murano in this cycle.

 That's why I've decided to drive this process forward and formalize the
 spec and requirements for the Actions feature in Murano.
 I've created a draft of the spec - please see the etherpad at [2] for
 details. I'd like some comments and discussion on the spec, and once we all
 agree on that, I will be happy to find a volunteer eager to implement this
 during Juno :)

 BTW, we have a number of blueprints already created on the topic ([3],
 [4], [5], [6]), but they lack the details and have some problems with
 terminology ('events' and 'actions' are definitely not the same, while the
 blueprints mix them). I think we should revisit these BPs and either change
 them to reflect the updated vision or to mark them as superseded and create
 more appropriate one.


 [1] https://etherpad.openstack.org/p/9XQ7Q2NQdv
 [2] https://etherpad.openstack.org/p/MuranoActions
 [3] https://blueprints.launchpad.net/murano/+spec/external-events
 [4] https://blueprints.launchpad.net/murano/+spec/api-list-events
 [5] https://blueprints.launchpad.net/murano/+spec/dsl-register-event
 [6]
 https://blueprints.launchpad.net/murano/+spec/ui-application-event-list


 --
 Regards,
 Alexander Tivelkov

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




 --
 Serg Melikyan, Senior Software Engineer at Mirantis, Inc.
 http://mirantis.com | smelik...@mirantis.com

 +7 (495) 640-4904, 0261
 +7 (903) 156-0836

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Murano] Improvements to MuranoPL contracts

2014-05-27 Thread Stan Lagun
Hi Alex,

1. This is directly related to the bug mentioned above.

Taking an example from launchpad

networks:
Contract:
  environment: $.bool().notNull()
  flat: $.bool().notNull()
  custom: [$.class(Network).notNull()]

Default:
  environment: true
  flat: false


currently if no value was passed for 'networks' property (and no value
means no such key, null value doesn't count) then default value would be
used ({environment: true, flat: false}) and contract would be applied
to that value. If you pass networks: {flat: true} you will get
exception as this value doesn't matched contract (for 'environment' key).
With new approach exactly the same Default is interpreted as a set of
independent default - default for 'environment' attribute and default for
'flat' attribute. So {flat: true} turns into {environment: true,
flat: true}. With proposed changes the example above can be rewritten as

networks:
Contract:
  environment: $.bool()
  flat: $.bool()
  custom: [$.class(Network)]

Default:
  environment: true

2. Yes, I do agree it breaks compatibility. I believe we will brake it
anyway because MuranoPL is still far from mature and it is very likely that
some new future would require incompatible change. Thats was one of the
purpose of app-incubator repository - if we make breaking changes we can
walk through all applications and fix them. I hope that all such changes
will be made in Juno time frame.
As for this particular change it is 1% chance that $.int() contract was
intentionally written to accept null and 99% that it was mistake. So making
$.int() not-nullable we do much more good than cause harm


Sincerely yours,
Stan Lagun
Principal Software Engineer @ Mirantis

 sla...@mirantis.com


On Tue, May 27, 2014 at 1:37 PM, Alexander Tivelkov
ativel...@mirantis.comwrote:

 Hi Stan,

 I don't understand well your third problem/solution statement. Could you
 please give an example of the proposed behaviour vs the current one?

 The last solution - to have not-nullable primitive types by default -
 looks good to me, but it breaks the compatibility with the existing
 contracts. If some package developers have already got $.int() in their
 classes, we cannot be sure if this is a mistake or an intention to make the
 property nullable. In the latter case their code will be broke, as the null
 values will not be accepted anymore.

 I understand that currently the adoption of MuranoPL is quite low and most
 of its users are murano-contributors and so are probably reading this
 mailing list, so this breaking change should not surprise them, but I'd
 rather be careful with such things anyway. I suggest to discuss this today
 on a weekly meeting in IRC. If everybody are fine with such changes, then
 let's do them, as they are reasonable.



 --
 Regards,
 Alexander Tivelkov


 On Mon, May 26, 2014 at 6:40 PM, Stan Lagun sla...@mirantis.com wrote:

 Hello everyone!

 Recently I've been looking for a best way to fix incorrect handling of
 defaults in MuranoPL's property contracts [1]. I analyzed how contracts
 used in applications that are currently in app-incubator, typical mistakes
 and usage patterns. I've found number of places where contract system can
 be significantly improved. Here is my list of proposed improvements I'd
 like to deliberate with you:

 Problem: when contract violation happens it is very hard to tell without
 debugger what went wrong. Especially this is a problem for complex nested
 structures. Currently it even impossible to tell which property caused
 exception during class load

 Solution: during contract traversal keep track of path to a part of
 contract being processed (for example 'foo/0/bar'). Include that path in
 every thrown exception alongside with human-readable description describing
 what value was processed and what contract was violated. Prepend property
 name to exception text so that is would be clear what property cannot be
 initialized. The same for method arguments.

 ---

 Problem: Single default value is not enough for properties that are not
 of scalar type.
 For example if we have contract like
 - name: $.string()
   disabled: $.bool()

 (array of structures) it is reasonable to have default value for
 disabled attribute of each list entry whereas there is no meaningful
 default for name attribute. Current approach allows to provide initial
 filling of entire array which is obviously not what developer expects.

 Solution: to have Default reflect contract structure so that different
 parts of Default can serve as an independent defaults for corresponding
 parts of the contract. Default need to be traversed in parallel with
 contract spec and provided value. Default value need not provide value for
 every single attribute mentioned in contract spec and thus can be simpler
 than contract itself.

 ---

 Problem: Default value is only used when no property value provided at
 all. Null (None) value is a valid value even

Re: [openstack-dev] [Murano] Improvements to MuranoPL contracts

2014-05-27 Thread Stan Lagun
You're correct for not-nullable contract types. I agree that it is less
obvious. But on the other side it simplifies contract understanding because
otherwise developer needs to know not only how null is treated but also how
NoValue is treated. This also make UI development easier as with null equal
to NoValue you can have templated object model that you substitute values
from web form into. With nulls the structure always remain the same only
values change while with NoValue you have to delete key from template to
get default.

What is better? I don't know. Lets wait for other opinions

Sincerely yours,
Stan Lagun
Principal Software Engineer @ Mirantis

 sla...@mirantis.com


On Tue, May 27, 2014 at 4:29 PM, Alexander Tivelkov
ativel...@mirantis.comwrote:

 Got it.
 What about this it is easier to set attribute value to None rather then
 deleting attribute from object if default value is desired, especially for
 generic clients?
 Do I understand correctly that setting some property to None should leads
 to automatic assignment of the default value?

 As for me, this looks not obvious, and I would prefer not to have such
 implicit assignments

 --
 Regards,
 Alexander Tivelkov


 On Tue, May 27, 2014 at 4:05 PM, Stan Lagun sla...@mirantis.com wrote:

 Hi Alex,

 1. This is directly related to the bug mentioned above.

 Taking an example from launchpad

 networks:
 Contract:
   environment: $.bool().notNull()
   flat: $.bool().notNull()
   custom: [$.class(Network).notNull()]

 Default:
   environment: true
   flat: false


 currently if no value was passed for 'networks' property (and no value
 means no such key, null value doesn't count) then default value would be
 used ({environment: true, flat: false}) and contract would be applied
 to that value. If you pass networks: {flat: true} you will get
 exception as this value doesn't matched contract (for 'environment' key).
 With new approach exactly the same Default is interpreted as a set of
 independent default - default for 'environment' attribute and default for
 'flat' attribute. So {flat: true} turns into {environment: true,
 flat: true}. With proposed changes the example above can be rewritten as

 networks:
 Contract:
   environment: $.bool()
   flat: $.bool()
   custom: [$.class(Network)]

 Default:
   environment: true

 2. Yes, I do agree it breaks compatibility. I believe we will brake it
 anyway because MuranoPL is still far from mature and it is very likely that
 some new future would require incompatible change. Thats was one of the
 purpose of app-incubator repository - if we make breaking changes we can
 walk through all applications and fix them. I hope that all such changes
 will be made in Juno time frame.
 As for this particular change it is 1% chance that $.int() contract was
 intentionally written to accept null and 99% that it was mistake. So making
 $.int() not-nullable we do much more good than cause harm


 Sincerely yours,
 Stan Lagun
 Principal Software Engineer @ Mirantis

  sla...@mirantis.com


 On Tue, May 27, 2014 at 1:37 PM, Alexander Tivelkov 
 ativel...@mirantis.com wrote:

 Hi Stan,

 I don't understand well your third problem/solution statement. Could you
 please give an example of the proposed behaviour vs the current one?

 The last solution - to have not-nullable primitive types by default -
 looks good to me, but it breaks the compatibility with the existing
 contracts. If some package developers have already got $.int() in their
 classes, we cannot be sure if this is a mistake or an intention to make the
 property nullable. In the latter case their code will be broke, as the null
 values will not be accepted anymore.

 I understand that currently the adoption of MuranoPL is quite low and
 most of its users are murano-contributors and so are probably reading this
 mailing list, so this breaking change should not surprise them, but I'd
 rather be careful with such things anyway. I suggest to discuss this today
 on a weekly meeting in IRC. If everybody are fine with such changes, then
 let's do them, as they are reasonable.



 --
 Regards,
 Alexander Tivelkov


 On Mon, May 26, 2014 at 6:40 PM, Stan Lagun sla...@mirantis.com wrote:

 Hello everyone!

 Recently I've been looking for a best way to fix incorrect handling of
 defaults in MuranoPL's property contracts [1]. I analyzed how contracts
 used in applications that are currently in app-incubator, typical mistakes
 and usage patterns. I've found number of places where contract system can
 be significantly improved. Here is my list of proposed improvements I'd
 like to deliberate with you:

 Problem: when contract violation happens it is very hard to tell
 without debugger what went wrong. Especially this is a problem for complex
 nested structures. Currently it even impossible to tell which property
 caused exception during class load

 Solution: during contract traversal keep track of path to a part

[openstack-dev] [Murano] Improvements to MuranoPL contracts

2014-05-26 Thread Stan Lagun
Hello everyone!

Recently I've been looking for a best way to fix incorrect handling of
defaults in MuranoPL's property contracts [1]. I analyzed how contracts
used in applications that are currently in app-incubator, typical mistakes
and usage patterns. I've found number of places where contract system can
be significantly improved. Here is my list of proposed improvements I'd
like to deliberate with you:

Problem: when contract violation happens it is very hard to tell without
debugger what went wrong. Especially this is a problem for complex nested
structures. Currently it even impossible to tell which property caused
exception during class load

Solution: during contract traversal keep track of path to a part of
contract being processed (for example 'foo/0/bar'). Include that path in
every thrown exception alongside with human-readable description describing
what value was processed and what contract was violated. Prepend property
name to exception text so that is would be clear what property cannot be
initialized. The same for method arguments.

---

Problem: Single default value is not enough for properties that are not of
scalar type.
For example if we have contract like
- name: $.string()
  disabled: $.bool()

(array of structures) it is reasonable to have default value for disabled
attribute of each list entry whereas there is no meaningful default for
name attribute. Current approach allows to provide initial filling of
entire array which is obviously not what developer expects.

Solution: to have Default reflect contract structure so that different
parts of Default can serve as an independent defaults for corresponding
parts of the contract. Default need to be traversed in parallel with
contract spec and provided value. Default value need not provide value for
every single attribute mentioned in contract spec and thus can be simpler
than contract itself.

---

Problem: Default value is only used when no property value provided at all.
Null (None) value is a valid value even if it not valid for particular
contract. Thus is null is passed Default is not used.

Solution: to treat every missing value as null. Missing Default is also
considered to be null. This greatly simplifies understanding of contracts
and client development (it is easier to set attribute value to None rather
then deleting attribute from object if default value is desired, especially
for generic clients)

---

Problem: most of the time developer writes $.int() contract he doesn't
realize that null is also valid value for such contract and the correct
form is $.int().notNull(). This is even more obvious in case of bool().
Developers are lazy and usually forget to wright long verbose contracts.

Solution: to make all primitive types be not-nullable with obvious defaults
(0 for int, false for bool).
This is enough for 95% of use cases. For the rest 5% where null value does
valid to have separate contract methods optionalInt(), optionalBool() etc.

As for strings I think that the same approach should be used - $.string()
is a non-nullable sequence of chars with empty string as a default. And
there should also be optionalString() that es equal to current $.string().
But in most cases when developer writes $.string() what he really wants is
$.string().notNull().trim().notEmpty(). So while it is reasonable to have
all this helper functions (notEmpty(), notBlank(), trim() etc.) is would be
great to have one contract function that does exactly that.

---

While some of proposed changes can possible break existing contracts in
practice they just legalize current state of affairs so nothing would break.

What do you think?

[1]: https://bugs.launchpad.net/murano/+bug/1313694

Sincerely yours,
Stan Lagun
Principal Software Engineer @ Mirantis

 sla...@mirantis.com
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [murano] Proposal to add Ruslan Kamaldinov to murano-core team

2014-04-17 Thread Stan Lagun
+1

Sincerely yours,
Stan Lagun
Principal Software Engineer @ Mirantis

 sla...@mirantis.com


On Thu, Apr 17, 2014 at 6:51 PM, Georgy Okrokvertskhov 
gokrokvertsk...@mirantis.com wrote:

 +1


 On Thu, Apr 17, 2014 at 6:01 AM, Dmitry Teselkin 
 dtesel...@mirantis.comwrote:

 +1

 Agree


 On Thu, Apr 17, 2014 at 4:51 PM, Alexander Tivelkov 
 ativel...@mirantis.com wrote:

 +1

 Totally agree

 --
 Regards,
 Alexander Tivelkov


 On Thu, Apr 17, 2014 at 4:37 PM, Timur Sufiev tsuf...@mirantis.comwrote:

 Guys,

 Ruslan Kamaldinov has been doing a lot of things for Murano recently
 (including devstack integration, automation scripts, making Murano
 more compliant with OpenStack standards and doing many reviews). He's
 actively participating in our ML discussions as well. I suggest to add
 him to the core team.

 Murano folks, please say your +1/-1 word.

 --
 Timur Sufiev

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




 --
 Thanks,
 Dmitry Teselkin
 Deployment Engineer
 Mirantis
 http://www.mirantis.com

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




 --
 Georgy Okrokvertskhov
 Architect,
 OpenStack Platform Products,
 Mirantis
 http://www.mirantis.com
 Tel. +1 650 963 9828
 Mob. +1 650 996 3284

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Heat] [Murano] [Solum] applications in the cloud

2014-04-04 Thread Stan Lagun
On Fri, Apr 4, 2014 at 9:05 PM, Clint Byrum cl...@fewbar.com wrote:

 IMO that is not really true and trying to stick all these databases into
 one SQL database interface is not a use case I'm interested in
 pursuing.


Indeed. Any SQL database is a useless interface. What I was trying to say
is that some apps may work just on any MySQL while others require some
specific version or variation or even impose some constraints on license or
underlying operating system. One possible solution was to have some sort of
interface hierarchy for that. Even better solution would be that all such
properties be declared somewhere in HOT so that consumer could say not just
I require MySQL-compatible template but I require MySQL-compatible
template with version = 5.0 and clustered = True. Probably you can come
with better example for this. Though interface alone is a good starting
point.

So for instance there is the non-Neutron LBaaS and the Neutron LBaaS, and
 both have their
 merits for operators, but are basically identical from an application
 standpoint.


While conforming to the same interface from consumer's point of view
different load-balancers has many configuration options (template
parameters) and many of them are specific to particular implementation
(otherwise all of them be functionally equal). This need to be addressed
somehow.

Sincerely yours,
Stan Lagun
Principal Software Engineer @ Mirantis

 sla...@mirantis.com
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Murano][Heat] MuranoPL questions?

2014-03-25 Thread Stan Lagun
On Tue, Mar 25, 2014 at 2:27 PM, Thomas Herve thomas.he...@enovance.comwrote:


  What I can say is that I'm not convinced. The only use-case for a DSL
 would
  be if you have to upload user-written code, but what you mentioned is a
 Web
  interface, where the user doesn't use the DSL, and the cloud provider
 is the
  developer. There is no reason in this case to have a secure environment
 for
  the code.
 
  I didn't say that. There are at least 2 different roles application
  developers/publishers and application users. Application developer is not
  necessary cloud provider. The whole point of AppCatalog is to support
  scenario when anyone can create and package some application and that
  package can be uploaded by user alone. Think Apple AppStore or Google
 Play.
  Some cloud providers may configure ACLs so that user be allowed to
 consume
  applications they decided while others may permit to upload applications
 to
  some configurable scope (e.g. apps that would be visible to all cloud
 users,
  to particular tenant or be private to the user). We also think to have
 some
  of peer relations so that it would be possible to have application
 upload in
  one catalog to become automatically available in all connected catalogs.
 
  This is similar to how Linux software repos work - AppCatalog is repo,
 Murano
  package is what DEB/RPMs are to repo and DSL is what DEB/RPMs manifests
 are
  to packages. Just that is run on cloud and designed to handle complex
  multi-node apps as well as trivial ones in which case this may be
 narrowed
  to actual installation of DEB/RPM

 I'm glad that you bring packages up. This is a really good example of why
 you don't need a new programming language. Packages uses whatever
 technology they prefer to handle their scripting needs. They then have an
 declarative interface which hides the imperative parts behind.


The same is true for Murano. MuranoPL is not used to express what should be
deployed. In Murano there is object model that describes view of the word.
It serves for the same purpose as HOT in Heat but it is simpler because it
says just what need to be deployed but not how it should be accomplished as
this information is already contained in application definitions. There is
REST API to edit/submit Object Model which is again has nothing to do with
MuranoPL. UI dashboard talks to AppCatalog to see what applications/classes
are available and AppCatalog also knows what are the properties of those
classes. This is needed for UI so that it can ask user for appropriate
input. This is similar to how Horizon asks user to input parameters that
are declared in HOT template. But all the imperative stuff is hidden inside
Murano packages and is not used for anything outside Murano engine.
MuranoPL is not a replacement for scripting languages. You still use
bash/puppet/PowerShell/whatever you like for actual deployment. No MuranoPL
code is executed on VM side. So the analogy with RPMs manifests is valid.




 You trust OpenStack developers with their code, you trust package
 developers with their code, why not trust catalog developers?


They do trust catalog developers (hopefully). But catalog developers have
nothing to do with catalog contents. Anyone can create and upload
application to App Catalog the same way how anyone can upload his
application to Google Play. The fact that I trust Google doesn't mean that
I trust all applications in Google Play. The fact that I trust catalog
developers doesn't mean that I (as a cloud operator) is going to allow
execution of untrusted code in that catalog. Unless that code is sandboxed
by design. Similar to that I can navigate to any web site google points me
out and let it execute any JavaScript it wants unless it it JavaScript and
not browser plugin or desktop application.




 --
 Thomas

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




-- 
Sincerely yours
Stanislav (Stan) Lagun
Senior Developer
Mirantis
35b/3, Vorontsovskaya St.
Moscow, Russia
Skype: stanlagun
www.mirantis.com
sla...@mirantis.com
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] Separating our Murano PL core in own package

2014-03-24 Thread Stan Lagun
I like dsl most because it is
a. Short. This is especially good when you have that awesome 79-chars
limitation
b. It leaves a lot of room for changes. MuranoPL can change name. DSL - not
:)


On Mon, Mar 24, 2014 at 1:43 PM, Alexander Tivelkov
ativel...@mirantis.comwrote:

 Hi Serg,

 Are you proposing to have a standalone git repository / stack forge
 project for that? Or just a separate package inside our primary murano repo?

 --
 Regards,
 Alexander Tivelkov


 On Mon, Mar 24, 2014 at 12:00 PM, Serg Melikyan smelik...@mirantis.comwrote:

 Programming Language, AFAIK


 On Mon, Mar 24, 2014 at 11:46 AM, Oleg Gelbukh ogelb...@mirantis.comwrote:

 What does PL stand for, anyway?

 --
 Best regards,
 Oleg Gelbukh


 On Mon, Mar 24, 2014 at 11:39 AM, Serg Melikyan 
 smelik...@mirantis.comwrote:

 because 'dsl'/'language' terms are too broad.
 Too broad in general, but we choose name for sub-package, and in murano
 term 'language' mean Murano PL.

 +1 for language


 On Mon, Mar 24, 2014 at 11:26 AM, Timur Sufiev tsuf...@mirantis.comwrote:

 +1 for muranoapi.engine.murano_pl, because 'dsl'/'language' terms are
 too broad.

 On Mon, Mar 24, 2014 at 12:48 AM, Timur Nurlygayanov
 tnurlygaya...@mirantis.com wrote:
  Hi Serg,
 
  This idea sounds good, I suggest to use name
 'murano.engine.murano_pl' (not
  just common name like 'language' or 'dsl', but name, which will be
 based on
  'MuranoPL')
 
  Do we plan to support the ability to define different languages for
 Murano
  Engine?
 
 
  Thank you!
 
 
  On Sun, Mar 23, 2014 at 1:05 PM, Serg Melikyan 
 smelik...@mirantis.com
  wrote:
 
  There is a idea to separate core of Murano PL implementation from
 engine
  specific code, like it was done in PoC. When this two things are
 separated
  in different packages, we will be able to track and maintain our
 language
  core as clean as possible from engine specific code. This will give
 to us an
  ability to easily separate our language implementation to a library.
 
  Questions is under what name we should place core of Murano PL?
 
  1) muranoapi.engine.language;
  2) muranoapi.engine.dsl;
  3) suggestions?
 
  --
  Serg Melikyan, Senior Software Engineer at Mirantis, Inc.
  http://mirantis.com | smelik...@mirantis.com
 
  +7 (495) 640-4904, 0261
  +7 (903) 156-0836
 
  ___
  OpenStack-dev mailing list
  OpenStack-dev@lists.openstack.org
  http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
 
 
 
 
  --
 
  Timur,
  QA Engineer
  OpenStack Projects
  Mirantis Inc
 
  ___
  OpenStack-dev mailing list
  OpenStack-dev@lists.openstack.org
  http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
 



 --
 Timur Sufiev

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




 --
 Serg Melikyan, Senior Software Engineer at Mirantis, Inc.
 http://mirantis.com | smelik...@mirantis.com

 +7 (495) 640-4904, 0261
 +7 (903) 156-0836

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




 --
 Serg Melikyan, Senior Software Engineer at Mirantis, Inc.
 http://mirantis.com | smelik...@mirantis.com

 +7 (495) 640-4904, 0261
 +7 (903) 156-0836

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




-- 
Sincerely yours
Stanislav (Stan) Lagun
Senior Developer
Mirantis
35b/3, Vorontsovskaya St.
Moscow, Russia
Skype: stanlagun
www.mirantis.com
sla...@mirantis.com
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Murano] API/data formats version change and compatibility

2014-03-24 Thread Stan Lagun
Let's discuss this in community meeting.

I would suggest drop support for older version at least until we release
Murano 1.0. As soon as we start to guarantee backward compatibility we will
introduce MinimalMuranoVersion instead of Version because format would not
change but some particular template may require feature that was introduced
after specified version.


On Mon, Mar 24, 2014 at 3:42 PM, Timur Sufiev tsuf...@mirantis.com wrote:

 Hi all!

 While adapting Murano's Dynamic UI to the new MuranoPL data input
 format, I've encountered the need to add some new fields to it, which
 meant that the 'Version' field also had to be added to Dynamic UI. So,
 Dynamic UI definition without Version field is supposed to be of v.1
 while upcoming Murano 0.5 release will support Dynamic UI v.2
 processing (which produces data suitable for the MuranoPL in 0.5).

 But then the next questions arised. What if Murano 0.5 Dynamic UI
 processor gets definition in v.1 format? Should it be able to process
 it as well? If it should, then to which murano-api endpoint should it
 pass the resulting object model?

 I suspect that we have to deal with a slightly broader scope: to which
 extent should Murano components of each new version support the data
 formats and APIs from the previous versions? I suggest to discuss this
 question on the upcoming Murano's community meeting.

 --
 Timur Sufiev

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




-- 
Sincerely yours
Stanislav (Stan) Lagun
Senior Developer
Mirantis
35b/3, Vorontsovskaya St.
Moscow, Russia
Skype: stanlagun
www.mirantis.com
sla...@mirantis.com
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Murano][Heat] MuranoPL questions?

2014-03-24 Thread Stan Lagun
On Mon, Mar 24, 2014 at 4:13 PM, Thomas Herve thomas.he...@enovance.comwrote:

What I can say is that I'm not convinced. The only use-case for a DSL would
 be if you have to upload user-written code, but what you mentioned is a Web
 interface, where the user doesn't use the DSL, and the cloud provider is
 the developer. There is no reason in this case to have a secure environment
 for the code.


I didn't say that. There are at least 2 different roles application
developers/publishers and application users. Application developer is not
necessary cloud provider. The whole point of AppCatalog is to support
scenario when anyone can create and package some application and that
package can be uploaded by user alone. Think Apple AppStore or Google Play.
Some cloud providers may configure ACLs so that user be allowed to consume
applications they decided while others may permit to upload applications to
some configurable scope (e.g. apps that would be visible to all cloud
users, to particular tenant or be private to the user). We also think to
have some of peer relations so that it would be possible to have
application upload in one catalog to become automatically available in all
connected catalogs.

This is similar to how Linux software repos work - AppCatalog is repo,
Murano package is what DEB/RPMs are to repo and DSL is what DEB/RPMs
manifests are to packages. Just that is run on cloud and designed to handle
complex multi-node apps as well as trivial ones in which case this may be
narrowed to actual installation of DEB/RPM



-- 
Sincerely yours
Stanislav (Stan) Lagun
Senior Developer
Mirantis
35b/3, Vorontsovskaya St.
Moscow, Russia
Skype: stanlagun
www.mirantis.com
sla...@mirantis.com
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Mistral][TaskFlow] Long running actions

2014-03-21 Thread Stan Lagun
Don't forget HA issues. Mistral can be restarted at any moment and need to
be able to proceed from the place it was interrupted on another instance.
In theory it can be addressed by TaskFlow but I'm not sure it can be done
without complete redesign of it


On Fri, Mar 21, 2014 at 8:33 AM, W Chan m4d.co...@gmail.com wrote:

 Can the long running task be handled by putting the target task in the
 workflow in a persisted state until either an event triggers it or timeout
 occurs?  An event (human approval or trigger from an external system) sent
 to the transport will rejuvenate the task.  The timeout is configurable by
 the end user up to a certain time limit set by the mistral admin.

 Based on the TaskFlow examples, it seems like the engine instance managing
 the workflow will be in memory until the flow is completed.  Unless there's
 other options to schedule tasks in TaskFlow, if we have too many of these
 workflows with long running tasks, seems like it'll become a memory issue
 for mistral...


 On Thu, Mar 20, 2014 at 3:07 PM, Dmitri Zimine d...@stackstorm.com wrote:


 For the 'asynchronous manner' discussion see http://tinyurl.com/n3v9lt8;
 I'm still not sure why u would want to make is_sync/is_async a primitive
 concept in a workflow system, shouldn't this be only up to the entity
 running the workflow to decide? Why is a task allowed to be sync/async,
 that has major side-effects for state-persistence, resumption (and to me is
 a incorrect abstraction to provide) and general workflow execution control,
 I'd be very careful with this (which is why I am hesitant to add it without
 much much more discussion).


 Let's remove the confusion caused by async. All tasks [may] run async
 from the engine standpoint, agreed.

 Long running tasks - that's it.

 Examples: wait_5_days, run_hadoop_job, take_human_input.
 The Task doesn't do the job: it delegates to an external system. The flow
 execution needs to wait  (5 days passed, hadoob job finished with data x,
 user inputs y), and than continue with the received results.

 The requirement is to survive a restart of any WF component without
 loosing the state of the long running operation.

 Does TaskFlow already have a way to do it? Or ongoing ideas,
 considerations? If yes let's review. Else let's brainstorm together.

 I agree,

 that has major side-effects for state-persistence, resumption (and to me
 is a incorrect abstraction to provide) and general workflow execution
 control, I'd be very careful with this

 But these requirement  comes from customers'  use cases: wait_5_day -
 lifecycle management workflow, long running external system - Murano
 requirements, user input - workflow for operation automations with control
 gate checks, provisions which require 'approval' steps, etc.

 DZ


 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




-- 
Sincerely yours
Stanislav (Stan) Lagun
Senior Developer
Mirantis
35b/3, Vorontsovskaya St.
Moscow, Russia
Skype: stanlagun
www.mirantis.com
sla...@mirantis.com
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Murano][Heat] MuranoPL questions?

2014-03-21 Thread Stan Lagun
 and not consume it directly. So you still write code in
Turing-complete sophisticated language Java

2. Our initial idea was to translate MuranoPL into Mistral workflows
(similar to compiling C++ to assembly) but it turns to be too difficult to
implement

3. The DSL Mistral currently has is very very limited and academic. You
have to be real genius with a twisted mind to write real-life workflows
with Mistral DSL.

4. Mistral is at PoC stage and it would take him years to grow to something
that is as powerful as MuranoPL (or maybe BPEL)

5. Mistral does not need all that OOP stuff, inheritance, constraints etc.
And in practice it is very hard to separate declarations from the workflow
code as they are very coupled in their nature

6. DSL is a hart and brain of Murano. It is the most vital part of it. You
cannot develop system of such level as Murano without having control over
it. Any decision taken by Mistral team can become fatal for Murano.

Anyway I hope that some day we will be able to somehow merge our
DSLs/codebase/whatever



On Fri, Mar 21, 2014 at 10:20 PM, Zane Bitter zbit...@redhat.com wrote:

 I completely agree with Georgy, but you raised some questions about Heat
 that I want to answer in the interests of spreading knowledge about how
 Heat works. A heavily-snipped response follows...


 On 21/03/14 05:11, Stan Lagun wrote:

 3. Despite HOT being more secure on the surface it is not necessary so
 in reality. There is a Python class behind each entry in resources
 section of HOT template. That Python code is run with root privileges
 and not guaranteed to be safe. People make mistakes, forget to validate
 parameters, make incorrect assumptions etc. Even if the code is proven
 to be secure every single commit can introduce security breach. And no
 testing system can detect this.


 Quite right, I should acknowledge that it would be crazy to assume that
 HOT is secure just because it is not a programming language, and I do not
 make that assumption. (Indeed, YAML itself has been the subject of many
 security problems, though afaik not in the safe mode that we use in Heat.)
 Thanks for pointing out that I was not clear.


  The operator can install whatever plugins they want.

 They do but that is a bad solution. The reason is that plugins can
 introduce additional resource types but they cannot modify existing
 code. Most of the time cloud operators need to customize existing
 resources' logic for their needs rather then rewriting it from scratch.
 And they want their changes to be opaque to end-users. Imagine that
 cloud operator need thats to get permission from his proprietary quota
 management system for each VM spawned. If he would create custom
 MyInstance resource type end-users could bypass it by using standard
 Instance resource rather than custom one. Patching existing Python code
 is not good in that then operator need to maintain his private fork of
 the Heat and have troubles with CD, upgrades to newer versions etc.


 It's not as bad as you think. All of the things you mentioned were
 explicit design goals of the plugin system. If you install a plug-in
 resource with the same type as a built-in resource then it replaces the
 built-in one. And of course you can inherit from the existing plugin to
 customise it.

 So in this example, the operator would create a plugin like this:

   from heat.engine.resources import server
   from my.package import do_my_proprietary_quota_thing

   class MyServer(server.Server):
   def handle_create(self):
   do_my_proprietary_quota_thing()
   return super(MyServer, self).handle_create()

   def resource_mapping():
   return {'OS::Nova::Server': MyServer}

 and drop it in /usr/lib/heat. As you can see, this is a simple
 customisation (10 lines of code), completely opaque to end users
 (OS::Nova::Server is replaced), and highly unlikely to be broken by any
 changes in Heat (we consider the public APIs of
 heat.engine.resource.Resource as a contract with existing plugins that we
 can't break, at least without a lot of notice).

 (I'm ignoring here that if this is needed for _every_ server, it makes no
 sense to do it in Heat, unless you don't expose the Nova API to users at
 all.)


  Besides plugin system is not secure because plugins run with the
 privileges of Heat engine and while I may trust Heat developers
 (community) but not necessary trust 3rd party proprietary plugin.


 I'm not sure who 'I' means in this context? As an end-user, you have no
 way of auditing what code your cloud provider is running in general.



  What
 if he wants that auto-scaling would be based on input from his
 existing
 Nagios infrastructure rather then Ceilometer?


 This is supported already in autoscaling. Ceilometer just hits a URL
 for an alarm, but you don't have to configure it this way. Anything
 can hit the URL.

 And this is a good example for our general approach - we provide

Re: [openstack-dev] [legal-discuss] [Marconi] Why is marconi a queue implementation vs a provisioning API?

2014-03-20 Thread Stan Lagun
Kurt,

Your point is that NoSQL solution may be required for innovative project.
And that is MongoDB. But what if come another amazing project that needs
CouchDB, neo4j, Riak, (put your favorite NoSQL DB here)? It would be in the
same position cause everyone would say hey, we already have NoSQL in
OpenStack so you have to use MongoDB which is not fair. But is also obvious
that OpenStack cannot demand cloud operators to maintain MySQL, MongoDB,
CouchDB, neo4j etc in simultaneously

I hate to say that (I happen to be MongoDB fan) but the only way we can
introduce external dependencies to OpenStack is by making technology that
would make possible the project to be responsible for deployment and
maintenance of that dependency (DBMS) rather then cloud operator. It seems
to me that the right way to introduce MongoDB is to invest in projects like
TripleO, Fuel, Murano, Heat and Ironic


On Thu, Mar 20, 2014 at 9:09 PM, Gil Yehuda gyeh...@yahoo-inc.com wrote:

 To be clear, Marconi does not incorporate any AGPL code itself; pymongo is
 Apache2 licensed.

 Understood, but here's the rub. Someone else is going to want to build on
 this (which it the point of this open source project). Whereas 'pymongo' is
 Apache licensed, since the copyright holder, MongoDB Inc. declared it as
 such, the authors of the other community drivers (for other language
 bindings and features of MongoDB, etc.) are also of releasing drivers under
 the Apache or BSD licenses too (thinking that's OK to do since no one is
 telling them otherwise). That community is unaware of their legal
 obligations when creating drivers to an AGPL database. Thus if one of those
 community drivers gets intertwined in a court case clarifying their license
 to be infringing on the AGPL terms, we've inadvertently impacted our
 community. This is a credible risk that is difficult for OpenStack to
 abate, since the problem lies with the way a different community chose to
 operate.

 There are three interconnected issues here:
 1. The confusion that MongoDB has created in Open Source licensing due to
 the asymmetric control they have on licensing terms.
 2. The diligence of Open Stack to remain careful with OpenStack's CLA
 compliance and Apache-friendly terms.
 3. The pragmatics of the effect MondgoDB would have onto OpenStack's
 economic viability and legal risks at large.

 The first problem is out of scope for this list. But I think people who
 rely upon Open Source for their business ought to understand what MongoDB
 is doing to open source software. The second is, to your point, the issue
 in this conversation. As long as Openstack only use Apache licensed code
 from MondgoDB Inc. and diligently avoids using any open source
 contributions from any community contributor to the MongoDB ecosystem, then
 you remain compliant the your CLA. But you will have to exclude the rest of
 the MongoDB community (which goes against the spirit of Open Source -- back
 to the problem #1, which is out of scope). As for #3, I think the
 foundation needs to weigh in on the pragmatics here, since this has an
 economic and legal impact to the entire endeavor, not just to persisting
 data in one component of the project.

 Gil Yehuda
 Sr. Director Of Open Source, Open Standards

 -Original Message-
 From: Kurt Griffiths [mailto:kurt.griffi...@rackspace.com]
 Sent: Thursday, March 20, 2014 9:29 AM
 To: OpenStack Development Mailing List (not for usage questions)
 Subject: Re: [openstack-dev] [legal-discuss] [Marconi] Why is marconi a
 queue implementation vs a provisioning API?

  The incorporation of AGPLv3 code Into OpenStack Project  is a
 significant decision



 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




-- 
Sincerely yours
Stanislav (Stan) Lagun
Senior Developer
Mirantis
35b/3, Vorontsovskaya St.
Moscow, Russia
Skype: stanlagun
www.mirantis.com
sla...@mirantis.com
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Marconi] Why is marconi a queue implementation vs a provisioning API?

2014-03-19 Thread Stan Lagun
 as a non-AGPL alternative to MongoDB.
 For large-scale production deployments, we currently provide the MongoDB
 driver and will likely add Redis as another option (there is already a POC
 driver). And, of course, operators can provide drivers for NewSQL
 databases, such as VelocityDB, that are very fast and scale extremely
 well. In Marconi, every queue can be associated with a different backend
 cluster. This allows operators to scale both up and out, according to what
 is most cost-effective for them. Marconi's app-level sharding is currently
 done using a lookup table to provide for maximum operator control over
 placement, but I personally think it would be great to see this opened up
 so that we can swap in other types of drivers, such as one based on hash
 rings (TBD).

 3. Cost-effective. The Marconi team has done a lot of work to (1) provide
 several dimensions for scaling deployments that can be used according to
 what is most cost-effective for a given use case, and (2) make the Marconi
 service as efficient as possible, including time spent optimizing the
 transport layer (using Falcon in lieu of Pecan, reducing the work that the
 request handlers do, etc.), and tuning the MongoDB storage driver (the
 SQLAlchemy driver is newer and we haven't had the chance to tune it yet,
 but are planning to do so during Juno). Turnaround on requests is in the
 low ms range (including dealing with HTTP), not the usec range, but that
 works perfectly well for a large class of applications. We've been
 benchmarking with Tsung for quite a while now, and we are working on
 making the raw data more accessible to folks outside our team. I'll try to
 get some of the latest data up on the wiki this week.

 Marconi was originally incubated because the community believed developers
 building their apps on top of OpenStack were looking for this kind of
 service, and it was a big missing gap in our portfolio. Since that time,
 the team has worked hard to fill that gap.

 Kurt

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




-- 
Sincerely yours
Stanislav (Stan) Lagun
Senior Developer
Mirantis
35b/3, Vorontsovskaya St.
Moscow, Russia
Skype: stanlagun
www.mirantis.com
sla...@mirantis.com
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Murano][Heat] MuranoPL questions?

2014-03-19 Thread Stan Lagun
:

 Georgy,

 In consideration of the can you express it instead of the who will
 generate it, I see Heat's HOT evolving to support the expression of
 complex
 multi-tier architectures and applications (I would argue you can already
 do
 this today, perhaps with some additional features desired, e.g. Ability
 to
 define cloud workflows and workflow execution rules which could come
 when we
 have a workflow service like Mistral).  Therefore, I would encourage
 Murano
 contributors to consider whether they can help make Heat sufficiently
 cover
 desired use cases.  I have never viewed Heat templates as isolated
 components of a multi-tier architecture.  Instead, a single template or a
 combination of master/subordinate templates together (using references,
 nesting, or inclusion) could express the complete architecture, both
 infrastructure and applications.

 If I've read your previous comments and threads correctly, you desire a
 way
 to express System Lifecycle Management across multiple related
 applications
 or components, whereby you view the System as a grouping of independently
 developed and/or deployed (but systematically related) components,
 whereby
 you view Components as individual disconnected Heat templates that
 independently describe different application stacks of the System.  Did I
 get that correct?   If so, perhaps the discussion here is one of scope
 of
 what can or should be expressed in a Heat template. Is it correct to
 state
 that your argument is that a separate system (such as Murano) should be
 used
 to express System Lifecycle Management as I've defined it here?  If so,
 why
 could we not use the Heat DSL to also define the System?  The System
 definition could be logically separated out into its own text file...
 But,
 we'd have a common DSL syntax and semantics for both lower level and
 higher
 level component interaction (a building block effect of sorts).

 As for who will generate it, ( with it being the Heat multi-tier
 application/infrastructure definition) I think that question will go
 through
 a lot more evolution and could be any number of sources: e.g. Solum,
 Murano,
 Horizon, Template Author with a text editor, etc.

 Basically, I'm a +1 for as few DSLs as possible. I support the position
 that
 we should evolve HOT if needed vs. having two separate DSLs that are both
 related to expressing application and infrastructure semantics.

 Workflow is quite interesting ... Should we be able to express imperative
 workflow semantics in HOT?  Or, should we only be able to declare
 workflow
 configurations that get configured in a service like Mistral whereby
 Mistral's execution of a workflow may need to invoke Heat hooks or Stack
 Updates?  Or, some other solution?

 I look forward to a design discussion on all this at the summit... This
 is
 fun stuff to think about!

 -Keith

 From: Georgy Okrokvertskhov gokrokvertsk...@mirantis.com

 Reply-To: OpenStack Development Mailing List (not for usage questions)
 openstack-dev@lists.openstack.org
 Date: Tuesday, March 18, 2014 1:49 PM

 To: OpenStack Development Mailing List (not for usage questions)
 openstack-dev@lists.openstack.org
 Subject: Re: [openstack-dev] [Murano][Heat] MuranoPL questions?

 I see this in the following way - who will generate HOT template for my
 complex multi-tier applications when I have only templates for
 components?


 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

  ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




-- 
Sincerely yours
Stanislav (Stan) Lagun
Senior Developer
Mirantis
35b/3, Vorontsovskaya St.
Moscow, Russia
Skype: stanlagun
www.mirantis.com
sla...@mirantis.com
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [openstack][Mistral] Adding new core reviewers

2014-03-19 Thread Stan Lagun
+1 for both


On Wed, Mar 19, 2014 at 3:35 PM, Renat Akhmerov rakhme...@mirantis.comwrote:

 Team,

 So far I've been just the only one core member of the team. I started
 feeling lonely :) Since the project team and the project itself has now
 grown (thanks to StackStorm and Intel) I think it's time to think about
 extending the core team.

 I would propose:

- Nikolay Makhotkin (nmakhotkin at launchpad). He's been working on
the project since almost the very beginning and made significant
contribution (design, reviews, code).
- Dmitri Zimine (i-dz at launchpad). Dmitri joined the project about 2
months ago. Since then he's made a series of important high-quality
commits, a lot of valuable reviews and, IMO most importantly, he has a
solid vision of the project in general (requirements, use cases, comparison
to other technologies) and has a pro-active viewpoint in all our
discussions.


 Thoughts?

 Renat Akhmerov
 @ Mirantis Inc.




 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




-- 
Sincerely yours
Stanislav (Stan) Lagun
Senior Developer
Mirantis
35b/3, Vorontsovskaya St.
Moscow, Russia
Skype: stanlagun
www.mirantis.com
sla...@mirantis.com
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Murano][Heat] MuranoPL questions?

2014-03-19 Thread Stan Lagun
 Management as I've defined it here?  If so,
 why
 could we not use the Heat DSL to also define the System?  The System
 definition could be logically separated out into its own text file...
 But,
 we'd have a common DSL syntax and semantics for both lower level and
 higher
 level component interaction (a building block effect of sorts).

 As for who will generate it, ( with it being the Heat multi-tier
 application/infrastructure definition) I think that question will go
 through
 a lot more evolution and could be any number of sources: e.g. Solum,
 Murano,
 Horizon, Template Author with a text editor, etc.

 Basically, I'm a +1 for as few DSLs as possible. I support the position
 that
 we should evolve HOT if needed vs. having two separate DSLs that are
 both
 related to expressing application and infrastructure semantics.

 Workflow is quite interesting ... Should we be able to express
 imperative
 workflow semantics in HOT?  Or, should we only be able to declare
 workflow
 configurations that get configured in a service like Mistral whereby
 Mistral's execution of a workflow may need to invoke Heat hooks or Stack
 Updates?  Or, some other solution?

 I look forward to a design discussion on all this at the summit... This
 is
 fun stuff to think about!

 -Keith

 From: Georgy Okrokvertskhov gokrokvertsk...@mirantis.com

 Reply-To: OpenStack Development Mailing List (not for usage questions)
 openstack-dev@lists.openstack.org
 Date: Tuesday, March 18, 2014 1:49 PM

 To: OpenStack Development Mailing List (not for usage questions)
 openstack-dev@lists.openstack.org
 Subject: Re: [openstack-dev] [Murano][Heat] MuranoPL questions?

 I see this in the following way - who will generate HOT template for my
 complex multi-tier applications when I have only templates for
 components?


 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

  ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




 --
 Sincerely yours
 Stanislav (Stan) Lagun
 Senior Developer
 Mirantis
 35b/3, Vorontsovskaya St.
 Moscow, Russia
 Skype: stanlagun
 www.mirantis.com
 sla...@mirantis.com


 ___
 OpenStack-dev mailing 
 listOpenStack-dev@lists.openstack.orghttp://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




-- 
Sincerely yours
Stanislav (Stan) Lagun
Senior Developer
Mirantis
35b/3, Vorontsovskaya St.
Moscow, Russia
Skype: stanlagun
www.mirantis.com
sla...@mirantis.com
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Murano][Heat] MuranoPL questions?

2014-03-19 Thread Stan Lagun
Ability to hook up to application deployment is indeed a good thing. And it
can be done both in HOT and MuranoPL. And Mistral is a good candidate to
handle those hooks. But this is not a replacement for MuranoPL but an
addition to it.

The problem with hooks is that you cannot hoot just into arbitrary place in
deployment workflow. And the author of Python code may not expose the exact
hook that you need. Hooks can work for logging purposes or triggering some
additional workflows but are not good for customization of your workflow
from outside. Hooked code may not have access to all engine's internal
state and workflow context and have even less chances to modify it in a
safe manner


On Thu, Mar 20, 2014 at 1:21 AM, Georgy Okrokvertskhov 
gokrokvertsk...@mirantis.com wrote:

 Hi,

 I think notification mechanism proposed in Heat will work fine for
 integration with external workflows. The approach which uses workflows
 outside of Heat engine sounds consistent with our current approach in
 Murano.

 I am looking into new TOSCA yaml format and I also ask Mirantis management
 to consider joining OASIS. The decision is not made yet, but hopefully will
 be made on next week. We eager to jump onto TOSCA standard work and
 contribute plan related parts.

 Thanks
 Georgy




 On Wed, Mar 19, 2014 at 1:38 PM, Thomas Spatzier 
 thomas.spatz...@de.ibm.com wrote:

 Excerpts from Zane Bitter's message on 19/03/2014 18:18:34:

  From: Zane Bitter zbit...@redhat.com
  To: openstack-dev@lists.openstack.org
  Date: 19/03/2014 18:21
  Subject: Re: [openstack-dev] [Murano][Heat] MuranoPL questions?
 
  On 19/03/14 05:00, Stan Lagun wrote:
   Steven,
  
   Agree with your opinion on HOT expansion. I see that inclusion of
   imperative workflows and ALM would require major Heat redesign and
   probably would be impossible without loosing compatibility with
 previous
   HOT syntax. It would blur Heat mission, confuse current users and rise
 a
   lot of questions what should and what should not be in Heat. Thats why
   we chose to built a system on top of Heat rather then expending HOT.
 
  +1, I agree (as we have discussed before) that it would be a mistake to
  shoehorn workflow stuff into Heat. I do think we should implement the
  hooks I mentioned at the start of this thread to allow tighter
  integration between Heat and a workflow engine (i.e. Mistral).

 +1 on not putting workflow stuff into Heat. Rather let's come up with a
 nice way of Heat and a workflow service to work together.
 That could be done in two ways: (1) let Heat hand off to a workflow
 service
 for certains tasks or (2) let people define workflow tasks that can easily
 work on Heat deployed resources. Maybe both make sense, but right now I am
 more leaning towards (2).

 
  So building a system on top of Heat is good. Building it on top of
  Mistral as well would also be good, and that was part of the feedback
  from the TC.
 
  To me, building on top means building on top of the languages (which
  users will have to invest a lot of work in learning) as well, rather
  than having a completely different language and only using the
  underlying implementation(s).

 That all sounds logical to me and would keep things clean, i.e. keep the
 HOT language clean by not mixing it with imperative expression, and keep
 the Heat engine clean by not blowing it up to act as a workflow engine.

 When I think about the two aspects that are being brought up in this
 thread
 (declarative description of a desired state and workflows) my thinking is
 that much (and actually as much as possible) can be done declaratively the
 way Heat does it with HOT. Then for bigger lifecycle management there will
 be a need for additional workflows on top, because at some point it will
 be
 hard to express management logic declaratively in a topology model.
 Those additional flows on-top will have to be aware of the instance
 created
 from a declarative template (i.e. a Heat stack) because it needs to act on
 the respective resources to do something in addition.

 So when thinking about a domain specific workflow language, it should be
 possible to define tasks (in a template aware manner) like on resource
 XYZ
 of the template, do something, or update resource XYZ of the template
 with this state, then do this etc. At runtime this would resolve to the
 actual resource instances created from the resource templates. Making such
 constructs available to the workflow authors would make sense. Having a
 workflow service able to execute this via the right underlying APIs would
 be the execution part. I think from an instance API perspective, Heat
 already brings a lot for this with the stack model, so workflow tasks
 could
 be written to use the stack API to access instance information. Things
 like
 update of resources is also something that is already there.

 BTW, we have a similar concept (or are working on a refinement of it based
 on latest discussions) in TOSCA and call it the plan

Re: [openstack-dev] MuranoPL questions?

2014-03-17 Thread Stan Lagun
Joshua,

Completely agree with you. We wouldn't be writing another language if we
knew how any of existing languages can be used for this particular purpose.
If anyone suggest such language and show how it can be used to solve those
issues DSL was designed to solve we will consider dropping MuranoPL. np

Surely DSL hasn't stood the test of time. It just hasn't had a chance yet.
100% of successful programming languages were in such position once.

Anyway it is the best time to come with your suggestions. If you know how
exactly DSL can be replaced or improved we would like you to share


On Wed, Mar 12, 2014 at 2:05 AM, Joshua Harlow harlo...@yahoo-inc.comwrote:

  I guess I might be a bit biased to programming; so maybe I'm not the
 target audience.

  I'm not exactly against DSL's, I just think that DSL's need to be really
 really proven to become useful (in general this applies to any language
 that 'joe' comp-sci student can create). Its not that hard to just make
 one, but the real hard part is making one that people actually like and use
 and survives the test of time. That's why I think its just nicer to use
 languages that have stood the test of time already (if we can), creating a
 new DSL (muranoPL seems to be slightly more than a DSL imho) means creating
 a new language that has not stood the test of time (in terms of lifetime,
 battle tested, supported over years) so that's just the concern I have.

  Of course we have to accept innovation and I hope that the DSL/s makes
 it easier/simpler, I just tend to be a bit more pragmatic maybe in this
 area.

  Here's hoping for the best! :-)

  -Josh

   From: Renat Akhmerov rakhme...@mirantis.com
 Reply-To: OpenStack Development Mailing List (not for usage questions) 
 openstack-dev@lists.openstack.org
 Date: Monday, March 10, 2014 at 8:36 PM
 To: OpenStack Development Mailing List (not for usage questions) 
 openstack-dev@lists.openstack.org
 Subject: Re: [openstack-dev] MuranoPL questions?

   Although being a little bit verbose it makes a lot of sense to me.

  @Joshua,

  Even assuming Python could be sandboxed and whatever else that's needed
 to be able to use it as DSL (for something like Mistral, Murano or Heat) is
 done  why do you think Python would be a better alternative for people who
 don't know neither these new DSLs nor Python itself. Especially, given the
 fact that Python has A LOT of things that they'd never use. I know many
 people who have been programming in Python for a while and they admit they
 don't know all the nuances of Python and actually use 30-40% of all of its
 capabilities. Even not in domain specific development. So narrowing a
 feature set that a language provides and limiting it to a certain domain
 vocabulary is what helps people solve tasks of that specific domain much
 easier and in the most expressive natural way. Without having to learn tons
 and tons of details that a general purpose language (GPL, hah :) ) provides
 (btw, the reason to write thick books).

  I agree with Stan, if you begin to use a technology you'll have to learn
 something anyway, be it TaskFlow API and principles or DSL. Well-designed
 DSL just encapsulates essential principles of a system it is used for. By
 learning DSL you're leaning the system itself, as simple as that.

  Renat Akhmerov
 @ Mirantis Inc.



  On 10 Mar 2014, at 05:35, Stan Lagun sla...@mirantis.com wrote:

I'd be very interested in knowing the resource controls u plan to
 add. Memory, CPU...
  We haven't discussed it yet. Any suggestions are welcomed

  I'm still trying to figure out where something like
 https://github.com/istalker2/MuranoDsl/blob/master/meta/com.mirantis.murano.demoApp.DemoInstance/manifest.yamlwould
  be beneficial, why not  just spend effort sand boxing lua, python...
 Instead of spending effort on creating a new language and then having to
 sandbox it as well... Especially if u picked languages that are made to be
 sandboxed from the start (not python)...

  1. See my detailed answer in Mistral thread why haven't we used any of
 those languages. There are many reasons besides sandboxing.

  2. You don't need to sandbox MuranoPL. Sandboxing is restricting some
 operations. In MuranoPL ALL operations (including operators in expressions,
 functions, methods etc.) are just those that you explicitly provided. So
 there is nothing to restrict. There are no builtins that throw
 AccessViolationError

  3. Most of the value of MuranoPL comes not form the workflow code but
 from class declarations. In all OOP languages classes are just a convenient
 to organize your code. There are classes that represent real-life objects
 and classes that are nothing more than data-structures, DTOs etc. In Murano
 classes in MuranoPL are deployable entities like Heat resources application
 components, services etc. In dashboard UI user works with those entities.
 He (in UI!) creates instances of those classes, fills their property
 values, binds objects together

Re: [openstack-dev] [Mistral][Taskflow][all] Mistral + taskflow

2014-03-14 Thread Stan Lagun
Joshua,

why wait? Why not just help Renat with his research on that integration and
bring your own vision to the table? Write some 1-page architecture
description on how Mistral can be built on top of TaskFlow and we discuss
pros and cons. In would be much more productive.


On Fri, Mar 14, 2014 at 11:35 AM, Joshua Harlow harlo...@yahoo-inc.comwrote:

  Thanks Renat,

  I'll keep waiting, and hoping that we can figure this out for everyone's
 benefit. Because in the end we are all much stronger working together and
 much weaker when not.

 Sent from my really tiny device...

 On Mar 13, 2014, at 11:41 PM, Renat Akhmerov rakhme...@mirantis.com
 wrote:

  Folks,

  Mistral and TaskFlow are significantly different technologies. With
 different set of capabilities, with different target audience.

  We may not be doing enough to clarify all the differences, I admit that.
 The challenge here is that people tend to judge having minimal amount of
 information about both things. As always, devil in the details. Stan is
 100% right, seems is not an appropriate word here. Java seems to be
 similar to C++ at the first glance for those who have little or no
 knowledge about them.

  To be more consistent I won't be providing all the general
 considerations that I've been using so far (in etherpads, MLs, in personal
 discussions), it doesn't seem to be working well, at least not with
 everyone. So to make it better, like I said in that different thread: we're
 evaluating TaskFlow now and will share the results. Basically, it's what
 Boris said about what could and could not be implemented in TaskFlow. But
 since the very beginning of the project I never abandoned the idea of using
 TaskFlow some day when it's possible.

  So, again: Joshua, we hear you, we're working in that direction.


 I'm reminded of

 http://www.slideshare.net/RenatAkhmerov/mistral-hong-kong-unconference-trac
 k/2http://www.slideshare.net/RenatAkhmerov/mistral-hong-kong-unconference-track/2where
  it seemed like we were doing much better collaboration, what has
 happened to break this continuity?


  Not sure why you think something is broken. We just want to finish the
 pilot with all the 'must' things working in it. This is a plan. Then we can
 revisit and change absolutely everything. Remember, to the great extent
 this is research. Joshua, this is what we talked about and agreed on many
 times. I know you might be anxious about that given the fact it's taking
 more time than planned but our vision of the project has drastically
 evolved and gone far far beyond the initial Convection proposal. So the
 initial idea of POC is no longer relevant. Even though we finished the
 first version in December, we realized it wasn't something that should have
 been shared with the community since it lacked some essential things.


  Renat Akhmerov
 @ Mirantis Inc.

   ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




-- 
Sincerely yours
Stanislav (Stan) Lagun
Senior Developer
Mirantis
35b/3, Vorontsovskaya St.
Moscow, Russia
Skype: stanlagun
www.mirantis.com
sla...@mirantis.com
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Mistral][Taskflow][all] Mistral + taskflow

2014-03-13 Thread Stan Lagun
Joshua, Boris, Renat,

I call for this discussion to be technical rather than emotional. Seems
is not appropriate word here. It seems like both Mistral and TaskFlow
duplicate many similar services and libraries outside of OpenStack (BTW,
what exactly in TaskFlow is OpenStack-specific?). So it is not a good idea
to judge who duplicates who and focus on how TaskFlow can help Mistral (or
vice versa)

Let us first list what can be considered to be inalienable parts of Mistral
(features and use cases that are vital to Mistral paradigm) and then
suggest how those use cases can be addressed by TaskFlow. I think it would
be best if Joshua propose detailed Mistral design based on TaskFlow that
would have all the listed futures and addresses all principal use cases.
After all it really doesn't matter if Mistral and TaskFlow has similar
pieces of code in their implementation or similar concepts unless those
pieces of code and concepts can be extracted from TaskFlow and reused in
Mistral


On Fri, Mar 14, 2014 at 3:17 AM, Boris Pavlovic bo...@pavlovic.me wrote:

 Joshua,


 Fully agree, seems like Mistral duplicates a lot of efforts of TaskFlow.
 I just don't see any reason why Mistral is reimplementing TaskFlow and not
 just adding new features to...



 Best regards,
 Boris Pavlovic


 On Fri, Mar 14, 2014 at 3:02 AM, Joshua Harlow harlo...@yahoo-inc.comwrote:

 Separating from the following:

 *
 http://lists.openstack.org/pipermail/openstack-dev/2014-March/029870.html

 What can we do to resolve this, and reduce duplication in effort, code and
 so on.

 I believe that we are really doing the same thing, but it appears that
 something (imho) is wrong with the process here.

 I'm unsure how to resolve the situationŠ

 I believe since as a community we should be working together instead of
 working apart in silos creating the same thing.

 I'm reminded of

 http://www.slideshare.net/RenatAkhmerov/mistral-hong-kong-unconference-trac
 k/2http://www.slideshare.net/RenatAkhmerov/mistral-hong-kong-unconference-track/2where
  it seemed like we were doing much better collaboration, what has
 happened to break this continuity?

 How can we fix it?

 -Josh


 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




-- 
Sincerely yours
Stanislav (Stan) Lagun
Senior Developer
Mirantis
35b/3, Vorontsovskaya St.
Moscow, Russia
Skype: stanlagun
www.mirantis.com
sla...@mirantis.com
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Mistral] Crack at a Real life workflow

2014-03-10 Thread Stan Lagun
On Mon, Mar 10, 2014 at 12:26 PM, Renat Akhmerov rakhme...@mirantis.comwrote:


 In case of Amazon SWF it works in the opposite way. First of all it's a
 language agnostic web service, then they have language specific frameworks
 working on top of the service.


The big question here is would SWF be popular (or at least usable in
real-world scenarios) without language-specific framework on top?


-- 
Sincerely yours
Stanislav (Stan) Lagun
Senior Developer
Mirantis
35b/3, Vorontsovskaya St.
Moscow, Russia
Skype: stanlagun
www.mirantis.com
sla...@mirantis.com
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] MuranoPL questions?

2014-03-09 Thread Stan Lagun
 and knew nothing
about it beforehand to write some application manifests using MuranoPL. We
gave them 10-minute introduction, link to DSL documentation blueprint and
examples in my github repository (you've seen both of them). The link above
to DemoInstance manifest points to the class that was created by one of
those guys. It took them several days to get used to it. But they never
even used Python before! And the feedback was very positive.

I don't really see a problem writing tutorials or even a book. Learning new
framework like TaskFlow is nowhere easier than learning DSL. Anyway you
need to learn something to start using new technology. There are hundreds
of DSL languages that are used in production and I don't feel like having
one more language is really a problem




On Mon, Mar 10, 2014 at 12:26 AM, Joshua Harlow harlo...@yahoo-inc.comwrote:

  I'd be very interested in knowing the resource controls u plan to add.
 Memory, CPU...

  I'm still trying to figure out where something like
 https://github.com/istalker2/MuranoDsl/blob/master/meta/com.mirantis.murano.demoApp.DemoInstance/manifest.yamlwould
  be beneficial, why not just spend effort sand boxing lua, python...
 Instead of spending effort on creating a new language and then having to
 sandbox it as well... Especially if u picked languages that are made to be
 sandboxed from the start (not python)...

  Who is going to train people on muranoPL, write language books and
 tutorials when the same amount of work has already been done for 10+ years
 for other languages

  I fail to see where muranoPL is a DSL when it contains a full language
 already with functions, objects, namespaces, conditionals... (what is the
 domain of it?), maybe I'm just confused though (quite possible, haha).

  Does this not seem awkward to anyone else??

 Sent from my really tiny device...

 On Mar 8, 2014, at 10:44 PM, Stan Lagun sla...@mirantis.com wrote:

First of all MuranoPL is not a host but hosted language. It never
 intended to replace Python and if Python can do the job it is probably
 better than MuranoPL for that job.
  The problem with Python is that you cannot have Python code as a part of
 your DSL if you need to evaluate that DSL on server-side. Using Python's
 eval() is not secure. And you don't have enough control on what evaled code
 is allowed to do. MuranoPL on the contrary are fully sandboxed. You have
 absolute control over what functions/methods/APIs are exposed to DSL and
 DSL code can do nothing except for what it allowed to do. Besides you
 typically do want your DSL to be domain-specific so general-purpose
 language like Python can be suboptimal.

  I don't say MuranoPL is good for all projects. It has many
 Murano-specific things after all. In most cases you don't need all those
 OOP features that MuranoPL has. But the code organization, how it uses
 YAML, block structures and especially YAQL expressions can be of a great
 value to many projects

  For examples of MuranoPL classes you can browse
 https://github.com/istalker2/MuranoDsl/tree/master/meta folder. This is
 my private repository that I was using to develop PoC for MuranoPL engine.
 We are on the way to create production-quality implementation with
 unit-tests etc. in regular Murano repositories on stackforge


 On Sun, Mar 9, 2014 at 7:33 AM, Joshua Harlow harlo...@yahoo-inc.comwrote:

 To continue from other thread

 
 Personally I believe that YAQL-based DSLs similar (but probably simlet
 than) MuranoPL can be of a great value for many OpenStack projects that
 have DSLs of different kind. Murano for App Catatalog, Mistral for
 workflows, Heat for HOT templates, Ceilometer for alarms  counter
 aggregation rules, Nova for customizable resource scheduling and probably
 many more.
 

 Isn't python a better host language for said DSLs than muranoPL? I am
 still pretty weary of a DSL that starts to grow so many features to
 encompass other DSLs since then it's not really a DSL but a non-DSL, and we
 already have one that everyone is familiar with (python).

 Are there any good examples if muranoPL that I can look at that take
 advantage of muranoPL classes, functions, namespaces which can be compared
 to there python equivalents. Has such an analysis/evaluation been done?

 Sent from my really tiny device...
 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




 --
 Sincerely yours
 Stanislav (Stan) Lagun
 Senior Developer
 Mirantis
 35b/3, Vorontsovskaya St.
 Moscow, Russia
 Skype: stanlagun
 www.mirantis.com
 sla...@mirantis.com

  ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi

Re: [openstack-dev] [Mistral] Crack at a Real life workflow

2014-03-08 Thread Stan Lagun
 deserialized version of it. Thus
you can control where those YAMLs are located. You can store them in
Glance, database, file-system or generate them on the fly without DSL even
notice.

9. With YAML you has less to explain (people already know it) and more of a
tooling and IDE support


Q: Isn't DSL of that power would be a subject to DoS attacks, resource
starvation attacks etc.?

A: This is a really good question cause nobody asked it yet :) And the
answer is NO. Here is why:

With MuranoPL you have complete control over everything. You can have time
quota for DSL code (remember 30 second quota for PHP scrips in typical web
setup?). Or you can limit DSL scripts to some reasonable number of executed
instructions. Or count raw CPU time for the script. You can make all loop
constructs to have upper limit on iterations. You can do the same with YAQL
functions.  You can enforce timeouts on all I/O operations. MuranoPL uses
green threads and you can have a lot of them (unlike regular threads) and
it is possible to limit total number of green threads used for DSL
execution. It is possible to even limit memory usage. DSL code cannot
allocate operating-system-level resources like file handles of TCP sockets.
And there is garbage collector as DSL in interpreted on top of Python.

How easy it would be to do the same with JavaScript/Lua/whatever?







On Sat, Mar 8, 2014 at 7:05 AM, Joshua Harlow harlo...@yahoo-inc.comwrote:

  What are the benefits of MuranoPL over a already established language?

  I noted the following and don't quite understand why they are needed
 (reinventing a language):

  - https://wiki.openstack.org/wiki/Murano/DSL/Blueprint#Extends
 - https://wiki.openstack.org/wiki/Murano/DSL/Blueprint#Block_constructs
 Q: where are those looping constructs executed? How hard is it to DOS
 murano servers (submitting jobs that loop forever). What execution limits
 are imposed? I noted that the parallel construct actually exposes the
 number of green threads (isn't this an avenue for resource starvation?).
 - https://wiki.openstack.org/wiki/Murano/DSL/Blueprint#Object_model

  IMHO, something just doesn't seem right when the above is created,
 fitting a language into YAML seems about as awkward as creating a language
 in XML (xquery[1] for example) . Why was this really preferred over just
 python or something simpler for example, [lua, javascript...], that already
 has language/object constructs... built-in and have runtimes that u can
 control the security domain of (python is not a good choice to run
 arbitrary code-in, find out how much energy google put into python +
 app-engine and u'll see what it takes).

  http://en.wikipedia.org/wiki/XQuery#Examples

   From: Stan Lagun sla...@mirantis.com

 Reply-To: OpenStack Development Mailing List (not for usage questions) 
 openstack-dev@lists.openstack.org
 Date: Friday, March 7, 2014 at 9:36 AM

 To: OpenStack Development Mailing List (not for usage questions) 
 openstack-dev@lists.openstack.org
 Subject: Re: [openstack-dev] [Mistral] Crack at a Real life workflow

  Hello everyone!

  Actually it is possible to construct YAML-based DSL that has all the
 constructs of regular OOP language like Python and at the same time be safe
 enough to be used for execution of untrusted code on shared server.

  Take a look at Murano DSL.
  For example the code above defines class Instance:
 https://github.com/istalker2/MuranoDsl/blob/master/meta/com.mirantis.murano.services.Instance/manifest.yaml
 The part that may be useful for Mistral is under Workflow key.
  Here is some doc on the language:
 https://wiki.openstack.org/wiki/Murano/DSL/Blueprint
   Technically you can code any workflow that you can in Python using such
 language (just don't look at all OOP-related stuff) and it will look very
 similar to Python but be safe as you can only call APIs that are explicitly
 provided for DSL

  Hope this might be helpful for Mistral



 On Fri, Mar 7, 2014 at 10:38 AM, Dmitri Zimine d...@stackstorm.com wrote:

 I just moved the sample to Git; let's leverage git review for specific
 comments on the syntax.


 https://github.com/dzimine/mistral-workflows/commit/d8c4a8c845e9ca49f6ea94362cef60489f2a46a3

  DZ

  On Mar 6, 2014, at 10:36 PM, Dmitri Zimine d...@stackstorm.com wrote:

  Folks, thanks for the input!

  @Joe:

  Hopefully Renat covered the differences.  Yet I am interested in how
 the same workflow can be expressed as Salt state(s) or Ansible playbooks.
 Can you (or someone else who knows them well) take a stub?


  @Joshua
 I am still new to Mistral and learning, but I think it _is_ relevant to
 taskflow. Should we meet, and you help me catch up? Thanks!

  @Sandy:
 Aaahr, I used the D word?!  :) I keep on arguing that YAML workflow
 representation doesn't make DSL.

  And YES to the object model first to define the workflow, with
 YAML/JSON/PythonDSL/what-else as a syntax to build it. We are having these
 discussions on another thread and reviews.

  Basically

Re: [openstack-dev] [Mistral] Crack at a Real life workflow

2014-03-08 Thread Stan Lagun
I would really like to answer all your questions but lets do it in separate
thread :)

 It will just be a tight rope walk to keep that DSL free of issues when it
enables all those features.
MuranoPL doesn't built on Python - MuranoPL engine does. You can
re-implement it on Java, Haskel whatever without loosing compatibility. So
you never get (and never will) access to Python built-ins, Python modules
and implementation details. DSL is safe to the level you want. All
functions, operators, classes etc. that are available DSL are those that
are written in MuranoPL or explicitly injected into DSL. So this is
completely sandboxed. And in Murano we are not going to provide any
operating-system level API. No file access, no direct networking. Just APIs
to talk to other OpenStack services similar to their native python-clients.
So there is no real need to physically isolate MuranoPL executions.

As for your questions on pointers to execution control and resource
constraints currently we do not have them. We are on PoC stage currently.
There is a PoC MuranoPL engine that can execute most of the things but it
lucks some advanced features like those you're talking about. But DSL
engine was designed with such features in mind and they will be implemented
during next several release cycles.

It relatively easy to limit execution time, number of green threads used
and number of executed instructions. As for memory usage the simplest way
of doing this is executing DSL in separate process and measure its memory
usage before execution and periodically during execution. Once memory usage
goes above threshold process is terminated by master process. This method
also increases security/isolation. Also note, that Murano engine is capable
of simultaneous execution of many DSL-based deployments and thus hanging of
one such execution doesn't block/affect other system users in any way.

Personally I believe that YAQL-based DSLs similar (but probably simlet
than) MuranoPL can be of a great value for many OpenStack projects that
have DSLs of different kind. Murano for App Catatalog, Mistral for
workflows, Heat for HOT templates, Ceilometer for alarms  counter
aggregation rules, Nova for customizable resource scheduling and probably
many more. If you like the idea we would appreciate your contributions and
help in language promotion so that together we could solve all the issues
you rightly pointed out


On Sat, Mar 8, 2014 at 9:05 PM, Joshua Harlow harlo...@yahoo-inc.comwrote:

  Thanks for the detailed explanation!

  I have done similar stuff to what google has done for python a long time
 ago for php (when rasmus, the php creator worked at y!) so I know it's not
 simple. The part of me that likes doing languages thinks all this muranoPL
 stuff is neat. But at the same time it worries me, since providing users
 with a avenue to run code (or a DSL that has similar concepts) scares me :P
 Especially if it's running in python. I noted u don't provide access to
 lower lever python functions (besides the green thread one) so that's
 great. It will just be a tight rope walk to keep that DSL free of issues
 when it enables all those features.

  Are there any pointers to the execution control that it is doing? Is
 there a external process time limiting the user code that is running (or is
 it ulimits?)? With raw loop access as it appears the DSL provides it's not
 exactly hard to lock up a whole process (which will lock up any other
 greenthreads to)... How is it that u can calculate ahead of time how much
 memory an object will use (python doesn't exactly provide this afaik) to
 impose memory limits. Are deployers of murano expected to isolate there
 murano deployments to avoid security problems (and resource contention)?

  As for lua and others (I know the javascript built into java also, due
 to some of this being used for something similar at y!), those languages
 have been designed to run in very minimal modes (no access to sockets or
 files by default...) and there run times have been built from the ground up
 to have the features needed to control execution time, memory and resource
 access so it might not really be as hard there as u would think (although
 it's still not easy of course). Python afaik was never really built to have
 these features, to run in very minimal modes without access to things like
 sockets or files... (although google apparently hacked enough of them in
 for usage in app-engine).

  Anyways maybe we should/can continue this on another thread :-)

 Sent from my really tiny device...

 On Mar 8, 2014, at 2:35 AM, Stan Lagun sla...@mirantis.com wrote:

This may be not proper thread to discuss MuranoPL but since I
 suggested Mistral to borrow some parts of it let me answer it here.

  Q: Why not use some existing language?

 A:
   1. There are not so many languages that can be embedded and securely
 used to run code in shared environments. And it cannot be Python itself
 (unless you're Google

Re: [openstack-dev] MuranoPL questions?

2014-03-08 Thread Stan Lagun
First of all MuranoPL is not a host but hosted language. It never intended
to replace Python and if Python can do the job it is probably better than
MuranoPL for that job.
The problem with Python is that you cannot have Python code as a part of
your DSL if you need to evaluate that DSL on server-side. Using Python's
eval() is not secure. And you don't have enough control on what evaled code
is allowed to do. MuranoPL on the contrary are fully sandboxed. You have
absolute control over what functions/methods/APIs are exposed to DSL and
DSL code can do nothing except for what it allowed to do. Besides you
typically do want your DSL to be domain-specific so general-purpose
language like Python can be suboptimal.

I don't say MuranoPL is good for all projects. It has many Murano-specific
things after all. In most cases you don't need all those OOP features that
MuranoPL has. But the code organization, how it uses YAML, block structures
and especially YAQL expressions can be of a great value to many projects

For examples of MuranoPL classes you can browse
https://github.com/istalker2/MuranoDsl/tree/master/meta folder. This is my
private repository that I was using to develop PoC for MuranoPL engine. We
are on the way to create production-quality implementation with unit-tests
etc. in regular Murano repositories on stackforge


On Sun, Mar 9, 2014 at 7:33 AM, Joshua Harlow harlo...@yahoo-inc.comwrote:

 To continue from other thread

 
 Personally I believe that YAQL-based DSLs similar (but probably simlet
 than) MuranoPL can be of a great value for many OpenStack projects that
 have DSLs of different kind. Murano for App Catatalog, Mistral for
 workflows, Heat for HOT templates, Ceilometer for alarms  counter
 aggregation rules, Nova for customizable resource scheduling and probably
 many more.
 

 Isn't python a better host language for said DSLs than muranoPL? I am
 still pretty weary of a DSL that starts to grow so many features to
 encompass other DSLs since then it's not really a DSL but a non-DSL, and we
 already have one that everyone is familiar with (python).

 Are there any good examples if muranoPL that I can look at that take
 advantage of muranoPL classes, functions, namespaces which can be compared
 to there python equivalents. Has such an analysis/evaluation been done?

 Sent from my really tiny device...
 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




-- 
Sincerely yours
Stanislav (Stan) Lagun
Senior Developer
Mirantis
35b/3, Vorontsovskaya St.
Moscow, Russia
Skype: stanlagun
www.mirantis.com
sla...@mirantis.com
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] MuranoPL questions?

2014-03-08 Thread Stan Lagun
Forgot to say that the discussion was started in [openstack-dev] [Mistral]
Crack at a Real life workflow thread


On Sun, Mar 9, 2014 at 10:39 AM, Stan Lagun sla...@mirantis.com wrote:

 First of all MuranoPL is not a host but hosted language. It never intended
 to replace Python and if Python can do the job it is probably better than
 MuranoPL for that job.
 The problem with Python is that you cannot have Python code as a part of
 your DSL if you need to evaluate that DSL on server-side. Using Python's
 eval() is not secure. And you don't have enough control on what evaled code
 is allowed to do. MuranoPL on the contrary are fully sandboxed. You have
 absolute control over what functions/methods/APIs are exposed to DSL and
 DSL code can do nothing except for what it allowed to do. Besides you
 typically do want your DSL to be domain-specific so general-purpose
 language like Python can be suboptimal.

 I don't say MuranoPL is good for all projects. It has many Murano-specific
 things after all. In most cases you don't need all those OOP features that
 MuranoPL has. But the code organization, how it uses YAML, block structures
 and especially YAQL expressions can be of a great value to many projects

 For examples of MuranoPL classes you can browse
 https://github.com/istalker2/MuranoDsl/tree/master/meta folder. This is
 my private repository that I was using to develop PoC for MuranoPL engine.
 We are on the way to create production-quality implementation with
 unit-tests etc. in regular Murano repositories on stackforge


 On Sun, Mar 9, 2014 at 7:33 AM, Joshua Harlow harlo...@yahoo-inc.comwrote:

 To continue from other thread

 
 Personally I believe that YAQL-based DSLs similar (but probably simlet
 than) MuranoPL can be of a great value for many OpenStack projects that
 have DSLs of different kind. Murano for App Catatalog, Mistral for
 workflows, Heat for HOT templates, Ceilometer for alarms  counter
 aggregation rules, Nova for customizable resource scheduling and probably
 many more.
 

 Isn't python a better host language for said DSLs than muranoPL? I am
 still pretty weary of a DSL that starts to grow so many features to
 encompass other DSLs since then it's not really a DSL but a non-DSL, and we
 already have one that everyone is familiar with (python).

 Are there any good examples if muranoPL that I can look at that take
 advantage of muranoPL classes, functions, namespaces which can be compared
 to there python equivalents. Has such an analysis/evaluation been done?

 Sent from my really tiny device...
 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




 --
 Sincerely yours
 Stanislav (Stan) Lagun
 Senior Developer
 Mirantis
 35b/3, Vorontsovskaya St.
 Moscow, Russia
 Skype: stanlagun
 www.mirantis.com
 sla...@mirantis.com




-- 
Sincerely yours
Stanislav (Stan) Lagun
Senior Developer
Mirantis
35b/3, Vorontsovskaya St.
Moscow, Russia
Skype: stanlagun
www.mirantis.com
sla...@mirantis.com
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Mistral] Crack at a Real life workflow

2014-03-07 Thread Stan Lagun
 biased).

 I agree with letting the native language implement the basics
 (expressions, assignment...) and then building the domain ontop of that.
 Just seems more natural IMHO, and is similar to what linq (in c#) has done.

 My 3 cents.

 Sent from my really tiny device...

 On Mar 6, 2014, at 5:33 AM, Sandy Walsh sandy.wa...@rackspace.com
 wrote:

 DSL's are tricky beasts. On one hand I like giving a tool to
 non-developers so they can do their jobs, but I always cringe when the
 DSL reinvents the wheel for basic stuff (compound assignment
 expressions, conditionals, etc).

 YAML isn't really a DSL per se, in the sense that it has no language
 constructs. As compared to a Ruby-based DSL (for example) where you
 still have Ruby under the hood for the basic stuff and extensions to the
 language for the domain-specific stuff.

 Honestly, I'd like to see a killer object model for defining these
 workflows as a first step. What would a python-based equivalent of that
 real-world workflow look like? Then we can ask ourselves, does the DSL
 make this better or worse? Would we need to expose things like email
 handlers, or leave that to the general python libraries?

 $0.02

 -S



 On 03/05/2014 10:50 PM, Dmitri Zimine wrote:
 Folks,

 I took a crack at using our DSL to build a real-world workflow.
 Just to see how it feels to write it. And how it compares with
 alternative tools.

 This one automates a page from OpenStack operation
 guide:
 http://docs.openstack.org/trunk/openstack-ops/content/maintenance.html#planned_maintenance_compute_node

 Here it is https://gist.github.com/dzimine/9380941
 or here http://paste.openstack.org/show/72741/

 I have a bunch of comments, implicit assumptions, and questions which
 came to mind while writing it. Want your and other people's opinions on
 it.

 But gist and paste don't let annotate lines!!! :(

 May be we can put it on the review board, even with no intention to
 check in,  to use for discussion?

 Any interest?

 DZ


 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




-- 
Sincerely yours
Stanislav (Stan) Lagun
Senior Developer
Mirantis
35b/3, Vorontsovskaya St.
Moscow, Russia
Skype: stanlagun
www.mirantis.com
sla...@mirantis.com
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] Incubation Request: Murano

2014-03-04 Thread Stan Lagun
 is XML based and it is
 quite painful to write a workflows without any additional instrument
 like IDE.


 It so happens that the OASIS's TOSCA technical committee are working as we
 speak on a TOSCA Simple Profile that will hopefully make things easier to
 use and includes a YAML representation (the latter is great IMHO, but the
 key to being able to do it is the former). Work is still at a relatively
 early stage and in my experience they are very much open to input from
 implementers.

 I would strongly encourage you to get involved in this effort (by joining
 the TOSCA TC), and also to architect Murano in such a way that it can
 accept input in multiple formats (this is something we are making good
 progress toward in Heat). Ideally the DSL format for Murano+Heat should be
 a trivial translation away from the relevant parts of the YAML
 representation of TOSCA Simple Profile.


 cheers,
 Zane.

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




-- 
Sincerely yours
Stanislav (Stan) Lagun
Senior Developer
Mirantis
35b/3, Vorontsovskaya St.
Moscow, Russia
Skype: stanlagun
www.mirantis.com
sla...@mirantis.com
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Murano] Object-oriented approach for defining Murano Applications

2014-02-24 Thread Stan Lagun
 for PrimaryController and
SecondaryControllers and a Deploy workflow which call appropriate
workflows on the controllers.


 The simplified class diagram may look like this:





 So, this approach allows to decompose the AD deployment workflow into
 simple isolated parts, explicitly manage the state and create reusable
 entities (of course classes like Instance, WindowsInstance, DomainMember
 may be used by other Murano Applications). For me this looks much, much
 better than the current implicit state machine which we run based on XML
 rules.

 What do you think about this approach, folks? Do you think it will be
 easily understood by application developers? Will it be easy to write
 workflows this way? Do you see any drawbacks here?

 Waiting for your feedback.


 --
 Regards,
 Alexander Tivelkov



 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




-- 
Sincerely yours
Stanislav (Stan) Lagun
Senior Developer
Mirantis
35b/3, Vorontsovskaya St.
Moscow, Russia
Skype: stanlagun
www.mirantis.com
sla...@mirantis.com
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Murano] Heat resource isolation withing single stack

2014-02-23 Thread Stan Lagun
 I would like the Murano project to evaluate the upcoming heat
 software-config resources to represent app configuration.

Yes, we are closely watching progress being made in Heat in that area and
evaluating how all new great features can be used in Murano

 Inferring from the above, it sounds like Murano is transforming an app
 definition into a single flat heat template.

It is just not that simple. Murano by itself neither generates any
templates nor deploys applications. Murano is just App Catalog.

Application definitions that Catalog consists of are more then just a
declarations. They also contains imperative executable parts written in
Murano DSL language.
Murano itself can be viewed as a runtime for application manifests.

Murano engine provides carefully designed APIs for DSL code to access Heat
and other OpenStack services. Those APIs are more restrictive and safe than
native OpenStack APIs (provided by various python-clients) and DSL cannot
access OpenStack services by any means other than those APIs. Heat API is
one of them. With this API DSL can access Heat stack and make some changes
in it.

It is up to Murano users (software publishers) who fill AppCatalog to
decide how and if to use Heat. They are free to use Heat just for
infrastructure-level management and do software config by some external
tools or use new software-config in Heat or even directly talk to Nova,
Trove etc.
But no matter how application decides to implement it, Murano must
guarantee that DSL code of one app cannot damage or compromise OpenStack
resources that belongs to another app. So even if new Heat feature makes
life easier we still need to guarantee safe behavior of DSL when using old
Heat template syntax or older Heat version that doesn't have that feature.




On Sun, Feb 23, 2014 at 11:50 PM, Steve Baker sba...@redhat.com wrote:

 On 22/02/14 03:37, Stan Lagun wrote:
  Hi Everyone,
 
  While looking through Heat templates generation code in Murano I've
  realized it has a major design flaw: there is no isolation between
  Heat resources generated by different apps.
 
  Every app manifest can access and modify its environment stack in any
  way. For example it can delete instances and other resources belonging
  to other applications. This may be not so bad for Murano 0.4 but it
  becomes critical for AppCatalog (0.5) as there is no trust relations
  between applications and it may be unacceptable that untrusted
  application can gain complete write access over the whole stack.
 
  There is also a problem of name collisions - resources generated by
  different applications may have the same names. This is especially
  probable between resources generated by different instances of the
  same app. This also affects Parameters/Output of Heat templates as
  each application instance must generate unique names for them (and do
  not forget them later as they are needed to read output results).
 
  I think we need at least to know how we going to solve it before 0.5
 
  Here is possible directions i can think of:
 
  1. Use nested Heat stacks. I'm not sure it solves naming collisions
  and that nested stacks can have their own Output
 
  2. Control all stack template modifications and track which resource
  was created by which app. Give applications read-only access to
  resources they don't own
 
  3. Auto-generate resource names. Auto-add prefixes/suffixes to
  resource/output etc names indicating owning app instance ID and remove
  them upon read access from workflow so that generated names would be
  invisible to workflow. That would also mean all VMs  would have
  generated names
 
  Hope to see better ideas and suggestions in this thread
 
 I would like the Murano project to evaluate the upcoming heat
 software-config resources to represent app configuration.
 Some patches are still in review [1][2] and I expect more changes before
 feature freeze, but now would be a good time for Murano to get involved
 to validate that software-config meets your needs.

 The current implementation of the SoftwareDeployment resource actually
 has the same issue mentioned above; any authenticated user can currently
 create a deployment which results in software being deployed to any
 server in that tenant/project. There may be times where this is useful,
 but there will soon be a default policy enforced which ensures that the
 deployment resource and the server resource belong to the same stack.

 Inferring from the above, it sounds like Murano is transforming an app
 definition into a single flat heat template. Using a combination of
 resource providers and software config should give you a significantly
 cleaner model, and you may even find that some of the problems you're
 solving in Murano are now solved in the Heat layer.

 [1]

 https://review.openstack.org/#/q/status:open+topic:bp/hot-software-config-rest,n,z
 [2]

 https://review.openstack.org/#/q/status:open+topic:bp/hot-software-config,n,z

[openstack-dev] [Murano] Heat resource isolation withing single stack

2014-02-21 Thread Stan Lagun
Hi Everyone,

While looking through Heat templates generation code in Murano I've
realized it has a major design flaw: there is no isolation between Heat
resources generated by different apps.

Every app manifest can access and modify its environment stack in any way.
For example it can delete instances and other resources belonging to other
applications. This may be not so bad for Murano 0.4 but it becomes critical
for AppCatalog (0.5) as there is no trust relations between applications
and it may be unacceptable that untrusted application can gain complete
write access over the whole stack.

There is also a problem of name collisions - resources generated by
different applications may have the same names. This is especially probable
between resources generated by different instances of the same app. This
also affects Parameters/Output of Heat templates as each application
instance must generate unique names for them (and do not forget them later
as they are needed to read output results).

I think we need at least to know how we going to solve it before 0.5

Here is possible directions i can think of:

1. Use nested Heat stacks. I'm not sure it solves naming collisions and
that nested stacks can have their own Output

2. Control all stack template modifications and track which resource was
created by which app. Give applications read-only access to resources they
don't own

3. Auto-generate resource names. Auto-add prefixes/suffixes to
resource/output etc names indicating owning app instance ID and remove them
upon read access from workflow so that generated names would be invisible
to workflow. That would also mean all VMs  would have generated names

Hope to see better ideas and suggestions in this thread

-- 
Sincerely yours
Stanislav (Stan) Lagun
Senior Developer
Mirantis
35b/3, Vorontsovskaya St.
Moscow, Russia
Skype: stanlagun
www.mirantis.com
sla...@mirantis.com
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Murano] Heat resource isolation withing single stack

2014-02-21 Thread Stan Lagun
Steve, thank you for very valuable suggestions. Your block post is really
great - I've read about environments in Heat documentation but didn't
really understood them until now.

Usage of nested stacks may or may not solve my problem depending on what is
possible to do within those stacks.
Let me explain with simple example.

As you probably know Murano uses Heat for all infrastructure-related
operations. This means if some application from Catalog needs VM instance
or any other type of OpenStack resource it creates it by inserting a
snippet into user's Heat stack template and executes UPDATE STACK command.

Now suppose there is WordPress application published in App Catalog.
WordPress app manifest says that it requires installation of MySql. There
is also another application in AppCatalog called GaleraMySql that is known
to be compatible with MySql. In Murano Dashboard user creates new
environment (this corresponds to Heat stack and is not related to what is
called environment in Heat)
and puts WordPress and GaleraMySql on it. Then he connects them so that
GaleraMySql instance would be used in WordPress for MySql requirement.

WordPress and GaleraMySql were developed by different vendors that are not
aware of each others presence. But because of unfortunate combination of
circumstances both vendors chose to merge exactly the same snippet into
user's stack:

Resources: {
myHost: {
Type: AWS::EC2::Instance,
Properties: {
InstanceType: large,
ImageId: someImage
}
}
}

Then instead of 2 different VMs there would be only one. Things would be
even worse if there was already resource myHost in user's stack.
It is more than a name-collision problem as incorrectly written application
manifest can cause any imaginable harm to the stack.

The obvious solution would be to give each app dedicated nested stack and
restrict it to that nested stack only. This would be a best solution. All I
need is to have the same level of control on nested stack I have on outer
stack - get stack template, modify and update them, access output
attributes. Is it possible to retrieve nested stack template, modify it and
populate it back to Heat?

Another option would be create separate top-level stacks for each app. But
in Murano applications themselves composed of smaller parts and in practice
this would lead to creation of dozen stacks with most of them containing
single resource. And then we would have to implement transaction update
between several stacks, coordinated deletion etc. This would also be bad
from a user's point of view at he doesn't expect to find long list of
stacks he has no idea where they came from.

My other options were on how nested stacks can be emulated on top of single
stack by controlling which app created which resource and dynamically
adjust resource names back and forth (myHost in example above) to some
unique values in a way that is opaque to application


On Fri, Feb 21, 2014 at 8:20 PM, Steven Hardy sha...@redhat.com wrote:

 On Fri, Feb 21, 2014 at 06:37:27PM +0400, Stan Lagun wrote:
  Hi Everyone,
 
  While looking through Heat templates generation code in Murano I've
  realized it has a major design flaw: there is no isolation between Heat
  resources generated by different apps.

 Can you define the requirement for isolation in more detail?  Are you
 referring simply to namespace isolation, or do you need auth level
 isolation, e.g something enforced via keystone?

  Every app manifest can access and modify its environment stack in any
 way.
  For example it can delete instances and other resources belonging to
 other
  applications. This may be not so bad for Murano 0.4 but it becomes
 critical
  for AppCatalog (0.5) as there is no trust relations between applications
  and it may be unacceptable that untrusted application can gain complete
  write access over the whole stack.

 All requests to Heat are scoped by tenant/project, so unless you enforce
 resource-level access policy (which we sort-of started looking at with
 OS::Heat::AccessPolicy), this is expected behavior.

  There is also a problem of name collisions - resources generated by
  different applications may have the same names. This is especially
 probable
  between resources generated by different instances of the same app. This
  also affects Parameters/Output of Heat templates as each application
  instance must generate unique names for them (and do not forget them
 later
  as they are needed to read output results).

 A heirarchy of nested stacks, with each application defined as a separate
 stack seems the obvious solution here.

  I think we need at least to know how we going to solve it before 0.5
 
  Here is possible directions i can think of:
 
  1. Use nested Heat stacks. I'm not sure it solves naming collisions and
  that nested stacks can have their own Output

 I think it does, and yes all stacks can have their own outputs

Re: [openstack-dev] [Murano] Need a new DSL for Murano

2014-02-17 Thread Stan Lagun
 to write workflows for application deployment and maintenance.

 Current version of workflow definition markup really have some design
 drawbacks which limit its potential adoption. They are caused by the fact
 that it was never intended for use for Application Catalog use-cases.


 Just curious, is there any reason you're not collaborating on Mistral
 for this rather than both having a workflow engine?

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




-- 
Sincerely yours
Stanislav (Stan) Lagun
Senior Developer
Mirantis
35b/3, Vorontsovskaya St.
Moscow, Russia
Skype: stanlagun
www.mirantis.com
sla...@mirantis.com
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Murano] Need a new DSL for Murano

2014-02-15 Thread Stan Lagun
 it gets u none of the benefits of syntax checking,
 parsing, validation (highlighting...) and all the pain of yaml.

  Something doesn't seem right about the approach of creating languages
 inside the yaml format (in a way it becomes like xsl, yet xsl at least has
 a spec and is well defined).

  My 2 cents

 Sent from my really tiny device...

 On Feb 14, 2014, at 7:22 PM, Alexander Tivelkov ativel...@mirantis.com
 wrote:

Hi folks,

  Murano matures, and we are getting more and more feedback from our
 early adopters. The overall reception is very positive, but at the same
 time there are some complaints as well. By now the most significant
 complaint is is hard to write workflows for application deployment and
 maintenance.

 Current version of workflow definition markup really have some design
 drawbacks which limit its potential adoption. They are caused by the fact
 that it was never intended for use for Application Catalog use-cases.

  I'll briefly touch these drawbacks first:


1. Murano's workflow engine is actually a state machine, however the
workflow markup does not explicitly define the states and transitions.
2. There is no data isolation within any environment, which causes
both potential security vulnerabilities and unpredictable workflow
behaviours.
3. There is no easy way to reuse the workflows and their related
procedures between several applications.
4. The markup uses JSONPath, which relies on Python's 'eval'
function. This is insecure and has to be avoided.
5. 5. The workflow markup is XML-based, which is not a common
practice in the OpenStack community.

   So, it turns out that we have to design and implement a new workflow
 definition notation, which will not have any of the issues mentioned above.

 At the same time, it should still allow to fully specify the
 configuration of any third-party Application, its dependencies with other
 Applications and define specific actions which are required for Application
 deployment, configuration and life cycle management.

 This new notation should allow to do the following:


-

List all the required configuration parameters and dependencies for a
given application
-

Validate user input and match it to the defined parameters
-

Define specific deployment actions and their execution order
-

Define behaviors to handle the events of changes in application's
environment


  Also, it should satisfy the following requirements:


-

Minimize the amount of configuration for common application parts,
i.e. reuse existing configuration parts and add only difference specific 
 to
the application.
-

Allow to use different deployment tools with using the same markup
constructs. i.e. provide a high-level abstraction on the underlying tools
(heat, shell, chef, puppet etc)
-

For security reasons it should NOT allow to execute arbitrary
operations - i.e. should allow to run only predefined set of meaningful
configuration actions.



  So, I would suggest to introduce a simple and domain specific notation
 which would satisfy these needs:

-

Application dependencies and configuration properties are defined
declaratively, in a way similar to how it is done in Heat templates.
-

Each property has special constraints and rules, allowing to validate
the input and applications relationship within the environment.
-

The workflows are defined in imperative way: as a sequence of actions
or method calls. This may include assigning data variables or calling the
workflows of other applications.
-

All of these may be packaged in a YAML format. The example may look
something like this [1]


  The final version may become a bit more complicated, but as the
 starting point this should look fine. I suggest to cover this in more
 details on our next IRC meeting on Tuesday.

  Any feedback or suggestions are appreciated.


  [1] https://etherpad.openstack.org/p/murano-new-dsl-example

  --
  Regards,
 Alexander Tivelkov

  ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




-- 
Sincerely yours
Stanislav (Stan) Lagun
Senior Developer
Mirantis
35b/3, Vorontsovskaya St.
Moscow, Russia
Skype: stanlagun
www.mirantis.com
sla...@mirantis.com
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack

Re: [openstack-dev] Designing and implementing Murano (App Catalog for OpenStack)

2013-11-15 Thread Stan Lagun
Hello,

As far as I see it now here is what will be required:

1. Ability to integrate support for Murano guest VM agent (probably as a
component type plugin).
2. Ability to control configuration actions order by explicitly specifying
pre-requirements for each component
3. All configuration actions that has all pre-requirements met need to be
executed simultaneously (green threads are ok). So that we can do
mass-configuration things like install XX on 100 machines (using 100
components in generated template)
4. Components need to have metadata and agent plugin implementation need to
have access to it so that the values could be passed to Murano Agent.
Standard HOT value referencing  syntax must be supported here as well.
5. Configuration actions performed by component may produce output values
that need to be accessible to other components later in dependency chain.
This may be in form of explicit output attributes (as in HOT template
itself) or by altering own metadata values in runtime
6. Ability to stop orchestration and/or rollback stack on configuration
failures
7. Ability to modify component graph for deployed stack and thus execute
some additional configuration actions on VMs created earlier.


Does this fit your vision of Heat orchestration?


On Fri, Nov 15, 2013 at 1:11 AM, Steve Baker sba...@redhat.com wrote:

  On 11/15/2013 08:30 AM, Alexander Tivelkov wrote:

  Hey Stackers,

  During HK summit  we’ve been discussing an approach to implement app
 catalog for Openstack. We’ve seen certain interest in the topic and heard
 from few people that they are working on similar concepts outside of
 community.

  Apparently Openstack is now mature enough to have a need for an
 application catalog which will fulfil several needs of cloud admins and
 users on platform level:

 - an integration point for applications and services running on top of OS

 - the level of management of lifecycle of applications, distribution
 channels and billing

 - self-service provisioning for applications

  Few weeks ago we proposed to extend the mission of Murano to be an
 Application Catalog for Openstack, simultaneously we decided to contribute
 several parts of the project to other OpenStack projects, mainly Heat and
 Mistral.

  Our intention is to make the mission and scope of the project as lean as
 possible, at the same time leveraging as much of the existing Openstack
 services as possible.

  As many others we’re excited about Solum project and looking forward to
 integrate with Solum and contribute there.

  Now the most exciting part - we are looking for contributors! Use cases,
 architecture, blueprints, code - anything would be helpful.

  Right now is the best time to start, as we are just started figuring out
 definition of Application Catalog, getting the requirements, considering
 use cases and design concepts.

  The project from day one is run on OpenStack infrastructure, following
 all standard OpenStack processes and practices and we’re really making an
 effort to make it very open.

  We believe that this is an initiative which can bring a lot of value to
 the community, let’s try to obtain synergy working on this all together.
 The earlier you express your interest explicitly, the more impact you’re
 going to have on the roadmap.

  The high-level overview of the proposal can be found at our wiki:
 https://wiki.openstack.org/wiki/Murano/ApplicationCatalog

 This is a work-in-progress document, and it is going to be changed - so
 your feedback is very welcome right now.

  We’ve created several etherpads to collaborate and gather the
 requirements together. Feel free to add anything which seems important for
 you:

 https://etherpad.openstack.org/p/AppCatalogUI - requirements for UI

 https://etherpad.openstack.org/p/AppCatalogRoadmap - roadmap with
 features and milestones

 https://etherpad.openstack.org/p/AppCatalogUseCases - some use-cases,
 giving examples of how different users interact with the Application Catalog

  Feel free to add anything to these etherpads - and share any ideas you
 might have with all the community.

 Looking forward for your participation!



   Can you outline what might need to be added to Heat to support Murano?

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




-- 
Sincerely yours
Stanislav (Stan) Lagun
Senior Developer
Mirantis
35b/3, Vorontsovskaya St.
Moscow, Russia
Skype: stanlagun
www.mirantis.com
sla...@mirantis.com
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


  1   2   >