Re: [openstack-dev] [mistal] Mistral logo ideas?

2016-06-24 Thread Nikolay Makhotkin
>
> I like the idea of the logo being a stylized wind turbine. Perhaps it
> could be
> a turbine with a gust of wind. Then we can show that Mistral harnesses the
> power of the wind :-)


I like this idea! Combine Mistral functionality symbol with wind :)

-- 
Best Regards,
Nikolay
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [mistral] Promoting Hardik Parekh to core reviewers

2016-05-10 Thread Nikolay Makhotkin
Welcome to Mistral cores, Hardik Parekh!

My vote is +1.

On Tue, May 10, 2016 at 11:49 AM, Renat Akhmerov 
wrote:

> I’d like to promote Hardik Parekh to Mistral core reviewers.
>
> He was #1 by number of commits and #3 by number of reviews in Mitaka cycle
> and I think he deserved it. His statistics for Mitaka is at [1].
>
> Please vote.
>
> [1]
> http://stackalytics.com/?release=mitaka=mistral-group=marks_id=hardik-parekh047
>
> Renat Akhmerov
> @Nokia
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>


-- 
Best Regards,
Nikolay
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Mistral] Mistral with IPV6

2016-03-09 Thread Nikolay Makhotkin
Hi, Guy!

I think there is the same issue as in ceilometerю Furthermore, Mistral and
Ceilometer are using the same framework for constructing their Rest API.
So, the fix probably should be the same.

On Wed, Mar 9, 2016 at 4:15 PM, Paz, Guy (Nokia - IL) 
wrote:

>
>
> Hello,
>
> We are trying to deploy mistral with IPV6 (Keystone is IPV6, the VM that
> we are installing on is IPV6 too, all OS Endpoints IPV6) – by changing In
> the ‘mistral.conf’ -> ‘bind_host=::’ .
> In Murano it was working pretty straight forward by changing this param..
> but unfortanly for now for Mistral not.
>
> A little but searching in the internet we found this for Ceilometer :
> https://bugs.launchpad.net/ceilometer/+bug/1269243 maybe same
> implementation needed here too.
>
> So we are wondering If in high level if Mistral can support IPV6 and if
> someone can advise here ?
>
> This is the exception when we are trying to start the mistral with
> ‘bind_host=::’
> [root@test3 ~(keystone_admin)]#  /usr/bin/python /usr/bin/mistral-server
> --server api --config-file /etc/mistral/mistral.conf --log-file
> /var/log/mistral/mistral-api.log
> /usr/lib/python2.7/site-packages/novaclient/client.py:784: UserWarning:
> 'get_client_class' is deprecated. Please use `novaclient.client.Client`
> instead.
>   warnings.warn(_LW("'get_client_class' is deprecated. "
>
> |\\//| //   // || |||\\   /\  ||
> ||\\  //||// ||   ||  || //\\ ||  \\// || || || ||
>  // //  \\||
> ||  \/  || ||  \\||   || \\//-||-\\   ||
> ||  || ||   ||   ||   ||  ||  //  \\  ||
> ||  || || _//||   ||  || //\\ |
>
> Mistral Workflow Service, version 1.0.1
>
> Launching server components ['api']...
> 2016-03-06 12:28:37.950 12246 DEBUG stevedore.extension [-] found
> extension EntryPoint.parse('qpid =
> oslo_messaging._drivers.impl_qpid:QpidDriver') _load_plugins
> /usr/lib/python2.7/site-packages/stevedore/extension.py:156
> 2016-03-06 12:28:37.951 12246 DEBUG stevedore.extension [-] found
> extension EntryPoint.parse('amqp =
> oslo_messaging._drivers.protocols.amqp.driver:ProtonDriver') _load_plugins
> /usr/lib/python2.7/site-packages/stevedore/extension.py:156
> 2016-03-06 12:28:37.951 12246 DEBUG stevedore.extension [-] found
> extension EntryPoint.parse('kombu =
> oslo_messaging._drivers.impl_rabbit:RabbitDriver') _load_plugins
> /usr/lib/python2.7/site-packages/stevedore/extension.py:156
> 2016-03-06 12:28:37.952 12246 DEBUG stevedore.extension [-] found
> extension EntryPoint.parse('rabbit =
> oslo_messaging._drivers.impl_rabbit:RabbitDriver') _load_plugins
> /usr/lib/python2.7/site-packages/stevedore/extension.py:156
> 2016-03-06 12:28:37.975 12246 DEBUG stevedore.extension [-] found
> extension EntryPoint.parse('fake =
> oslo_messaging._drivers.impl_fake:FakeDriver') _load_plugins
> /usr/lib/python2.7/site-packages/stevedore/extension.py:156
> 2016-03-06 12:28:37.975 12246 DEBUG stevedore.extension [-] found
> extension EntryPoint.parse('zmq =
> oslo_messaging._drivers.impl_zmq:ZmqDriver') _load_plugins
> /usr/lib/python2.7/site-packages/stevedore/extension.py:156
> Server started.
> 2016-03-06 12:28:37.994 12246 DEBUG oslo_db.sqlalchemy.engines [-] MySQL
> server mode set to
> STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION
> _check_effective_sql_mode
> /usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/engines.py:256
> 2016-03-06 12:28:38.001 12246 DEBUG oslo_concurrency.lockutils [-] Lock
> "service_coordinator" acquired by
> "mistral.coordination.register_membership" :: waited 0.000s inner
> /usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py:253
> 2016-03-06 12:28:38.002 12246 DEBUG oslo_concurrency.lockutils [-] Lock
> "service_coordinator" released by
> "mistral.coordination.register_membership" :: held 0.001s inner
> /usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py:265
> Traceback (most recent call last):
>   File "/usr/lib/python2.7/site-packages/eventlet/hubs/hub.py", line 457,
> in fire_timers
> timer()
>   File "/usr/lib/python2.7/site-packages/eventlet/hubs/timer.py", line 58,
> in __call__
> cb(*args, **kw)
>   File "/usr/lib/python2.7/site-packages/eventlet/greenthread.py", line
> 214, in main
> result = function(*args, **kwargs)
>   File "/usr/lib/python2.7/site-packages/mistral/cmd/launch.py", line 138,
> in launch_api
> app.setup_app()
>   File "/usr/lib64/python2.7/wsgiref/simple_server.py", line 144, in
> make_server
> server = server_class((host, port), handler_class)
>   File "/usr/lib64/python2.7/SocketServer.py", line 419, in __init__
> self.server_bind()
>   File "/usr/lib64/python2.7/wsgiref/simple_server.py", line 48, in
> server_bind
> HTTPServer.server_bind(self)
>   File "/usr/lib64/python2.7/BaseHTTPServer.py", line 108, in server_bind
>SocketServer.TCPServer.server_bind(self)

[openstack-dev] [mistral] Mistral team meeting minutes

2016-02-15 Thread Nikolay Makhotkin
Thank you for attending the meeting today!

Next meeting is scheduled on 22 Feb. It is non-working day in Russia so
Renat, Anastasia and I very likely won't come to the meeting.

Minutes:
http://eavesdrop.openstack.org/meetings/mistral/2016/mistral.2016-02-15-16.00.html
 Log:
http://eavesdrop.openstack.org/meetings/mistral/2016/mistral.2016-02-15-16.00.log.html


-- 
Best Regards,
Nikolay
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [mistral] Promoting Anastasia Kuznetsova to core reviewers

2016-01-31 Thread Nikolay Makhotkin
Hi, great work, Anastasia!

+1 for you!

On Fri, Jan 29, 2016 at 11:27 AM, Lingxian Kong 
wrote:

> +1 from me, welcome Anastasia!
>
> On Fri, Jan 29, 2016 at 9:00 PM, Igor Marnat  wrote:
>
>> Folks,
>> I'm not a core reviewer, but if I was, I'd definitely vote with +1.
>>
>> Good job, Anastasia! Keep going!
>>
>> Regards,
>> Igor Marnat
>>
>> On Fri, Jan 29, 2016 at 10:23 AM, Elisha, Moshe (Nokia - IL) <
>> moshe.eli...@nokia.com> wrote:
>>
>>> A very big +1
>>> 
>>> From: Renat Akhmerov [rakhme...@mirantis.com]
>>> Sent: Friday, January 29, 2016 8:13 AM
>>> To: OpenStack Development Mailing List (not for usage questions)
>>> Subject: [openstack-dev] [mistral] Promoting Anastasia Kuznetsova to
>>> core   reviewers
>>>
>>> Team,
>>>
>>> I’d like to promote Anastasia Kuznetsova to the core team. What she’s
>>> been doing for 2 years is hard to overestimate. She’s done a huge amount of
>>> work reviewing code, bugfixing and participating in public discussions
>>> including our IRC channel #openstack-mistral. She is very reliable,
>>> diligent and consistent about her work.
>>>
>>> Please vote.
>>>
>>> Renat Akhmerov
>>> @ Mirantis Inc.
>>>
>>>
>>>
>>>
>>>
>>> __
>>> OpenStack Development Mailing List (not for usage questions)
>>> Unsubscribe:
>>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>>
>>> __
>>> OpenStack Development Mailing List (not for usage questions)
>>> Unsubscribe:
>>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>
>
> --
> *Regards!*
> *---*
> *Lingxian Kong*
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>


-- 
Best Regards,
Nikolay
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


[openstack-dev] [mistral] N. Makhotkin availability next 2 weeks

2016-01-15 Thread Nikolay Makhotkin
Hi team!

I will be on vacation next 2 weeks right till 31 Jan. Please ping me via
email if I am needed for reviewing some patches.

-- 
Best Regards,
Nikolay
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


[openstack-dev] [OpenStack-Infra] [fuel-plugin-mistral] Request to add group member into the new Openstack project

2016-01-11 Thread Nikolay Makhotkin
Hi openstack-infra team,

Kindly please add nmakhot...@mirantis.com to the following Gerrit ACL groups:

https://review.openstack.org/#/admin/groups/1156,members

https://review.openstack.org/#/admin/groups/1155,members

(ref. https://review.openstack.org/#/c/238890/)

-- 
Best Regards,
Nikolay
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


[openstack-dev] [mistral] Mistral team meeting reminder

2015-12-14 Thread Nikolay Makhotkin
Hi,

This is a reminder that we’ll have a team meeting today at #openstack-
meeting at 16.00 UTC.

Agenda:

   - Review action items
   - Current status (progress, issues, roadblocks, further plans)
   - M-2 status and planning
   - Open discussion


-- 
Best Regards,
Nikolay
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


[openstack-dev] [mistral] Mistral team meeting minutes 12/14/2015

2015-12-14 Thread Nikolay Makhotkin
Hi,


Thank you guys to join us today!

Meeting minutes -
http://eavesdrop.openstack.org/meetings/mistral/2015/mistral.2015-12-14-16.01.html

Meeting full log -
http://eavesdrop.openstack.org/meetings/mistral/2015/mistral.2015-12-14-16.01.log.html



Best Regards,
Nikolay
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


[openstack-dev] Mistral team meeting minutes - 12/07/2015

2015-12-07 Thread Nikolay Makhotkin
Hi,

Thank you for coming us today!

Meeting minutes:
http://eavesdrop.openstack.org/meetings/mistral/2015/mistral.2015-12-07-15.59.html

Log:
http://eavesdrop.openstack.org/meetings/mistral/2015/mistral.2015-12-07-15.59.log.html


Best Regards,
Nikolay Makhotkin
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


[openstack-dev] [pecan] [mistral] Pecan python3 compatibility

2015-10-01 Thread Nikolay Makhotkin
Hi, pecan folks!

I have an question for you about python3 support in pecan library.

In Mistral, we are trying to fix the codebase for supporting python3, but
we are not able to do this since we faced issue with pecan library. If you
want to see the details, see this traceback - [1].
(Actually, something is wrong with HooksController and walk_controller
method)

Does pecan officially support python3 (especially, python3.4 or python3.5)
or not?
I didn't find any info about that in pecan repository.

[1] http://paste.openstack.org/show/475041/

-- 
Best Regards,
Nikolay Makhotkin
@Mirantis Inc.
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [pecan] [mistral] Pecan python3 compatibility

2015-10-01 Thread Nikolay Makhotkin
>
>
> If value is a bounded method, you can replace value.im_class with
> type(value.__self__).
>
>
Yes. Python 3 does not support im_class attribute for methods.

-- 
Best Regards,
Nikolay
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


[openstack-dev] [mistral] Cancelling team meeting today (09/28/2015)

2015-09-28 Thread Nikolay Makhotkin
Mistral Team,

We’re cancelling today’s team meeting because a number of key members won’t
be able to attend.

The next one is scheduled for 5 Oct.


Best Regards,
Nikolay Makhotkin
@Mirantis Inc.
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [mistral] Define better terms for WAITING and DELAYED states

2015-09-25 Thread Nikolay Makhotkin
Thank you Robert for the explanation in details!



>- RUNNING_DELAYED - a substate of RUNNING and it has exactly this
>meaning: it’s generally running but delayed till some later time.
>- WAITING - it is not a substate of RUNNING and hence it means a task
>has not started yet
>
>
Yes, I agree, need to introduce RUNNING_DELAYED state. It reflects the task
is already running but delayed for certain amount of time.

So, we can proceed with this right in Liberty cycle.

-- 
Best Regards,
Nikolay
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


[openstack-dev] [mistral] Team meeting minutes

2015-09-07 Thread Nikolay Makhotkin
Thanks for joining the meeting today!

Meeting minutes:
*http://eavesdrop.openstack.org/meetings/mistral/2015/mistral.2015-09-07-16.00.html
*
Meeting log: 
*http://eavesdrop.openstack.org/meetings/mistral/2015/mistral.2015-09-07-16.00.log.html
*

The next meeting will be on Sept 14. You can post your agenda items at
https://wiki.openstack.org/wiki/Meetings/MistralAgenda

-- 
Best Regards,
Nikolay
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [mistral][yaql] Addressing task result using YAQL function

2015-09-02 Thread Nikolay Makhotkin
Hi,

I also thought about that recently. So, I absolutely agree with this
proposal. It would be nice to see this feature in Liberty.

On Wed, Sep 2, 2015 at 2:17 PM, Renat Akhmerov 
wrote:

> FYI
>
> Renat Akhmerov
> @ Mirantis Inc.
>
>
>
> Begin forwarded message:
>
> *From: *Renat Akhmerov 
> *Subject: **[openstack-dev][mistral][yaql] Addressing task result using
> YAQL function*
> *Date: *2 Sep 2015 17:16:27 GMT+6
> *To: *"OpenStack Development Mailing List (not for usage questions)" <
> openstack-dev@lists.openstack.org>
>
> Hi,
>
> I’d like to propose a few changes after transition to yaql 1.0:
>
> We already moved from using “$.__execution” in DSL to "execution()” and
> from “$.__env” to “env()” where “execution()” and “env()" are registered
> yaql functions. We had to do it because double underscored are prohibited
> in the new yaql.
>
>
> In the spirit of these changes I’m proposing a similar change for
> addressing task result in Mistral DSL.
>
> Currently we have the syntax “$.task_name” that we can use in yaql
> expressions to refer to corresponding task result. The current
> implementation is of that is kind of tricky and, IMO, conceptually wrong
> because referencing this kind of key leads to DB read operation that’s
> requried to fetch task result (it may be big as megabytes so it can’t be
> stored in workflow context all the time. In other words, we create a
> dictionary with side effect and change the initial semantics of dictionary.
> Along with mentioned trickiness of this approach it’s not convenient, for
> example, to debug the code because we can accidentally cause a DB
> operation.
>
> So the solution I’m proposing is to have an explicit yaql function called
> “res” or “result” to extract task result.
>
> *res()* - extracts the result of the current task, i.e. in “publish”
> clause.
> *res(‘task_name’)* - extracts the result of the task with the specified
> name. Can also be used in “publish” clause, if needed
>
> This approach seems more flexible (cause we can add any other functions
> w/o having to make significant changes in WF engine) and expressive because
> user won’t confuse $.task_name with addressing a regular workflow context
> variable.
>
> Of course, this to some extent breaks backwards compatibility. But we
> already changed yaql version which was necessary anyway so it seems like a
> good time to do it.
>
> I’d very much like to have your input on this.
>
> Renat Akhmerov
> @ Mirantis Inc.
>
>
>
>
>


-- 
Best Regards,
Nikolay
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


[openstack-dev] [mistral] Team meeting minutes

2015-08-24 Thread Nikolay Makhotkin
Thanks for joining the meeting today!

Meeting minutes:
http://eavesdrop.openstack.org/meetings/mistral/2015/mistral.2015-08-24-16.01.html
Meeting log:
http://eavesdrop.openstack.org/meetings/mistral/2015/mistral.2015-08-24-16.01.log.html

The next meeting will be on Aug 3. You can post your agenda items at
https://wiki.openstack.org/wiki/Meetings/MistralAgenda

-- 
Best Regards,
Nikolay
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


[openstack-dev] [requirements] [global-requirements] [mistral] Using pika library

2015-08-19 Thread Nikolay Makhotkin
Hi, folks!

Recently we've had the discussion with oslo.messaging team about
acknowledge feature in RabbitMQ:
http://lists.openstack.org/pipermail/openstack-dev/2015-July/068806.html

After that Mistral team did the research about using *kombu *or *pika*
library for implementation of remote procedure call server. Both kombu and
pika was met our requirements. But we would prefer to use *pika* by next
reasons:

 1. It is less complex than *kombu*;
 2. *kombu* itself is abstraction of messaging (but we don't need that);
 3. We had the discussion with one of RabbitMQ developer which advised us
to use *pika* (comparing to *kombu*)


Is it possible to add *pika *to the global-requirements for our needs?

Thanks.
-- 
Best Regards,
Nikolay
@ Mirantis Inc.
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [requirements] [global-requirements] [mistral] Using pika library

2015-08-19 Thread Nikolay Makhotkin
Hi, Davanum!

Have you already read the thread [1]? It is about acknowledge feature in
oslo.messaging. Particularly, about the absent of this feature in
oslo.messaging.

The guys from messaging said that it is very problematically to add that
kind of feature to oslo.messaging because it does not fit to
oslo.messaging's approach.

[1] http://lists.openstack.org/pipermail/openstack-dev/2015-July/068806.html

On Wed, Aug 19, 2015 at 2:35 PM, Davanum Srinivas dava...@gmail.com wrote:

 Nikolay,

 Do you mean that you will *NOT* use oslo.messaging and use pika directly.
 I'd strongly discourage that possibility.

 Few members on the oslo.messaging team are looking at pika library already
 and are doing a prototype:

 https://github.com/dukhlov/oslo.messaging/blob/master/oslo_messaging/_drivers/impl_pika.py

 If you are interested in that effort, please let us know.

 Thanks,
 Dims


 On Wed, Aug 19, 2015 at 7:21 AM, Nikolay Makhotkin 
 nmakhot...@mirantis.com wrote:

 Hi, folks!

 Recently we've had the discussion with oslo.messaging team about
 acknowledge feature in RabbitMQ:
 http://lists.openstack.org/pipermail/openstack-dev/2015-July/068806.html

 After that Mistral team did the research about using *kombu *or *pika*
 library for implementation of remote procedure call server. Both kombu and
 pika was met our requirements. But we would prefer to use *pika* by next
 reasons:

  1. It is less complex than *kombu*;
  2. *kombu* itself is abstraction of messaging (but we don't need that);
  3. We had the discussion with one of RabbitMQ developer which advised us
 to use *pika* (comparing to *kombu*)


 Is it possible to add *pika *to the global-requirements for our needs?

 Thanks.
 --
 Best Regards,
 Nikolay
 @ Mirantis Inc.

 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe:
 openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




 --
 Davanum Srinivas :: https://twitter.com/dims

 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




-- 
Best Regards,
Nikolay
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


[openstack-dev] [mistral] Team meeting reminder - 08/03/2015

2015-08-03 Thread Nikolay Makhotkin
Hi,

This is a reminder that we’ll have a team meeting today at
#openstack-meeting at 16.00 UTC.

Agenda:

   - Review action items
   - Current status (progress, issues, roadblocks, further plans)
   - Liberty-2 release
   - Open discussion


Add your own items either by replying to this email or modifying
https://wiki.openstack.org/wiki/Meetings/MistralAgenda.

-- 
Best Regards,
Nikolay
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [mistral] [murano] An online YAQL evaluator

2015-08-02 Thread Nikolay Makhotkin
Hi guys!

That's awesome! It is very useful for all us!

-- 
Best Regards,
Nikolay
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


[openstack-dev] [mistral] Team meeting minutes

2015-07-27 Thread Nikolay Makhotkin
Thanks for joining the meeting today!

Meeting minutes:
http://eavesdrop.openstack.org/meetings/mistral/2015/mistral.2015-07-27-16.01.html
Meeting log:
http://eavesdrop.openstack.org/meetings/mistral/2015/mistral.2015-07-27-16.01.log.html

The next meeting will be on Aug 3. You can post your agenda items at
https://wiki.openstack.org/wiki/Meetings/MistralAgenda

-- 
Best Regards,
Nikolay
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


[openstack-dev] [mistral] Team meeting reminder - 07/20/2015

2015-07-20 Thread Nikolay Makhotkin
Hi,

This is a reminder that we’ll have a team meeting today at
#openstack-meeting at 16.00 UTC.

Agenda:

   - Review action items
   - Current status (progress, issues, roadblocks, further plans)
   - Execution expiration policy discussion
   - Open discussion


Add your own items either by replying to this email or modifying
https://wiki.openstack.org/wiki/Meetings/MistralAgenda.

-- 
Best Regards,
Nikolay
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


[openstack-dev] [mistral] Team meeting minutes/log - 07/20/2015

2015-07-20 Thread Nikolay Makhotkin
Thanks for joining the meeting today!

Meeting minutes:
http://eavesdrop.openstack.org/meetings/mistral/2015/mistral.2015-07-20-16.00.html
Meeting log:
http://eavesdrop.openstack.org/meetings/mistral/2015/mistral.2015-07-20-16.00.log.html

The next meeting will be on July 27. You can post your agenda items at
https://wiki.openstack.org/wiki/Meetings/MistralAgenda

-- 
Best Regards,
Nikolay
@ Mirantis Inc.
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


[openstack-dev] [oslo.messaging] [mistral] Acknowledge feature of RabbitMQ in oslo.messaging

2015-07-07 Thread Nikolay Makhotkin
Hi,

I am using RabbitMQ as the backend and searched oslo.messaging for message
acknowledgement feature but I found only [1] what is wrong using of
acknowledgement since it acknowledges incoming message before it has been
processed (while it should be done only after processing the message,
otherwise we can lost given message if, say, the server suddenly goes
down).

So, my questions: does oslo.messaging indeed not support correct
acknowledgement feature? Or it is impossible to do for oslo.messaging
paradighm? Or is it somehow connected with that oslo.messaging has to
support a lot of transport types?

Can't it be implemented somehow in oslo.messaging?

[1]
https://github.com/openstack/oslo.messaging/blob/master/oslo_messaging/rpc/dispatcher.py#L135


Best Regards,
Nikolay
@Mirantis Inc.
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [mistral] Proposing Lingxian Kong as a core reviewer

2015-06-22 Thread Nikolay Makhotkin
Hi,

+1 for Lingxian!

On Mon, Jun 22, 2015 at 7:08 PM, Nikolay Makhotkin nmakhot...@mirantis.com
wrote:

 Hi,

 +1 for Lingxian!

 --
 Best Regards,
 Nikolay




-- 
Best Regards,
Nikolay
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Solum][Mistral] Help with a patch

2015-06-18 Thread Nikolay Makhotkin
Hi, Devdatta!

Thank you for catching this and for the patch. I already reviewed it and it
has been merged.

-- 
Best Regards,
Nikolay
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


[openstack-dev] [mistral] Cancelling today's team meeting - 06/08/2015

2015-06-08 Thread Nikolay Makhotkin
Team,

We decided to cancel today’s meeting because a number of key members won’t
be able to attend.

-- 
Best Regards,
Nikolay
@Mirantis Inc.
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Mistral] remind me about with_items concurrency

2015-05-13 Thread Nikolay Makhotkin
Hi, Dmitri!

AFAIK, we made the decision that 'concurrency' is a policy (and our schema
allows to add 'concurrency' property in DSL).

On Wed, May 13, 2015 at 7:37 PM, Dmitri Zimine dzim...@stackstorm.com
wrote:

 Folks, pls remind me where we ended up with ‘concurrency’?

 In the current implementation concurrency is a task policy (and not sure
 how much we tested it, not with unit testing/automated testing).

 Also I recall discussing/ going back and forth re if concurrency is a task
 policy or it belongs to with_items, and at some point we admitted that
 Nikolay was right from the beginning when advocating for concurrency as
 with_item property.

 What’s the desired?

 This is not reopening the discussion: I just need to recall what the
 decision was :)

 DZ.




-- 
Best Regards,
Nikolay
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [mistral] Break_on in Retry policy

2015-04-22 Thread Nikolay Makhotkin
So, in this case I guess 'break-on' will work correctly now:
https://github.com/stackforge/mistral/blob/master/mistral/engine/policies.py#L295-L296

On Wed, Apr 22, 2015 at 2:58 PM, Renat Akhmerov rakhme...@mirantis.com
wrote:

 Lingxian, yes, that’s basically what I suggest too.

 Renat Akhmerov
 @ Mirantis Inc.



  On 22 Apr 2015, at 16:03, Lingxian Kong anlin.k...@gmail.com wrote:
 
  Hi all,
 
  In my understanding, the meaning of the 'break-on' in 'retry' policy
  is just give an oppotunity to end the task execution earlier, i.e. we
  don't need to wait for all the iterations. I prefer that we keep the
  ERROR state, and keep it simple.
 
  On Wed, Apr 22, 2015 at 5:06 PM, Renat Akhmerov rakhme...@mirantis.com
 wrote:
  Ok, after all thinking my suggestion is to leave break-on” but clarify
 its
  semantics and behavior a little bit as follow:
 
  As Dmitri wrote before “success-on” and “error-on” may be easily
 confused
  with “on-success” and “on-error”.
  “retry policy loop may only stop if:
 
  Task action/workflow completed successfully (no need to retry anymore).
 This
  case has nothing to do with “break-on”.
  Task action/workflow failed and some condition (“break-on”) evaluates to
  True. So in this case I don’t think we need to give a user opportunity
 to
  change semantics of task behavior and be able to say “although task
  action/workflow failed I want this task to finish with success”. IMO,
 it may
  be 1) confusing 2) error-prone 3) complicating our understanding of how
  workflow works. In other works, I’m now against of this behavior and I
 think
  that success/error of action/workflow should exactly match to
 success/error
  of task.
 
  Based on the previous thoughts evaluation of “break-on” condition should
  work against task inbound context that doesn’t contain a task result
 itself
  (because the action failed) but may include results of other tasks
 completed
  in parallel branches. The general use case for this is to “stop waiting
 for
  something if we see that fundamental requirements for that are not met”.
 
 
  Thoughts?
 
  Renat Akhmerov
  @ Mirantis Inc.
 
 
 
  On 21 Apr 2015, at 14:11, Nikolay Makhotkin nmakhot...@mirantis.com
 wrote:
 
  Any more suggestions/thoughts here? Dmitri?
 
  More words: succeed-on / fail-on, success-expr / error-expr.
 
  --
  Best Regards,
  Nikolay
 
 __
  OpenStack Development Mailing List (not for usage questions)
  Unsubscribe:
 openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
  http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
 
 
 
 
 __
  OpenStack Development Mailing List (not for usage questions)
  Unsubscribe:
 openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
  http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
 
 
 
 
  --
  Regards!
  ---
  Lingxian Kong
 
 
 __
  OpenStack Development Mailing List (not for usage questions)
  Unsubscribe:
 openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
  http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




-- 
Best Regards,
Nikolay
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [mistral] Break_on in Retry policy

2015-04-21 Thread Nikolay Makhotkin
Any more suggestions/thoughts here? Dmitri?

More words: succeed-on / fail-on, success-expr / error-expr.

-- 
Best Regards,
Nikolay
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Mistral] [Murano] Mistral devstack installation is failing in murano gate job

2015-04-13 Thread Nikolay Makhotkin

 We are facing an issue with Mistral devstack installation in our gate job
 testing murano-congress-mistral integration (policy enforcement) [1] .
 Mistral devstack scripts are failing with following import error [2]


Hi, Filip!

Recently Mistral has moved to new YAQL, and it seems this dependency is
missed (yaql 1.0, currently yaql 1.0.0b2)

I think the root of problem is that Murano and Mistral have different yaql
versions installed.

-- 
Best Regards,
Nikolay
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Mistral] Propose Winson Chan m4dcoder for core team

2015-04-07 Thread Nikolay Makhotkin
It would be good to see Winson as the core contributor in Mistral.

+1 for Winson!



On Tue, Apr 7, 2015 at 11:53 AM, Renat Akhmerov rakhme...@mirantis.com
wrote:

 +1 with my both hands. Winson has been working on Mistral for about a
 year, was actively participating in the very first workflow engine version
 (what we called PoC) and keeps bringing his experience and excellent
 engineering skills to the project.

 Thanks Winson for your passionate work!

 Renat Akhmerov
 @ Mirantis Inc.



  On 07 Apr 2015, at 03:04, Dmitri Zimine dzim...@stackstorm.com wrote:
 
  Hi folks,
 
  I’d like to propose Winson Chan (m4dcoder) to become Mistral core team
 member.
 
  Winson brings unique mix of field experience implementing Mistral
 workflows in user environments, and solid development skills.
 
  He has been contributing to Mistral since Mar 2014. Winson did a 23
 commits - a number of them are non-trivial, like moving RPC to
 oslo-messaging, or introducing environments, or making full validation. He
 has submitted 14 blueprints and detailed design proposals, contributing to
 design and directions. He found, reported, and fixed bugs. He is becoming
 more active on the review board (would encourage to do more).
 
  I am +1 for m4dcoder as a core team member.
 
  DZ.
 
 
 
 
 
 
 __
  OpenStack Development Mailing List (not for usage questions)
  Unsubscribe:
 openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
  http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




-- 
Best Regards,
Nikolay
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [mistral] Break_on in Retry policy

2015-03-04 Thread Nikolay Makhotkin
Ok, we will proceed with error-on and success-on.

Thanks for the reply!

-- 
Best Regards,
Nikolay
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


[openstack-dev] [mistral] Break_on in Retry policy

2015-03-03 Thread Nikolay Makhotkin
Hello,

Recently we've found that break_on property of RetryPolicy is not working
now.

I tried to solve this problem but faced the another problem: How does
'break_on' supposed to work?

Will 'break_on' change the task state to ERROR or SUCCESS?
if ERROR, it means 'we interrupt all retry iterations and keep the state
which was before'.
But if SUCCESS, it means 'we interrupt all retry iterations and assume
SUCCESS state from task because we cought this condition'.

This is ambiguous.

There is a suggestion to use not just 'break_on' but, say, another, more
explicit properties which will delete this ambiguity. For example,
'success_on' and 'error_on'.

Thoughts?

-
Best Regards,
Nikolay
@Mirantis Inc.
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Mistral] Changing expression delimiters in Mistral DSL

2015-02-20 Thread Nikolay Makhotkin
Ok, we will proceed with % % option.

---
Best Regards,
Nikolay
@Mirantis Inc.
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Mistral] Changing expression delimiters in Mistral DSL

2015-02-19 Thread Nikolay Makhotkin
Hi !

From those three I'd choose only { ... }, it looks better for YAML while
'%' sign looks foreign for YAML. Moreover, it needs extra spaces for
writing expressions:

Compare:
1. %$.var + 1%
2. % $.var + 1 %
3. {$.var + 1}

One more point from me: We can't do things just beacuse it is familiar with
N things and it should be so.


Best Regards,
Nikolay
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [keystone] [trusts] [all] How trusts should work by design?

2015-02-18 Thread Nikolay Makhotkin
Hello!

Nova client's CLI parameter 'bypass_url' helps me. The client's API also
has 'management_url' attribute, if this one is specified - the client
doesn't reauthenticate. Also the most of clients have 'endpoint' argument,
so client doesn't make extra call to keystone to retrieve new token and
service_catalog.

Thank you for clarification!


On Mon, Feb 16, 2015 at 11:30 PM, Jamie Lennox jamielen...@redhat.com
wrote:



 - Original Message -
  From: Alexander Makarov amaka...@mirantis.com
  To: OpenStack Development Mailing List (not for usage questions) 
 openstack-dev@lists.openstack.org
  Sent: Tuesday, 17 February, 2015 4:00:05 AM
  Subject: Re: [openstack-dev] [keystone] [trusts] [all] How trusts should
 work by design?
 
 
 https://blueprints.launchpad.net/keystone/+spec/trust-scoped-re-authentication
 
  On Mon, Feb 16, 2015 at 7:57 PM, Alexander Makarov 
 amaka...@mirantis.com 
  wrote:
 
 
 
  We could soften this limitation a little by returning token client tries
 to
  authenticate with.
  I think we need to discuss it in community.
 
  On Mon, Feb 16, 2015 at 6:47 PM, Steven Hardy  sha...@redhat.com 
 wrote:
 
 
  On Mon, Feb 16, 2015 at 09:02:01PM +0600, Renat Akhmerov wrote:
   Yeah, clarification from keystone folks would be really helpful.
   If Nikolaya**s info is correct (I believe it is) then I actually
 dona**t
   understand why trusts are needed at all, they seem to be useless. My
   assumption is that they can be used only if we send requests directly
 to
   OpenStack services (w/o using clients) with trust scoped token
 included in
   headers, that might work although I didna**t checked that yet myself.
   So please help us understand which one of my following assumptions is
   correct?
   1. We dona**t understand what trusts are.
   2. We use them in a wrong way. (If yes, then whata**s the correct
 usage?)
 
  One or both of these seems likely, possibly combined with bugs in the
  clients where they try to get a new token instead of using the one you
  provide (this is a common pattern in the shell case, as the token is
  re-requested to get a service catalog).
 
  This provides some (heat specific) information which may help somewhat:
 
 
 http://hardysteven.blogspot.co.uk/2014/04/heat-auth-model-updates-part-1-trusts.html
 
   3. Trust mechanism itself is in development and cana**t be used at this
   point.
 
  IME trusts work fine, Heat has been using them since Havana with few
  problems.
 
   4. OpenStack clients need to be changed in some way to somehow bypass
   this keystone limitation?
 
  AFAICS it's not a keystone limitation, the behavior you're seeing is
  expected, and the 403 mentioned by Nikolay is just trusts working as
  designed.
 
  The key thing from a client perspective is:
 
  1. If you pass a trust-scoped token into the client, you must not request
  another token, normally this means you must provide an endpoint as you
  can't run the normal auth code which retrieves the service catalog.
 
  2. If you could pass a trust ID in, with a non-trust-scoped token, or
  username/password, the above limitation is removed, but AFAIK none of the
  CLI interfaces support a trust ID yet.
 
  3. If you're using a trust scoped token, you cannot create another trust
  (unless you've enabled chained delegation, which only landed recently in
  keystone). This means, for example, that you can't create a heat stack
  with a trust scoped token (when heat is configured to use trusts), unless
  you use chained delegation, because we create a trust internally.
 
  When you understand these constraints, it's definitely possible to
 create a
  trust and use it for requests to other services, for example, here's how
  you could use a trust-scoped token to call heat:
 
  heat --os-auth-token trust-scoped-token --os-no-client-auth
  --heat-url http://192.168.0.4:8004/v1/ project-id stack-list
 
  The pattern heat uses internally to work with trusts is:
 
  1. Use a trust_id and service user credentials to get a trust scoped
 token
  2. Pass the trust-scoped token into python clients for other projects,
  using the endpoint obtained during (1)
 
  This works fine, what you can't do is pass the trust scoped token in
  without explicitly defining the endpoint, because this triggers
  reauthentication, which as you've discovered, won't work.
 
  Hope that helps!
 
  Steve
 

 So I think what you are seeing, and what heat has come up against in the
 past is a limitation of the various python-*clients and not a problem of
 the actual delegation mechanism from the keystone point of view. This is a
 result of the basic authentication code being copied around between clients
 and then not being kept updated since... probably havana.

 The good news is that if you go with the session based approach then you
 can share these tokens amongst clients without the hacks.

 The identity authentication plugins that keystoneclient offers (v2 and v3
 api for Token and Password) both accept a 

Re: [openstack-dev] [Mistral] Changing expression delimiters in Mistral DSL

2015-02-17 Thread Nikolay Makhotkin
Some suggestions from me:

1. y 1 + $.var  # (short from yaql).
2. { 1 + $.var }  # as for me, looks more elegant than % %. And
visually it is more strong

A also like p7 and p8 suggested by Renat.

On Tue, Feb 17, 2015 at 11:43 AM, Renat Akhmerov rakhme...@mirantis.com
wrote:

 One more:

 p9: \{1 + $.var} # That’s pretty much what
 https://review.openstack.org/#/c/155348/ addresses but it’s not exactly
 that. Note that we don’t have to put it in quotes in this case to deal with
 YAML {} semantics, it’s just a string



 Renat Akhmerov
 @ Mirantis Inc.



 On 17 Feb 2015, at 13:37, Renat Akhmerov rakhme...@mirantis.com wrote:

 Along with % % syntax here are some other alternatives that I checked
 for YAML friendliness with my short comments:

 p1: ${1 + $.var} # Here it’s bad that $ sign is used for two
 different things
 p2: ~{1 + $.var} # ~ is easy to miss in a text
 p3: ^{1 + $.var} # For someone may be associated with regular
 expressions
 p4: ?{1 + $.var}
 p5: {1 + $.var} # This is kinda crazy
 p6: e{1 + $.var} # That looks a pretty interesting option to me, “e”
 could mean “expression” here.
 p7: yaql{1 + $.var} # This is interesting because it would give a clear
 and easy mechanism to plug in other expression languages, “yaql” here is a
 used dialect for the following expression
 p8: y{1 + $.var} # “y” here is just shortened “yaql


 Any ideas and thoughts would be really appreciated!

 Renat Akhmerov
 @ Mirantis Inc.



 On 17 Feb 2015, at 12:53, Renat Akhmerov rakhme...@mirantis.com wrote:

 Dmitri,

 I agree with all your reasonings and fully support the idea of changing
 the syntax now as well as changing system’s API a little bit due to
 recently found issues in the current engine design that don’t allow us, for
 example, to fully implement ‘with-items’ (although that’s a little bit
 different story).

 Just a general note about all changes happening now: *Once we release
 kilo stable release our API, DSL of version 2 must be 100% stable*. I was
 hoping to stabilize it much earlier but the start of production use
 revealed a number of things (I think this is normal) which we need to
 address, but not later than the end of Kilo.

 As far as % % syntax. I see that it would solve a number of problems
 (YAML friendliness, type ambiguity) but my only not strong argument is that
 it doesn’t look that elegant in YAML as it looks, for example, in ERB
 templates. It really reminds me XML/HTML and looks like a bear in a grocery
 store (tried to make it close to one old russian saying :) ). So just for
 this only reason I’d suggest we think about other alternatives, maybe not
 so familiar to Ruby/Chef/Puppet users but looking better with YAML and at
 the same time being YAML friendly.

 I would be good if we could here more feedback on this, especially from
 people who started using Mistral.

 Thanks

 Renat Akhmerov
 @ Mirantis Inc.



 On 17 Feb 2015, at 03:06, Dmitri Zimine dzim...@stackstorm.com wrote:

 SUMMARY:
 

 We are changing the syntax for inlining YAQL expressions in Mistral YAML
 from {1+$.my.var} (or “{1+$.my.var}”) to % 1+$.my.var %

 Below I explain the rationale and the criteria for the choice. Comments
 and suggestions welcome.

 DETAILS:
 -

 We faced a number of problems with using YAQL expressions in Mistral DSL:
 [1] must handle any YAQL, not only the ones started with $; [2] must
 preserve types and [3] must comply with YAML. We fixed these problems by
 applying Ansible style syntax, requiring quotes around delimiters (e.g.
 “{1+$.my.yaql.var}”). However, it lead to unbearable confusion in DSL
 readability, in regards to types:

 publish:
intvalue1: {1+1}” # Confusing: you expect quotes to be string.
intvalue2: {int(1+1)}” # Even this doestn’ clean the confusion
whatisthis:{$.x + $.y}” # What type would this return?

 We got a very strong push back from users in the filed on this syntax.

 The crux of the problem is using { } as delimiters YAML. It is plain wrong
 to use the reserved character. The clean solution is to find a delimiter
 that won’t conflict with YAML.

 Criteria for selecting best alternative are:
 1) Consistently applies to to all cases of using YAML in DSL
 2) Complies with YAML
 3) Familiar to target user audience - openstack and devops

 We prefer using two-char delimiters to avoid requiring extra escaping
 within the expressions.

 The current winner is % %. It fits YAML well. It is familiar to
 openstack/devops as this is used for embedding Ruby expressions in Puppet
 and Chef (for instance, [4]). It plays relatively well across all cases of
 using expressions in Mistral (see examples in [5]):

 ALTERNATIVES considered:
 --

 1) Use Ansible-like syntax:
 http://docs.ansible.com/YAMLSyntax.html#gotchas
 Rejected for confusion around types. See above.

 2) Use functions, like Heat HOT or TOSCA:

 HOT templates and TOSCA doesn’t seem to have a 

[openstack-dev] [mistral] Team meeting minutes 02/16/2015

2015-02-16 Thread Nikolay Makhotkin
Thanks for joining our team meeting today!

 * Meeting minutes:
http://eavesdrop.openstack.org/meetings/mistral/2015/mistral.2015-02-16-16.00.html
 * Meeting log:
http://eavesdrop.openstack.org/meetings/mistral/2015/mistral.2015-02-16-16.00.log.html

The next meeting is scheduled for Feb 23 at 16.00 UTC.
-- 
Best Regards,
Nikolay
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


[openstack-dev] [keystone] [trusts] [all] How trusts should work by design?

2015-02-16 Thread Nikolay Makhotkin
Hello,

Decided to start a new thread due to too much technical details in old
thread.
(You can see thread *[openstack-dev] [keystone] [nova]* )

*The problem:* Trusts can not be used to retrieve a token for further work
with python-projectclient.

I made some research for trust's use cases. The main goal of trusts is
clear to me: delegation of privileges of one user to another on specific
time (or limitless). But if I get a trust and then get a token from it, it
can not be used in any python-client. The reason why it happens so - is
'authenticate' method in almost all python-clients. This method request a
keystone for authentication and get a new auth token. But in case of
trust-scoped token it can't be true - this method always return '403
Forbidden' [1]

*The question:* Is there a way to create a trust and use it for requests to
any other service? E.g., We can get a token from trust and use it (but
actually, we are not).

Or am I misunderstanding trust's purpose? How are trusts should worked?


[1]
https://github.com/openstack/keystone/blob/master/keystone/token/controllers.py#L154-L156


Best Regards,
Nikolay Makhotkin
@Mirantis
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [keystone] [nova]

2015-02-16 Thread Nikolay Makhotkin
Well, if we use trust-scoped token for getting server-list from nova
(simply use nova.servers.list() ),

Novaclient somehow tries to get another token:
https://github.com/openstack/python-novaclient/blob/master/novaclient/client.py#L690-L724

Actually, novaclient does this request: (found from debug of novaclient)

  REQ: curl -i 'http://my_host:5000/v2.0/tokens' -X POST -H Accept:
application/json -H Content-Type: application/json -H User-Agent:
python-novaclient -d '{auth: {token: {id:
78c71fb549244075b3a5d994baa326b3}, tenantName:
b0c9bbb541d541b098c3c0a92412720d}}'

I.e., this is the request for another auth token from keystone. Keystone
here returns 403 because token in request is trust-scoped.

Why I can't do this simple command using trust-scoped token?

Note: Doing the keystone --os-token 5483086d91094a3886ccce1442b538a0
--os-endpoint http://my_host:5000/v2.0 tenant-list, it returns
tenant-list (not 403).
Note2: Doing the server-list request directly to api with trust-scoped
token, it returns 200, not 403:

curl -H X-Auth-Token: 5483086d91094a3886ccce1442b538a0
http://192.168.0.2:8774/v3/servers

{
servers: [ list_of_servers ]
}

How I can use trust-scoped tokrn via client?

On Fri, Feb 13, 2015 at 9:16 PM, Alexander Makarov amaka...@mirantis.com
wrote:

 Adam, Nova client does it for some reason during a call to
 nova.servers.list()


 On Thu, Feb 12, 2015 at 10:03 PM, Adam Young ayo...@redhat.com wrote:

  On 02/12/2015 10:40 AM, Alexander Makarov wrote:

 A trust token cannot be used to get another token:

 https://github.com/openstack/keystone/blob/master/keystone/token/controllers.py#L154-L156
 You have to make your Nova client use the very same trust scoped token
 obtained from authentication using trust without trying to authenticate
 with it one more time.



 Actually, there have been some recent changes to allow re-delegation of
 Trusts, but for older deployments, you are correct.  I hadn't seen anywhere
 here that he was trying to use a trust token to get another token, though.



 On Wed, Feb 11, 2015 at 9:10 PM, Adam Young ayo...@redhat.com wrote:

  On 02/11/2015 12:16 PM, Nikolay Makhotkin wrote:

 No, I just checked it. Nova receives trust token and raise this error.

  In my script, I see:

  http://paste.openstack.org/show/171452/

  And as you can see, token from trust differs from direct user's token.


  The original user needs to have the appropriate role to perform the
 operation on the specified project.  I see the admin role is created on the
 trust. If the trustor did not have that role, the trustee would not be able
 to exececute the trust and get a token.  It looks like you were able to
 execute the trust and get a token,  but I would like you to confirm that,
 and not just trust the keystone client:  either put debug statements in
 Keystone or call the POST to tokens from curl with the appropriate options
 to get a trust token.  In short, make sure you have not fooled yourself.
 You can also look in the token table inside Keystone to see the data for
 the trust token, or validate the token  via curl to see the data in it.  In
 all cases, there should be an OS-TRUST stanza in the token data.


 If it is still failing, there might be some issue on the Policy side.  I
 have been assuming that you are running with the default policy for Nova.

 http://git.openstack.org/cgit/openstack/nova/tree/etc/nova/policy.json

 I'm not sure which rule matches for list servers (Nova developer input
 would be appreciated)  but I'm guessing it is executing the rule

 admin_or_owner: is_admin:True or project_id:%(project_id)s,

 Since that is the default. I am guessing that the project_id in question
 comes from the token here, as that seems to be common, but if not, it might
 be that the two values are mismatched. Perhaps there Proejct ID value from
 the client env var is sent, and matches what the trustor normally works as,
 not the project in question.  If these two values don't match, then, yes,
 the rule would fail.




 On Wed, Feb 11, 2015 at 7:55 PM, Adam Young ayo...@redhat.com wrote:

   On 02/11/2015 10:52 AM, Nikolay Makhotkin wrote:

 Hi !

  I investigated trust's use cases and encountered the problem: When I
 use auth_token obtained from keystoneclient using trust, I get *403*
 Forbidden error:  *You are not authorized to perform the requested
 action.*

  Steps to reproduce:

  - Import v3 keystoneclient (used keystone and keystoneclient from
 master, tried also to use stable/icehouse)
 - Import v3 novaclient
 - initialize the keystoneclient:
   keystone = keystoneclient.Client(username=username,
 password=password, tenant_name=tenant_name, auth_url=auth_url)

  - create a trust:
   trust = keystone.trusts.create(
 keystone.user_id,
 keystone.user_id,
 impersonation=True,
 role_names=['admin'],
 project=keystone.project_id
   )

  - initialize new keystoneclient:
client_from_trust = keystoneclient.Client(
 username=username, password

Re: [openstack-dev] [keystone] [nova]

2015-02-11 Thread Nikolay Makhotkin
No, I just checked it. Nova receives trust token and raise this error.

In my script, I see:

http://paste.openstack.org/show/171452/

And as you can see, token from trust differs from direct user's token.

On Wed, Feb 11, 2015 at 7:55 PM, Adam Young ayo...@redhat.com wrote:

  On 02/11/2015 10:52 AM, Nikolay Makhotkin wrote:

 Hi !

  I investigated trust's use cases and encountered the problem: When I use
 auth_token obtained from keystoneclient using trust, I get *403*
 Forbidden error:  *You are not authorized to perform the requested
 action.*

  Steps to reproduce:

  - Import v3 keystoneclient (used keystone and keystoneclient from
 master, tried also to use stable/icehouse)
 - Import v3 novaclient
 - initialize the keystoneclient:
   keystone = keystoneclient.Client(username=username, password=password,
 tenant_name=tenant_name, auth_url=auth_url)

  - create a trust:
   trust = keystone.trusts.create(
 keystone.user_id,
 keystone.user_id,
 impersonation=True,
 role_names=['admin'],
 project=keystone.project_id
   )

  - initialize new keystoneclient:
client_from_trust = keystoneclient.Client(
 username=username, password=password,
 trust_id=trust.id, auth_url=auth_url,
   )

  - create nova client using new token from new client:
nova = novaclient.Client(
 auth_token=client_from_trust.auth_token,
 auth_url=auth_url_v2,
 project_id=from_trust.project_id,
 service_type='compute',
 username=None,
 api_key=None
   )

  - do simple request to nova:
   nova.servers.list()

  - get the error described above.


 Maybe I misunderstood something but what is wrong? I supposed I just can
 work with nova like it was initialized using direct token.


 From what you wrote here it should work, but since Heat has been doing
 stuff like this for a while, I'm pretty sure it is your setup and not a
 fundamental problem.

 I'd take a look at what is going back and forth on the wire and make sure
 the right token is being sent to Nova.  If it is the original users token
 and not the trust token, then you would see that error.


  --
  Best Regards,
 Nikolay


 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: 
 openstack-dev-requ...@lists.openstack.org?subject:unsubscribehttp://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




-- 
Best Regards,
Nikolay
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


[openstack-dev] [keystone] [nova]

2015-02-11 Thread Nikolay Makhotkin
Hi !

I investigated trust's use cases and encountered the problem: When I use
auth_token obtained from keystoneclient using trust, I get *403* Forbidden
error:  *You are not authorized to perform the requested action.*

Steps to reproduce:

- Import v3 keystoneclient (used keystone and keystoneclient from master,
tried also to use stable/icehouse)
- Import v3 novaclient
- initialize the keystoneclient:
  keystone = keystoneclient.Client(username=username, password=password,
tenant_name=tenant_name, auth_url=auth_url)

- create a trust:
  trust = keystone.trusts.create(
keystone.user_id,
keystone.user_id,
impersonation=True,
role_names=['admin'],
project=keystone.project_id
  )

- initialize new keystoneclient:
  client_from_trust = keystoneclient.Client(
username=username, password=password,
trust_id=trust.id, auth_url=auth_url,
  )

- create nova client using new token from new client:
  nova = novaclient.Client(
auth_token=client_from_trust.auth_token,
auth_url=auth_url_v2,
project_id=from_trust.project_id,
service_type='compute',
username=None,
api_key=None
  )

- do simple request to nova:
  nova.servers.list()

- get the error described above.


Maybe I misunderstood something but what is wrong? I supposed I just can
work with nova like it was initialized using direct token.

-- 
Best Regards,
Nikolay
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Mistral] Converging discussions on WF context and WF/task/action inputs

2014-12-25 Thread Nikolay Makhotkin

 One thing that I strongly suggest is that we clearly define all reserved
 keys like “env”, “__actions” etc. I think it’d be better if they all
 started with the same prefix, for example, double underscore.


I absolutely agree here. We should use specific keywords with __ prefix
like we used __executions.

On Thu, Dec 25, 2014 at 8:51 AM, Renat Akhmerov rakhme...@mirantis.com
wrote:


 On 24 Dec 2014, at 23:37, W Chan m4d.co...@gmail.com wrote:

 * 2) Retuning to first example:
 ** ...
 **  action: std.sql conn_str={$.env.conn_str} query={$.query}
 ** ...
 ** $.env - is it a name of environment or it will be a registered syntax to 
 getting access to values from env ?
 *

 I was actually thinking the environment will use the reserved word env in 
 the WF context.  The value for the env key will be the dict supplied either 
 DB lookup by name, by dict, or by JSON from CLI.

 Ok, probably here’s the place where I didn’t understand you before. I
 thought “env” here is just a arbitrary key that users themselves may want
 to have to just group some variables under a single umbrella. What you’re
 saying is that whatever is under “$.env” is just the exact same environment
 that we passed when we started the workflow? If yes then it definitely
 makes sense to me (it just allows to explicitly access environment, not
 through the implicit variable lookup). Please confirm.

 One thing that I strongly suggest is that we clearly define all reserved
 keys like “env”, “__actions” etc. I think it’d be better if they all
 started with the same prefix, for example, double underscore.

 The nested dict for __actions (and all other keys with double underscore) 
 is special system purpose, in this case declaring defaults for action inputs. 
  Similar to __execution where it's for containing runtime data for the WF 
 execution.

 Yes, that’s clear


 Renat Akhmerov
 @ Mirantis Inc.


 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




-- 
Best Regards,
Nikolay
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


[openstack-dev] [Mistral] For-each

2014-12-15 Thread Nikolay Makhotkin
Hi,

Here is the doc with suggestions on specification for for-each feature.

You are free to comment and ask questions.

https://docs.google.com/document/d/1iw0OgQcU0LV_i3Lnbax9NqAJ397zSYA3PMvl6F_uqm0/edit?usp=sharing



-- 
Best Regards,
Nikolay
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Mistral]

2014-12-09 Thread Nikolay Makhotkin
Guys,

May be I misunderstood something here but what is the difference between
this one and
https://blueprints.launchpad.net/mistral/+spec/mistral-execution-environment
?

On Tue, Dec 9, 2014 at 5:35 PM, Dmitri Zimine dzim...@stackstorm.com
wrote:

 Winson,

 thanks for filing the blueprint:
 https://blueprints.launchpad.net/mistral/+spec/mistral-global-context,

 some clarification questions:
 1) how exactly would the user describe these global variables
 syntactically? In DSL? What can we use as syntax? In the initial workflow
 input?
 2) what is the visibility scope: this and child workflows, or truly
 global”?
 3) What are the good default behavior?

 Let’s detail it a bit more.

 DZ



 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




-- 
Best Regards,
Nikolay
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Mistral] Query on creating multiple resources

2014-12-08 Thread Nikolay Makhotkin
Hi, Sushma!

Can we create multiple resources using a single task, like multiple
 keypairs or security-groups or networks etc?


Yes, we can. This feature is in the development now and it is considered as
experimental -
https://blueprints.launchpad.net/mistral/+spec/mistral-dataflow-collections

Just clone the last master branch from mistral.

You can specify for-each task property and provide the array of data to
your workflow:

 

version: '2.0'

name: secgroup_actions

workflows:
  create_security_group:
type: direct
input:
  - array_with_names_and_descriptions

tasks:
  create_secgroups:

for-each:

  data: $.array_with_names_and_descriptions
action: nova.security_groups_create name={$.data.name}
description={$.data.description}


On Mon, Dec 8, 2014 at 6:36 PM, Zane Bitter zbit...@redhat.com wrote:

 On 08/12/14 09:41, Sushma Korati wrote:

 Can we create multiple resources using a single task, like multiple
 keypairs or security-groups or networks etc?


 Define them in a Heat template and create the Heat stack as a single task.

 - ZB

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




-- 
Best Regards,
Nikolay
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


[openstack-dev] [mistral] Team meeting minutes/log 12/08/2014

2014-12-08 Thread Nikolay Makhotkin
Thanks for joining our team meeting today!

 * Meeting minutes:
*http://eavesdrop.openstack.org/meetings/mistral/2014/mistral.2014-12-08-16.04.log.html
http://eavesdrop.openstack.org/meetings/mistral/2014/mistral.2014-12-08-16.04.log.html*
 * Meeting log:
*http://eavesdrop.openstack.org/meetings/mistral/2014/mistral.2014-12-08-16.04.html
http://eavesdrop.openstack.org/meetings/mistral/2014/mistral.2014-12-08-16.04.html*

The next meeting is scheduled for Dec 15 at 16.00 UTC.

-- 
Best Regards,
Nikolay
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


[openstack-dev] [pecan] [WSME] Different content-type in request and response

2014-11-25 Thread Nikolay Makhotkin
Hi, folks!

I try to create a controller which should receive one http content-type in
request but it should be another content-type in response. I tried to use
pecan and wsme decorators for controller's methods.

I just want to receive text on server and send json-encoded string from
server (request has text/plain and response - application/json)

I tried:

class MyResource(resource.Resource):
id = wtypes.text
name = wtypes.text


class MyResourcesController(rest.RestController):
@wsexpose(MyResource, body=wtypes.text)
def put(self, text):
return MyResource(id='1', name=text)


According to WSME documentation (
http://wsme.readthedocs.org/en/latest/integrate.html#module-wsmeext.pecan)
signature wsexpose method as following:

  wsexpose(*return_type*, **arg_types*, ***options*)

Ok, I just set MyResource as return_type and body to text type. But it
didn't work as expected:
http://paste.openstack.org/show/138268/

I looked at pecan documentation at
https://media.readthedocs.org/pdf/pecan/latest/pecan.pdf but I didn't find
anything that can fit to my case.

Also, I tried:

class MyResource(resource.Resource):
id = wtypes.text
name = wtypes.text


class MyResourcesController(rest.RestController):
@expose('json')
@expose(content_type=text/plain)
def put(self):
text = pecan.request.text
return MyResource(id='1', name=text).to_dict()

It worked just in case if request and response have the same content-type.
(application/json-application/json, text/plain-text/plain)

I also tried a lot of combination of parameters but it is still not worked.

Does anyone know what the problem is?
How it can be done using WSME and/or Pecan?

Sorry if I misunderstand something.
-- 
Best Regards,
Nikolay
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


[openstack-dev] [mistral] Team meeting - 11/24/2014

2014-11-24 Thread Nikolay Makhotkin
This is a reminder about our team meeting scheduled for today 16.00 UTC.

Agenda:

   - Review action items
   - Paris Summit results
   - Current status (progress, issues, roadblocks, further plans)
   - Release 0.2 progress
   - Open discussion


-- 
Best Regards,
Nikolay
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


[openstack-dev] [mistral] Team meeting minutes/log - 11/24/2014

2014-11-24 Thread Nikolay Makhotkin
Thanks for joining us today,

Here are the links to the meeting minutes and full log:

   - Minutes -
   
http://eavesdrop.openstack.org/meetings/mistral/2014/mistral.2014-11-24-16.00.html
   - Full log -
*http://eavesdrop.openstack.org/meetings/mistral/2014/mistral.2014-11-24-16.00.log.html
   
http://eavesdrop.openstack.org/meetings/mistral/2014/mistral.2014-11-24-16.00.log.html*


The next meeting will be next Monday Dec 1 at 16.00 UTC.

-- 
Best Regards,
Nikolay
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Mistral] Event Subscription

2014-11-11 Thread Nikolay Makhotkin
Hi, Winson,

This is great!

But I have a question:

 9. Operation in python-mistralclient to re-publish events for a given
 workflow execution (in case where client applications was downed and need
 the data to recover).

How does Mistral know about all the events which it has already sent to
subscriber?

I guess you propose to store this data in EventSubscriber object. Correct
me if I mistaked.

On Tue, Nov 11, 2014 at 11:28 PM, W Chan m4d.co...@gmail.com wrote:

 Regarding blueprint to register event listeners to notify client
 applications on state changes (
 https://blueprints.launchpad.net/mistral/+spec/mistral-event-listeners-http),
 I want to propose the following.

 1. Refer this feature as event subscription instead of callback
 2. Event subscription supports only HTTP web hooks with retry policy in
 this first attempt
 3. Event subscription can be defined for a list of specific events,
 workflows, projects, domains, or any combinations (all if list is empty).
 4. Decorator to publish event (similar to logging) and place decorators at
 Engine and TaskPolicy methods @
 https://github.com/stackforge/mistral/blob/master/mistral/engine1/base.py
 .
 5. Events should be published to a queue and then processed by a worker so
 not to disrupt actual workflow/task executions.
 6. API controller to register event subscriber
 a. New resource type named EventSubscriber
 b. New REST controller named EventSubscribersController and CRUD
 operations
 7. DB v2 sqlalchemy model named EventSubscriber and appropriate CRUD
 methods
 8. Operations in python-mistralclient to manage CRUD for subscribers.
 9. Operation in python-mistralclient to re-publish events for a given
 workflow execution (in case where client applications was downed and need
 the data to recover).


 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




-- 
Best Regards,
Nikolay
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


[openstack-dev] [mistral] Team meeting minutes 10/06/2014

2014-10-06 Thread Nikolay Makhotkin
Thanks everyone for participating the meeting today!

In case you’d like to see what we discussed, here’s

Meeting minutes:
http://eavesdrop.openstack.org/meetings/mistral/2014/mistral.2014-10-06-16.00.html
http://eavesdrop.openstack.org/meetings/mistral/2014/mistral.2014-10-06-16.00.html
Meeting full log:
http://eavesdrop.openstack.org/meetings/mistral/2014/mistral.2014-10-06-16.00.log.html
The next meeting is scheduled for Oct 13.

-- 
Best Regards,
Nikolay
@Mirantis Inc.
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


[openstack-dev] [mistral] [fuel] Executor task affinity

2014-10-02 Thread Nikolay Makhotkin
Hi, folks!

I drafted the document where we can see how task affinity will be applied
to Mistral:

https://docs.google.com/a/mirantis.com/document/d/17O51J1822G9KY_Fkn66Ul2fc56yt9T4NunnSgmaehmg/edit

-- 
Best Regards,
Nikolay
@Mirantis Inc.
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


[openstack-dev] [mistral] Team meeting minutes 09/29/2014

2014-09-29 Thread Nikolay Makhotkin
Thanks everyone for participating the meeting today!

In case you’d like to see what we discussed, here’s

Meeting minutes:
http://eavesdrop.openstack.org/meetings/mistral/2014/mistral.2014-09-29-16.01.html
Meeting full log:
http://eavesdrop.openstack.org/meetings/mistral/2014/mistral.2014-09-29-16.01.log.html

The next meeting is scheduled for Oct 6.

-- 
Best Regards,
Nikolay
@ Mirantis Inc.
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Mistral] task update at end of handle_task in executor

2014-03-27 Thread Nikolay Makhotkin

 In case of async tasks, executor keeps the task status at RUNNING, and a
 3rd party system will call convey_task_resutls on engine.


Yes, it is correct.  With this approach (in case sync task), we set task
state to SUCCESS if it returns a result, or ERROR if we can't see a result
and exception is raised.

 It is a bug and should be done before line 119:  self._do_task_action(
 db_task).


And also lines 120-123 (
https://github.com/stackforge/mistral/blob/master/mistral/engine/scalable/executor/server.py#L120-L123)
are incorrect since in _do_task_action updates the task state. But we have
two different types of task (async, sync) and I think we should update task
state to RUNNING before invoking _do_task_action and remove this lines -
https://github.com/stackforge/mistral/blob/master/mistral/engine/scalable/executor/server.py#L58-L61
.




On Thu, Mar 27, 2014 at 9:47 AM, Manas Kelshikar ma...@stackstorm.comwrote:

 Yes. It is a bug and should be done before line 119:  self._do_task_action
 (db_task). It can definitely lead to bugs especially since
 _do_task_action itself updates the status.




 On Wed, Mar 26, 2014 at 8:46 PM, W Chan m4d.co...@gmail.com wrote:

 In addition, for sync tasks, it'll overwrite the task state from SUCCESS
 to RUNNING.


 On Wed, Mar 26, 2014 at 8:41 PM, Dmitri Zimine d...@stackstorm.com wrote:

 My understanding is: it's the engine which finalizes the task results,
 based on the status returned by the task via convey_task_result call.


 https://github.com/stackforge/mistral/blob/master/mistral/engine/abstract_engine.py#L82-L84

 https://github.com/stackforge/mistral/blob/master/mistral/engine/scalable/executor/server.py#L44-L66

 In case of async tasks, executor keeps the task status at RUNNING, and a
 3rd party system will call convey_task_resutls on engine.


 https://github.com/stackforge/mistral/blob/master/mistral/engine/scalable/executor/server.py#L123
 ,


 This line however looks like a bug to me:  at best it doesnt do much and
 at worst it overwrites the ERROR previously set in here
 http://tinyurl.com/q5lps2h

 Nicolay, any better explanation?


 DZ

 On Mar 26, 2014, at 6:20 PM, W Chan m4d.co...@gmail.com wrote:

 Regarding
 https://github.com/stackforge/mistral/blob/master/mistral/engine/scalable/executor/server.py#L123,
 should the status be set to SUCCESS instead of RUNNING?  If not, can
 someone clarify why the task should remain RUNNING?

 Thanks.
 Winson

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




-- 
Best Regards,
Nikolay
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Mistral] Error on running tox

2014-03-12 Thread Nikolay Makhotkin
maybe something wrong with python2.6?

.tox/py26/lib/python2.6/site-packages/mock.py, line 1201, in patched


what if you try it on py27?



On Wed, Mar 12, 2014 at 10:08 AM, Renat Akhmerov rakhme...@mirantis.comwrote:

 Ok. I might be related with oslo.messaging change that we merged in
 yesterday but I don't see at this point how exactly.

 Renat Akhmerov
 @ Mirantis Inc.



 On 12 Mar 2014, at 12:38, Manas Kelshikar ma...@stackstorm.com wrote:

 Yes it is 100% reproducible.

 Was hoping it was environmental i.e. missing some dependency etc. but
 since that does not seem to be the case I shall debug locally and report
 back.

 Thanks!


 On Tue, Mar 11, 2014 at 9:54 PM, Renat Akhmerov rakhme...@mirantis.comwrote:

 Hm.. Interesting. CI wasn't able to reveal this for some reason.

 My first guess is that there's a race condition somewhere. Did you try to
 debug it? And is this error 100% repeatable?

 Renat Akhmerov
 @ Mirantis Inc.



 On 12 Mar 2014, at 11:18, Manas Kelshikar ma...@stackstorm.com wrote:

 I see this error when I run tox. I pulled down a latest copy of master
 and tried to setup the environment. Any ideas?

 See http://paste.openstack.org/show/73213/ for details. Any help is
 appreciated.


 Thanks,

 Manas
 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




-- 
Best Regards,
Nikolay
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Mistral] DSL model vs. DB model, renaming

2014-03-06 Thread Nikolay Makhotkin
Manas, Renat, no problem :)

The commit is sent already - https://review.openstack.org/#/c/75888/


On Thu, Mar 6, 2014 at 12:14 PM, Manas Kelshikar ma...@stackstorm.comwrote:

 I agree, let's rename data to spec and unblock the check-in.

 Nikolay - Sorry for the trouble :)
 On Mar 5, 2014 10:17 PM, Renat Akhmerov rakhme...@mirantis.com wrote:

 Alright, good input Manas, appreciate.

 My comments are below...

 On 06 Mar 2014, at 10:47, Manas Kelshikar ma...@stackstorm.com wrote:


- Do we have better ideas on how to work with DSL? A good mental
exercise here would be to imagine that we have more than one DSL, not only
YAML but say XML. How would it change the picture?

 [Manas] As long as we form an abstraction between the DSL format i.e.
 YAML/XML and it consumption we will be able to move between various
 formats. (wishful) My personal preference is to not even have DSL show up
 anywhere in Mistral code apart from take it as input and transforming into
 this first level specification model - I know this is not the current state.


 Totally agree with your point. That's what we're trying to achieve.


- How can we clearly distinguish between these two models so that it
wouldn't be confusing?
- Do we have a better naming in mind?

 [Manas] I think we all would agree that the best approach is to have
 precise naming.

 I see your point of de-normalizing the dsl data into respective db model
 objects.

 In a previous email I suggested using *Spec. I will try to build on this -
 1. Everything specified via the YAML input is a specification or
 definition or template. Therefore I suggest we suffix all these types with
 Spec/Definition/Template. So TaskSpec/TaskDefinition/TaskTemplate etc.. As
 per the latest change these are TaskData ... ActionData.


 After all the time I spent thinking about it I would choose Spec suffix
 since it's short and expresses the intention well enough. In conjunction
 with workbook package name it would look very nice (basically we get
 specification of workbook which is what we're talking about, right?)

 So if you agree then let's change to TaskSpec, ActionSpec etc. Nikolay,
 sorry for making you change this patch again and again :) But it's really
 important and going to have a long-term effect at the entire system.

 2. As per current impl the YAML is stored as a key-value in the DB. This
 is fine since that is front-ended by objects that Nikolay has introduced.
 e.g. TaskData, ActionData etc.


 Yep, right. The only thing I would suggest is to avoid DB fields like
 task_dsl like we have now. The alternative could be task_spec.

 3. As per my thinking a model object that ends up in the DB or a model
 object that is in memory all can reside in the same module. I view
 persistence as an orthogonal concern so no real reason to distinguish the
 module names of the two set of models. If we do choose to distinguish as
 per latest change i.e. mistral/workbook that works too.


 Sorry, I believe I wasn't clear enough on this thing. I think we
 shouldn't have these two models in the same package since what I meant by
 DB model is actually execution and task that carry workflow runtime
 information and refer to a particular execution (we could also call it
 session). So my point is that these are fundamentally different types of
 models. The best analogy that comes to my mind is the relationship class
 - instance where in our case class = Specification (TaskSpec etc.) and
 instance = Execution/Task. Does it make any sense?

 @Nikolay - I am generally ok with the approach. I hope that this helps
 clarify my thinking and perception. Please ask more questions.

 Overall I like the approach of formalizing the 2 models. I am ok with
 current state of the review and have laid out my preferences.


 I like the current state of this patch. The only thing I would do is
 renaming Data to Spec.

 Thank you.

 Renat Akhmerov
 @ Mirantis Inc.



 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




-- 
Best Regards,
Nikolay
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Mistral] DSL model vs. DB model, renaming

2014-03-05 Thread Nikolay Makhotkin
I think today and I have a good name for package (instead of
'mistral/model')

How do you think about to name it 'mistral/workbook'? I.e., It means that
it contains modules for work with workbook representation - tasks,
services, actions and workflow.

This way we able to get rid of any confusing.


Best Regards,
Nikolay

On Wed, Mar 5, 2014 at 8:50 AM, Renat Akhmerov rakhme...@mirantis.comwrote:

 I think we forgot to point to the commit itself. Here it is:
 https://review.openstack.org/#/c/77126/

 Manas, can you please provide more details on your suggestion?

 For now let me just describe the background of Nikolay's question.

 Basically, we are talking about how we are working with data inside
 Mistral. So far, for example, if a user sent a request to Mistral start
 workflow then Mistral would do the following:

- Get workbook DSL (YAML) from the DB (given that it's been already
persisted earlier).
- Load it into a dictionary-like structure using standard 'yaml'
library.
- Based on this dictionary-like structure create all necessary DB
objects to track the state of workflow execution objects and individual
tasks.
- Perform all the necessary logic in engine and so on. The important
thing here is that DB objects contain corresponding DSL snippets as they
are described in DSL (e.g. tasks have property task_dsl) to reduce the
complexity of relational model that we have in DB. Otherwise it would be
really complicated and most of the queries would contain lots of joins. The
example of non-trivial relation in DSL is task-action
name-service-service actions-action, as you can see it would be
hard to navigate to action in the DB from a task if our relational model
matches to what we have in DSL. this approach leads to the problem of
addressing any dsl properties using hardcoded strings which are spread
across the code and that brings lots of pain when doing refactoring, when
trying to understand the structure of the model we describe in DSL, it
doesn't allow to do validation easily and so on.


 So what we have in DB we've called model so far and we've called just
 dsl the dictionary structure coming from DSL. So if we got a part of the
 structure related to a task we would call it dsl_task.

 So what Nikolay is doing now is he's reworking the approach how we work
 with DSL. Now we assume that after we parsed a workbook DSL we get some
 model. So that we don't use dsl in the code anywhere this model
 describes basically the structure of what we have in DSL and that would
 allow to address the problems I mentioned above (hardcoded strings are
 replaced with access methods, we clearly see the structure of what we're
 working with, we can easily validate it and so on). So when we need to
 access some DSL properties we would need to get workbook DSL from DB, build
 this model out of it and continue to work with it.

 Long story short, this model parsed from DSL is not the model we store in
 DB but they're both called model which may be confusing. For me this
 non-DB model more looks like domain model or something like this. So the
 question I would ask ourselves here:

- Is the approach itself reasonable?
- Do we have better ideas on how to work with DSL? A good mental
exercise here would be to imagine that we have more than one DSL, not only
YAML but say XML. How would it change the picture?
- How can we clearly distinguish between these two models so that it
wouldn't be confusing?
- Do we have a better naming in mind?


 Thanks.

 Renat Akhmerov
 @ Mirantis Inc.



 On 05 Mar 2014, at 08:56, Manas Kelshikar ma...@stackstorm.com wrote:

 Since the renaming is for types in mistral.model.*. I am thinking we
 suffix with Spec e.g.

 TaskObject - TaskSpec
 ActionObject - ActionSpec and so on.

 The Spec suggest that it is a specification of the final object that
 ends up in the DB and not the actual object. Multiple actual objects can be
 derived from these Spec objects which fits well with the current paradigm.
 Thoughts?


 On Mon, Mar 3, 2014 at 9:43 PM, Manas Kelshikar ma...@stackstorm.comwrote:

 Hi Nikolay -

 Is your concern that mistral.db.sqlalchemy.models.* and mistral.model.*
 will lead to confusion or something else?

 IMHO as per your change model seems like the appropriate usage while what
 is stored in the DB is also a model. If we pick appropriate names to
 distinguish between nature of the objects we should be able to avoid any
 confusion and whether or not model appears in the module name should not
 matter much.

 Thanks,
 Manas


 On Mon, Mar 3, 2014 at 8:43 AM, Nikolay Makhotkin 
 nmakhot...@mirantis.com wrote:

 Hi, team!

 Please look at the commit .

 Module 'mistral/model' now is responsible for object model
 representation which is used for accessing properties of actions, tasks etc.

 We have a name problem - looks like we should rename module
 'mistral/model' since we have DB models

Re: [openstack-dev] [Mistral] std:repeat action

2014-03-04 Thread Nikolay Makhotkin
Ok, Manas, I'll comment it very soon :)


On Tue, Mar 4, 2014 at 6:47 AM, Manas Kelshikar ma...@stackstorm.comwrote:

 Ping!

 @Nikolay - Can you take a look at the etherpad discussion and provide
 comments. I am going to start working on option (I) as that is the one
 which seems to make most sense. Thoughts?


 On Thu, Feb 27, 2014 at 8:24 AM, Renat Akhmerov rakhme...@mirantis.comwrote:

 Thanks Manas!

 This is one of the important things we need to get done within the next
 couple of weeks. Since it's going to affect engine I think we need to wait
 for a couple of days with the implementation till we merge the changes that
 are being worked on and that also affect engine significantly.

 Team, please research carefully this etherpad and leave your comments.
 It's a pretty tricky thing and we need to figure out the best strategy how
 to approach this kind of things. We're going to have more problems similar
 to this one.

 Renat Akhmerov
 @ Mirantis Inc.



 On 25 Feb 2014, at 10:07, manas kelshikar mana...@gmail.com wrote:

 Hi everyone,

 I have put down my thoughts about the standard repeat action blueprint.

 https://blueprints.launchpad.net/mistral/+spec/mistral-std-repeat-action

 I have added link to an etherpad document which explore a few
 alternatives of the approach. I have explored details of how the std:repeat
 action should behave as defined in the blueprint. Further there are some
 thoughts on how it could be designed to remove ambiguity in the chaining.

 Please take a look.

 Thanks,
 Manas
 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




-- 
Best Regards,
Nikolay
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


[openstack-dev] [Mistral] DSL model vs. DB model, renaming

2014-03-03 Thread Nikolay Makhotkin
Hi, team!

Please look at the commit .

Module 'mistral/model' now is responsible for object model representation
which is used for accessing properties of actions, tasks etc.

We have a name problem - looks like we should rename module 'mistral/model'
since we have DB models and they are absolutely different.


Thoughts?


Best Regards,
Nikolay
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Mistral] Defining term DSL

2014-02-28 Thread Nikolay Makhotkin
Yes, I also think this changes more refer to model than DSL


On Fri, Feb 28, 2014 at 1:41 PM, Renat Akhmerov rakhme...@mirantis.comwrote:

 Yes. Guys, thanks for your feedback. I had a conversation with Dmitri
 today and realized you guys are right here. We should think about building
 basically a domain model which the system operates with and once we built
 it we should forget that we have some DSL or whatever we used to describe
 this model (could be other language, for example). Our initial intention
 actually was different but anyway what you're saying is valid. Looks like
 Nikolay agrees with me too and he's now reworking this commit. Coming up
 soon.

 Renat Akhmerov
 @ Mirantis Inc.



 On 27 Feb 2014, at 23:36, Manas Kelshikar ma...@stackstorm.com wrote:

 I looked at the review prior to looking at the discussion and even I was
 confused by names like DSL*. The way I see it DSL is largely syntatic sugar
 and therefore it will be good to have a clear separation between DSL and
 model. The fact that something is defined in a DSL is irrelevant once it
 crosses mistral API border in effect within mistral itself DSLTask,
 DSLAction etc are simply description objects and how they were defined does
 not matter to mistral implementation.

 Each description object being a recipe to eventually execute a task. We
 therefore already see these two manifestations in current code i.e.
 DSLTask(per Nikolay's change) and Task (
 https://github.com/stackforge/mistral/blob/master/mistral/api/controllers/v1/task.py#L30
 ).

 To me it seems like we only need to agree upon names. Here are my
 suggestions -

 i)
 DSLTask - Task
 Task - TaskInstance
 (Similarly for workflow, action etc.)

 OR

 ii)
 DSLTask - TaskSpec
 Task - Task
 (Similarly for workflow, action etc.)



 On Wed, Feb 26, 2014 at 9:31 PM, Renat Akhmerov rakhme...@mirantis.comwrote:


 On 26 Feb 2014, at 22:54, Dmitri Zimine d...@stackstorm.com wrote:

 Based on the terminology from [1], it's not part of the model, but the
 language that describes the model in the file.


 Sorry, I'm having a hard time trying to understand this phrase :) What do
 you mean by model here? And why should DSL be a part of the model?

 And theoretically this may be not the only language to express the
 workflow.


 Sure, from that perspective, for example, JVM has many DSLs: Java,
 Scala, Groovy etc.

 Once the file is parsed, we operate on model, not on the language.


 How does it influence using term DSL? DSL is, in fact, a user interface.
 Model is something we build inside a system to process DSL in a more
 convenient way.


 I am afraid we are breaking an abstraction when begin to call things
 DSLWorkbook or DSLWorkflow. What is the difference between Workbook and
 DSLWorkbook, and how DSL is relevant here?


 Prefix DSL tells that this exactly matches the structure of an object
 declared with using DSL. But, for example, a workbook in a database may
 have (and it has) a different structure better suitable for storing it in a
 relational model.
 So I'm not sure what you mean by saying we are breaking an abstraction
 here. What abstraction?

 [1] https://wiki.openstack.org/wiki/Mistral,




 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




-- 
Best Regards,
Nikolay
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Mistral] Understanding parameters for tasks and actions

2014-02-26 Thread Nikolay Makhotkin
Hi, Renat!



*Suggestions*
 *1*. Define input and output at task level like this:
  createVM:
input:
  image_id: $.image_id
output: vm_id
 Where output: vm_id is basically a replacement for store-as: vm_id at
 action level, i.e. it's a hint to Mistral to store the output of this task
 under vm_id key in execution context. Again, the idea is to define task
 and action responsibilities more strictly:

- *Task is a high-level workflow building block which defines workflow
logical step and how it modifies workflow data. Task doesn't contain
technical details on how it's implemented.*


- *Action is an implementor of the workflow logical step defined by a
task. Action defines specific algorithm of how task is implemented.*


 *2*. User parameters only for actions to specify their additional
 properties influencing their nature (like method for HTTP actions).


Just clarify your thoughts:
 - All static keys and parameters should be in actions
 - All dynamic keys and parameters should be in tasks block
 - We may use our context to define some parameters in action or task

And, yes, I think it is a good idea to differentiate this. It is become
easier
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Mistral] Understanding parameters for tasks and actions

2014-02-26 Thread Nikolay Makhotkin
Timur, '$' here is mean referrence to context at stage which it has been
given to.

'$.image_id' takes 'image_id' from current workflow execution context
variable


On Wed, Feb 26, 2014 at 12:30 PM, Nikolay Makhotkin nmakhot...@mirantis.com
 wrote:

 Hi, Renat!



 *Suggestions*
 *1*. Define input and output at task level like this:
  createVM:
input:
  image_id: $.image_id
output: vm_id
 Where output: vm_id is basically a replacement for store-as: vm_id at
 action level, i.e. it's a hint to Mistral to store the output of this task
 under vm_id key in execution context. Again, the idea is to define task
 and action responsibilities more strictly:

- *Task is a high-level workflow building block which defines
workflow logical step and how it modifies workflow data. Task doesn't
contain technical details on how it's implemented.*


- *Action is an implementor of the workflow logical step defined by a
task. Action defines specific algorithm of how task is implemented.*


 *2*. User parameters only for actions to specify their additional
 properties influencing their nature (like method for HTTP actions).


 Just clarify your thoughts:
  - All static keys and parameters should be in actions
  - All dynamic keys and parameters should be in tasks block
  - We may use our context to define some parameters in action or task

 And, yes, I think it is a good idea to differentiate this. It is become
 easier




-- 
Best Regards,
Nikolay
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Mistral] Renaming action types

2014-02-26 Thread Nikolay Makhotkin
Agree, I don't see any thing that makes sense with words 'REST' and 'API'
too.


On Wed, Feb 26, 2014 at 11:38 AM, Renat Akhmerov rakhme...@mirantis.comwrote:

 Folks,

 I'm proposing to rename these two action types REST_API and
 MISTRAL_REST_API to HTTP and MISTRAL_HTTP. Words REST and API don't
 look correct to me, if you look at

 Services:
   Nova:
 type: REST_API
 parameters:
   baseUrl: {$.novaURL}
 actions:
   createVM:
 parameters:
   url: /servers/{$.vm_id}
   method: POST

 There's no information about REST or API here. It's just a spec how to
 form an HTTP request.

 Thoughts?

 Renat Akhmerov
 @ Mirantis Inc.




 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




-- 
Best Regards,
Nikolay
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


[openstack-dev] [Mistral] Defining term DSL

2014-02-26 Thread Nikolay Makhotkin
Due to the comment to https://review.openstack.org/#/c/75888/1 there is a
quiestion:

Do we use term DSL or something else?
I think the word 'DSL' is more fit thing that we call 'workbook
definition', some text describing workflows, services, tasks and actions.
And processing module for this also has name 'dsl'.

Thoughts? Dmitri?

Nikolay,
Mirantis Inc.
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Mistral] Porting executor and engine to oslo.messaging

2014-02-25 Thread Nikolay Makhotkin
Looks good. Thanks, Winson!

Renat, What do you think?


On Wed, Feb 26, 2014 at 10:00 AM, W Chan m4d.co...@gmail.com wrote:

 The following link is the google doc of the proposed engine/executor
 message flow architecture.
 https://drive.google.com/file/d/0B4TqA9lkW12PZ2dJVFRsS0pGdEU/edit?usp=sharing

 The diagram on the right is the scalable engine where one or more engine
 sends requests over a transport to one or more executors.  The executor
 client, transport, and executor server follows the RPC client/server
 design 
 patternhttps://github.com/openstack/oslo.messaging/tree/master/oslo/messaging/rpcin
  oslo.messaging.

 The diagram represents the local engine.  In reality, it's following the
 same RPC client/server design pattern.  The only difference is that it'll
 be configured to use a 
 fakehttps://github.com/openstack/oslo.messaging/blob/master/oslo/messaging/_drivers/impl_fake.pyRPC
  backend driver.  The fake driver uses in process
 queues http://docs.python.org/2/library/queue.html#module-Queue shared
 between a pair of engine and executor.

 The following are the stepwise changes I will make.
 1) Keep the local and scalable engine structure intact.  Create the
 Executor Client at ./mistral/engine/scalable/executor/client.py.  Create
 the Executor Server at ./mistral/engine/scalable/executor/service.py and
 implement the task operations under
 ./mistral/engine/scalable/executor/executor.py.  Delete
 ./mistral/engine/scalable/executor/executor.py.  Modify the launcher
 ./mistral/cmd/task_executor.py.  Modify ./mistral/engine/scalable/engine.py
 to use the Executor Client instead of sending the message directly to
 rabbit via pika.  The sum of this is the atomic change that keeps existing
 structure and without breaking the code.
 2) Remove the local engine.
 https://blueprints.launchpad.net/mistral/+spec/mistral-inproc-executor
 3) Implement versioning for the engine.
 https://blueprints.launchpad.net/mistral/+spec/mistral-engine-versioning
 4) Port abstract engine to use oslo.messaging and implement the engine
 client, engine server, and modify the API layer to consume the engine
 client.
 https://blueprints.launchpad.net/mistral/+spec/mistral-engine-standalone-process
 .

 Winson


 On Mon, Feb 24, 2014 at 8:07 PM, Renat Akhmerov rakhme...@mirantis.comwrote:


 On 25 Feb 2014, at 02:21, W Chan m4d.co...@gmail.com wrote:

 Renat,

 Regarding your comments on change https://review.openstack.org/#/c/75609/,
 I don't think the port to oslo.messaging is just a swap from pika to
 oslo.messaging.  OpenStack services as I understand is usually implemented
 as an RPC client/server over a messaging transport.  Sync vs async calls
 are done via the RPC client call and cast respectively.  The messaging
 transport is abstracted and concrete implementation is done via
 drivers/plugins.  So the architecture of the executor if ported to
 oslo.messaging needs to include a client, a server, and a transport.  The
 consumer (in this case the mistral engine) instantiates an instance of the
 client for the executor, makes the method call to handle task, the client
 then sends the request over the transport to the server.  The server picks
 up the request from the exchange and processes the request.  If cast
 (async), the client side returns immediately.  If call (sync), the client
 side waits for a response from the server over a reply_q (a unique queue
 for the session in the transport).  Also, oslo.messaging allows versioning
 in the message. Major version change indicates API contract changes.  Minor
 version indicates backend changes but with API compatibility.


 My main concern about this patch is not related with messaging
 infrastructure. I believe you know better than me how it should look like.
 I'm mostly concerned with the way of making changes you chose. From my
 perspective, it's much better to make atomic changes where every changes
 doesn't affect too much in existing architecture. So the first step could
 be to change pika to oslo.messaging with minimal structural changes without
 introducing versioning (could be just TODO comment saying that the
 framework allows it and we may want to use it in the future, to be decide),
 without getting rid of the current engine structure (local, scalable). Some
 of the things in the file structure and architecture came from the
 decisions made by many people and we need to be careful about changing them.


 So, where I'm headed with this change...  I'm implementing the basic
 structure/scaffolding for the new executor service using oslo.messaging
 (default transport with rabbit).  Since the whole change will take a few
 rounds, I don't want to disrupt any changes that the team is making at the
 moment and so I'm building the structure separately.  I'm also adding
 versioning (v1) in the module structure to anticipate any versioning
 changes in the future.   I expect the change request will lead to some
 discussion as we are doing here.  I will migrate the 

Re: [openstack-dev] [Mistral] Notes on action YAML declaration and naming

2014-02-14 Thread Nikolay Makhotkin
Current DSL snippet:
actions:
   my-action
  parameters:
  foo: bar
  response: # just agreed to change to 'results'
  select: '$.server_id'
  store_as: v1

'result' sounds better than 'response' and, I think, more fit to action
description.
And I suggest for a new word - 'output'; actually, this block describe how
the output will be taken and stored.

However, I agree that this block should be at action-property level:

actions:
   my-action
  result:
 select: '$.server_id'
 store_as: vm_id
  parameters:
 foo: bar



On Fri, Feb 14, 2014 at 12:36 PM, Renat Akhmerov rakhme...@mirantis.comwrote:


 On 14 Feb 2014, at 15:02, Dmitri Zimine d...@stackstorm.com wrote:

 Current DSL snippet:
 actions:
my-action
   parameters:
   foo: bar
   response: # just agreed to change to 'results'


 Just a note: response indentation here is not correct, it's not a
 parameter called response but rather a property of my-action.

   select: '$.server_id'
   store_as: v1

 In the code, we refer to action.result_helper

 1) Note that *response* is not exactly a parameter. It doesn't doesn't
 refer to data. It's  (query, variable) pairs, that are used to parse the
 results and post data to global context [1]. The terms response, or result,
 do not reflect what is actually happening here. Suggestions? Save? Publish?
 Result Handler?


 For explicitness we can use something like result-handler and initially
 I thought about this option. But I personally love conciseness and I think
 name result would be ok for this section meaning it defines the structure
 of the action result. handler is not 100% precise too because we don't
 actually handle a result here, we define the rules how to get this result.

 I would appreciate to see other suggestions though.

 2) Whichever name we use for this output transformer, shall it be under
 parameters?


 No, what we have in this section is like a return value type for a regular
 method. Parameters define action input.

 3) how do we call action/task parameters? Think 'model' (which reflects in
 yaml,  code, docs, talk, etc.)
input and output? (+1)
in and out? (-1)
request and response? (-1) good for WebServices but not generic enough
parameters and results? (ok)


 Could you please clarify your questions here? Not sure I'm following...

 4) Syntax simplification: can we drop 'parameters' keyword? Anything under
 action is action parameters, unless it's a reserved keyword, which the
 implementation can parse out.

 actions:
my-action
   foo: bar
   task-parameters: # not a keyword, specific to REST_API
   flavor_id:
   image_id:
   publish:  # keyword
   select: '$.server_id'
   store_as: v1


 It will create problems like name ambiguity in case we need to have a
 parameter with the same names as keywords (task-parameters and publish
 in your example). My preference would be to leave it explicit.

 Renat


 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




-- 
Best Regards,
Nikolay
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Mistral] Notes on action YAML declaration and naming

2014-02-14 Thread Nikolay Makhotkin
Dmitri, in our concerns under word 'input' we assume a block contains the
info about how the input data will be taken for corresponding task from
initial context. So, it will be a kind of expression (e.g. YAQL).

Renat, am I right?


On Fri, Feb 14, 2014 at 9:51 PM, Dmitri Zimine d...@stackstorm.com wrote:

 I like output, too. But it should go with 'input'
 In summary, there are two alternatives.
 Note that I moved task-parameters under parameters. Ok with this?

 actions:
my-action
   input:
  foo: bar
  task-parameters:
 flavor_id:
 image_id:
   output:
   select: '$.server_id'
   store_as: v1

 this maps to action(input, *output)

 actions:
my-action
   parameters:
  foo: bar
  task-parameters:
 flavor_id:
 image_id:
   result:
   select: '$.server_id'
   store_as: v1

 this maps to result=action(parameters)


 On Feb 14, 2014, at 8:40 AM, Renat Akhmerov rakhme...@mirantis.com
 wrote:

 output looks nice!


 Renat Akhmerov
 @ Mirantis Inc.

 On 14 Feb 2014, at 20:26, Nikolay Makhotkin nmakhot...@mirantis.com
 wrote:

 Current DSL snippet:
 actions:
my-action
   parameters:
   foo: bar
   response: # just agreed to change to 'results'
   select: '$.server_id'
   store_as: v1

 'result' sounds better than 'response' and, I think, more fit to action
 description.
 And I suggest for a new word - 'output'; actually, this block describe how
 the output will be taken and stored.

 However, I agree that this block should be at action-property level:

 actions:
my-action
   result:
  select: '$.server_id'
  store_as: vm_id
   parameters:
  foo: bar



 On Fri, Feb 14, 2014 at 12:36 PM, Renat Akhmerov 
 rakhme...@mirantis.comwrote:


 On 14 Feb 2014, at 15:02, Dmitri Zimine d...@stackstorm.com wrote:

 Current DSL snippet:
 actions:
my-action
   parameters:
   foo: bar
   response: # just agreed to change to 'results'


 Just a note: response indentation here is not correct, it's not a
 parameter called response but rather a property of my-action.

   select: '$.server_id'
   store_as: v1

 In the code, we refer to action.result_helper

 1) Note that *response* is not exactly a parameter. It doesn't doesn't
 refer to data. It's  (query, variable) pairs, that are used to parse the
 results and post data to global context [1]. The terms response, or result,
 do not reflect what is actually happening here. Suggestions? Save? Publish?
 Result Handler?


 For explicitness we can use something like result-handler and initially
 I thought about this option. But I personally love conciseness and I think
 name result would be ok for this section meaning it defines the structure
 of the action result. handler is not 100% precise too because we don't
 actually handle a result here, we define the rules how to get this result.

 I would appreciate to see other suggestions though.

 2) Whichever name we use for this output transformer, shall it be under
 parameters?


 No, what we have in this section is like a return value type for a
 regular method. Parameters define action input.

 3) how do we call action/task parameters? Think 'model' (which reflects
 in yaml,  code, docs, talk, etc.)
input and output? (+1)
in and out? (-1)
request and response? (-1) good for WebServices but not generic enough
parameters and results? (ok)


 Could you please clarify your questions here? Not sure I'm following...

 4) Syntax simplification: can we drop 'parameters' keyword? Anything
 under action is action parameters, unless it's a reserved keyword, which
 the implementation can parse out.

 actions:
my-action
   foo: bar
   task-parameters: # not a keyword, specific to REST_API
   flavor_id:
   image_id:
   publish:  # keyword
   select: '$.server_id'
   store_as: v1


 It will create problems like name ambiguity in case we need to have a
 parameter with the same names as keywords (task-parameters and publish
 in your example). My preference would be to leave it explicit.

 Renat


 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




 --
 Best Regards,
 Nikolay
  ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



 ___
 OpenStack-dev mailing list

Re: [openstack-dev] [savanna] What's the recipe to build Oozie-4.0.0.tar.gz?

2013-10-23 Thread Nikolay Makhotkin
Hi, Matthew!

Note, apache does not make oozie builds, and we have to build it manually.

To build oozie.tar.gz you should follow the steps below:

1. Download oozie distribution from apache mirror (e.g.
http://apache-mirror.rbc.ru/pub/apache/oozie/4.0.0)
2. Build includes maven project, so you should have installed maven
3. Download ExtJS library (extJS-2.2) (http://extjs.com/deploy/ext-2.2.zip)
for enabling Oozie web console in build
4. run mkdistro.sh -DskipTests in oozie distribution directory (some tests
are failed so we don't need it to pass)
5. copy this build file (in distro/target/) to newly created hadoop cluster
and unpack
6. copy hadoop jars (including hadoop-core, hadoop-client, hadoop-auth) to
oozie-dir/libext/ directory (you should create it)
7. copy ext-2.2.zip to libext/ directory too
8. run   $ bin/oozie-setup.sh prepare-war -d libext

Then, your oozie package is ready, pack it to tar.gz and deploy on clusters

Similar instruction to build oozie.tar.gz you may find here:
http://oozie.apache.org/docs/4.0.0/DG_QuickStart.html#Building_Oozie


On Wed, Oct 23, 2013 at 2:01 PM, Alexander Ignatov aigna...@mirantis.comwrote:




  Original Message   Subject: [openstack-dev] [savanna]
 What's the recipe to build Oozie-4.0.0.tar.gz?  Date: Tue, 22 Oct 2013
 15:42:49 -0400  From: Matthew Farrellee m...@redhat.comm...@redhat.com  
 Reply-To:
 OpenStack Development Mailing List 
 openstack-dev@lists.openstack.orgopenstack-dev@lists.openstack.org  To:
 OpenStack Development Mailing List 
 openstack-dev@lists.openstack.orgopenstack-dev@lists.openstack.org

 Having diskimage-create.sh is a great addition for the Savanna user
 community. It greatly simplifies the image building process (using DIB
 for those of you not familiar), making it repeatable and giving everyone
 a hope of debugging issues.

 One thing it does is install oozie. It pulls oozie from 
 http://savanna-files.mirantis.com/oozie-4.0.0.tar.gz

 What's the recipe to create oozie-4.0.0.tar.gz?

 Best,


 matt

 ___
 OpenStack-dev mailing 
 listOpenStack-dev@lists.openstack.orghttp://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev






-- 
Best Regards,
Nikolay Makhotkin,
Intern Software Engineer,
Mirantis Inc.
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev