Re: [openstack-dev] [oslo] Asyncio and oslo.messaging

2014-07-14 Thread Jay Pipes
On 07/09/2014 11:39 AM, Clint Byrum wrote: Excerpts from Yuriy Taraday's message of 2014-07-09 03:36:00 -0700: On Tue, Jul 8, 2014 at 11:31 PM, Joshua Harlow harlo...@yahoo-inc.com wrote: I think clints response was likely better than what I can write here, but I'll add-on a few things,

Re: [openstack-dev] [oslo] Asyncio and oslo.messaging

2014-07-14 Thread Mike Bayer
On Jul 14, 2014, at 9:46 AM, Jay Pipes jaypi...@gmail.com wrote: The point of eventlet, I thought, was to hide the low-level stuff so that developers could focus on higher-level (and more productive) abstractions. Introducing asyncio contructs into the higher level code like Nova and

Re: [openstack-dev] [oslo] Asyncio and oslo.messaging

2014-07-12 Thread Yuriy Taraday
On Fri, Jul 11, 2014 at 10:34 PM, Joshua Harlow harlo...@outlook.com wrote: S, how about we can continue this in #openstack-state-management (or #openstack-oslo). Since I think we've all made the point and different viewpoints visible (which was the main intention). Overall, I'd like

Re: [openstack-dev] [oslo] Asyncio and oslo.messaging

2014-07-11 Thread Flavio Percoco
On 07/10/2014 06:46 PM, Mark McLoughlin wrote: On Thu, 2014-07-03 at 16:27 +0100, Mark McLoughlin wrote: Hey This is an attempt to summarize a really useful discussion that Victor, Flavio and I have been having today. At the bottom are some background links - basically what I have open in my

Re: [openstack-dev] [oslo] Asyncio and oslo.messaging

2014-07-11 Thread Victor Stinner
Hi, Le lundi 7 juillet 2014, 19:18:38 Mark McLoughlin a écrit : I'd expect us to add e.g. @asyncio.coroutine def call_async(self, ctxt, method, **kwargs): ... to RPCClient. Perhaps we'd need to add an AsyncRPCClient in a separate module and only add the method there - I don't

Re: [openstack-dev] [oslo] Asyncio and oslo.messaging

2014-07-11 Thread Yuriy Taraday
On Thu, Jul 10, 2014 at 11:51 PM, Outlook harlo...@outlook.com wrote: On Jul 10, 2014, at 3:48 AM, Yuriy Taraday yorik@gmail.com wrote: On Wed, Jul 9, 2014 at 7:39 PM, Clint Byrum cl...@fewbar.com wrote: Excerpts from Yuriy Taraday's message of 2014-07-09 03:36:00 -0700: On Tue, Jul 8,

Re: [openstack-dev] [oslo] Asyncio and oslo.messaging

2014-07-11 Thread Joshua Harlow
S, how about we can continue this in #openstack-state-management (or #openstack-oslo). Since I think we've all made the point and different viewpoints visible (which was the main intention). Overall, I'd like to see asyncio more directly connected into taskflow so we can have the best of

Re: [openstack-dev] [oslo] Asyncio and oslo.messaging

2014-07-10 Thread Yuriy Taraday
On Wed, Jul 9, 2014 at 7:39 PM, Clint Byrum cl...@fewbar.com wrote: Excerpts from Yuriy Taraday's message of 2014-07-09 03:36:00 -0700: On Tue, Jul 8, 2014 at 11:31 PM, Joshua Harlow harlo...@yahoo-inc.com wrote: I think clints response was likely better than what I can write here, but

Re: [openstack-dev] [oslo] Asyncio and oslo.messaging

2014-07-10 Thread Victor Stinner
Le jeudi 10 juillet 2014, 14:48:04 Yuriy Taraday a écrit : I'm not suggesting that taskflow is useless and asyncio is better (apple vs oranges). I'm saying that using coroutines (asyncio) can improve ways we can use taskflow and provide clearer method of developing these flows. This was mostly

Re: [openstack-dev] [oslo] Asyncio and oslo.messaging

2014-07-10 Thread Clint Byrum
Excerpts from Victor Stinner's message of 2014-07-10 05:57:38 -0700: Le jeudi 10 juillet 2014, 14:48:04 Yuriy Taraday a écrit : I'm not suggesting that taskflow is useless and asyncio is better (apple vs oranges). I'm saying that using coroutines (asyncio) can improve ways we can use

Re: [openstack-dev] [oslo] Asyncio and oslo.messaging

2014-07-10 Thread Mark McLoughlin
On Thu, 2014-07-03 at 16:27 +0100, Mark McLoughlin wrote: Hey This is an attempt to summarize a really useful discussion that Victor, Flavio and I have been having today. At the bottom are some background links - basically what I have open in my browser right now thinking through all of

Re: [openstack-dev] [oslo] Asyncio and oslo.messaging

2014-07-10 Thread Mark McLoughlin
On Mon, 2014-07-07 at 12:48 +0200, Nikola Đipanov wrote: When I read all of this stuff and got my head around it (took some time :) ), a glaring drawback of such an approach, and as I mentioned on the spec proposing it [1] is that we would not really doing asyncio, we would just be pretending

Re: [openstack-dev] [oslo] Asyncio and oslo.messaging

2014-07-10 Thread Joshua Harlow
This is not supposed to be from 'Outlook', haha. Using my adjusted mail account so that its don't go to spam on the receiving end due to DMARC/DKIM issues... My fault ;) -Josh On Jul 10, 2014, at 12:51 PM, Outlook harlo...@outlook.com wrote: On Jul 10, 2014, at 3:48 AM, Yuriy Taraday

Re: [openstack-dev] [oslo] Asyncio and oslo.messaging

2014-07-09 Thread Yuriy Taraday
On Tue, Jul 8, 2014 at 11:31 PM, Joshua Harlow harlo...@yahoo-inc.com wrote: I think clints response was likely better than what I can write here, but I'll add-on a few things, How do you write such code using taskflow? @asyncio.coroutine def foo(self): result = yield from

Re: [openstack-dev] [oslo] Asyncio and oslo.messaging

2014-07-09 Thread Clint Byrum
Excerpts from Yuriy Taraday's message of 2014-07-09 03:36:00 -0700: On Tue, Jul 8, 2014 at 11:31 PM, Joshua Harlow harlo...@yahoo-inc.com wrote: I think clints response was likely better than what I can write here, but I'll add-on a few things, How do you write such code using

Re: [openstack-dev] [oslo] Asyncio and oslo.messaging

2014-07-08 Thread Victor Stinner
Le lundi 7 juillet 2014, 19:14:45 Angus Salkeld a écrit : 4) retraining of OpenStack developers/reviews to understand the new event loop. (eventlet has warts, but a lot of devs know about them). Wait, what? Sorry if it was not clear, but the *whole* point of replacing eventlet with asyncio

Re: [openstack-dev] [oslo] Asyncio and oslo.messaging

2014-07-08 Thread Gordon Sim
On 07/07/2014 07:18 PM, Mark McLoughlin wrote: I'd expect us to add e.g. @asyncio.coroutine def call_async(self, ctxt, method, **kwargs): ... to RPCClient. Perhaps we'd need to add an AsyncRPCClient in a separate module and only add the method there - I don't have a good sense of

Re: [openstack-dev] [oslo] Asyncio and oslo.messaging

2014-07-08 Thread Clint Byrum
Excerpts from Victor Stinner's message of 2014-07-08 05:47:36 -0700: Hi Joshua, You asked a lot of questions. I will try to answer. Le lundi 7 juillet 2014, 17:41:34 Joshua Harlow a écrit : * Why focus on a replacement low level execution model integration instead of higher level

Re: [openstack-dev] [oslo] Asyncio and oslo.messaging

2014-07-08 Thread Joshua Harlow
:43 AM To: openstack-dev openstack-dev@lists.openstack.org Subject: Re: [openstack-dev] [oslo] Asyncio and oslo.messaging Excerpts from Victor Stinner's message of 2014-07-08 05:47:36 -0700: Hi Joshua, You asked a lot of questions. I will try to answer. Le lundi 7 juillet 2014, 17:41:34

Re: [openstack-dev] [oslo] Asyncio and oslo.messaging

2014-07-08 Thread Joshua Harlow
I think clints response was likely better than what I can write here, but I'll add-on a few things, How do you write such code using taskflow? @asyncio.coroutine def foo(self): result = yield from some_async_op(...) return do_stuff(result) The idea (at a very high level) is

Re: [openstack-dev] [oslo] Asyncio and oslo.messaging

2014-07-08 Thread Joshua Harlow
This is the part that I really wonder about. Since asyncio isn't just a drop-in replacement for eventlet (which hid the async part under its *black magic*), I very much wonder how the community will respond to this kind of mindset change (along with its new *black magic*). I disagree with you,

Re: [openstack-dev] [oslo] Asyncio and oslo.messaging

2014-07-07 Thread Nikola Đipanov
On 07/03/2014 05:27 PM, Mark McLoughlin wrote: Hey This is an attempt to summarize a really useful discussion that Victor, Flavio and I have been having today. At the bottom are some background links - basically what I have open in my browser right now thinking through all of this. We're

Re: [openstack-dev] [oslo] Asyncio and oslo.messaging

2014-07-07 Thread Victor Stinner
Hi, Le lundi 7 juillet 2014, 12:48:59 Nikola Đipanov a écrit : When I read all of this stuff and got my head around it (took some time :) ), a glaring drawback of such an approach, and as I mentioned on the spec proposing it [1] is that we would not really doing asyncio, we would just be

Re: [openstack-dev] [oslo] Asyncio and oslo.messaging

2014-07-07 Thread Victor Stinner
Le lundi 7 juillet 2014, 11:26:27 Gordon Sim a écrit : When we switch to asyncio's event loop, all of this code needs to be ported to asyncio's explicitly asynchronous approach. We might do: @asyncio.coroutine def foo(self): result = yield from some_async_op(...)

Re: [openstack-dev] [oslo] Asyncio and oslo.messaging

2014-07-07 Thread Gordon Sim
On 07/07/2014 03:12 PM, Victor Stinner wrote: The first step is to patch endpoints to add @trollius.coroutine to the methods, and add yield From(...) on asynchronous tasks. What are the 'endpoints' here? Are these internal to the oslo.messaging library, or external to it? Later we may

Re: [openstack-dev] [oslo] Asyncio and oslo.messaging

2014-07-07 Thread Mike Bayer
On 7/4/14, 4:45 AM, Julien Danjou wrote: On Thu, Jul 03 2014, Mark McLoughlin wrote: We're attempting to take baby-steps towards moving completely from eventlet to asyncio/trollius. The thinking is for Ceilometer to be the first victim. Thumbs up for the plan, that sounds like a good

Re: [openstack-dev] [oslo] Asyncio and oslo.messaging

2014-07-07 Thread Nikola Đipanov
On 07/07/2014 02:58 PM, Victor Stinner wrote: Hi, Le lundi 7 juillet 2014, 12:48:59 Nikola Đipanov a écrit : When I read all of this stuff and got my head around it (took some time :) ), a glaring drawback of such an approach, and as I mentioned on the spec proposing it [1] is that we would

Re: [openstack-dev] [oslo] Asyncio and oslo.messaging

2014-07-07 Thread Joshua Harlow
So I've been thinking how to respond to this email, and here goes (shields up!), First things first; thanks mark and victor for the detailed plan and making it visible to all. It's very nicely put together and the amount of thought put into it is great to see. I always welcome an effort to move

Re: [openstack-dev] [oslo] Asyncio and oslo.messaging

2014-07-07 Thread Angus Salkeld
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 03/07/14 05:30, Mark McLoughlin wrote: Hey This is an attempt to summarize a really useful discussion that Victor, Flavio and I have been having today. At the bottom are some background links - basically what I have open in my browser right

Re: [openstack-dev] [oslo] Asyncio and oslo.messaging

2014-07-07 Thread Mark McLoughlin
On Sun, 2014-07-06 at 09:28 -0400, Eoghan Glynn wrote: This is an attempt to summarize a really useful discussion that Victor, Flavio and I have been having today. At the bottom are some background links - basically what I have open in my browser right now thinking through all of this.

Re: [openstack-dev] [oslo] Asyncio and oslo.messaging

2014-07-07 Thread Mark McLoughlin
On Mon, 2014-07-07 at 15:53 +0100, Gordon Sim wrote: On 07/07/2014 03:12 PM, Victor Stinner wrote: The first step is to patch endpoints to add @trollius.coroutine to the methods, and add yield From(...) on asynchronous tasks. What are the 'endpoints' here? Are these internal to the

Re: [openstack-dev] [oslo] Asyncio and oslo.messaging

2014-07-07 Thread Chris Behrens
On Jul 7, 2014, at 11:11 AM, Angus Salkeld angus.salk...@rackspace.com wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 03/07/14 05:30, Mark McLoughlin wrote: Hey This is an attempt to summarize a really useful discussion that Victor, Flavio and I have been having today. At the

Re: [openstack-dev] [oslo] Asyncio and oslo.messaging

2014-07-07 Thread Mark McLoughlin
On Mon, 2014-07-07 at 18:11 +, Angus Salkeld wrote: On 03/07/14 05:30, Mark McLoughlin wrote: Hey This is an attempt to summarize a really useful discussion that Victor, Flavio and I have been having today. At the bottom are some background links - basically what I have open in my

Re: [openstack-dev] [oslo] Asyncio and oslo.messaging

2014-07-07 Thread Joshua Harlow
Glynn egl...@redhat.com Reply-To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org Date: Sunday, July 6, 2014 at 6:28 AM To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org Subject: Re: [openstack-dev] [oslo

Re: [openstack-dev] [oslo] Asyncio and oslo.messaging

2014-07-07 Thread Angus Salkeld
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 07/07/14 08:28, Mark McLoughlin wrote: On Mon, 2014-07-07 at 18:11 +, Angus Salkeld wrote: On 03/07/14 05:30, Mark McLoughlin wrote: Hey This is an attempt to summarize a really useful discussion that Victor, Flavio and I have been having

Re: [openstack-dev] [oslo] Asyncio and oslo.messaging

2014-07-07 Thread Clint Byrum
Excerpts from Joshua Harlow's message of 2014-07-07 10:41:34 -0700: So I've been thinking how to respond to this email, and here goes (shields up!), First things first; thanks mark and victor for the detailed plan and making it visible to all. It's very nicely put together and the amount of

Re: [openstack-dev] [oslo] Asyncio and oslo.messaging

2014-07-06 Thread Eoghan Glynn
This is an attempt to summarize a really useful discussion that Victor, Flavio and I have been having today. At the bottom are some background links - basically what I have open in my browser right now thinking through all of this. Thanks for the detailed summary, it puts a more flesh on

Re: [openstack-dev] [oslo] Asyncio and oslo.messaging

2014-07-04 Thread Julien Danjou
On Thu, Jul 03 2014, Mark McLoughlin wrote: We're attempting to take baby-steps towards moving completely from eventlet to asyncio/trollius. The thinking is for Ceilometer to be the first victim. Thumbs up for the plan, that sounds like a good approach from what I got. I just think there's a

Re: [openstack-dev] [oslo] Asyncio and oslo.messaging

2014-07-04 Thread victor stinner
Hi, I promise a status of my work on Trollius and greenio to Mark, but it's not easy to summarize it because there are still a few pending patches to implement the final greenio executor. There are different parts: asyncio, Trollius, greenio, Olso Messaging. The design of the asyncio is the

Re: [openstack-dev] [oslo] Asyncio and oslo.messaging

2014-07-03 Thread Doug Hellmann
On Thu, Jul 3, 2014 at 11:27 AM, Mark McLoughlin mar...@redhat.com wrote: Hey This is an attempt to summarize a really useful discussion that Victor, Flavio and I have been having today. At the bottom are some background links - basically what I have open in my browser right now thinking