Re: [openstack-dev] [oslo] Add a new aiogreen executor for Oslo Messaging

2014-11-24 Thread Joshua Harlow
Doug Hellmann wrote: On Nov 24, 2014, at 12:57 PM, Mike Bayer wrote: On Nov 24, 2014, at 12:40 PM, Doug Hellmann wrote: This is a good point. I’m not sure we can say “we’ll only use explicit/implicit async in certain cases" because most of our apps actually mix the cases. We have WSGI app

Re: [openstack-dev] [oslo] Add a new aiogreen executor for Oslo Messaging

2014-11-24 Thread Doug Hellmann
On Nov 24, 2014, at 12:57 PM, Mike Bayer wrote: > >> On Nov 24, 2014, at 12:40 PM, Doug Hellmann wrote: >> >> >> This is a good point. I’m not sure we can say “we’ll only use >> explicit/implicit async in certain cases" because most of our apps actually >> mix the cases. We have WSGI apps

Re: [openstack-dev] [oslo] Add a new aiogreen executor for Oslo Messaging

2014-11-24 Thread Mike Bayer
> On Nov 24, 2014, at 12:40 PM, Doug Hellmann wrote: > > > This is a good point. I’m not sure we can say “we’ll only use > explicit/implicit async in certain cases" because most of our apps actually > mix the cases. We have WSGI apps that send RPC messages and we have other > apps that recei

Re: [openstack-dev] [oslo] Add a new aiogreen executor for Oslo Messaging

2014-11-24 Thread Doug Hellmann
On Nov 24, 2014, at 11:30 AM, Jay Pipes wrote: > On 11/24/2014 10:43 AM, Mike Bayer wrote: >>> On Nov 24, 2014, at 9:23 AM, Adam Young wrote: >>> For pieces such as the Nova compute that talk almost exclusively on >>> the Queue, we should work to remove Monkey patching and use a clear >>> progr

Re: [openstack-dev] [oslo] Add a new aiogreen executor for Oslo Messaging

2014-11-24 Thread Jay Pipes
On 11/24/2014 10:43 AM, Mike Bayer wrote: On Nov 24, 2014, at 9:23 AM, Adam Young wrote: For pieces such as the Nova compute that talk almost exclusively on the Queue, we should work to remove Monkey patching and use a clear programming model. If we can do that within the context of Eventlet, g

Re: [openstack-dev] [oslo] Add a new aiogreen executor for Oslo Messaging

2014-11-24 Thread Mike Bayer
> On Nov 24, 2014, at 9:23 AM, Adam Young wrote: > > > > For pieces such as the Nova compute that talk almost exclusively on the > Queue, we should work to remove Monkey patching and use a clear programming > model. If we can do that within the context of Eventlet, great. If we need > to

Re: [openstack-dev] [oslo] Add a new aiogreen executor for Oslo Messaging

2014-11-24 Thread Mike Bayer
> On Nov 23, 2014, at 9:24 PM, Donald Stufft wrote: > > > There’s a long history of implicit context switches causing buggy software > that breaks. As far as I can tell the only downsides to explicit context > switches that don’t stem from an inferior interpreter seem to be “some > particula

Re: [openstack-dev] [oslo] Add a new aiogreen executor for Oslo Messaging

2014-11-24 Thread Adam Young
On 11/23/2014 06:13 PM, Robert Collins wrote: On WSGI - if we're in an asyncio world, I don't think WSGI has any relevance today - it has no async programming model. While is has incremental apis and supports generators, thats not close enough to the same thing: so we're going to have to port our

Re: [openstack-dev] [oslo] Add a new aiogreen executor for Oslo Messaging

2014-11-23 Thread Donald Stufft
> On Nov 23, 2014, at 9:09 PM, Mike Bayer wrote: > > >> On Nov 23, 2014, at 8:23 PM, Donald Stufft wrote: >> >> I don’t really take performance issues that seriously for CPython. If you >> care about performance you should be using PyPy. I like that argument though >> because the same argum

Re: [openstack-dev] [oslo] Add a new aiogreen executor for Oslo Messaging

2014-11-23 Thread Mike Bayer
> On Nov 23, 2014, at 8:23 PM, Donald Stufft wrote: > > I don’t really take performance issues that seriously for CPython. If you > care about performance you should be using PyPy. I like that argument though > because the same argument is used against the GCs which you like to use as an > ex

Re: [openstack-dev] [oslo] Add a new aiogreen executor for Oslo Messaging

2014-11-23 Thread Donald Stufft
> On Nov 23, 2014, at 7:55 PM, Mike Bayer wrote: > >> >> On Nov 23, 2014, at 7:30 PM, Donald Stufft wrote: >> >> >>> On Nov 23, 2014, at 7:21 PM, Mike Bayer wrote: >>> >>> Given that, I’ve yet to understand why a system that implicitly defers CPU >>> use when a routine encounters IO, defe

Re: [openstack-dev] [oslo] Add a new aiogreen executor for Oslo Messaging

2014-11-23 Thread Mike Bayer
> On Nov 23, 2014, at 7:30 PM, Donald Stufft wrote: > > >> On Nov 23, 2014, at 7:21 PM, Mike Bayer wrote: >> >> Given that, I’ve yet to understand why a system that implicitly defers CPU >> use when a routine encounters IO, deferring to other routines, is relegated >> to the realm of “magic

Re: [openstack-dev] [oslo] Add a new aiogreen executor for Oslo Messaging

2014-11-23 Thread Donald Stufft
> On Nov 23, 2014, at 7:29 PM, Mike Bayer wrote: > >> >> Glyph wrote a good post that mirrors my opinions on implicit vs explicit >> here: https://glyph.twistedmatrix.com/2014/02/unyielding.html. > > this is the post that most makes me think about the garbage collector > analogy, re: “gevent

Re: [openstack-dev] [oslo] Add a new aiogreen executor for Oslo Messaging

2014-11-23 Thread Donald Stufft
> On Nov 23, 2014, at 7:21 PM, Mike Bayer wrote: > > Given that, I’ve yet to understand why a system that implicitly defers CPU > use when a routine encounters IO, deferring to other routines, is relegated > to the realm of “magic”. Is Python reference counting and garbage > collection “mag

Re: [openstack-dev] [oslo] Add a new aiogreen executor for Oslo Messaging

2014-11-23 Thread Mike Bayer
> On Nov 23, 2014, at 6:35 PM, Donald Stufft wrote: > > > For whatever it’s worth, I find explicit async io to be _way_ easier to > understand for the same reason I find threaded code to be a rats nest. web applications aren’t explicitly “threaded”. You get a request, load some data, manipu

Re: [openstack-dev] [oslo] Add a new aiogreen executor for Oslo Messaging

2014-11-23 Thread Mike Bayer
> On Nov 23, 2014, at 6:13 PM, Robert Collins wrote: > > > So - the technical bits of the plan sound fine. > > On WSGI - if we're in an asyncio world, *looks around*, we are? when did that happen?Assuming we’re talking explicit async. Rewriting all our code as verbose, “inside out

Re: [openstack-dev] [oslo] Add a new aiogreen executor for Oslo Messaging

2014-11-23 Thread Robert Collins
On 24 November 2014 at 12:30, Monty Taylor wrote: > I'm not going to comment on the pros and cons - I think we all know I'm > a fan of threads. But I have been around a while, so - for those who > haven't been: FWIW we have *threads* today as a programming model. The implementation is green, but

Re: [openstack-dev] [oslo] Add a new aiogreen executor for Oslo Messaging

2014-11-23 Thread Robert Collins
On 24 November 2014 at 12:35, Donald Stufft wrote: > > For whatever it’s worth, I find explicit async io to be _way_ easier to > understand for the same reason I find threaded code to be a rats nest. > > The co-routine style of asyncio (or Twisted’s inlineCallbacks) solves > almost all of the prob

Re: [openstack-dev] [oslo] Add a new aiogreen executor for Oslo Messaging

2014-11-23 Thread Donald Stufft
> On Nov 23, 2014, at 6:30 PM, Monty Taylor wrote: > > On 11/23/2014 06:13 PM, Robert Collins wrote: >> On 24 November 2014 at 11:01, victor stinner >> wrote: >>> Hi, >>> >>> I'm happy to announce you that I just finished the last piece of the puzzle >>> to add support for trollius coroutines

Re: [openstack-dev] [oslo] Add a new aiogreen executor for Oslo Messaging

2014-11-23 Thread Monty Taylor
On 11/23/2014 06:13 PM, Robert Collins wrote: > On 24 November 2014 at 11:01, victor stinner > wrote: >> Hi, >> >> I'm happy to announce you that I just finished the last piece of the puzzle >> to add support for trollius coroutines in Oslo Messaging! See my two changes: >> >> * Add a new aiogree

Re: [openstack-dev] [oslo] Add a new aiogreen executor for Oslo Messaging

2014-11-23 Thread Robert Collins
On 24 November 2014 at 11:01, victor stinner wrote: > Hi, > > I'm happy to announce you that I just finished the last piece of the puzzle > to add support for trollius coroutines in Oslo Messaging! See my two changes: > > * Add a new aiogreen executor: > https://review.openstack.org/#/c/136653/