On Feb 7, 2014, at 2:59 PM, Victor Stinner wrote:
> I don't see why external libraries should be modified. Only the few libraries
> sending HTTP queries and requests to the database should handle asyncio.
> Dummy
> example: the iso8601 module used to parse time doesn't need to be aware of
>
Le vendredi 7 février 2014, 09:37:27 Vishvananda Ishaya a écrit :
> To be clear, since many people weren’t around in ye olde days, nova started
> using tornado. We exchanged tornado for twisted, and finally moved to
> eventlet. People have suggested gevent and threads in the past, and now
> asyncio
Le jeudi 6 février 2014, 22:32:26 Zane Bitter a écrit :
> [BTW am I right in thinking that in trollius you _have_ to drive your
> co-routines from a top-level Task? That's not the case in Heat (although
> we do anyway), or IIUC in asyncio - I was expecting e.g. the exception
> handling for Return t
Le mardi 4 février 2014, 12:53:24 Kevin Conway a écrit :
> One of the great benefits of using a green thread abstraction, like
> eventlet or gevent, is that it lets you write normal Python code and slap
> your concurrency management over the top. It even lets us program in
> familiar models by pret
On Feb 7, 2014, at 10:25 AM, Yuriy Taraday wrote:
> Hello, Vish!
>
> I hope you can provide some historical data.
>
> On Fri, Feb 7, 2014 at 9:37 PM, Vishvananda Ishaya
> wrote:
> To be clear, since many people weren’t around in ye olde days, nova started
> using tornado. We exchanged torna
Hello, Vish!
I hope you can provide some historical data.
On Fri, Feb 7, 2014 at 9:37 PM, Vishvananda Ishaya wrote:
>
> To be clear, since many people weren’t around in ye olde days, nova
> started using tornado. We exchanged tornado for twisted, and finally moved
> to eventlet. People have sugge
I want to address some of Chuck’s post, but I think we should come up with a
list of requirements. Replies to Chuck inline, and then some requirements below:
On Feb 7, 2014, at 8:38 AM, Chuck Thier wrote:
> Concurrency is hard, let's blame the tools!
>
> Any lib that we use in python is going
On Feb 7, 2014, at 8:21 AM, Jesse Noller wrote:
>
> On Feb 7, 2014, at 1:51 AM, Chris Behrens wrote:
>
>>
>> On Feb 6, 2014, at 11:07 PM, Joshua Harlow wrote:
>>
>>> +1
>>>
>>> To give an example as to why eventlet implicit monkey patch the world isn't
>>> especially great (although it's
On Feb 7, 2014, at 11:12 AM, Chris Behrens wrote:
>
> On Feb 7, 2014, at 8:21 AM, Jesse Noller wrote:
>
>> It seems that baking concurrency models into the individual clients /
>> services adds some opinionated choices that may not scale, or fit the needs
>> of a large-scale deployment. Thi
On Feb 7, 2014, at 8:21 AM, Jesse Noller wrote:
> It seems that baking concurrency models into the individual clients /
> services adds some opinionated choices that may not scale, or fit the needs
> of a large-scale deployment. This is one of the things looking at the client
> tools I’ve not
Concurrency is hard, let's blame the tools!
Any lib that we use in python is going to have a set of trade-offs.
Looking at a couple of the options on the table:
1. Threads: Great! code doesn't have to change too much, but now that
code *will* be preempted at any time, so now we have to worry a
On Feb 7, 2014, at 1:51 AM, Chris Behrens wrote:
>
> On Feb 6, 2014, at 11:07 PM, Joshua Harlow wrote:
>
>> +1
>>
>> To give an example as to why eventlet implicit monkey patch the world isn't
>> especially great (although it's what we are currently using throughout
>> openstack).
>>
>> T
On Feb 6, 2014, at 11:07 PM, Joshua Harlow wrote:
> +1
>
> To give an example as to why eventlet implicit monkey patch the world isn't
> especially great (although it's what we are currently using throughout
> openstack).
>
> The way I think about how it works is to think about what librarie
+1
To give an example as to why eventlet implicit monkey patch the world isn't
especially great (although it's what we are currently using throughout
openstack).
The way I think about how it works is to think about what libraries that a
single piece of code calls and how it is very hard to pre
ge questions)"
Date: Thursday, February 6, 2014 at 1:55 AM
To: "OpenStack Development Mailing List (not for usage questions)"
Subject: Re: [openstack-dev] Asynchrounous programming: replace eventlet
with asyncio
Sean Dague wrote:
First, very cool!
Thanks.
This is very promising work.
On 04/02/14 13:53, Kevin Conway wrote:
On 2/4/14 12:07 PM, "victor stinner" wrote:
>The purpose of replacing eventlet with asyncio is to get a well defined
>control flow, no more surprising task switching at random points.
I disagree with this. Eventlet and gevent yield the execution context
te workers
-Original Message-
From: Clint Byrum
Reply-To: "OpenStack Development Mailing List (not for usage questions)"
Date: Thursday, February 6, 2014 at 12:46 PM
To: openstack-dev
Subject: Re: [openstack-dev] Asynchrounous programming: replace
eventletwith asyncio
>
Hello, Kevin.
On Fri, Feb 7, 2014 at 12:32 AM, Kevin Conway wrote:
> There's an incredibly valid reason why we use green thread abstractions
> like eventlet and gevent in Python. The CPython implementation is
> inherently single threaded so we need some other form of concurrency to get
> the most
s).
>
> -Josh
>
> From: Yuriy Taraday
>
> Reply-To: "OpenStack Development Mailing List (not for usage questions)"
>
> Date: Thursday, February 6, 2014 at 10:06 AM
>
> To: "OpenStack Development Mailing List (not for usage questions)"
>
> Subje
ling List (not for usage questions)"
>
> Date: Thursday, February 6, 2014 at 1:55 AM
> To: "OpenStack Development Mailing List (not for usage questions)"
>
> Subject: Re: [openstack-dev] Asynchrounous programming: replace eventlet
> with asyncio
>
> >
or usage questions)" <
> openstack-dev@lists.openstack.org>
> Date: Thursday, February 6, 2014 at 10:06 AM
>
> To: "OpenStack Development Mailing List (not for usage questions)" <
> openstack-dev@lists.openstack.org>
> Subject: Re: [openstack-dev] Asynchrounous pr
On Thu, Feb 6, 2014 at 10:34 PM, Joshua Harlow wrote:
> Its a good question, I see openstack as mostly like the following 2
> groups of applications.
>
> Group 1:
>
> API entrypoints using [apache/nginx]+wsgi (nova-api, glance-api…)
>
> In this group we can just let the underlying framework/ap
ment Mailing List (not for usage questions)"
mailto:openstack-dev@lists.openstack.org>>
Subject: Re: [openstack-dev] Asynchrounous programming: replace eventlet with
asyncio
Hello.
On Tue, Feb 4, 2014 at 5:38 PM, victor stinner
mailto:victor.stin...@enovance.com>> wrote:
I would like to repl
iginal Message-
From: victor stinner
Reply-To: "OpenStack Development Mailing List (not for usage questions)"
Date: Thursday, February 6, 2014 at 1:55 AM
To: "OpenStack Development Mailing List (not for usage questions)"
Subject: Re: [openstack-dev] Asynchrounous programming:
Hello.
On Tue, Feb 4, 2014 at 5:38 PM, victor stinner
wrote:
> I would like to replace eventlet with asyncio in OpenStack for the
> asynchronous programming. The new asyncio module has a better design and is
> less "magical". It is now part of python 3.4 arguably becoming the de-facto
> standard
Sean Dague wrote:
> First, very cool!
Thanks.
> This is very promising work. It might be really interesting to figure
> out if there was a smaller project inside of OpenStack that could be
> test ported over to this (even as a stackforge project), and something
> we could run in the gate.
Oslo M
Hi,
Joshua Harlow:
> Any mysql DB drivers (I think the majority of openstack deployments use
> mysql?).
I don't know. Here are some asynchronous clients for MySQL:
https://github.com/PyMySQL/PyMySQL/
https://launchpad.net/myconnpy
https://github.com/hybridlogic/txMySQL
http://chartio.com/blog/20
ictor stinner
Reply-To: "OpenStack Development Mailing List (not for usage questions)"
Date: Wednesday, February 5, 2014 at 3:00 AM
To: "OpenStack Development Mailing List (not for usage questions)"
Subject: Re: [openstack-dev] Asynchrounous programming:
replace eventlet
On 02/05/2014 09:44 PM, victor stinner wrote:
> Hi,
>
> Thierry Carrez wrote:
>>> The problem is that the asyncio module was written for Python 3.3, whereas
>>> OpenStack is not fully Python 3 compatible (yet). To easy the transition I
>>> have ported asyncio on Python 2, it's the new Trollis proj
Hi,
Thierry Carrez wrote:
> > The problem is that the asyncio module was written for Python 3.3, whereas
> > OpenStack is not fully Python 3 compatible (yet). To easy the transition I
> > have ported asyncio on Python 2, it's the new Trollis project which
> > supports Python 2.6-3.4:
> >https:
victor stinner wrote:
> [...]
> The problem is that the asyncio module was written for Python 3.3, whereas
> OpenStack is not fully Python 3 compatible (yet). To easy the transition I
> have ported asyncio on Python 2, it's the new Trollis project which supports
> Python 2.6-3.4:
>https://bi
Hi,
Chris Behrens wrote:
> Interesting thread. I have been working on a side project that is a
> gevent/eventlet replacement [1] that focuses on thread-safety and
> performance. This came about because of an outstanding bug we have with
> eventlet not being Thread safe. (We cannot safely enable th
uestions)"
Date: Tuesday, February 4, 2014 at 12:38 PM
To: "OpenStack Development Mailing List (not for usage questions)"
Subject: Re: [openstack-dev] Asynchrounous programming: replace eventlet
with asyncio
>Kevin Conway wrote:
>> Switching our async IO management from eventlet
Hi,
Interesting thread. I have been working on a side project that is a
gevent/eventlet replacement [1] that focuses on thread-safety and performance.
This came about because of an outstanding bug we have with eventlet not being
Thread safe. (We cannot safely enable thread pooling for DB calls
Kevin Conway wrote:
> Switching our async IO management from eventlet to asyncio would not be a
> trivial task. Tell me when I'm wrong, but it would require that we
> completely change our programming model from typical, function-call based
> programming to use generator-iterators for everything.
On Tue, Feb 04 2014, Kevin Conway wrote:
> I disagree with this. Eventlet and gevent yield the execution context
> anytime an IO call is made or the 'sleep()' function is called explicitly.
> The order in which greenthreads grain execution context is deterministic
> even if not entirely obvious. T
On 2/4/14 12:07 PM, "victor stinner" wrote:
>The purpose of replacing eventlet with asyncio is to get a well defined
>control flow, no more surprising task switching at random points.
I disagree with this. Eventlet and gevent yield the execution context
anytime an IO call is made or the 'sleep(
;
Subject: Re: [openstack-dev] Asynchrounous programming: replace eventlet
withasyncio
>Hi Joshua,
>
>> Why not just create a eventlet "micro" version that underneath uses
>>asyncio?
>> Then code isn't riddled with yield from or Return() or similar. It seem
Hi Joshua,
> Why not just create a eventlet "micro" version that underneath uses asyncio?
> Then code isn't riddled with yield from or Return() or similar. It seems
> possible (maybe?) to replace the eventlet hub with one that uses asyncio?
> This is then a nice change for all those who are using
Hi,
> Thanks for taking this on, Victor.
>
> Do you have a small eventlet program example that you can show the
> transformation to asyncio?
Example getting the body of an HTML page and display it to the terminal. The
HTTP query is invalid, the Host header is missing, but supporting HTTP/1.0 an
A follow up kind of question,
Why not just create a eventlet "micro" version that underneath uses asyncio?
Then code isn't riddled with yield from or Return() or similar. It seems
possible (maybe?) to replace the eventlet hub with one that uses asyncio? This
is then a nice change for all those
Thanks for taking this on, Victor.
Do you have a small eventlet program example that you can show the
transformation to asyncio?
Thanks, -peter
On Tue, Feb 4, 2014 at 8:38 AM, victor stinner
wrote:
> Hi,
>
> I would like to replace eventlet with asyncio in OpenStack for the
> asynchronous prog
Hi,
I would like to replace eventlet with asyncio in OpenStack for the asynchronous
programming. The new asyncio module has a better design and is less "magical".
It is now part of python 3.4 arguably becoming the de-facto standard for
asynchronous programming in Python world.
The asyncio mod
43 matches
Mail list logo
