Re: [openstack-dev] [Octavia] Which threading library?

[Mike Bayer]

On Sep 12, 2014, at 5:53 PM, Eichberger, German  
wrote:

> Hi,
>  
> I think the “standard” threading library for OpenStack is eventlet – however, 
> it seems that Oslo is spearheading efforts to get to a more compatible one 
> (see http://techs.enovance.com/6562/asyncio-openstack-python3) I am now 
> wondering since we are starting fresh if we should embrace (a potential) 
> future or stick with eventlet and all its flaws?

the so-called “flaws” of implicit async are up for debate.A “flaw” of 
asyncio is that it requires a full rewrite of code that uses it, as well as all 
libraries which it consumes that also happen to use IO.This includes the 
particularly salient topic that none of the Python database APIs that exist, 
other than psycopg2, have any support for true non-blocking code at the IO 
level, without using monkeypatching.  the standard Python DBAPI (pep 249) has 
no support for explicit async, so any APIs that do so are ad-hoc (see 
http://initd.org/psycopg/docs/advanced.html#async-support for psycopg2’s very 
well done, but entirely non-standard API in this regard).

Once you’re on explicit async, simple imperative code that happens to make 
calls which imply IO is no longer possible.   This is kind of a blocker to 
end-to-end integration of asyncio in all of Openstack, rather than just making 
use of it in those areas where it is already directly applicable.

This has been discussed in depth at 
lists.openstack.org/pipermail/openstack-dev/2014-July/039291.html.


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Octavia] Which threading library?

[Joshua Harlow]

So my personal opinion is that u should design your API independent of either 
of these (asyncio, eventlet, threads...),

I think if u design the key parts of your API with the future[1] concept in 
mind then you can easily connect into the future (asyncio or other).

The asyncio layer has a future wrapper of its own (it actually has its own 
future related class[2], both of which are in the standard library to add to 
the confusion/fun...),

But if you target that kind of API (one that can return async objects, futures 
or other) then I think you will be tolerant of changes for asyncio or other...

With taskflow[3] I've tried to do this. It runs tasks/atoms (a taskflow 
concept) via futures and executors (one of its executors actually runs things 
remotely), which hopefully makes it easier to switch in a asyncio executor or 
other in the future, and it also has connections to make it transparent that 
eventlet is just another future executor[4]. But I guess it becomes a question 
of what platform (3.4+) do you want to support  and what design decisions you 
are ok with making that don't affect your current system to drastically...

[1] https://docs.python.org/dev/library/concurrent.futures.html#future-objects
[2] https://docs.python.org/3/library/asyncio-task.html#future
[3] http://docs.openstack.org/developer/taskflow/
[4] 
https://github.com/openstack/taskflow/blob/master/taskflow/utils/eventlet_utils.py#L98

My 2 cents,

Josh

On Sep 12, 2014, at 2:53 PM, Eichberger, German  
wrote:

> Hi,
>  
> I think the “standard” threading library for OpenStack is eventlet – however, 
> it seems that Oslo is spearheading efforts to get to a more compatible one 
> (seehttp://techs.enovance.com/6562/asyncio-openstack-python3) I am now 
> wondering since we are starting fresh if we should embrace (a potential) 
> future or stick with eventlet and all its flaws?
>  
> Thoughts?
>  
> German
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev