Re: [openstack-dev] [solum] async / threading for python 2 and 3
On 21/02/14 11:03 +0100, Victor Stinner wrote: Le vendredi 21 février 2014, 09:27:49 Angus Salkeld a écrit : Honestly, I have no answer to your question right now (How useful is trollius ...). (...) I asked your question on Tulip mailing list to see how a single code base could support Tulip (yield from) and Trollius (yield). At least check if it's technically possible. Short answer: it's possible to write code working on Trollius (Python 2 and 3) / Tulip (Python 3.3+) / CPython 3.4 (asyncio) if you use callbacks. The core of the asyncio module (event loop and scheduler) uses callbacks. If you only uses callbacks, you can also support Twisted and Tornado frameworks. Ok, that is a good idea. For example, the AutobahnPython project adopted this design and thanks to that, it supports Trollius, Tulip and CPython asyncio, but also Twisted: https://github.com/tavendo/AutobahnPython So you have to find a WSGI server using callbacks instead of yield from. It should not be hard since asyncio is young, callbacks was the only option before greenlet/eventlet, and Twisted and Tornado (which use callbacks) are still widely used. Makes sense. Thanks for taking the time to look into it. -Angus Victor ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [solum] async / threading for python 2 and 3
Le vendredi 21 février 2014, 09:27:49 Angus Salkeld a écrit : Honestly, I have no answer to your question right now (How useful is trollius ...). (...) I asked your question on Tulip mailing list to see how a single code base could support Tulip (yield from) and Trollius (yield). At least check if it's technically possible. Short answer: it's possible to write code working on Trollius (Python 2 and 3) / Tulip (Python 3.3+) / CPython 3.4 (asyncio) if you use callbacks. The core of the asyncio module (event loop and scheduler) uses callbacks. If you only uses callbacks, you can also support Twisted and Tornado frameworks. For example, the AutobahnPython project adopted this design and thanks to that, it supports Trollius, Tulip and CPython asyncio, but also Twisted: https://github.com/tavendo/AutobahnPython So you have to find a WSGI server using callbacks instead of yield from. It should not be hard since asyncio is young, callbacks was the only option before greenlet/eventlet, and Twisted and Tornado (which use callbacks) are still widely used. Victor ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [solum] async / threading for python 2 and 3
On 20/02/14 13:52 +0100, victor stinner wrote: Hi, On 19/02/14 10:09 +0100, Julien Danjou wrote: On Wed, Feb 19 2014, Angus Salkeld wrote: 2) use tulip and give up python 2 + use trollius to have Python 2 support. https://pypi.python.org/pypi/trollius So I have been giving this a go. FYI I'm the author of Trollius project. Cool, Hi. We use pecan and wsme (like ceilometer), I wanted to use a httpserver library in place of wsgiref.server so had a look at a couple and can't use them as they all have yield from all over the place (i.e. python 3 only). The quesion I have is: How useful is trollius if we can't use other thirdparty libraries written for asyncio? https://github.com/KeepSafe/aiohttp/blob/master/aiohttp/server.py#L171 Maybe I am missing something? (Tulip and Trollius unit tests use wsgiref.simple_server module of the standard library. It works but you said that you don't want to use it.) I saw that, but it's used as a test server and doesn't run an asyncio loop. wsgiref.server does not have any of the yields that we need around reads/writes but more importantly we need an add_reader() so the whole server does not block waiting for a new connection. My usecase is to receive a REST request return a 202 (Accepted) and process the request asyncronously. If the http server then goes around it's loop and blocks on a read (waiting for the next request) I never get any processing time scheduled. Honestly, I have no answer to your question right now (How useful is trollius ...). asyncio developers are working on fixing last bugs in asyncio (Trollius is a fork, I merge regulary updates from Tulip into Trollius) and adding some late features before the Python 3.4 release. This Python release will be somehow the version 1.0 of asyncio and will freeze the API. Right now, I'm working on a proof-on-concept of eventlet hub using asyncio event loop. So it may be possible to use eventlet and asyncio APIs are the same time. And maybe slowly replace eventlet with asyncio, or at least use asyncio in new code. Ok, thanks for the update. -Angus I asked your question on Tulip mailing list to see how a single code base could support Tulip (yield from) and Trollius (yield). At least check if it's technically possible. Victor ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [solum] async / threading for python 2 and 3
On Wed, Feb 19 2014, Angus Salkeld wrote: 2) use tulip and give up python 2 + use trollius to have Python 2 support. https://pypi.python.org/pypi/trollius -- Julien Danjou /* Free Software hacker http://julien.danjou.info */ signature.asc Description: PGP signature ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [solum] async / threading for python 2 and 3
On 19/02/14 10:09 +0100, Julien Danjou wrote: On Wed, Feb 19 2014, Angus Salkeld wrote: 2) use tulip and give up python 2 + use trollius to have Python 2 support. https://pypi.python.org/pypi/trollius So I have been giving this a go. We use pecan and wsme (like ceilometer), I wanted to use a httpserver library in place of wsgiref.server so had a look at a couple and can't use them as they all have yield from all over the place (i.e. python 3 only). The quesion I have is: How useful is trollius if we can't use other thirdparty libraries written for asyncio? https://github.com/KeepSafe/aiohttp/blob/master/aiohttp/server.py#L171 Maybe I am missing something? http://code.google.com/p/tulip/wiki/ThirdParty -Angus -- Julien Danjou /* Free Software hacker http://julien.danjou.info */ ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [solum] async / threading for python 2 and 3
Hi all I need to use some async / threaded behaviour to solum for image creation (all I need right now is to run a job asyncronously). eventlet is python 2 only tulip is python 3 only tornado (supports 2 + 3) http://www.tornadoweb.org twisted pyev etc... Options: 1) use eventlet and have the same path of migration as the rest of openstack. Basically give up python 3 for now. 2) use tulip and give up python 2 3) choose an existing framework that supports both py2+3 Thoughts? Since we are starting out fresh, I'd suggest 2). This will mean some learning, but that is always fun and would be benefical to other projects to see how this code looks. I am not sure how important support for python 2 is, I'd suggest supporting python 3 is more important. -Angus ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [solum] async / threading for python 2 and 3
Hi Angus, I think that we need to keep python 2 as a lot of enterprise customers still use python 2.x versions. If you target Solum only for python 3 users you will significantly reduce potential customer adoption. I would rather spend some time and research round option #3. I saw in openstack-dev mailing list that there is a work around asyncio native in py3 and its port for python2. Here is related BP https://wiki.openstack.org/wiki/Oslo/blueprints/asyncio Here is the e-mail thread: http://lists.openstack.org/pipermail/openstack-dev/2014-February/026237.html Thanks Gosha On Tue, Feb 18, 2014 at 4:53 PM, Angus Salkeld angus.salk...@rackspace.comwrote: Hi all I need to use some async / threaded behaviour to solum for image creation (all I need right now is to run a job asyncronously). eventlet is python 2 only tulip is python 3 only tornado (supports 2 + 3) http://www.tornadoweb.org twisted pyev etc... Options: 1) use eventlet and have the same path of migration as the rest of openstack. Basically give up python 3 for now. 2) use tulip and give up python 2 3) choose an existing framework that supports both py2+3 Thoughts? Since we are starting out fresh, I'd suggest 2). This will mean some learning, but that is always fun and would be benefical to other projects to see how this code looks. I am not sure how important support for python 2 is, I'd suggest supporting python 3 is more important. -Angus ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Georgy Okrokvertskhov Architect, OpenStack Platform Products, Mirantis http://www.mirantis.com Tel. +1 650 963 9828 Mob. +1 650 996 3284 ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
