Re: [Python-Dev] News from asyncio
Latest asyncio update: it has a new asyncio.subprocess submodule which provides a high-level API to control subprocesses, similar to subprocess.Popen but using asyncio event loop (and so is asynchronous). It solves the following old and tricky issue: http://bugs.python.org/issue12187 subprocess.wait() with a timeout uses polling on POSIX yield from asyncio.wait_for(proc.wait(), timeout) is no more a busy loop ;-) Victor ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] News from asyncio
On 27Jan2014 09:15, Devin Jeanpierre jeanpierr...@gmail.com wrote: On Mon, Jan 27, 2014 at 5:21 AM, Victor Stinner victor.stin...@gmail.com wrote: - asyncio.IncompleReadError.expected is the total expected size, not the remaining size Why not be consistent with the meaning of http.client.IncompleteRead.expected? The current meaning can be recovered via len(e.partial) + e.expected. Which you could also expose as a property name .requested, since this value must have been thought useful or Victor wouldn't have defined .expected to be it:-) Seems reasonable, for all that the above computation is trivial. Just a thought, -- Cameron Simpson c...@zip.com.au Artificial intelligence won't make a micrometer out of a monkeywrench. - Rick Gordon ri...@crl.com ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] News from asyncio
Hi, I'm working for eNovance on the asyncio module, the goal is to use it in the huge OpenStack project (2.5 millions line of code) which currently uses eventlet. I'm trying to fix remaining issues in the asyncio module before Python 3.4 final. The asyncio project is very active but discussions are splitted between its own dedicated mailing list (python-tulip Google group), Tulip bug tracker and Python bug tracker. Please join Tulip mailing list if you are interested to contribute. http://code.google.com/p/tulip/ I would like to share with you the status of the module. Many bugs have been fixed recently. I suppose that new bugs are found because new developers started to play with asyncio since Python 3.4 beta 1. asyncio issues fixed in Python 3.4 beta 3, in a random order: - I wrote most of the asyncio documentation, please help me to improve it! I tried to add many short examples, each time explaining one feature or concept (ex: callbacks, signals, futures, etc.): http://docs.python.org/dev/library/asyncio.html - Characters devices (TTY/PTY) are now supported, useful to control real terminal (not pipes) for subprocesses. On Mac OS X older than Maverick (10.9), the SelectSelector should be used instead of KqueueSelector (kqueue didnd't support character devices) - Tulip #111: StreamReader.readexactly() now raises an IncompleteReadError if the end of stream is reached before we received enough bytes, instead of returning less bytes than requested. - Python #20311: asyncio had a performance issue related to the resolution of selectors and clocks. For example, selectors.EpollSelector has a resolution of 1 millisecond (10^-3), whereas asyncio uses arbitrary timestamps. The issue was fixed by adding a resolution attribute to selectors and a private granularity attribute to asyncio.BaseEventLoop, and use the granularity in asyncio event loop to round times. - New Task.current_task() class method - Guido wrote a web crawler, see examples/crawl.py in Tulip - More symbols are exported in the main asyncio module (ex: Queue, iscouroutine(), etc.) - Charles-François improved the signal handlers: SA_RESTART flag is now set to limit EINTR errors in syscalls - Some optimizations (ex: don't call logger.log() when it's not needed) - Many bugfixes - (sorry if I forgot other changes, see also Tulip history and history of the asyncio module in Python) I also would like to change asyncio to support a stream-like API for subprocesses, see Tulip issue #115 (and Python issue #20400): http://code.google.com/p/tulip/issues/detail?id=115 I ported ayncio on Python 2.6 and 2.7, because today OpenStack only uses these Python versions. I created a new project called Trollius (instead of Tulip) because the syntax is a little bit different. yield from becomes yield, and return x becomes raise Return(x): https://bitbucket.org/enovance/trollius https://pypi.python.org/pypi/trollius If you are interested by the OpenStack part, see my blueprint (something similar to PEPs but for smaller changes) for Oslo Messaing: https://wiki.openstack.org/wiki/Oslo/blueprints/asyncio There is an ongoing effort to port OpenStack to Python 3, eNovance is also working on the portage: https://wiki.openstack.org/wiki/Python3 Victor ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] News from asyncio
On Mon, 27 Jan 2014 10:45:37 +0100 Victor Stinner victor.stin...@gmail.com wrote: - Tulip #111: StreamReader.readexactly() now raises an IncompleteReadError if the end of stream is reached before we received enough bytes, instead of returning less bytes than requested. Why not simply EOFError? Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] News from asyncio
2014-01-27 Antoine Pitrou solip...@pitrou.net: On Mon, 27 Jan 2014 10:45:37 +0100 Victor Stinner victor.stin...@gmail.com wrote: - Tulip #111: StreamReader.readexactly() now raises an IncompleteReadError if the end of stream is reached before we received enough bytes, instead of returning less bytes than requested. Why not simply EOFError? IncompleteReadError has two additionnal attributes: - partial: incomplete received bytes - expected: total number of expected bytes (n parameter of readexactly) I prefer to use a different exception to ensure that these attributes are present. I don't like having to check hasattr(exc, ...). Before this change, readexactly(n) returned the partial received bytes if the end of the stream was reached. Victor ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] News from asyncio
On 27 January 2014 10:55, Victor Stinner victor.stin...@gmail.com wrote: 2014-01-27 Antoine Pitrou solip...@pitrou.net: On Mon, 27 Jan 2014 10:45:37 +0100 Victor Stinner victor.stin...@gmail.com wrote: - Tulip #111: StreamReader.readexactly() now raises an IncompleteReadError if the end of stream is reached before we received enough bytes, instead of returning less bytes than requested. Why not simply EOFError? IncompleteReadError has two additionnal attributes: - partial: incomplete received bytes - expected: total number of expected bytes (n parameter of readexactly) I prefer to use a different exception to ensure that these attributes are present. I don't like having to check hasattr(exc, ...). Before this change, readexactly(n) returned the partial received bytes if the end of the stream was reached. I had the same doubt. Note also that IncompleteReadError is a subclass of EOFError, so you can catch EOFError if you like. Victor ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/gjcarneiro%40gmail.com -- Gustavo J. A. M. Carneiro Gambit Research LLC The universe is always one step beyond logic. -- Frank Herbert ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] News from asyncio
2014-01-27 Gustavo Carneiro gjcarne...@gmail.com: Why not simply EOFError? IncompleteReadError has two additionnal attributes: - partial: incomplete received bytes - expected: total number of expected bytes (n parameter of readexactly) I prefer to use a different exception to ensure that these attributes are present. I don't like having to check hasattr(exc, ...). Before this change, readexactly(n) returned the partial received bytes if the end of the stream was reached. I had the same doubt. Note also that IncompleteReadError is a subclass of EOFError, so you can catch EOFError if you like. Oops, I forgot to mention that :-) I just documented the new IncompleteReadError in asyncio doc. Victor ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] News from asyncio
27.01.14 12:55, Victor Stinner написав(ла): IncompleteReadError has two additionnal attributes: - partial: incomplete received bytes - expected: total number of expected bytes (n parameter of readexactly) This looks similar to http.client.IncompleteRead. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] News from asyncio
2014-01-27 Serhiy Storchaka storch...@gmail.com: 27.01.14 12:55, Victor Stinner написав(ла): IncompleteReadError has two additionnal attributes: - partial: incomplete received bytes - expected: total number of expected bytes (n parameter of readexactly) This looks similar to http.client.IncompleteRead. Please read the original issue for more information: http://code.google.com/p/tulip/issues/detail?id=111 I mentionned http.client.IncompleRead exception there. The HTTP exception is similar but also different: - asyncio.IncompleReadError inherits from EOFError, not from HTTPException (which inherits from Exception) - asyncio.IncompleReadError.expected is the total expected size, not the remaining size Victor ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] News from asyncio
On Mon, Jan 27, 2014 at 5:21 AM, Victor Stinner victor.stin...@gmail.com wrote: - asyncio.IncompleReadError.expected is the total expected size, not the remaining size Why not be consistent with the meaning of http.client.IncompleteRead.expected? The current meaning can be recovered via len(e.partial) + e.expected. -- Devin ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
