[issue26221] awaiting asyncio.Future swallows StopIteration
Ian Kelly added the comment: Chris Angelico suggested on python-list that another possibly useful thing to do would be to add a "from __future__ import generator_stop" to asyncio/futures.py. This would at least have the effect of causing "await future" to raise a RuntimeError instead of silently returning None if a StopIteration is set on the future. Future.__iter__ is the only generator in the file, so this change shouldn't have any other effects. -- title: asynco run_in_executor swallows StopIteration -> awaiting asyncio.Future swallows StopIteration ___ Python tracker <http://bugs.python.org/issue26221> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26221] asynco run_in_executor swallows StopIteration
Ian Kelly added the comment: The place I'd expect to find it is in https://docs.python.org/3/library/asyncio-task.html#coroutines, in the list of "things a coroutine can do". The first two bullets in the list say that any exceptions raised will be propagated. Maybe there should be a note after the bullet list to the effect that "StopIteration carries special meaning to coroutines and will not be propagated if raised by an awaited coroutine or future." -- ___ Python tracker <http://bugs.python.org/issue26221> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26221] asynco run_in_executor swallows StopIteration
Ian Kelly added the comment: Fair enough. I think there should be some documentation though to the effect that coroutines aren't robust to passing StopIteration across coroutine boundaries. It's particularly surprising with PEP-492 coroutines, since those aren't even iterators and intuitively should ignore StopIteration like normal functions do. As it happens, this variation (moving the try-except into the executor thread) does turn out to work but is probably best avoided for the same reason. I don't think it's obviously bad code though: class AsyncIteratorWrapper: def __init__(self, iterable, loop=None, executor=None): self._iterator = iter(iterable) self._loop = loop or asyncio.get_event_loop() self._executor = executor async def __aiter__(self): return self async def __anext__(self): def _next(iterator): try: return next(iterator) except StopIteration: raise StopAsyncIteration return await self._loop.run_in_executor( self._executor, _next, self._iterator) -- ___ Python tracker <http://bugs.python.org/issue26221> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26221] asynco run_in_executor swallows StopIteration
Ian Kelly added the comment:
The idea is that the wrapped iterator is something potentially blocking, like a
database cursor that doesn't natively support asyncio. Usage would be something
like this:
async def get_data():
cursor.execute('select * from stuff')
async for row in AsyncIteratorWrapper(cursor):
process(row)
Investigating this further, I think the problem is actually in await, not
run_in_executor:
>>> async def test():
... fut = asyncio.Future()
... fut.set_exception(StopIteration())
... print(await fut)
...
>>> loop.run_until_complete(test())
None
--
___
Python tracker
<http://bugs.python.org/issue26221>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26221] asynco run_in_executor swallows StopIteration
New submission from Ian Kelly: I was playing around with this class for adapting regular iterators to async iterators using BaseEventLoop.run_in_executor: import asyncio class AsyncIteratorWrapper: def __init__(self, iterable, loop=None, executor=None): self._iterator = iter(iterable) self._loop = loop or asyncio.get_event_loop() self._executor = executor async def __aiter__(self): return self async def __anext__(self): try: return await self._loop.run_in_executor( self._executor, next, self._iterator) except StopIteration: raise StopAsyncIteration Unfortunately this fails because when next raises StopIteration, run_in_executor swallows the exception and just returns None back to the coroutine, resulting in an infinite iterator of Nones. -- components: asyncio messages: 259036 nosy: gvanrossum, haypo, ikelly, yselivanov priority: normal severity: normal status: open title: asynco run_in_executor swallows StopIteration versions: Python 3.5 ___ Python tracker <http://bugs.python.org/issue26221> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue25675] doc for BaseEventLoop.run_in_executor() says its a coroutine, but it is not
Ian Kelly added the comment: The asyncio docs also have this note, so this is technically not a bug: Note: In this documentation, some methods are documented as coroutines, even if they are plain Python functions returning a Future. This is intentional to have a freedom of tweaking the implementation of these functions in the future. If such a function is needed to be used in a callback-style code, wrap its result with ensure_future(). Since the intention seems to be to document something that can be awaited without specifying the implementation, I think that these functions should be documented as returning awaitables. However GvR in python-ideas said: IMO [the docs] should be very clear about the distinction between functions that return Futures and functions that return coroutines (of either kind). I think it's fine if they are fuzzy about whether the latter return a PEP 492 style coroutine (i.e. defined with async def) or a pre-PEP-492 coroutine (marked with @asyncio.coroutine), since those are almost entirely interchangeable, and the plan is to eventually make everything a PEP 492 coroutine. Source: http://thread.gmane.org/gmane.comp.python.ideas/38045/focus=38046 -- nosy: +ikelly ___ Python tracker <http://bugs.python.org/issue25675> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23460] Decimals do not obey ':g' exponential notation formatting rules
New submission from Ian Kelly:
>>> '{:g}'.format(D('0.01'))
'0.01'
Formatted with '{:e}', the exponent would be -6, so per the formatting rules
described under the 'g' specifier at
https://docs.python.org/3/library/string.html#format-specification-mini-language
the above should be formatted using exponential notation.
--
components: Library (Lib)
messages: 235904
nosy: ikelly
priority: normal
severity: normal
status: open
title: Decimals do not obey ':g' exponential notation formatting rules
type: behavior
versions: Python 3.4
___
Python tracker
<http://bugs.python.org/issue23460>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22911] Segfault on recursive itertools.chain.from_iterable
New submission from Ian Kelly: I expect this should result in a recursion depth exceeded error, not a segmentation fault. $ cat test.py import itertools l = [] it = itertools.chain.from_iterable(l) l.append(it) next(it) $ python3 test.py Segmentation fault (core dumped) -- components: Library (Lib) messages: 231489 nosy: ikelly priority: normal severity: normal status: open title: Segfault on recursive itertools.chain.from_iterable versions: Python 3.4 ___ Python tracker <http://bugs.python.org/issue22911> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17697] Incorrect stacktrace from pdb
Ian Kelly added the comment: The bug also occurs using 32-bit Python 3.3.1 on Windows 7. -- nosy: +ikelly ___ Python tracker <http://bugs.python.org/issue17697> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1167] gdbm/ndbm 1.8.1+ needs libgdbm_compat.so
Ian Kelly added the comment: I'm not sure why you think this patch would be backwards incompatible. I've tested it with gdbm-1.8.0 and gdbm-1.7.3, and it works for those. __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1167> __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1167] gdbm/ndbm 1.8.1+ needs libgdbm_compat.so
New submission from Ian Kelly: The ndbm functions in gdbm 1.8.1+ require the gdbm_compat library in addition to gdbm. -- components: Build, Extension Modules files: gdbm_ndbm.diff messages: 55939 nosy: ikelly severity: normal status: open title: gdbm/ndbm 1.8.1+ needs libgdbm_compat.so type: compile error versions: Python 2.5, Python 2.6, Python 3.0 __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1167> __ gdbm_ndbm.diff Description: Binary data ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
