[issue29942] Stack overflow in itertools.chain.from_iterable.

Changes by Thomas Wouters : -- resolution: -> fixed stage: patch review -> resolved status: open -> closed ___ Python tracker ___

[issue29942] Stack overflow in itertools.chain.from_iterable.

Thomas Wouters added the comment: New changeset d694a06206fc09b76b4507aacde5e69a248f434f by T. Wouters in branch '2.7': bpo-29942: Fix the use of recursion in itertools.chain.from_iterable. (#913) https://github.com/python/cpython/commit/d694a06206fc09b76b4507aacde5e69a248f434f --

[issue29942] Stack overflow in itertools.chain.from_iterable.

Thomas Wouters added the comment: New changeset 9273dfe1800fc7241d69f4d523d748ebd35b3801 by T. Wouters in branch '3.5': bpo-29942: Fix the use of recursion in itertools.chain.from_iterable. (#912) https://github.com/python/cpython/commit/9273dfe1800fc7241d69f4d523d748ebd35b3801 --

[issue29942] Stack overflow in itertools.chain.from_iterable.

Thomas Wouters added the comment: New changeset 599bb181036f724629a515317f0f39520950d51c by T. Wouters in branch '3.6': bpo-29942: Fix the use of recursion in itertools.chain.from_iterable. (#911) https://github.com/python/cpython/commit/599bb181036f724629a515317f0f39520950d51c --

[issue29942] Stack overflow in itertools.chain.from_iterable.

Serhiy Storchaka added the comment: Possible workaround: use chain.from_iterable(filter(None, iterables)) instead of chain.from_iterable(iterables). But this works only when iterables are collections, not iterators. -- nosy: +serhiy.storchaka ___

[issue29942] Stack overflow in itertools.chain.from_iterable.

Changes by Thomas Wouters : -- pull_requests: +811 ___ Python tracker ___ ___

[issue29942] Stack overflow in itertools.chain.from_iterable.

Changes by Thomas Wouters : -- pull_requests: +812 ___ Python tracker ___ ___

[issue29942] Stack overflow in itertools.chain.from_iterable.

Changes by Thomas Wouters : -- pull_requests: +813 ___ Python tracker ___ ___

[issue29942] Stack overflow in itertools.chain.from_iterable.

Thomas Wouters added the comment: FWIW, we ran into this in real-world cases (Youtube, I think), when we switched from using a pre-built Python interpreter to one built from source using the same optimisation and debug levels as we use for all other C/C++ code. Even so, the accompanying test

[issue29942] Stack overflow in itertools.chain.from_iterable.

Thomas Wouters added the comment: New changeset 5466d4af5fe76ec0a5fbc8a05675287d9e8e9d14 by T. Wouters in branch 'master': bpo-29942: Fix the use of recursion in itertools.chain.from_iterable. (#889) https://github.com/python/cpython/commit/5466d4af5fe76ec0a5fbc8a05675287d9e8e9d14 --

[issue29942] Stack overflow in itertools.chain.from_iterable.

Mark Dickinson added the comment: > I would have guessed that the C compiler would have automatically removed the > tail recursion I think it probably does, unless optimisation is turned off: I'm unable to reproduce except in debug builds of Python. -- nosy: +mark.dickinson

[issue29942] Stack overflow in itertools.chain.from_iterable.

Raymond Hettinger added the comment: This looks fine. Feel free to apply and to backport this to earlier versions. I would have guessed that the C compiler would have automatically removed the tail recursion, but your experience would indicate otherwise. -- assignee: -> twouters

[issue29942] Stack overflow in itertools.chain.from_iterable.

Changes by Serhiy Storchaka : -- nosy: +rhettinger stage: -> patch review versions: +Python 2.7, Python 3.5, Python 3.6, Python 3.7 ___ Python tracker

[issue29942] Stack overflow in itertools.chain.from_iterable.

New submission from Thomas Wouters: itertools.chain.from_iterable (somewhat ironically) uses recursion to resolve the next iterator, which means it can run out of the C stack when there's a long run of empty iterables. This is most obvious when building with low optimisation modes, or with

[issue29942] Stack overflow in itertools.chain.from_iterable.

Changes by Thomas Wouters : -- type: -> crash ___ Python tracker ___ ___ Python-bugs-list