[issue31815] Make itertools iterators interruptible

Raymond Hettinger added the comment: I don't think a new public API should be introduced. This is at best an implementation detail. Also, I really don't want to garbage-up the inner-loop code for the itertools. I've spent a good deal of time micro-optimizing this code and don't want to

[issue31815] Make itertools iterators interruptible

Nick Coghlan added the comment: The purpose would be two-fold: 1. The presence of the `check_signals()` wrapper provides a way to more explicitly document that the other itertools iterators *don't* implicitly check for signals, so if you want to combine them with consumers that also don't

[issue31815] Make itertools iterators interruptible

Raymond Hettinger added the comment: > What if itertools were to offer an opt-in ... This doesn't make sense to me. As far as I can tell, the only time this issue has ever arisen in the past 15 or 16 years is when someone was trying to create an unbreakable infinite loop on-purpose. In a

[issue31815] Make itertools iterators interruptible

Nick Coghlan added the comment: As a potential stepping stone towards possible future changes in the default behaviour here, what if itertools were to offer an opt-in "check_signals(itr, *, iterations=100_000)" helper function that was essentially a more efficient version of:: def

[issue31815] Make itertools iterators interruptible

Nick Coghlan added the comment: Note: I've filed the "raise TypeError in __length_hint__" suggestion for infinite iterators separately in https://bugs.python.org/issue33939 -- ___ Python tracker

[issue31815] Make itertools iterators interruptible

Koos Zevenhoven added the comment: For the interactive user who uses an interactive environment such as the repl or a Jupyter notebook, the situation is a little different from "CPython as programming language runtime". The docs say a KeyboardInterrupt is "Raised when the

[issue31815] Make itertools iterators interruptible

Tim Peters added the comment: Segfaults are different: they usually expose an error in CPython's implementation. We don't prioritize them because the user may have to restart their program (who cares? <0.5 wink>), but because they demonstrate the language implementation is

[issue31815] Make itertools iterators interruptible

Nick Coghlan added the comment: I'd personally be happy enough if the infinite iterators implemented __length_hint__() as always raising TypeError so the machine-breaking cases of incremental consumption of ever-increasing amounts of memory were blocked - I was suggesting

[issue31815] Make itertools iterators interruptible

Raymond Hettinger added the comment: I respectfully disagree that this just happens to people accidentally -- Every single day, I work with either Python professionals or Python students and never see this situation occur, nor have I had a single report of it from

[issue31815] Make itertools iterators interruptible

Nick Coghlan added the comment: To put this another way: I see an uninterruptible infinite loop as a data loss bug on par with a segfault, since there's no graceful way to terminate the process and allow cleanup code to run. For segfaults, we're willing to tolerate them,

[issue31815] Make itertools iterators interruptible

Nick Coghlan added the comment: Defensive coding and the complications it brings is a fact of life when providing a widely used platform. Sure, we're free to say "We don't care about minor user experience irritations like Ctrl-C not always being reliable, users should

[issue31815] Make itertools iterators interruptible

Koos Zevenhoven added the comment: To repeat one of my points in the linked threads, I'm not convinced that infinite iterators are the most common case for the problem of long uninterruptible loops. A general mechanism that can be easily used in many places with minimal

[issue31815] Make itertools iterators interruptible

Serhiy Storchaka added the comment: I concur with Raymond. I cited the same arguments in the discussion on Python-ideas. But the other solution that was suggested in this discussion will add more complexity and can't solve all cases. --

[issue31815] Make itertools iterators interruptible

Serhiy Storchaka added the comment: With optimized repeat(): $ ./python -m perf timeit --compare-to=../cpython-release/python -s 'from itertools import repeat' 'list(repeat(None, 100))' /home/serhiy/py/cpython-release/python: . 3.77 ms +-

[issue31815] Make itertools iterators interruptible

Serhiy Storchaka added the comment: Microbenchmark results: $ ./python -m perf timeit --compare-to=../cpython-release/python -s 'from itertools import repeat' 'list(repeat(None, 100))' /home/serhiy/py/cpython-release/python: . 3.79 ms +-

[issue31815] Make itertools iterators interruptible

Raymond Hettinger added the comment: When I have time, I would like to re-launch a python-dev discussion on this. It is my feeling that this solves an invented problem. In my experience, it only ever happens to people who have intentionally trying to create this

[issue31815] Make itertools iterators interruptible

Change by Raymond Hettinger : -- assignee: -> rhettinger ___ Python tracker ___

[issue31815] Make itertools iterators interruptible

Change by Serhiy Storchaka : -- title: Make itertools iterators interrable -> Make itertools iterators interruptible ___ Python tracker