On Tue, Oct 12, 2021 at 8:43 PM Oscar Benjamin
<oscar.j.benja...@gmail.com> wrote:
> A leaky StopIteration can wreak all sorts of havoc. There was a PEP that
> attempted to solve this by turning StopIteration into RuntimeError if it gets
> caught in a generator but that PEP (which was rushed through very quickly
> IIRC) missed the fact that generators are not the only iterators. It remains
> a problem that leaking a StopIteration into map, filter etc will terminate
> iteration of an outer loop.
>
Generators are special because they never mention StopIteration. They
are written like functions, but behave like iterators. That is why
StopIteration leaking is such a problem.
In every other situation, StopIteration is part of the API of what
you're working with. It is a bug to call next() without checking for
StopIteration (or knowingly and intentionally permitting it to
bubble).
> The culprit for the problem of leaking StopIteration is next itself which in
> the 1-arg form is only really suitable for use when implementing an iterator
> and not for the much more common case of simply wanting to extract something
> from an iterable. Numerous threads here and on stackoverflow and elsewhere
> suggesting that you can simply use next(iter(obj)) are encouraging bug magnet
> code. Worse, the bug when it arises will easily manifest in something like
> silent data loss and can be hard to debug.
>
That's no worse than getattr() and AttributeError. If you call getattr
and you aren't checking for AttributeError, then you could be running
into the exact same sorts of problems, because AttributeError is part
of the function's API.
> The correct usage of next/iter in most cases would be something like:
>
> try:
> val = next(iter(obj))
> except StopIteration:
> raise AnotherError
Yes. Or whatever other method you have for coping with the lack of a
first element.
> or perhaps
>
> val = next(iter(obj), None)
> if val is None:
> raise AnotherError
Definitely not. The two-arg form is a short-hand for this:
try:
val = next(iter(obj))
except StopIteration:
val = None
If your except clause would simply set a default, use two-arg next.
Otherwise, don't open yourself up to data-specific bugs.
> The real advantage of providing first (or "take" or any of the other names
> that have been proposed in the past) is that it should raise a different
> exception like ValueError so that it would be safe to use by default.
>
ValueError is no safer. The first() function would have, as its API,
"returns the first element or raises ValueError if there is none". So
now the caller of first() has to use try/except to handle the case
where there is no value. Failing to do so is *just as buggy* as
leaking a StopIteration.
A leaky StopIteration is a majorly confusing bug inside a __next__
function, because StopIteration is part of that function's API. A
leaky KeyError is a majorly confusing bug inside a __getitem__
function, for the same reason. A leaky AttributeError inside a
__getattr__ function, ditto. Anywhere else, those exceptions will all
just bubble up normally, and most likely get printed to the console.
>>> def leak(): return next(iter([])) # ooops
...
>>> for foo in leak(): print("Hello")
...
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 1, in leak
StopIteration
>>> for foo in "test": print(leak())
...
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 1, in leak
StopIteration
They don't prematurely end the loop because they're not happening in
places where that's the API you're working with.
The only times you should need to think about StopIteration are
calling next(), and implementing __next__.
ChrisA
_______________________________________________
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at
https://mail.python.org/archives/list/python-ideas@python.org/message/EBI3YBM3TV72PBWPBAHWOJLB4PNNRTHM/
Code of Conduct: http://python.org/psf/codeofconduct/
