On 28Feb2021 23:56, Irit Katriel <iritkatr...@googlemail.com> wrote:
>If you go long, I go longer :)
:-)
>On Sun, Feb 28, 2021 at 10:51 PM Cameron Simpson <c...@cskk.id.au> wrote:
>> On 28Feb2021 10:40, Irit Katriel <iritkatr...@googlemail.com> wrote:
>> >split() and subgroup() take care to preserve the correct metadata on
>> >all
>> >the internal nodes, and if you just use them you only make safe
>> operations.
>> >This is why I am hesitating to add iteration utilities to the API. Like we
>> >did, people will naturally try that first, and it's not the safest API.
>>
>> Wouldn't it be safe if the ExceptionGroup were immutable, as you plan?
>> Or have you another hazard in mind?
>
>Making them immutable won't help the metadata issue. split() and subgroup()
>copy the (context, cause traceback) from the original ExceptionGroups (root
>and internal nodes of the tree) to the result trees. If you DIY creating
>new ExceptionGroups you need to take care of that.
Ah, right. Yes.
The overflows into my request for a factory method to construct a "like"
ExceptionGroup with a new exception tree lower down.
>> But all that said, being able to iterable the subexceptions seems a
>> natural way to manage that:
>>
>> unhandled = []
>> try:
>> .........
>> except *OSError as eg:
>> for e in eg:
>> if an ok exception:
>> handle it
>> else:
>> unhandled.append(e)
>> if unhandled:
>> raise ExceptionGroup("unhandled", unhandled)
>
>
>You just lost the metadata of eg. It has no context, cause and its
>traceback begins here.
Aye. Hence a wish, again lower down, for some reference to the source
ExceptionGroup and therefore a handy factory for making a new group with
the right metadata.
>And the exceptions contained in it, if they came
>from a deeper tree that you flattened into the list, now look like their
>traceback jumps straight to here from the place they were actually first
>inserted into an ExceptionGroup. This may well be an impossible code path.
Perhaps so. But it doesn't detract from how useful it is to iterate over
the inner exceptions. I see this as an argument for making it possible
to obtain the correct metadata, not against iteration itself. Even if
the iteration yielded some proxy or wrapper for the inner exception
instead of the naked exception itself.
>Here's an example:
[... flattened ExceptionGroup with uninformative tracebacks ...]
>>>> import traceback
>>>> def flatten(exc):
>... if isinstance(exc, ExceptionGroup):
>... for e in exc.errors:
>... yield from flatten(e)
>... else:
>... yield exc
[...]
>>>> traceback.print_exception(flat_h())
>Traceback (most recent call last):
> File "<stdin>", line 3, in flat_h
>ExceptionGroup: flat_h
> ------------------------------------------------------------
> Traceback (most recent call last):
> File "<stdin>", line 3, in f
> ValueError: 42
>
>traceback.print_exception(h()) prints a reasonable traceback - h() called
>g() called f().
>
>But according to traceback.print_exception(flat_h()), flat_h() called
>f().
>
>You can preserve the metadata (and the nested structure with all its
>metadata) if you replace the last line with:
> raise eg.subgroup(lambda e: e in unhandled)
Ok. That works for me as my desired factory. Verbose maybe, but
workable. Um. It presumes exception equality will do - that feels
slightly error prone (including some similar but not unhandled
exceptions). But I can write something more pointed based on id().
>And for the part before that, iteration, Guido's pattern showed that you
>can roll it into the subgroup callback.
Aye.
>> There are some immediate shortcomings above. In particular, I have no
>> way of referencing the original ExceptionGroup without surprisingly
>> cumbersome:
>
>> try:
>> .........
>> except ExceptionGroup as eg0:
>> unhandled = []
>> eg, os_eg = eg0.split(OSError)
>> if os_eg:
>> for e in os_eg:
>> if an ok exception:
>> handle it
>> else:
>> unhandled.append(e)
>> if eg:
>> eg, value_eg = eg.split(ValueError)
>> if value_eg:
>> for e in value_eg:
>> if some_ValueError_we_understand:
>> handle it
>> else:
>> unhandled.append(e)
>> if eg:
>> unhandled.append(eg)
>> if unhandled:
>> raise ExceptionGroup("unhandled", unhandled) from eg0
>
>This is where except* can help:
>
> try:
> .........
> except except *OSError as eg:
> unhandled = []
> handled, unhandled = eg.split(lambda e: e is an ok exception) #
>with side effect to handle e
> if unhandled:
> raise unhandled
> except *ValueError as eg:
> handled, unhandled = eg.split(lambda e: e is a value error we
>understand) # with side effect to handle e
> if unhandled:
Alas, that raises within each branch. I want to gather all the unhandled
exceptions together into single ExceptionGroup. Using split gets me a
bunch of groups, were I to defer the raise to after all the checking (as
I want to in my pattern).
So I've have to combine multiple groups back together.
unhandled_groups = []
try:
.........
except except *OSError as eg:
handled, unhandled = eg.split(lambda e: e is an ok exception) # with
side effect to handle e
if unhandled:
unhandled_groups.append(unhandled)
except *ValueError as eg:
handled, unhandled = eg.split(lambda e: e is a value error we
understand) # with side effect to handle e
if unhandled:
unhandled_groups.append(unhandled)
if unhandled_groups:
# combine them here?
new_eg = eg0.subgroup(lambda e: e in
all-the-es-from-all-the-unhandled-groups)
That last line needs eg0, the original ExceptionGroup (unavailable
AFAICT), _and_ something to test for a naked exception being one of the
unhandled ones.
I _could_ gather the latter as a side effect of my split() lambda, but
that makes things even more elaborate. A closure, even!
If I could just iterate over the nested naked exceptions _none_ of this
complexity would be required.
>I have the following concerns with the pattern above:
>>
>> There's no way to make a _new_ ExceptionGroup with the same __cause__
>> and __context__ and message as the original group: not that I can't
>> assign to these, but I'd need to enuerate them; what if the exceptions
>> grew new attributes I wanted to replicate?
>
>As I said, split() and subgroup() do that for you.
They do if I'm only concerned with the particular exception subclass (eg
OSError) i.e. my work is complete within the single except* clause. If I
use except*, but later want to build an overarching "unhandled
exceptions group", I have nothing to build on.
>> This cries out for another factory method like .subgroup but which
>> makes
>> a new ExceptionGroup from an original group, containing a new sequence
>> of exceptions but the same message and coontext. Example:
>>
>> unhandled_eg = eg0.with_exceptions(unhandled)
>
>Why same message and context but not same cause and traceback?
Insufficient verbiage. I meant same message and context and cause. The
point being that it is exceptly such a missing out of some attribute
that I'd like to avoid by not having to enumerate the preserved
attributes - the factory should do that, knowing the ExceptionGroup
internals.
Anyway, as you point out, .subgroup does this.
>> I don't see a documented way to access the group's message.
>
>In the PEP: "The ExceptionGroup class exposes these parameters in the
>fields message and errors".
Hmm. Missed that. Thanks.
I did find the PEP hard to read in some places. I think it could well do
with saying "eg" instead of "e" throughout where "e" is an
ExceptionGroup - I had a distinct tendency to want to read "e" as one of
the naked exceptions and not the group.
>> I'm quite unhappy about .subgroup (and presumably .split) returning
>> None
>> when the group is empty. The requires me to have the gratuitous "if eg:"
>> and "if value_eg:" if-statements in the example above.
>>
>> If, instead, ExceptionGroups were like any other container I could
>> just test if they were empty:
>>
>> if eg:
>>
>> _and_ even if they were empty, iterate over them. Who cares if the loop
>> iterates zero times?
>> [...]
>> Anyway, I'm strongly of the opinion that ExceptionGroups should look
>> like containers, be iterable, be truthy/falsey based on empty/nonempty
>> and that .split and .subgroup should return empty subgroups instead of
>> None.
>
>This would be true if iteration was a useful pattern for working with
>ExceptionGroup, but I still think subgroup/split is a better tool in most
>cases.
I think I want to iterate over these things - it is useful to me. I want
an ExceptionGroup to look like a container. I accept that subgroup makes
a new group with metadata intact, but often that is not what _I_ care
about, particularly when I'm just winnowing some special known cases.
I cannot see that returning empty groups instead of None does any harm
at all, and brings benefits in streamlining testing and processing.
I guess if we don't get iteration and containerness I'll just have to
subclass ExceptionGroup for my own use, giving it iterations and
truthiness. It is hard to override subgroup and split to return empty
ExceptionGroups though, without hacking with internals.
Let's turn this on its head:
- what specific harm comes from giving EGs container truthiness for size
testing?
- what specific harm comes from returning an empty EG from split on no
match instead of None?
- what specific harm comes from supporting iteration with a caveat about
metadata in the docstring, and maybe a recommendation of subgroup?
- if I wanted to override subgroup and split to not return None, is that
even possible with the current ? i.e. can I make a clean metadata
preserved EG from an empty list? For example:
eg2 = eg.subgroup(lambda e: False)
Does that get me None, or an empty group? If the latter, I can roll my
own subclass with my desired features. If not, I can't AFAICT.
EGs _have_ a .errors attribute which has all these aspects, why not
expand it to the class as a whole?
You seem very happen to implement 80% of what I want using callbacks
(lambda e: ...), but I find explicit iteration much easier to read. I
rarely use filter() for example, and often prefer a generator expression
of list comprehension.
Cheers,
Cameron Simpson <c...@cskk.id.au>
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at
https://mail.python.org/archives/list/python-dev@python.org/message/46SOJ67QFCC4KEWJOKK53RSV3ND2YBBX/
Code of Conduct: http://python.org/psf/codeofconduct/
