subject:"\[Python\-Dev\] Re\: PEP 654 \-\- Exception Groups and except\* \: request for feedback for SC submission"

On Wed, Mar 3, 2021 at 10:39 PM Greg Ewing 
wrote:

> On 4/03/21 5:37 am, Paul Moore wrote:
> > frameworks and libraries typically have to interact with other users'
> > code, and there the contract has changed from "do what you like in
> > your code and I'll cope" to "do what you like in your code as long as
> > you don't let an exception group escape, and I'll cope"... And I have
> > to change *my* code to get the old contract back.
>
> Seems to me this whole issue would go away if the ordinary
> except statement were to look inside ExceptionGroups.
>
> In other words, the only difference between except and
> except* would be that multiple except* clauses can be run,
> whereas only one except clause will run (the first one that
> matches something in the ExceptionGroup).
>
> Is there any good reason not to do things that way?
>

That's an interesting idea.

Do you mean that one exception gets handled and the rest of the group is
reraised? Or discarded?

The value of sys.exc_info() (and the e in "except T as e:") needs to be a
single naked exception. So if there is more than one match in the group we
would need to pick one (let's say the first in DFS order).

If we do this, then we have this situation. Before ExceptionGroups, you got
to choose which of the exceptions you have is the most important, and you
raised only that one. Now you raise a bunch of them and the order of the
except clauses in caller's code determines which one of them counts and
which ones are discarded. What do you make of that?

Irit
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/O72A2J4HRHEKYBRL5NRL4YEUQVSL3TZQ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 654 -- Exception Groups and except* : request for feedback for SC submission

2021-03-03 Thread Greg Ewing


On 4/03/21 5:37 am, Paul Moore wrote:

frameworks and libraries typically have to interact with other users'
code, and there the contract has changed from "do what you like in
your code and I'll cope" to "do what you like in your code as long as
you don't let an exception group escape, and I'll cope"... And I have
to change *my* code to get the old contract back.


Seems to me this whole issue would go away if the ordinary
except statement were to look inside ExceptionGroups.

In other words, the only difference between except and
except* would be that multiple except* clauses can be run,
whereas only one except clause will run (the first one that
matches something in the ExceptionGroup).

Is there any good reason not to do things that way?

--
Greg
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/NDIO4533M4OQZZWT65EKPPIXZU6ZJQUY/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 654 -- Exception Groups and except* : request for feedback for SC submission

2021-03-03 Thread Guido van Rossum

On Wed, Mar 3, 2021 at 11:26 AM Irit Katriel via Python-Dev <
python-dev@python.org> wrote:

>
>
> On Wed, Mar 3, 2021 at 6:57 PM Paul Moore  wrote:
>
>> Sorry, I keep thinking I've finished and you keep making interesting
>> points :-)
>>
>> On Wed, 3 Mar 2021 at 17:01, Irit Katriel 
>> wrote:
>>
>> > Raising an ExceptionGroup is an API change.  If you call APIs that say
>> they will raise ExceptionGroups you need to update your code accordingly.
>> If a library doesn't document that it raises ExceptionGroups and then one
>> of those escapes, then that library has a bug. Just like with any other
>> exception type.
>>
>> In my experience, libraries don't document what exceptions they raise
>> very well. You can call that a bug, but it's a fact of life, I'm
>> afraid. The problem here isn't so much that the library code now
>> raises an exception that it used not to raise, but rather that *the
>> user hitting Ctrl-C* can now result in a different exception surfacing
>> in my code than it used to. Libraries don't re-wrap KeyboardInterrupt,
>> as you pointed out in a previous response, so I can currently write
>> code that traps KeyboardInterrupt, safe in the knowledge that by doing
>> so I'll handle that user action properly. But with PEP 654, libraries
>> might well (indeed, some libraries almost certainly will) start
>> wrapping KeyboardInterrupt in an exception group. That's a backward
>> incompatible change from the perspective of my code's interaction with
>> the user, and I need to re-code my application to deal with it (and
>> worse still, writing that new code in a way that is portable between
>> versions is not particularly straightforward).
>>
>
> This is also true for MemoryError, and many other errors. What makes
> KeyboardInterrupt special?
>

Users get really grumpy if they can't stop a runaway program with ^C --
they want it to either terminate the script or app, or go back into the
toplevel REPL if there is one. And users write runaway code all the time.

asyncio in particular catches BaseException but excludes (reraises)
KeyboardInterrupt and SystemExit.

OTOH MemoryError is rarely a problem -- and Linux for example kills the
process rather than failing to allocate more memory, so you won't even get
an exception, so how it's treated by the exception machinery doesn't
matter. The MemoryErrors you do get tend to be recoverable, as in
'x'*1.

-- 
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/4DDHVB3VSWEKPZABKO5C3LDFCPTQKRIN/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 654 -- Exception Groups and except* : request for feedback for SC submission

Hi Sven,

This is all about the popularity of "except Exception".

Look at the hierarchy of builtin exceptions:
https://docs.python.org/3/library/exceptions.html#exception-hierarchy

Most exceptions are subclasses of Exception. There are a few which are not,
because they typically mean "this process should exit" so you should not
usually handle them in your code. People use "except Exception" as a way to
"catch almost everything, but not the critical stuff like SystemExit".

If we make ExceptionGroup be a BaseException, then "except Exception"
doesn't catch it. So we make it a subclass of Exception. But then we can't
make it wrap things like SystemExit, which people expect will not be caught
by "except Exception".  So we add BaseExceptionGroup, which is a subclass
of BaseException and therefore is not caught by "except Exception", so it
can wrap SystemExit.

Why is the choice automated?  Because it can be. You look at what you're
wrapping. If it's all subclasses of Exception, then it can be
ExceptionGroup. If there are BaseExceptions, then it needs to be
BaseExceptionGroup. There is no reason to ever do anything else.

I hope that makes sense.

On Wed, Mar 3, 2021 at 7:32 PM Sven R. Kunze  wrote:

> Hey Irit,
>
> find my 3 answers below:
>
> On 03.03.21 13:17, Irit Katriel wrote:
> > Hi Sven,
> >
> > I like your formatting suggestion, thanks. I will do something like that.
>
> You're welcome.
>
> >
> > I'm not sure I understand your question. ExceptionGroup is a subclass
> > of Exception (which is a subclass of BaseException). So ExceptionGroup
> > is caught by "except Exception" or "except BaseException".
>
> 1) So I understand "try-except BaseException" (cf. concurrent.futures)
> will work without fixes (i.e. produce the same results).
>
>
> > BaseExceptionGroup is a subclass only of BaseException so it is caught
> > by "except BaseException" but not "except Exception". And
> > ExceptionGroup is allowed to wrap only Exceptions while BaseException
> > can wrap Exceptions and and BaseExceptions. Makes sense?
>
>
> 2) Can you add motivating examples for "BaseExceptionGroup vs
> ExceptionGroup" in the PEP? Right now, I only see what the consequences
> are but not why it was done this way.
>
> 3) Can you explain (and show the reasoning behind) this automatic choice
> in the PEP? Sounds a bit like hidden magic to me.
>
>
> Referring to: "The difference between them is that ExceptionGroup can
> only wrap Exception subclasses while BaseExceptionGroup can wrap any
> BaseException subclass. A factory method that inspects the nested
> exceptions and selects between ExceptionGroup and BaseExceptionGroup
> makes the choice automatic."
>
>
> Best
> Sven
>
>
> PS:
>
> the reason why I was a bit puzzled by the
> BaseExceptionGroup/ExceptionGroup issue is that:
> - if it doesn't matter (so we do it automatically, because we do not
> want to bother anybody), why do we need ExceptionGroup at all,
> BaseExceptionGroup seems more flexible?
> - if it does matter, why is the choice automatic and what if it was the
> wrong choice?
>
>
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/JZJFDTZKPIY477EAVWHWEHLX2XFEAXJK/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 654 -- Exception Groups and except* : request for feedback for SC submission

2021-03-03 Thread Sven R. Kunze

Hey Irit,

find my 3 answers below:

On 03.03.21 13:17, Irit Katriel wrote:

Hi Sven,

I like your formatting suggestion, thanks. I will do something like that.

You're welcome.

I'm not sure I understand your question. ExceptionGroup is a subclass
of Exception (which is a subclass of BaseException). So ExceptionGroup
is caught by "except Exception" or "except BaseException".

1) So I understand "try-except BaseException" (cf. concurrent.futures)
will work without fixes (i.e. produce the same results).

BaseExceptionGroup is a subclass only of BaseException so it is caught
by "except BaseException" but not "except Exception". And
ExceptionGroup is allowed to wrap only Exceptions while BaseException
can wrap Exceptions and and BaseExceptions. Makes sense?

2) Can you add motivating examples for "BaseExceptionGroup vs
ExceptionGroup" in the PEP? Right now, I only see what the consequences
are but not why it was done this way.

3) Can you explain (and show the reasoning behind) this automatic choice
in the PEP? Sounds a bit like hidden magic to me.

Referring to: "The difference between them is that ExceptionGroup can
only wrap Exception subclasses while BaseExceptionGroup can wrap any
BaseException subclass. A factory method that inspects the nested
exceptions and selects between ExceptionGroup and BaseExceptionGroup
makes the choice automatic."

Best
Sven

PS:

the reason why I was a bit puzzled by the
BaseExceptionGroup/ExceptionGroup issue is that:
- if it doesn't matter (so we do it automatically, because we do not
want to bother anybody), why do we need ExceptionGroup at all,
BaseExceptionGroup seems more flexible?
- if it does matter, why is the choice automatic and what if it was the
wrong choice?

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at
https://mail.python.org/archives/list/python-dev@python.org/message/4TEOPEX7EELLZUUFAODZ6XU7HF7FUE54/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 654 -- Exception Groups and except* : request for feedback for SC submission

On Wed, Mar 3, 2021 at 6:57 PM Paul Moore  wrote:

> Sorry, I keep thinking I've finished and you keep making interesting
> points :-)
>
> On Wed, 3 Mar 2021 at 17:01, Irit Katriel 
> wrote:
>
> > Raising an ExceptionGroup is an API change.  If you call APIs that say
> they will raise ExceptionGroups you need to update your code accordingly.
> If a library doesn't document that it raises ExceptionGroups and then one
> of those escapes, then that library has a bug. Just like with any other
> exception type.
>
> In my experience, libraries don't document what exceptions they raise
> very well. You can call that a bug, but it's a fact of life, I'm
> afraid. The problem here isn't so much that the library code now
> raises an exception that it used not to raise, but rather that *the
> user hitting Ctrl-C* can now result in a different exception surfacing
> in my code than it used to. Libraries don't re-wrap KeyboardInterrupt,
> as you pointed out in a previous response, so I can currently write
> code that traps KeyboardInterrupt, safe in the knowledge that by doing
> so I'll handle that user action properly. But with PEP 654, libraries
> might well (indeed, some libraries almost certainly will) start
> wrapping KeyboardInterrupt in an exception group. That's a backward
> incompatible change from the perspective of my code's interaction with
> the user, and I need to re-code my application to deal with it (and
> worse still, writing that new code in a way that is portable between
> versions is not particularly straightforward).
>

This is also true for MemoryError, and many other errors. What makes
KeyboardInterrupt special?


>
> > For older Pythons you would have to do something like
> >
> > except KeyboardInterrupt:
> >...
> > except BaseExceptionGroup:  # some stub type in old versions
> ># inspect the contents
> ># if there's a KeyboardInterrupt do what you need to do
> ># reraise the rest
>
> I'd be inclined to suggest that a complete version of this should be
> included in the "Backward compatibility" part of the PEP, as I
> honestly don't really know how I'd write that without doing more
> research. But such an example would make the KeyboardInterrupt case
> seem more important than it is. Maybe if it's framed as "how to write
> calling code that's compatible with older versions of Python but still
> able to handle called code potentially raising exceptions that you
> need to trap as part of a group", that would be a useful general
> example.


> Or maybe it's not actually something that will be needed that often.
> I'm not sure - I'm trying to think in terms of pip, where we can't use
> new features in our own code until we drop support for older versions,
> but we might potentially rely on a library that uses exception
> grouping internally on versions where it's available (and that code
> lets those exception groups escape). It feels like a stretch to claim
> this is particularly likely, but conversely it's something I can
> easily imagine *could* happen...



If a library starts raising ExceptionGroups from version 3.X it should
probably do that from a new API so people won't have to worry about it just
because they are bumping Python version. So I think the cross-version issue
is in the case of "I'm calling a user function and I don't know what it is
or what it raises", or the "I just want to write all exceptions to the log
and ignore them". So this is the "except Exception" case, which will work.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/BF6R7Y6UAPSAFZYBQ2GKAMO7RJ6FYDAY/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 654 -- Exception Groups and except* : request for feedback for SC submission

Sorry, I keep thinking I've finished and you keep making interesting points :-)

On Wed, 3 Mar 2021 at 17:01, Irit Katriel  wrote:

> Raising an ExceptionGroup is an API change.  If you call APIs that say they 
> will raise ExceptionGroups you need to update your code accordingly. If a 
> library doesn't document that it raises ExceptionGroups and then one of those 
> escapes, then that library has a bug. Just like with any other exception type.

In my experience, libraries don't document what exceptions they raise
very well. You can call that a bug, but it's a fact of life, I'm
afraid. The problem here isn't so much that the library code now
raises an exception that it used not to raise, but rather that *the
user hitting Ctrl-C* can now result in a different exception surfacing
in my code than it used to. Libraries don't re-wrap KeyboardInterrupt,
as you pointed out in a previous response, so I can currently write
code that traps KeyboardInterrupt, safe in the knowledge that by doing
so I'll handle that user action properly. But with PEP 654, libraries
might well (indeed, some libraries almost certainly will) start
wrapping KeyboardInterrupt in an exception group. That's a backward
incompatible change from the perspective of my code's interaction with
the user, and I need to re-code my application to deal with it (and
worse still, writing that new code in a way that is portable between
versions is not particularly straightforward).

> For older Pythons you would have to do something like
>
> except KeyboardInterrupt:
>...
> except BaseExceptionGroup:  # some stub type in old versions
># inspect the contents
># if there's a KeyboardInterrupt do what you need to do
># reraise the rest

I'd be inclined to suggest that a complete version of this should be
included in the "Backward compatibility" part of the PEP, as I
honestly don't really know how I'd write that without doing more
research. But such an example would make the KeyboardInterrupt case
seem more important than it is. Maybe if it's framed as "how to write
calling code that's compatible with older versions of Python but still
able to handle called code potentially raising exceptions that you
need to trap as part of a group", that would be a useful general
example.

Or maybe it's not actually something that will be needed that often.
I'm not sure - I'm trying to think in terms of pip, where we can't use
new features in our own code until we drop support for older versions,
but we might potentially rely on a library that uses exception
grouping internally on versions where it's available (and that code
lets those exception groups escape). It feels like a stretch to claim
this is particularly likely, but conversely it's something I can
easily imagine *could* happen...

Paul
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/ITBRA7QAU64BN3XKBYV6G35NXKK2ESXX/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 654 -- Exception Groups and except* : request for feedback for SC submission

On Wed, Mar 3, 2021 at 4:37 PM Paul Moore  wrote:

>
> Similar to the argument for "except Exception". Applications that trap
> KeyboardInterrupt so that they can exit cleanly without an ugly
> traceback will no longer trap *all* keyboard interrupts, as they could
> miss grouped ones.
>

See below.


>
> If we accept that grouped exceptions should never escape out of a
> well-defined context, then this wouldn't be such an issue. But there's
> nothing in the PEP that enforces that, and there *is* code that needs
> to be prepared for "any sort of result". It's the except Exception
> argument again.
>
> So code that wants to exit cleanly in the face of Ctrl-C will need to
> be rewritten from:
>
> try:
> main()
> except KeyboardInterrupt:
> print("User interrupted the program. Exiting")
> sys.exit(1)
>
> to:
>
> try:
> try:
> main()
> except KeyboardInterrupt:
> print("User interrupted the program. Exiting")
> sys.exit(1)
> except *KeyboardInterrupt:
> print("User interrupted the program. Exiting")
> sys.exit(1)
>

It suffices to do

try:
main()
  except *KeyboardInterrupt:
print("User interrupted the program. Exiting")
sys.exit(1)

because "except *T" catches Ts as well.



>
> Did I miss an easier way of writing this code? And worse, how would I
> write it so that it was portable between Python 3.9 and later versions
> (which is a common requirement for library code - admittedly library
> code wouldn't normally be doing this sort of top-level trap, but it
> could just as easily be "catch Ctrl-C and do a bit of tidy-up and
> re-raise").
>

For older Pythons you would have to do something like

except KeyboardInterrupt:
   ...
except BaseExceptionGroup:  # some stub type in old versions
   # inspect the contents
   # if there's a KeyboardInterrupt do what you need to do
   # reraise the rest



>
> > I think the only reason you're comfortable with having to select between
> the exceptions that were raised and discard some of them is because that's
> where we are today. The PEP lists several standard library and other APIs
> that discard errors because they need to pick one. That's what we're trying
> to fix.
>
> Maybe. But I'm not looking at it as being "comfortable" with the
> current situation, but rather as "I don't use any of these new
> features, why am I having to change my code to accommodate stuff I
> don't use?" If I own the full stack, that's not an issue, but
> frameworks and libraries typically have to interact with other users'
> code, and there the contract has changed from "do what you like in
> your code and I'll cope" to "do what you like in your code as long as
> you don't let an exception group escape, and I'll cope"... And I have
> to change *my* code to get the old contract back.
>

 "except Exception"/"except BaseException" is the special case of "I don't
know what I'm calling and I want to catch everything".  And that (to
repeat, just in case) will work as you expect.

If you are actually handling exceptions selectively, then I can break you
already in 3.9 just by raising a different exception type to the one you
are catching. How is this different?

Raising an ExceptionGroup is an API change.  If you call APIs that say they
will raise ExceptionGroups you need to update your code accordingly. If a
library doesn't document that it raises ExceptionGroups and then one of
those escapes, then that library has a bug. Just like with any other
exception type.


>
> But it's a small point in the wider scheme of things, and I'm not
> going to labour the point any more. Thanks for listening and taking
> the time to reply.


It's an important one though. Thanks for asking good questions.

Irit
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/SDHZAEUNHMOC2NBCFQZEDTHZ4B6QGEXO/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 654 -- Exception Groups and except* : request for feedback for SC submission

2021-03-03 Thread Joao S. O. Bueno

Just to add +1 for Paul's concerns.

Even though ExceptionGroups "are not supposed" to not
leak into caller code, don't mean they "won't". Making  "except Exception"
catch them would make this part a non issue, and the
feature looks great otherwise.

On Wed, 3 Mar 2021 at 13:44, Paul Moore  wrote:

> On Wed, 3 Mar 2021 at 14:59, Irit Katriel 
> wrote:
> >> 2. Given the above, why even have a means of grouping BaseExceptions
> >> at all? Why not just have ExceptionGroup that can only catch instances
> >> of Exception?
> >
> > Because the IGotInterrupted alternative involves wrapping a
> BaseException by an Exception, which is not something we should push people
> into doing (it's not that different from allowing ExceptionGroup to wrap
> BaseExceptions directly).
>
> That's a fair point.
>
> > What's the harm/complication in offering a
> BaseExceptionGroup(BaseException) in addition to
> ExceptionGroup(BaseExceptionGroup, Exception)?
>
> Similar to the argument for "except Exception". Applications that trap
> KeyboardInterrupt so that they can exit cleanly without an ugly
> traceback will no longer trap *all* keyboard interrupts, as they could
> miss grouped ones.
>
> If we accept that grouped exceptions should never escape out of a
> well-defined context, then this wouldn't be such an issue. But there's
> nothing in the PEP that enforces that, and there *is* code that needs
> to be prepared for "any sort of result". It's the except Exception
> argument again.
>
> So code that wants to exit cleanly in the face of Ctrl-C will need to
> be rewritten from:
>
> try:
> main()
> except KeyboardInterrupt:
> print("User interrupted the program. Exiting")
> sys.exit(1)
>
> to:
>
> try:
> try:
> main()
> except KeyboardInterrupt:
> print("User interrupted the program. Exiting")
> sys.exit(1)
> except *KeyboardInterrupt:
> print("User interrupted the program. Exiting")
> sys.exit(1)
>
> Did I miss an easier way of writing this code? And worse, how would I
> write it so that it was portable between Python 3.9 and later versions
> (which is a common requirement for library code - admittedly library
> code wouldn't normally be doing this sort of top-level trap, but it
> could just as easily be "catch Ctrl-C and do a bit of tidy-up and
> re-raise").
>
> > I think the only reason you're comfortable with having to select between
> the exceptions that were raised and discard some of them is because that's
> where we are today. The PEP lists several standard library and other APIs
> that discard errors because they need to pick one. That's what we're trying
> to fix.
>
> Maybe. But I'm not looking at it as being "comfortable" with the
> current situation, but rather as "I don't use any of these new
> features, why am I having to change my code to accommodate stuff I
> don't use?" If I own the full stack, that's not an issue, but
> frameworks and libraries typically have to interact with other users'
> code, and there the contract has changed from "do what you like in
> your code and I'll cope" to "do what you like in your code as long as
> you don't let an exception group escape, and I'll cope"... And I have
> to change *my* code to get the old contract back.
>
> But it's a small point in the wider scheme of things, and I'm not
> going to labour the point any more. Thanks for listening and taking
> the time to reply.
>
> Paul
>
> Paul
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/WSUEOGDCBBOZ7PCQGUCXKIZEZ7RK34LK/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/CYV3RDTUZM2LBF6NLCHA6PTJMWG4CKF6/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 654 -- Exception Groups and except* : request for feedback for SC submission

On Wed, 3 Mar 2021 at 14:59, Irit Katriel  wrote:
>> 2. Given the above, why even have a means of grouping BaseExceptions
>> at all? Why not just have ExceptionGroup that can only catch instances
>> of Exception?
>
> Because the IGotInterrupted alternative involves wrapping a BaseException by 
> an Exception, which is not something we should push people into doing (it's 
> not that different from allowing ExceptionGroup to wrap BaseExceptions 
> directly).

That's a fair point.

> What's the harm/complication in offering a BaseExceptionGroup(BaseException) 
> in addition to ExceptionGroup(BaseExceptionGroup, Exception)?

Similar to the argument for "except Exception". Applications that trap
KeyboardInterrupt so that they can exit cleanly without an ugly
traceback will no longer trap *all* keyboard interrupts, as they could
miss grouped ones.

If we accept that grouped exceptions should never escape out of a
well-defined context, then this wouldn't be such an issue. But there's
nothing in the PEP that enforces that, and there *is* code that needs
to be prepared for "any sort of result". It's the except Exception
argument again.

So code that wants to exit cleanly in the face of Ctrl-C will need to
be rewritten from:

try:
main()
except KeyboardInterrupt:
print("User interrupted the program. Exiting")
sys.exit(1)

to:

try:
try:
main()
except KeyboardInterrupt:
print("User interrupted the program. Exiting")
sys.exit(1)
except *KeyboardInterrupt:
print("User interrupted the program. Exiting")
sys.exit(1)

Did I miss an easier way of writing this code? And worse, how would I
write it so that it was portable between Python 3.9 and later versions
(which is a common requirement for library code - admittedly library
code wouldn't normally be doing this sort of top-level trap, but it
could just as easily be "catch Ctrl-C and do a bit of tidy-up and
re-raise").

> I think the only reason you're comfortable with having to select between the 
> exceptions that were raised and discard some of them is because that's where 
> we are today. The PEP lists several standard library and other APIs that 
> discard errors because they need to pick one. That's what we're trying to fix.

Maybe. But I'm not looking at it as being "comfortable" with the
current situation, but rather as "I don't use any of these new
features, why am I having to change my code to accommodate stuff I
don't use?" If I own the full stack, that's not an issue, but
frameworks and libraries typically have to interact with other users'
code, and there the contract has changed from "do what you like in
your code and I'll cope" to "do what you like in your code as long as
you don't let an exception group escape, and I'll cope"... And I have
to change *my* code to get the old contract back.

But it's a small point in the wider scheme of things, and I'm not
going to labour the point any more. Thanks for listening and taking
the time to reply.

Paul

Paul
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/WSUEOGDCBBOZ7PCQGUCXKIZEZ7RK34LK/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 654 -- Exception Groups and except* : request for feedback for SC submission

Hi Paul,

On Wed, Mar 3, 2021 at 2:20 PM Paul Moore  wrote:

>
> 1. Having now read the PEP, I don't actually see a use case for
> grouping BaseExceptions. Why would anyone catch KeyboardInterrupt or
> SystemExit and wrap them in a BaseExceptionGroup anyway? It seems to
> me that the right thing to do, even in async or parallel code, is to
> just propogate the KeyboardInterrupt/SystemExit up to the main
> program. Losing a ValueError that happened at the exact same time as
> the user pressed Ctrl-C seems like it's not going to be a problem in
> practice...
>

It is possible that your program wants to do something other than just
terminate when it gets a KeyboardInterrupt. Otherwise why does the
interpreter bother propagating the KeyboardInterrupt rather than just
terminate the program there and then?

And the ValueError you lost may be some other error type that involves
cleanup.

If you do this:

try:
async.gather(...)  # raises BaseExceptionGroup( DoCleanup(),
KeyboardInterrupt())
except * DoCleanup:
do_cleanup() # log stuff, send notifications, close sockets, whatever

That will do the cleanup in addition to propagating a
BaseExceptionGroup(KeyboardInterrupt())

Also, KeyboardInterrupt/SystemExit are not the only BaseExceptions.

> 2. Given the above, why even have a means of grouping BaseExceptions
> at all? Why not just have ExceptionGroup that can only catch instances
> of Exception?
>

Because the IGotInterrupted alternative involves wrapping a BaseException
by an Exception, which is not something we should push people into doing
(it's not that different from allowing ExceptionGroup to wrap
BaseExceptions directly).

What's the harm/complication in offering a
BaseExceptionGroup(BaseException) in addition to
ExceptionGroup(BaseExceptionGroup, Exception)?

I think the only reason you're comfortable with having to select between
the exceptions that were raised and discard some of them is because that's
where we are today. The PEP lists several standard library and other APIs
that discard errors because they need to pick one. That's what we're trying
to fix.

Irit
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/7HHNSKCF4DEBAWR2PUAVQ2QXNSJMWMHS/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 654 -- Exception Groups and except* : request for feedback for SC submission

On Wed, 3 Mar 2021 at 12:37, Irit Katriel  wrote:
> This is covered the PEP, but TL;DR:
> If we could make "except KeyboardInterrupt" catch 
> BaseExceptionGroup([KeyboardInterrupt]) in a reasonably backwards compatible 
> way then we wouldn't need except*.
[...]
> As you noted, no library is under any obligation to wrap KeyboardInterrupts 
> into the exception groups it raises. You may decide it's a bad idea and not 
> do it.  What we are discussing here is what the language should make 
> possible. We agree that wrapping a BaseException by an Exception is something 
> we should definitely block. When it's wrapping a BaseException by another, 
> new BaseException type, in my view that's ok. You may have a bug where you 
> don't catch an exception you want to catch, because you are using a new API 
> incorrectly. But you won't have bugs where you swallow an exception that you 
> didn't swallow before.

Thanks for the explanation. I understand your point here, and I see
what you're saying. But I have a couple of questions still:

1. Having now read the PEP, I don't actually see a use case for
grouping BaseExceptions. Why would anyone catch KeyboardInterrupt or
SystemExit and wrap them in a BaseExceptionGroup anyway? It seems to
me that the right thing to do, even in async or parallel code, is to
just propogate the KeyboardInterrupt/SystemExit up to the main
program. Losing a ValueError that happened at the exact same time as
the user pressed Ctrl-C seems like it's not going to be a problem in
practice...
2. Given the above, why even have a means of grouping BaseExceptions
at all? Why not just have ExceptionGroup that can only catch instances
of Exception?

If there really *was* a low-level case where some code absolutely had
to (temporarily) group KeyboardInterrupt, for example, it could be
temporarily wrapped:

class IGotInterrupted(Exception):
def __init__(self, exc):
self.exc = exc

def might_be_interrupted():
try:
critical_stuff()
except KeyboardInterrupt as exc:
raise IGotInterrupted(exc)

def funky_parallel_stuff():
try:
do_in_parallel(might_be_interrupted)
except *IGotInterrupted as e:
# Please excuse the fact that I don't know the
# correct way to re-raise here. Might need raise from.
raise e.exc
# Do we also need to consider an *ungrouped* IGotInterrupted?
# That's for the library to decide, and is why not letting it
escape is a good thing...

That would be appropriate for really low-level code, and it would be a
fairly obvious anti-pattern for an exception class like
IGotInterrupted to "escape" so that user code would ever see it.

I guess the argument for grouping BaseExceptions is "why make it
necessary to do complicated wrapping like this?" To which my answer is
"because it should almost never be something you do, and so why add
support for it when it can be done *without* needing explicit support
in the PEP?" And by not having stdlib support for wrapping
BaseExceptions, we're signalling that we don't think people should be
doing it casually...

Paul
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/5LVHBCIWOLYVUYAMFYIZAPEDV7JUVBYS/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 654 -- Exception Groups and except* : request for feedback for SC submission

Hi Paul,

I agree that your condition (1) is essential, while condition (2) is
desirable.   (I explained in the email you replied to why I think 2 is less
critical than 1).

The current state of the PEP is that ExceptionGroup does not wrap
BaseExceptions and is caught by "except Exception", while
BaseExceptionGroup wraps BaseException and is only caught by "except
BaseException" but not "except Exception".

This is covered the PEP, but TL;DR:
If we could make "except KeyboardInterrupt" catch
BaseExceptionGroup([KeyboardInterrupt]) in a reasonably backwards
compatible way then we wouldn't need except*.

For example, suppose that this:

try:
raise BaseExceptionGroup([KeyboardInterrupt()])
except  KeyboardInterrupt:
   email_the_boss()

worked as you suggest. Then what should this do?  Send two emails to the
boss?

try:
raise BaseExceptionGroup([KeyboardInterrupt(), KeyboardInterrupt() ])
except  KeyboardInterrupt:
   email_the_boss()

As you noted, no library is under any obligation to wrap KeyboardInterrupts
into the exception groups it raises. You may decide it's a bad idea and not
do it.  What we are discussing here is what the language should make
possible. We agree that wrapping a BaseException by an Exception is
something we should definitely block. When it's wrapping a BaseException by
another, new BaseException type, in my view that's ok. You may have a bug
where you don't catch an exception you want to catch, because you are using
a new API incorrectly. But you won't have bugs where you swallow an
exception that you didn't swallow before.

Irit

On Wed, Mar 3, 2021 at 8:30 AM Paul Moore  wrote:

> On Tue, 2 Mar 2021 at 21:46, Irit Katriel via Python-Dev
>  wrote:
> > As an aside - I found it interesting that the option to wrap
> BaseException instances by an Exception, which came up a couple of times in
> this thread, didn't generate any anxiety.
>
> Probably because it wasn't clear that was ever being proposed... (or
> at least the implication wasn't obvious - presumably this is somehow
> related to BaseExceptions being accumulated in ExceptionGroups?) :-(
>
> I would consider it essential that if someone hits Ctrl-C and that
> generates a KeyboardInterrupt, then:
>
> 1. That KeyboardInterrupt will *not* get caught by exception handlers
> only interested in Exception instances
> 2. That KeyboardInterrupt *will* get caught by any handler that does
> an explicit `except KeyboardInterrupt` or an `except BaseException`.
>
> To me, that's pretty much essential to correct Ctrl-C handling in any
> app (never ignore a user's Ctrl-C and always exit cleanly if one is
> raised).
>
> That might mean that BaseException instances shouldn't be "groupable",
> but I don't want to comment on that until I've properly read the PEP
> (I've skimmed it now, but only superficially). At a minimum, I'd
> consider it a bug for library code to manually wrap a
> KeyboardInterrupt in an exception group (just like code that catches
> KeyboardInterrupt and re-raises it as a ValueError would be today).
>
> Paul
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/5M6YSXXZWGUHE5VOFLJWKRSUY4TIXLTH/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 654 -- Exception Groups and except* : request for feedback for SC submission

Hi Sven,

I like your formatting suggestion, thanks. I will do something like that.

I'm not sure I understand your question. ExceptionGroup is a subclass of
Exception (which is a subclass of BaseException). So ExceptionGroup is
caught by "except Exception" or "except BaseException". BaseExceptionGroup
is a subclass only of BaseException so it is caught by "except
BaseException" but not "except Exception". And ExceptionGroup is allowed to
wrap only Exceptions while BaseException can wrap Exceptions and and
BaseExceptions. Makes sense?

Irit


On Tue, Mar 2, 2021 at 7:25 PM Sven R. Kunze  wrote:

> Just to be on the safe side here, I would want to asked a little bit
> further.
>
>
> On 02.03.21 12:22, Irit Katriel wrote:
>
>
> 1) What is the right "except pattern" when ExceptionGroup is introduced
>> for the use cases described above (remote or concurrent python processes)?
>>
>
> If you want to catch the whole ExceptionGroup and format its traceback,
> then you can just do "except ExceptionGroup as eg" and then
> traceback.print_exception(eg).
>
> The except* syntax is for when you want to handle only certain types of
> exceptions from the group, selectively.
>
> Just to make sure, I understand this completely.
>
>
> In order to make it more tangible here is an example from the stdlib:
>
>
> https://github.com/python/cpython/blob/bf2e7e55d7306b1e2fce7dce767e8df5ff42cf1c/Lib/concurrent/futures/process.py#L215
>
> As you can see, BaseException is caught multiple times as the only
> except-clause. _sendback_result(...) then used to transfer the
> return_val/exception back to parent process.
>
>
> How is the supposed way of handling a raised ExceptionGroup?
>
>
> a) will the existing code simply work as it?
> b) if not what are the changes to this lib specifically as a blueprint
> example for others
>
>
> Reading from the other subthread for this PEP, I am not 100% sure now that
> "except BaseException" will suffice if ExceptionGroup inherits from
> Exception instead of BaseException because it seems that ExceptionGroup
> somehow is handled specially. So, why I try to approach this very issue
> with existing code. Once that is clear, it should be easy for everybody to
> apply the same pattern to their code. Here is the copy from github:
>
>
> try:
> r = call_item.fn(*call_item.args, **call_item.kwargs)
> except BaseException as e:
> exc = _ExceptionWithTraceback(e, e.__traceback__)
> _sendback_result(result_queue, call_item.work_id, exception=exc)
> else:
> _sendback_result(result_queue, call_item.work_id, result=r)
> del r
>
>
> Maybe even _sendback_result could benefit from using ExceptionGroup
> itself. You can see there another "except BaseException" in case of an
> error. But that's maybe a different topic.
>
>
>
>> 2) What is the recommended approach of printing the traceback potentially
>> incorporating multiple tracebacks - I couldn't find it in the PEP and
>> tracebacks are a really neat tool for error hunting.
>>
>
> Part of the proposal of the PEP is that we teach the builtin traceback
> formatting code to display ExceptionGroups with all that information.
>
>
> As long as there's the possibility to receive the usual (two-line)
> entry-based list, I guess that is enough for every one to work with it
> (e.g. hiding unnecessary tb entries).
>
>
> The reference implementation has this, and the examples in the PEP were
> produced with it. Some of the examples (the early ones) show exceptions
> that were never raised so there is no traceback. But if you scroll down to
> the later examples, you see the output for exceptions with tracebacks,
> cause, context etc.
>
>
> Ah I see them now. Thank you. :-)
>
>
> We didn't put too much effort into making the traceback output pretty, so
> at the moment it's a draft. If there is an ascii artist out there who has
> suggestions on improving this, that would be great.
>
>
> Well, yes. It's not really pretty. I would recommend a tree-based solution
> such as the following:
>
>
> Traceback (most recent call last):
>   File "", line 3, in 
> ExceptionGroup
> |
> +---+   Traceback (most recent call last):
> |   | File "", line 3, in 
> |   |   ExceptionGroup: eg
> |   |   |
> |   |   +---+ ValueError: a
> |   |
> |   |   During handling of the above exception, another exception occurred:
> |   |
> |   |   Traceback (most recent call last):
> |   | File "", line 5, in 
> |   |   KeyError: 'x'
> |
> |
> +---+   Traceback (most recent call last):
> |   | File "", line 3, in 
> |   |   ExceptionGroup: eg
> |   |   |
> |   |   +---+ TypeError: b
>
>
> I used a similar pattern for the remote-execution lib and received good
> user feedback (even requesting color encoding for each layer of the tree
> (not the text), so people know what belongs together). Besides colors, I
> used https://en.wikipedia.org/wiki/Box-drawing_character but I guess
> pipes, dashes and pluses could suffice for CPython.
>
>
> One other remark from my side here: in the

[Python-Dev] Re: PEP 654 -- Exception Groups and except* : request for feedback for SC submission

On Tue, 2 Mar 2021 at 21:46, Irit Katriel via Python-Dev
 wrote:
> As an aside - I found it interesting that the option to wrap BaseException 
> instances by an Exception, which came up a couple of times in this thread, 
> didn't generate any anxiety.

Probably because it wasn't clear that was ever being proposed... (or
at least the implication wasn't obvious - presumably this is somehow
related to BaseExceptions being accumulated in ExceptionGroups?) :-(

I would consider it essential that if someone hits Ctrl-C and that
generates a KeyboardInterrupt, then:

1. That KeyboardInterrupt will *not* get caught by exception handlers
only interested in Exception instances
2. That KeyboardInterrupt *will* get caught by any handler that does
an explicit `except KeyboardInterrupt` or an `except BaseException`.

To me, that's pretty much essential to correct Ctrl-C handling in any
app (never ignore a user's Ctrl-C and always exit cleanly if one is
raised).

That might mean that BaseException instances shouldn't be "groupable",
but I don't want to comment on that until I've properly read the PEP
(I've skimmed it now, but only superficially). At a minimum, I'd
consider it a bug for library code to manually wrap a
KeyboardInterrupt in an exception group (just like code that catches
KeyboardInterrupt and re-raises it as a ValueError would be today).

Paul
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/S5SPIFHGWASEGKBITWPDDRXOU33NLG46/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 654 -- Exception Groups and except* : request for feedback for SC submission

2021-03-02 Thread Guido van Rossum

On Tue, Mar 2, 2021 at 1:53 PM Irit Katriel via Python-Dev <
python-dev@python.org> wrote:

> [...]
> I can't imagine people building deep trees of exceptions in practice (at
> least not on purpose).  But some nesting may show up naturally, and we need
> to support it because otherwise it can get awkward if, for example,
> asyncio.gather() needs to wrap an exception group that came from
> a TemporaryDirectory.__exit__().
>

I assume that in serious asyncio apps it is not uncommon to see some
operation use gather() to wrap several tasks, and then to see that
operation itself be passed to an outer gather() call together with some
other tasks. If several things go wrong you could easily end up with a
nested EG. Same in Trio -- you could have multiple nested "nurseries".

If you're catching such exceptions it may not be so important to be aware
of the nesting, but if you're logging tracebacks (or if the exception is
never caught and the program exits with a traceback) it is pretty important
to be able to show everything that went wrong without unnecessary
duplication of stack traces.

-- 
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/ZKFEK5GZWXCINXDZLONSQDUZPPPVX3OV/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 654 -- Exception Groups and except* : request for feedback for SC submission

2021-03-02 Thread Irit Katriel via Python-Dev

Hi Barry,

On Mon, Mar 1, 2021 at 6:12 PM Barry Scott  wrote:

>
> Have you considered alternative syntax to the "except*"?
> I feel that expect* is too easy to confuse with plain except.
>

We briefly considered things like "except Group[ValueError, TypeError]".
Do you (or anyone else) have any suggestions on syntax?

>
> As others have said needing to change all except Exception:
> handlers would be a breaking change for code that I maintain.
>

> I think its accepted that that idiom should continue to be as
> reliable as it currently is. Can you confirm that I have understood
> the conversation on this point?
>

That's not happening. The PEP has been updated regarding this.  It was
never going to be all "except Exception", most of them would not be
affected. But we have a better plan now so this discussion is moot.

As an aside - I found it interesting that the option to wrap BaseException
instances by an Exception, which came up a couple of times in this thread,
didn't generate any anxiety. In my day job I maintain a proprietary DSL
that ~4000 developers use, and my experience has been that a bug that
swallows an exception you need is worse than a bug that makes you get an
exception you did not expect. The latter can hurt you once (and then you
fix it). But the former is (1) harder to detect (2) longer to debug (3)
once you know about it there is nothing you can do to work around it.
Problems like that cost us years and years of work.

>
> I have been thinking about why I would be getting an ExceptionGroup
> with, for example, a number of OSError for a handler to deal with.
> Would it not be a difficult piece of code to write without the context of
> each OSError?
>
> What I'm think is that the lower level pieces would be catching the OSError
> closer to the source of the problem and raising a App/Lib specific
> exception
> that provides the context. For example AppFailedToSaveConfigError and
> AppFailedToSaveDataError as oppose to a two permission OSError's.
>

I think you're right about this. OSError is an example of an exception with
a data field that is often inspected for its value, so we picked on it to
talk about how you would do something like this. Most of the time this
won't be needed, but we still need to know that it's possible and
reasonably ergonomic to do it.

> With context I can imagine that handling the ExceptionGroup would be
> a lot easier for the App designer.
>
> If that the pattern that emerges then is the complexity of nested
> ExceptionGroups going to only rarely used?
>

I can't imagine people building deep trees of exceptions in practice (at
least not on purpose).  But some nesting may show up naturally, and we need
to support it because otherwise it can get awkward if, for example,
asyncio.gather() needs to wrap an exception group that came from
a TemporaryDirectory.__exit__().

Irit
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/3W6C3Y6WVKGYL4H63VDLB7VF6JH52TV4/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 654 -- Exception Groups and except* : request for feedback for SC submission

2021-03-02 Thread Sven R. Kunze


Hey Irit,

cool proposal.

I just have two questions regarding "except Exception" or "except 
BaseException" as it is used e.g. by concurrent.futures.process (def 
_process_worker) from the stdlib.


Almost similarly, I maintain a library using this pattern to wrap/unwrap 
exceptions from remote Python processes to create nicely formatted 
tracebacks (also recursively of course if needed) at the calling python 
process.


Usually these exceptions are wrapped, transferred to the parent process, 
there is the current call stack added, then reraised as a different 
exception, and so on and so forth if a chain of parents exist. The 
outermost parent process takes care of printing the tb.



My two questions regarding PEP 654:


1) What is the right "except pattern" when ExceptionGroup is introduced 
for the use cases described above (remote or concurrent python processes)?



2) What is the recommended approach of printing the traceback 
potentially incorporating multiple tracebacks - I couldn't find it in 
the PEP and tracebacks are a really neat tool for error hunting.



Best Regards,
Sven



On 23.02.21 01:24, Irit Katriel via Python-Dev wrote:


Hi all,

We would like to request feedback on PEP 654 -- Exception Groups and 
except*.


https://www.python.org/dev/peps/pep-0654/ 



It proposes language extensions that allow programs to raise and 
handle multiple unrelated
exceptions simultaneously, motivated by the needs of asyncio and other 
concurrency libraries,

but with other use cases as well.

* A new standard exception type, ExceptionGroup, to represent multiple 
exceptions with

  shared traceback.
* Updates to the traceback printing code to display (possibly nested) 
ExceptionGroups.

* A new syntax except* for handling ExceptionGroups.

A reference implementation (unreviewed) can be found at:
https://github.com/iritkatriel/cpython/pull/10 



Thank you for your help

Kind regards
Irit, Yury & Guido



___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/L5Q27DVKOKZCDNCAWRIQVOZ5DZCZHLRM/
Code of Conduct: http://python.org/psf/codeofconduct/
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/BY27BOGXFGEYVXKO3CDOHHZFYRAKY2TP/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 654 -- Exception Groups and except* : request for feedback for SC submission

2021-03-02 Thread Barry Scott

> On 23 Feb 2021, at 00:24, Irit Katriel via Python-Dev  
> wrote:
> 
> 
> Hi all,
> 
> We would like to request feedback on PEP 654 -- Exception Groups and except*.
> 
> https://www.python.org/dev/peps/pep-0654/ 
> 
> 
> It proposes language extensions that allow programs to raise and handle 
> multiple unrelated
> exceptions simultaneously, motivated by the needs of asyncio and other 
> concurrency libraries,
> but with other use cases as well.
> 
> * A new standard exception type,  ExceptionGroup, to represent multiple 
> exceptions with
>   shared traceback.
> * Updates to the traceback printing code to display (possibly nested) 
> ExceptionGroups.
> * A new syntax except* for handling ExceptionGroups.
> 
> A reference implementation (unreviewed) can be found at:
> https://github.com/iritkatriel/cpython/pull/10 
> 
Have you considered alternative syntax to the "except*"?
I feel that expect* is too easy to confuse with plain except.

As others have said needing to change all except Exception:
handlers would be a breaking change for code that I maintain.

I think its accepted that that idiom should continue to be as
reliable as it currently is. Can you confirm that I have understood
the conversation on this point?

I have been thinking about why I would be getting an ExceptionGroup
with, for example, a number of OSError for a handler to deal with.
Would it not be a difficult piece of code to write without the context of
each OSError?

What I'm think is that the lower level pieces would be catching the OSError
closer to the source of the problem and raising a App/Lib specific exception
that provides the context. For example AppFailedToSaveConfigError and
AppFailedToSaveDataError as oppose to a two permission OSError's.

With context I can imagine that handling the ExceptionGroup would be
a lot easier for the App designer.

If that the pattern that emerges then is the complexity of nested
ExceptionGroups going to only rarely used?

Barry

> 
> Thank you for your help
> 
> Kind regards
> Irit, Yury & Guido
> 
> 
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at 
> https://mail.python.org/archives/list/python-dev@python.org/message/L5Q27DVKOKZCDNCAWRIQVOZ5DZCZHLRM/
> Code of Conduct: http://python.org/psf/codeofconduct/

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/DPIXB6UFJFL2SIWPHNG2ZUURRJO7DTAV/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 654 -- Exception Groups and except* : request for feedback for SC submission

2021-03-02 Thread Irit Katriel via Python-Dev

Hi Sven,

On Mon, Mar 1, 2021 at 8:59 PM Sven R. Kunze  wrote:

> Hey Irit,
>
> cool proposal.
>

Thank you.

> I just have two questions regarding "except Exception" or "except
> BaseException" as it is used e.g. by concurrent.futures.process (def
> _process_worker) from the stdlib.
>
> Almost similarly, I maintain a library using this pattern to wrap/unwrap
> exceptions from remote Python processes to create nicely formatted
> tracebacks (also recursively of course if needed) at the calling python
> process.
>
> Usually these exceptions are wrapped, transferred to the parent process,
> there is the current call stack added, then reraised as a different
> exception, and so on and so forth if a chain of parents exist. The
> outermost parent process takes care of printing the tb.
>
>
> My two questions regarding PEP 654:
>
>
> 1) What is the right "except pattern" when ExceptionGroup is introduced
> for the use cases described above (remote or concurrent python processes)?
>

If you want to catch the whole ExceptionGroup and format its traceback,
then you can just do "except ExceptionGroup as eg" and then
traceback.print_exception(eg).

The except* syntax is for when you want to handle only certain types of
exceptions from the group, selectively.

> 2) What is the recommended approach of printing the traceback potentially
> incorporating multiple tracebacks - I couldn't find it in the PEP and
> tracebacks are a really neat tool for error hunting.
>

Part of the proposal of the PEP is that we teach the builtin traceback
formatting code to display ExceptionGroups with all that information. The
reference implementation has this, and the examples in the PEP were
produced with it. Some of the examples (the early ones) show exceptions
that were never raised so there is no traceback. But if you scroll down to
the later examples, you see the output for exceptions with tracebacks,
cause, context etc.

We didn't put too much effort into making the traceback output pretty, so
at the moment it's a draft. If there is an ascii artist out there who has
suggestions on improving this, that would be great.

Irit
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/N6ACTVYIPUXQGUXF7EGFXTX27VAA45L3/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 654 -- Exception Groups and except* : request for feedback for SC submission

2021-03-02 Thread Sven R. Kunze

Just to be on the safe side here, I would want to asked a little bit 
further.



On 02.03.21 12:22, Irit Katriel wrote:


1) What is the right "except pattern" when ExceptionGroup is
introduced for the use cases described above (remote or concurrent
python processes)?

If you want to catch the whole ExceptionGroup and format its 
traceback, then you can just do "except ExceptionGroup as eg" and then 
traceback.print_exception(eg).


The except* syntax is for when you want to handle only certain types 
of exceptions from the group, selectively.


Just to make sure, I understand this completely.


In order to make it more tangible here is an example from the stdlib:

https://github.com/python/cpython/blob/bf2e7e55d7306b1e2fce7dce767e8df5ff42cf1c/Lib/concurrent/futures/process.py#L215

As you can see, BaseException is caught multiple times as the only 
except-clause. _sendback_result(...) then used to transfer the 
return_val/exception back to parent process.



How is the supposed way of handling a raised ExceptionGroup?


a) will the existing code simply work as it?
b) if not what are the changes to this lib specifically as a blueprint 
example for others



Reading from the other subthread for this PEP, I am not 100% sure now 
that "except BaseException" will suffice if ExceptionGroup inherits from 
Exception instead of BaseException because it seems that ExceptionGroup 
somehow is handled specially. So, why I try to approach this very issue 
with existing code. Once that is clear, it should be easy for everybody 
to apply the same pattern to their code. Here is the copy from github:



try:
    r = call_item.fn(*call_item.args, **call_item.kwargs)
except BaseException as e:
    exc = _ExceptionWithTraceback(e, e.__traceback__)
    _sendback_result(result_queue, call_item.work_id, exception=exc)
else:
    _sendback_result(result_queue, call_item.work_id, result=r)
    del r


Maybe even _sendback_result could benefit from using ExceptionGroup 
itself. You can see there another "except BaseException" in case of an 
error. But that's maybe a different topic.



2) What is the recommended approach of printing the traceback
potentially incorporating multiple tracebacks - I couldn't find it
in the PEP and tracebacks are a really neat tool for error hunting.


Part of the proposal of the PEP is that we teach the builtin traceback 
formatting code to display ExceptionGroups with all that information.



As long as there's the possibility to receive the usual (two-line) 
entry-based list, I guess that is enough for every one to work with it 
(e.g. hiding unnecessary tb entries).



The reference implementation has this, and the examples in the PEP 
were produced with it. Some of the examples (the early ones) show 
exceptions that were never raised so there is no traceback. But if you 
scroll down to the later examples, you see the output for exceptions 
with tracebacks, cause, context etc.



Ah I see them now. Thank you. :-)


We didn't put too much effort into making the traceback output pretty, 
so at the moment it's a draft. If there is an ascii artist out there 
who has suggestions on improving this, that would be great.



Well, yes. It's not really pretty. I would recommend a tree-based 
solution such as the following:



Traceback (most recent call last):
  File "", line 3, in 
ExceptionGroup
|
+---+   Traceback (most recent call last):
|   | File "", line 3, in 
|   |   ExceptionGroup: eg
|   |   |
|   |   +---+ ValueError: a
|   |
|   |   During handling of the above exception, another exception occurred:
|   |
|   |   Traceback (most recent call last):
|   | File "", line 5, in 
|   |   KeyError: 'x'
|
|
+---+   Traceback (most recent call last):
|   | File "", line 3, in 
|   |   ExceptionGroup: eg
|   |   |
|   |   +---+ TypeError: b


I used a similar pattern for the remote-execution lib and received good 
user feedback (even requesting color encoding for each layer of the tree 
(not the text), so people know what belongs together). Besides colors, I 
used https://en.wikipedia.org/wiki/Box-drawing_character but I guess 
pipes, dashes and pluses could suffice for CPython.



One other remark from my side here: in the example of the PEP there seem 
to be missing a space before 'File ""'. Comparing outer tbs with 
inner tbs, you can see that the "F" of "File" is not underneath the a of 
"Traceback" but underneath the "r" of it.


Regards,
Sven

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/6I2SLGQEGOI4XMWPLBUC2SMQYVKTRQJ5/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 654 -- Exception Groups and except* : request for feedback for SC submission

On 28Feb2021 20:05, Guido van Rossum  wrote:
>I'm trying to shorten this again...
>
>On Sun, Feb 28, 2021 at 5:54 PM Cameron Simpson  wrote:
>> Let's turn this on its head:
>> - what specific harm comes from giving EGs container truthiness for 
>> size
>>   testing?
>
>For something that's not a pure container, this is an anti-pattern. No
>other subclass of BaseException (that I know of) can ever be falsey, so
>existing code would be forgiven to write "if e:" where "if e is not None:"
>was meant.

I that's how you feel, I'll withdraw my advocacy. I agree it's a 
difference.

I'll outline my view below the objections below, but agree to disagree 
here - I'm not seeking to convince you any more.

>> - what specific harm comes from returning an empty EG from split on 
>> no
>>   match instead of None?
>
>It would be an exception. What is raising an empty EG supposed to do?

It'd be rather pointless. It assumes a little care on the person making 
it (well, using it where I'd asked subgroup to make it). I guess if 
there's a fall through in the except* which reraises, its a burden on 
the clause author to check such a contingency.  But wouldn't such a 
check normally be written:

if eg:
raise

on the presumption that None (falsey) means nothing to raise? And 
wouldn't the same code work if eg looks like a container in truthiness?

>> - what specific harm comes from supporting iteration with a caveat 
>> about
>>   metadata in the docstring, and maybe a recommendation of subgroup?
>
>Iteration is such an easy-to-use API that it would become an attractive
>nuisance -- people would use it regardless of whether it's the right tool
>for the job.

Perhaps. But an easy to use API is normally what one wants. For you, 
this is a downside because you hold that it is also usually a poor 
choice for this facility. Maybe my use case and habits are unusual.

>> - if I wanted to override subgroup and split to not return None, is 
>> that
>>   even possible with the current ? i.e. can I make a clean metadata
>>   preserved EG from an empty list? For example:
>>
>> eg2 = eg.subgroup(lambda e: False)
>>
>> Does that get me None, or an empty group? If the latter, I can roll my
>> own subclass with my desired features. If not, I can't AFAICT.
>
>It returns None. Writing a subclass that behaves differently than the base
>class will just confuse users who aren't aware of your overrides.

Yeah, a subclass might be misleading. Maybe all I need is a help 
function or class.

Cheers,
Cameron Simpson 
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/Z4BM5JR7XQOIXCLW6GCE5N6V5Q72XE7W/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 654 -- Exception Groups and except* : request for feedback for SC submission

2021-02-28 Thread Guido van Rossum

I'm trying to shorten this again...

On Sun, Feb 28, 2021 at 5:54 PM Cameron Simpson  wrote:

>
> Let's turn this on its head:
>
> - what specific harm comes from giving EGs container truthiness for size
>   testing?
>

For something that's not a pure container, this is an anti-pattern. No
other subclass of BaseException (that I know of) can ever be falsey, so
existing code would be forgiven to write "if e:" where "if e is not None:"
was meant.

> - what specific harm comes from returning an empty EG from split on no
>   match instead of None?
>

It would be an exception. What is raising an empty EG supposed to do?

> - what specific harm comes from supporting iteration with a caveat about
>   metadata in the docstring, and maybe a recommendation of subgroup?
>

Iteration is such an easy-to-use API that it would become an attractive
nuisance -- people would use it regardless of whether it's the right tool
for the job.

>
> - if I wanted to override subgroup and split to not return None, is that
>   even possible with the current ? i.e. can I make a clean metadata
>   preserved EG from an empty list? For example:
>
> eg2 = eg.subgroup(lambda e: False)
>
> Does that get me None, or an empty group? If the latter, I can roll my
> own subclass with my desired features. If not, I can't AFAICT.
>

It returns None. Writing a subclass that behaves differently than the base
class will just confuse users who aren't aware of your overrides.

> EGs _have_ a .errors attribute which has all these aspects, why not
> expand it to the class as a whole?
>

They are different. The `errors` attribute may contain other EGs. But
presumably iteration would recurse into those so as to flatten the tree and
yield only leaf nodes. The attribute is necessary so it is possible to
write traversal functions (including that iterator you so desperately want
-- as a helper function).

> You seem very happen to implement 80% of what I want using callbacks
> (lambda e: ...), but I find explicit iteration much easier to read. I
> rarely use filter() for example, and often prefer a generator expression
> of list comprehension.
>

Our (the PEP authors') belief is that in most cases users are better off
not handling the leaf exceptions. If you want to handle e.g. KeyError, the
place to do that is in the try/except clause immediately surrounding the
offending `__getitem__` operation (which will never raise an EG), not
several stack frames below, where you're potentially dealing with an
aggregate of many KeyErrors, or perhaps a KeyError and a TypeError. At that
point the most common option is to ignore or log some or all exception
types.

-- 
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/GBUJJLRV5OH3ZE4OGWVVOBY2YLHFOWRR/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 654 -- Exception Groups and except* : request for feedback for SC submission

On 28Feb2021 23:56, Irit Katriel  wrote:
>If you go long, I go longer :)

:-)

>On Sun, Feb 28, 2021 at 10:51 PM Cameron Simpson  wrote:
>> On 28Feb2021 10:40, Irit Katriel  wrote:
>> >split() and subgroup() take care to preserve the correct metadata on
>> >all
>> >the internal nodes, and if you just use them you only make safe
>> operations.
>> >This is why I am hesitating to add iteration utilities to the API. Like we
>> >did, people will naturally try that first, and it's not the safest API.
>>
>> Wouldn't it be safe if the ExceptionGroup were immutable, as you plan?
>> Or have you another hazard in mind?
>
>Making them immutable won't help the metadata issue. split() and subgroup()
>copy the (context, cause traceback) from the original ExceptionGroups (root
>and internal nodes of the tree) to the result trees.   If you DIY creating
>new ExceptionGroups you need to take care of that.

Ah, right. Yes.

The overflows into my request for a factory method to construct a "like" 
ExceptionGroup with a new exception tree lower down.

>> But all that said, being able to iterable the subexceptions seems a
>> natural way to manage that:
>>
>> unhandled = []
>> try:
>> .
>> except *OSError as eg:
>> for e in eg:
>> if an ok exception:
>> handle it
>> else:
>> unhandled.append(e)
>> if unhandled:
>> raise ExceptionGroup("unhandled", unhandled)
>
>
>You just lost the metadata of eg. It has no context, cause and its
>traceback begins here.

Aye. Hence a wish, again lower down, for some reference to the source 
ExceptionGroup and therefore a handy factory for making a new group with 
the right metadata.

>And the exceptions contained in it, if they came
>from a deeper tree that you flattened into the list, now look like their
>traceback jumps straight to here from the place they were actually first
>inserted into an ExceptionGroup. This may well be an impossible code path.

Perhaps so. But it doesn't detract from how useful it is to iterate over 
the inner exceptions. I see this as an argument for making it possible 
to obtain the correct metadata, not against iteration itself. Even if 
the iteration yielded some proxy or wrapper for the inner exception 
instead of the naked exception itself.

>Here's an example:
[... flattened ExceptionGroup with uninformative tracebacks ...]
 import traceback
 def flatten(exc):
>... if isinstance(exc, ExceptionGroup):
>... for e in exc.errors:
>...yield from flatten(e)
>... else:
>... yield exc
[...]
 traceback.print_exception(flat_h())
>Traceback (most recent call last):
>  File "", line 3, in flat_h
>ExceptionGroup: flat_h
>   
>   Traceback (most recent call last):
> File "", line 3, in f
>   ValueError: 42
>
>traceback.print_exception(h()) prints a reasonable traceback - h() called
>g() called f().
>
>But according to  traceback.print_exception(flat_h()),   flat_h() called
>f().
>
>You can preserve the metadata (and the nested structure with all its
>metadata) if you replace the last line with:
>raise eg.subgroup(lambda e: e in  unhandled)

Ok. That works for me as my desired factory. Verbose maybe, but 
workable. Um. It presumes exception equality will do - that feels 
slightly error prone (including some similar but not unhandled 
exceptions).  But I can write something more pointed based on id().

>And for the part before that, iteration, Guido's pattern showed that you
>can roll it into the subgroup callback.

Aye.

>> There are some immediate shortcomings above. In particular, I have no
>> way of referencing the original ExceptionGroup without surprisingly
>> cumbersome:
>
>> try:
>> .
>> except ExceptionGroup as eg0:
>> unhandled = []
>> eg, os_eg = eg0.split(OSError)
>> if os_eg:
>> for e in os_eg:
>> if an ok exception:
>> handle it
>> else:
>> unhandled.append(e)
>> if eg:
>> eg, value_eg = eg.split(ValueError)
>> if value_eg:
>> for e in value_eg:
>> if some_ValueError_we_understand:
>> handle it
>> else:
>> unhandled.append(e)
>> if eg:
>> unhandled.append(eg)
>> if unhandled:
>> raise ExceptionGroup("unhandled", unhandled) from eg0
>
>This is where except* can help:
>
>  try:
>.
>except except *OSError as eg:
>unhandled = []
>handled, unhandled = eg.split(lambda e: e is an ok exception)  #
>with side effect to handle e
>if unhandled:
>raise unhandled
>except *ValueError as eg:
>handled, unhandled = eg.split(lambda e: e is a value error we
>understand)  # with side effect to handle

[Python-Dev] Re: PEP 654 -- Exception Groups and except* : request for feedback for SC submission

2021-02-28 Thread Irit Katriel via Python-Dev

Hi Cameron,

If you go long, I go longer :)

On Sun, Feb 28, 2021 at 10:51 PM Cameron Simpson  wrote:

> On 28Feb2021 10:40, Irit Katriel  wrote:
> >split() and subgroup() take care to preserve the correct metadata on
> >all
> >the internal nodes, and if you just use them you only make safe
> operations.
> >This is why I am hesitating to add iteration utilities to the API. Like we
> >did, people will naturally try that first, and it's not the safest API.
>
> Wouldn't it be safe if the ExceptionGroup were immutable, as you plan?
> Or have you another hazard in mind?
>

Making them immutable won't help the metadata issue. split() and subgroup()
copy the (context, cause traceback) from the original ExceptionGroups (root
and internal nodes of the tree) to the result trees.   If you DIY creating
new
ExceptionGroups you need to take care of that.

> But all that said, being able to iterable the subexceptions seems a
> natural way to manage that:
>
> unhandled = []
> try:
> .
> except *OSError as eg:
> for e in eg:
> if an ok exception:
> handle it
> else:
> unhandled.append(e)
> if unhandled:
> raise ExceptionGroup("unhandled", unhandled)
>

You just lost the metadata of eg. It has no context, cause and its
traceback begins here. And the exceptions contained in it, if they came
from a deeper tree that you flattened into the list, now look like their
traceback jumps straight to here from the place they were actually first
inserted into an ExceptionGroup. This may well be an impossible code path.
Here's an example:

>>> import traceback
>>>
>>> def flatten(exc):
... if isinstance(exc, ExceptionGroup):
... for e in exc.errors:
...yield from flatten(e)
... else:
... yield exc
...
>>> def f():
... try:
... raise ValueError(42)
... except ValueError as e:
... return e
...
>>> def g():
... try:
... raise ExceptionGroup("g", [f()])
... except ExceptionGroup as e:
... return e
...
>>> def h():
... try:
... raise ExceptionGroup("h", [g()])
... except ExceptionGroup as e:
... return e
...
>>> def flat_h():
... try:
... raise ExceptionGroup("flat_h", list(flatten(h(
... except ExceptionGroup as e:
... return e
...
>>>
>>> traceback.print_exception(h())
Traceback (most recent call last):
  File "", line 3, in h
ExceptionGroup: h

   Traceback (most recent call last):
 File "", line 3, in g
   ExceptionGroup: g

  Traceback (most recent call last):
File "", line 3, in f
  ValueError: 42

>>> traceback.print_exception(flat_h())
Traceback (most recent call last):
  File "", line 3, in flat_h
ExceptionGroup: flat_h

   Traceback (most recent call last):
 File "", line 3, in f
   ValueError: 42
>>>

traceback.print_exception(h()) prints a reasonable traceback - h() called
g() called f().

But according to  traceback.print_exception(flat_h()),   flat_h() called
f().

You can preserve the metadata (and the nested structure with all its
metadata) if you replace the last line with:
raise eg.subgroup(lambda e: e in  unhandled)
And for the part before that, iteration, Guido's pattern showed that you
can roll it into the subgroup callback.

>
> There are some immediate shortcomings above. In particular, I have no
> way of referencing the original ExceptionGroup without surprisingly
> cumbersome:

> try:
> .
> except ExceptionGroup as eg0:
> unhandled = []
> eg, os_eg = eg0.split(OSError)
> if os_eg:
> for e in os_eg:
> if an ok exception:
> handle it
> else:
> unhandled.append(e)
> if eg:
> eg, value_eg = eg.split(ValueError)
> if value_eg:
> for e in value_eg:
> if some_ValueError_we_understand:
> handle it
> else:
> unhandled.append(e)
> if eg:
> unhandled.append(eg)
> if unhandled:
> raise ExceptionGroup("unhandled", unhandled) from eg0
>

This is where except* can help:

  try:
.
except except *OSError as eg:
unhandled = []
handled, unhandled = eg.split(lambda e: e is an ok exception)  #
with side effect to handle e
if unhandled:
raise unhandled
except *ValueError as eg:
handled, unhandled = eg.split(lambda e: e is a value error we
understand)  # with side effect to handle e
if  unhandled:

I have the following concerns with the pattern above:
>
> There's no way to make a _new_ ExceptionGroup with

[Python-Dev] Re: PEP 654 -- Exception Groups and except* : request for feedback for SC submission

2021-02-28 Thread Marco Sulla

On Sat, 27 Feb 2021 at 00:35, Guido van Rossum  wrote:
> On Fri, Feb 26, 2021 at 3:18 PM Marco Sulla  
> wrote:
>>
>> Excuse me if I post here. Maybe is a stupid question: why, instead of
>> introducing except*, Python can't extend the functionality of except,
>> so it can do what except* would do?
>
>
> Good question. Here's an example:
> ```
> try:
> . . .
> except OSError as err:
> if err.errno != ENOENT:
> raise
> . . .
> ```
> If this would catch ExceptionGroup(OSError), the `err` variable would be an 
> ExceptionGroup instance, which does not have an `errno` attribute.

Thank you for the clarification :)

I must admit I read the PEP quickly, so I thought that the
subexception will be raised by except*, not the exception group. But
obviously this can't work.

The fact is I am really used to think that

except OsError as e

means "except for any OsError, which we name `e` from this moment on"

that I thought it applied also for except*. But, if I understood well

except* OsError as eg

means "except for any OsError contained in an ExceptionGroup. If the
OsError is not contained in an ExceptionGroup, Python wraps it in a
new ExceptionGroup. The ExceptionGroup is named `eg` from this moment
on"

I have to say that it's not obvious for me reading except* this way.
Anyway, I can't find a more readable semantic that does not use a new
keyword and it's short ^^'
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/XZRM57YQU3NRU5JLAHDULVR7TKUF5FT3/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 654 -- Exception Groups and except* : request for feedback for SC submission

On 27Feb2021 00:54, Irit Katriel  wrote:
>On Sat, Feb 27, 2021 at 12:47 AM Jim J. Jewett  wrote:
>> > Is this not allowed?
>>
>> >try:
>> >try:
>> >obj.func()# function that raises ExceptionGroups
>> >except AttributeError:
>> >logger.info("obj doesn't have a func")
>> >except *(AttributeError, SyntaxError):
>> >logger.info("func had some problems")
>>
>> Allowed, but probably in error ... no AttributeError will get through to
>> the except * unless it happened inside the except AttributeError handler.
>> Did you mean:
>
>If obj.func() raises an ExceptionGroup that contains AttributeError then
>"except AttributeError" doesn't catch it. So it will get through.

And I, for one, would expect that. And _want_ that: I want the code to 
do what I said, not have some magic which silently/invisibly intercepts 
ExceptionGroups which contain something buried deep in their subgroup 
tree.

We already allow "deep" exceptions out, to be caught at an arbitrary 
outer call stack level. I don't see why ExceptionGroups should be any 
different.

I certainly do not want ExceptionGroup([AttributeError]) conflated with 
AttributeError. That fills me with horror.

Cheers,
Cameron Simpson 
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/CSJJSEEW2EY56JKMBCSD6R5SEJOF2ZVU/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 654 -- Exception Groups and except* : request for feedback for SC submission

This message is longer than I had anticipated.

To aid comprehension, I'm:

- accepting that .split and .subgroup help my "handle some excpetions 
  but not others" situation, barely

- arguing for ExceptionGroups acting like other containers: truthy if 
  nonempty, falsey if empty; iterable; .subgroup and .split _not_ 
  returning None for an empty subgroup, so that the container-like 
  aspects can be used directly

On 28Feb2021 10:40, Irit Katriel  wrote:
>split() and subgroup() take care to preserve the correct metadata on 
>all
>the internal nodes, and if you just use them you only make safe operations.
>This is why I am hesitating to add iteration utilities to the API. Like we
>did, people will naturally try that first, and it's not the safest API.

Wouldn't it be safe if the ExceptionGroup were immutable, as you plan?  
Or have you another hazard in mind?

>We actually have the  OSErrors example in the PEP, just above
>https://www.python.org/dev/peps/pep-0654/#caught-exception-objects:
>
>try:
>low_level_os_operation()
>except *OSerror as errors:
>raise errors.subgroup(lambda e: e.errno != errno.EPIPE) from None

Indeed. That basicly addresses my pattern. Along with:

>On Sun, Feb 28, 2021 at 6:30 AM Guido van Rossum  
>wrote:
>> There’s a pattern for what you propose using the split() method and a
>> lambda, or you can keep the exceptions in a list and re-wrap them at the
>> end.

The keep-a-list approach was my fallback, absent a way to push an 
unhandled exception back in some sense.

Guido:
>> We really don’t want users pushing non-exceptions into the list, nor 
>> do we
>> want e.g. KeyboardInterrupt to be added to a (non-Base-) ExceptionGroup.

I was only imagining pushing exceptions from the original group back in.  
Enforcing that would be tedious and confusing though, so I was imagining 
some way of marking specific subexeceptions as handled or not handled.

But I had not understood the subgroup method.

I think my handled/unhandled concerns are (barely) sufficient addressed 
above.  If I wanted to sweep the group for handled exceptions and then 
reraise the unhandled ones in their own ExceptionGroup at the end, that 
seems tractable.

But all that said, being able to iterable the subexceptions seems a 
natural way to manage that:

unhandled = []
try:
.
except *OSError as eg:
for e in eg:
if an ok exception:
handle it
else:
unhandled.append(e)
if unhandled:
raise ExceptionGroup("unhandled", unhandled)

There are some immediate shortcomings above. In particular, I have no 
way of referencing the original ExceptionGroup without surprisingly 
cumbersome:

try:
.
except ExceptionGroup as eg0:
unhandled = []
eg, os_eg = eg0.split(OSError)
if os_eg:
for e in os_eg:
if an ok exception:
handle it
else:
unhandled.append(e)
if eg:
eg, value_eg = eg.split(ValueError)
if value_eg:
for e in value_eg:
if some_ValueError_we_understand:
handle it
else:
unhandled.append(e)
if eg:
unhandled.append(eg)
if unhandled:
raise ExceptionGroup("unhandled", unhandled) from eg0

I have the following concerns with the pattern above:

There's no way to make a _new_ ExceptionGroup with the same __cause__ 
and __context__ and message as the original group: not that I can't 
assign to these, but I'd need to enuerate them; what if the exceptions 
grew new attributes I wanted to replicate?

This cries out for another factory method like .subgroup but which makes 
a new ExceptionGroup from an original group, containing a new sequence 
of exceptions but the same message and coontext. Example:

unhandled_eg = eg0.with_exceptions(unhandled)

I don't see a documented way to access the group's message.

I'm quite unhappy about .subgroup (and presumably .split) returning None 
when the group is empty. The requires me to have the gratuitous "if eg:" 
and "if value_eg:" if-statements in the example above.

If, instead, ExceptionGroups were like any other container I could just 
test if they were empty:

if eg:

_and_ even if they were empty, iterate over them. Who cares if the loop 
iterates zero times? The example code would become:

try:
.
except ExceptionGroup as eg0:
unhandled = []
eg, os_eg = eg0.split(OSError)
for e in os_eg:
if an ok exception:
handle it
else:
unhandled.append(e)
eg, value_eg = eg.split(ValueError)
for e in value_eg:
if some_ValueError_we_understand:
handle it
else:
unhandled.append(e)
if eg:

[Python-Dev] Re: PEP 654 -- Exception Groups and except* : request for feedback for SC submission

2021-02-28 Thread Guido van Rossum

You can pass an arbitrary function to eg.split() and eg.subgroup(), and
that function can have a side effect. Suppose you want to log and ignore
OSError with errno==ENOENT but re-raise the rest, you can do this:
```
def log_and_ignore(err):
if err.errno == ENOENT:
log(err)
return False
else:
return True

try:
. . .
except *OSError as eg:
eg = eg.subgroup(log_and_ignore)
if eg is not None:
raise eg
```


On Sun, Feb 28, 2021 at 10:35 AM Jim J. Jewett  wrote:

> Looking at the following PEP example, I'm still not sure what he should do
> to handle some but not all OSError instances:
>
> ...raise ExceptionGroup(
> ... "eg",
> ... [
> ... ValueError(1),
> ... TypeError(2),
> ... OSError(3),
> ... ExceptionGroup(
> ... "nested",
> ... [OSError(4), TypeError(5), ValueError(6)])
> ... ]
>
> except *OSError seems to get him an ExceptionGroup that he still has to
> manually tree-walk to handle the OSErrors that he can actually handle.
>
> Once he has manually tree-walked, he has to manually re-raise the other
> OSErrors that he hasn't filtered out, and they become a sibling of the
> original ExceptionGroup, because they're (now) being raised at a different
> location.  (Or, not *really* the original ExceptionGroup, but one that
> looks like it without any OSErrors.)  This new sibling relationship is
> there to show that a failed or incomplete attempt was made at resolving the
> OSErrors, but no such attempt was made at the ValueErrors or TypeErrors.
>
> Alternatively, he *might* be able to just grab Exception, then use a
> complicated callback to subgroup only the OSErrors that he will be able to
> handle, and raise the complement of that.  It will still add another layer
> of ExceptionGroup, but won't split the remainders.
>
> This seems like an awful lot of work to preserve and elaborate structural
> relations between different exceptions that were originally accidental.  (I
> am assuming that the original relation was "these all happened, probably in
> different execution threads" and the new relations are just accidents of
> what got partially handled along the way, or maybe the order in which
> different execution threads got merged.)
>
> I can't imagine ever needing to care about the full tree structure unless
> I'm re-implementing asycio/trio/curio.  Even if that structural information
> is important to my own framework, that just means I should create more
> intermediate tasks to handle it more quickly, before the next level of
> aggregation.
>
> I would much prefer an API that let me pattern-match (even just by class
> of the exception instances, though preferably also by attributes), process
> each exception (as though it were the only one) within the code block that
> match selects, and indicate whether that one exception should be raised
> further or not.
>
> -jJ
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/WTZUGBFWFXMGAFHNTTBMUYFGMQCPV5N6/
> Code of Conduct: http://python.org/psf/codeofconduct/
>


-- 
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/BJYWU6VZNYTTV43T5ZL5CFXLXEOCBDEQ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 654 -- Exception Groups and except* : request for feedback for SC submission

2021-02-28 Thread Jim J. Jewett

Looking at the following PEP example, I'm still not sure what he should do to
handle some but not all OSError instances:

...raise ExceptionGroup(
... "eg",
... [
... ValueError(1),
... TypeError(2),
... OSError(3),
... ExceptionGroup(
... "nested",
... [OSError(4), TypeError(5), ValueError(6)])
... ]

except *OSError seems to get him an ExceptionGroup that he still has to
manually tree-walk to handle the OSErrors that he can actually handle.

Once he has manually tree-walked, he has to manually re-raise the other
OSErrors that he hasn't filtered out, and they become a sibling of the original
ExceptionGroup, because they're (now) being raised at a different location.
(Or, not *really* the original ExceptionGroup, but one that looks like it
without any OSErrors.) This new sibling relationship is there to show that a
failed or incomplete attempt was made at resolving the OSErrors, but no such
attempt was made at the ValueErrors or TypeErrors.

Alternatively, he *might* be able to just grab Exception, then use a
complicated callback to subgroup only the OSErrors that he will be able to
handle, and raise the complement of that. It will still add another layer of
ExceptionGroup, but won't split the remainders.

This seems like an awful lot of work to preserve and elaborate structural
relations between different exceptions that were originally accidental. (I am
assuming that the original relation was "these all happened, probably in
different execution threads" and the new relations are just accidents of what
got partially handled along the way, or maybe the order in which different
execution threads got merged.)

I can't imagine ever needing to care about the full tree structure unless I'm
re-implementing asycio/trio/curio. Even if that structural information is
important to my own framework, that just means I should create more
intermediate tasks to handle it more quickly, before the next level of
aggregation.

I would much prefer an API that let me pattern-match (even just by class of the
exception instances, though preferably also by attributes), process each
exception (as though it were the only one) within the code block that match
selects, and indicate whether that one exception should be raised further or
not.

-jJ
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at
https://mail.python.org/archives/list/python-dev@python.org/message/WTZUGBFWFXMGAFHNTTBMUYFGMQCPV5N6/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 654 -- Exception Groups and except* : request for feedback for SC submission

2021-02-28 Thread Irit Katriel via Python-Dev

On Sun, Feb 28, 2021 at 6:17 PM Guido van Rossum  wrote:

> On Sun, Feb 28, 2021 at 2:40 AM Irit Katriel 
> wrote:
>
>>
>> In earlier versions of the PEP ExceptionGroup was not immutable and split
>> actually removed matching exceptions from it.
>> It was also iterable so you could get a flat list of all the contained
>> leaf exceptions. Then we changed it.
>>
>> ExceptionGroup is a tree of exceptions, and the internal nodes of the
>> tree (which are ExceptionGroups) have metadata on them -  context, cause,
>> traceback.
>> If you want the full traceback of a leaf exception you need to
>> concatenate the tracebacks of all the exceptions on the path from the root
>> to the leaf.
>> If you flatten an ExceptionGroup and create a new list
>> ExceptionGroup(list(eg)) you don't lose metadata (unless eg's tree has
>> depth 1).
>>
>
> Is this a typo? Did you mean "If you flatten [it] and create a new list...
> you *do* lose metadata (unless ... depth 1)"?
>

It is a typo, thanks.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/M3DF5VIJWVE73SU4576ICKOGLEEWQBNO/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 654 -- Exception Groups and except* : request for feedback for SC submission

2021-02-28 Thread Guido van Rossum

On Sun, Feb 28, 2021 at 2:40 AM Irit Katriel 
wrote:

>
> In earlier versions of the PEP ExceptionGroup was not immutable and split
> actually removed matching exceptions from it.
> It was also iterable so you could get a flat list of all the contained
> leaf exceptions. Then we changed it.
>
> ExceptionGroup is a tree of exceptions, and the internal nodes of the tree
> (which are ExceptionGroups) have metadata on them -  context, cause,
> traceback.
> If you want the full traceback of a leaf exception you need to concatenate
> the tracebacks of all the exceptions on the path from the root to the leaf.
> If you flatten an ExceptionGroup and create a new list
> ExceptionGroup(list(eg)) you don't lose metadata (unless eg's tree has
> depth 1).
>

Is this a typo? Did you mean "If you flatten [it] and create a new list...
you *do* lose metadata (unless ... depth 1)"?


> Similarly, if you add an exception it needs to have matching metadata to
> the ExceptionGroup you are adding it to. It's too easy to get that wrong.
>
> split() and subgroup() take care to preserve the correct metadata on all
> the internal nodes, and if you just use them you only make safe operations.
> This is why I am hesitating to add iteration utilities to the API. Like we
> did, people will naturally try that first, and it's not the safest API.
>
> We actually have the  OSErrors example in the PEP, just above
> https://www.python.org/dev/peps/pep-0654/#caught-exception-objects:
>
> try:
> low_level_os_operation()
> except *OSerror as errors:
> raise errors.subgroup(lambda e: e.errno != errno.EPIPE) from None
>
>
> --
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/JCRKPDWQMYPSQJUXBZDGSVDI2T67ZSC3/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 654 -- Exception Groups and except* : request for feedback for SC submission

2021-02-28 Thread Irit Katriel via Python-Dev

In earlier versions of the PEP ExceptionGroup was not immutable and split
actually removed matching exceptions from it.
It was also iterable so you could get a flat list of all the contained leaf
exceptions. Then we changed it.

ExceptionGroup is a tree of exceptions, and the internal nodes of the tree
(which are ExceptionGroups) have metadata on them -  context, cause,
traceback.
If you want the full traceback of a leaf exception you need to concatenate
the tracebacks of all the exceptions on the path from the root to the leaf.
If you flatten an ExceptionGroup and create a new list
ExceptionGroup(list(eg)) you don't lose metadata (unless eg's tree has
depth 1).
Similarly, if you add an exception it needs to have matching metadata to
the ExceptionGroup you are adding it to. It's too easy to get that wrong.

split() and subgroup() take care to preserve the correct metadata on all
the internal nodes, and if you just use them you only make safe operations.
This is why I am hesitating to add iteration utilities to the API. Like we
did, people will naturally try that first, and it's not the safest API.

We actually have the  OSErrors example in the PEP, just above
https://www.python.org/dev/peps/pep-0654/#caught-exception-objects:

try:
low_level_os_operation()
except *OSerror as errors:
raise errors.subgroup(lambda e: e.errno != errno.EPIPE) from None

On Sun, Feb 28, 2021 at 6:30 AM Guido van Rossum  wrote:

> We really don’t want users pushing non-exceptions into the list, nor do we
> want e.g. KeyboardInterrupt to be added to a (non-Base-) ExceptionGroup.
>
> There’s a pattern for what you propose using the split() method and a
> lambda, or you can keep the exceptions in a list and re-wrap them at the
> end.
>
> On Sat, Feb 27, 2021 at 20:41 Cameron Simpson  wrote:
>
>> On 26Feb2021 02:44, Irit Katriel  wrote:
>> >On Fri, Feb 26, 2021 at 2:00 AM Guido van Rossum 
>> wrote:
>> >> OT: Is ExceptionGroup *really* immutable in the current
>> >> implementation? As
>> >> long as the 'errors' field is a list, I think one could mutate the list
>> >> directly.
>> >
>> >It's not, but we were going to make it an immutable tuple.
>>
>> Could you say why? Other than wanting to discourage mutation happy code
>> getting out there?
>>
>> The reason I ask is that the scenario which comes to my mind is
>> something like:
>>
>> except *OSError as e:
>>
>> AIUI "e" is an ExceptionGroup containing only OSErrors. So, one common
>> thing in my own code is this:
>>
>> try:
>> do something with a file
>> except OSError as e:
>> if e.errno == ENOENT:
>> # file is missing, but that is ok
>> # because we treat it like an empty file
>> elif ... some other ok situation ...
>> else:
>> raise
>>
>> My natural inclination with an ExceptionGroup would be to winnow the
>> OSErrors I'm handed, and push the _unhandled_ errors back into the
>> original ExceptionGroup. That way, after the various except* clauses, a
>> nonempty ExceptionGroup would remain with the unhandled errors, and it
>> might perhaps be reraised then.
>>
>> Cheers,
>> Cameron Simpson 
>>
> --
> --Guido (mobile)
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/JAVBJENJCBIMC6EVUVZFJJYYPGKRC3R2/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 654 -- Exception Groups and except* : request for feedback for SC submission

2021-02-27 Thread Guido van Rossum

We really don’t want users pushing non-exceptions into the list, nor do we
want e.g. KeyboardInterrupt to be added to a (non-Base-) ExceptionGroup.

There’s a pattern for what you propose using the split() method and a
lambda, or you can keep the exceptions in a list and re-wrap them at the
end.

On Sat, Feb 27, 2021 at 20:41 Cameron Simpson  wrote:

> On 26Feb2021 02:44, Irit Katriel  wrote:
> >On Fri, Feb 26, 2021 at 2:00 AM Guido van Rossum 
> wrote:
> >> OT: Is ExceptionGroup *really* immutable in the current
> >> implementation? As
> >> long as the 'errors' field is a list, I think one could mutate the list
> >> directly.
> >
> >It's not, but we were going to make it an immutable tuple.
>
> Could you say why? Other than wanting to discourage mutation happy code
> getting out there?
>
> The reason I ask is that the scenario which comes to my mind is
> something like:
>
> except *OSError as e:
>
> AIUI "e" is an ExceptionGroup containing only OSErrors. So, one common
> thing in my own code is this:
>
> try:
> do something with a file
> except OSError as e:
> if e.errno == ENOENT:
> # file is missing, but that is ok
> # because we treat it like an empty file
> elif ... some other ok situation ...
> else:
> raise
>
> My natural inclination with an ExceptionGroup would be to winnow the
> OSErrors I'm handed, and push the _unhandled_ errors back into the
> original ExceptionGroup. That way, after the various except* clauses, a
> nonempty ExceptionGroup would remain with the unhandled errors, and it
> might perhaps be reraised then.
>
> Cheers,
> Cameron Simpson 
>
-- 
--Guido (mobile)
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/KLQTLTUCUGQ4ZKQA5NFW4SR7MIDPUME4/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 654 -- Exception Groups and except* : request for feedback for SC submission

2021-02-27 Thread Cameron Simpson

On 26Feb2021 02:44, Irit Katriel  wrote:
>On Fri, Feb 26, 2021 at 2:00 AM Guido van Rossum  wrote:
>> OT: Is ExceptionGroup *really* immutable in the current 
>> implementation? As
>> long as the 'errors' field is a list, I think one could mutate the list
>> directly.
>
>It's not, but we were going to make it an immutable tuple.

Could you say why? Other than wanting to discourage mutation happy code 
getting out there?

The reason I ask is that the scenario which comes to my mind is 
something like:

except *OSError as e:

AIUI "e" is an ExceptionGroup containing only OSErrors. So, one common 
thing in my own code is this:

try:
do something with a file
except OSError as e:
if e.errno == ENOENT:
# file is missing, but that is ok
# because we treat it like an empty file
elif ... some other ok situation ...
else:
raise

My natural inclination with an ExceptionGroup would be to winnow the 
OSErrors I'm handed, and push the _unhandled_ errors back into the 
original ExceptionGroup. That way, after the various except* clauses, a 
nonempty ExceptionGroup would remain with the unhandled errors, and it 
might perhaps be reraised then.

Cheers,
Cameron Simpson 
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/FZI5JRQKTSMF7BYSIM2PJTO4E6IHSPDA/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 654 -- Exception Groups and except* : request for feedback for SC submission

2021-02-27 Thread Larry Hastings



On 2/27/21 2:37 AM, Paul Moore wrote:

On Fri, 26 Feb 2021 at 23:36, Jim J. Jewett  wrote:

Whenever I've used except Exception or stronger, it was a sanitary barrier 
around code that might well do unpredictable or even stupid things.  Adding a 
new kind of exception that I hadn't predicted -- including ExceptionGroup -- 
would certainly fit this description, and I want my driver loop to do what I 
told it.  (Probably log an unexpected exception, and continue with the next 
record.  I honestly don't even want a page, let alone a crash, because data 
from outside that barrier ... is often bad, and almost never in ways the 
on-call person can safely fix.  And if they don't have time to find it in the 
logs, then it isn't a priority that week.)

This is my biggest concern. Disclaimer: I've yet to read the PEP,
because async makes my head hurt, but I am aware of some of the
background with Trio. Please take this as the perspective of someone
thinking "I don't use async/await in my code, can I assume this
doesn't affect me?"


I haven't read the PEP either.  But I assume it could (should?) affect 
anyone managing multiple simultaneous /things/ in Python:


 * async code, "fibers", "greenlets", Stackless "microthreads",
   "cooperative multitasking", or any other userspace mechanism where
   you manage multiple "threads" of execution with multiple stacks
 * code managing multiple OS-level threads
 * code managing multiple processes

It seems to me that any of those could raise multiple heterogeneous 
exceptions, and Python doesn't currently provide a mechanism to manage 
this situation.  My dim understanding is that ExceptionGroup proposes a 
mechanism to handle exactly this thing.



Cheers,


//arry/

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/2EDUBSWHB6QP5ITFSDV6OPHUAJ2NXMKC/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 654 -- Exception Groups and except* : request for feedback for SC submission

2021-02-27 Thread Cameron Simpson

On Fri, 26 Feb 2021 at 23:36, Jim J. Jewett  wrote:
> Whenever I've used except Exception or stronger, it was a sanitary
> barrier around code that might well do unpredictable or even stupid
> things.  Adding a new kind of exception that I hadn't predicted --
> including ExceptionGroup -- would certainly fit this description, and 
> I want my driver loop to do what I told it.

This is my concern as well. I've got plenty of "except Exception" and 
IMO none consititutes "abuse" - they're there for the same reason Jim 
cites above: a barrier around _arbitrary_ callback code - so that the 
outer control facility continues to roll on as expected.

All of these are handler frameworks for use by other code - they 
inherently call an unknown and arbitrary thing at some point and expect 
to catch an Exception raised and suitably handle it (log it, record it 
for the caller to see later, whatever).

Now _all_ those handlers will need a special except* handler for 
ExceptionGroup, because they DO NOT KNOW whether the called code might 
raise it. And "except Exception", the longstanding recommended way to 
catch "everything (well almost everything)" no longer works.

Now, I've long wanted something like MultiError or ExceptionGroup to 
gather miltiple failures for raising together at a suitable 
slightly-later point.

[...]
On 27Feb2021 19:06, Guido van Rossum  wrote:
>Our goal for the PEP is that *unless* you're going to use APIs that are
>documented to raise ExceptionGroup, you won't have to use `except *` nor
>will you have to deal with ExceptionGroup otherwise.

But.. this isn't Java, where the (hah!) type annotations document the 
exceptions it raises.

Jim again, catching Exception to protect calling code:
>> def safe_fn():
>> try:
>> do_something_funky()
>> return True
>> except Exception:
>> print("Well, that was a shame...")
>> return False
>>
>> then I am intending to guarantee that calling safe_fn() will never
>> raise an exception. [...]

Guido:
>The modification I proposed where we have both BaseExceptionGroup and
>ExceptionGroup will satisfy this need. It basically means that *unless* you
>are explicitly using an API that is documented to raise
>[Base]ExceptionGroup (such as a future variant of asyncio.gather()), you
>don't have to care about it. [...] that's why I am proposing to change the PEP 
>so that your 
>code will remain safe.

That would be welcome to me, too.

Cheers,
Cameron Simpson 
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/IKGK5YEPUPPWCAQ6NQEVGMKA63KPFUJQ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 654 -- Exception Groups and except* : request for feedback for SC submission

2021-02-27 Thread Guido van Rossum

On Sat, Feb 27, 2021 at 1:09 PM Jim J. Jewett  wrote:

> > In fact, if you're happy with RuntimeError here, making
> > ExceptionGroup inherit from Exception (instead of
> > BaseException) would do just as well -- after all RuntimeError
> > is pretty arbitrary
>
> Agreed, and I think it should inherit from Exception.
>
> > (in fact, it's wrong, because the problem is a static bug in the
> > code, not something that went wrong at run time).
>
> Something went wrong deep inside an individual task.  The only clear
> static bug is that the async library changed how it reported that, and my
> except clause hasn't done the make-work to keep in step.
>
> > Let's see how this compares to the alternatives. First let's define
> > three key examples.
>
> > Example 1:
>
> > try:
> >raise ExceptionGroup(ValueError)
> > except Exception:
>
> > Example 2:
>
> > try:
> > raise ExceptionGroup(ValueError)
> > except ValueError:
>
> Example 2(a) :
>
>  try:
>  raise ExceptionGroup(ValueError,OtherError)
>  except ValueError:
>
> > Example 3:
>
> > try:
> >raise ExceptionGroup(asyncio.CancelledError)
> > except Exception:
>
> I would prefer that the except clause be executed in all of the examples,
> but I admit that 2a is a problem, because it could end up silently losing
> OtherError.  And I admit getting example 2 to do the right thing if 2a
> doesn't might be too much of a contortion.
>
> For example 3, I may be missing a subtle point, but I feel that by the
> time you get to code which doesn't expect asyncio.CancelledError, then you
> have already cancelled as far as you should.  Cancelling may want to bubble
> up through several other tasks, but it shouldn't kill the whole server,
> just because the trampoline got sloppy.
>

If you're using 'except Exception:' and an asyncio.CancelledError  (or
trio.Cancelled) exception escapes, you're already dead, because those don't
inherit from Exception, only from BaseException. And everything inherits
from BaseException, including [Base]ExceptionGroup.

-- 
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/TJGM3DBVXOUJU4LCD2ULICN4I537HHCY/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 654 -- Exception Groups and except* : request for feedback for SC submission

2021-02-27 Thread Guido van Rossum

On Sat, Feb 27, 2021 at 2:39 AM Paul Moore  wrote:

> On Fri, 26 Feb 2021 at 23:36, Jim J. Jewett  wrote:
> >
> > Whenever I've used except Exception or stronger, it was a sanitary
> barrier around code that might well do unpredictable or even stupid
> things.  Adding a new kind of exception that I hadn't predicted --
> including ExceptionGroup -- would certainly fit this description, and I
> want my driver loop to do what I told it.  (Probably log an unexpected
> exception, and continue with the next record.  I honestly don't even want a
> page, let alone a crash, because data from outside that barrier ... is
> often bad, and almost never in ways the on-call person can safely fix.  And
> if they don't have time to find it in the logs, then it isn't a priority
> that week.)
>
> This is my biggest concern. Disclaimer: I've yet to read the PEP,
> because async makes my head hurt, but I am aware of some of the
> background with Trio. Please take this as the perspective of someone
> thinking "I don't use async/await in my code, can I assume this
> doesn't affect me?"
>

The PEP doesn't actually do anything with async -- it just uses asyncio as
one of the motivating use cases. The spec itself does not use async or
asyncio. At some point in the future we might consider adding new APIs to
asyncio that can raise ExceptionGroup, but other uses of 'async def' will
definitely not be affected.

Our goal for the PEP is that *unless* you're going to use APIs that are
documented to raise ExceptionGroup, you won't have to use `except *` nor
will you have to deal with ExceptionGroup otherwise.


> For me, the biggest requirement I would have is that if I have code like:
>
> def safe_fn():
> try:
> do_something_funky()
> return True
> except Exception:
> print("Well, that was a shame...")
> return False
>
> then I am intending to guarantee that calling safe_fn() will never
> raise an exception. Obviously, the example is a toy (in reality, I'd
> log the error, skip a record, whatever) but that's not the point here
> - the point is that I consider it reasonable to expect `except
> Exception` to be a hard barrier that allows me to be sure I've covered
> everything.
>

The modification I proposed where we have both BaseExceptionGroup and
ExceptionGroup will satisfy this need. It basically means that *unless* you
are explicitly using an API that is documented to raise
[Base]ExceptionGroup (such as a future variant of asyncio.gather()), you
don't have to care about it. The other exception is if you're writing a
library for formatting exceptions (like the stdlib traceback.py module).


> My worry is that if ExceptionGroup exceptions are *not* trapped by
> `except Exception`, then code like this is no longer safe. And by
> making ExceptionGroup a BaseException subclass, that's what will
> happen.
>

Right, and that's why I am proposing to change the PEP so that your code
will remain safe.

Note that the class headings are like this:
```
class BaseExceptionGroup(BaseException): ...
class ExceptionGroup(BaseExceptionGroup, Exception): ...
```
which produces the following MROs:
```
ExceptionGroup -> BaseExceptionGroup -> Exception -> BaseException -> object
  BaseExceptionGroup ->  BaseException -> object
```


Ultimately, there's a genuine use case for code that says "whatever
> happens in the code I call, this is my last line of defense and I want
> to be as near to 100% sure as I can be that I regain control at this
> point". At the moment, that is spelled `except Exception`. If you're
> proposing to change that, even if it's just to require that it be
> spelled differently, you're going to break quite a lot of code - and
> worse, the code you're breaking will be code that has taken care to
> ensure it's written safely, so you're penalising people who *tried to
> get stuff right*, which IMO is the worst type of breakage.
>

I got you.


> I will at some point read the PEP fully, to better understand the
> async side of the story, but for now please consider this as the
> perspective of someone who doesn't expect to care about async, and
> therefore wants to feel safe in assuming this PEP won't affect them.
>

There really is no async side. And you will be safe.


> Paul
>
> PS If you're thinking "but if you were using async, you'd know about
> it", consider that I could be using a PyPI library that, for example,
> returned stock values. That library switches to using async, and has a
> bug that lets an exception group escape. My `except Exception` code is
> designed *precisely* to protect me against such bugs, without me
> needing to know anything about how the library works...
>

And it will.

-- 
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*

___
Python-Dev mailing list --

[Python-Dev] Re: PEP 654 -- Exception Groups and except* : request for feedback for SC submission

2021-02-27 Thread Jim J. Jewett

> In fact, if you're happy with RuntimeError here, making
> ExceptionGroup inherit from Exception (instead of
> BaseException) would do just as well -- after all RuntimeError
> is pretty arbitrary

Agreed, and I think it should inherit from Exception.

> (in fact, it's wrong, because the problem is a static bug in the
> code, not something that went wrong at run time).

Something went wrong deep inside an individual task.  The only clear static bug 
is that the async library changed how it reported that, and my except clause 
hasn't done the make-work to keep in step.

> Let's see how this compares to the alternatives. First let's define 
> three key examples.

> Example 1:

> try:
>raise ExceptionGroup(ValueError)
> except Exception:

> Example 2:

> try:
> raise ExceptionGroup(ValueError)
> except ValueError:

Example 2(a) :

 try:
 raise ExceptionGroup(ValueError,OtherError)
 except ValueError:
 
> Example 3:

> try:
>raise ExceptionGroup(asyncio.CancelledError)
> except Exception:

I would prefer that the except clause be executed in all of the examples, but I 
admit that 2a is a problem, because it could end up silently losing OtherError. 
 And I admit getting example 2 to do the right thing if 2a doesn't might be too 
much of a contortion.

For example 3, I may be missing a subtle point, but I feel that by the time you 
get to code which doesn't expect asyncio.CancelledError, then you have already 
cancelled as far as you should.  Cancelling may want to bubble up through 
several other tasks, but it shouldn't kill the whole server, just because the 
trampoline got sloppy.

-jJ
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/RHL3E2IBCIW2SP5L5W3BHWIWB4DV6U55/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 654 -- Exception Groups and except* : request for feedback for SC submission

2021-02-27 Thread Paul Moore

On Fri, 26 Feb 2021 at 23:36, Jim J. Jewett  wrote:
>
> Whenever I've used except Exception or stronger, it was a sanitary barrier 
> around code that might well do unpredictable or even stupid things.  Adding a 
> new kind of exception that I hadn't predicted -- including ExceptionGroup -- 
> would certainly fit this description, and I want my driver loop to do what I 
> told it.  (Probably log an unexpected exception, and continue with the next 
> record.  I honestly don't even want a page, let alone a crash, because data 
> from outside that barrier ... is often bad, and almost never in ways the 
> on-call person can safely fix.  And if they don't have time to find it in the 
> logs, then it isn't a priority that week.)

This is my biggest concern. Disclaimer: I've yet to read the PEP,
because async makes my head hurt, but I am aware of some of the
background with Trio. Please take this as the perspective of someone
thinking "I don't use async/await in my code, can I assume this
doesn't affect me?"

For me, the biggest requirement I would have is that if I have code like:

def safe_fn():
try:
do_something_funky()
return True
except Exception:
print("Well, that was a shame...")
return False

then I am intending to guarantee that calling safe_fn() will never
raise an exception. Obviously, the example is a toy (in reality, I'd
log the error, skip a record, whatever) but that's not the point here
- the point is that I consider it reasonable to expect `except
Exception` to be a hard barrier that allows me to be sure I've covered
everything.

My worry is that if ExceptionGroup exceptions are *not* trapped by
`except Exception`, then code like this is no longer safe. And by
making ExceptionGroup a BaseException subclass, that's what will
happen.

Ultimately, there's a genuine use case for code that says "whatever
happens in the code I call, this is my last line of defense and I want
to be as near to 100% sure as I can be that I regain control at this
point". At the moment, that is spelled `except Exception`. If you're
proposing to change that, even if it's just to require that it be
spelled differently, you're going to break quite a lot of code - and
worse, the code you're breaking will be code that has taken care to
ensure it's written safely, so you're penalising people who *tried to
get stuff right*, which IMO is the worst type of breakage.

I will at some point read the PEP fully, to better understand the
async side of the story, but for now please consider this as the
perspective of someone who doesn't expect to care about async, and
therefore wants to feel safe in assuming this PEP won't affect them.

Paul

PS If you're thinking "but if you were using async, you'd know about
it", consider that I could be using a PyPI library that, for example,
returned stock values. That library switches to using async, and has a
bug that lets an exception group escape. My `except Exception` code is
designed *precisely* to protect me against such bugs, without me
needing to know anything about how the library works...
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/SKU4RIRPRRGESKRMP55ZPZNYL4XWFFUR/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 654 -- Exception Groups and except* : request for feedback for SC submission

> Why would you want a different flavor of ExceptionGroup?

The easiest example you took care of by saving the exceptions in a list instead 
of a set.  Next would be the ExceptionGroup vs BaseExceptionGroup that I 
*don't* like, but someone might.  There are also libraries that promise to 
raise only exceptions derived (possibly through multiple inheritance) from a 
particular marker exception.

But honestly, my biggest concern is that it just seems wrong for any class to 
be final, and it has sometimes been an irritant even for such obvious cases 
cases as Boolean.  So why is this so special?

-jJ
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/2AJJIOLL4RB2TKLYDA2Y7HGDRWEBUD5J/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 654 -- Exception Groups and except* : request for feedback for SC submission

ValueError("Partner: 3 File: 127 record: 93 is missing field: currency") tells 
the production support people who to contact and what to say.  
 
I'm not sure what additional context would be helpful, let alone how it might 
have been available "at the time", but lost now that the ValueAttribute is 
collected into an ExceptionGroup.

-jJ
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/GXUJTE25TWR5AKAYCTETKL4YWXQC2FR5/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 654 -- Exception Groups and except* : request for feedback for SC submission

On Sat, Feb 27, 2021 at 12:35 AM Greg Ewing 
wrote:

> While I don't particularly mind if we get ExceptionGroup, giving
> it special magical semantics, and especially new syntax, seems like
> bringing a massively overpowered weapon to bear on something that
> will only be used rarely.
>
> Handling multiple exceptions from an ExceptionGroup could be done
> using a loop with pattern matching, so is except* really justified?
>
>
It is of course an option to split the PEP into two, add ExceptionGroup
first and then ask ourselves if we want except*.
We do have some experience with this from the Trio experiments with
MultiError though, so we are not starting from scratch.

Can you spell out how you see ExceptionGroup handling work with pattern
matching?
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/XZ5EGRGO5LMAO4DYRX2VZLFRBEVHWTIK/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 654 -- Exception Groups and except* : request for feedback for SC submission

On Fri, Feb 26, 2021 at 11:43 PM Guido van Rossum  wrote:

> On Fri, Feb 26, 2021 at 3:18 PM Marco Sulla 
> wrote:
>
>> Excuse me if I post here. Maybe is a stupid question: why, instead of
>> introducing except*, Python can't extend the functionality of except,
>> so it can do what except* would do?
>>
>
> Good question. Here's an example:
> ```
> try:
> . . .
> except OSError as err:
> if err.errno != ENOENT:
> raise
> . . .
> ```
> If this would catch ExceptionGroup(OSError), the `err` variable would be
> an ExceptionGroup instance, which does not have an `errno` attribute.
>
> (Irit: Does the PEP answer this question? I couldn't quickly find it in
> the rejected ideas. I think it's a reasonable question and we should answer
> it, either in the Rationale or in Rejected Ideas.)
>

Added here:  https://github.com/python/peps/pull/1846



>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/K7VCM7OZDIGZDTIZGJZJGWJOBXB65IU4/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 654 -- Exception Groups and except* : request for feedback for SC submission

On Sat, Feb 27, 2021 at 12:47 AM Jim J. Jewett  wrote:

>
> > Is this not allowed?
>
> >try:
> >try:
> >obj.func()# function that raises ExceptionGroups
> >except AttributeError:
> >logger.info("obj doesn't have a func")
> >except *(AttributeError, SyntaxError):
> >logger.info("func had some problems")
>
> Allowed, but probably in error ... no AttributeError will get through to
> the except * unless it happened inside the except AttributeError handler.
> Did you mean:
>


If obj.func() raises an ExceptionGroup that contains AttributeError then
"except AttributeError" doesn't catch it. So it will get through.



>
> try:
> try:
> obj.func# function that raises ExceptionGroups
> except AttributeError:
> logger.info("obj doesn't have a func")
> obj.func()
> except *(AttributeError, SyntaxError):
> logger.info("func had some problems")
>
> I see this as an argument that the except/except* split is tricky, but I
> don't think it says anything about whether except* clauses should be able
> to see into nested ExceptionGroups ... nor am I at all confident that I
> understood your intent.
>
> -jJ
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/XOAB7IJNXOHRL3HRVZ5VZON6MVHOPXB3/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/SVADA4PBND7ADSGTMC37XCQVNSOQ57LM/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 654 -- Exception Groups and except* : request for feedback for SC submission

> Thank you for turning to what happens with 'except ValueError' when an
> ExceptionGroup[ValueError] is raised, this is important.

> I'm not sure it's safe to assume that it is necessarily a > programming
>error, and that the interpreter can essentially break the program in this
> case.

I'm betting it means a ValueError was raised, but then something (probably an 
asynchronous framework) aggregated it.  I won't swear you would never want to 
distinguish the two cases, or to distinguish them both from 
ExceptionGroup[ExceptionGroup[ExceptionGroup[ValueError]]], but ... normally 
you wouldn't.

> Is this not allowed?

>try:
>try:
>obj.func()# function that raises ExceptionGroups
>except AttributeError:
>logger.info("obj doesn't have a func")
>except *(AttributeError, SyntaxError):
>logger.info("func had some problems")

Allowed, but probably in error ... no AttributeError will get through to the 
except * unless it happened inside the except AttributeError handler.  Did you 
mean:

try:
try:
obj.func# function that raises ExceptionGroups
except AttributeError:
logger.info("obj doesn't have a func")
obj.func()
except *(AttributeError, SyntaxError):
logger.info("func had some problems")

I see this as an argument that the except/except* split is tricky, but I don't 
think it says anything about whether except* clauses should be able to see into 
nested ExceptionGroups ... nor am I at all confident that I understood your 
intent. 

-jJ
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/XOAB7IJNXOHRL3HRVZ5VZON6MVHOPXB3/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 654 -- Exception Groups and except* : request for feedback for SC submission

2021-02-26 Thread Greg Ewing


While I don't particularly mind if we get ExceptionGroup, giving
it special magical semantics, and especially new syntax, seems like
bringing a massively overpowered weapon to bear on something that
will only be used rarely.

Handling multiple exceptions from an ExceptionGroup could be done
using a loop with pattern matching, so is except* really justified?

--
Greg
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/BRQDP4KNRYU6UCKQBHRAUAHWWCENNIVM/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 654 -- Exception Groups and except* : request for feedback for SC submission

You still need except* for the (unusual?) case where the ExceptionGroup 
contains multiple individual Exceptions, and you want them all to be processed. 
 (This possibility is the justification for the PEP, but the difficulty of 
associating an exception with the specific task that raised it suggests that 
exceptions are still intended to be rare, rather than a back-channel 
communications band.)

As written, you also need except* to unwrap, so that ExceptionGroup(ValueError) 
can be handled by "except ValueError" instead of "except RuntimeError" (or 
except ExceptionGroup) followed by cause-unwrapping.

-jJ
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/B6JKV2JIFA76CMDOK2IBVBA2MIQKYDVR/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 654 -- Exception Groups and except* : request for feedback for SC submission

FWIW, the only situation I can think of where you would care that the enclosed 
exception instances are BaseException but not regular Exception is interactive 
debugging, and even then there are enough other ways to kill the whole process 
that I think most people would use one of them instead of wanting a different 
BaseExceptionGroup that breaks the server instead of just the servlet.

I suppose you could argue that the distinction encourages the "good practice" 
of defensively wrapping "except Exception" in an "except BaseException" that is 
itself wrapped in a bare except.  I suspect it would actually just push people 
to replace that "except Exception" with the bare except and give up on the 
logging, because that is a quicker adjustment.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/IDVNUNBAMFKZVT64TFV5QYLB3BJTABBD/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 654 -- Exception Groups and except* : request for feedback for SC submission

2021-02-26 Thread Guido van Rossum

On Fri, Feb 26, 2021 at 3:18 PM Marco Sulla 
wrote:

> Excuse me if I post here. Maybe is a stupid question: why, instead of
> introducing except*, Python can't extend the functionality of except,
> so it can do what except* would do?
>

Good question. Here's an example:
```
try:
. . .
except OSError as err:
if err.errno != ENOENT:
raise
. . .
```
If this would catch ExceptionGroup(OSError), the `err` variable would be an
ExceptionGroup instance, which does not have an `errno` attribute.

(Irit: Does the PEP answer this question? I couldn't quickly find it in the
rejected ideas. I think it's a reasonable question and we should answer it,
either in the Rationale or in Rejected Ideas.)

-- 
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/3VQEWV26SD6NH3ALOLORMZ2G3DTFOJZ4/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 654 -- Exception Groups and except* : request for feedback for SC submission

> We keep the ability to wrap any exception, while we lose the "fail-fast if
> you forget to handle an ExceptionGroup" feature, which was intended as a
> kindness towards those who abuse "except Exception".

How is this a kindness?

Whenever I've used except Exception or stronger, it was a sanitary barrier 
around code that might well do unpredictable or even stupid things.  Adding a 
new kind of exception that I hadn't predicted -- including ExceptionGroup -- 
would certainly fit this description, and I want my driver loop to do what I 
told it.  (Probably log an unexpected exception, and continue with the next 
record.  I honestly don't even want a page, let alone a crash, because data 
from outside that barrier ... is often bad, and almost never in ways the 
on-call person can safely fix.  And if they don't have time to find it in the 
logs, then it isn't a priority that week.)

> If we adopt this solution then letting an ExceptionGroup escape from code
> that is not supposed to raise it, is not a fatal error, it's just some
> exception like any other.

Good!  If we're not coordinating so closely that I already know to handle the 
ExceptionGroup in advance, then that is exactly what should happen (and except 
Exception with a log and log analysis is the right way to deal with it).

> So there is no longer a distinction between code that raises
> ExceptionGroups and code that doesn't. Any code can propagate them, like
> any code can raise any other exception.

Good;  special cases and all.

> Does this mean that more code needs to be aware of the possibility of them
> showing up?  Is that a problem? 

Honestly, no.  You seem to be assuming a very well-controlled environment where 
any potential problems would be caught long before production.  My experience 
is that such optimism is never warranted, *particularly* at places that claim 
to have a heavy process to ensure such early (or in-time) bug-catches.

> What would we have done here if we were building Python from scratch?

Raise anything you want.  Or maybe only strings and exceptions.  Or maybe only 
stuff inheriting from a marker class named BaseException ... but we probably 
wouldn't add a parallel base marker that catch-all code *also* needs to be 
aware of.  (And since we would be starting from scratch, the catch-all wrapper 
code would certainly not have to be deployed on a conservatively managed 
server, before lightweight exploratory less-centralized client code could start 
using it.) 

-jJ
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/TEBYSGMMT3SJRVTPCZQFMHJQPBOV6Q74/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 654 -- Exception Groups and except* : request for feedback for SC submission

2021-02-26 Thread Guido van Rossum

On Thu, Feb 25, 2021 at 9:40 PM Nathaniel Smith  wrote:

> [...]
> Yury/I/others did discuss the idea of a
> BaseExceptionGroup/ExceptionGroup split a lot, and I think the general
> feeling is that it could potentially work, but feels like a
> complicated and awkward hack, so no-one was super excited about it.
>

Oh, I didn't realize (or recall) the idea was considered before. Given the
pseudo code I showed I don't think it's particularly complicated or
awkward, although it's of course more complicated than the design that's
currently in our PEP.


> For a while we also had a compromise design where only
> BaseExceptionGroup was built-in, but we left it non-final specifically
> so asyncio could define an ExceptionsOnlyExceptionGroup.
>

I did notice that. Given how many distinct use cases we've discovered by
now I think this is a dead end.


> Another somewhat-related awkward part of the API is how ExceptionGroup
> and plain-old 'except' should interact *in general*. The intuition is
> that if you have 'except ValueError' and you get an
> 'ExceptionGroup(ValueError)', then the user's code has some kind of
> problem and we should probably do something? to let them know? One
> idea I had was that we should raise a RuntimeError if this happens,
> sort of similar to PEP 479. But I could never quite figure out how
> this would help (gunicorn crashing with a RuntimeError isn't obviously
> better than gunicorn crashing with an ExceptionGroup).
>

I'm not sure I agree with the intuition that such user code definitely has
a problem. Irit tried to give an example using AttributeError, and while
that particular example is easily countered, I think it's not as
straightforward as you state. For example there could be a framework
exception that we know statically will not be wrapped in ExceptionGroup.
Basically, if we have some code that can either raise some exceptions of
category A wrapped in ExceptionGroup, or a bare exception of category B
(without overlap between the categories), writing code that catches
category B using `except B:` should not affect the propagation of
ExceptionGroup.

Or what about code that uses this idiom?
```
try:
. . .
except Exception:
. . .
raise
```
There should be no need to switch this to `except *` just to prevent the
ExceptionGroup from becoming a RuntimeError.


> == NEW IDEA THAT MAYBE SOLVES BOTH PROBLEMS ==
>
> Proposal:
>
> - any time an unwinding ExceptionGroup encounters a traditional
> try/except, then it gets replaced with a RuntimeError whose __cause__
> is set to the original ExceptionGroup and whose first traceback entry
> points to the offending try/except block
>
> - CUTE BIT I ONLY JUST THOUGHT OF: this substitution happens right
> *before* we start evaluating 'except' clauses for this try/except
>
> So for example:
>
> If an ExceptionGroup hits an 'except Exception': The ExceptionGroup is
> replaced by a RuntimeError. RuntimeError is an Exception, so the
> 'except Exception' clause catches it. And presumably logs it or
> something. This way your log contains both a notification that you
> might want to switch to except* (from the RuntimeError), *along with*
> the full original exception details (from the __cause__ attribute). If
> it was an ExceptionGroup(KeyboardInterrupt), then it still gets caught
> and that's not so great, but at least you get the RuntimeError to
> point out that something has gone wrong and tell you where?
>
> If an ExceptionGroup(ValueError) hits an 'except ValueError': it
> doesn't get caught, *but* a RuntimeError keeps propagating out to tell
> you you have a problem. And when that RuntimeError eventually hits the
> top of your program or ends up in your webserver logs or whatever,
> then the RuntimeError's traceback will point you to the 'except
> ValueError' that needs to be fixed.
>
> If you write 'except ExceptionGroup': this clause is a no-op that will
> never execute, because it's impossible to still have an ExceptionGroup
> when we start matching 'except' clauses. (We could additionally emit a
> diagnostic if we want.)
>
> If you write bare 'except:', or 'except BaseException': the clause
> always executes (as before), but they get the RuntimeError instead of
> the ExceptionGroup. If you really *wanted* the ExceptionGroup, you can
> retrieve it from the __cause__ attribute. (The only case I can think
> of where this would be useful is if you're writing code that has to
> straddle both old and new Python versions *and* wants to do something
> clever with ExceptionGroups. I think this would happen if you're
> implementing Trio, or implementing a higher-level backport library for
> catching ExceptionGroups, something like that. So this only applies to
> like half a dozen users total, but they are important users :-).)
>

This is all very clever, but something feels off with it. Perhaps it
improves this:
```
try:
raise ExceptionGroup(ValueError)  # Hidden in more complex code
except ValueError:
. . .
```
(Though how?

[Python-Dev] Re: PEP 654 -- Exception Groups and except* : request for feedback for SC submission

2021-02-26 Thread Marco Sulla

Excuse me if I post here. Maybe is a stupid question: why, instead of
introducing except*, Python can't extend the functionality of except,
so it can do what except* would do?
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/34E3TMVLHRT2UQSH5RWZXOPT6AQ6DTYZ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 654 -- Exception Groups and except* : request for feedback for SC submission

On Fri, Feb 26, 2021 at 2:42 PM Nathaniel Smith  wrote:

> On Fri, Feb 26, 2021 at 5:05 AM Irit Katriel 
> wrote:
> > I'm not sure it's safe to assume that it is necessarily a programming
> error, and that the interpreter can essentially break the program in this
> case.
> > Is this not allowed?
> >
> > try:
> > try:
> > obj.func()# function that raises ExceptionGroups
> > except AttributeError:
> > logger.info("obj doesn't have a func")
> > except *(AttributeError, SyntaxError):
> > logger.info("func had some problems")
>
> I'd be fine with disallowing that. The intuition is that things will
> be simplest if ExceptionGroup is kept as transparent and meaningless
> as possible, i.e. ExceptionGroup(ValueError) and ValueError mean
> exactly the same thing -- "some code inside this block raised
> ValueError" -- and ideally should be processed in exactly the same
> way.

But they don't mean the same thing if this prints hello:

try:
raise ValueError()
except ValueError:
   print('hello')

and this raises a RuntimeError:

try:
raise ExceptionGroup("", [ValueError()])
except ValueError:
   print('hello')

What am I missing?
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/ZQYMOWWCFBP3TBBNZGD5X45CWJNC4HFF/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 654 -- Exception Groups and except* : request for feedback for SC submission

2021-02-26 Thread Emily Bowman

A side benefit is that if an Exception somehow propagates up where only
ExceptionGroup is defined, except *() could just work anyway, though it
might take a little magic to make sure they act the same. Like Guido said,
I don't think it can be retrofitted into existing *-less APIs, and it'll
either need a new API and deprecation, or dumping the old for the new
hotness if that's the maintenance strategy, but as long as docs call out
that this is now returning collective exceptions, I don't see a problem.

I don't think it's been mentioned, but it's a nice synergy with pattern
matching, even if it's wholly unrelated, and having both in one release
will actually make sense.

-Em


On Fri, Feb 26, 2021 at 6:45 AM Nathaniel Smith  wrote:

> On Fri, Feb 26, 2021 at 5:05 AM Irit Katriel 
> wrote:
> > I'm not sure it's safe to assume that it is necessarily a programming
> error, and that the interpreter can essentially break the program in this
> case.
> > Is this not allowed?
> >
> > try:
> > try:
> > obj.func()# function that raises ExceptionGroups
> > except AttributeError:
> > logger.info("obj doesn't have a func")
> > except *(AttributeError, SyntaxError):
> > logger.info("func had some problems")
>
> I'd be fine with disallowing that. The intuition is that things will
> be simplest if ExceptionGroup is kept as transparent and meaningless
> as possible, i.e. ExceptionGroup(ValueError) and ValueError mean
> exactly the same thing -- "some code inside this block raised
> ValueError" -- and ideally should be processed in exactly the same
> way. (Of course we can't quite achieve that due to backcompat issues,
> but the closer we can get, the better, I think?)
>
> If you need to distinguish between the AttributeError from
> 'obj.__getattr__("func")' vs the AttributeError from the call to
> func(), then there's already an obvious way to do that, that works for
> all functions, not just ones that happen to raise ExceptionGroups:
>
> try:
> f = obj.func
> except AttributeError:
> ...
> try:
> f()
> except ...:# or except *...:
> ...
>
> -n
>
> --
> Nathaniel J. Smith -- https://vorpus.org
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/W3GMOA7RVADLBL7L3VB2R3SZULS7QZ7W/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 654 -- Exception Groups and except* : request for feedback for SC submission

2021-02-26 Thread Nathaniel Smith

On Fri, Feb 26, 2021 at 5:05 AM Irit Katriel  wrote:
> I'm not sure it's safe to assume that it is necessarily a programming error, 
> and that the interpreter can essentially break the program in this case.
> Is this not allowed?
>
> try:
> try:
> obj.func()# function that raises ExceptionGroups
> except AttributeError:
> logger.info("obj doesn't have a func")
> except *(AttributeError, SyntaxError):
> logger.info("func had some problems")

I'd be fine with disallowing that. The intuition is that things will
be simplest if ExceptionGroup is kept as transparent and meaningless
as possible, i.e. ExceptionGroup(ValueError) and ValueError mean
exactly the same thing -- "some code inside this block raised
ValueError" -- and ideally should be processed in exactly the same
way. (Of course we can't quite achieve that due to backcompat issues,
but the closer we can get, the better, I think?)

If you need to distinguish between the AttributeError from
'obj.__getattr__("func")' vs the AttributeError from the call to
func(), then there's already an obvious way to do that, that works for
all functions, not just ones that happen to raise ExceptionGroups:

try:
f = obj.func
except AttributeError:
...
try:
f()
except ...:# or except *...:
...

-n

-- 
Nathaniel J. Smith -- https://vorpus.org
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/VSIGKUNU2GNLMKKH4GLMRSICTQYVLVCZ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 654 -- Exception Groups and except* : request for feedback for SC submission

FTR: nobody on this long thread so far has suggested that there are no
valid use cases for `except Exception`.

Thank you for turning to what happens with 'except ValueError' when an
ExceptionGroup[ValueError] is raised, this is important.

I'm not sure it's safe to assume that it is necessarily a programming
error, and that the interpreter can essentially break the program in this
case.
Is this not allowed?

try:
try:
obj.func()# function that raises ExceptionGroups
except AttributeError:
logger.info("obj doesn't have a func")
except *(AttributeError, SyntaxError):
logger.info("func had some problems")



On Fri, Feb 26, 2021 at 5:40 AM Nathaniel Smith  wrote:

> On Thu, Feb 25, 2021 at 2:13 PM Guido van Rossum  wrote:
> >
> > So is "fail-fast if you forget to handle an ExceptionGroup" really a
> feature? (Do we call this out in the PEP?)
> >
> > We may believe that "except Exception" is an abuse, but it is too common
> to dismiss out of hand. I think if some app has e.g. a main loop where they
> repeatedly do something that may fail in many ways (e.g. handle a web
> request), catch all errors and then just log the error and continue from
> the top, it's a better experience if it logs "ExceptionGroup: 
> []" than if it crashes.
>
> Yeah, 'except Exception' happens a lot in the wild, and what to do
> about that has been a major sticking point in the ExceptionGroup
> debates all along. I wouldn't say that 'except Exception' is an abuse
> even -- what do you want gunicorn to do if your buggy flask app raises
> some random exception? Crash your entire web server, or log it and
> attempt to keep going? (This is almost your example, but adding in the
> part where gunicorn is reliable and well-respected, and that its whole
> job is to invoke arbitrarily flaky code written by random users.)
> Yury/I/others did discuss the idea of a
> BaseExceptionGroup/ExceptionGroup split a lot, and I think the general
> feeling is that it could potentially work, but feels like a
> complicated and awkward hack, so no-one was super excited about it.
> For a while we also had a compromise design where only
> BaseExceptionGroup was built-in, but we left it non-final specifically
> so asyncio could define an ExceptionsOnlyExceptionGroup.
>
> Another somewhat-related awkward part of the API is how ExceptionGroup
> and plain-old 'except' should interact *in general*. The intuition is
> that if you have 'except ValueError' and you get an
> 'ExceptionGroup(ValueError)', then the user's code has some kind of
> problem and we should probably do something? to let them know? One
> idea I had was that we should raise a RuntimeError if this happens,
> sort of similar to PEP 479. But I could never quite figure out how
> this would help (gunicorn crashing with a RuntimeError isn't obviously
> better than gunicorn crashing with an ExceptionGroup).
>
> == NEW IDEA THAT MAYBE SOLVES BOTH PROBLEMS ==
>
> Proposal:
>
> - any time an unwinding ExceptionGroup encounters a traditional
> try/except, then it gets replaced with a RuntimeError whose __cause__
> is set to the original ExceptionGroup and whose first traceback entry
> points to the offending try/except block
>
> - CUTE BIT I ONLY JUST THOUGHT OF: this substitution happens right
> *before* we start evaluating 'except' clauses for this try/except
>
> So for example:
>
> If an ExceptionGroup hits an 'except Exception': The ExceptionGroup is
> replaced by a RuntimeError. RuntimeError is an Exception, so the
> 'except Exception' clause catches it. And presumably logs it or
> something. This way your log contains both a notification that you
> might want to switch to except* (from the RuntimeError), *along with*
> the full original exception details (from the __cause__ attribute). If
> it was an ExceptionGroup(KeyboardInterrupt), then it still gets caught
> and that's not so great, but at least you get the RuntimeError to
> point out that something has gone wrong and tell you where?
>
> If an ExceptionGroup(ValueError) hits an 'except ValueError': it
> doesn't get caught, *but* a RuntimeError keeps propagating out to tell
> you you have a problem. And when that RuntimeError eventually hits the
> top of your program or ends up in your webserver logs or whatever,
> then the RuntimeError's traceback will point you to the 'except
> ValueError' that needs to be fixed.
>
> If you write 'except ExceptionGroup': this clause is a no-op that will
> never execute, because it's impossible to still have an ExceptionGroup
> when we start matching 'except' clauses. (We could additionally emit a
> diagnostic if we want.)
>
> If you write bare 'except:', or 'except BaseException': the clause
> always executes (as before), but they get the RuntimeError instead of
> the ExceptionGroup. If you really *wanted* the ExceptionGroup, you can
> retrieve it from the __cause__ attribute. (The only case I can think
> of where this would be useful is if you're writing code that has to
>

[Python-Dev] Re: PEP 654 -- Exception Groups and except* : request for feedback for SC submission

2021-02-25 Thread Nathaniel Smith

On Thu, Feb 25, 2021 at 10:23 PM Glenn Linderman  wrote:
> So then why do you need  except*  at all?  Only to catch unwrapped
> ExceptionGroup before it gets wrapped?
>
> So why not write except ExceptionGroup, and let it catch unwrapped
> ExceptionGroup?
>
> That "CUTE BIT" could be done only when hitting an except chain that
> doesn't include an except ExceptionGroup.
>
> Nope, I didn't read the PEP, and don't understand the motivation, but
> the discussion sure sounded confusing. This is starting to sound almost
> reasonable.

I'm not sure what to make of the complaint that not reading the
motivation makes the motivation confusing :-).

But very briefly: the core reason we need ExceptionGroup is because in
concurrent programs, multiple things can go wrong at the same time, so
our error handling system needs to have some way to represent and cope
with that. This means that in concurrent programs (e.g. anything using
asyncio, trio, etc.), it's safest to assume that *any* exception could
be wrapped in an ExceptionGroup and use except* for everything. The
question is how to make error handling in those programs as ergonomic
as Python exceptions are currently in non-concurrent programs, without
creating too many issues for existing code.

So in programs like this, you have to assume that 'except
ExceptionGroup' catches *all* exceptions, or worse, a
random/unpredictable subset. Saying that that would be good enough is
like saying that bare 'except:' is good enough for regular
non-concurrent Python (after all, you can always do an isinstance
check inside the 'except' block and re-raise the ones you don't want,
right?), and that 'except ' is a pointless frivolity. I
think most people would disagree with that :-).

-n

-- 
Nathaniel J. Smith -- https://vorpus.org
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/BYCDFS5FVIF4FYNKUPVEVWRSCCEIU6CB/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 654 -- Exception Groups and except* : request for feedback for SC submission

2021-02-25 Thread Glenn Linderman

On 2/25/2021 9:40 PM, Nathaniel Smith wrote:

On Thu, Feb 25, 2021 at 2:13 PM Guido van Rossum wrote:

So is "fail-fast if you forget to handle an ExceptionGroup" really a feature?
(Do we call this out in the PEP?)

We may believe that "except Exception" is an abuse, but it is too common to dismiss out of hand. I
think if some app has e.g. a main loop where they repeatedly do something that may fail in many ways (e.g.
handle a web request), catch all errors and then just log the error and continue from the top, it's a better
experience if it logs "ExceptionGroup: []" than if it
crashes.

Yeah, 'except Exception' happens a lot in the wild, and what to do
about that has been a major sticking point in the ExceptionGroup
debates all along. I wouldn't say that 'except Exception' is an abuse
even -- what do you want gunicorn to do if your buggy flask app raises
some random exception? Crash your entire web server, or log it and
attempt to keep going? (This is almost your example, but adding in the
part where gunicorn is reliable and well-respected, and that its whole
job is to invoke arbitrarily flaky code written by random users.)
Yury/I/others did discuss the idea of a
BaseExceptionGroup/ExceptionGroup split a lot, and I think the general
feeling is that it could potentially work, but feels like a
complicated and awkward hack, so no-one was super excited about it.
For a while we also had a compromise design where only
BaseExceptionGroup was built-in, but we left it non-final specifically
so asyncio could define an ExceptionsOnlyExceptionGroup.

Another somewhat-related awkward part of the API is how ExceptionGroup
and plain-old 'except' should interact *in general*. The intuition is
that if you have 'except ValueError' and you get an
'ExceptionGroup(ValueError)', then the user's code has some kind of
problem and we should probably do something? to let them know? One
idea I had was that we should raise a RuntimeError if this happens,
sort of similar to PEP 479. But I could never quite figure out how
this would help (gunicorn crashing with a RuntimeError isn't obviously
better than gunicorn crashing with an ExceptionGroup).

== NEW IDEA THAT MAYBE SOLVES BOTH PROBLEMS ==

Proposal:

- any time an unwinding ExceptionGroup encounters a traditional
try/except, then it gets replaced with a RuntimeError whose __cause__
is set to the original ExceptionGroup and whose first traceback entry
points to the offending try/except block

- CUTE BIT I ONLY JUST THOUGHT OF: this substitution happens right
*before* we start evaluating 'except' clauses for this try/except

So for example:

If an ExceptionGroup hits an 'except Exception': The ExceptionGroup is
replaced by a RuntimeError. RuntimeError is an Exception, so the
'except Exception' clause catches it. And presumably logs it or
something. This way your log contains both a notification that you
might want to switch to except* (from the RuntimeError), *along with*
the full original exception details (from the __cause__ attribute). If
it was an ExceptionGroup(KeyboardInterrupt), then it still gets caught
and that's not so great, but at least you get the RuntimeError to
point out that something has gone wrong and tell you where?

If an ExceptionGroup(ValueError) hits an 'except ValueError': it
doesn't get caught, *but* a RuntimeError keeps propagating out to tell
you you have a problem. And when that RuntimeError eventually hits the
top of your program or ends up in your webserver logs or whatever,
then the RuntimeError's traceback will point you to the 'except
ValueError' that needs to be fixed.

If you write 'except ExceptionGroup': this clause is a no-op that will
never execute, because it's impossible to still have an ExceptionGroup
when we start matching 'except' clauses. (We could additionally emit a
diagnostic if we want.)

If you write bare 'except:', or 'except BaseException': the clause
always executes (as before), but they get the RuntimeError instead of
the ExceptionGroup. If you really *wanted* the ExceptionGroup, you can
retrieve it from the __cause__ attribute. (The only case I can think
of where this would be useful is if you're writing code that has to
straddle both old and new Python versions *and* wants to do something
clever with ExceptionGroups. I think this would happen if you're
implementing Trio, or implementing a higher-level backport library for
catching ExceptionGroups, something like that. So this only applies to
like half a dozen users total, but they are important users :-).)

-n

I wondered why an ExceptionGroup couldn't just be an Exception. This
effectively makes it so (via wrapping).

So then why do you need except* at all? Only to catch unwrapped
ExceptionGroup before it gets wrapped?

So why not write except ExceptionGroup, and let it catch unwrapped
ExceptionGroup?

That "CUTE BIT" could be done only when hitting an except chain that
doesn't include an except ExceptionGroup.

Nope, I didn't read the PEP, and don't

[Python-Dev] Re: PEP 654 -- Exception Groups and except* : request for feedback for SC submission

2021-02-25 Thread Nathaniel Smith

On Thu, Feb 25, 2021 at 2:13 PM Guido van Rossum  wrote:
>
> So is "fail-fast if you forget to handle an ExceptionGroup" really a feature? 
> (Do we call this out in the PEP?)
>
> We may believe that "except Exception" is an abuse, but it is too common to 
> dismiss out of hand. I think if some app has e.g. a main loop where they 
> repeatedly do something that may fail in many ways (e.g. handle a web 
> request), catch all errors and then just log the error and continue from the 
> top, it's a better experience if it logs "ExceptionGroup:  [ subexceptions>]" than if it crashes.

Yeah, 'except Exception' happens a lot in the wild, and what to do
about that has been a major sticking point in the ExceptionGroup
debates all along. I wouldn't say that 'except Exception' is an abuse
even -- what do you want gunicorn to do if your buggy flask app raises
some random exception? Crash your entire web server, or log it and
attempt to keep going? (This is almost your example, but adding in the
part where gunicorn is reliable and well-respected, and that its whole
job is to invoke arbitrarily flaky code written by random users.)
Yury/I/others did discuss the idea of a
BaseExceptionGroup/ExceptionGroup split a lot, and I think the general
feeling is that it could potentially work, but feels like a
complicated and awkward hack, so no-one was super excited about it.
For a while we also had a compromise design where only
BaseExceptionGroup was built-in, but we left it non-final specifically
so asyncio could define an ExceptionsOnlyExceptionGroup.

Another somewhat-related awkward part of the API is how ExceptionGroup
and plain-old 'except' should interact *in general*. The intuition is
that if you have 'except ValueError' and you get an
'ExceptionGroup(ValueError)', then the user's code has some kind of
problem and we should probably do something? to let them know? One
idea I had was that we should raise a RuntimeError if this happens,
sort of similar to PEP 479. But I could never quite figure out how
this would help (gunicorn crashing with a RuntimeError isn't obviously
better than gunicorn crashing with an ExceptionGroup).

== NEW IDEA THAT MAYBE SOLVES BOTH PROBLEMS ==

Proposal:

- any time an unwinding ExceptionGroup encounters a traditional
try/except, then it gets replaced with a RuntimeError whose __cause__
is set to the original ExceptionGroup and whose first traceback entry
points to the offending try/except block

- CUTE BIT I ONLY JUST THOUGHT OF: this substitution happens right
*before* we start evaluating 'except' clauses for this try/except

So for example:

If an ExceptionGroup hits an 'except Exception': The ExceptionGroup is
replaced by a RuntimeError. RuntimeError is an Exception, so the
'except Exception' clause catches it. And presumably logs it or
something. This way your log contains both a notification that you
might want to switch to except* (from the RuntimeError), *along with*
the full original exception details (from the __cause__ attribute). If
it was an ExceptionGroup(KeyboardInterrupt), then it still gets caught
and that's not so great, but at least you get the RuntimeError to
point out that something has gone wrong and tell you where?

If an ExceptionGroup(ValueError) hits an 'except ValueError': it
doesn't get caught, *but* a RuntimeError keeps propagating out to tell
you you have a problem. And when that RuntimeError eventually hits the
top of your program or ends up in your webserver logs or whatever,
then the RuntimeError's traceback will point you to the 'except
ValueError' that needs to be fixed.

If you write 'except ExceptionGroup': this clause is a no-op that will
never execute, because it's impossible to still have an ExceptionGroup
when we start matching 'except' clauses. (We could additionally emit a
diagnostic if we want.)

If you write bare 'except:', or 'except BaseException': the clause
always executes (as before), but they get the RuntimeError instead of
the ExceptionGroup. If you really *wanted* the ExceptionGroup, you can
retrieve it from the __cause__ attribute. (The only case I can think
of where this would be useful is if you're writing code that has to
straddle both old and new Python versions *and* wants to do something
clever with ExceptionGroups. I think this would happen if you're
implementing Trio, or implementing a higher-level backport library for
catching ExceptionGroups, something like that. So this only applies to
like half a dozen users total, but they are important users :-).)

-n

-- 
Nathaniel J. Smith -- https://vorpus.org
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/JYRWNHRBPFQQPT44TZQRTVFNXPAY27UL/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 654 -- Exception Groups and except* : request for feedback for SC submission

On Fri, Feb 26, 2021 at 2:00 AM Guido van Rossum  wrote:

> Then we should get some more opinions on this. I think it's the best idea
> so far for kindness towards code using `except Exception:`.
>

I agree that it's the best idea so far.
If someone says 'except Exception' they better mean it because we'll
believe them, and if someone forgets to handle an ExceptionGroup then they
have a bug and that's how it is.



> OT: Is ExceptionGroup *really* immutable in the current implementation? As
> long as the 'errors' field is a list, I think one could mutate the list
> directly.
>

It's not, but we were going to make it an immutable tuple.


> Which brings me to the question, do you have a branch that matches the PEP
> yet?
>
>
The implementation matches the PEP less and less every day because the PEP
is developing faster than the implementation.
But these aren't differences that are more than a technicality to fix
(rename things, move an error from runtime to the parser, things like that).
The except* implementation is probably pretty close to the PEP because it's
the most recent bit.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/I473Z5E3FA3SV7MLT74UNPWEG3W2FOCA/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 654 -- Exception Groups and except* : request for feedback for SC submission

Great. Let’s wait for others to catch up with the discussion then.

On Thu, Feb 25, 2021 at 18:44 Irit Katriel 
wrote:

>
>
> On Fri, Feb 26, 2021 at 2:00 AM Guido van Rossum  wrote:
>
>> Then we should get some more opinions on this. I think it's the best idea
>> so far for kindness towards code using `except Exception:`.
>>
>
> I agree that it's the best idea so far.
> If someone says 'except Exception' they better mean it because we'll
> believe them, and if someone forgets to handle an ExceptionGroup then they
> have a bug and that's how it is.
>
>
>
>> OT: Is ExceptionGroup *really* immutable in the current implementation?
>> As long as the 'errors' field is a list, I think one could mutate the list
>> directly.
>>
>
> It's not, but we were going to make it an immutable tuple.
>
>
>> Which brings me to the question, do you have a branch that matches the
>> PEP yet?
>>
>>
> The implementation matches the PEP less and less every day because the PEP
> is developing faster than the implementation.
> But these aren't differences that are more than a technicality to fix
> (rename things, move an error from runtime to the parser, things like that).
> The except* implementation is probably pretty close to the PEP because
> it's the most recent bit.
>
>
>
>
>
>
> --
--Guido (mobile)
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/VS2JC7OK67D3DA3E45IR3IKHZWSLGBFO/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 654 -- Exception Groups and except* : request for feedback for SC submission

On Thu, Feb 25, 2021 at 9:06 PM Guido van Rossum  wrote:

>
> Hm, a different idea: maybe it's simple enough that we can just add an
> example showing how to do this? Then people can tailor that e.g. to use
> various traversal orders. (We could just link to the code in traceback.py,
> but it probably is full of distractions.)
>

I've added it here: https://github.com/python/peps/pull/1841
I'd rather do this for now, and add it to the standard library only when we
have a better idea about common patterns // best practices for handling
ExceptionGroups.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/4UDHU7ZMZBGS5V6A62HHYBPG4JCJICQZ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 654 -- Exception Groups and except* : request for feedback for SC submission

Then we should get some more opinions on this. I think it's the best idea
so far for kindness towards code using `except Exception:`. I even think
that the names I came up with are reasonable. The pseudo-code needs work:
when instantiating ExceptionGroup, all errors should inherit from
Exception, and I can't decide which base class for ExceptionGroup should
come first: Exception or BaseExceptionGroup. I'm currently leaning slightly
towards putting BaseExceptionGroup first. This is my current attempt:
```
class BaseExceptionGroup(BaseException):
def __new__(cls, msg, errors):
if cls is BaseExceptionGroup and all(isinstance(e, Exception) for e
in errors):
cls = ExceptionGroup
else:
assert all(isinstance(e, Exception) for e in errors)
return super().__new__(cls, msg, errors)
def __init__(self, msg, errors):
self.msg = msg
self.errors = errors
class ExceptionGroup(BaseExceptionGroup, Exception):
pass
```

OT: Is ExceptionGroup *really* immutable in the current implementation? As
long as the 'errors' field is a list, I think one could mutate the list
directly. Which brings me to the question, do you have a branch that
matches the PEP yet?

--Guido

On Thu, Feb 25, 2021 at 3:26 PM Irit Katriel 
wrote:

>
> We don't call it out but we talked about this at the sprint.  I think the
> reason we didn't come up with this solution then is that at the time
> ExceptionGroups in our plans were not immutable, so this would not have
> been possible (you don't know at construction time what it is going to
> contain).
>
>
> On Thu, Feb 25, 2021 at 10:08 PM Guido van Rossum 
> wrote:
>
>> So is "fail-fast if you forget to handle an ExceptionGroup" really a
>> feature? (Do we call this out in the PEP?)
>>
>> We may believe that "except Exception" is an abuse, but it is too common
>> to dismiss out of hand. I think if some app has e.g. a main loop where they
>> repeatedly do something that may fail in many ways (e.g. handle a web
>> request), catch all errors and then just log the error and continue from
>> the top, it's a better experience if it logs "ExceptionGroup: 
>> []" than if it crashes.
>>
>> There's also code that catches Exception, logs the error and some
>> relevant data, and then re-raises it. The logged error may go to a more
>> specialized destination than the generic traceback, and the log message may
>> contain additional data that's only available in that stack frame.
>>
>> So I think there are enough serious use cases that we should do what's
>> best for those use cases, and not try to lecture users about abuse of the
>> idiom.
>>
>> I don't know what we would have done if we were building Python from
>> scratch. Possibly we would not have BaseException at all, and the whole
>> mess would go away. (But there are some good reasons why we introduced
>> BaseException, so I don't know that that would really be a better overall
>> experience.)
>>
>> --Guido
>>
>>
>>

-- 
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/ZAEF52BHSTDPD5GF2O7YDYL6OSTOEJS2/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 654 -- Exception Groups and except* : request for feedback for SC submission

We don't call it out but we talked about this at the sprint.  I think the
reason we didn't come up with this solution then is that at the time
ExceptionGroups in our plans were not immutable, so this would not have
been possible (you don't know at construction time what it is going to
contain).


On Thu, Feb 25, 2021 at 10:08 PM Guido van Rossum  wrote:

> So is "fail-fast if you forget to handle an ExceptionGroup" really a
> feature? (Do we call this out in the PEP?)
>
> We may believe that "except Exception" is an abuse, but it is too common
> to dismiss out of hand. I think if some app has e.g. a main loop where they
> repeatedly do something that may fail in many ways (e.g. handle a web
> request), catch all errors and then just log the error and continue from
> the top, it's a better experience if it logs "ExceptionGroup: 
> []" than if it crashes.
>
> There's also code that catches Exception, logs the error and some relevant
> data, and then re-raises it. The logged error may go to a more specialized
> destination than the generic traceback, and the log message may contain
> additional data that's only available in that stack frame.
>
> So I think there are enough serious use cases that we should do what's
> best for those use cases, and not try to lecture users about abuse of the
> idiom.
>
> I don't know what we would have done if we were building Python from
> scratch. Possibly we would not have BaseException at all, and the whole
> mess would go away. (But there are some good reasons why we introduced
> BaseException, so I don't know that that would really be a better overall
> experience.)
>
> --Guido
>
>
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/DJ265GSMTHIIDLKR5JTV3NILOYCLVJDX/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 654 -- Exception Groups and except* : request for feedback for SC submission

So is "fail-fast if you forget to handle an ExceptionGroup" really a
feature? (Do we call this out in the PEP?)

We may believe that "except Exception" is an abuse, but it is too common to
dismiss out of hand. I think if some app has e.g. a main loop where they
repeatedly do something that may fail in many ways (e.g. handle a web
request), catch all errors and then just log the error and continue from
the top, it's a better experience if it logs "ExceptionGroup: 
[]" than if it crashes.

There's also code that catches Exception, logs the error and some relevant
data, and then re-raises it. The logged error may go to a more specialized
destination than the generic traceback, and the log message may contain
additional data that's only available in that stack frame.

So I think there are enough serious use cases that we should do what's best
for those use cases, and not try to lecture users about abuse of the idiom.

I don't know what we would have done if we were building Python from
scratch. Possibly we would not have BaseException at all, and the whole
mess would go away. (But there are some good reasons why we introduced
BaseException, so I don't know that that would really be a better overall
experience.)

--Guido

On Thu, Feb 25, 2021 at 2:46 AM Irit Katriel 
wrote:

>
>
> On Thu, Feb 25, 2021 at 5:59 AM Guido van Rossum  wrote:
>
>>
>> Here's a potentially alternative plan, which is also complex, but doesn't
>> require asyncio or other use cases to define special classes. Let's define
>> two exceptions, BaseExceptionGroup which wraps BaseException instances, and
>> ExceptionGroup which only wraps Exception instances. (Names to be
>> bikeshedded.) They could share a constructor (always invoked via
>> BaseExceptionGroup) which chooses the right class depending on whether
>> there are any non-Exception instances being wrapped -- this would do the
>> right thing for split() and subgroup() and re-raising unhandled exceptions.
>>
>> Then 'except Exception:' would catch ExceptionGroup but not
>> BaseExceptionGroup, so if a group wraps e.g. KeyboardError it wouldn't be
>> caught (even if there's also e.g. a ValueError among the wrapped errors).
>>
>> Pseudo-code:
>>
>> class BaseExceptionGroup(BaseException):
>> def __new__(cls, msg, errors):
>> if cls is BaseExceptionGroup and all(isinstance(e, Exception) for
>> e in errors):
>> cls = ExceptionGroup
>> return BaseException.__new__(cls, msg, errors)
>>
>> class ExceptionGroup(Exception, BaseExceptionGroup):
>> pass
>>
>
>
> This could be a valid compromise.
>
> We keep the ability to wrap any exception, while we lose the "fail-fast if
> you forget to handle an ExceptionGroup" feature, which was intended as a
> kindness towards those who abuse "except Exception".
>
> If we adopt this solution then letting an ExceptionGroup escape from code
> that is not supposed to raise it, is not a fatal error, it's just some
> exception like any other.
> So there is no longer a distinction between code that raises
> ExceptionGroups and code that doesn't. Any code can propagate them, like
> any code can raise any other exception.
> Does this mean that more code needs to be aware of the possibility of them
> showing up?  Is that a problem?  Maybe this a simpler state of affairs
> overall.
>
> What would we have done here if we were building Python from scratch?
>
> Irit
>
>
>
>
>
>
>
>

-- 
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/45U4VDHH3E7JJNX2L5JCUE4CJJEMFRNH/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 654 -- Exception Groups and except* : request for feedback for SC submission

Good question. The main use case I see for iterating over individual
exceptions is logging frameworks that want to format their own exception
tracebacks for whatever reason. I know that the traceback.py module has
extensive utilities for exactly that, but I betcha there are plenty of
people who still want to roll their own, e.g. for compatibility with other
tooling on their platform.

Presumably the code for traversing a tree of exceptions exists in
traceback.py. I honestly don't think it matters much whether we do it as an
iterator or using callbacks, as long as it visits all the leaves in the
tree.

Hm, a different idea: maybe it's simple enough that we can just add an
example showing how to do this? Then people can tailor that e.g. to use
various traversal orders. (We could just link to the code in traceback.py,
but it probably is full of distractions.)

On Thu, Feb 25, 2021 at 3:01 AM Irit Katriel 
wrote:

>
>
> On Thu, Feb 25, 2021 at 5:19 AM Guido van Rossum  wrote:
>
>> [Subthread: handling individual errors]
>>
>> > Rather than iterator, I think we should add a visitor that calls a
>> function for each leaf exception,
>>
>> I agree that we shouldn't add an `__iter__` method. But what if we added
>> a separate method that returns an iterator? Then you could write
>> ```
>> for e in eg.something():
>> print("caught", e)
>> ```
>> This looks cleaner to me than having to write
>> ```
>> def handler(e):
>> print("caught", e)
>> eg.something(handler)
>> ```
>> (Does my bias against callback functions shine through? :-)
>>
>
>
> I hear you. I suggested a visitor to make it a bit awkward to use because
> I'm not sure why people should iterate over individual exceptions in an
> ExceptionGroup, and I think that by providing an iteration utility we are
> implying that this is what you should do.
> So let me be more direct instead of proposing passive-aggressive APIs.
>
> Can we take a step back and talk about how we think people would want to
> handle ExceptionGroups?
>
> In the rejected ideas section we discuss this a bit, with a reference to
> Yury's writeup:
> https://github.com/python/exceptiongroups/issues/3#issuecomment-716203284
>
> TL;DR:  when you get a group of exceptions from asyncio or the like, you
> may want to query it for exception types is contains (with subgroup()), you
> may want to format it into a log (with traceback.* methods), but you are
> unlikely to care whether there are 1, 2 or 300 ValueErrors. Your program
> will probably do the same thing regardless. If you allowed your ValueError
> get collected into an ExceptionGroup you already lost the context in which
> it happened to it's unlikely that you can make a targeted recovery which is
> relevant to this particular exception.
>
> So, what is the use case for iterating over single exceptions?
>
>
>>

-- 
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/KA6PVBBGZUNJ43YF44C63SAPQOODUPJH/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 654 -- Exception Groups and except* : request for feedback for SC submission

On Thu, Feb 25, 2021 at 5:19 AM Guido van Rossum  wrote:

> [Subthread: handling individual errors]
>
> > Rather than iterator, I think we should add a visitor that calls a
> function for each leaf exception,
>
> I agree that we shouldn't add an `__iter__` method. But what if we added a
> separate method that returns an iterator? Then you could write
> ```
> for e in eg.something():
> print("caught", e)
> ```
> This looks cleaner to me than having to write
> ```
> def handler(e):
> print("caught", e)
> eg.something(handler)
> ```
> (Does my bias against callback functions shine through? :-)
>

I hear you. I suggested a visitor to make it a bit awkward to use because
I'm not sure why people should iterate over individual exceptions in an
ExceptionGroup, and I think that by providing an iteration utility we are
implying that this is what you should do.
So let me be more direct instead of proposing passive-aggressive APIs.

Can we take a step back and talk about how we think people would want to
handle ExceptionGroups?

In the rejected ideas section we discuss this a bit, with a reference to
Yury's writeup:
https://github.com/python/exceptiongroups/issues/3#issuecomment-716203284

TL;DR:  when you get a group of exceptions from asyncio or the like, you
may want to query it for exception types is contains (with subgroup()), you
may want to format it into a log (with traceback.* methods), but you are
unlikely to care whether there are 1, 2 or 300 ValueErrors. Your program
will probably do the same thing regardless. If you allowed your ValueError
get collected into an ExceptionGroup you already lost the context in which
it happened to it's unlikely that you can make a targeted recovery which is
relevant to this particular exception.

So, what is the use case for iterating over single exceptions?

>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/GYSBF6MZDAYG7SMRZGTDXTXY3TYWZAT3/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 654 -- Exception Groups and except* : request for feedback for SC submission

On Thu, Feb 25, 2021 at 5:59 AM Guido van Rossum  wrote:

>
> Here's a potentially alternative plan, which is also complex, but doesn't
> require asyncio or other use cases to define special classes. Let's define
> two exceptions, BaseExceptionGroup which wraps BaseException instances, and
> ExceptionGroup which only wraps Exception instances. (Names to be
> bikeshedded.) They could share a constructor (always invoked via
> BaseExceptionGroup) which chooses the right class depending on whether
> there are any non-Exception instances being wrapped -- this would do the
> right thing for split() and subgroup() and re-raising unhandled exceptions.
>
> Then 'except Exception:' would catch ExceptionGroup but not
> BaseExceptionGroup, so if a group wraps e.g. KeyboardError it wouldn't be
> caught (even if there's also e.g. a ValueError among the wrapped errors).
>
> Pseudo-code:
>
> class BaseExceptionGroup(BaseException):
> def __new__(cls, msg, errors):
> if cls is BaseExceptionGroup and all(isinstance(e, Exception) for
> e in errors):
> cls = ExceptionGroup
> return BaseException.__new__(cls, msg, errors)
>
> class ExceptionGroup(Exception, BaseExceptionGroup):
> pass
>

This could be a valid compromise.

We keep the ability to wrap any exception, while we lose the "fail-fast if
you forget to handle an ExceptionGroup" feature, which was intended as a
kindness towards those who abuse "except Exception".

If we adopt this solution then letting an ExceptionGroup escape from code
that is not supposed to raise it, is not a fatal error, it's just some
exception like any other.
So there is no longer a distinction between code that raises
ExceptionGroups and code that doesn't. Any code can propagate them, like
any code can raise any other exception.
Does this mean that more code needs to be aware of the possibility of them
showing up?  Is that a problem?  Maybe this a simpler state of affairs
overall.

What would we have done here if we were building Python from scratch?

Irit
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/L3GKLVTZWICQYH5JCJRGQ6K5QX7EWIZX/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 654 -- Exception Groups and except* : request for feedback for SC submission

On Wed, Feb 24, 2021 at 1:38 AM Irit Katriel 
wrote:

>
>
> On Wed, Feb 24, 2021 at 4:39 AM Guido van Rossum  wrote:
>
>>
>> OTOH we might reconsider deriving ExceptionGroup from BaseException --
>> maybe it's better to inherit from Exception? I don't think the PEP
>> currently has a clear reason why it must derive from BaseException. I
>> believe it has to do with the possibility that ExceptionGroup might wrap a
>> BaseException instance (e.g. KeyboardInterrupt or SystemExit).
>>
>
>
> That was the reason, and we also said that an ExceptionGroup escaping
> something that isn't supposed to be raising ExceptionGroups is a bug, so if
> you call an API that raises ExceptionGroups it is your responsibility to
> handle them.
>
> We could make ExceptionGroup be an Exception, and refuse to wrap anything
> which is not an Exception. So asyncio either raises KeyboardInterrupt or an
> ExceptionGroup of user exceptions.
>
> Those are quite different directions.
>
>>
Indeed. I don't think we require that all wrapped exceptions derive from
Exception; an important special case in asyncio is CancelledError, which
doesn't derive from Exception. (Ditto for trio.Cancelled.)

It is really worth carefully reading the section I mentioned in Trio's
MultiError v2 issue. But its "plan" seems complex, as it would require
asyncio to define `class TaskGroupError(ExceptionGroup, Exception)` -- that
was fine when we considered this *only* for asyncio, but now that we've
found several other use cases it really doesn't feel right.

Here's a potentially alternative plan, which is also complex, but doesn't
require asyncio or other use cases to define special classes. Let's define
two exceptions, BaseExceptionGroup which wraps BaseException instances, and
ExceptionGroup which only wraps Exception instances. (Names to be
bikeshedded.) They could share a constructor (always invoked via
BaseExceptionGroup) which chooses the right class depending on whether
there are any non-Exception instances being wrapped -- this would do the
right thing for split() and subgroup() and re-raising unhandled exceptions.

Then 'except Exception:' would catch ExceptionGroup but not
BaseExceptionGroup, so if a group wraps e.g. KeyboardError it wouldn't be
caught (even if there's also e.g. a ValueError among the wrapped errors).

Pseudo-code:

class BaseExceptionGroup(BaseException):
def __new__(cls, msg, errors):
if cls is BaseExceptionGroup and all(isinstance(e, Exception) for e
in errors):
cls = ExceptionGroup
return BaseException.__new__(cls, msg, errors)

class ExceptionGroup(Exception, BaseExceptionGroup):
pass

-- 
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/ZFXN3GCNWC6GMZMC3UD56NLGAHGZBLTY/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 654 -- Exception Groups and except* : request for feedback for SC submission

On Wed, Feb 24, 2021 at 7:09 PM Emily Bowman 
wrote:

> After reading through the PEP and skimming the code (but I didn't build
> it), something I didn't see: What happens to a currently conforming except
> check?
>
> try:
> async with trio.open_nursery() as nursery:
> # Make two concurrent calls to child()
> nursery.start_soon(child)
> nursery.start_soon(child)
> except ValueError:
> pass
>
> I've removed the * from the example: Say the interface was built for 3.7,
> but the "trio" module has been upgraded to use ExceptionGroups which can't
> fall back to a standalone exception.
>
> Silently hand back the first exception, or the first matching exception?
> Deprecation warning? Silently skip? Context-specific error? Run the default
> error handler?
>
> I think that deserves a statement in the PEP.
>

The ExceptionGroup would bubble up. IIRC this Trio behavior is considered
problematic by the Trio developers, because it means that if *two* children
fail with ValueError, it will raise MultiError and the except clause will
not trigger (see the "MultiError v2" issue linked from the PEP).

But changing trio.open_nursery() to raise ExceptionGroup would be
considered backwards compatible. I would recommend introducing a new API
instead. This is discussed (though maybe not in enough detail?) under
Backwards Compatibility in the PEP.

-- 
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/OYKCVAOYAJKOI24DRWMUSJUJKGQ74CQ5/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 654 -- Exception Groups and except* : request for feedback for SC submission

On Wed, Feb 24, 2021 at 5:58 PM Jim J. Jewett  wrote:

> If it (compatible __new__ and __init__) needs to be checked at definition
> time, just try create an instance passing the same arguments you would pass
> to the base class.  If the creation it doesn't raise an exception, that is
> good enough.
>

User code can do that, the interpreter should not use such heuristics.

> This isn't about theoretical type safety against malice; it is about
> defining the minimal protocol for an ExceptionGrouping that has to be
> supported by someone who wants something other than the default flavor.
>

You still haven't shown a use case for wanting to subclass ExceptionGroup.
It's easy to allow this later if there's a valid use case (it wouldn't
require a PEP, just a well-reasoned use case). But if we allow it now and
in the future we discover it causes subtle bugs, it would be difficult to
roll back (requiring deprecation over several releases). Why would you want
a different flavor of ExceptionGroup?

-- 
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/SMEJMIMSIMWOXW47KDMRL3POCRHNTSXC/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 654 -- Exception Groups and except* : request for feedback for SC submission

On Wed, Feb 24, 2021 at 5:51 PM Jim J. Jewett  wrote:

> Are you saying that
>
> except *ValueError as e:
>
> will catch
>
> ValueError
> and
> ExceptionGroup(ValueError)
> but miss
> ExceptionGroup(ExceptionGroup(ValueError))
> ?
>

No, it will catch it regardless of how many levels of ExceptionGroup wrap
it (zero or more, in fact).

-- 
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/3XIBSH7IJDIAM2YMGI42ZWQSOIHTNW2R/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 654 -- Exception Groups and except* : request for feedback for SC submission

[Subthread: handling individual errors]

> Rather than iterator, I think we should add a visitor that calls a
function for each leaf exception,

I agree that we shouldn't add an `__iter__` method. But what if we added a
separate method that returns an iterator? Then you could write
```
for e in eg.something():
print("caught", e)
```
This looks cleaner to me than having to write
```
def handler(e):
print("caught", e)
eg.something(handler)
```
(Does my bias against callback functions shine through? :-)

> plus a utility that returns the traceback of a leaf exception
> (as a single list, copying frames from the tracebacks of ExceptionGroups
so that it's not destructive.)

Hm, what would the arguments for that utility be? Since ExceptionGroups can
be nested, if you pass in the root EG and the leaf exception, it would
require a full tree search to find the nodes that have all the needed
tracebacks.

Maybe the API for the iterator-returning method should be that it returns
an iterator of (exception, list-of-tracebacks) pairs? Or maybe there should
be two iterator-returning methods, one that yields just exceptions and one
that yields such pairs, so that if you don't need the tracebacks we don't
have to construct them. (Separate methods so the static type doesn't depend
on the value of a flag.) If in the end we were to go for something with a
handler callback we could do a similar thing.

On Wed, Feb 24, 2021 at 1:27 AM Irit Katriel 
wrote:

>
>
> On Wed, Feb 24, 2021 at 4:55 AM Guido van Rossum  wrote:
>
> However there may be an omission in the PEP -- what if we want to do
>> something special for each suberror? If we just iterate over `eg.errors` we
>> would have to do something recursive for exceptions wrapped inside multiple
>> nested groups. We could add a helper method to ExceptionGroup that iterates
>> over suberrors, flattening nested groups. But we'd need to do something
>> special to recover the tracebacks there (groups share the common part of
>> the traceback). Or we could say "if you need the traceback you're going to
>> have to write something recursive" -- like we have to do in traceback.py.
>> But we might as well generalize whatever we're doing there. (Irit: I
>> suspect there's a piece of API design we missed here.)
>>
>
> This is mentioned briefly where we talk about why ExceptionGroup is not
> iterable: https://www.python.org/dev/peps/pep-0654/#the-exceptiongroup-api
> We left it out pending a use case.
>
> Rather than iterator, I think we should add a visitor that calls a
> function for each leaf exception, plus a utility that returns the traceback
> of a leaf
> exception (as a single list, copying frames from the tracebacks of
> ExceptionGroups so that it's not destructive.) This way the expensive
> traceback
> construction is happening explicitly when it is needed.
>
> I think accessing one exception at a time to do something other than
> formatting it is more likely to be overused than actually needed.
> We did that in an earlier version of the PEP for the "filter OSErrors by
> type" example, which we did with iteration and now do with subgroup:
>
> try:
> low_level_os_operation()
> except *OSerror as errors:
> raise errors.subgroup(lambda e: e.errno != errno.EPIPE) from None
>
>
>
>

-- 
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/JEXBOJQKIGIEOYJZIBJSYTFNVUUBTGLL/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 654 -- Exception Groups and except* : request for feedback for SC submission

2021-02-24 Thread Emily Bowman

After reading through the PEP and skimming the code (but I didn't build
it), something I didn't see: What happens to a currently conforming except
check?

try:
async with trio.open_nursery() as nursery:
# Make two concurrent calls to child()
nursery.start_soon(child)
nursery.start_soon(child)
except ValueError:
pass

I've removed the * from the example: Say the interface was built for 3.7,
but the "trio" module has been upgraded to use ExceptionGroups which can't
fall back to a standalone exception.

Silently hand back the first exception, or the first matching exception?
Deprecation warning? Silently skip? Context-specific error? Run the default
error handler?

I think that deserves a statement in the PEP.

-Em

On Mon, Feb 22, 2021 at 4:48 PM Irit Katriel via Python-Dev <
python-dev@python.org> wrote:

>
> Hi all,
>
> We would like to request feedback on PEP 654 -- Exception Groups and
> except*.
>
> https://www.python.org/dev/peps/pep-0654/
>
> It proposes language extensions that allow programs to raise and handle
> multiple unrelated
> exceptions simultaneously, motivated by the needs of asyncio and other
> concurrency libraries,
> but with other use cases as well.
>
> * A new standard exception type,  ExceptionGroup, to represent multiple
> exceptions with
>   shared traceback.
> * Updates to the traceback printing code to display (possibly nested)
> ExceptionGroups.
> * A new syntax except* for handling ExceptionGroups.
>
> A reference implementation (unreviewed) can be found at:
> https://github.com/iritkatriel/cpython/pull/10
>
> Thank you for your help
>
> Kind regards
> Irit, Yury & Guido
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/2ABNKHUQK2GZZBPEOSOKZFVZTQCQ2QOK/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 654 -- Exception Groups and except* : request for feedback for SC submission

Ideally, (at least) trivial subclasses could be declared, and the class itself 
would serve as the marker.  I would prefer regular subclasses, so that they 
could offer methods as well.  Alternatively, at least copy the instance 
__dict__ to the new ExceptionGroup instance.

By compatible __init__ and __new__, I mean "whatever you do in 
ExceptionGroup.subgroup to create a new instance with fewer contained 
Exceptions ... do the same with MyExceptionGroup."  I'm assuming that 
"whatever" is to call __new__ and __init__ with "a message string and a 
sequence of the nested exceptions, for example: ExceptionGroup('issues', 
[ValueError('bad value'), TypeError('bad type')])."

-jJ
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/SFE46EGTGLXLCJDXZ4EI6HOEATYREOL4/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 654 -- Exception Groups and except* : request for feedback for SC submission

If it (compatible __new__ and __init__) needs to be checked at definition time, 
just try create an instance passing the same arguments you would pass to the 
base class.  If the creation it doesn't raise an exception, that is good enough.

This isn't about theoretical type safety against malice; it is about defining 
the minimal protocol for an ExceptionGrouping that has to be supported by 
someone who wants something other than the default flavor.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/JW4BFAR7SN23NF7S6TRGOROJIMPDGQEJ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 654 -- Exception Groups and except* : request for feedback for SC submission

Are you saying that 

except *ValueError as e:

will catch

ValueError
and
ExceptionGroup(ValueError)
but miss
ExceptionGroup(ExceptionGroup(ValueError))
?
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/VLPYAILCWTGG25WYJXUA47LIQPULQ7ZM/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 654 -- Exception Groups and except* : request for feedback for SC submission

By the time a KeyboardInterrupt or SystemExit is being grouped into an 
ExceptionGroup, it already isn't being treated as an immediate interruption ... 
it has (presumably) already killed its own execution path, but it should not 
kill the program as a whole.  Whether ExceptionGroup inherits from Exception or 
BaseException shouldn't matter for deciding what it can group.

-jJ
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/XSEL22SDNEX2BWT7GS7OUPDAA2UVMH42/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 654 -- Exception Groups and except* : request for feedback for SC submission

Hi Ethan,

See Irit's response to my about why ExceptionGroup doesn't subclass
Exception -- I think this is still a little bit of an open issue.

Another response (which I also read somewhere in one of Irit's messages) is
that a library API shouldn't go from raising subclasses of Exception only
to raising ExceptionGroups. That would be a backwards incompatible change
to an API.

--Guido

On Tue, Feb 23, 2021 at 8:52 PM Ethan Furman  wrote:

> On 2/23/21 7:56 PM, Guido van Rossum wrote:
> > On Tue, Feb 23, 2021 at 7:37 PM Ethan Furman wrote:
>
> >> It sounds like the long-term goal is to move away from `except` and
> >> replace it with `except *` -- is that correct?
> >
> > I don't think so -- if we expected that to happen the extra '*' in the
> > syntax would be a nuisance. The premise of the PEP is rather that
> > raising and catching multiple exceptions at once is always going
> > to be an esoteric hobby. The most common case by far would be in async
> > frameworks, but we don't expect 'async def' to eventually become the
> > standard function definition either (nor 'await f()' the standard call
> :-).
> >
> > And even in an async framework, I don't expect that every async function
> > would be "upgraded" to raise ExceptionGroup -- only specific APIs like
> > gather() or create_connection(). (Even for those, backwards compatibility
> > concerns will probably mean that we'll have to introduce *new* APIs that
> > can raise ExceptionGroup.) The PEP lists a handful of other use cases,
> > none of which seem to suggest to me that this is going to be a common
> thing.
>
> I can see the value in `except *` for concurrent code; my concern is how
> it will operate with the standard `try/except` framework.  With the new
> non-Exception-derived ExceptionGroup class, existing try-excepts that catch
> Exception will fail for every ExceptionGroup that leaks through; the fix
> for that is a doubly-nested try-except/try-except* -- what do we get for
> that code churn?  What is the advantage in not deriving from Exception?  If
> I recall correctly, we have only three exception types that derive directly
> from BaseException: Exception, SystemExit, and KeyboardInterrupt -- with
> SystemExit and KeyboardInterrupt both being concerned with making sure an
> application can quit.
>
> --
> ~Ethan~
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/445RAHP7XXLQFFGFKCXITPQPGAB55MDY/
> Code of Conduct: http://python.org/psf/codeofconduct/
>


-- 
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/5JEJ2TM7VQIJPPH6SIJNTWX5BOCTS3CH/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 654 -- Exception Groups and except* : request for feedback for SC submission

Hi Jim,

Let me echo Irit's response here -- what's your use case for subclassing
ExceptionGroup?

Requiring that subclasses have a compatible __init__ or __new__ sounds like
a bug magnet, since this can't be checked when the subclass is being
defined, and it goes against conventional rules for those signatures.

--Guido

On Wed, Feb 24, 2021 at 6:22 AM Jim J. Jewett  wrote:

> The "no subclasses" seems pretty severe, particularly if you can't even
> use marker attributes because it isn't clear when a different instance of
> container ExceptionGroup will be substituted.
>
> The justification for this restriction was that .split() had to be be able
> to instantiate ... wouldn't it be enough to just say that subclasses need a
> compatible __init__ and __new__ for that reason, and then leave it to
> consenting adults?
>
> -jJ
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/5OLFDKJCNKLIJJQMNYYLQGV53Z7ZTRC5/
> Code of Conduct: http://python.org/psf/codeofconduct/
>


-- 
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/BXW7E5UEWIJDCA2JN5QTBZXDFKQ4RFEK/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 654 -- Exception Groups and except* : request for feedback for SC submission

2021-02-24 Thread Irit Katriel via Python-Dev

Hi Jim,

On Wed, Feb 24, 2021 at 2:16 PM Jim J. Jewett  wrote:

> Petr:  What about except *(TypeError, ExceptionGroup):?
>
> Irit:  Good question. We should block that as well.
>
> But https://www.python.org/dev/peps/pep-0654/#backwards-compatibility
> seems to explicitly recommend the very similar:
>
> except (Exception, ExceptionGroup):

That's except, not except*.

Please include an example for:
>
> except *ExceptionGroup: pass
>

There is one here:
https://www.python.org/dev/peps/pep-0654/#forbidden-combinations
It raises a Runtime Error.

The "no subclasses" seems pretty severe, particularly if you can't even use
> marker attributes because it isn't clear when a different instance of
> container ExceptionGroup will be substituted.
>
> The justification for this restriction was that .split() had to be be able
> to instantiate ... wouldn't it be enough to just say that subclasses need a
> compatible __init__ and __new__ for that reason, and then leave it to
> consenting adults?

What do you mean by compatible?  How would one's marker attributes be
copied to the new instances?

split() currently copies (context, cause, traceback) from the original
ExceptionGroup to the new ones. We could have some protocol, but I'm not
sure what it would look like. I'm pretty sure there will be a cost, and I'm
not sure I understand why no subclassing is a serious enough limitation to
justify that.

Irit
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/NMJSQW5D6O43KMLMOHY6X6Z2PXT5XYFB/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 654 -- Exception Groups and except* : request for feedback for SC submission

2021-02-24 Thread Irit Katriel via Python-Dev

On Wed, Feb 24, 2021 at 4:39 AM Guido van Rossum  wrote:

>
> OTOH we might reconsider deriving ExceptionGroup from BaseException --
> maybe it's better to inherit from Exception? I don't think the PEP
> currently has a clear reason why it must derive from BaseException. I
> believe it has to do with the possibility that ExceptionGroup might wrap a
> BaseException instance (e.g. KeyboardInterrupt or SystemExit).
>

That was the reason, and we also said that an ExceptionGroup escaping
something that isn't supposed to be raising ExceptionGroups is a bug, so if
you call an API that raises ExceptionGroups it is your responsibility to
handle them.

We could make ExceptionGroup be an Exception, and refuse to wrap anything
which is not an Exception. So asyncio either raises KeyboardInterrupt or an
ExceptionGroup of user exceptions.

Those are quite different directions.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/EO6Z3DRWFXCOMEYENLDVZSIN27T24BAI/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 654 -- Exception Groups and except* : request for feedback for SC submission

2021-02-24 Thread Irit Katriel via Python-Dev

On Wed, Feb 24, 2021 at 4:55 AM Guido van Rossum  wrote:

However there may be an omission in the PEP -- what if we want to do
> something special for each suberror? If we just iterate over `eg.errors` we
> would have to do something recursive for exceptions wrapped inside multiple
> nested groups. We could add a helper method to ExceptionGroup that iterates
> over suberrors, flattening nested groups. But we'd need to do something
> special to recover the tracebacks there (groups share the common part of
> the traceback). Or we could say "if you need the traceback you're going to
> have to write something recursive" -- like we have to do in traceback.py.
> But we might as well generalize whatever we're doing there. (Irit: I
> suspect there's a piece of API design we missed here.)
>

This is mentioned briefly where we talk about why ExceptionGroup is not
iterable: https://www.python.org/dev/peps/pep-0654/#the-exceptiongroup-api
We left it out pending a use case.

Rather than iterator, I think we should add a visitor that calls a function
for each leaf exception, plus a utility that returns the traceback of a leaf
exception (as a single list, copying frames from the tracebacks of
ExceptionGroups so that it's not destructive.) This way the expensive
traceback
construction is happening explicitly when it is needed.

I think accessing one exception at a time to do something other than
formatting it is more likely to be overused than actually needed.
We did that in an earlier version of the PEP for the "filter OSErrors by
type" example, which we did with iteration and now do with subgroup:

try:
low_level_os_operation()
except *OSerror as errors:
raise errors.subgroup(lambda e: e.errno != errno.EPIPE) from None
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/5VX7AG266VCSZUV4QYXSWD77W7LOC75X/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 654 -- Exception Groups and except* : request for feedback for SC submission

Please include an example for:

except *ExceptionGroup: pass

I *think* it swallows all exceptions, instead of re-raising.
I *think* it also catches (and then swallows) bare exception that were not 
initially part of an ExceptionGroup.
I *think* it catches anything that gets raised, not just subclasses of 
Exception (or even BaseException).
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/NTQ22LUIKNRNA2YHKOLVPR5GI2KW6TLW/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 654 -- Exception Groups and except* : request for feedback for SC submission

The "no subclasses" seems pretty severe, particularly if you can't even use 
marker attributes because it isn't clear when a different instance of container 
ExceptionGroup will be substituted.

The justification for this restriction was that .split() had to be be able to 
instantiate ... wouldn't it be enough to just say that subclasses need a 
compatible __init__ and __new__ for that reason, and then leave it to 
consenting adults?

-jJ
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/5OLFDKJCNKLIJJQMNYYLQGV53Z7ZTRC5/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 654 -- Exception Groups and except* : request for feedback for SC submission