[issue39725] unrelated `from None` exceptions lose prior exception information

[Ido Michael]

Ido Michael  added the comment:

Can I take this change into a PR?
Or is it still in the process?

--
nosy: +Ido Michael

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39725] unrelated `from None` exceptions lose prior exception information

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

Because there may be more exceptions than the code writer know. The code raised 
OverflowError and ValueError may be in different function, so the code writer 
know only about ValueError, but OverflowError is an implementation detail (and 
may be absent in other versions).

Currently the code writer has to write the cumbersome code

was_raised = False
try:
foo()  # may raise ValueError -> OverflowError
except:
was_raised = True
if was_raised:
raise KeyError from None

if they want to suppress only exceptions raised in foo(). And it is worse in 
case of finally.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39725] unrelated `from None` exceptions lose prior exception information

[Terry J. Reedy]

Terry J. Reedy  added the comment:

Would it generaly work better if 'from None' only suppressed the previous 
exception?  That is the only one that the code writer knows about.

--
nosy: +terry.reedy

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39725] unrelated `from None` exceptions lose prior exception information

[Serhiy Storchaka]

Change by Serhiy Storchaka :


--
keywords: +patch
pull_requests: +17996
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/18629

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39725] unrelated `from None` exceptions lose prior exception information

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

It is more complicated. In the following example

try:
try:
raise TypeError
except:
try:
try:
raise OverflowError
except:
raise ValueError
except:
raise KeyError from None
except BaseException as exc:
e = exc

the __context__ chain is KeyError -> ValueError -> OverflowError -> TypeError. 
We want to suppress ValueError and OverflowError, and keep KeyError -> 
TypeError. I afraid that the chain of exceptions just do not have enough 
information to do this. Maybe analyzing tracebacks can help, but it looks too 
complex.

Maybe the simpler solution is to "cut" __context__ when we leave an exception 
handler. If __suppress_context__ is true and the current handled exception is 
not __context__, set it as __context__.

Here is a PR which implements this solution. I am not sure that it is correct, 
and it may miss some corner cases, it is just an example. Pay attention to the 
ipaddress module: in some cases we want to suppress more than one exception, 
and currently it is inconvenient.


Maybe we incorrectly handle exception chains. Maybe they should be build in a 
way similar to tracebacks: initially set __context__ to None and add a handled 
exception to the end of the chain when leave an exception handler.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39725] unrelated `from None` exceptions lose prior exception information

[Ethan Furman]

Ethan Furman  added the comment:

Can this be fixed in the traceback module or is there C code behind it?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39725] unrelated `from None` exceptions lose prior exception information

[Ethan Furman]

Ethan Furman  added the comment:

I think the way forward is going to be a recursive walk back to the first 
exception, and if __suppress_context__ is true for any exception then only the 
previous exception is omitted.

For example, the above example had the following chain:

Exception__context____cause____suppress_context__
-----
TypeError: None   None  False
KeyError:  TypeError  None  False
KeyError:  KeyError   None  True

When walking the stack to decide which items get printed, the final KeyError is 
printed (obviously, as it's the last one), but because its __suppress_context__ 
is True then the immediately preceding exception, the first KeyError, is 
skipped; however, the first KeyError's __suppress_context__ is False, so we do 
print its __context__, which is the TypeError.  Giving us a traceback similar 
to:

-
Traceback (most recent call last):
TypeError: str expected, not type

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
KeyError: 'NEW_VARIABLE'
-

Which is entirely correct.

In cases where NewException has it's __cause__ set, we should print that as 
normal, then keep following the NewException.__context__ chain (with some kind 
of verbage or visual indicator between the __context__ and the __cause__).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39725] unrelated `from None` exceptions lose prior exception information

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

For reference: PEP 3134 (and superseded PEP 344), PEP 409, PEP 415.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39725] unrelated `from None` exceptions lose prior exception information

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

Interesting problem.

Example which does not depend on os.environ:

import traceback
try:
try:
raise TypeError
except:
try:
raise ValueError
except:
raise KeyError from None
except BaseException as exc:
e = exc

print([e, e.__cause__, e.__context__, e.__suppress_context__])
traceback.print_exception(type(e), e, e.__traceback__)


"from None" sets __suppress_context__ to True. Currently, if 
__suppress_context__ is true, the output of __context__ will be suppressed in 
the traceback. But what if we suppress only a part __context__ and output the 
"original" exception. The problem is what is the "original" exception. Is it 
__context__.__context__? Or __context__.__cause__ if it is not None? Should we 
take __context__.__suppress_context__ into account? Maybe go down recursively?

--
nosy: +benjamin.peterson

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39725] unrelated `from None` exceptions lose prior exception information

[Ethan Furman]

New submission from Ethan Furman :

Using the example from https://bugs.python.org/msg293185:

---
--> import os
--> try:
... os.environ["NEW_VARIABLE"] = bug  # bug is not a str
... finally:
... del os.environ["NEW_VARIABLE"]  # KeyError
... 
Traceback (most recent call last):
  ...
KeyError: 'NEW_VARIABLE'
---

We lost the original exception, `TypeError: str expected, not object`, because 
in os.py we have:

def __delitem__(self, key):
encodedkey = self.encodekey(key)
unsetenv(encodedkey)
try:
del self._data[encodedkey]
except KeyError:
# raise KeyError with the original key value
raise KeyError(key) from None


If we remove the `from None` the result of the above code is:

---
Traceback (most recent call last):
TypeError: str expected, not type

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
KeyError: b'NEW_VARIABLE'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
KeyError: 'NEW_VARIABLE'
---


There are various tricks we can do to fix this isolated issue (and others like 
it), but the real problem is that one exception handler's work was destroyed by 
an unrelated exception handler.

The intent of `from None` is to get rid of any exception details in the 
try/except block it is contained within, not to lose details from exceptions 
that were already in play when its try/except block was entered.

Any ideas on how to correct this?

--
messages: 362478
nosy: ethan.furman, ncoghlan, rhettinger, serhiy.storchaka
priority: normal
severity: normal
status: open
title: unrelated  `from None` exceptions lose prior exception information
type: behavior
versions: Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com