[issue40222] "Zero cost" exception handling

STINNER Victor added the comment: I created bpo-47236 "Document types.CodeType.replace() changes about co_exceptiontable". -- ___ Python tracker ___

[issue40222] "Zero cost" exception handling

Guido van Rossum added the comment: > See bpo-47185: code.replace(co_code=new_code) no longer catch exceptions on > Python 3.11. Surely the bigger issue is that the contents of new_code itself must be totally different? Also there are other tables that need to be adjusted if you really do

[issue40222] "Zero cost" exception handling

STINNER Victor added the comment: See bpo-47185: code.replace(co_code=new_code) no longer catch exceptions on Python 3.11. -- nosy: +vstinner ___ Python tracker ___

[issue40222] "Zero cost" exception handling

Mark Shannon added the comment: Yes. -- priority: release blocker -> resolution: -> fixed stage: patch review -> resolved status: open -> closed ___ Python tracker ___

[issue40222] "Zero cost" exception handling

Pablo Galindo Salgado added the comment: Can this be closed? -- ___ Python tracker ___ ___ Python-bugs-list mailing list

[issue40222] "Zero cost" exception handling

Irit Katriel added the comment: New changeset 7989e9dff68fd681657aa9f93ac6e8fa12c534e4 by Irit Katriel in branch 'main': bpo-40222: update doc entry with respect to the change in WITH_EXCEPT_START (GH-29975) https://github.com/python/cpython/commit/7989e9dff68fd681657aa9f93ac6e8fa12c534e4

[issue40222] "Zero cost" exception handling

Change by Irit Katriel : -- nosy: +iritkatriel nosy_count: 16.0 -> 17.0 pull_requests: +28199 pull_request: https://github.com/python/cpython/pull/29975 ___ Python tracker ___

[issue40222] "Zero cost" exception handling

Change by Irit Katriel : -- nosy: +iritkatriel, iritkatriel nosy_count: 16.0 -> 17.0 pull_requests: +28199, 28200 pull_request: https://github.com/python/cpython/pull/29975 ___ Python tracker

[issue40222] "Zero cost" exception handling

Change by STINNER Victor : -- nosy: -vstinner ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue40222] "Zero cost" exception handling

Change by Ruairidh MacLeod : -- nosy: +rkm ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue40222] "Zero cost" exception handling

Guido van Rossum added the comment: I've started a thread on python-dev. https://mail.python.org/archives/list/python-...@python.org/thread/ZWTBR5ESYR26BUIVMXOKPFRLGGYDJSFC/#ZWTBR5ESYR26BUIVMXOKPFRLGGYDJSFC -- ___ Python tracker

[issue40222] "Zero cost" exception handling

Guido van Rossum added the comment: > >> I want to get rid of PyCode_NewWithPosArgs() and just have PyCode_New(). > That as added because of PEP 387 and unfortunately removing it is backwards > incompatible. Is changing the signature allowed? Because it *must* be changed (at the very

[issue40222] "Zero cost" exception handling

Pablo Galindo Salgado added the comment: >> I want to get rid of PyCode_NewWithPosArgs() and just have PyCode_New(). That as added because of PEP 387 and unfortunately removing it is backwards incompatible. >> I propose we declare all APIs for code objects *unstable*, liable to change >>

[issue40222] "Zero cost" exception handling

Guido van Rossum added the comment: I propose we declare all APIs for code objects *unstable*, liable to change each (feature) release. I want to get rid of PyCode_NewWithPosArgs() and just have PyCode_New(). All callers to either one must be changed anyways. (And we’re not done changing

[issue40222] "Zero cost" exception handling

Pablo Galindo Salgado added the comment: > Is there a separate issue for how we are handling CodeType()? No, that's why this is marked as release blocker, because this is the first issue where CodeType was changed. If you wish to close this one, please open a new issue explaining the

[issue40222] "Zero cost" exception handling

Mark Shannon added the comment: I'd like to close this, as the exception handling is all done and working correctly. Is there a separate issue for how we are handling CodeType()? -- ___ Python tracker

[issue40222] "Zero cost" exception handling

Guido van Rossum added the comment: Ah, okay. So we're not on the hook to decide this today. :-) -- ___ Python tracker ___ ___

[issue40222] "Zero cost" exception handling

Pablo Galindo Salgado added the comment: Small note, as I mentioned in my correction email (https://mail.python.org/archives/list/python-...@python.org/message/GBD57CUUU4K5NMQDTEZXNCX76YISEIGQ/), this is a release blocker for 3.11 (it was not marked in the issue what Python version was

[issue40222] "Zero cost" exception handling

Pablo Galindo Salgado added the comment: > I think we're waiting here for the release manager to decide, right? As Petr mentions, the release manager doesn't have authority to decide if the backwards compatibility policy can be ignored, only the Steering Council. > Should we roll back the

[issue40222] "Zero cost" exception handling

Guido van Rossum added the comment: I think we're waiting here for the release manager to decide, right? Should we roll back the change to PyCode_NewWithPosOnlyArgs() or keep it? Presumably the requested docs aren't the (beta) release blocker? --

[issue40222] "Zero cost" exception handling

STINNER Victor added the comment: Mark: Can you please document your change on types.CodeType? In: * https://docs.python.org/dev/library/types.html#types.CodeType * https://docs.python.org/dev/whatsnew/3.11.html The change broke the Genshi project:

[issue40222] "Zero cost" exception handling

Change by Mark Shannon : -- pull_requests: +24699 pull_request: https://github.com/python/cpython/pull/26059 ___ Python tracker ___

[issue40222] "Zero cost" exception handling

Stefan Behnel added the comment: Just a comment regarding the change to "PyCode_NewWithPosOnlyArgs()". As Pablo mentioned, this has happened before. And that's OK! Exactly because this has happened before, it's clearly not a part of the API that is meant to be stable. I can easily adapt

[issue40222] "Zero cost" exception handling

Pablo Galindo Salgado added the comment: > It is very little effort to add back the old function, so that isn't the > problem. It won't work properly, but it never did anyway. So I guess that's > sort of compatible. It won't work properly is an incompatible change. Before, if you extract

[issue40222] "Zero cost" exception handling

Mark Shannon added the comment: It is very little effort to add back the old function, so that isn't the problem. It won't work properly, but it never did anyway. So I guess that's sort of compatible. Maybe the best thing is to put a big red warning in the docs and hope that warns away

[issue40222] "Zero cost" exception handling

Pablo Galindo Salgado added the comment: > I think API breaks like this do need coordination at the project level. Absolutely, that's why I said before: > As I mentioned, I simphatise with your argument and I think it makes sense, but we cannot just do it in a BPO issue, I'm afraid.

[issue40222] "Zero cost" exception handling

Petr Viktorin added the comment: Then, according to PEP 387, "The steering council may grant exceptions to this policy." I think API breaks like this do need coordination at the project level. -- ___ Python tracker

[issue40222] "Zero cost" exception handling

Pablo Galindo Salgado added the comment: > The PyAPI_FUNC makes it part of the public *API*. It needs to be source- > compatible; the number of arguments can't change. Could yo u add a new > function? Unfortunately, no: new functions cannot be added easily because the new field that

[issue40222] "Zero cost" exception handling

Petr Viktorin added the comment: PyCode_NewWithPosOnlyArgs is not part of the stable ABI. It is OK to break its ABI in a minor version (i.e. 3.11). The PyAPI_FUNC makes it part of the public *API*. It needs to be source- compatible; the number of arguments can't change. Could yo u add a new

[issue40222] "Zero cost" exception handling

Pablo Galindo Salgado added the comment: In any case, if we decide to let it stay, at the very least this behaviour (that these functions are not stable) needs to be documented everywhere: what's new, C-API docs...etc And probably we need to somehow add it to the future deprecations of 3.10

[issue40222] "Zero cost" exception handling

Pablo Galindo Salgado added the comment: I agree with you but we already went through this when I added positional-only arguments and everyone complained that Cython and other projects were broken and we changed a stable API function so I am just mentioning that we are here again.

[issue40222] "Zero cost" exception handling

Mark Shannon added the comment: I know PyCode_NewWithPosOnlyArgs is declared as "PyAPI_FUNC" but that can't make it part of the ABI unless it has stable behavior. It can't have stable behavior because its inputs are complex, undefined, have altered semantics and are interlinked in complex

[issue40222] "Zero cost" exception handling

Change by Pablo Galindo Salgado : -- nosy: +petr.viktorin ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue40222] "Zero cost" exception handling

Pablo Galindo Salgado added the comment: It seems that we have broken the stable ABI of PyCode_NewWithPosOnlyArgs as it has now another parameter. We need to either create a new private constructor or a new public constructor, but the ABI cannot change. --

[issue40222] "Zero cost" exception handling

Change by Mark Shannon : -- pull_requests: +24671 pull_request: https://github.com/python/cpython/pull/26021 ___ Python tracker ___

[issue40222] "Zero cost" exception handling

Mark Shannon added the comment: Thanks everyone for the triaging and fixing. -- ___ Python tracker ___ ___ Python-bugs-list

[issue40222] "Zero cost" exception handling

Change by Dennis Sweeney : -- pull_requests: +24662 pull_request: https://github.com/python/cpython/pull/26012 ___ Python tracker ___

[issue40222] "Zero cost" exception handling

Dennis Sweeney added the comment: I tried some debugging code: diff --git a/Python/ceval.c b/Python/ceval.c index f745067069..a8668dbac2 100644 --- a/Python/ceval.c +++ b/Python/ceval.c @@ -4864,6 +4864,18 @@ get_exception_handler(PyCodeObject *code, int index) return res;

[issue40222] "Zero cost" exception handling

Ammar Askar added the comment: Seconded, also seeing the same ASAN failure on the fuzzers with a blame for this commit. -- nosy: +ammar2 ___ Python tracker ___

[issue40222] "Zero cost" exception handling

Pablo Galindo Salgado added the comment: To reproduce with a modern gcc: % export ASAN_OPTIONS=detect_leaks=0:allocator_may_return_null=1:handle_segv=0 % ./configure --with-address-sanitizer --without-pymalloc % make -j -s % ./python -m test test_statistics

[issue40222] "Zero cost" exception handling

Change by Pablo Galindo Salgado : -- priority: normal -> release blocker ___ Python tracker ___ ___ Python-bugs-list mailing list

[issue40222] "Zero cost" exception handling

Pablo Galindo Salgado added the comment: This seems to have broken the address sanitizer buildbot: https://buildbot.python.org/all/#/builders/582/builds/116/steps/5/logs/stdio Example error: ==28597==ERROR: AddressSanitizer:

[issue40222] "Zero cost" exception handling

Change by Mark Shannon : -- keywords: +patch pull_requests: +24440 stage: -> patch review pull_request: https://github.com/python/cpython/pull/25729 ___ Python tracker ___

[issue40222] "Zero cost" exception handling

Mark Shannon added the comment: I've played around with a few formats, and what I've ended up with is this: 1. Use the >> marker for for exception targets, as well as normal branch targets. 2. Add a text version of the exception handler table at the end of the disassembly. This has all the

[issue40222] "Zero cost" exception handling

Mark Shannon added the comment: Responding to Serhiy's suggestions: 1 Add another column: Adding another column makes for lots of repetition in larger try blocks, and pushes useful information further to the right. 2 Add pseudo-instructions I find those misleading, as they aren't really

[issue40222] "Zero cost" exception handling

Mark Shannon added the comment: > BTW, what are the three POP_TOP op codes in a row popping? When exceptions are pushed to the stack, they are pushed as a triple: (exc, type, traceback) so when we pop them, we need three pops. -- ___ Python

[issue40222] "Zero cost" exception handling

Change by Erlend Egeberg Aasland : -- nosy: +erlendaasland ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue40222] "Zero cost" exception handling

Guido van Rossum added the comment: I like Serhiy’s idea. BTW, what are the three POP_TOP op codes in a row popping? -- nosy: +Guido.van.Rossum ___ Python tracker ___

[issue40222] "Zero cost" exception handling

Serhiy Storchaka added the comment: We can add a new column for the offset or the index of the error handler. Or add pseudo-instructions (which do not correspond to any bytecode) at boundaries of the code with some error handler. -- ___ Python

[issue40222] "Zero cost" exception handling

Mark Shannon added the comment: The changes to pyc format aren't user visible so shouldn't matter, but what about the dis output? Consider this program: def f(): try: 1/0 except: return "fail" Currently it compiles to: 2 0 SETUP_FINALLY7 (to

[issue40222] "Zero cost" exception handling

Change by Guido van Rossum : -- nosy: +gvanrossum ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue40222] "Zero cost" exception handling

Serhiy Storchaka added the comment: The only observable changes will be changes in the code object: new attributes and constructor parameters, changed .pyc format, dis output, etc. -- ___ Python tracker

[issue40222] "Zero cost" exception handling

Chris Jerdonek added the comment: To clarify, would there be any observable difference in behavior aside from speed? And would there be any limitations in when the speedup can be applied? -- nosy: +chris.jerdonek ___ Python tracker

[issue40222] "Zero cost" exception handling

Dong-hee Na added the comment: +1 -- nosy: +corona10 ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue40222] "Zero cost" exception handling

Change by Yonatan Goldschmidt : -- nosy: +Yonatan Goldschmidt ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue40222] "Zero cost" exception handling

Raymond Hettinger added the comment: This is an exciting prospect. Am looking forward to it :-) -- nosy: +rhettinger ___ Python tracker ___

[issue40222] "Zero cost" exception handling

Change by Shantanu : -- nosy: +hauntsaninja ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue40222] "Zero cost" exception handling

Serhiy Storchaka added the comment: +1! I was going to implement this, but first I wanted to implement support of line number ranges instead of just line numbers (co_lineno). We need to design some compact portable format for address to address mapping (or address range to address mapping

[issue40222] "Zero cost" exception handling

New submission from Mark Shannon : C++ and Java support what is known as "zero cost" exception handling. The "zero cost" refers to the cost when no exception is raised. There is still a cost when exceptions are thrown. The basic principle is that the compiler generates tables indicating where