[issue46090] C extensions can't swap out live frames anymore

Brandt Bucher added the comment: New changeset 77195cd44b2506cda88a3cfc98918526068b1d46 by Brandt Bucher in branch 'main': bpo-46090: Allow PyThreadState.datastack_* members to be NULL (GH-30234) https://github.com/python/cpython/commit/77195cd44b2506cda88a3cfc98918526068b1d46 --

[issue46090] C extensions can't swap out live frames anymore

Change by Brandt Bucher : -- resolution: -> fixed stage: patch review -> resolved status: open -> closed ___ Python tracker ___

[issue46090] C extensions can't swap out live frames anymore

Change by Barry A. Warsaw : -- nosy: +barry ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue46090] C extensions can't swap out live frames anymore

Change by Brandt Bucher : -- keywords: +patch pull_requests: +28455 stage: -> patch review pull_request: https://github.com/python/cpython/pull/30234 ___ Python tracker ___

[issue46090] C extensions can't swap out live frames anymore

Brandt Bucher added the comment: After further discussion with the team this morning, we decided that the proposed "dummy chunk" workaround is unacceptable (the interpreter may end up trying free it when shutting down a thread). Allowing the tstate->datastack_* members to be NULL is very

[issue46090] C extensions can't swap out live frames anymore

Brandt Bucher added the comment: Fixing this actually ended up being a lot simpler than I anticipated (and it requires no changes on our end). All that's really required is saving and restoring: - tstate->cframe->current_frame - tstate->datastack_chunk - tstate->datastack_top -

[issue46090] C extensions can't swap out live frames anymore

Pablo Galindo Salgado added the comment: Relevant section: https://docs.python.org/3/c-api/init.html#non-python-created-threads -- ___ Python tracker ___

[issue46090] C extensions can't swap out live frames anymore

Pablo Galindo Salgado added the comment: That is weird, but can happen for example if some pthread of some c extensión for example picks up the Gil, causing a new PyThreadState to be created and linked but has not called into Python yet. Basically, an external native thread that calls

[issue46090] C extensions can't swap out live frames anymore

Guido van Rossum added the comment: A thread without a frame looks like a natural end case when you consider the frames as a linked list. Could it be that exec()/eval() run in the current frame, and if there isn't a frame, they refuse to run? (Sorry, I don't recall anything more specific

[issue46090] C extensions can't swap out live frames anymore

Brandt Bucher added the comment: Also, it appears that in earlier versions of CPython, it was okay to set tstate->frame = NULL. Some of Greenlet's tests *seem* to be doing this (although it also possible that my current scaffolding has just broken Greenlet). Does anybody know off the top

[issue46090] C extensions can't swap out live frames anymore

Brandt Bucher added the comment: I'm going to take a stab at making this work today. The idea is to do something like this when setting a new frame: - Clear and pop tstate->cframe->current_frame and all of its linked predecessors. - These should all belong to the same CFrame, right? -

[issue46090] C extensions can't swap out live frames anymore

Change by Jason Madden : -- nosy: +jmadden ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue46090] C extensions can't swap out live frames anymore

New submission from Brandt Bucher : I'm specifically concerned about Greenlet here (since it's a dependency of pyperformance), but this discussion is equally relevant to any library like it that dynamically swaps out the currently executing frame. CPython 3.11 makes several major changes to