[issue35886] Move PyInterpreterState into Include/internal/pycore_pystate.h

STINNER Victor added the comment: > FWIW, I was hoping to the same for PyThreadState but it looks like 6 of its > fields are exposed in "stable" header files via the following macros: (...) I created bpo-39947: "[C API] Make the PyThreadState structure opaque (move it to the internal C

[issue35886] Move PyInterpreterState into Include/internal/pycore_pystate.h

Eric Snow added the comment: +1 to the suggested "whatsnew" updates. If no one beats me to it I'll work on a PR soon. -- ___ Python tracker ___

[issue35886] Move PyInterpreterState into Include/internal/pycore_pystate.h

Change by Vinay Sajip : -- nosy: +vinay.sajip ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue35886] Move PyInterpreterState into Include/internal/pycore_pystate.h

STINNER Victor added the comment: Note: IMHO the "The PyInterpreterState struct has been moved into the “internal” header files (...)" entry should be documented in the "Changes in the C API" section, rather than in the "Changes in the Python API" section of:

[issue35886] Move PyInterpreterState into Include/internal/pycore_pystate.h

STINNER Victor added the comment: This change broke at least two projects: * Blender: access "interp->modules" but also "interp->builtins" * FreeCAD: access "interp->modules" It would be nice to document replacement solutions at:

[issue35886] Move PyInterpreterState into Include/internal/pycore_pystate.h

Armin Rigo added the comment: Done. -- ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue35886] Move PyInterpreterState into Include/internal/pycore_pystate.h

Stéphane Wirtel added the comment: @arigo Yep, I am interested because I would like to execute the tests of the major projects/libraries (django, flasks, pandas, requests, ...) and create issues for the maintainer. the sooner we get feedback, the sooner we can fix the bugs. --

[issue35886] Move PyInterpreterState into Include/internal/pycore_pystate.h

Armin Rigo added the comment: Cool. Also, no bugfix release of cffi was planned, but I can make one if you think it might help for testing the current pre-release of CPython 3.8. -- ___ Python tracker

[issue35886] Move PyInterpreterState into Include/internal/pycore_pystate.h

Nick Coghlan added the comment: Oh, cool (both the fact the issue here is only with building cffi itself, and that cffi creates extension modules that build with PY_LIMITED_API). I've filed https://bugs.python.org/issue36124 to follow up on the PyInterpreter_GetDict API idea. --

[issue35886] Move PyInterpreterState into Include/internal/pycore_pystate.h

Armin Rigo added the comment: @nick the C sources produced by cffi don't change. When they are compiled, they use Py_LIMITED_API so you can continue using a single compiled module version for any recent-enough Python 3.x. The required fix is only inside the cffi module itself. --

[issue35886] Move PyInterpreterState into Include/internal/pycore_pystate.h

Stéphane Wirtel added the comment: @nick which indirectly broke httpbin and this one is used by python-requests and we can't execute the tests of requests. -- nosy: +matrixise ___ Python tracker

[issue35886] Move PyInterpreterState into Include/internal/pycore_pystate.h

Nick Coghlan added the comment: (On closer inspection, that's actually be the same breakage as already mentioned above) However, what I'm not clear on is how this would affect projects that had *already* generated their cffi code, and include that in their sdist. Are all those sdists going

[issue35886] Move PyInterpreterState into Include/internal/pycore_pystate.h

Nick Coghlan added the comment: Next incompatibility: https://github.com/python-hyper/brotlipy/issues/147 (which indirectly broke httpbin) -- ___ Python tracker ___

[issue35886] Move PyInterpreterState into Include/internal/pycore_pystate.h

Eric Snow added the comment: On Mon, Feb 25, 2019 at 3:02 AM STINNER Victor wrote: > But that's where I suggested to test "popular C extensions" with a C API > change before merging it. It's ok if issues are discovered later, but the > author of backward incompatible C API changes should

[issue35886] Move PyInterpreterState into Include/internal/pycore_pystate.h

Eric Snow added the comment: @Armin, thanks for fixing things on your end so quickly. :) Adding PyInterpreterState.dict and a public getter function (a la PyThreadState) is probably fine. I'm just not sure how that relates to the problem with cffi's use of the "builtins" field. If it

[issue35886] Move PyInterpreterState into Include/internal/pycore_pystate.h

Nathaniel Smith added the comment: The cffi issue is: https://bitbucket.org/cffi/cffi/issues/403/build-fails-on-38-dev-pyinterpreterstate Armin already went ahead and committed the change to make cffi include internal/pycore_pystate.h. I guess this might not be the ideal final outcome

[issue35886] Move PyInterpreterState into Include/internal/pycore_pystate.h

STINNER Victor added the comment: > This broke cffi: Well, that's not a surprise :-) It's a deliberate backward incompatible change. But that's where I suggested to test "popular C extensions" with a C API change before merging it. It's ok if issues are discovered later, but the author of

[issue35886] Move PyInterpreterState into Include/internal/pycore_pystate.h

Armin Rigo added the comment: Just so you know, when we look at the changes to CPython, the easiest fix is to add these lines in cffi: #if PY_VERSION_HEX >= 0x0308 # define Py_BUILD_CORE # include "internal/pycore_pystate.h" # undef Py_BUILD_CORE #endif But if we're

[issue35886] Move PyInterpreterState into Include/internal/pycore_pystate.h

Nathaniel Smith added the comment: This broke cffi: gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -g -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security -fPIC -DUSE__THREAD -DHAVE_SYNC_SYNCHRONIZE

[issue35886] Move PyInterpreterState into Include/internal/pycore_pystate.h

Change by Eric Snow : -- resolution: -> fixed stage: patch review -> resolved status: open -> closed ___ Python tracker ___ ___

[issue35886] Move PyInterpreterState into Include/internal/pycore_pystate.h

Eric Snow added the comment: New changeset be3b295838547bba267eb08434b418ef0df87ee0 by Eric Snow in branch 'master': bpo-35886: Make PyInterpreterState an opaque type in the public API. (GH-11731) https://github.com/python/cpython/commit/be3b295838547bba267eb08434b418ef0df87ee0 --

[issue35886] Move PyInterpreterState into Include/internal/pycore_pystate.h

Eric Snow added the comment: Thanks, Victor! python-dev: https://mail.python.org/pipermail/python-dev/2019-February/156344.html Also, my PR already has a What's New entry in the porting section. :) -- ___ Python tracker

[issue35886] Move PyInterpreterState into Include/internal/pycore_pystate.h

STINNER Victor added the comment: > @Victor, do you see any problems with doing this? It will help simplify > other changes I'm working on. I'm quite sure that they are users of the PyInterpreterState structure outside CPython internals and stdlib, but I expect that the number is quite

[issue35886] Move PyInterpreterState into Include/internal/pycore_pystate.h

Change by Eric Snow : -- nosy: +ncoghlan ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue35886] Move PyInterpreterState into Include/internal/pycore_pystate.h

Eric Snow added the comment: @Victor, do you see any problems with doing this? It will help simplify other changes I'm working on. -- ___ Python tracker ___

[issue35886] Move PyInterpreterState into Include/internal/pycore_pystate.h

Change by Eric Snow : -- pull_requests: -11624 ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue35886] Move PyInterpreterState into Include/internal/pycore_pystate.h

Change by Eric Snow : -- pull_requests: -11625 ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue35886] Move PyInterpreterState into Include/internal/pycore_pystate.h

Eric Snow added the comment: FWIW, I was hoping to the same for PyThreadState but it looks like 6 of its fields are exposed in "stable" header files via the following macros: # Include/object.h Py_TRASHCAN_SAFE_BEGIN Py_TRASHCAN_SAFE_END Include.ceval.h Py_EnterRecursiveCall

[issue35886] Move PyInterpreterState into Include/internal/pycore_pystate.h

Change by Eric Snow : -- keywords: +patch, patch, patch pull_requests: +11623, 11624, 11625 stage: needs patch -> patch review ___ Python tracker ___

[issue35886] Move PyInterpreterState into Include/internal/pycore_pystate.h

Change by Eric Snow : -- keywords: +patch, patch pull_requests: +11623, 11624 stage: needs patch -> patch review ___ Python tracker ___

[issue35886] Move PyInterpreterState into Include/internal/pycore_pystate.h

Change by Eric Snow : -- keywords: +patch pull_requests: +11623 stage: needs patch -> patch review ___ Python tracker ___ ___

[issue35886] Move PyInterpreterState into Include/internal/pycore_pystate.h

New submission from Eric Snow : In November Victor created the Include/cpython directory and moved a decent amount of public (but not limited) API there. This included the PyInterpreterState struct. I'd like to move it into the "internal" headers since it is somewhat coupled to the