[issue39573] Make PyObject an opaque structure in the limited C API

STINNER Victor added the comment: I created bpo-40170 "[C API] Make PyTypeObject structure an opaque structure in the public C API". -- ___ Python tracker ___

[issue39573] Make PyObject an opaque structure in the limited C API

STINNER Victor added the comment: New changeset 557287075c264d2458cd3e1b45e9b8ee5341e0a1 by Andy Lester in branch 'master': bpo-39573: Use Py_IS_TYPE() macro to check for types (GH-18809) https://github.com/python/cpython/commit/557287075c264d2458cd3e1b45e9b8ee5341e0a1 --

[issue39573] Make PyObject an opaque structure in the limited C API

Change by Andy Lester : -- pull_requests: +18167 pull_request: https://github.com/python/cpython/pull/18809 ___ Python tracker ___

[issue39573] Make PyObject an opaque structure in the limited C API

Change by STINNER Victor : -- pull_requests: +18161 pull_request: https://github.com/python/cpython/pull/18804 ___ Python tracker ___

[issue39573] Make PyObject an opaque structure in the limited C API

STINNER Victor added the comment: New changeset 8767ce92d24d3687405848442e6c67cf0af1c657 by Andy Lester in branch 'master': bpo-39573: Make Py_IS_TYPE() take constant parameters (GH-18799) https://github.com/python/cpython/commit/8767ce92d24d3687405848442e6c67cf0af1c657 --

[issue39573] Make PyObject an opaque structure in the limited C API

Change by Andy Lester : -- pull_requests: +18155 pull_request: https://github.com/python/cpython/pull/18799 ___ Python tracker ___

[issue39573] Make PyObject an opaque structure in the limited C API

Change by Andy Lester : -- pull_requests: +18154 pull_request: https://github.com/python/cpython/pull/18798 ___ Python tracker ___

[issue39573] Make PyObject an opaque structure in the limited C API

Change by Andy Lester : -- pull_requests: +18146 pull_request: https://github.com/python/cpython/pull/18789 ___ Python tracker ___

[issue39573] Make PyObject an opaque structure in the limited C API

STINNER Victor added the comment: New changeset dffe4c07095e0c693e094d3c140e85a68bd8128e by Andy Lester in branch 'master': bpo-39573: Finish converting to new Py_IS_TYPE() macro (GH-18601) https://github.com/python/cpython/commit/dffe4c07095e0c693e094d3c140e85a68bd8128e --

[issue39573] Make PyObject an opaque structure in the limited C API

Andy Lester added the comment: Just added a new PR to finish off the remaining places to use Py_IS_TYPE() https://github.com/python/cpython/pull/18601 -- ___ Python tracker

[issue39573] Make PyObject an opaque structure in the limited C API

Change by Andy Lester : -- pull_requests: +17968 pull_request: https://github.com/python/cpython/pull/18601 ___ Python tracker ___

[issue39573] Make PyObject an opaque structure in the limited C API

Andy Lester added the comment: All I'm saying is that I think Py_IS_TYPE is a great idea, and that Py_IS_TYPE should take const arguments, since its arguments are not modified. If you think that should go in a different ticket, then I can make that happen. --

[issue39573] Make PyObject an opaque structure in the limited C API

STINNER Victor added the comment: > If we can keep functions that don't modify the object to accept const > PyObject* it will help make things safer in the long run. In my experience, trying to add "const" is quite painful, since the "const" has to be propagated to all functions called by

[issue39573] Make PyObject an opaque structure in the limited C API

Andy Lester added the comment: > Would you mind to explain how it's an issue to modify PyObject* temporarily > during a function call? It's not a problem to modify the PyObject* during a function call. However, many functions don't need to modify the object, but are still taking non-const

[issue39573] Make PyObject an opaque structure in the limited C API

STINNER Victor added the comment: New changeset 1b55b65638254aa78b005fbf0b71fb02499f1852 by Dong-hee Na in branch 'master': bpo-39573: Clean up modules and headers to use Py_IS_TYPE() function (GH-18521) https://github.com/python/cpython/commit/1b55b65638254aa78b005fbf0b71fb02499f1852

[issue39573] Make PyObject an opaque structure in the limited C API

STINNER Victor added the comment: > Getting away from Py_TYPE(op) would also mean a move to making the internals > const-correct. Would you mind to explain how it's an issue to modify PyObject* temporarily during a function call? It's common to increase the object reference count to ensure

[issue39573] Make PyObject an opaque structure in the limited C API

Change by Dong-hee Na : -- pull_requests: +17898 pull_request: https://github.com/python/cpython/pull/18521 ___ Python tracker ___

[issue39573] Make PyObject an opaque structure in the limited C API

Andy Lester added the comment: I'm hoping that a goal here is to make static inline int _Py_IS_TYPE(PyObject *ob, PyTypeObject *type) actually be static inline int _Py_IS_TYPE(const PyObject *ob, const PyTypeObject *type) -- ___ Python tracker

[issue39573] Make PyObject an opaque structure in the limited C API

Andy Lester added the comment: @vstinner would it be helpful if I went on a sweep looking for places we can use the new Py_IS_TYPE macro? Getting away from Py_TYPE(op) would also mean a move to making the internals const-correct. -- nosy: +petdance

[issue39573] Make PyObject an opaque structure in the limited C API

STINNER Victor added the comment: New changeset 9aeb0ef9309384099e2f23bcee2240fbc096568e by Dong-hee Na in branch 'master': bpo-39573: Update clinic to use Py_IS_TYPE() function (GH-18507) https://github.com/python/cpython/commit/9aeb0ef9309384099e2f23bcee2240fbc096568e --

[issue39573] Make PyObject an opaque structure in the limited C API

STINNER Victor added the comment: New changeset d212c3c55d414203b0579e000d9f340f8cd11be7 by Dong-hee Na in branch 'master': bpo-39573: PyXXX_Check() macros use Py_IS_TYPE() (GH-18508) https://github.com/python/cpython/commit/d212c3c55d414203b0579e000d9f340f8cd11be7 --

[issue39573] Make PyObject an opaque structure in the limited C API

Change by Dong-hee Na : -- pull_requests: +17884 pull_request: https://github.com/python/cpython/pull/18508 ___ Python tracker ___

[issue39573] Make PyObject an opaque structure in the limited C API

Change by Dong-hee Na : -- pull_requests: +17883 pull_request: https://github.com/python/cpython/pull/18507 ___ Python tracker ___

[issue39573] Make PyObject an opaque structure in the limited C API

hai shi added the comment: > By the way, please find another more inclusive way to say hi, see: https://heyguys.cc/ Oh, copy that. Sorry for my poor english. -- ___ Python tracker

[issue39573] Make PyObject an opaque structure in the limited C API

STINNER Victor added the comment: > Hi, guys. By the way, please find another more inclusive way to say hi, see: https://heyguys.cc/ -- ___ Python tracker ___

[issue39573] Make PyObject an opaque structure in the limited C API

STINNER Victor added the comment: New changeset d905df766c367c350f20c46ccd99d4da19ed57d8 by Dong-hee Na in branch 'master': bpo-39573: Add Py_IS_TYPE() function (GH-18488) https://github.com/python/cpython/commit/d905df766c367c350f20c46ccd99d4da19ed57d8 --

[issue39573] Make PyObject an opaque structure in the limited C API

STINNER Victor added the comment: New changeset 968dcd9e7a4d3aa9aaa1dfca693adf60d6b71ce7 by Brandt Bucher in branch 'master': bpo-39573: Fix bad copy-paste in Py_SET_SIZE (GH-18496) https://github.com/python/cpython/commit/968dcd9e7a4d3aa9aaa1dfca693adf60d6b71ce7 --

[issue39573] Make PyObject an opaque structure in the limited C API

hai shi added the comment: >"obj == Py_None" is a very common pattern. >You have check how it is done in HPy: https://github.com/pyhandle/hpy >See also bpo-39511: "[subinterpreters] Per-interpreter singletons (None, >>True, False, etc.)". Thanks, I will check it. --

[issue39573] Make PyObject an opaque structure in the limited C API

STINNER Victor added the comment: > Hi, guys. Is there value in adding `PyNone_Check` macro? "obj == Py_None" is a very common pattern. You have check how it is done in HPy: https://github.com/pyhandle/hpy See also bpo-39511: "[subinterpreters] Per-interpreter singletons (None, True,

[issue39573] Make PyObject an opaque structure in the limited C API

hai shi added the comment: Hi, guys. Is there value in adding `PyNone_Check` macro?(`_PyNone_Type` is not esposed to CAPI directly, so I am not sure about it) If the answer is 'yes', i can add it ;) -- nosy: +shihai1991 ___ Python tracker

[issue39573] Make PyObject an opaque structure in the limited C API

Change by Brandt Bucher : -- pull_requests: +17870 pull_request: https://github.com/python/cpython/pull/18496 ___ Python tracker ___

[issue39573] Make PyObject an opaque structure in the limited C API

Change by Dong-hee Na : -- pull_requests: +17861 pull_request: https://github.com/python/cpython/pull/18488 ___ Python tracker ___

[issue39573] Make PyObject an opaque structure in the limited C API

Dong-hee Na added the comment: FYI, I am working on to add Py_IS_TYPE macro. :) -- nosy: +corona10 ___ Python tracker ___ ___

[issue39573] Make PyObject an opaque structure in the limited C API

Change by hai shi : -- pull_requests: +17795 pull_request: https://github.com/python/cpython/pull/18419 ___ Python tracker ___ ___

[issue39573] Make PyObject an opaque structure in the limited C API

STINNER Victor added the comment: New changeset 7f6f7eef5206858030cbe4f80a7c04b02781cc9a by Dong-hee Na in branch 'master': bpo-39573: Use Py_TYPE() macro in ctypes.h (GH-18411) https://github.com/python/cpython/commit/7f6f7eef5206858030cbe4f80a7c04b02781cc9a --

[issue39573] Make PyObject an opaque structure in the limited C API

STINNER Victor added the comment: > "static inline" functions are not opaque I'm well aware of that :-) But once the CPython code base will stop accessing directly PyObject fields directly, it would become possible to experiment changing PyObject layout, at least testing it in CPython.

[issue39573] Make PyObject an opaque structure in the limited C API

Steve Dower added the comment: "static inline" functions are not opaque - as they get inlined into 3rd-party compiled code, we can't change anything they reference, and so the structure layout is still fixed and has to be visible to the user's compiler. I'm not totally against the changes,

[issue39573] Make PyObject an opaque structure in the limited C API

Change by Dong-hee Na : -- pull_requests: +17785 pull_request: https://github.com/python/cpython/pull/18411 ___ Python tracker ___

[issue39573] Make PyObject an opaque structure in the limited C API

STINNER Victor added the comment: New changeset 60ac6ed5579f130fc264d3b748ee9575e3aa by Victor Stinner in branch 'master': bpo-39573: Use Py_SET_SIZE() function (GH-18402) https://github.com/python/cpython/commit/60ac6ed5579f130fc264d3b748ee9575e3aa --

[issue39573] Make PyObject an opaque structure in the limited C API

STINNER Victor added the comment: > You have merged so much PRs today. What they do? I merged changes which prepares CPython code base to make PyObject opaque. I only merged changes which should have no impact on performance, but prepare the API to make the structure opaque. Right now,

[issue39573] Make PyObject an opaque structure in the limited C API

Serhiy Storchaka added the comment: You have merged so much PRs today. What they do? PyObject cannot just be made an opaque structure. The user code reads and writes its fields directly and via macros. This change would break working code. We can encourage the user code to prepare to making

[issue39573] Make PyObject an opaque structure in the limited C API

Change by STINNER Victor : -- pull_requests: +17778 pull_request: https://github.com/python/cpython/pull/18402 ___ Python tracker ___

[issue39573] Make PyObject an opaque structure in the limited C API

STINNER Victor added the comment: New changeset b10dc3e7a11fcdb97e285882eba6da92594f90f9 by Victor Stinner in branch 'master': bpo-39573: Add Py_SET_SIZE() function (GH-18400) https://github.com/python/cpython/commit/b10dc3e7a11fcdb97e285882eba6da92594f90f9 --

[issue39573] Make PyObject an opaque structure in the limited C API

Change by STINNER Victor : -- pull_requests: +17776 pull_request: https://github.com/python/cpython/pull/18400 ___ Python tracker ___

[issue39573] Make PyObject an opaque structure in the limited C API

STINNER Victor added the comment: New changeset c65b320a95784d2b2133926921d67ac439259e9f by Victor Stinner in branch 'master': bpo-39573: Use Py_TYPE() macro in object.c (GH-18398) https://github.com/python/cpython/commit/c65b320a95784d2b2133926921d67ac439259e9f --

[issue39573] Make PyObject an opaque structure in the limited C API

STINNER Victor added the comment: Would a Py_TYPE_IS() macro help code readability? For example: #define Future_CheckExact(obj) (Py_TYPE(obj) == ) would become: #define Future_CheckExact(obj) (Py_TYPE_IS(obj, )) Py_TYPE_IS() would be more efficient for tagged pointers. I'm not sure

[issue39573] Make PyObject an opaque structure in the limited C API

Change by STINNER Victor : -- pull_requests: +17774 pull_request: https://github.com/python/cpython/pull/18398 ___ Python tracker ___

[issue39573] Make PyObject an opaque structure in the limited C API

STINNER Victor added the comment: To make PyObject opaque, we would have to convert Py_INCREF() and Py_DECREF() to opaque function calls. Example: #define Py_XINCREF(op) Py_IncRef(op) #define Py_XDECREF(op) Py_DecRef(op) Benchmarks should be run to measure to overhead and balance the

[issue39573] Make PyObject an opaque structure in the limited C API

STINNER Victor added the comment: New changeset d2ec81a8c99796b51fb8c49b77a7fe369863226f by Victor Stinner in branch 'master': bpo-39573: Add Py_SET_TYPE() function (GH-18394) https://github.com/python/cpython/commit/d2ec81a8c99796b51fb8c49b77a7fe369863226f --

[issue39573] Make PyObject an opaque structure in the limited C API

Change by STINNER Victor : -- pull_requests: +17770 pull_request: https://github.com/python/cpython/pull/18394 ___ Python tracker ___

[issue39573] Make PyObject an opaque structure in the limited C API

STINNER Victor added the comment: New changeset daa9756cb6395323d6f291efe5c7d7fdc6b2e9d8 by Victor Stinner in branch 'master': bpo-39573: Use Py_TYPE() macro in Modules directory (GH-18393) https://github.com/python/cpython/commit/daa9756cb6395323d6f291efe5c7d7fdc6b2e9d8 --

[issue39573] Make PyObject an opaque structure in the limited C API

Change by STINNER Victor : -- pull_requests: +17769 pull_request: https://github.com/python/cpython/pull/18393 ___ Python tracker ___

[issue39573] Make PyObject an opaque structure in the limited C API

STINNER Victor added the comment: New changeset 58ac700fb09497df14d4492b6f820109490b2b88 by Victor Stinner in branch 'master': bpo-39573: Use Py_TYPE() macro in Objects directory (GH-18392) https://github.com/python/cpython/commit/58ac700fb09497df14d4492b6f820109490b2b88 --

[issue39573] Make PyObject an opaque structure in the limited C API

STINNER Victor added the comment: Make PyObject an opaque structure is also a first step towards the more ambitious project "HPy" project which is fully opaque: https://github.com/pyhandle/hpy This API is written from scratch and currently implemented on top on the existing C API. The

[issue39573] Make PyObject an opaque structure in the limited C API

Change by STINNER Victor : -- pull_requests: +17767 pull_request: https://github.com/python/cpython/pull/18392 ___ Python tracker ___

[issue39573] Make PyObject an opaque structure in the limited C API

STINNER Victor added the comment: New changeset a102ed7d2f0e7e05438f14d5fb72ca0358602249 by Victor Stinner in branch 'master': bpo-39573: Use Py_TYPE() macro in Python and Include directories (GH-18391) https://github.com/python/cpython/commit/a102ed7d2f0e7e05438f14d5fb72ca0358602249

[issue39573] Make PyObject an opaque structure in the limited C API

STINNER Victor added the comment: Py_TYPE() is commonly used to render the type name in an error message. Example: PyErr_Format(PyExc_TypeError, "cannot convert '%.200s' object to bytearray", Py_TYPE(arg)->tp_name); This code has multiple issues: * It truncates

[issue39573] Make PyObject an opaque structure in the limited C API

Change by STINNER Victor : -- pull_requests: +17765 pull_request: https://github.com/python/cpython/pull/18391 ___ Python tracker ___

[issue39573] Make PyObject an opaque structure in the limited C API

STINNER Victor added the comment: New changeset 0d76d2bd28ac815dabae8b07240ed002ac8fce2d by Victor Stinner in branch 'master': bpo-39573: Use Py_TYPE() in abstract.c (GH-18390) https://github.com/python/cpython/commit/0d76d2bd28ac815dabae8b07240ed002ac8fce2d --

[issue39573] Make PyObject an opaque structure in the limited C API

Change by STINNER Victor : -- pull_requests: +17764 pull_request: https://github.com/python/cpython/pull/18390 ___ Python tracker ___

[issue39573] Make PyObject an opaque structure in the limited C API

Change by STINNER Victor : -- nosy: +nascheme ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue39573] Make PyObject an opaque structure in the limited C API

STINNER Victor added the comment: New changeset c86a11221df7e37da389f9c6ce6e47ea22dc44ff by Victor Stinner in branch 'master': bpo-39573: Add Py_SET_REFCNT() function (GH-18389) https://github.com/python/cpython/commit/c86a11221df7e37da389f9c6ce6e47ea22dc44ff --

[issue39573] Make PyObject an opaque structure in the limited C API

STINNER Victor added the comment: TODO: Add Py_IS_TYPE() macro: #define Py_IS_TYPE(ob, tp) (Py_TYPE(ob) == (tp)) For example, replace: #define PyBool_Check(x) (Py_TYPE(x) == _Type) with: #define PyBool_Check(x) Py_IS_TYPE(x, _Type) IMHO it makes the code more readable.

[issue39573] Make PyObject an opaque structure in the limited C API

STINNER Victor added the comment: In the limited C API, Py_REFCNT() should be converted to: static inline Py_ssize_t _Py_REFCNT(const PyObject *ob) { return ob->ob_refcnt; } #define Py_REFCNT(ob) _Py_REFCNT(_PyObject_CAST(ob)) It would enforce the usage of newly added Py_SET_REFCNT() (PR

[issue39573] Make PyObject an opaque structure in the limited C API

STINNER Victor added the comment: > it can prepare CPython to experiment tagged pointers In September 2018, Neil Schemenauer did an experiment: * https://mail.python.org/archives/list/capi-...@python.org/thread/EGAY55ZWMF2WSEMP7VAZSFZCZ4VARU7L/ *

[issue39573] Make PyObject an opaque structure in the limited C API

Change by STINNER Victor : -- pull_requests: +17763 pull_request: https://github.com/python/cpython/pull/18389 ___ Python tracker ___

[issue39573] Make PyObject an opaque structure in the limited C API

STINNER Victor added the comment: New changeset a93c51e3a8e15f1a486d11d5b55a64f3381babe0 by Victor Stinner in branch 'master': bpo-39573: Use Py_REFCNT() macro (GH-18388) https://github.com/python/cpython/commit/a93c51e3a8e15f1a486d11d5b55a64f3381babe0 --

[issue39573] Make PyObject an opaque structure in the limited C API

STINNER Victor added the comment: > Another idea would be to convert some C extensions of the standard library to > the limited C API. It would ensure that the limited C API contains enough > functions to be useful, but would also notify us directly if the API is > broken. First issues

[issue39573] Make PyObject an opaque structure in the limited C API

Change by STINNER Victor : -- keywords: +patch pull_requests: +17762 stage: -> patch review pull_request: https://github.com/python/cpython/pull/18388 ___ Python tracker ___

[issue39573] Make PyObject an opaque structure in the limited C API

New submission from STINNER Victor : Today, CPython is leaking too many implementation through its public C API. We cannot easily change the "default" C API, but we can enhance the "limited" C API (when Py_LIMITED_API macro is defined). Example of leaking implementation details: memory