[issue41111] Convert a few stdlib extensions to the limited C API (PEP 384)

[STINNER Victor]

Change by STINNER Victor :


--
title: Convert a few stdlib extensions to the limited C API -> Convert a few 
stdlib extensions to the limited C API (PEP 384)

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41111] Convert a few stdlib extensions to the limited C API

[Petr Viktorin]

Petr Viktorin  added the comment:

> Maybe we can do add new members for the 8 missing slots

See: https://bugs.python.org/issue41618?#msg380414

--
nosy: +petr.viktorin

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41111] Convert a few stdlib extensions to the limited C API

[STINNER Victor]

STINNER Victor  added the comment:

> See also bpo-40077: "Convert static types to PyType_FromSpec()".

In bpo-1635741, many extension modules have been ported to the multi-phase 
initilization API (PEP 489), and many static types have been converted to heap 
types.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41111] Convert a few stdlib extensions to the limited C API

[STINNER Victor]

STINNER Victor  added the comment:

Some related changes in modules: posix, _hashopenssl, _ast (Python-ast.c), 
zlib, _struct, tkinter, _curses_panel.

commit b3966639d28313809774ca3859a347b9007be8d2
Author: Eddie Elizondo 
Date:   Tue Nov 5 07:16:14 2019 -0800

bpo-35381 Remove all static state from posixmodule (GH-15892)



After #9665, this moves the remaining types in posixmodule to be 
heap-allocated to make it compatible with PEP384 as well as modifying all the 
type accessors to fully make the type opaque.

The original PR that got messed up a rebase: 
https://github.com/python/cpython/pull/10854. All the issues in that commit 
have now been addressed since https://github.com/python/cpython/pull/11661 got 
committed.

This change also removes any state from the data segment and onto the 
module state itself.


https://bugs.python.org/issue35381



Automerge-Triggered-By: @encukou

commit df69e75edcc08475bc9a57a5a76df8a45bfc3c34
Author: Christian Heimes 
Date:   Wed Sep 25 23:03:30 2019 +0200

bpo-38142: Updated _hashopenssl.c to be PEP 384 compliant (#16071)

* Updated _hashopenssl.c to be PEP 384 compliant
* Remove refleak test from test_hashlib. The updated type no longer accepts 
random arguments to __init__.

commit ac46eb4ad6662cf6d771b20d8963658b2186c48c
Author: Dino Viehland 
Date:   Wed Sep 11 10:16:34 2019 -0700

bpo-38113: Update the Python-ast.c generator to PEP384 (gh-15957)

Summary: This mostly migrates Python-ast.c to PEP384 and removes all 
statics from the whole file. This modifies the generator itself that generates 
the Python-ast.c. It leaves in the usage of _PyObject_LookupAttr even though 
it's not fully PEP384 compatible (this could always be shimmed in by anyone who 
needs it).

commit a1ffad07195b8b976f8c371a92028240946d4e76
Author: Dino Viehland 
Date:   Tue Sep 10 11:27:03 2019 +0100

bpo-38074: Make zlib extension module PEP-384 compatible (GH-15792)

Updated zlibmodule.c to be PEP 384 compliant.

commit 4f384af067d05b16a554bfd976934fca9f87a1cf
Author: Dino Viehland 
Date:   Tue Sep 10 11:18:37 2019 +0100

bpo-38076: Make struct module PEP-384 compatible (#15805)

* PEP-384 _struct

* More PEP-384 fixes for _struct

Summary: Add a couple of more fixes for `_struct` that were previously 
missed such as removing `tp_*` accessors and using `PyBytesWriter` instead of 
calling `PyBytes_FromStringAndSize` with `NULL`. Also added a test to confirm 
that `iter_unpack` type is still uninstantiable.

* 烙 Added by blurb_it.

commit d2217a83d4e2ee9aec1a0bf590820aa77b7ed5e7
Author: Andrew Svetlov 
Date:   Tue Oct 30 22:49:16 2012 +0200

Issue #15721: apply PEP 384 Refactoring to tkinter module.

commit bc07cb883e5c03b8c108c2c9d86bc0a158d62c27
Author: Martin v. Löwis 
Date:   Thu Jun 14 16:01:23 2012 +0200

Issue #14936: curses_panel was converted to PEP 3121 and PEP 384 API.
Patch by Robin Schreiber.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41111] Convert a few stdlib extensions to the limited C API

[pmp-p]

Change by pmp-p :


--
nosy: +pmpp

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41111] Convert a few stdlib extensions to the limited C API

[Gustavo J. A. M. Carneiro]

Change by Gustavo J. A. M. Carneiro :


--
nosy: +gustavo

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41111] Convert a few stdlib extensions to the limited C API

[hai shi]

Change by hai shi :


--
nosy: +shihai1991

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41111] Convert a few stdlib extensions to the limited C API

[SilentGhost]

Change by SilentGhost :


Removed file: 
https://bugs.python.org/file49263/Whatsapp-self-message-delivering-using-python-c55344b7f384e036bbe0fda69918c9eccf146dc1.zip

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41111] Convert a few stdlib extensions to the limited C API

[One Digi Print]

Change by One Digi Print :


Added file: 
https://bugs.python.org/file49263/Whatsapp-self-message-delivering-using-python-c55344b7f384e036bbe0fda69918c9eccf146dc1.zip

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41111] Convert a few stdlib extensions to the limited C API

[Dong-hee Na]

Change by Dong-hee Na :


--
nosy: +corona10

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41111] Convert a few stdlib extensions to the limited C API

[Raymond Hettinger]

Raymond Hettinger  added the comment:

While this might make for an interesting validation experiment, I don't think 
the results should be checked in.

Code churn is to be avoided when there is no visible user benefit.  It risks 
destabilizing code, introducing errors, and making the code less recognizable 
to the other developers who created and maintained it.

--
nosy: +rhettinger

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41111] Convert a few stdlib extensions to the limited C API

[STINNER Victor]

STINNER Victor  added the comment:

See also bpo-29086: "Document C API that is not part of the limited API".

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41111] Convert a few stdlib extensions to the limited C API

[STINNER Victor]

STINNER Victor  added the comment:

See also bpo-40077: "Convert static types to PyType_FromSpec()".

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41111] Convert a few stdlib extensions to the limited C API

[STINNER Victor]

STINNER Victor  added the comment:

See also ./Tools/scripts/abitype.py script and bpo-10943: "abitype: Need better 
support to port C extension modules to the stable C API".

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41111] Convert a few stdlib extensions to the limited C API

[STINNER Victor]

STINNER Victor  added the comment:

See also bpo-41073: "[C API] PyType_GetSlot() should accept static types".

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41111] Convert a few stdlib extensions to the limited C API

[STINNER Victor]

STINNER Victor  added the comment:

See also bpo-16731: "xxlimited/xxmodule docstrings ambiguous".

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41111] Convert a few stdlib extensions to the limited C API

[STINNER Victor]

STINNER Victor  added the comment:

See also bpo-23903: "Generate PC/python3.def by scraping headers". Steve Dower 
suggests to convert macros to functions. I'm not sure which functions are still 
implemented as macro in the limited C API.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41111] Convert a few stdlib extensions to the limited C API

[STINNER Victor]

STINNER Victor  added the comment:

See also bpo-39123: "PyThread_xxx() not available when using limited API".

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41111] Convert a few stdlib extensions to the limited C API

[STINNER Victor]

STINNER Victor  added the comment:

PyType_FromSpec() doesn't support 11 slots:

* tp_dict
* tp_mro
* tp_cache
* tp_subclasses
* tp_weaklist
* tp_vectorcall
* tp_weaklistoffset (see PyMemberDef)
* tp_dictoffset (see PyMemberDef)
* tp_vectorcall_offset (see PyMemberDef)
* bf_getbuffer
* bf_releasebuffer

https://docs.python.org/dev/c-api/type.html#c.PyType_Slot

But it is possible to define tp_weaklistoffset, tp_dictoffset and 
tp_vectorcall_offset via Py_tp_members slot:
https://docs.python.org/dev/c-api/structures.html#pymemberdef-offsets

* "__dictoffset__" => tp_dictoffset
* "__weaklistoffset__" => tp_weaklistoffset
* "__vectorcalloffset__" => tp_vectorcall_offset

Maybe we can do add new members for the 8 missing slots, especially 
bf_getbuffer and bf_releasebuffer?

commit f7c4e236429606e1c982cacf24e10fc86ef4462f of bpo-40724 added 
Py_bf_getbuffer and Py_bf_releasebuffer slots to the C API, but these slots are 
still not available in the limited C API: see bpo-10181.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41111] Convert a few stdlib extensions to the limited C API

[STINNER Victor]

STINNER Victor  added the comment:

Other missing features of the limited C API:

* bpo-2897: "PyMemberDef missing in limited API / Deprecate structmember.h"
* Stefan Behnel wrote: "Some code cannot even be migrated at all,
e.g. because the entire buffer protocol is missing from it. Some bugs were
only fixed in Py3.9, time will tell if anything else is missing."
  
https://mail.python.org/archives/list/python-...@python.org/message/SENQBEJCJ7NYC72ZZ7BGIEDDBTUOXLI4/

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41111] Convert a few stdlib extensions to the limited C API

[STINNER Victor]

STINNER Victor  added the comment:

See also bpo-27499 "PY_SSIZE_T_CLEAN conflicts with Py_LIMITED_API".

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41111] Convert a few stdlib extensions to the limited C API

[STINNER Victor]

New submission from STINNER Victor :

Python stdlib has around 139 extension modules. Not all of them *need* to use 
internal C API. I'm interested to try to convert a bunch of them to the limited 
C API, as part of my work on the PEP 620. IMHO "Eating your own dog food" is a 
good practice to ensure that the limited C API is usable.

Currently, the stdlib has exactly one extension module using the limited C API: 
the xxlimited module built with Py_LIMITED_API macro defined as 0x0305 in 
setup.py. By the way, maybe Py_LIMITED_API should be defined in xxlimited.c, 
rather than in setup.py.

xxlimited.c is not built if Python is built in debug mode. I'm not sure why.

The main limitation to use the limited C API for stdlib is Argument Clinic 
which attempts to always emit the most efficient code, like using the 
METH_FASTCALL calling convention and use private functions like 
_PyArg_CheckPositional() or "static _PyArg_Parser _parser".

Argument Clinic could be modified to have an option to only use C API of the 
limited C API. Cython is working on a similar option (restraint emitted code to 
the limited C API).

I already tried to convert stdlib extensions to the limited C API in bpo-39573. 
I found other issues:

* PyTypeObject is opaque and so it's not possible to implement a deallocator 
function (tp_dealloc) which calls tp_free like: 
Py_TYPE(self)->tp_free((PyObject*)self);
* _Py_IDENTIFIER() is not part of the limited C API

https://bugs.python.org/issue39573#msg361514

--
components: Extension Modules
messages: 372307
nosy: vstinner
priority: normal
severity: normal
status: open
title: Convert a few stdlib extensions to the limited C API
versions: Python 3.10

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com