[issue40421] [C API] Add public getter functions for the internal PyFrameObject structure
Change by STINNER Victor : -- pull_requests: +30442 pull_request: https://github.com/python/cpython/pull/32417 ___ Python tracker <https://bugs.python.org/issue40421> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue47250] Add object.__getstate__() introduced refleaks
Change by STINNER Victor : -- title: Add object.__getstate__() introduced a refleak -> Add object.__getstate__() introduced refleaks ___ Python tracker <https://bugs.python.org/issue47250> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26579] Support pickling slots in subclasses of common classes
STINNER Victor added the comment: > bpo-26579: Add object.__getstate__(). (GH-2821) > https://github.com/python/cpython/commit/884eba3c76916889fd6bff3b37b8552bfb4f9566 This change introduced reference leaks: see bpo-47250. -- ___ Python tracker <https://bugs.python.org/issue26579> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue47250] Add object.__getstate__() introduced a refleak
New submission from STINNER Victor : The following change introduced reference leaks: --- commit 884eba3c76916889fd6bff3b37b8552bfb4f9566 Author: Serhiy Storchaka Date: Wed Apr 6 20:00:14 2022 +0300 bpo-26579: Add object.__getstate__(). (GH-2821) Copying and pickling instances of subclasses of builtin types bytearray, set, frozenset, collections.OrderedDict, collections.deque, weakref.WeakSet, and datetime.tzinfo now copies and pickles instance attributes implemented as slots. --- Example of buildbot failure. AMD64 RHEL8 Refleaks 3.x: https://buildbot.python.org/all/#builders/259/builds/355 14 tests failed: test_bytes test_copy test_datetime test_deque test_descr test_distutils test_minidom test_ordered_dict test_pickle test_set test_shutil test_tarfile test_typing test_weakset Example of leak: $ ./python -m test -R 3:3 test_descr -m test_issue24097 (...) beginning 6 repetitions 123456 .. test_descr leaked [1, 1, 1] references, sum=3 test_descr failed (reference leak) (...) -- components: Interpreter Core messages: 416930 nosy: corona10, erlendaasland, pablogsal, serhiy.storchaka, vstinner priority: normal severity: normal status: open title: Add object.__getstate__() introduced a refleak versions: Python 3.11 ___ Python tracker <https://bugs.python.org/issue47250> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue35134] Add a new Include/cpython/ subdirectory for the "CPython API" with implementation details
STINNER Victor added the comment: New changeset 85addfb9c6496eb3d26082348cf5aca848c877ef by Victor Stinner in branch 'main': bpo-35134: Remove the Include/code.h header file (GH-32385) https://github.com/python/cpython/commit/85addfb9c6496eb3d26082348cf5aca848c877ef -- ___ Python tracker <https://bugs.python.org/issue35134> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue35134] Add a new Include/cpython/ subdirectory for the "CPython API" with implementation details
STINNER Victor added the comment: New changeset 5c4d1f6e0e192653560ae2941a6677fbf4fbd1f2 by Victor Stinner in branch 'main': bpo-35134: Add Include/cpython/setobject.h header (GH-32384) https://github.com/python/cpython/commit/5c4d1f6e0e192653560ae2941a6677fbf4fbd1f2 -- ___ Python tracker <https://bugs.python.org/issue35134> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue35134] Add a new Include/cpython/ subdirectory for the "CPython API" with implementation details
STINNER Victor added the comment: New changeset ca219f6dfc57f8f4984f96df0f733b7de92fe91c by Victor Stinner in branch 'main': bpo-35134: Add Include/cpython/complexobject.h header (GH-32383) https://github.com/python/cpython/commit/ca219f6dfc57f8f4984f96df0f733b7de92fe91c -- ___ Python tracker <https://bugs.python.org/issue35134> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue35134] Add a new Include/cpython/ subdirectory for the "CPython API" with implementation details
Change by STINNER Victor : -- pull_requests: +30423 pull_request: https://github.com/python/cpython/pull/32385 ___ Python tracker <https://bugs.python.org/issue35134> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue35134] Add a new Include/cpython/ subdirectory for the "CPython API" with implementation details
Change by STINNER Victor : -- pull_requests: +30422 pull_request: https://github.com/python/cpython/pull/32384 ___ Python tracker <https://bugs.python.org/issue35134> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue35134] Add a new Include/cpython/ subdirectory for the "CPython API" with implementation details
Change by STINNER Victor : -- pull_requests: +30421 pull_request: https://github.com/python/cpython/pull/32383 ___ Python tracker <https://bugs.python.org/issue35134> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24778] mailcap.findmatch: document shell command Injection danger in filename parameter
STINNER Victor added the comment: In 2022, Python 3.11 still has the issue: vstinner@apu$ python3.11 -m mailcap Mailcap files: /home/vstinner/.mailcap /etc/mailcap (...) Mailcap entries: (...) text/html copiousoutput lineno 5 view/usr/bin/xdg-open %s $ python3 -m mailcap text/html 'filename; pwd' Executing: /usr/bin/xdg-open filename; pwd (...) /home/vstinner/python/main Maybe subst() can be modified to work on a list (as Bernd Dietzel proposed) and then use subprocess to avoid shell and so avoid having to pass a single string, but pass a *list* of arguments (strings). The problem is that it would change the public mailcap.findmatch() API: "Return a 2-tuple; the first element is a string containing the command line to be executed (which can be passed to os.system()), (...)" https://docs.python.org/dev/library/mailcap.html#mailcap.findmatch Adding a new findmatch_list() function avoids the backward compatibility issue, but the existing findmatch() function would remain vulnerable. The other problem is that the mailcap.findmatch() function supports "test" command which executes os.system() on string created by mailcap.subst(). Is the mailcap format (RFC 1524) still used in 2022? Does the mailcap module still belong to the Python stdlib in 2022? I propose to: * (1) Document the shell injection vulnerability: the caller is responsible to validate the filename * (2) Deprecate the mailcap module A code search in the top 5000 PyPI projects (at 2022-01-26) did not find any Python source code using the "mailcap" module. I only found the word "mailcap" used to refer to other things: * https://docs.djangoproject.com/en/4.0/ref/contrib/staticfiles/ mentions a "mailcap" RHEL package: "This can be achieved, for example, by installing or updating the mailcap package on a Red Hat distribution, mime-support on a Debian distribution, or by editing the keys under HKEY_CLASSES_ROOT in the Windows registry." * wxPython refers to "KDE< mailcap and mime.types" https://docs.djangoproject.com/en/4.0/ref/contrib/staticfiles/ -- nosy: +vstinner versions: +Python 3.11 -Python 2.7, Python 3.5, Python 3.6, Python 3.7 ___ Python tracker <https://bugs.python.org/issue24778> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40421] [C API] Add public getter functions for the internal PyFrameObject structure
STINNER Victor added the comment: The PyFrameObject structure was made opaque by bpo-46836. -- ___ Python tracker <https://bugs.python.org/issue40421> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue47241] [C API] Move the PyCodeObject structure to the internal C API (make the structure opaque)
New submission from STINNER Victor :
The PyCodeObject structure is documented at:
https://docs.python.org/dev/c-api/code.html
The structured evolved a lot in Python 3.11 to optimize Python/ceval.c
performance:
* read-only co_code (object) was replaced with modifiable co_code_adaptive
(char[])
* co_varnames, co_freevars and co_cellvars were removed: merged into
co_localsplusnames with co_localspluskinds
* co_cell2arg was removed
* co_zombieframe ("free list") was removed
* co_opcache, co_opcache_map, co_opcache_flag and co_opcache_size were removed:
see PEP 659
https://peps.python.org/pep-0659/
New members:
* co_exceptiontable
* co_warmup, co_code_adaptive
* co_nlocalsplus, co_nplaincellvars, co_ncellvars, co_nfreevars
* co_localsplusnames, co_localspluskinds
* co_qualname
* co_endlinetable, co_columntable
The PyCodeObject structure should be made opaque in the public C API and only
accessed with function calls.
In Python 3.11, the PyFrameObject structure was made opaque (bpo-46836) and
multiple getters were added (bpo-40421). The idea is similar, but it might be
too late to do that in Python 3.11 (beta1 feature freeze is close).
---
By the way, it was proposed multiple times on python-dev to mark the
PyCode_New() function as "unstable" since its API changed often. PEP 670
"Python Positional-Only Parameters" caused a lot of troubles in Cython when it
added a new parameter to PyCode_New(). The change was reverted: instead, a new
PyCode_NewWithPosOnlyArgs() function was added.
* PyCode_New() has 19 parameters!
* PyCode_NewWithPosOnlyArgs() has 20 parameters!!
On Python 3.11a1 and newer, Cython uses the code.replace() method (added to
Python 3.8) to build new code objects. Otherwise, it just calls directly
PyCode_New().
https://docs.python.org/dev/library/types.html#types.CodeType.replace
--
components: C API
messages: 416869
nosy: vstinner
priority: normal
severity: normal
status: open
title: [C API] Move the PyCodeObject structure to the internal C API (make the
structure opaque)
versions: Python 3.11
___
Python tracker
<https://bugs.python.org/issue47241>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39947] [C API] Make the PyThreadState structure opaque (move it to the internal C API)
STINNER Victor added the comment: I would be nice to make the PyThreadState opaque in Python 3.12. IMO it's too late for Python 3.11. Hopefully, Cython should be prepared for such change. At the beginning, maybe Cython can just use the internal C API, as it does to access the internal PyFrameObject structure. -- versions: +Python 3.11 -Python 3.9 ___ Python tracker <https://bugs.python.org/issue39947> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40421] [C API] Add public getter functions for the internal PyFrameObject structure
STINNER Victor added the comment: IMO the initial goal is now reached. I close the issue. Thanks to everyone who helped implementing these changes! The PyFrameObject structure is now opaque in Python 3.11. New getter functions of the Python 3.11 C API: * PyFrame_GetBuiltins() * PyFrame_GetGenerator() * PyFrame_GetGlobals() * PyFrame_GetLocals() Finally, the PyFrameObject structure now has its own page in the C API documentation: https://docs.python.org/dev/c-api/frame.html As explained in previous comments, the work is not done: Cython, greenlet, gevent and coverage still need more getter and/or setter functions. Adding more functions is being discussed in this external issue: https://github.com/faster-cpython/ideas/issues/309 I propose to open new specific issues to add new functions. For example, open an issue to add a getter for the f_lasti member. -- resolution: -> fixed stage: patch review -> resolved status: open -> closed ___ Python tracker <https://bugs.python.org/issue40421> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40421] [C API] Add public getter functions for the internal PyFrameObject structure
STINNER Victor added the comment: New changeset 14a9b4895b61bcd46ed968c43c5eec27670a0aac by Victor Stinner in branch 'main': bpo-40421: test_capi uses assertEqual(), not assertEquals() (GH-32361) https://github.com/python/cpython/commit/14a9b4895b61bcd46ed968c43c5eec27670a0aac -- ___ Python tracker <https://bugs.python.org/issue40421> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40421] [C API] Add public getter functions for the internal PyFrameObject structure
Change by STINNER Victor : -- pull_requests: +30410 pull_request: https://github.com/python/cpython/pull/32361 ___ Python tracker <https://bugs.python.org/issue40421> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46850] [C API] Move _PyEval_EvalFrameDefault() to the internal C API
STINNER Victor added the comment: The change was rejected by the Steering Council: https://mail.python.org/archives/list/python-...@python.org/message/GFOMU7LP63JUVFMWNJNZJLUMZDRPTUYJ/ -- resolution: -> rejected stage: patch review -> resolved status: open -> closed ___ Python tracker <https://bugs.python.org/issue46850> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46850] [C API] Move _PyEval_EvalFrameDefault() to the internal C API
STINNER Victor added the comment: New changeset 2b4f2f5fa4d1123c19bf0643cfa5a4fe8df4175c by Victor Stinner in branch 'main': Revert "bpo-46850: Move _PyEval_EvalFrameDefault() to internal C API (GH-32052)" (GH-32343) https://github.com/python/cpython/commit/2b4f2f5fa4d1123c19bf0643cfa5a4fe8df4175c -- ___ Python tracker <https://bugs.python.org/issue46850> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue36207] robotsparser deny all with some rules
STINNER Victor added the comment: I removed two comments: none of the mentioned URL contains a "Disallow: ?" rule and the comments didn't add any value to this issue. It looks like regular spam (SEO). -- ___ Python tracker <https://bugs.python.org/issue36207> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue36207] robotsparser deny all with some rules
Change by STINNER Victor : -- Removed message: https://bugs.python.org/msg416847 ___ Python tracker <https://bugs.python.org/issue36207> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue36207] robotsparser deny all with some rules
Change by STINNER Victor : -- Removed message: https://bugs.python.org/msg416767 ___ Python tracker <https://bugs.python.org/issue36207> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue45210] tp_dealloc docs should mention error indicator may be set
STINNER Victor added the comment:
I wrote GH-32357 to check in debug mode that tp_dealloc functions leave the
current exception unchanged. You can use attached unicode_dealloc_bug.patch to
inject a bug to test my PR. Example ("make" is enough to trigger the bug):
---
Fatal Python error: _Py_Dealloc: Deallocator of type 'str' cleared the current
exception
Python runtime state: initialized
Current thread 0x7ff28d45a740 (most recent call first):
File "/home/vstinner/python/main/Lib/sysconfig.py", line 349 in
_parse_makefile
File "/home/vstinner/python/main/Lib/sysconfig.py", line 470 in
_generate_posix_vars
File "/home/vstinner/python/main/Lib/sysconfig.py", line 845 in _main
File "/home/vstinner/python/main/Lib/sysconfig.py", line 857 in
File "", line 88 in _run_code
File "", line 198 in _run_module_as_main
generate-posix-vars failed
---
--
Added file: https://bugs.python.org/file50724/unicode_dealloc_bug.patch
___
Python tracker
<https://bugs.python.org/issue45210>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue45210] tp_dealloc docs should mention error indicator may be set
Change by STINNER Victor : -- pull_requests: +30407 pull_request: https://github.com/python/cpython/pull/32357 ___ Python tracker <https://bugs.python.org/issue45210> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40222] "Zero cost" exception handling
STINNER Victor added the comment: I created bpo-47236 "Document types.CodeType.replace() changes about co_exceptiontable". -- ___ Python tracker <https://bugs.python.org/issue40222> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue47185] code.replace(co_code=new_code) no longer catch exceptions on Python 3.11
STINNER Victor added the comment: Ok sure, I created bpo-47236 "Document types.CodeType.replace() changes about co_exceptiontable". -- ___ Python tracker <https://bugs.python.org/issue47185> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue47236] Document types.CodeType.replace() changes about co_exceptiontable
New submission from STINNER Victor : It would be nice to document the bpo-40222 changes in What's New in Python 3.11 and in the CodeType documentation: * https://docs.python.org/dev/library/types.html#types.CodeType * https://docs.python.org/dev/whatsnew/3.11.html Users of types.CodeType.replace(co_code=new_code) must now pass co_exceptiontable or exception handling will no longer work as expected. Callers are responsible to build co_exceptiontable. Currently, the format of this table is documented at: https://github.com/python/cpython/blob/main/Objects/exception_handling_notes.txt There is no public function to encode this table, it's also the responsibility of the caller. See also bpo-47185 "code.replace(co_code=new_code) no longer catch exceptions on Python 3.11" closed as "won't fix" for the background. -- assignee: docs@python components: Documentation messages: 416831 nosy: docs@python, vstinner priority: normal severity: normal status: open title: Document types.CodeType.replace() changes about co_exceptiontable versions: Python 3.11 ___ Python tracker <https://bugs.python.org/issue47236> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue47185] code.replace(co_code=new_code) no longer catch exceptions on Python 3.11
STINNER Victor added the comment: >>> def f(): ... foo() ... try: ... bar() ... except: ... pass ... >>> def g(): ... try: ... foo() ... bar() ... except: ... pass ... >>> dis.dis(f) 1 0 RESUME 0 2 2 LOAD_GLOBAL 1 (NULL + foo) 14 PRECALL 0 18 CALL 0 28 POP_TOP 3 30 NOP 4 32 LOAD_GLOBAL 3 (NULL + bar) 44 PRECALL 0 48 CALL 0 58 POP_TOP 60 LOAD_CONST 0 (None) 62 RETURN_VALUE >> 64 PUSH_EXC_INFO 5 66 POP_TOP 6 68 POP_EXCEPT 70 LOAD_CONST 0 (None) 72 RETURN_VALUE >> 74 COPY 3 76 POP_EXCEPT 78 RERAISE 1 ExceptionTable: 32 to 58 -> 64 [0] 64 to 66 -> 74 [1] lasti >>> dis.dis(g) 1 0 RESUME 0 2 2 NOP 3 4 LOAD_GLOBAL 1 (NULL + foo) 16 PRECALL 0 20 CALL 0 30 POP_TOP 4 32 LOAD_GLOBAL 3 (NULL + bar) 44 PRECALL 0 48 CALL 0 58 POP_TOP 60 LOAD_CONST 0 (None) 62 RETURN_VALUE >> 64 PUSH_EXC_INFO 5 66 POP_TOP 6 68 POP_EXCEPT 70 LOAD_CONST 0 (None) 72 RETURN_VALUE >> 74 COPY 3 76 POP_EXCEPT 78 RERAISE 1 ExceptionTable: 4 to 58 -> 64 [0] 64 to 66 -> 74 [1] lasti Oh, I didn't follow recent bytecode changes. Ok, now I see that it is not longer possible to build the exception table just from the bytecode. The purpose of the exception table is to handle exceptions: the opcodes related to exception handles are simply gone in Python 3.11. I was thinking about looking for things like PUSH_EXC_INFO or POP_EXCEPT, but as Guido shows, it doesn't work: the start of the "try" block cannot be detected in the bytecode anymore in Python 3.11. > If code.replace() is not updated to recompute co_exceptiontable, at least, it > would be nice to document the bpo-40222 changes in What's New in Python 3.11 > and in the CodeType documentation You closed the issue. I understand that you don't want to document CodeType.replace() changes neither. Users of this API should follow Python development and notice that their code no longer works with Python 3.11. -- ___ Python tracker <https://bugs.python.org/issue47185> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46607] Add DeprecationWarning to configparser's LegacyInterpolation
STINNER Victor added the comment: Thanks Hugo for the your contribution. I close the issue. > Searching the top 5,000 PyPI sdists, there's very little (if any "real") use > of LegacyInterpolation. Details: https://bugs.python.org/issue45173#msg409685 You can offer them a PR to avoid the deprecated API, or at least notify them by creating an issue that this API is deprecated in Python 3.11. -- resolution: -> fixed stage: patch review -> resolved status: open -> closed ___ Python tracker <https://bugs.python.org/issue46607> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue47231] TarFile.getmember cannot work on tar sourced directory over 100 characters
Change by STINNER Victor : -- nosy: -vstinner ___ Python tracker <https://bugs.python.org/issue47231> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46850] [C API] Move _PyEval_EvalFrameDefault() to the internal C API
Change by STINNER Victor : -- pull_requests: +30397 pull_request: https://github.com/python/cpython/pull/32343 ___ Python tracker <https://bugs.python.org/issue46850> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43557] Deprecate getlocale() and normalize() functions
Change by STINNER Victor : -- title: Deprecate getdefaultlocale(), getlocale() and normalize() functions -> Deprecate getlocale() and normalize() functions ___ Python tracker <https://bugs.python.org/issue43557> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue45410] python -m test -jN: write stderr in stdout to get messages in order
Change by STINNER Victor : -- resolution: -> fixed stage: patch review -> resolved status: open -> closed ___ Python tracker <https://bugs.python.org/issue45410> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue47089] Avoid sporadic failure of test_compileall on Windows
STINNER Victor added the comment: bpo-37387 was closed as a duplicate of this issue. -- ___ Python tracker <https://bugs.python.org/issue47089> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue37387] test_compileall fails randomly on Windows when tests are run in parallel
STINNER Victor added the comment: I close this issue as a duplicate of bpo-47089. -- resolution: -> duplicate stage: -> resolved status: open -> closed superseder: -> Avoid sporadic failure of test_compileall on Windows ___ Python tracker <https://bugs.python.org/issue37387> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue47152] Reorganize the re module sources
STINNER Victor added the comment: The re.template() function and the re.TEMPLATE functions are not documented and not tested. The re.Scanner class is not documented but has a test_scanner() test in test_re. -- ___ Python tracker <https://bugs.python.org/issue47152> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue47152] Reorganize the re module sources
STINNER Victor added the comment: See also bpo-40259: "re.Scanner groups". -- ___ Python tracker <https://bugs.python.org/issue47152> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue47152] Reorganize the re module sources
STINNER Victor added the comment: Old python-dev discussions on re.Scanner from 2000 to 2004: * "[Python-Dev] A standard lexer?" (July 2000) https://mail.python.org/archives/list/python-...@python.org/message/MQ4OMCVIVRJWNGHYGI3OUVZQPN5NNNAU/ thread: https://mail.python.org/archives/list/python-...@python.org/thread/DLMYLYW3QRAAIZDEL3VA7M3TTUWMSPPB/#MQ4OMCVIVRJWNGHYGI3OUVZQPN5NNNAU * "Scanner" (May 2001) https://mail.python.org/archives/list/python-...@python.org/thread/7FGWHTFA2JT23TMVQXLGZLSKG7EGM44Q/#SVQBSSDWPYVHPRS363RWXWGKJTSEYQDP * "iterator support for SRE?" (Oct 2001): https://mail.python.org/archives/list/python-...@python.org/thread/IPJJX6MEW4ATOWHSRKLITL4CAZXDEJ5I/#IPJJX6MEW4ATOWHSRKLITL4CAZXDEJ5I * "should sre.Scanner be exposed through re and documented?" (April 2003) https://mail.python.org/archives/list/python-...@python.org/thread/BHVWYZVMDUJZIJMSSBAAXEH3JI7MTOIJ/#DDFDBY4D6OITPWO26Q5XPBFU7A5X6LXN * "pre-PEP: Complete, Structured Regular Expression Group Matching" (Aug 2004) https://mail.python.org/archives/list/python-...@python.org/thread/5M4YIZ2UFZF5AEWT3CGG74ZHERC6JV3B/#SNURCRGEYANPQVVQFZTY3LTXE2TFEKEP Search for "sre.Scanner". See also: "Using Regular Expressions for Lexical Analysis" (Feb 2002) by Fredrik Lundh https://web.archive.org/web/20200220172033/http://effbot.org/zone/xml-scanner.htm -- ___ Python tracker <https://bugs.python.org/issue47152> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue47143] Add types.copy_class() which updates closures
STINNER Victor added the comment: Lumír Balhar: > Do you think it's a good idea to start a PR with a copy of the implementation > from attrs for Python 3.11? We can then add tests for the new function and > also some for dataclasses where this new function is needed and try to find > all corner cases. I'm worried that attrs license is MIT with an "advertisement clause" (the MIT license must be mentioned), whereas Python has a different license. I'm planning to contact the 3 authors of the code to ask their permission. Also, I expect more feedback on this "idea" first: > I would like to implement this function, but first I would like to discuss if > it makes sense to add such function and check if it's the right abstraction. -- ___ Python tracker <https://bugs.python.org/issue47143> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue47152] Reorganize the re module sources
STINNER Victor added the comment: It's funny to still see mentions of "experimental stuff" in Python 3.11 (2022), whereas these "experimental stuff" are there for 20 years. *Maybe* it's time to consider that re.template() and re.Scanner are no longer experimental? Maybe change their status to alpha or beta? :-D commit 770617b23e286f1147f9480b5f625e88e7badd50 Author: Fredrik Lundh Date: Sun Jan 14 15:06:11 2001 + SRE fixes for 2.1 alpha: +# sre extensions (experimental, don't rely on these) +T = TEMPLATE = sre_compile.SRE_FLAG_TEMPLATE # disable backtracking commit 7cafe4d7e466996d5fc32e871fe834e0e0c94282 Author: Fredrik Lundh Date: Sun Jul 2 17:33:27 2000 + - actually enabled charset anchors in the engine (still not used by the code generator) - changed max repeat value in engine (to match earlier array fix) - added experimental "which part matched?" mechanism to sre; see http://hem.passagen.se/eff/2000_07_01_bot-archive.htm#416954 or python-dev for details. +# experimental stuff (see python-dev discussions for details) + +class Scanner: (...) -- ___ Python tracker <https://bugs.python.org/issue47152> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue47152] Reorganize the re module sources
STINNER Victor added the comment: Is the "import _locale" still used in re/__init__.py? It cannot see any reference to it in the code and test_re still if it's removed. The last reference to the _locale module has been removed in 2017 by the commit 898ff03e1e7925ecde3da66327d3cdc7e07625ba. diff --git a/Lib/re/__init__.py b/Lib/re/__init__.py index c47a2650e3..b887722bbb 100644 --- a/Lib/re/__init__.py +++ b/Lib/re/__init__.py @@ -124,10 +124,6 @@ import enum from . import _compiler, _parser import functools -try: -import _locale -except ImportError: -_locale = None # public symbols -- ___ Python tracker <https://bugs.python.org/issue47152> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue47152] Reorganize the re module sources
STINNER Victor added the comment: $ ls Lib/re/ _compiler.py _constants.py __init__.py _parser.py Thanks, that's a nice enhancement! Serhiy: Would you mind to explicitly document the 3 deprecated modules in What's New in Python 3.11? https://docs.python.org/dev/whatsnew/3.11.html#deprecated -- ___ Python tracker <https://bugs.python.org/issue47152> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue47089] Avoid sporadic failure of test_compileall on Windows
STINNER Victor added the comment: Thanks for this enhancement. I close the issue. Do you know if it does fix the old bpo-37387 issue? -- resolution: -> fixed stage: patch review -> resolved status: open -> closed ___ Python tracker <https://bugs.python.org/issue47089> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue47089] Avoid sporadic failure of test_compileall on Windows
STINNER Victor added the comment: New changeset 306a93b4819db611041da735ca3b34117a3bc961 by Jeremy Kloth in branch '3.9': [3.9] bpo-47089: Avoid test_compileall failures on Windows (GH-32037). (GH-32240) https://github.com/python/cpython/commit/306a93b4819db611041da735ca3b34117a3bc961 -- ___ Python tracker <https://bugs.python.org/issue47089> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue47185] code.replace(co_code=new_code) no longer catch exceptions on Python 3.11
STINNER Victor added the comment: Guido (msg416498) > Surely the bigger issue is that the contents of new_code itself must be > totally different? Also there are other tables that need to be adjusted if > you really do change co_code, e.g. the debugging tables. Do you consider that .replace() must reject changing co_code if other tables are not updated? Debugging tables are not strictly required just to *execute* code, no? If you consider that the caller *must* update co_exceptiontable, replace() must raise an exception in this case, to prevent creating a code object which would behave in a strange way (broken exception handling). If someone really wants testing an empty exception table just for fun, it would still be possible to pass co_exceptiontable=b''. My concern is more about people upgrading to Python 3.11 and who "suddenly" don't get their exceptions handled anymore. I would prefer catching such bug at the replace() call, rather than having to execute the code (and only notice the bug in production? oops). -- ___ Python tracker <https://bugs.python.org/issue47185> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue47089] Avoid sporadic failure of test_compileall on Windows
STINNER Victor added the comment: New changeset 76b8a075b8a79b08468fd0ed06a489a5c815bc11 by Jeremy Kloth in branch 'main': bpo-47089: Avoid test_compileall failures on Windows (GH-32037) https://github.com/python/cpython/commit/76b8a075b8a79b08468fd0ed06a489a5c815bc11 -- ___ Python tracker <https://bugs.python.org/issue47089> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue47185] code.replace(co_code=new_code) no longer catch exceptions on Python 3.11
STINNER Victor added the comment: python-dev thread: https://mail.python.org/archives/list/python-...@python.org/thread/KWSPCLXDHBAP2U4LBSMLQEOC7LREDMPB/ Mark wrote: "You can pass the exception table the same way you pass all the other arguments. The exception table depends on the code, but that is nothing new. The bytecode library already recomputes the consts, names, etc." Constants and names are easy to build, it's just an array and the bytecode refers to their index. Building the exception table is more complicated. It's nice that the format is documented in https://github.com/python/cpython/blob/main/Objects/exception_handling_notes.txt but it would be more convenient to put it in the regular Python documentation (docs.python.org), no? I discovered that file by mistake with filename completion in my editor while looking for Objects/exceptions.c :-) -- ___ Python tracker <https://bugs.python.org/issue47185> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue47185] code.replace(co_code=new_code) no longer catch exceptions on Python 3.11
STINNER Victor added the comment: > How would you compute the exception table from the bytecode? There are no > clues in the bytecode about where the try and except blocks are. Disassemble the bytecode to rebuild basic blocks and detect which ones are except blocks. I don't know how the exception table works :-) It's just a guess. Or do you think that this job should be done by the caller? -- ___ Python tracker <https://bugs.python.org/issue47185> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40222] "Zero cost" exception handling
STINNER Victor added the comment: See bpo-47185: code.replace(co_code=new_code) no longer catch exceptions on Python 3.11. -- nosy: +vstinner ___ Python tracker <https://bugs.python.org/issue40222> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue47185] code.replace(co_code=new_code) no longer catch exceptions on Python 3.11
New submission from STINNER Victor :
Since bpo-40222 "Zero cost exception handling", code object created by from
bytecode with code.replace(co_code=new_code) no longer catch exceptions on
Python 3.11, unless an exception table is set explicitly.
Example:
---
def f():
try:
print("raise")
raise ValueError
except ValueError:
print("except")
else:
print("else")
print("exit func")
def g(): pass
if 1:
code = f.__code__
g.__code__ = g.__code__.replace(
co_code=code.co_code,
co_consts=code.co_consts,
co_names=code.co_names,
co_flags=code.co_flags,
co_stacksize=code.co_stacksize)
else:
g.__code__ = f.__code__ # this code path works on Python 3.11
g()
---
Output with Python 3.10 (ok):
---
raise
except
exit func
---
Output with Python 3.11 (oops):
---
raise
Traceback (most recent call last):
...
ValueError
---
Would it make sense to automatically compute co_exceptiontable on
code.replace(co_code=new_code)? If it's computed automatically, I don't know if
it makes still sense to call code.replace(co_code=new_code,
co_exceptiontable=new_table).
It seems like currently, the only implementation to build an exception table
lives in Python/compile.c which compiles AST to bytecode. It cannot be reused
for code.replace() which takes bytecode as input, not AST.
--
If code.replace() is not updated to recompute co_exceptiontable, at least, it
would be nice to document the bpo-40222 changes in What's New in Python 3.11
and in the CodeType documentation:
* https://docs.python.org/dev/library/types.html#types.CodeType
* https://docs.python.org/dev/whatsnew/3.11.html
--
components: Interpreter Core
messages: 416479
nosy: vstinner
priority: normal
severity: normal
status: open
title: code.replace(co_code=new_code) no longer catch exceptions on Python 3.11
versions: Python 3.11
___
Python tracker
<https://bugs.python.org/issue47185>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46850] [C API] Move _PyEval_EvalFrameDefault() to the internal C API
STINNER Victor added the comment: New changeset f877b40e3f7e0d97878884d80fbec879a85ab7e8 by Victor Stinner in branch 'main': bpo-46850: Move _PyInterpreterState_SetEvalFrameFunc() to internal C API (GH-32054) https://github.com/python/cpython/commit/f877b40e3f7e0d97878884d80fbec879a85ab7e8 -- ___ Python tracker <https://bugs.python.org/issue46850> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46850] [C API] Move _PyEval_EvalFrameDefault() to the internal C API
STINNER Victor added the comment: New changeset b9a5522dd952125a99ff554f01f311cae25f5a91 by Victor Stinner in branch 'main': bpo-46850: Move _PyEval_EvalFrameDefault() to internal C API (GH-32052) https://github.com/python/cpython/commit/b9a5522dd952125a99ff554f01f311cae25f5a91 -- ___ Python tracker <https://bugs.python.org/issue46850> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue47152] Reorganize the re module sources
STINNER Victor added the comment: sre_constants, sre_compile and sre_parse are not tested and are not documented. I don't consider them as public API currently. If someone has good reason to use them, IMO we must clearly define which exact API is needed, properly document and test it. If we expose something, I don't think that the API would be exposed as re.sre_xxx.xxx, but as re.xxx. I suggest to hide sre_xxx submodules by adding an underscore to their name. Moreover, the "sre_" prefix is now redundant. I suggest renaming: * sre_constants => re._constants * sre_compile => re._compile * sre_parse => re._parse -- nosy: +vstinner ___ Python tracker <https://bugs.python.org/issue47152> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue47179] pymalloc should align to max_align_t
STINNER Victor added the comment: Oh, it seems like this issue is a duplicate of bpo-31912 created in 2017. -- ___ Python tracker <https://bugs.python.org/issue47179> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue47179] pymalloc should align to max_align_t
STINNER Victor added the comment: Objects/obmalloc.c currently relies on the SIZEOF_VOID_P macro: --- #if SIZEOF_VOID_P > 4 #define ALIGNMENT 16 /* must be 2^N */ #define ALIGNMENT_SHIFT 4 #else #define ALIGNMENT 8 /* must be 2^N */ #define ALIGNMENT_SHIFT 3 #endif --- If we want to respect sizeof(max_align_t) alignment, we can compute sizeof(max_align_t) in configure and uses the result in obmalloc.c. I expect that it's either 16 or 32, so we can maybe just hardcode ALIGNMENT_SHIFT using something like: "if == 32 ... #elif == 16 ... #else #error ...". On x86 (32-bit) Fedora 35, gcc says 48 for sizeof(max_align_t) which is way larger than the current alignment to 8 bytes! -- ___ Python tracker <https://bugs.python.org/issue47179> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue47179] pymalloc should align to max_align_t
STINNER Victor added the comment:
On my x86-64 Fedora 35, gcc says 32 bytes for sizeof(max_align_t). By the way,
g++ also says 32 bytes for sizeof(std::max_align_t).
GCC 11.2.1 defines max_align_t as:
---
#if (defined (__STDC_VERSION__) && __STDC_VERSION__ >= 201112L) \
|| (defined(__cplusplus) && __cplusplus >= 201103L)
#ifndef _GCC_MAX_ALIGN_T
#define _GCC_MAX_ALIGN_T
/* Type whose alignment is supported in every context and is at least
as great as that of any standard type not using alignment
specifiers. */
typedef struct {
long long __max_align_ll __attribute__((__aligned__(__alignof__(long long;
long double __max_align_ld __attribute__((__aligned__(__alignof__(long
double;
/* _Float128 is defined as a basic type, so max_align_t must be
sufficiently aligned for it. This code must work in C++, so we
use __float128 here; that is only available on some
architectures, but only on i386 is extra alignment needed for
__float128. */
#ifdef __i386__
__float128 __max_align_f128
__attribute__((__aligned__(__alignof(__float128;
#endif
} max_align_t;
#endif
#endif /* C11 or C++11. */
---
file: /usr/lib/gcc/x86_64-redhat-linux/11/include/stddef.h
It's not an union but a structure with 2 fields (1 long long, 1 long double).
The __i386__ macro is not defined on Linux x86-64, so the struct does not have
the 3rd 128-bit float field.
align.c:
---
#include
int main() { return sizeof(max_align_t); }
---
Build and run (C):
---
$ gcc align.c -o align && (./align; echo $?)
32
---
align.cpp:
---
int main() { return sizeof(std::max_align_t); }
---
Build and run (C++):
---
$ g++ align.cpp -o align && (./align; echo $?)
32
---
--
___
Python tracker
<https://bugs.python.org/issue47179>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue47164] [C API] Add private "CAST" macros to clean up casts in C code
Change by STINNER Victor : -- pull_requests: +30286 pull_request: https://github.com/python/cpython/pull/32210 ___ Python tracker <https://bugs.python.org/issue47164> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue47164] [C API] Add private "CAST" macros to clean up casts in C code
STINNER Victor added the comment: New changeset 7fc39a21cb85163a456eab91b52e5fe85e7f7e3e by Victor Stinner in branch 'main': bpo-47164: Add _PyCFunctionObject_CAST() macr (GH-32190) https://github.com/python/cpython/commit/7fc39a21cb85163a456eab91b52e5fe85e7f7e3e -- ___ Python tracker <https://bugs.python.org/issue47164> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue47164] [C API] Add private "CAST" macros to clean up casts in C code
STINNER Victor added the comment: New changeset f0bc69485677ae8973685866ada0982976d3878f by Victor Stinner in branch 'main': bpo-47164: Add _PyCFunction_CAST() macro (GH-32192) https://github.com/python/cpython/commit/f0bc69485677ae8973685866ada0982976d3878f -- ___ Python tracker <https://bugs.python.org/issue47164> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue47164] [C API] Add private "CAST" macros to clean up casts in C code
STINNER Victor added the comment: New changeset c14d7e4b816134b8e93ece4066a86d229631ce96 by Victor Stinner in branch 'main': bpo-47164: Add _PyASCIIObject_CAST() macro (GH-32191) https://github.com/python/cpython/commit/c14d7e4b816134b8e93ece4066a86d229631ce96 -- ___ Python tracker <https://bugs.python.org/issue47164> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue47164] [C API] Add private "CAST" macros to clean up casts in C code
STINNER Victor added the comment: After reading Mark's comments, I reworked my GH-32190 PR to only use the CAST macros in other macros, not in the C code. The CAST macros are not used in such code pattern: else if (PyCFunction_Check(func)) -return ((PyCFunctionObject*)func)->m_ml->ml_name; +return _PyCFunctionObject_CAST(func)->m_ml->ml_name; In fact, this change doesn't bring any value. -- ___ Python tracker <https://bugs.python.org/issue47164> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46964] The global config should not be stored on each interpreter
STINNER Victor added the comment: > If the invariant bits are moved out, then it clearly indicates when code can > safely assume that every interpreter will be seeing the same value. What is the benefit of that? Do you have an example? -- ___ Python tracker <https://bugs.python.org/issue46964> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue47164] [C API] Add private "CAST" macros to clean up casts in C code
STINNER Victor added the comment: > I've had to debug a segfault before only because the inline function > implicitly cast its arguments, and it was accessing a non-existent member. If > it were a macro it would access the struct member directly, and the compiler > would be able to catch that and warn me before runtime. This is part of Python C API legacy and as I wrote, it's not going to change soon. The minor enhancement that we can do is to inject an assertion when Python is built in debug mode. -- ___ Python tracker <https://bugs.python.org/issue47164> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue47164] [C API] Add private "CAST" macros to clean up casts in C code
STINNER Victor added the comment: GH-32192 "Add _PyCFunction_CAST() macro" is special. It's used to define functions in PyTypeObject.tp_methods or PyModuleDef.m_methods. Casting a function pointer can cause issues with Control Flow Integrity (CFI) implemented in LLVM. The _thread module has an undefined behavior fixed by the commit 9eea6eaf23067880f4af3a130e3f67c9812e2f30 of bpo-33015. Previously, a cast ignored that the callback has no return value, whereas pthread_create() requires a function which as a void* return value. To fix compiler warnings, the (PyCFunction)func cast was replaced with the (PyCFunction)(void(*)(void))func cast to fix bpo-bpo-33012 GCC 8 compiler warning: warning: cast between incompatible function types (...) -- ___ Python tracker <https://bugs.python.org/issue47164> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue47164] [C API] Add private "CAST" macros to clean up casts in C code
STINNER Victor added the comment: Hum, there are two things and maybe we are not talking about the same thing. * (A) Modifying macros defined in the Python C API (Include/ header files) doing cast on their arguments to use CAST macros * (B) Modify C code doing casts to use CAST macros I created this issue for (A). I propose to do (B), but this part is optional. Mark, Ken: are you fine with (A)? Is your concern about (B)? But modifying macros to remove the cast of their argument is out of the scope of this issue. As I wrote, it would add compiler warnings, something that I would like to avoid. See: https://peps.python.org/pep-0670/#cast-pointer-arguments -- ___ Python tracker <https://bugs.python.org/issue47164> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue47164] [C API] Add private "CAST" macros to clean up casts in C code
STINNER Victor added the comment: > By the way, the current Python C API is not fully compatible with C++. (...) I created bpo-47165 "[C API] Test that the Python C API is compatible with C++". -- ___ Python tracker <https://bugs.python.org/issue47164> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39355] The Python library will not compile with a C++2020 compiler because the code uses the reserved “module” keyword
STINNER Victor added the comment: Follow-up issue: bpo-47165 "[C API] Test that the Python C API is compatible with C++". -- ___ Python tracker <https://bugs.python.org/issue39355> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue47165] [C API] Test that the Python C API is compatible with C++
New submission from STINNER Victor : There are more and more popular projects using the Python C API. The first big player is pybind11: "Seamless operability between C++11 and Python" https://pybind11.readthedocs.io/ Recently, I proposed a PR to add Python 3.11 support to the datatable project: https://github.com/h2oai/datatable/pull/3231 My PR uses pythoncapi_compat.h header file which provides recent C API functions to old Python functions. The header file implements these functions as static inline function. Problem: a static inline function implemented in a C header file used in a C++ code file can emit C++ compiler warnings. In datatable, I got two kinds of C++ compiler warnings: * Usage of the C NULL constant: C++ prefers nullptr * "Old-style" cast like (PyObject*)obj: C++ prefers static_cast, reinterpret_cast, etc. It seems like these compiler warnings are not enabled by default. The datatable project seems enabling them in its CI and I was asked to fix these warnings. In the pythoncapi-compat project (*), I chose to use nullptr and reinterpret_cast if the "__cplusplus" macro is defined. Example: --- // C++ compatibility #ifdef __cplusplus # define PYCAPI_COMPAT_CAST(TYPE, EXPR) reinterpret_cast(EXPR) # define PYCAPI_COMPAT_NULL nullptr #else # define PYCAPI_COMPAT_CAST(TYPE, EXPR) ((TYPE)(EXPR)) # define PYCAPI_COMPAT_NULL NULL #endif // Cast argument to PyObject* type. #ifndef _PyObject_CAST # define _PyObject_CAST(op) PYCAPI_COMPAT_CAST(PyObject*, op) #endif --- (*) https://github.com/python/pythoncapi_compat It's unclear to me if the Python C API has or has not the same issue than pythoncapi_compat.h. Last years, some old macros of the Python C API have been converted to static inline functions, like Py_INCREF(). It's unclear to me if these compiler warnings happen on Py_INCREF(). I don't understand why, but static inline macros from Python.h didn't emit compiler warnings in datatable, whereas similar static inline functions of pythoncapi_compat.h emitted compiler warnings. Maybe there is a difference between and "Python.h". Or maybe it depends if the header file is a "local" file, or a "system" header file (ex: installed in /usr/include/ on Linux). A first step would be to build a C++ extension as part of the Python test suite and check that there is no compiler warning. My GH-32175 PR is a proof-of-concept of that. I don't know which C++ version we should target. pybind11 targets C++11. See bpo-39355 for a discussion about C++20: usage of the C++20 "module" keyword... which is a "contextual keyword" in practice. -- components: C API messages: 416359 nosy: vstinner priority: normal severity: normal status: open title: [C API] Test that the Python C API is compatible with C++ type: enhancement versions: Python 3.11 ___ Python tracker <https://bugs.python.org/issue47165> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue47164] [C API] Add private "CAST" macros to clean up casts in C code
STINNER Victor added the comment: > I think that adding macros makes readability worse. See the attached PRs, I can remove multiple layers of parenthesis in macros by using CAST macros. > ``` > PyObject *func = ...; > int flags = PyCFunction_GET_FLAGS(func); > ``` > > is dangerous. Right. PEP 670 proposes to remove the cast, but only in the limited C API version 3.11 or newer. Sadly, I don't think that we can remove the cast in the general Python C API, it would simply add too many compiler warnings in existing C extensions which previously built without warnings. (See also PEP 670 discussions on python-dev). Currently, PyCFunction_GET_FLAGS() doesn't check its argument. The macro documentation is quite explicit about it: /* Macros for direct access to these values. Type checks are *not* done, so use with care. */ My GH-32190 PR adds a check in debug mode. So using PyCFunction_GET_FLAGS() in debug mode is safer with this PR. -- ___ Python tracker <https://bugs.python.org/issue47164> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue47164] [C API] Add private "CAST" macros to clean up casts in C code
Change by STINNER Victor : -- pull_requests: +30270 pull_request: https://github.com/python/cpython/pull/32192 ___ Python tracker <https://bugs.python.org/issue47164> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue47164] [C API] Add private "CAST" macros to clean up casts in C code
Change by STINNER Victor : -- pull_requests: +30269 pull_request: https://github.com/python/cpython/pull/32191 ___ Python tracker <https://bugs.python.org/issue47164> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue47164] [C API] Add private "CAST" macros to clean up casts in C code
Change by STINNER Victor : -- keywords: +patch pull_requests: +30268 stage: -> patch review pull_request: https://github.com/python/cpython/pull/32190 ___ Python tracker <https://bugs.python.org/issue47164> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue47164] [C API] Add private "CAST" macros to clean up casts in C code
New submission from STINNER Victor : Last years, I started to add "CAST" macros like: #define _PyObject_CAST(op) ((PyObject*)(op)) #define _PyType_CAST(op) (assert(PyType_Check(op)), (PyTypeObject*)(op)) Example of usage: #define PyObject_TypeCheck(ob, type) PyObject_TypeCheck(_PyObject_CAST(ob), type) These macros avoids parenthesis. Example without CAST macro: #define PyCFunction_GET_FLAGS(func) \ (((PyCFunctionObject *)func) -> m_ml -> ml_flags) Currently, inline cast requires adding parenthesis: ((PyCFunctionObject *)func) IMO it's more readable with a CAST macro: #define _PyCFunction_CAST(func) ((PyCFunctionObject *)func) #define PyCFunction_GET_FLAGS(func) \ (_PyCFunction_CAST(func)->m_ml->ml_flags) I propose to add more CAST macros. By the way, the current Python C API is not fully compatible with C++. "(type)expr" C syntax is seen as "old-style cast" in C++ which recommends static_cast(expr), reinterpret_cast(expr), or another kind of cast. But I prefer to discuss C++ in a separated issue ;-) IMO without considering C++, adding CAST macros is worth it for readability. I am preparing pull requests for add CAST macros and use existing CAST macros. -- components: C API messages: 416350 nosy: vstinner priority: normal severity: normal status: open title: [C API] Add private "CAST" macros to clean up casts in C code versions: Python 3.11 ___ Python tracker <https://bugs.python.org/issue47164> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue47151] subprocess fails when used as init, vfork() returns EINVAL if PID=1
STINNER Victor added the comment: > Any possibility that you can test the attached PR as pid 1? Python should be built as a static binary and can be used as the init process on the kernel command line, no? I'm not sure that "static binary" is a requirement, since I'm commonly using init=/usr/bin/bash to fix a broken Linux, and on my Fedora, this program is "dynamically linked". Who uses Python as pid 1? I'm now curious :-) -- nosy: +vstinner ___ Python tracker <https://bugs.python.org/issue47151> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39355] The Python library will not compile with a C++2020 compiler because the code uses the reserved “module” keyword
STINNER Victor added the comment: The C++20 "module" keyword is "contextual keyword". It's only a keyword if the first line if a file contains "module". * https://en.cppreference.com/w/cpp/language/modules#Module_declarations * https://en.cppreference.com/w/cpp/keyword/module It's not the case in any .h file of the Python C API, so Python doesn't need to be changed. I close the issue. -- resolution: -> not a bug stage: patch review -> resolved status: open -> closed ___ Python tracker <https://bugs.python.org/issue39355> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39355] The Python library will not compile with a C++2020 compiler because the code uses the reserved “module” keyword
STINNER Victor added the comment: I wrote the draft PR GH-32175 to test https://bugs.python.org/issue39355 and GH-31282. Problem: I don't get any compiler warning or error about the "module" C++20 keyword. I tested GCC 11.2.1 and clang 13.0.0 of Fedora 35. -- ___ Python tracker <https://bugs.python.org/issue39355> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39355] The Python library will not compile with a C++2020 compiler because the code uses the reserved “module” keyword
Change by STINNER Victor : -- pull_requests: +30252 pull_request: https://github.com/python/cpython/pull/32175 ___ Python tracker <https://bugs.python.org/issue39355> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39355] The Python library will not compile with a C++2020 compiler because the code uses the reserved “module” keyword
STINNER Victor added the comment: STINNER Victor: > What is the error message? How can the error be reproduced? Keith (aCuria): > Compile with a compiler supporting the C++20 core feature (Modules) > https://en.cppreference.com/w/cpp/compiler_support I built a C++ extension which calls PyModule_AddType(): I get no warning. I tested GCC and LLVM clang. Commands: clang -Wno-unused-result -g -Og -Wall -O0 -fPIC -I/home/vstinner/python/main/Include -I/home/vstinner/python/main -c Modules/_testcppext.cpp -o build/temp.linux-x86_64-3.11-pydebug/Modules/_testcppext.o -I/home/vstinner/python -Werror -Wall -Wextra -Wconversion -Wno-typedef-redefinition -std=c++20 -Wzero-as-null-pointer-constant -Wold-style-cast gcc -Wno-unused-result -g -Og -Wall -O0 -fPIC -I/home/vstinner/python/main/Include -I/home/vstinner/python/main -c Modules/_testcppext.cpp -o build/temp.linux-x86_64-3.11-pydebug/Modules/_testcppext.o -I/home/vstinner/python -Werror -Wall -Wextra -Wconversion -Wno-typedef-redefinition -std=c++20 -Wzero-as-null-pointer-constant -Wold-style-cast Reformatted commands: ['clang', '-Wno-unused-result', '-g', '-Og', '-Wall', '-O0', '-fPIC', '-I/home/vstinner/python/main/Include', '-I/home/vstinner/python/main', '-c', 'Modules/_testcppext.cpp', '-o', 'build/temp.linux-x86_64-3.11-pydebug/Modules/_testcppext.o', '-I/home/vstinner/python', '-Werror', '-Wall', '-Wextra', '-Wconversion', '-Wno-typedef-redefinition', '-std=c++20', '-Wzero-as-null-pointer-constant', '-Wold-style-cast'] ['gcc', '-Wno-unused-result', '-g', '-Og', '-Wall', '-O0', '-fPIC', '-I/home/vstinner/python/main/Include', '-I/home/vstinner/python/main', '-c', 'Modules/_testcppext.cpp', '-o', 'build/temp.linux-x86_64-3.11-pydebug/Modules/_testcppext.o', '-I/home/vstinner/python', '-Werror', '-Wall', '-Wextra', '-Wconversion', '-Wno-typedef-redefinition', '-std=c++20', '-Wzero-as-null-pointer-constant', '-Wold-style-cast'] -- versions: +Python 3.10, Python 3.11 -Python 3.7, Python 3.8 ___ Python tracker <https://bugs.python.org/issue39355> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39355] The Python library will not compile with a C++2020 compiler because the code uses the reserved “module” keyword
STINNER Victor added the comment: When I wrote a PR to use the C header file pythoncapi_compat.h in the datatable C++ project, I got multiple C++ compiler warnings in static inline functions and in some macros: https://github.com/h2oai/datatable/pull/3231#issuecomment-1032864790 * Usage of NULL * Usage of "old-style cast" like (ssize_t)1 or (PyObject*)obj I solved this issue in pythoncapi_compat.h by using reinterpret_cast and nullptr: * https://github.com/python/pythoncapi_compat/commit/347746379f79fa091017e23427932c9f9980234d * https://github.com/python/pythoncapi_compat/pull/18 * https://github.com/h2oai/datatable/pull/3237 By the way, pythoncapi_compat.h no longer uses "module": https://github.com/python/pythoncapi_compat/pull/22 The Python C API has similar issues, but warnings about NULL and old-style cast depend on the C++ compiler flags: * -Wzero-as-null-pointer-constant * -Wold-style-cast -- ___ Python tracker <https://bugs.python.org/issue39355> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39355] The Python library will not compile with a C++2020 compiler because the code uses the reserved “module” keyword
STINNER Victor added the comment:
If I build a C++ extension with -std=c++20, I get a compiler error on
PyModuleDef_HEAD_INIT:
Modules/_testcppext.cpp:419:5: error: either all initializer clauses should be
designated or none of them should be
419 | .m_name = "_testcppext",
| ^
Code:
---
static struct PyModuleDef module = {
PyModuleDef_HEAD_INIT,
.m_name = "_testcppext",
.m_doc = module_doc,
...
};
---
Macro defined as (simplified code):
---
#define PyObject_HEAD_INIT(type) \
{ 1, type },
#define PyModuleDef_HEAD_INIT { \
PyObject_HEAD_INIT(NULL)\
NULL, /* m_init */ \
0,/* m_index */ \
NULL, /* m_copy */ \
}
---
--
___
Python tracker
<https://bugs.python.org/issue39355>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue47143] Add types.copy_class() which updates closures
STINNER Victor added the comment: Jelle Zijlstra: > I believe the attrs code wouldn't work if a method is decorated with a > decorator that wraps the original function, such as @functools.cache. What do you mean by "wouldn't work"? Do you mean that the semantics of "copy_class()" should be better defined? Currently, copy.deepcopy(MyClass) doesn't copy the class at all, it returns the class unmodified :-) @functools.cache is designed for unbound methods. Example: --- import attr import functools @attr.s(slots=True) class A: @staticmethod @functools.cache def incr(x): return x + 1 @staticmethod @functools.lru_cache def incr_lru(x): return x + 1 obj = A() print(obj.incr(1)) print(obj.incr_lru(2)) --- Output (current Python main branch, attrs 21.4.0): --- 2 3 --- @attr.s(slots=True) copies a class but then drops the original class. It doesn't create two classes which share methods, functools caches, etc. -- ___ Python tracker <https://bugs.python.org/issue47143> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue47143] Add types.copy_class() which updates closures
STINNER Victor added the comment: The stdlib types module looks like a better place for such new function, rather than the functools module. The types module documentation starts with: "This module defines utility functions to assist in dynamic creation of new types." https://docs.python.org/3/library/types.html -- title: Add functools.copy_class() which updates closures -> Add types.copy_class() which updates closures ___ Python tracker <https://bugs.python.org/issue47143> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39805] Copying functions doesn't actually copy them
STINNER Victor added the comment: See also bpo-47143 "Add functools.copy_class() which updates closures". -- nosy: +vstinner ___ Python tracker <https://bugs.python.org/issue39805> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue29270] ctypes: fail to create a _ctypes._SimpleCData subclass using a closure like calling super() without arguments
STINNER Victor added the comment:
CreateSwappedType() is an helper function used by the _ctypes.PyCSimpleType
type. Python script reproducing this ctypes bug:
---
class PyCSimpleType(type):
def __new__(cls, name, bases, dct):
print(f"PyCSimpleType: create {name} class")
cell = dct.get('__classcell__', None)
# type.__new__() sets __classcell__
x = super().__new__(cls, name, bases, dct)
if cell is not None:
print(f"PyCSimpleType: after first type.__new__() call:
__classcell__={cell.cell_contents}")
name2 = name + "_Swapped"
dct2 = dict(dct, __qualname__=name2)
# Calling type.__new__() again with the same cell object overrides
# __classcell__
x.__ctype_be__ = super().__new__(cls, name2, bases, dct2)
if cell is not None:
print(f"PyCSimpleType: after second type.__new__() call:
__classcell__={cell.cell_contents}")
return x
class BaseItem:
pass
class Item(BaseItem, metaclass=PyCSimpleType):
def get_class(self):
# get '__class__' to create a closure
return __class__
# Alternative to create a closure:
#def __repr__(self):
#return super().__repr__()
---
Output:
---
PyCSimpleType: create Item class
PyCSimpleType: after first type.__new__() call: __classcell__=
PyCSimpleType: after second type.__new__() call: __classcell__=
Traceback (most recent call last):
File "meta.py", line 23, in
class Item(BaseItem, metaclass=PyCSimpleType):
TypeError: __class__ set to defining 'Item' as
---
It's not a bug in Python types, but a bug specific to the _ctypes.PyCSimpleType
type which prevents creating subclasses which use closures ("__class__",
"super()", etc.).
--
nosy: +vstinner
title: super call in ctypes subclass fails -> ctypes: fail to create a
_ctypes._SimpleCData subclass using a closure like calling super() without
arguments
versions: +Python 3.11 -Python 3.8
___
Python tracker
<https://bugs.python.org/issue29270>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue47143] Add functools.copy_class() which updates closures
STINNER Victor added the comment: Note: Implementing a metaclass in Python is hard, it's easy to mess up with closures: see bpo-29270 "ctypes: fail to create a _ctypes._SimpleCData subclass using a closure like calling super() without arguments". type.__new__() is called twice on the same type dict, and the second call overrides the __classcell__ cell value. -- ___ Python tracker <https://bugs.python.org/issue47143> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue47143] Add functools.copy_class() which updates closures
STINNER Victor added the comment: > The same problem exists at the function level: see bpo-39805: "Copying > functions doesn't actually copy them". See also bpo-14369 "make function __closure__ writable". -- ___ Python tracker <https://bugs.python.org/issue47143> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14369] make function __closure__ writable
Change by STINNER Victor : -- title: make __closure__ writable -> make function __closure__ writable ___ Python tracker <https://bugs.python.org/issue14369> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue47143] Add functools.copy_class() which updates closures
STINNER Victor added the comment: bpo-32176 "Zero argument super is broken in 3.6 for methods with a hacked __class__ cell" added test_code.test_closure_injection() and fixed the CO_NOFREE flag in the code object constructor (types.CodeType). -- ___ Python tracker <https://bugs.python.org/issue47143> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue47143] Add functools.copy_class() which updates closures
STINNER Victor added the comment: > If I understand correctly, a cell content can be modified since Python 3.7: > since commit 64505a1f6c0af4574e17e823b27ffe24eca44df5 of bpo-30486 Moreover, it's possible to create a cell object since Python 3.8, commit df8d2cde63c865446468351f8f648e1c7bd45109 of bpo-35911. -- nosy: +pitrou ___ Python tracker <https://bugs.python.org/issue47143> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue47143] Add functools.copy_class() which updates closures
STINNER Victor added the comment: See also the types.new_class() function: https://docs.python.org/dev/library/types.html#types.new_class Oh, I didn't know this function! -- ___ Python tracker <https://bugs.python.org/issue47143> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue29944] Argumentless super() fails in classes constructed with type()
STINNER Victor added the comment: See also bpo-47143 "Add functools.copy_class() which updates closures". -- nosy: +vstinner ___ Python tracker <https://bugs.python.org/issue29944> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue47143] Add functools.copy_class() which updates closures
STINNER Victor added the comment: > * old fixed attrs issue: https://github.com/python-attrs/attrs/issues/102 > * attrs issue with Python 3.11: > https://github.com/python-attrs/attrs/issues/907 > * dataclasses issues with slots=True: https://bugs.python.org/issue46404 Similar bug without attrs nor dataclasses: bpo-29944 "Argumentless super() fails in classes constructed with type()". -- ___ Python tracker <https://bugs.python.org/issue47143> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue47143] Add functools.copy_class() which updates closures
STINNER Victor added the comment: The same problem exists at the function level: see bpo-39805: "Copying functions doesn't actually copy them". For example, copy.deepcopy(func) returns func unchanged if it's a function. Example: --- import copy def make_closure(): closure = [] def append(value): closure.append(value) return append, closure func, closure = make_closure() func(1) func2 = copy.deepcopy(func) func2(2) print(func2 is func) print(closure) --- Output: --- True [1, 2] --- -- ___ Python tracker <https://bugs.python.org/issue47143> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue47143] Add functools.copy_class() which updates closures
STINNER Victor added the comment: In the Python C API, PEP 384 added PyType_FromSpec(). There is also PyStructSequence_NewType(). PEP 3121 proposed PyType_Copy() but it was never implemented: see bpo-3760. But in C, closures are implemented using a module state, or previously using a global or static variable: cell objects are not used for types implemented in C. -- ___ Python tracker <https://bugs.python.org/issue47143> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue47143] Add functools.copy_class() which updates closures
STINNER Victor added the comment: If I understand correctly, a cell content can be modified since Python 3.7: since commit 64505a1f6c0af4574e17e823b27ffe24eca44df5 of bpo-30486: bpo-30486: Allow setting cell value (#1840) Antoine Pitrou created bpo-30486 for cloudpickle: "There are use cases for setting a cell value. One such use case is for (un)pickling recursive closures (see heroic workaround here: https://github.com/cloudpipe/cloudpickle/pull/90/files#diff-d2a3618afedd4e124c532151eedbae09R74 ). Other use cases may include tinkering around and general education value." -- ___ Python tracker <https://bugs.python.org/issue47143> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue47143] Add functools.copy_class() which updates closures
STINNER Victor added the comment: More recent copy.copy() change: commit 5c1c3b4f197c57952760be37d77d73669284a607 of bpo-11480: Issue #11480: Fixed copy.copy to work with classes with custom metaclasses. +try: +issc = issubclass(cls, type) +except TypeError: # cls is not a class +issc = False +if issc: +# treat it as a regular class: +return _copy_immutable(x) -- ___ Python tracker <https://bugs.python.org/issue47143> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue47143] Add functools.copy_class() which updates closures
STINNER Victor added the comment: pickle.dump(x) checks if x is a type since commit f048a8f6d79173cc1da1bf12c60ae06fea36762c (March 2002) of bpo-494904: Pickler.save(): Fix for SF bug #494904: Cannot pickle a class with a metaclass, reported by Dan Parisien. +if issubclass(t, TypeType): +self.save_global(object) +return Followed by a minor fix: commit 85ee491b3af3e1c124522249a52443b4d8c34c88 of bpo-502085: Don't die when issubclass(t, TypeType) fails. -if issubclass(t, TypeType): +try: +issc = issubclass(t, TypeType) +except TypeError: # t is not a class +issc = 0 copy.deepcopy(x) returns x if it's a type since commit 11ade1ddc053dcec884e2431b55fb1c1727c65d7 (June 2002) of bpo-560794. SF patch 560794 (Greg Chapman): deepcopy can't handle custom metaclasses. try: -copier = x.__deepcopy__ -except AttributeError: +issc = issubclass(type(x), type) +except TypeError: +issc = 0 +if issc: +y = _deepcopy_dispatch[type](x, memo) +else: (...) -- ___ Python tracker <https://bugs.python.org/issue47143> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue47143] Add functools.copy_class() which updates closures
STINNER Victor added the comment:
The pickle module doesn't copy a type but gets it from its module. The Python
implementation is pickle._Pickler.save_type() which calls
pickle._Pickler.save_global().
The cloudpickle module doesn't copy types neither: same behavior than pickle.
Example:
---
import pickle
import pickletools
class A:
pass
data = pickle.dumps(A)
pickletools.dis(data)
---
Output:
---
0: \x80 PROTO 4
2: \x95 FRAME 18
11: \x8c SHORT_BINUNICODE '__main__'
21: \x94 MEMOIZE(as 0)
22: \x8c SHORT_BINUNICODE 'A'
25: \x94 MEMOIZE(as 1)
26: \x93 STACK_GLOBAL
27: \x94 MEMOIZE(as 2)
28: .STOP
highest protocol among opcodes = 4
---
In short, it's implemented as:
getattr(__import__('__main__'), 'A')
--
___
Python tracker
<https://bugs.python.org/issue47143>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue47143] Add functools.copy_class() which updates closures
STINNER Victor added the comment: It seems like the copy module doesn't support copying a class. copy.deepcopy(cls) doesn't copy a class but returns its argument, the class unchanged. -- ___ Python tracker <https://bugs.python.org/issue47143> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46404] RFE: @dataclasses.dataclass(slots=True) doesn't support methods using closures
STINNER Victor added the comment: See also bpo-45520: "Frozen dataclass deep copy doesn't work with __slots__" which is related but a different issue. -- ___ Python tracker <https://bugs.python.org/issue46404> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
