[issue42671] Make the Python finalization more deterministic

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

I experimented with different approaches last evening. Just changing the order 
of clearing the module dict does not help your first example, because the 
__main__ module is garbage collected, but its dict is not cleared. Adding 
_PyModule_ClearDict() in tp_dealloc and tp_clear helps with that example, but 
it causes several failures and crashes in tests. I have found issue7140 and 
issue18214 for not clearing module dicts.

So in conclusion, your approach may work. It clears dicts of all modules, but 
only at shutdown. It did not worked before, when the order of dicts was 
undetermined, but now we can utilize deterministic order of sys.modules and 
module dicts.

--
nosy: +eric.snow, pitrou

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1635741] Py_Finalize() doesn't clear all Python objects at exit

[STINNER Victor]

STINNER Victor  added the comment:

> * bpo-15651: PEP 3121, 384 refactoring applied to elementtree module

Oops, this was a mistake. This one is not fixed yet.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15651] PEP 3121, 384 refactoring applied to elementtree module

[STINNER Victor]

STINNER Victor  added the comment:

> Fixed by: (...) bpo-1635741: Convert _sre types to heap types...

Oops, I commented the wrong issue. I reopen it.

--
resolution: duplicate -> 
status: closed -> open
superseder: Py_Finalize() doesn't clear all Python objects at exit -> 

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1635741] Py_Finalize() doesn't clear all Python objects at exit

[STINNER Victor]

STINNER Victor  added the comment:

Statistics on C extension modules using the new API (PyModuleDef_Init) / total:

* 3.9: 52 / 120 (43%)
* master: 94 / 119 (79%)

25 extension modules are still using the old API (PyModule_Create).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15725] PyType_FromSpecWithBases bugfix

[STINNER Victor]

STINNER Victor  added the comment:

Tons of extension modules have been ported to the multiphase initialization API 
(PEP 489) in bpo-1635741 without this fix. I don't think that it's needed.

--
nosy: +vstinner
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1635741] Py_Finalize() doesn't clear all Python objects at exit

[STINNER Victor]

STINNER Victor  added the comment:

I marked the following issues as duplicates of this one:

* bpo-15651: PEP 3121, 384 refactoring applied to elementtree module
* bpo-15650: PEP 3121, 384 refactoring applied to dbm module
* bpo-15712: PEP 3121, 384 Refactoring applied to unicodedata module
* bpo-15674: PEP 3121, 384 Refactoring applied to _thread module
* bpo-15669: PEP 3121, 384 Refactoring applied to sre module

I also closed bpo-15848 "PEP 3121, 384 Refactoring applied to xxsubtype module" 
(but not marked it as a duplicate of this one).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15669] PEP 3121, 384 Refactoring applied to sre module

[STINNER Victor]

STINNER Victor  added the comment:

Fixed by:

commit a6109ef68d421712ba368ef502c4789e8de113e0
Author: Erlend Egeberg Aasland 
Date:   Fri Nov 20 13:36:23 2020 +0100

bpo-1635741: Convert _sre types to heap types and establish module state 
(PEP 384) (GH-23393)

--
nosy: +vstinner
resolution:  -> fixed
stage:  -> resolved
status: open -> closed
superseder:  -> Py_Finalize() doesn't clear all Python objects at exit

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15848] PEP 3121, 384 Refactoring applied to xxsubtype module

[STINNER Victor]

STINNER Victor  added the comment:

See bpo-4 "[C API] Convert a few stdlib extensions to the limited C API 
(PEP 384)" for the PEP 384 part.

--
resolution:  -> fixed
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15651] PEP 3121, 384 refactoring applied to elementtree module

[STINNER Victor]

STINNER Victor  added the comment:

Fixed by:

commit a6109ef68d421712ba368ef502c4789e8de113e0
Author: Erlend Egeberg Aasland 
Date:   Fri Nov 20 13:36:23 2020 +0100

bpo-1635741: Convert _sre types to heap types and establish module state 
(PEP 384) (GH-23393)

--
nosy: +vstinner
resolution:  -> duplicate
stage: patch review -> resolved
status: open -> closed
superseder:  -> Py_Finalize() doesn't clear all Python objects at exit

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15650] PEP 3121, 384 refactoring applied to dbm module

[STINNER Victor]

STINNER Victor  added the comment:

Fixed by:

commit bf69a8f99f1b0e19a59509c6c4d7015a31d881a1
Author: Dong-hee Na 
Date:   Tue Jun 16 01:20:54 2020 +0900

bpo-1635741: Port _dbm module to multiphase initialization (GH-20848)

--
nosy: +vstinner
resolution:  -> duplicate
stage:  -> resolved
status: open -> closed
superseder:  -> Py_Finalize() doesn't clear all Python objects at exit

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15712] PEP 3121, 384 Refactoring applied to unicodedata module

[STINNER Victor]

STINNER Victor  added the comment:

Fixed by multiple changes, especially:

commit c8c4200b65b2159bbb13cee10d67dfb3676fef26
Author: Victor Stinner 
Date:   Mon Oct 26 23:19:22 2020 +0100

bpo-42157: Convert unicodedata.UCD to heap type (GH-22991)

Convert the unicodedata extension module to the multiphase
initialization API (PEP 489) and convert the unicodedata.UCD static
type to a heap type.

Co-Authored-By: Mohamed Koubaa 

--
nosy: +vstinner
resolution:  -> duplicate
stage:  -> resolved
status: open -> closed
superseder:  -> Cleanup the unicodedata module

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15674] PEP 3121, 384 Refactoring applied to _thread module

[STINNER Victor]

STINNER Victor  added the comment:

Fixed by:

New changeset 6104013838e181e3c698cb07316f449a0c31ea96 by Victor Stinner in 
branch 'master':
bpo-1635741: Port _thread to multiphase init (GH-23811)
https://github.com/python/cpython/commit/6104013838e181e3c698cb07316f449a0c31ea96

--
nosy: +vstinner
resolution:  -> duplicate
stage:  -> resolved
status: open -> closed
superseder:  -> Py_Finalize() doesn't clear all Python objects at exit

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1635741] Py_Finalize() doesn't clear all Python objects at exit

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 6104013838e181e3c698cb07316f449a0c31ea96 by Victor Stinner in 
branch 'master':
bpo-1635741: Port _thread to multiphase init (GH-23811)
https://github.com/python/cpython/commit/6104013838e181e3c698cb07316f449a0c31ea96


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42199] bytecode_helper assertNotInBytecode fails too eagerly

[Dino Viehland]

Dino Viehland  added the comment:


New changeset 6e799be0a18d0bb5bbbdc77cd3c30a229d31dfb4 by Max Bernstein in 
branch 'master':
bpo-42199: Fix bytecode_helper assertNotInBytecode (#23031)
https://github.com/python/cpython/commit/6e799be0a18d0bb5bbbdc77cd3c30a229d31dfb4


--
nosy: +dino.viehland

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42600] Cancelling tasks waiting for asyncio.Conditions crashes w/ RuntimeError: Lock is not acquired.

[Yurii Karabas]

Yurii Karabas <1998uri...@gmail.com> added the comment:

Hey Yurii, I have fixed this issue.
The actual problem was that `wait_for` cancels passed future but doesn't wait 
until it finished.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42600] Cancelling tasks waiting for asyncio.Conditions crashes w/ RuntimeError: Lock is not acquired.

[Yurii Karabas]

Change by Yurii Karabas <1998uri...@gmail.com>:


--
keywords: +patch
nosy: +uriyyo
nosy_count: 4.0 -> 5.0
pull_requests: +22689
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/23829

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42652] recursive in finally clause cause Python interpreter crash.

[Steve Stagg]

Steve Stagg  added the comment:

This was also fixed by bpo-42500, in commit 4e7a69bdb6 
===
user@obsidian ~/t/f/cpython ((93a0ef76…))> git checkout 4e7a69bd^
HEAD is now at 93a0ef7647 Correct return type in 
Modules/_ssl.c::sslmodule_legacy (GH-23609)
+ make distclean
+ CFLAGS=-O0
+ ./configure --prefix=/home/user/prefix --exec-prefix=/home/user/prefix 
--cache-file=../config.cache --with-pydebug --without-ensurepip
+ CFLAGS=-O0
+ make -j8 install
/home/user/cpython/Lib/runpy.py:111: PendingDeprecationWarning: lib2to3 package 
is deprecated and may not be able to parse Python 3.10+
  __import__(pkg_name)
/home/user/cpython/Lib/runpy.py:111: PendingDeprecationWarning: lib2to3 package 
is deprecated and may not be able to parse Python 3.10+
  __import__(pkg_name)
++ realpath /home/user/prefix/include/python3.10d/
+ INCLUDE_DIR=/home/user/prefix/include/python3.10d
+ set +e
+ PYTHONHOME=/home/user/prefix
+ /home/user/prefix/bin/python3 ../test.py
Traceback (most recent call last):
  File "/home/user/cpython/../test.py", line 5, in foo
1/0
ZeroDivisionError: division by zero
Traceback (most recent call last):
  File "/home/user/cpython/../test.py", line 5, in foo
1/0
ZeroDivisionError: division by zero
...
  File "/home/user/cpython/../test.py", line 10 in foo
  File "/home/user/cpython/../test.py", line 10 in foo
  ...
../test.sh: line 33: 384713 Aborted (core dumped) 
PYTHONHOME=$PREFIX $EPREFIX/bin/python3 ../test.py
+ RESULT=134
+ [[ 134 -eq 139 ]]
+ [[ 134 -eq 134 ]]
+ exit 1

user@obsidian ~/t/f/cpython ((93a0ef76…)) [1]> git checkout 4e7a69bd
Previous HEAD position was 93a0ef7647 Correct return type in 
Modules/_ssl.c::sslmodule_legacy (GH-23609)
HEAD is now at 4e7a69bdb6 bpo-42500: Fix recursion in or after except (GH-23568)

user@obsidian ~/t/f/cpython ((4e7a69bd…))> ../test.sh
+ make distclean
+ CFLAGS=-O0
+ ./configure --prefix=/home/user/prefix --exec-prefix=/home/user/prefix 
--cache-file=../config.cache --with-pydebug --without-ensurepip
+ CFLAGS=-O0
+ make -j8 install
/home/user/cpython/Lib/runpy.py:111: PendingDeprecationWarning: lib2to3 package 
is deprecated and may not be able to parse Python 3.10+
  __import__(pkg_name)
/home/user/cpython/Lib/runpy.py:111: PendingDeprecationWarning: lib2to3 package 
is deprecated and may not be able to parse Python 3.10+
  __import__(pkg_name)
+ /home/user/prefix/bin/python3 ../test.py
Traceback (most recent call last):
  File "/home/user/cpython/../test.py", line 5, in foo
1/0
...
  File "/home/user/cpython/../test.py", line 10, in foo
foo()   
  File "/home/user/cpython/../test.py", line 10, in foo
foo()   
  [Previous line repeated 996 more times]
RecursionError: maximum recursion depth exceeded
+ RESULT=1
+ [[ 1 -eq 139 ]]
+ [[ 1 -eq 134 ]]
+ exit 0

--
nosy: +stestagg

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42671] Make the Python finalization more deterministic

[STINNER Victor]

STINNER Victor  added the comment:

I took some notes on latest changes of the Python finalization order:
https://pythondev.readthedocs.io/finalization.html#reorder-python-finalization

Not directly related, I also took notes on the "weird GC behavior during Python 
finalization" (such as the issue that I have with my _thread change):
https://pythondev.readthedocs.io/finalization.html#weird-gc-behavior-during-python-finalization

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42671] Make the Python finalization more deterministic

[STINNER Victor]

STINNER Victor  added the comment:

> Modules are already cleared in the reverse order.

In Python 3.9, _PyImport_Cleanup() of Python/import.c contains the comment:

/* Since dict is ordered in CPython 3.6+, modules are saved in
   importing order.  First clear modules imported later. */

But using Python 3.9, the z.py example output is:
---
a
b
c
---

The most recently imported module is the last one to be deleted.

The first loop iterates on sys.modules, not on reversed(sys.modules).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42671] Make the Python finalization more deterministic

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

Modules are already cleared in the reverse order.

Clearing module attributes in the reverse order looks interesting. It increases 
chance that modules and other imported names which can be used in __del__ will 
be deleted after executing __del__ if the module has references to an instance 
of a class defined in the same module. And if the class was defined in other 
module, it will be deleted later because dependencies are cleared after 
depended modules.

On other hand, private attributes are currently removed first, so we perhaps 
need to change it to make the strategy working.

--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42671] Make the Python finalization more deterministic

[STINNER Victor]

STINNER Victor  added the comment:

I wrote this issue to attempt to fix a leak in PR 23811 which converts the 
_thread extension module to the multiphase initialization API (PEP 489).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42651] Recursive traceback crashes Python Interpreter

[Steve Stagg]

Steve Stagg  added the comment:

Sorry, previous should have read "Fixes the abort"

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42671] Make the Python finalization more deterministic

[STINNER Victor]

Change by STINNER Victor :


--
keywords: +patch
pull_requests: +22688
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/23826

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42651] Recursive traceback crashes Python Interpreter

[Steve Stagg]

Steve Stagg  added the comment:

Confirmed that bpo-42500 fixes this traceback, in commit 4e7a69bdb6 

Parent commit:

user@obsidian ~/t/f/cpython (master)> git checkout  93a0ef7
Note: switching to '93a0ef7'.
HEAD is now at 93a0ef7647 Correct return type in 
Modules/_ssl.c::sslmodule_legacy (GH-23609)

user@obsidian ~/t/f/cpython ((93a0ef76…))> ../test.sh
+ make distclean
+ CFLAGS=-O0
+ ./configure --prefix=/home/user/prefix --exec-prefix=/home/user/prefix 
--cache-file=../config.cache --with-pydebug --without-ensurepip
+ CFLAGS=-O0
+ make -j8 install
/home/user/cpython/Lib/runpy.py:111: PendingDeprecationWarning: lib2to3 package 
is deprecated and may not be able to parse Python 3.10+
  __import__(pkg_name)
/home/user/cpython/Lib/runpy.py:111: PendingDeprecationWarning: lib2to3 package 
is deprecated and may not be able to parse Python 3.10+
  __import__(pkg_name)
++ realpath /home/user/prefix/include/python3.10d/
+ INCLUDE_DIR=/home/user/prefix/include/python3.10d
+ set +e
+ PYTHONHOME=/home/user/prefix
+ /home/user/prefix/bin/python3 ../test.py
NoneType: None
...
NoneType: None
Fatal Python error: _Py_CheckRecursiveCall: Cannot recover from stack overflow.
Python runtime state: initialized

Current thread 0x7f483d4bd740 (most recent call first):
  File "/home/user/prefix/lib/python3.10/traceback.py", line 165 in _some_str
  File "/home/user/prefix/lib/python3.10/traceback.py", line 524 in __init__
  File "/home/user/prefix/lib/python3.10/traceback.py", line 113 in 
print_exception
  File "/home/user/prefix/lib/python3.10/traceback.py", line 173 in print_exc
  File "/home/user/cpython/../test.py", line 4 in foo
  ...
../test.sh: line 33: 50081 Aborted (core dumped) 
PYTHONHOME=$PREFIX $EPREFIX/bin/python3 ../test.py
+ [[ 134 -eq 139 ]]
+ exit 0

After fix merged:

user@obsidian ~/t/f/cpython ((93a0ef76…)) [1]> git checkout 4e7a69bdb6
Previous HEAD position was 93a0ef7647 Correct return type in 
Modules/_ssl.c::sslmodule_legacy (GH-23609)
HEAD is now at 4e7a69bdb6 bpo-42500: Fix recursion in or after except (GH-23568)
user@obsidian ~/t/f/cpython ((4e7a69bd…))> ../test.sh
+ make distclean
+ CFLAGS=-O0
+ ./configure --prefix=/home/user/prefix --exec-prefix=/home/user/prefix 
--cache-file=../config.cache --with-pydebug --without-ensurepip
+ CFLAGS=-O0
+ make -j8 install
/home/user/cpython/Lib/runpy.py:111: PendingDeprecationWarning: lib2to3 package 
is deprecated and may not be able to parse Python 3.10+
  __import__(pkg_name)
/home/user/cpython/Lib/runpy.py:111: PendingDeprecationWarning: lib2to3 package 
is deprecated and may not be able to parse Python 3.10+
  __import__(pkg_name)
++ realpath /home/user/prefix/include/python3.10d/
+ INCLUDE_DIR=/home/user/prefix/include/python3.10d
+ set +e
+ PYTHONHOME=/home/user/prefix
+ /home/user/prefix/bin/python3 ../test.py
NoneType: None
...
NoneType: None
Traceback (most recent call last):
  File "/home/user/prefix/lib/python3.10/traceback.py", line 165, in _some_str
return str(value)
RecursionError: maximum recursion depth exceeded while calling a Python object

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/user/cpython/../test.py", line 7, in 
foo()
  File "/home/user/cpython/../test.py", line 5, in foo
foo() 
  File "/home/user/cpython/../test.py", line 5, in foo
foo() 
  File "/home/user/cpython/../test.py", line 5, in foo
foo() 
  [Previous line repeated 989 more times]
  File "/home/user/cpython/../test.py", line 4, in foo
traceback.print_exc()
  File "/home/user/prefix/lib/python3.10/traceback.py", line 173, in print_exc
print_exception(*sys.exc_info(), limit=limit, file=file, chain=chain)
  File "/home/user/prefix/lib/python3.10/traceback.py", line 113, in 
print_exception
for line in TracebackException(
  File "/home/user/prefix/lib/python3.10/traceback.py", line 632, in format
yield from self.format_exception_only()
  File "/home/user/prefix/lib/python3.10/traceback.py", line 580, in 
format_exception_only
yield _format_final_exc_line(stype, self._str)
  File "/home/user/prefix/lib/python3.10/traceback.py", line 156, in 
_format_final_exc_line
valuestr = _some_str(value)
  File "/home/user/prefix/lib/python3.10/traceback.py", line 167, in _some_str
return '' % type(value).__name__
RecursionError: maximum recursion depth exceeded while calling a Python object
+ [[ 1 -eq 139 ]]
+ exit 0

--
nosy: +stestagg

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42672] tkinter/__init__.py raises a NameError if NoDefaultRoot()

[Ivo Shipkaliev]

Change by Ivo Shipkaliev :


--
components: Tkinter
files: default_root.diff
keywords: patch
nosy: shippo_
priority: normal
severity: normal
status: open
title: tkinter/__init__.py raises a NameError if NoDefaultRoot()
type: behavior
versions: Python 3.10
Added file: https://bugs.python.org/file49690/default_root.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42660] _zoneinfo.c incorrectly checks bounds of `day` variable in calenderrule_new

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

PR 23825 refactors parsing. There is now a single function for parsing numbers. 
It saves around 70 lines of code and makes easy to implement support of full 
range of transition hours.

It could be possible to represent transition time as 32-bit offset in seconds 
in CalendarRule and DayRule instead of 16-bit hour, and 8-bit minute and 
second. In any case we need that value instead of separate values of hour, 
minute and second.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42671] Make the Python finalization more deterministic

[STINNER Victor]

New submission from STINNER Victor :

At exit, Python calls Py_Finalize() which tries to clear every single Python 
objects. The order is which Python objects are cleared is not fully 
deterministic. Py_Finalize() uses an heuristic to attempt to clear modules of 
sys.modules in the "best" order.

The current code creates a weak reference to a module, set sys.modules[name] to 
None, and then clears the module attribute if and only if the module object was 
not destroyed (if the weak reference still points to the module).

The problem is that even if a module object is destroyed, the module dictionary 
can remain alive thanks for various kinds of strong references to it.

Worst case example:
---
class VerboseDel:
def __del__(self):
print("Goodbye Cruel World")
obj = VerboseDel()

def func():
pass

import os
os.register_at_fork(after_in_child=func)
del os
del func

print("exit")
---

Output:
---
$ python3.9 script.py
exit
---

=> The VerboseDel object is never destroyed :-( BUG!


Explanation:

* os.register_at_fork(after_in_child=func) stores func in 
PyInterpreterState.after_forkers_child -> func() is kept alive until 
interpreter_clear() calls Py_CLEAR(interp->after_forkers_child);

* func() has reference to the module dictionary

I'm not sure why the VerboseDel object is not destroyed.


I propose to rewrite the finalize_modules() to clear modules in a more 
deterministic order:

* start by clearing __main__ module variables
* then iterate on reversed(sys.modules.values()) and clear the module variables
* Module attributes are cleared by _PyModule_ClearDict(): iterate on 
reversed(module.__dict__) and set dict values to None


Drawback: it is a backward incompatible change. Code which worked by luck 
previously no longer works. I'm talking about applications which rely on 
__del__() methods being calling in an exact order and expect Python being in a 
specific state.

Example:
---
class VerboseDel:
def __init__(self, name):
self.name = name
def __del__(self):
print(self.name)

a = VerboseDel("a")
b = VerboseDel("b")
c = VerboseDel("c")
---

Output:
---
c
b
a
---

=> Module attributes are deleted in the reverse order of their definition: the 
most recent object is deleted first, the oldest is deleted last.


Example 2 with 3 modules (4 files):
---
$ cat a.py 
class VerboseDel:
def __init__(self, name):
self.name = name
def __del__(self):
print(self.name)

a = VerboseDel("a")


$ cat b.py 
class VerboseDel:
def __init__(self, name):
self.name = name
def __del__(self):
print(self.name)

b = VerboseDel("b")


$ cat c.py 
class VerboseDel:
def __init__(self, name):
self.name = name
def __del__(self):
print(self.name)

c = VerboseDel("c")


$ cat z.py 
import a
import b
import c
---

Output:
---
$ ./python z.py 
c
b
a
---

=> Modules are deleted from the most recently imported (import c) to the least 
recently imported module (import a).

--
components: Interpreter Core
messages: 383265
nosy: vstinner
priority: normal
severity: normal
status: open
title: Make the Python finalization more deterministic
versions: Python 3.10

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42660] _zoneinfo.c incorrectly checks bounds of `day` variable in calenderrule_new

[Serhiy Storchaka]

Change by Serhiy Storchaka :


--
keywords: +patch
pull_requests: +22687
stage: needs patch -> patch review
pull_request: https://github.com/python/cpython/pull/23825

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42663] zoneinfo does not support full range of allowed transition hours in fallback string

[Serhiy Storchaka]

Change by Serhiy Storchaka :


--
keywords: +patch
nosy: +serhiy.storchaka
nosy_count: 1.0 -> 2.0
pull_requests: +22686
stage: needs patch -> patch review
pull_request: https://github.com/python/cpython/pull/23825

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue10496] Python startup should not require passwd entry

[Peixing Xin]

Change by Peixing Xin :


--
nosy: +pxinwr
nosy_count: 28.0 -> 29.0
pull_requests: +22685
pull_request: https://github.com/python/cpython/pull/23530

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42600] Cancelling tasks waiting for asyncio.Conditions crashes w/ RuntimeError: Lock is not acquired.

[Yury Selivanov]

Yury Selivanov  added the comment:

Hey Hynek! :) Can you submit a PR?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42670] Missing word in itertools.product

[miss-islington]

Change by miss-islington :


--
pull_requests: +22684
pull_request: https://github.com/python/cpython/pull/23824

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42670] Missing word in itertools.product

[miss-islington]

miss-islington  added the comment:


New changeset 074ad5123f18923bdb5fa0b6e4bf24de45e32ba9 by Zackery Spytz in 
branch 'master':
bpo-42670: Fix a missing word in the itertools.product() docs (GH-23823)
https://github.com/python/cpython/commit/074ad5123f18923bdb5fa0b6e4bf24de45e32ba9


--
nosy: +miss-islington

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42115] Caching infrastructure for the evaluation loop: specialised opcodes

[Yury Selivanov]

Yury Selivanov  added the comment:

> I will try to do some prototyping around that to see how much can we gain in 
> that route. In any case, adding LOAD_METHOD support for all kinds of calls 
> should be an improvement by itself even without caching, no?

Exactly.

As one argument for generalizing of LOAD_METHOD is that I, for example, try not 
to use kwargs in hot paths because I know it will be slower, which feels very 
wrong. I shouldn't care of internal implementation details of CPython and focus 
on writing readable code.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42115] Caching infrastructure for the evaluation loop: specialised opcodes

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

> IMO you need to implement LOAD_METHOD support for all kinds of calls, 
> including the ones that use kwargs, to see any improvement.

I will try to do some prototyping around that to see how much can we gain in 
that route. In any case, adding LOAD_METHOD support for all kinds of calls 
should be an improvement by itself even without caching, no?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42115] Caching infrastructure for the evaluation loop: specialised opcodes

[Yury Selivanov]

Yury Selivanov  added the comment:

> I think I am closing the PR as it seems that the gains are not good enough 
> (and there is quite a lot of noise by comparing the benchmarks together).

IMO you need to implement LOAD_METHOD support for all kinds of calls, including 
the ones that use kwargs, to see any improvement.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31904] Python should support VxWorks RTOS

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 75dabfe7a8324a16687959cc401deb72b104a575 by pxinwr in branch 
'master':
bpo-31904: posixpath.expanduser() handles None user home on VxWorks (GH-23530)
https://github.com/python/cpython/commit/75dabfe7a8324a16687959cc401deb72b104a575


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42667] shelve module is not thread-safe during accessing different databases from different threads

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

I added import experts because the issue looks relates to race condition in 
import.

Willy, can the issue be reproduced on more modern Python releases (3.8 or 3.9)? 
It could help if you provide minimal script which reproduces it.

--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42667] shelve module is not thread-safe during accessing different databases from different threads

[Brett Cannon]

Change by Brett Cannon :


--
nosy:  -brett.cannon

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42670] Missing word in itertools.product

[Zackery Spytz]

Change by Zackery Spytz :


--
keywords: +patch
nosy: +ZackerySpytz
nosy_count: 2.0 -> 3.0
pull_requests: +22683
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/23823

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42670] Missing word in itertools.product

[Scott Noyes]

New submission from Scott Noyes :

-Accordingly, it only useful with finite inputs.
+Accordingly, it is only useful with finite inputs.

--
assignee: docs@python
components: Documentation
messages: 383257
nosy: docs@python, snoyes
priority: normal
severity: normal
status: open
title: Missing word in itertools.product
type: behavior
versions: Python 3.10, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42529] CPython DLL initialization routine failed from PYC cache file

[Karl Nelson]

Karl Nelson  added the comment:

Well that gives me a place to search from.   Unfortunately statics are not 
likely the source of the issue.   I scrubbed all C++ structures from the 
project into a big structure which has only a single global pointer which I 
initialize in the init method.   It is possible that one crept back in at some 
point, but it is a possibility.   The use of actual C++ in this project is 
nearly zero.  Other than exceptions, strings and the vector on the buried in 
some class, the rest has been purged already.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42529] CPython DLL initialization routine failed from PYC cache file

[Steve Dower]

Steve Dower  added the comment:

You'll find it leads up to a LoadLibraryExW call in Python/dynload_win.c that 
is failing with ERROR_DLL_INIT_FAILED (1114) in GetLastError(). From that point 
on, it's looking solely at _jpype.pyd and its dependencies, and it could be any 
of them failing to load.

You can rule out anything that's already loaded at that stage, since it won't 
be calling the initialization routine. And anything in your module 
initialization would cause an error later than this, so it's clean too.

However, I think some static variables may be initialised at this point? So if 
you've got any statics, those might be working too hard (and statics in C++ can 
easily do too much work).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42529] CPython DLL initialization routine failed from PYC cache file

[Karl Nelson]

Karl Nelson  added the comment:

Without the pyc everything goes fine...

```
...
_bootlocale' # <_frozen_importlib_external.SourceFileLoader object at 
0x0222E9FD5A30>
import 'site' # <_frozen_importlib_external.SourceFileLoader object at 
0x0222E9F88F40>
Python 3.9.0 (tags/v3.9.0:9cf6752, Oct  5 2020, 15:34:40) [MSC v.1927 64 bit 
(AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
# code object from 
c:\users\nelson85\documents\devel\open\jpype\jpype\__init__.py
# created 
'c:\\users\\nelson85\\documents\\devel\\open\\jpype\\jpype\\__pycache__\\__init__.cpython-39.pyc'
# extension module '_jpype' loaded from 
'c:\\users\\nelson85\\documents\\devel\\open\\jpype\\_jpype.cp39-win_amd64.pyd'
# extension module '_jpype' executed from 
'c:\\users\\nelson85\\documents\\devel\\open\\jpype\\_jpype.cp39-win_amd64.pyd'
import '_jpype' # <_frozen_importlib_external.ExtensionFileLoader object at 
0x0222EA4C6250>
# 
c:\users\nelson85\documents\devel\open\jpype\jpype\__pycache__\_jinit.cpython-39.pyc
 matches c:\users\nelson85\documents\devel\open\jpype\jpype\_jinit.py
# code o
...
```

With the pyc file, failure... 

```
...
_bootlocale' # <_frozen_importlib_external.SourceFileLoader object at 
0x015278445A30>
import 'site' # <_frozen_importlib_external.SourceFileLoader object at 
0x0152783F8F40>
Python 3.9.0 (tags/v3.9.0:9cf6752, Oct  5 2020, 15:34:40) [MSC v.1927 64 bit 
(AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
# 
c:\users\nelson85\documents\devel\open\jpype\jpype\__pycache__\__init__.cpython-39.pyc
 matches c:\users\nelson85\documents\devel\open\jpype\jpype\__init__.py
# code object from 
'c:\\users\\nelson85\\documents\\devel\\open\\jpype\\jpype\\__pycache__\\__init__.cpython-39.pyc'
Traceback (most recent call last):
  File "C:\Users\nelson85\Documents\test.py", line 1, in 
import jpype
  File "", line 1007, in _find_and_load
  File "", line 986, in _find_and_load_unlocked
  File "", line 680, in _load_unlocked
  File "", line 790, in exec_module
  File "", line 228, in _call_with_frames_removed
  File "c:\users\nelson85\documents\devel\open\jpype\jpype\__init__.py", line 
18, in 
import _jpype
  File "", line 1007, in _find_and_load
  File "", line 986, in _find_and_load_unlocked
  File "", line 666, in _load_unlocked
  File "", line 565, in module_from_spec
  File "", line 1108, in create_module
  File "", line 228, in _call_with_frames_removed
ImportError: DLL load failed while importing _jpype: A dynamic link library 
(DLL) initialization routine failed.
# clear builtins._
# clear sys.path
# clear sys.argv
# clear sys.ps1
# clear sys.ps2
# clear sys.last_type
# clear sys.last_value
# clear sys.last_traceback
# destroy jpype
# clear sys.path_hooks
# clear sys.path_importer_cache
# clear sys.meta_path
# clear sys.__interactivehook__
# restore sys.stdin
# restore sys.stdout
# restore sys.stderr
...
```

I see the codecs all get loaded earlier in the trace, so that can't be the 
issue.  There were no differences (other than memory addresses) up to this 
point.  So it must be going wrong at bootstrap_external:1108.

So guessing from the message we failed in.

```
def exec_module(self, module):
"""Initialize an extension module"""
_bootstrap._call_with_frames_removed(_imp.exec_dynamic, module)
_bootstrap._verbose_message('extension module {!r} executed from {!r}',
 self.name, self.path)
```

After that point it gets really fuzzy.  I would guess we are going to 
import.c#_imp_create_dynamic_impl which takes us to 
_PyImport_LoadDynamicModuleWithSpec.  But I will need some time to read code to 
see if this guess is correct.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42651] Recursive traceback crashes Python Interpreter

[Xinmeng Xia]

Xinmeng Xia  added the comment:

In issue#36272, they discuss a bug in logging module. They show an example 
which is similar as this one. So I think I think there might be a similar bug 
in traceback module.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42652] recursive in finally clause cause Python interpreter crash.

[Xinmeng Xia]

Xinmeng Xia  added the comment:

In issue #42500, crashes is resulted by recursion and try-except. Program like 
following will crash the interpreter.
=
def foo():
try:
1/0
except:
pass
foo()
foo()
=
However with traceback module, program will no longer crash the interpreter. A 
recursive Error is returned as expected. 
=
import traceback
def foo():
try:
1/0
except:
traceback.print_exc()
foo()
foo()
=
But it is still crash the interpreter in finally clause. I think this might be 
a new but and it is different from #42500. It should be related to traceback 
module, finally clause and recursion. what do you think?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42529] CPython DLL initialization routine failed from PYC cache file

[Karl Nelson]

Karl Nelson  added the comment:

Thanks, I will see if I can get additional diagnostics today.  Otherwise I will 
have to start recompiling Python with diagnostic hooks.  I have never had to 
rebuild python on windows so it may take a while to figure out how to make 
progress.

The monkey patch is less scary than it looks.   Since I only need these slots 
of specific basic types that I designate (not user selected), the extra types 
all extend without altering the underlaying type, and the class layout is 
controlled by a meta class so that only Java can create these type objects, it 
is reasonably robust.   However, yes changes in Python can break it.  It is 
basically an attempt to create a mixin for concrete classes without the benefit 
of a dictoffset type slot.  It took about a month do develop something that 
safely added the slot without overrunning, leaking or creating other memory 
issues (and played nice when Python added its own dict slot). 

Unfortunately, as I do many 100s on JSlot tests during method resolution using 
a dict based approach was unacceptably slow.  I would love if there was a 
formal method to create multiple inheritance mixins for Python basic types, but 
that is another issue.


```
/**
 * Allocate a new Python object with a slot for Java.
 *
 * We need extra space to store our values, but because there
 * is no way to do so without disturbing the object layout.
 * Fortunately, Python already handles this for dict and weakref.
 * Python aligns the ends of the structure and increases the
 * base type size to add additional slots to a standard object.
 *
 * We will use the same trick to add an additional slot for Java
 * after the end of the object outside of where Python is looking.
 * As the memory is aligned this is safe to do.  We will use
 * the alloc and finalize slot to recognize which objects have this
 * extra slot appended.
 */
PyObject* PyJPValue_alloc(PyTypeObject* type, Py_ssize_t nitems )
{
// Modification from Python to add size elements
const size_t size = _PyObject_VAR_SIZE(type, nitems + 1) + sizeof 
(JPValue);
PyObject *obj = (PyType_IS_GC(type)) ? _PyObject_GC_Malloc(size)
: (PyObject *) PyObject_MALLOC(size);
if (obj == NULL)
return PyErr_NoMemory(); // GCOVR_EXCL_LINE
memset(obj, 0, size);

Py_ssize_t refcnt = ((PyObject*) type)->ob_refcnt;
if (type->tp_itemsize == 0)
PyObject_Init(obj, type);
else
PyObject_InitVar((PyVarObject *) obj, type, nitems);

// This line is required to deal with Python bug (GH-11661)
// Some versions of Python fail to increment the reference counter of
// heap types properly.
if (refcnt == ((PyObject*) type)->ob_refcnt)
Py_INCREF(type);  // GCOVR_EXCL_LINE

if (PyType_IS_GC(type))
{
PyObject_GC_Track(obj);
}
return obj;
}

bool PyJPValue_hasJavaSlot(PyTypeObject* type)
{
if (type == NULL
|| type->tp_alloc != (allocfunc) PyJPValue_alloc
|| type->tp_finalize != (destructor) PyJPValue_finalize)
return false;  // GCOVR_EXCL_LINE
return true;
}

// Check for a Java slot (must work on all object types)
Py_ssize_t PyJPValue_getJavaSlotOffset(PyObject* self)
{
PyTypeObject *type = Py_TYPE(self);
if (type == NULL
|| type->tp_alloc != (allocfunc) PyJPValue_alloc
|| type->tp_finalize != (destructor) PyJPValue_finalize)
return 0;
Py_ssize_t offset;
Py_ssize_t sz = Py_SIZE(self);
// I have no clue what negative sizes mean
if (sz < 0)
sz = -sz;
if (type->tp_itemsize == 0)
offset = _PyObject_VAR_SIZE(type, 1);
else
offset = _PyObject_VAR_SIZE(type, sz + 1);
return offset;
}

JPValue* PyJPValue_getJavaSlot(PyObject* self)
{
Py_ssize_t offset = PyJPValue_getJavaSlotOffset(self);
if (offset == 0)
return NULL;
JPValue* value = (JPValue*) (((char*) self) + offset);
if (value->getClass() == NULL)
return NULL;
return value;
}

void PyJPValue_free(void* obj)
{
// Normally finalize is not run on simple classes.
PyTypeObject *type = Py_TYPE(obj);
if (type->tp_finalize != NULL)
type->tp_finalize((PyObject*) obj);
if (type->tp_flags & Py_TPFLAGS_HAVE_GC)
PyObject_GC_Del(obj);
else
PyObject_Free(obj);  // GCOVR_EXCL_LINE
}

void PyJPValue_finalize(void* obj)
{
JPValue* value = PyJPValue_getJavaSlot((PyObject*) obj);
... destroy the objects hidden slot if it exists
}

```

--

___
Python tracker 

_

[issue42652] recursive in finally clause cause Python interpreter crash.

[Ronald Oussoren]

Ronald Oussoren  added the comment:

This might be related to #42500

--
nosy: +ronaldoussoren

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42651] Recursive traceback crashes Python Interpreter

[Ronald Oussoren]

Ronald Oussoren  added the comment:

This might be related to #42500

--
nosy: +ronaldoussoren

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42668] re.escape does not correctly escape newlines

[Serhiy Storchaka]

Change by Serhiy Storchaka :


--
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42613] freeze.py doesn't support sys.platlibdir different than lib nor multiarch

[STINNER Victor]

Change by STINNER Victor :


--
versions: +Python 3.8, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42613] freeze.py doesn't support sys.platlibdir different than lib nor multiarch

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 79c535796da4a1377364910e309b97e53c3e51ef by Miss Islington (bot) 
in branch '3.9':
bpo-42613: Fix freeze.py config directory (GH-23792) (GH-23817)
https://github.com/python/cpython/commit/79c535796da4a1377364910e309b97e53c3e51ef


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42668] re.escape does not correctly escape newlines

[Martin Altmayer]

Martin Altmayer  added the comment:

Thanks for the explanation, I did not know re.VERBOSE. I still think the 
behavior is a bit confusing, but it's probably not worth the effort to change 
this.

--
type: behavior -> enhancement

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26564] Malloc debug hooks: display memory block traceback on error

[STINNER Victor]

STINNER Victor  added the comment:

> I am wondering why 53/22? 5/2 is as good for 8-bytes long -- both gives 
> 21-bytes buffer.

53/22 is closer to math.log(256)/math.log(10) than 5/2, and 8*53 should not 
overflow :-)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42645] break/continue or return in finally block occurs twice in trace.

[Mark Shannon]

Change by Mark Shannon :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42246] Implement PEP 626 -- Precise line numbers for debugging

[Mark Shannon]

Mark Shannon  added the comment:


New changeset bf353f3c2d937772a8cf30b15fd8eb7b82665ccb by Mark Shannon in 
branch 'master':
bpo-42246: Make sure that `f_lasti`, and thus `f_lineno`, is set correctly 
after raising or reraising an exception (GH-23803)
https://github.com/python/cpython/commit/bf353f3c2d937772a8cf30b15fd8eb7b82665ccb


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26564] Malloc debug hooks: display memory block traceback on error

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

I am wondering why 53/22? 5/2 is as good for 8-bytes long -- both gives 
21-bytes buffer.

I do not propose to change anything. It is just curiosity.

--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42669] "except" documentation still suggests nested tuples are allowed

[Colin Watson]

Change by Colin Watson :


--
keywords: +patch
pull_requests: +22682
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/23822

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42669] "except" documentation still suggests nested tuples are allowed

[Colin Watson]

New submission from Colin Watson :

In Python 2, it was possible to use `except` with a nested tuple, and 
occasionally natural.  For example, `zope.formlib.interfaces.InputErrors` is a 
tuple of several exception classes, and one might reasonably think to do 
something like this (this is real code used in several places in 
https://git.launchpad.net/launchpad):

try:
self.getInputValue()
return True
except (InputErrors, SomethingElse):
return False

As of Python 3.0, this raises "TypeError: catching classes that do not inherit 
from BaseException is not allowed" instead: one must instead either break it up 
into multiple "except" clauses or flatten the tuple.  The change was mentioned 
in https://bugs.python.org/issue2380 and seems to have been intentional: I'm 
not requesting that the previous behaviour be restored, since it's a fairly 
rare porting issue and by now well-established in Python 3.

However, the relevant sentences of documentation in 
https://docs.python.org/2/reference/compound_stmts.html#try and 
https://docs.python.org/3/reference/compound_stmts.html#the-try-statement are 
identical aside from punctuation, and they both read:

 For an except clause with an expression, that expression is evaluated, and 
the clause matches the exception if the resulting object is “compatible” with 
the exception.  An object is compatible with an exception if it is the class or 
a base class of the exception object or a tuple containing an item compatible 
with the exception.

I think this admits a recursive reading: I certainly read it that way.  It 
should make it clear that nested tuples are not allowed.

--
assignee: docs@python
components: Documentation
messages: 383243
nosy: cjwatson, docs@python
priority: normal
severity: normal
status: open
title: "except" documentation still suggests nested tuples are allowed
versions: Python 3.10

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26564] Malloc debug hooks: display memory block traceback on error

[Irit Katriel]

Change by Irit Katriel :


--
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26564] Malloc debug hooks: display memory block traceback on error

[miss-islington]

miss-islington  added the comment:


New changeset 1c70d40e94741578ce28b8851fb65372ac2e7942 by Miss Islington (bot) 
in branch '3.9':
bpo-26564: fix obsolete comment in traceback.c (GH-23819)
https://github.com/python/cpython/commit/1c70d40e94741578ce28b8851fb65372ac2e7942


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26564] Malloc debug hooks: display memory block traceback on error

[miss-islington]

miss-islington  added the comment:


New changeset cecbaa3a80d5ae28cdd4129d6d2c4395c12a89e4 by Miss Islington (bot) 
in branch '3.8':
bpo-26564: fix obsolete comment in traceback.c (GH-23819)
https://github.com/python/cpython/commit/cecbaa3a80d5ae28cdd4129d6d2c4395c12a89e4


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42668] re.escape does not correctly escape newlines

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

> If the escape-result still contains a verbatim newline, why escape this 
> character at all?

Because in verbose mode it is ignored. This is why \n and other ASCII 
whitespace characters, and '#', which starts a comment, should be escaped.

'\\\n' and '\\n' has identical meaning independent of mode and context, so 
there is no bug here. Escaping '\n' as '\\\n' is slightly simpler.

--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42668] re.escape does not correctly escape newlines

[Matthew Barnett]

Matthew Barnett  added the comment:

In a regex, putting a backslash before any character that's not an ASCII-range 
letter or digit makes it a literal. re.escape doesn't special-case control 
characters. Its purpose is to make a string that might contain metacharacters 
into one that's a literal, and it already does that, although it sometimes 
escapes more than necessary.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26564] Malloc debug hooks: display memory block traceback on error

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 40125ab3252453bf205ed906e46bf9741c27bf9d by Irit Katriel in 
branch 'master':
bpo-26564: fix obsolete comment in traceback.c (GH-23819)
https://github.com/python/cpython/commit/40125ab3252453bf205ed906e46bf9741c27bf9d


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26564] Malloc debug hooks: display memory block traceback on error

[miss-islington]

Change by miss-islington :


--
pull_requests: +22681
pull_request: https://github.com/python/cpython/pull/23821

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26564] Malloc debug hooks: display memory block traceback on error

[miss-islington]

Change by miss-islington :


--
nosy: +miss-islington
nosy_count: 4.0 -> 5.0
pull_requests: +22680
pull_request: https://github.com/python/cpython/pull/23820

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42667] shelve module is not thread-safe during accessing different databases from different threads

[Serhiy Storchaka]

Change by Serhiy Storchaka :


--
nosy: +brett.cannon, eric.snow, ncoghlan

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42668] re.escape does not correctly escape newlines

[Martin Altmayer]

New submission from Martin Altmayer :

re.escape('\n') returns '\\\n', i.e. a string consisting of a backslash and a 
newline. I believe it should return '\\n', i.e. a backslash and an 'n'. If the 
escape-result still contains a verbatim newline, why escape this character at 
all?

Note that Python's regular expressions engine allows newlines, so 
re.match(re.escape('\n'), '\n') gives a match. Thus, while this looks like an 
undesired behavior, it is not functionally broken.

The same problem applies to some other characters: \t\r\v\f

--
components: Regular Expressions
files: test.py
messages: 383237
nosy: MartinAltmayer, ezio.melotti, mrabarnett
priority: normal
severity: normal
status: open
title: re.escape does not correctly escape newlines
type: behavior
versions: Python 3.10, Python 3.7, Python 3.8, Python 3.9
Added file: https://bugs.python.org/file49689/test.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26564] Malloc debug hooks: display memory block traceback on error

[Irit Katriel]

Change by Irit Katriel :


--
pull_requests: +22679
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/23819

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26564] Malloc debug hooks: display memory block traceback on error

[Irit Katriel]

Irit Katriel  added the comment:

I'm reopening this to commit a fix for a comment which became obsolete 
following this change: https://hg.python.org/cpython/rev/18a19e62bac5.

--
nosy: +iritkatriel
status: closed -> open

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42613] freeze.py doesn't support sys.platlibdir different than lib nor multiarch

[miss-islington]

miss-islington  added the comment:


New changeset 829272e67bbd4b2cc76c01cd20265eb114b392a2 by Miss Islington (bot) 
in branch '3.8':
bpo-42613: Fix freeze.py config directory (GH-23792)
https://github.com/python/cpython/commit/829272e67bbd4b2cc76c01cd20265eb114b392a2


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42529] CPython DLL initialization routine failed from PYC cache file

[Steve Dower]

Steve Dower  added the comment:

Monkeypatching native types looks suspect as anything, but probably not the 
cause here. Good luck :)

So I checked some of our code around dynamic loads and it's got to be caused by 
the loader in our LoadLibrary call, not when we call your module init function. 
That rules out basically anything you're doing once the module is loaded.

The only thing I can think of that may be missing in the .pyc case is 
encodings. They *should* be being imported automatically on load, but perhaps 
not as we're constantly trying to improve startup time. You could try adding 
"import codecs" and then codecs.lookup some codecs.

You might also try running Python in verbose mode ("python -v ...") and 
comparing the output between the two cases. That could show up what extra 
things are being skipped in the failing case.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42613] freeze.py doesn't support sys.platlibdir different than lib nor multiarch

[miss-islington]

Change by miss-islington :


--
nosy: +miss-islington
nosy_count: 2.0 -> 3.0
pull_requests: +22677
pull_request: https://github.com/python/cpython/pull/23817

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42613] freeze.py doesn't support sys.platlibdir different than lib nor multiarch

[miss-islington]

Change by miss-islington :


--
pull_requests: +22678
pull_request: https://github.com/python/cpython/pull/23818

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42375] subprocess DragonFlyBSD build update

[STINNER Victor]

Change by STINNER Victor :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed
versions: +Python 3.8, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42375] subprocess DragonFlyBSD build update

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 4f65907f38339809532bc06ffc1382c2c09180e2 by Miss Islington (bot) 
in branch '3.8':
bpo-42375: subprocess DragonFlyBSD build update. (GH-23320) (GH-23389)
https://github.com/python/cpython/commit/4f65907f38339809532bc06ffc1382c2c09180e2


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42375] subprocess DragonFlyBSD build update

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 8d0a01c99b2a810bfe13d654b294c25a10151aa4 by Miss Islington (bot) 
in branch '3.9':
bpo-42375: subprocess DragonFlyBSD build update. (GH-23320) (GH-23388)
https://github.com/python/cpython/commit/8d0a01c99b2a810bfe13d654b294c25a10151aa4


--
nosy: +vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41804] test_epoll fails test_control_and_wait() randomly on aarch64 RHEL8 Refleaks 3.9

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 718bf1a7a1a39ca6f2381a299d00d8318732104a by Miss Islington (bot) 
in branch '3.8':
bpo-41804: Enhance test_epoll.test_control_and_wait() (GH-23795) (GH-23814)
https://github.com/python/cpython/commit/718bf1a7a1a39ca6f2381a299d00d8318732104a


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42625] Segmentation fault of PyState_AddModule()

[hai shi]

hai shi  added the comment:

Thanks victor, petr for your comment and review. I closed this bpo and PR.

I paste victor's explanation in here in case other developers have the same 
question too: Getting a crash is the
expected behaviour if you don't respect a function API. Most C
functions are designed like that. Only very few functions check their
arguments, tests are usually omitted for best performances.

Sorry for this noisy bpo.

--
resolution:  -> rejected
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42667] shelve module is not thread-safe during accessing different databases from different threads

[gardener.willy]

New submission from gardener.willy :

Shelve module uses "import dbm" instruction while opening database. Dbm module 
has global dictionary "_modules". This dictionary modifies during database 
opening operation. When different threads simultaneously try to open different 
databases, unexpected behavior occurred. In my case I've got such message: 

with shelve.open('some_file') as f:
  File "/usr/local/lib/python3.6/shelve.py", line 243, in open
return DbfilenameShelf(filename, flag, protocol, writeback)
  File "/usr/local/lib/python3.6/shelve.py", line 227, in init
Shelf.init(self, dbm.open(filename, flag), protocol, writeback)
  File "/usr/local/lib/python3.6/dbm/__init__.py", line 94, in open
return mod.open(file, flag, mode)
AttributeError: module 'dbm.ndbm' has no attribute 'open'

Behavior is the same on python 3.6 and 3.7. Error is spontaneous.

--
components: Library (Lib)
messages: 383229
nosy: gardener.willy
priority: normal
severity: normal
status: open
title: shelve module is not thread-safe during accessing different databases 
from different threads
type: behavior
versions: Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23522] Misleading note in Statistics module documentation

[Irit Katriel]

Irit Katriel  added the comment:

I agree with Jake's comment, but I think the solution is to remove that Note 
altogether. This document is a software manual, not a statistics textbook, and 
as such it should just state clearly what the statistics module does. If 
someone doesn't know whether they need the mean or the median, they really need 
to read a more fundamental text before writing their code.

--
nosy: +iritkatriel

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42665] Should PyLong_AsLongAndOverflow raise exception on overflow?

[Zackery Spytz]

Zackery Spytz  added the comment:

> No, it should not.

I agree.

--
nosy: +ZackerySpytz

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31904] Python should support VxWorks RTOS

[Andrew Svetlov]

Andrew Svetlov  added the comment:


New changeset ba760f3710eccdfae2b680a5f94fe0160ddb1536 by pxinwr in branch 
'master':
bpo-31904: Skip some asyncio tests on VxWorks (#23815)
https://github.com/python/cpython/commit/ba760f3710eccdfae2b680a5f94fe0160ddb1536


--
nosy: +asvetlov

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42665] Should PyLong_AsLongAndOverflow raise exception on overflow?

[Serhiy Storchaka]

New submission from Serhiy Storchaka :

No, it should not.

--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42666] getting a class source regresses in Python 3.9

[Thomas Viehmann]

Thomas Viehmann  added the comment:

I might add that this is a case I hit in the PyTorch test suite.

I tried to fix this a long time ago in a pricipled way in
https://bugs.python.org/issue33826

and Guido explained that fixing it was too heavy handed to be worthwile in

https://mail.python.org/archives/list/python-...@python.org/thread/AC5B5VORS6FF6KJ3EG5DIV3ZLAQB6DIY/#UL376F3Y6FKJUZ2HZDA3ERFUDDOX67X4

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42666] getting a class source regresses in Python 3.9

[Thomas Viehmann]

New submission from Thomas Viehmann :

getting a class source regresses in Python 3.9 onwards.
The following worked in Python 3.8, now it doesn't anymore for 3.9.1 and 
3.10.0a2:

(save as foo.py)

import inspect

class Foo:
def spam(self):
global Bar
class Bar:
pass
print(inspect.getsource(Bar))

Foo().spam()


It seems that getsource is very brittle for classes.

--
components: Library (Lib)
messages: 383223
nosy: t-vi
priority: normal
severity: normal
status: open
title: getting a class source regresses in Python 3.9
type: behavior
versions: Python 3.10, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42665] Should PyLong_AsLongAndOverflow raise exception on overflow?

[Ganesh Kathiresan]

Change by Ganesh Kathiresan :


--
components: C API
nosy: ganesh3597
priority: normal
severity: normal
status: open
title: Should PyLong_AsLongAndOverflow raise exception on overflow?
type: behavior
versions: Python 3.10

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22228] Adapt bash readline operate-and-get-next function

[Lele Gaifax]

Lele Gaifax  added the comment:

Apparently, now the operate-and-get-next function has been moved to the 
readline library, so hopefully it will be easier to get its advantage.

See point 2.j in https://lwn.net/Articles/839213/

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41804] test_epoll fails test_control_and_wait() randomly on aarch64 RHEL8 Refleaks 3.9

[miss-islington]

miss-islington  added the comment:


New changeset e962e3ad225a211b9f9689742db6e9771d07c505 by Miss Islington (bot) 
in branch '3.9':
bpo-41804: Enhance test_epoll.test_control_and_wait() (GH-23795)
https://github.com/python/cpython/commit/e962e3ad225a211b9f9689742db6e9771d07c505


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com