[issue40052] Incorrect pointer alignment in _PyVectorcall_Function() of cpython/abstract.h

[Andreas Schneider]

Andreas Schneider  added the comment:

I forgot, for detecting alignment issues or strict aliasing and this also falls 
under strict aliasing, you need to turn on optimizations.

clang -O2 -Werror -Wcast-align ...

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39943] Meta: Clean up various issues in C internals

[Andy Lester]

Change by Andy Lester :


--
pull_requests: +18513
pull_request: https://github.com/python/cpython/pull/19152

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40031] Python Configure IDLE 'Ok' and 'Apply' buttons do not seem to work.

[Terry J. Reedy]

Terry J. Reedy  added the comment:

Once you touch the idlelib/*.def files or manually edit the .idlerc/*.cfg 
files, you are on your own.  You could rename any *.def files and try to 
'repair' the installation, or delete the *.cfg files and rebuild then through 
the dialog.  

You could also start IDLE from Command Prompt with 'py -m idlelib' and report 
any error messages.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1635741] Py_Finalize() doesn't clear all Python objects at exit

[Paulo Henrique Silva]

Paulo Henrique Silva  added the comment:

Thanks for the clarifications. I will keep looking for simple modules, no state 
and easy to migrate but also dedicate more time to work on the more complex 
like datetime. I'm working on PR19122 corrections.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40039] [CVE-2020-10796] Python multiprocessing Remote Code Execution vulnerability

[Junyu Zhang]

Junyu Zhang  added the comment:

Thank you for your reply. Yes, under normal circumstances, keys are generally 
not leaked. I may have only considered the following attacks at the time:
1. If the client script of the distributed process is on another machine, or 
the key is leaked due to accidental leak.
2. When the attacker has obtained some server permissions, but not the highest 
permissions, and this distributed service process runs with the highest 
management permissions, and the attacker has read permissions to the script 
code, this may cause a Simple elevation.

Of course, after thinking about it carefully, I found that the above problem is 
just a conjecture, so now I have decided to give up reporting it as CVE, unless 
I find such a situation.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1635741] Py_Finalize() doesn't clear all Python objects at exit

[STINNER Victor]

STINNER Victor  added the comment:

Hum, some clarification is needed here.

"Port xxx extension module to multiphase initialization (PEP 489)" changes are 
helping to fix "Py_Finalize() doesn't clear all Python objects at exit", but 
alone they don't fix all issues.

--

For example, if a module still uses globals using "static ..." in C, these 
globals will not be cleared magically. Example with _datetimemodule.c:

static PyObject *us_per_hour = NULL;/* 1e6 * 3600 as Python int */
static PyObject *us_per_day = NULL; /* 1e6 * 3600 * 24 as Python int */
static PyObject *us_per_week = NULL;/* 1e6*3600*24*7 as Python int */

These variables initialized once in PyInit__datetime():

us_per_hour = PyLong_FromDouble(36.0);
us_per_day = PyLong_FromDouble(864.0);
us_per_week = PyLong_FromDouble(6048.0);

Converting the module to multiphase initialization will not magically clear 
these variables at exit. The _datetime module should be modified to store these 
variables in a module state: this module could be cleared at exit.

The binascii is a good example: it has a module state, traverse, clear and free 
methods, and it uses the multiphase initialization. This module can be fully 
unloaded at exit.

It's a "simple" module: it doesn't define types for example.

--

Another issue is that converting a module to the multiphase initialization 
doesn't magically fully isolate two instances of the module. For exmaple, the 
_abc module still uses a type defined statically:

static PyTypeObject _abc_data_type = {
PyVarObject_HEAD_INIT(NULL, 0)
"_abc_data",/*tp_name*/
sizeof(_abc_data),  /*tp_basicsize*/
.tp_dealloc = (destructor)abc_data_dealloc,
.tp_flags = Py_TPFLAGS_DEFAULT,
.tp_alloc = PyType_GenericAlloc,
.tp_new = abc_data_new,
};

Example:

vstinner@apu$ ./python
Python 3.9.0a5+ (heads/pr/19122:0ac3031a80, Mar 25 2020, 02:25:19) 
>>> import _abc
>>> class Bla: pass
... 
>>> _abc._abc_init(Bla)
>>> type(Bla._abc_impl)


# load a second instance of the module
>>> import sys; del sys.modules['_abc']
>>> import _abc as _abc2
>>> class Bla2: pass
... 
>>> _abc._abc_init(Bla2)

>>> type(Bla2._abc_impl)


# _abc and _abc2 have exactly the same type,
# they are not fully isolated
>>> type(Bla2._abc_impl) is type(Bla._abc_impl)
True


That's more an issue for subinterpreters: each interpreter should have its own 
fully isolated instance of an C extension module.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1635741] Py_Finalize() doesn't clear all Python objects at exit

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 7dd549eb08939e1927fba818116f5202e76f8d73 by Paulo Henrique Silva 
in branch 'master':
bpo-1635741: Port _functools module to multiphase initialization (PEP 489) 
(GH-19151)
https://github.com/python/cpython/commit/7dd549eb08939e1927fba818116f5202e76f8d73


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1635741] Py_Finalize() doesn't clear all Python objects at exit

[STINNER Victor]

STINNER Victor  added the comment:


New changeset f3d5ac47720045a72f7ef5af13046d9531e6007b by Paulo Henrique Silva 
in branch 'master':
bpo-1635741: Port operator module to multiphase initialization (PEP 489) 
(GH-19150)
https://github.com/python/cpython/commit/f3d5ac47720045a72f7ef5af13046d9531e6007b


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40058] Running test_datetime twice fails with: module 'datetime' has no attribute '_divide_and_round'

[STINNER Victor]

New submission from STINNER Victor :

vstinner@apu$ ./python -m test -v test_datetime test_datetime -m 
test_divide_and_round
== CPython 3.9.0a5+ (heads/pr/19122:0ac3031a80, Mar 25 2020, 02:25:19) [GCC 
9.2.1 20190827 (Red Hat 9.2.1-1)]
== Linux-5.5.9-200.fc31.x86_64-x86_64-with-glibc2.30 little-endian
== cwd: /home/vstinner/python/master/build/test_python_233006
== CPU count: 8
== encodings: locale=UTF-8, FS=utf-8
0:00:00 load avg: 0.82 Run tests sequentially
0:00:00 load avg: 0.82 [1/2] test_datetime
test_divide_and_round (test.datetimetester.TestModule_Pure) ... ok
test_divide_and_round (test.datetimetester.TestModule_Fast) ... skipped 'Only 
run for Pure Python implementation'

--

Ran 2 tests in 0.002s

OK (skipped=1)
0:00:00 load avg: 0.82 [2/2] test_datetime
test_divide_and_round (test.datetimetester.TestModule_Pure) ... ERROR
test_divide_and_round (test.datetimetester.TestModule_Fast) ... skipped 'Only 
run for Pure Python implementation'

==
ERROR: test_divide_and_round (test.datetimetester.TestModule_Pure)
--
Traceback (most recent call last):
  File "/home/vstinner/python/master/Lib/test/datetimetester.py", line 87, in 
test_divide_and_round
dar = datetime_module._divide_and_round
AttributeError: module 'datetime' has no attribute '_divide_and_round'

--

Ran 2 tests in 0.006s

FAILED (errors=1, skipped=1)
test test_datetime failed
test_datetime failed

== Tests result: FAILURE ==

1 test OK.

1 test failed:
test_datetime

Total duration: 448 ms
Tests result: FAILURE

--
components: Tests
messages: 364970
nosy: vstinner
priority: normal
severity: normal
status: open
title: Running test_datetime twice fails with: module 'datetime' has no 
attribute '_divide_and_round'
versions: Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36144] Dictionary union. (PEP 584)

[Guido van Rossum]

Guido van Rossum  added the comment:


New changeset 8f1ed21ecf57cc8b8095d9d1058af2b9b3ed0413 by Curtis Bucher in 
branch 'master':
bpo-36144: Add union operators to WeakValueDictionary584 (#19127)
https://github.com/python/cpython/commit/8f1ed21ecf57cc8b8095d9d1058af2b9b3ed0413


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1635741] Py_Finalize() doesn't clear all Python objects at exit

[Paulo Henrique Silva]

Paulo Henrique Silva  added the comment:

Updating on my findings on msg364833.

It looks like encodings module is not being destoyed at all and keeping all the 
encoding refs alive. Looks like some cycle but I am not sure yet how to solve 
it.

To validate this, I:
 - removed codec_search_cach of PyInterpreterState.
 - Py_DECREFd(encodings) after loading it on codecs.c.

Before: 4376 refs left (37fcbb65d4)
After :  352 refs left (-92%)

I've updated the changes at 
https://github.com/python/cpython/compare/master...phsilva:remove-codec-caches 
(not a proposed patch, just to validate the idea)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1635741] Py_Finalize() doesn't clear all Python objects at exit

[Paulo Henrique Silva]

Change by Paulo Henrique Silva :


--
pull_requests: +18512
pull_request: https://github.com/python/cpython/pull/19151

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1635741] Py_Finalize() doesn't clear all Python objects at exit

[Paulo Henrique Silva]

Change by Paulo Henrique Silva :


--
pull_requests: +18511
pull_request: https://github.com/python/cpython/pull/19150

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40025] enum: _generate_next_value_ is not called if its definition occurs after calls to auto()

[Jonathan Hsu]

Jonathan Hsu  added the comment:

Thank you for the explanation.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39812] Avoid daemon threads in concurrent.futures

[Kyle Stanley]

Change by Kyle Stanley :


--
keywords: +patch
pull_requests: +18510
stage: needs patch -> patch review
pull_request: https://github.com/python/cpython/pull/19149

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40039] [CVE-2020-10796] Python multiprocessing Remote Code Execution vulnerability

[STINNER Victor]

STINNER Victor  added the comment:

> if authkey is not set or leaked, it will cause RCE on the server side

In which situation the authkey can be empty?

Lib/mulitprocessing/process.py creates an authkey of 256 bits of entropy using:
AuthenticationString(os.urandom(32))

It's used by default if I understand correctly. I understand that the authkey 
can only be empty if the developer explicitly pass an empty string to authkey 
when the manager is created. Am I right?

--

About leaking the authkey: I don't know how the authkey is transfered to the 
child processes. Through a pipe controlled by the parent process?

--

> it will cause RCE on the server side

I read somewhere that multiprocessing is now supposed to accept other 
serialization protocol than pickle, but I failed to find the documentation :-( 
pickle remains the default.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40039] [CVE-2020-10796] Python multiprocessing Remote Code Execution vulnerability

[STINNER Victor]

Change by STINNER Victor :


Added file: https://bugs.python.org/file48999/poc.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40039] [CVE-2020-10796] Python multiprocessing Remote Code Execution vulnerability

[STINNER Victor]

STINNER Victor  added the comment:

Oh, I missed that the PDF contains a link to a PoC:
https://github.com/RGDZ-GZU/Python-Remote-code-exec/tree/master/poc

I attach a copy to this issue: server.py and poc.py.

--
Added file: https://bugs.python.org/file48998/server.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40024] Add _PyModule_AddType private helper function

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 37fcbb65d4589fbb5a72153e9338cf8e6495f64f by Dong-hee Na in branch 
'master':
bpo-40024: Update C extension modules to use PyModule_AddType() (GH-19119)
https://github.com/python/cpython/commit/37fcbb65d4589fbb5a72153e9338cf8e6495f64f


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40050] importlib: module.__spec__ leaks importlib namespaces (test_importlib leaked xxx references)

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +18509
pull_request: https://github.com/python/cpython/pull/19148

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40057] Missing mention of some class attributes in socketserver documentation

[OverMighty]

Change by OverMighty :


--
keywords: +patch
pull_requests: +18507
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/19147

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40057] Missing mention of some class attributes in socketserver documentation

[OverMighty]

New submission from OverMighty :

The documentation of the `socketserver` module of the Python standard library, 
available here: https://docs.python.org/3/library/socketserver.html, doesn't 
mention the existence of the following class attributes:

StreamRequestHandler.connection (defined at line 764 of socketserver.py)
DatagramRequestHandler.packet (defined at line 812 of socketserver.py)
DatagramRequestHandler.socket (defined at line 812 of socketserver.py)

--
assignee: docs@python
components: Documentation
messages: 364962
nosy: docs@python, overmighty
priority: normal
severity: normal
status: open
title: Missing mention of some class attributes in socketserver documentation
type: enhancement
versions: Python 3.5, Python 3.6, Python 3.7, Python 3.8, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40052] Incorrect pointer alignment in _PyVectorcall_Function() of cpython/abstract.h

[Andreas Schneider]

Andreas Schneider  added the comment:

clang -Werror -Wcast-align ...

rpm -q clang9
clang9-9.0.1-8.1.x86_64

Does that help? Found in CI of

https://gitlab.com/cwrap/pam_wrapper

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39949] truncating match in regular expression match objects repr

[Seth Troisi]

Change by Seth Troisi :


--
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40056] more user-friendly turtledemo

[Evin Liang]

New submission from Evin Liang :

[minor]
1. Display underscores as spaces in menu bar
2. Allow user to run custom code

--
components: Library (Lib)
messages: 364961
nosy: Evin Liang
priority: normal
pull_requests: 18506
severity: normal
status: open
title: more user-friendly turtledemo
type: enhancement
versions: Python 3.6, Python 3.7, Python 3.8, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40029] test_importlib.test_zip requires zlib but not marked

[Jason R. Coombs]

Jason R. Coombs  added the comment:

Thanks for the report and the fix.

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40029] test_importlib.test_zip requires zlib but not marked

[Jason R. Coombs]

Jason R. Coombs  added the comment:


New changeset 15e5024d04fc89d948ae761d88048bc58a56b650 by Roman Yurchak in 
branch 'master':
bpo-40029 mark test_importlib.test_zip as requiring zlib (#19105)
https://github.com/python/cpython/commit/15e5024d04fc89d948ae761d88048bc58a56b650


--
nosy: +jaraco

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39798] Update and Improve README.AIX

[Michael Felt]

Michael Felt  added the comment:

@BTaskaya - can you elaborate on what issues you ran into? Perhaps open an 
issue on the repository I started to work on getting this improved.

Thanks!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40014] os.getgrouplist() can fail on macOS if the user has more than 17 groups

[STINNER Victor]

STINNER Victor  added the comment:

Ned: so Linux was also impacted, but Linux only has an issue with more than 
65536 groups :-D macOS MAX_GROUPS is now only 16 (Python uses MAX_GROUPS+1)! 
Maybe MAX_GROUPS was reduced recently. I'm not sure.

Anyway, os.getgrouplist() does no longer depend on MAX_GROUPS hardcoded limit, 
but grow the group list dynamically. It should now work with any number of 
groups an all platforms (which provide the getgrouplist() function ;-)) on 
Python 3.7, 3.8 and master branches.

Thanks Dong-hee Na for the bug report and your PR.

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40014] os.getgrouplist() can fail on macOS if the user has more than 17 groups

[miss-islington]

miss-islington  added the comment:


New changeset af6fd1faa68f57c11c862624798f8510b7cac68a by Miss Islington (bot) 
in branch '3.8':
bpo-40014: Fix os.getgrouplist() (GH-19126)
https://github.com/python/cpython/commit/af6fd1faa68f57c11c862624798f8510b7cac68a


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40014] os.getgrouplist() can fail on macOS if the user has more than 17 groups

[miss-islington]

miss-islington  added the comment:


New changeset 5753fc69977bd9f70ecb4d466bda650efccf9e0a by Miss Islington (bot) 
in branch '3.7':
bpo-40014: Fix os.getgrouplist() (GH-19126)
https://github.com/python/cpython/commit/5753fc69977bd9f70ecb4d466bda650efccf9e0a


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40052] Incorrect pointer alignment in _PyVectorcall_Function() of cpython/abstract.h

[Emmanuel Arias]

Emmanuel Arias  added the comment:

Hi! I cannot reproduce the error. Could you provide the way to reproduce?

thanks

--
nosy: +eamanu

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40050] importlib: module.__spec__ leaks importlib namespaces (test_importlib leaked xxx references)

[STINNER Victor]

STINNER Victor  added the comment:

Since this reference leak is fixed, I reapplied Hai Shi's change for _weakref 
in bpo-1635741:

New changeset 93460d097f50db0870161a63911d61ce3c5f4583 by Victor Stinner in 
branch 'master':
bpo-1635741: Port _weakref extension module to multiphase initialization (PEP 
489) (GH-19140)
https://github.com/python/cpython/commit/93460d097f50db0870161a63911d61ce3c5f4583

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1635741] Py_Finalize() doesn't clear all Python objects at exit

[Alex Budovski]

Change by Alex Budovski :


--
nosy:  -Alex Budovski

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1635741] Py_Finalize() doesn't clear all Python objects at exit

[STINNER Victor]

STINNER Victor  added the comment:

I managed to identify bpo-40050 (test_importlib reference leak) root issue and 
to fix it, so I reapplied Hai Shi's change for _weakref.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1635741] Py_Finalize() doesn't clear all Python objects at exit

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 93460d097f50db0870161a63911d61ce3c5f4583 by Victor Stinner in 
branch 'master':
bpo-1635741: Port _weakref extension module to multiphase initialization (PEP 
489) (GH-19140)
https://github.com/python/cpython/commit/93460d097f50db0870161a63911d61ce3c5f4583


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40014] os.getgrouplist() can fail on macOS if the user has more than 17 groups

[miss-islington]

Change by miss-islington :


--
pull_requests: +18505
pull_request: https://github.com/python/cpython/pull/19144

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40014] os.getgrouplist() can fail on macOS if the user has more than 17 groups

[miss-islington]

Change by miss-islington :


--
pull_requests: +18504
pull_request: https://github.com/python/cpython/pull/19143

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40014] os.getgrouplist() can fail on macOS if the user has more than 17 groups

[STINNER Victor]

STINNER Victor  added the comment:


New changeset f5c7cabb2be4e42a5975ba8aac8bb458c8d9d6d7 by Victor Stinner in 
branch 'master':
bpo-40014: Fix os.getgrouplist() (GH-19126)
https://github.com/python/cpython/commit/f5c7cabb2be4e42a5975ba8aac8bb458c8d9d6d7


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40050] importlib: module.__spec__ leaks importlib namespaces (test_importlib leaked xxx references)

[STINNER Victor]

STINNER Victor  added the comment:

I close the issue.

I pushed commit 83d46e0622d2efdf5f3bf8bf8904d0dcb55fc322 which should not be 
controversial. In short, the fix is to remove two unused imports :-D

The fix doesn't remove module.__spec__ nor avoid usage of _weakref, it only fix 
this issue (reference leak) by copying code from _bootstrap.py to 
_bootstrap_external.py. In fact, modules like _io were already correctly 
imported by _bootstrap_external._setup(). Only _thread, _weakref and winreg 
imports caused this bug.

I don't think that it's worth it to backport the change to stable branches 3.7 
and 3.8, since stable branches don't use multiphase initialization for 
_weakref. The two unused imports don't cause any harm in 
importlib._bootstrap_external.


> Can I ask why you suddenly want to throw __spec__ objects out due to a leak 
> tied back to _weakref switching to multi-phase initialization?

I was thinking just aloud to try to find a solution for this reference leak.

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40013] CSV DictReader parameter documentation

[miss-islington]

Change by miss-islington :


--
nosy: +miss-islington
nosy_count: 4.0 -> 5.0
pull_requests: +18502
pull_request: https://github.com/python/cpython/pull/19141

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40013] CSV DictReader parameter documentation

[miss-islington]

Change by miss-islington :


--
pull_requests: +18503
pull_request: https://github.com/python/cpython/pull/19142

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40013] CSV DictReader parameter documentation

[Karthikeyan Singaravelan]

Karthikeyan Singaravelan  added the comment:


New changeset 4b3252cb764807fdb3a661b458d43e4af55cf4df by Juhana Jauhiainen in 
branch 'master':
bpo-40013: Clarify documentation of restval in csv.DictReader (GH-19099)
https://github.com/python/cpython/commit/4b3252cb764807fdb3a661b458d43e4af55cf4df


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40050] importlib: module.__spec__ leaks importlib namespaces (test_importlib leaked xxx references)

[Brett Cannon]

Brett Cannon  added the comment:

>From Victor:
* Maybe the test_importlib should before save/restore the the "Python state" 
rather than modifying modules

You will have to be more specific than that as there is an import_state() 
context manager to control import state via the sys module.


* Maybe module.__spec__ should leak less importlib internals: explicitly clear 
namespaces? Use static methods? I'm not sure how to do that.

Are you saying change everything on __spec__ objects to be static methods? That 
won't work because the whole point of __spec__ objects is to essentially be 
data classes to store details of a module.

* Remove module.__spec__? ... that would mean rejecting PEP 451 implemented in 
Python 3.4, that sounds like a major regression :-(

No. You would break the world and undo years of work to clean up import 
semantics with no good way to go back to the old way.

Can I ask why you suddenly want to throw __spec__ objects out due to a leak 
tied back to _weakref switching to multi-phase initialization? Also note that 
the use of weakrefs by importlib isn't tied to __spec__ objects but to 
per-module import locks in importlib._bootstrap.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1635741] Py_Finalize() doesn't clear all Python objects at exit

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +18501
pull_request: https://github.com/python/cpython/pull/19140

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40050] importlib: module.__spec__ leaks importlib namespaces (test_importlib leaked xxx references)

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 83d46e0622d2efdf5f3bf8bf8904d0dcb55fc322 by Victor Stinner in 
branch 'master':
bpo-40050: Fix importlib._bootstrap_external (GH-19135)
https://github.com/python/cpython/commit/83d46e0622d2efdf5f3bf8bf8904d0dcb55fc322


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20526] python: Modules/gcmodule.c:379: visit_decref: Assertion `((gc)->gc.gc_refs >> (1)) != 0' failed.

[STINNER Victor]

STINNER Victor  added the comment:

Just in case, I also ran asyncio_gc.py 10x times on the master branch. I just 
replaced asyncio.async with asyncio.ensure_future. In short, I consider that 
the bug is now fixed.


I interrupted the test two times with CTRL+c. 9 runs were stopped correctly. 1 
run failed with:

Fatal Python error: _enter_buffered_busy: could not acquire lock for 
<_io.BufferedWriter name=''> at interpreter shutdown, possibly due to 
daemon threads

But this is a different issue, unrelated to "Modules/gcmodule.c:379: 
visit_decref: Assertion `((gc)->gc.gc_refs >> (1)) != 0' failed". It's just 
that the test uses print() in threads, whereas using print() during Python 
finalization is not reliable. Using os.write() or avoiding print() would avoid 
the issue.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20526] python: Modules/gcmodule.c:379: visit_decref: Assertion `((gc)->gc.gc_refs >> (1)) != 0' failed.

[STINNER Victor]

STINNER Victor  added the comment:

I ran threading_shutdown_interrupted.py 10 times on the master branch (at 
commit 9b8e74ca77da7167033917d155e5f55c67b92f14): it does no longer crash.

I consider that the root issue is now fixed, so I close again the bug.

Thanks Pablo for the helpful debugging session!

--
components: +Interpreter Core
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed
versions: +Python 3.7, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40055] test___all__ and test_distutils alters the enviroinment: pkg_resources.PEP440Warning

[STINNER Victor]

Change by STINNER Victor :


--
keywords: +patch
pull_requests: +18500
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/19139

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20526] python: Modules/gcmodule.c:379: visit_decref: Assertion `((gc)->gc.gc_refs >> (1)) != 0' failed.

[STINNER Victor]

STINNER Victor  added the comment:


New changeset d1c09896c3b91d0ad7e3a14fabecde268f70dac7 by Victor Stinner in 
branch '3.7':
bpo-20526: Fix PyThreadState_Clear(): don't decref frame (GH-19120) (GH-19136) 
(GH-19137)
https://github.com/python/cpython/commit/d1c09896c3b91d0ad7e3a14fabecde268f70dac7


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40055] test___all__ and test_distutils alters the enviroinment: pkg_resources.PEP440Warning

[STINNER Victor]

STINNER Victor  added the comment:

> pkg_resources comes from python3-setuptools-41.6.0-1.fc31.noarch package.

Here is the line which alters warnings filters:

$ grep warnings /usr/lib/python3.7/site-packages/pkg_resources/__init__.py
(...)
warnings.filterwarnings("ignore", category=PEP440Warning, append=True)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40055] test___all__ and test_distutils alters the enviroinment: pkg_resources.PEP440Warning

[STINNER Victor]

New submission from STINNER Victor :

Even when no test is run, test_distutils alters the environment:

$ ./python -m test -v --fail-env-changed test_distutils -m DONTEXISTS
== CPython 3.7.7+ (heads/3.7:1cdc61c767, Mar 24 2020, 17:25:30) [GCC 9.2.1 
20190827 (Red Hat 9.2.1-1)]
== Linux-5.5.9-200.fc31.x86_64-x86_64-with-fedora-31-Thirty_One little-endian
== cwd: /home/vstinner/python/3.7/build/test_python_157151
== CPU count: 8
== encodings: locale=UTF-8, FS=utf-8
0:00:00 load avg: 0.37 Run tests sequentially
0:00:00 load avg: 0.37 [1/1] test_distutils

--

Ran 0 tests in 0.001s

OK
Warning -- warnings.filters was modified by test_distutils
  Before: (140048876788832, [], [])
  After:  (140048876788832, [], [('ignore', None, , None, 0)]) 
test_distutils run no tests

== Tests result: NO TEST RUN ==

1 test run no tests:
test_distutils

Total duration: 655 ms
Tests result: NO TEST RUN


The problem comes from Lib/distutils/tests/test_check.py: "from 
distutils.command.check import check, HAS_DOCUTILS" imports indirectly the 
docutils module which imports pkg_resources.

pkg_resources changes warnings filters.


docutils is installed by python3-docutils-0.15.2-1.fc31.noarch package and 
pkg_resources comes from python3-setuptools-41.6.0-1.fc31.noarch package.


Attached PR disables docutils to avoid side effects of "import docutils" like 
pkg_resources modifying warnings filters.

--
components: Distutils, Tests
messages: 364941
nosy: dstufft, eric.araujo, vstinner
priority: normal
severity: normal
status: open
title: test___all__ and test_distutils alters the enviroinment: 
pkg_resources.PEP440Warning
versions: Python 3.7, Python 3.8, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36852] Python3.7.2 fails to cross-compile (yocto / openembedded) when target is mips softfloat

[khoja]

khoja  added the comment:

I had an issue with yocto warrior and zeus branch for target simulation 
Qemu_x86. 
I did a report http://bugzilla.yoctoproject.org/show_bug.cgi?id=13842

Thank you to let me know what you think about It.

--
nosy: +medhi
type:  -> compile error
versions:  -Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40054] Allow formatted strings as docstrings

[Karthikeyan Singaravelan]

Karthikeyan Singaravelan  added the comment:

See also https://bugs.python.org/issue28739

--
nosy: +xtreak

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20526] python: Modules/gcmodule.c:379: visit_decref: Assertion `((gc)->gc.gc_refs >> (1)) != 0' failed.

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +18499
pull_request: https://github.com/python/cpython/pull/19137

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20526] python: Modules/gcmodule.c:379: visit_decref: Assertion `((gc)->gc.gc_refs >> (1)) != 0' failed.

[STINNER Victor]

STINNER Victor  added the comment:


New changeset e97c8b0688bc62959ced477d842fcd37992ef649 by Victor Stinner in 
branch '3.8':
bpo-20526: Fix PyThreadState_Clear(): don't decref frame (GH-19120) (GH-19136)
https://github.com/python/cpython/commit/e97c8b0688bc62959ced477d842fcd37992ef649


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20899] Nested namespace imports do not work inside zip archives

[Thomas Heller]

Change by Thomas Heller :


--
nosy:  -theller

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40054] Allow formatted strings as docstrings

[Eric V. Smith]

Eric V. Smith  added the comment:

The problem is that __doc__ is set at compile time, not run time. The ''.format 
call (and f-strings) are evaluated at run time.

--
nosy: +eric.smith

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20899] Nested namespace imports do not work inside zip archives

[Jonathan Hsu]

Jonathan Hsu  added the comment:

It appears this issue has been fixed, as I am unable to reproduce it on Windows 
10/Python 3.7:

Python 3.7.7 (tags/v3.7.7:d7c567b08f, Mar 10 2020, 10:41:24) [MSC v.1900 64 bit 
(AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.path += ['project1', 'project2.zip', 'project3', 'project4.zip']
>>> import parent.child.hello1
Hello 1
>>> import parent.child.hello2
Hello 2
>>> import parent.child.hello3
Hello 3
>>> import parent.child.hello4
Hello 4
>>> import boo
boo!
>>> import parent.boo
boo!

--
nosy: +Jonathan Hsu

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20526] python: Modules/gcmodule.c:379: visit_decref: Assertion `((gc)->gc.gc_refs >> (1)) != 0' failed.

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +18498
pull_request: https://github.com/python/cpython/pull/19136

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20526] python: Modules/gcmodule.c:379: visit_decref: Assertion `((gc)->gc.gc_refs >> (1)) != 0' failed.

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 5804f878e779712e803be927ca8a6df389d82cdf by Victor Stinner in 
branch 'master':
bpo-20526: Fix PyThreadState_Clear(): don't decref frame (GH-19120)
https://github.com/python/cpython/commit/5804f878e779712e803be927ca8a6df389d82cdf


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40054] Allow formatted strings as docstrings

[Richard Neumann]

New submission from Richard Neumann :

Currently only plain strings can be used as docstrings, such as:


class Foo:
"""Spamm eggs."""

For dynamic class generation, it would be useful to allow format strings as 
docstrings as well:

doc = 'eggs'

class Foo:
"""Spamm {}.""".format(doc)

or:

doc = 'eggs'

class Foo:
f"""Spamm {doc}."""

A current use case in which I realized that this feature was missing is:


class OAuth2ClientMixin(Model, ClientMixin):   # pylint: disable=R0904
"""An OAuth 2.0 client mixin for peewee models."""



@classmethod
def get_related_models(cls, model=Model):
"""Yields related models."""
for mixin, backref in CLIENT_RELATED_MIXINS:
yield cls._get_related_model(model, mixin, backref)

@classmethod
def _get_related_model(cls, model, mixin, backref):
"""Returns an implementation of the related model."""
class ClientRelatedModel(model, mixin):
f"""Implementation of {mixin.__name__}."""
client = ForeignKeyField(
cls, column_name='client', backref=backref,
on_delete='CASCADE', on_update='CASCADE')

return ClientRelatedModel

It actually *is* possible to dynamically set the docstring via the __doc__ 
attribute:

doc = 'eggs'

class Foo:
pass

Foo.__doc__ = doc


Allowing format strings would imho be more obvious when reading the code as it 
is set, where a docstring is expected i.e. below the class / function 
definition.

--
messages: 364934
nosy: conqp
priority: normal
severity: normal
status: open
title: Allow formatted strings as docstrings
type: enhancement
versions: Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1635741] Py_Finalize() doesn't clear all Python objects at exit

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +18497
pull_request: https://github.com/python/cpython/pull/19135

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40050] importlib: module.__spec__ leaks importlib namespaces (test_importlib leaked xxx references)

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +18496
pull_request: https://github.com/python/cpython/pull/19135

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40050] importlib: module.__spec__ leaks importlib namespaces (test_importlib leaked xxx references)

[Dong-hee Na]

Change by Dong-hee Na :


--
nosy: +corona10

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39689] struct and memoryview tests rely on undefined behavior (as revealed by clang 9)

[Petr Viktorin]

Petr Viktorin  added the comment:

Moved to Discourse, IMO that's a better place for maintainers of other 
PEP-3118-compatible libraries to chime in:
https://discuss.python.org/t/behavior-of-struct-format-native-bool/3774

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40050] importlib: module.__spec__ leaks importlib namespaces (test_importlib leaked xxx references)

[STINNER Victor]

STINNER Victor  added the comment:

> Before _weakref is converted to multiphase initialization:
> (...)
> => same module

That's because modules which don't use the multiphase initialization are cached 
in _PyImport_FixupExtensionObject(): see msg364914 for details. Next calls to 
_imp.create_builtin("_weakref") simply returned the cached _weakref module. 
It's loaded exactly once.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40050] importlib: module.__spec__ leaks importlib namespaces (test_importlib leaked xxx references)

[STINNER Victor]

STINNER Victor  added the comment:

Quick & dirty workaround:

diff --git a/Lib/importlib/_bootstrap_external.py 
b/Lib/importlib/_bootstrap_external.py
index 7353bf9a78..d988552f2d 100644
--- a/Lib/importlib/_bootstrap_external.py
+++ b/Lib/importlib/_bootstrap_external.py
@@ -1620,7 +1620,10 @@ def _setup(_bootstrap_module):
 setattr(self_module, '_thread', thread_module)
 
 # Directly load the _weakref module (needed during bootstrap).
-weakref_module = _bootstrap._builtin_from_name('_weakref')
+if '_weakref' not in sys.modules:
+weakref_module = _bootstrap._builtin_from_name('_weakref')
+else:
+weakref_module = sys.modules['_weakref']
 setattr(self_module, '_weakref', weakref_module)
 
 # Directly load the winreg module (needed during bootstrap).


But I think that the issue is larger than just _weakref.

* Maybe the test_importlib should before save/restore the the "Python state" 
rather than modifying modules
* Maybe module.__spec__ should leak less importlib internals: explicitly clear 
namespaces? Use static methods? I'm not sure how to do that.
* Remove module.__spec__? ... that would mean rejecting PEP 451 implemented in 
Python 3.4, that sounds like a major regression :-(

--
nosy: +brett.cannon, eric.snow, ncoghlan
title: test_importlib leaked [6303, 6299, 6303] references -> importlib: 
module.__spec__ leaks  importlib namespaces (test_importlib leaked xxx 
references)

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40050] test_importlib leaked [6303, 6299, 6303] references

[STINNER Victor]

STINNER Victor  added the comment:

> Code extremely simplified: (...)

Relationship between origin unmodified code and the extremely simplified code:

* Lib/test/test_importlib/__init__.py: load_tests() is called at each "test -R 
3:3" iteration
* Lib/test/test_importlib/test_windows.py: at each iteartion:

  * machinery = 
test.test_importlib.util.import_importlib('importlib.machinery') is called: it 
reloads the whole importlib package, with _frozen_importlib and 
_frozen_importlib_external blocked (force to reload pure Python 
importlib._bootstrap and importlib._bootstrap_external modules)
  * support.import_module('winreg', required_on=['win']) is called with the 
fresh new importlib package: in short, it tries to import winreg which fails 
with ModuleNotFoundError

* Lib/importlib/__init__.py: each time the module is reloaded

  * importlib._bootstrap._setup() is called: it copies '_thread', '_warnings' 
and '_weakref' from sys.modules into importlib._bootstrap namespace (module 
globals) using setattr(self_module, ...)
  * importlib._bootstrap_external._setup() is called: it calls 
_bootstrap._builtin_from_name('_weakref')


When importlib._bootstrap_external._setup() calls 
_bootstrap._builtin_from_name('_weakref'), _init_module_attrs() copies the 
"spec" into module.__spec__. But the spec contains a reference to 
importlib._bootstrap namespace.


Before _weakref is converted to multiphase initialization:

$ ./python
>>> import _weakref
>>> id(_weakref)
140119879580176
>>> id(_weakref.__spec__.__init__.__globals__['_weakref'])
140119879580176

=> same module


After the change:

$ ./python
>>> import _weakref
>>> id(_weakref)
140312826159952
>>> id(_weakref.__spec__.__init__.__globals__['_weakref'])
140312826366288

=> two different objects


The problem is that module.__spec__ pulls the whole importlib package which 
contains tons of objects.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40049] tarfile cannot extract from stdin

[Danijel]

Change by Danijel :


--
type:  -> crash

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40050] test_importlib leaked [6303, 6299, 6303] references

[STINNER Victor]

STINNER Victor  added the comment:

Code extremely simplified:
---
import unittest
import sys
import _imp

class ModuleSpec:
pass

class Tests(unittest.TestCase):
def test_import_fresh(self):
spec = ModuleSpec()
spec.sys_weakref = sys.modules["_weakref"]
spec.name = "_weakref"
module = _imp.create_builtin(spec)
module.__spec__ = spec
sys.modules["_weakref"] = module
---

I still get a leak:
---
test_leak leaked [34, 34, 34] references, sum=102
test_leak leaked [11, 11, 11] memory blocks, sum=33
---

I understand that sys.modules["_weakref"] is overriden by the test with a new 
module, and the new module holds a reference somehow to the previous _weakref 
module. The old and the new modules remain alive.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39689] struct and memoryview tests rely on undefined behavior (as revealed by clang 9)

[Petr Viktorin]

Petr Viktorin  added the comment:

I see. Thanks for your patience explaining this to me!

I will merge and continue in a different issue.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39689] struct and memoryview tests rely on undefined behavior (as revealed by clang 9)

[miss-islington]

miss-islington  added the comment:


New changeset 472fc843ca816d65c12f9508ac762ca492165c45 by Stefan Krah in branch 
'master':
bpo-39689: Do not use native packing for format "?" with standard size 
(GH-18969)
https://github.com/python/cpython/commit/472fc843ca816d65c12f9508ac762ca492165c45


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40050] test_importlib leaked [6303, 6299, 6303] references

[STINNER Victor]

STINNER Victor  added the comment:

> File 1:

Oops, you should read:

File 1: Lib/test/test_leak.py

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40050] test_importlib leaked [6303, 6299, 6303] references

[STINNER Victor]

STINNER Victor  added the comment:

With the latest Lib/importlib3.py, there are less leaked references, but there 
are still leaked references:

test_leak leaked [139, 139, 139] references, sum=417
test_leak leaked [42, 42, 42] memory blocks, sum=126

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40050] test_importlib leaked [6303, 6299, 6303] references

[STINNER Victor]

STINNER Victor  added the comment:

I can reproduce the leak with command:

./python -m test -R 3:3 test_leak

File 1:
---
import unittest
import sys
from importlib import _bootstrap as BOOTSTRAP


def _save_and_remove_module(name, orig_modules):
"""Helper function to save and remove a module from sys.modules

Raise ImportError if the module can't be imported.
"""
# try to import the module and raise an error if it can't be imported
if name not in sys.modules:
__import__(name)
del sys.modules[name]
for modname in list(sys.modules):
if modname == name or modname.startswith(name + '.'):
orig_modules[modname] = sys.modules[modname]
del sys.modules[modname]

def _save_and_block_module(name, orig_modules):
"""Helper function to save and block a module in sys.modules

Return True if the module was in sys.modules, False otherwise.
"""
saved = True
try:
orig_modules[name] = sys.modules[name]
except KeyError:
saved = False
sys.modules[name] = None
return saved



def import_fresh_module():
name = 'importlib3'
fresh = ('importlib',)
blocked = ('_frozen_importlib', '_frozen_importlib_external')
orig_modules = {}
names_to_remove = []
_save_and_remove_module(name, orig_modules)
try:
for fresh_name in fresh:
_save_and_remove_module(fresh_name, orig_modules)
for blocked_name in blocked:
if not _save_and_block_module(blocked_name, orig_modules):
names_to_remove.append(blocked_name)

with BOOTSTRAP._ModuleLockManager(name):
spec = BOOTSTRAP._find_spec(name, None)
module = BOOTSTRAP.module_from_spec(spec)
sys.modules[spec.name] = module
spec.loader.exec_module(module)
del sys.modules[spec.name]


finally:
for orig_name, module in orig_modules.items():
sys.modules[orig_name] = module
for name_to_remove in names_to_remove:
del sys.modules[name_to_remove]

class Tests(unittest.TestCase):
def test_import_fresh(self):
import_fresh_module()
---

File 2: Lib/importlib3.py
---
import _imp
import sys

try:
import _frozen_importlib as _bootstrap
case = 1
except ImportError:
case = 2

if case == 2:
_bootstrap = type(sys)("_bootstrap")
_bootstrap._weakref = sys.modules['_weakref']

class ModuleSpec:
@staticmethod
def method():
pass

spec = ModuleSpec()
spec.name = "_weakref"

module = _imp.create_builtin(spec)
module.__spec__ = spec

sys.modules["_weakref"] = module
---

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39615] cpython/abstract.h not compatible with C90

[Peter Eisentraut]

Peter Eisentraut  added the comment:

3.9.0a5 fixes my original issue.  Thanks.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11218] pattern=None when following documentation for load_tests and unittest.main()

[Zbynek Winkler]

Change by Zbynek Winkler :


--
nosy: +Zbynek.Winkler

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23882] unittest discovery doesn't detect namespace packages when given no parameters

[Zbynek Winkler]

Change by Zbynek Winkler :


--
nosy: +Zbynek.Winkler

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39672] Segmentation fault on shutdown with shelve & c pickle

[zd nex]

Change by zd nex :


--
title: SIGSEGV crash on shutdown with shelve & c pickle -> Segmentation fault 
on shutdown with shelve & c pickle

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40053] Document the behavior that no interplotation is applied when no *args are passed in for logging statements

[Eric V. Smith]

Eric V. Smith  added the comment:

I think it's important that logging has this feature for the reasons stated, so 
it should be documented.

And hopefully it's also tested for, but I haven't looked.

--
assignee:  -> docs@python
components: +Documentation -Library (Lib)
nosy: +docs@python, eric.smith
stage:  -> needs patch
versions: +Python 3.7, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36759] astimezone() fails on Windows for pre-epoch times

[Ard Kuijpers]

Ard Kuijpers  added the comment:

It would be helpful to have a better error message than '[Errno 22] Invalid 
argument' on Windows, so a ValueError seems to be a good idea at the moment.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40053] Document the behavior that no interplotation is applied when no *args are passed in for logging statements

[Karthikeyan Singaravelan]

Change by Karthikeyan Singaravelan :


--
nosy: +vinay.sajip

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40052] Incorrect pointer alignment in _PyVectorcall_Function() of cpython/abstract.h

[Andreas Schneider]

Change by Andreas Schneider :


--
type:  -> compile error
versions: +Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40053] Document the behavior that no interplotation is applied when no *args are passed in for logging statements

[Nan Hua]

New submission from Nan Hua :

As I see, Python's logging module's implementation has a nice property that, 
when no additional args are passed in, the msg (first argument) will be 
directly printed.

For example, logging.error('abc %s') can be handled peacefully with printing 
"ERROR:root:abc %s" in the log.

However, the logging's documentation only said the followings:
"The msg is the message format string, and the args are the arguments which are 
merged into msg using the string formatting operator."

>From what I see, this implementation (seems the case for both Python2 and 
>Python3) has many benefits: saving CPU resources, safe handling pre-formated 
>string, etc. More importantly, it also de-facto allows using the convenient 
>f-string in logging statement, e.g. 

logging.error(f'Started at {start_time}, finished at {finish_time}'
  f' by user {user}')

can run correctly and smoothly even with user containing %s inside.

In summary, I hope this de-facto actual behavior can be officially endorsed, 
with wordings like,
"When *args is empty, i.e. no additional positional arguments passed in, the 
msg be of any string (no need to be a format string) and will be directly used 
as is without interpolation."

What do you think? Thank you a lot!

--
components: Library (Lib)
messages: 364920
nosy: nhua
priority: normal
severity: normal
status: open
title: Document the behavior that no interplotation is applied when no *args 
are passed in for logging statements
type: behavior
versions: Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40052] Incorrect pointer alignment in _PyVectorcall_Function() of cpython/abstract.h

[Roundup Robot]

Change by Roundup Robot :


--
keywords: +patch
nosy: +python-dev
nosy_count: 1.0 -> 2.0
pull_requests: +18494
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/19133

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40052] Incorrect pointer alignment in _PyVectorcall_Function() of cpython/abstract.h

[Andreas Schneider]

New submission from Andreas Schneider :

In file included from /builds/cryptomilk/pam_wrapper/src/python/pypamtest.c:21:
In file included from /usr/include/python3.8/Python.h:147:
In file included from /usr/include/python3.8/abstract.h:837:
/usr/include/python3.8/cpython/abstract.h:91:11: error: cast from 'char *' to 
'vectorcallfunc *' (aka 'struct _object *(**)(struct _object *, struct _object 
*const *, unsigned long, struct _object *)') increases required alignment from 
1 to 8 [-Werror,-Wcast-align]
ptr = (vectorcallfunc*)(((char *)callable) + offset);
  ^~
1 error generated.


The correct way to do it would be:

union {
   char *data;
   vectorcallfunc *ptr;
} vc;

vc.data = (char *)callable + offset;
return *vc.ptr;

--
components: C API
messages: 364919
nosy: asn
priority: normal
severity: normal
status: open
title: Incorrect pointer alignment in _PyVectorcall_Function() of 
cpython/abstract.h

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com