[issue36759] astimezone() fails on Windows for pre-epoch times

[Jonathan Hsu]

Jonathan Hsu  added the comment:

This exception is raised because astimezone() ends up calling time.localtime() 
to determine the appropriate time zone. If the datetime object has a pre-epoch 
value, it passes a negative timestamp to time.localtime(). On Windows, 
time.localtime() does not accept values greater than 0 (more discussion in 
issue #35796).

This is the minimal code required to reproduce the error:

from datetime import datetime
datetime(1969, 1, 1).astimezone()

Without the ability to ascertain the time zone with localtime(), I'm not sure 
if the time zone can be accurately determined. It's not clear what the proper 
behavior is. Maybe raise a ValueError?

PEP 615 proposes to include the IANA tz database, which would negate the need 
for a system call. Should we wait for this PEP before fixing this issue? 
Thoughts?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40051] Dead link in help(lib2to3)

[wyz23x2]

New submission from wyz23x2 :

When typing this in shell:
>>> import lib2to3
>>> help(lib2to3)

The output contains this link:
--snip--
MODULE REFERENCE
https://docs.python.org/3.8/library/lib2to3 <--

The following documentation is automatically generated from the Python
--snip--

But when you access it, 404!
This works:
https://docs.python.org/3.8/library/2to3.html#module-lib2to3

Please change it. Thanks!

--
assignee: docs@python
components: 2to3 (2.x to 3.x conversion tool), Documentation, Library (Lib)
messages: 364917
nosy: docs@python, wyz23x2
priority: normal
severity: normal
status: open
title: Dead link in help(lib2to3)
type: performance
versions: Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40050] test_importlib leaked [6303, 6299, 6303] references

[hai shi]

Change by hai shi :


--
nosy: +shihai1991

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36054] On Linux, os.count() should read cgroup cpu.shares and cpu.cfs (CPU count inside docker container)

[Manjusaka]

Manjusaka  added the comment:

Actually, we already have some third party libs to support cgroup. But most of 
them get these questions

1. They are not std lib

2. They are just support cgroup1

But if we want to add a new std lib. Should we create a PEP?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40017] Please support CLOCK_TAI in the time module.

[Benjamin Peterson]

Benjamin Peterson  added the comment:


New changeset 687fe979705dc588a46a35da884cc0198c67 by Russell Owen in 
branch 'master':
closes bpo-40017: Add CLOCK_TAI constant to the time module. (GH-19096)
https://github.com/python/cpython/commit/687fe979705dc588a46a35da884cc0198c67


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40050] test_importlib leaked [6303, 6299, 6303] references

[STINNER Victor]

STINNER Victor  added the comment:

Before commit 8334f30a74abcf7e469b901afc307887aa85a888, at the first 
_imp.create_builtin() calls, it calls _PyImport_FixupExtensionObject() which 
stores the created module in an internal "extensions" dictionary. 
PyInit__weakref() returns a module object. Next calls to 
_imp.create_builtin("_weakref") simply returned the cached _weakref module.

At commit 8334f30a74abcf7e469b901afc307887aa85a888, _imp.create_builtin() 
doesn't store it in the internal "extensions" dictionary, but calls 
PyModule_FromDefAndSpec() instead. PyInit__weakref() returns a module 
definition (PyModuleDef_Type: "moduledef"). Each 
_imp.create_builtin("_weakref") creates a new module.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38972] [venv] Link to instructions to change PowerShell execution policy for venv activation

[Derek Keeler]

Change by Derek Keeler :


--
keywords: +patch
pull_requests: +18493
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/19131

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40034] cgi.parse() does not work with multipart POST requests.

[San]

Change by San :


--
keywords: +patch
pull_requests: +18492
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/19130

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40041] Typo in argparse ja-document wrong:'append', correct:'extend'

[Yasunobu Imamura]

Yasunobu Imamura  added the comment:

moved to https://github.com/python-doc-ja/python-doc-ja/issues/825

--
resolution:  -> rejected
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40018] test_ssl fails with OpenSSL 1.1.1e

[Charalampos Stratakis]

Charalampos Stratakis  added the comment:

Still searching the issue and created a first draft PR. With it, tesT_ssl and 
test_imaplib pass now, urllib2_localnet still has issues.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40018] test_ssl fails with OpenSSL 1.1.1e

[Charalampos Stratakis]

Change by Charalampos Stratakis :


--
keywords: +patch
pull_requests: +18491
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/19129

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40050] test_importlib leaked [6303, 6299, 6303] references

[STINNER Victor]

STINNER Victor  added the comment:

I can reproduce the leak with the following test, stored as 
Lib/test/test_leak.py:
---
from test import support
import unittest

class Tests(unittest.TestCase):
def test_import_fresh(self):
support.import_fresh_module('importlib.machinery',
fresh=('importlib',),
blocked=('_frozen_importlib', '_frozen_importlib_external'))
---

Extract of "./python -m test -R 3:3 test_leak" output:
---
test_leak leaked [6303, 6299, 6303] references, sum=18905
test_leak leaked [2022, 2020, 2022] memory blocks, sum=6064
---

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[RELEASE] Python 3.9.0a5 is now available for testing

[Łukasz Langa]

On behalf of the entire Python development community, and the currently serving 
Python release team in particular, I’m pleased to announce the release of 
Python 3.9.0a5. Get it here:

https://www.python.org/downloads/release/python-390a5/ 


This is an early developer preview of Python 3.9

Python 3.9 is still in development. This releasee, 3.9.0a5 is the fifth of six 
planned alpha releases. Alpha releases are intended to make it easier to test 
the current state of new features and bug fixes and to test the release 
process. During the alpha phase, features may be added up until the start of 
the beta phase (2020-05-18) and, if necessary, may be modified or deleted up 
until the release candidate phase (2020-08-10). Please keep in mind that this 
is a preview release and its use is not recommended for production environments.

Major new features of the 3.9 series, compared to 3.8

Many new features for Python 3.9 are still being planned and written. Among the 
new major new features and changes so far:

PEP 584 , Union Operators in dict
PEP 593 , Flexible function and 
variable annotations
PEP 602 , Python adopts a stable 
annual release cadence
BPO 38379 , garbage collection does not 
block on resurrected objects;
BPO 38692 , os.pidfd_open added that allows 
process management without races and signals;
BPO 39926 , Unicode support updated to 
version 13.0.0
BPO 1635741 , when Python is initialized 
multiple times in the same process, it does not leak memory anymore
A number of Python builtins (range, tuple, set, frozenset, list) are now sped 
up using PEP 570  vectorcall
A number of standard library modules (audioop, ast, grp, _hashlib, pwd, 
_posixsubprocess, random, select, struct, termios, zlib) are now using the 
stable ABI defined by PEP 384 .
(Hey, fellow core developer, if a feature you find important is missing from 
this list, let Łukasz know .)
The next pre-release, the last alpha release of Python 3.9, will be 3.9.0a6. It 
is currently scheduled for 2020-04-22. Until then, stay safe!

Your friendly release team,
Ned Deily @nad 
Steve Dower @steve.dower 
Łukasz Langa @ambv 
--
Python-announce-list mailing list -- python-announce-list@python.org
To unsubscribe send an email to python-announce-list-le...@python.org
https://mail.python.org/mailman3/lists/python-announce-list.python.org/

Support the Python Software Foundation:
http://www.python.org/psf/donations/

[issue1635741] Py_Finalize() doesn't clear all Python objects at exit

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 188078c39dec24aa5b3f2073bdc9a68ebaae42de by Victor Stinner in 
branch 'master':
Revert "bpo-1635741: Port _weakref extension module to multiphase 
initialization (PEP 489) (GH-19084)" (#19128)
https://github.com/python/cpython/commit/188078c39dec24aa5b3f2073bdc9a68ebaae42de


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40050] test_importlib leaked [6303, 6299, 6303] references

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 188078c39dec24aa5b3f2073bdc9a68ebaae42de by Victor Stinner in 
branch 'master':
Revert "bpo-1635741: Port _weakref extension module to multiphase 
initialization (PEP 489) (GH-19084)" (#19128)
https://github.com/python/cpython/commit/188078c39dec24aa5b3f2073bdc9a68ebaae42de


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[RELEASE] Python 3.9.0a5 is now available for testing

[Łukasz Langa]

On behalf of the entire Python development community, and the currently serving 
Python release team in particular, I’m pleased to announce the release of 
Python 3.9.0a5. Get it here:

https://www.python.org/downloads/release/python-390a5/ 


This is an early developer preview of Python 3.9

Python 3.9 is still in development. This releasee, 3.9.0a5 is the fifth of six 
planned alpha releases. Alpha releases are intended to make it easier to test 
the current state of new features and bug fixes and to test the release 
process. During the alpha phase, features may be added up until the start of 
the beta phase (2020-05-18) and, if necessary, may be modified or deleted up 
until the release candidate phase (2020-08-10). Please keep in mind that this 
is a preview release and its use is not recommended for production environments.

Major new features of the 3.9 series, compared to 3.8

Many new features for Python 3.9 are still being planned and written. Among the 
new major new features and changes so far:

PEP 584 , Union Operators in dict
PEP 593 , Flexible function and 
variable annotations
PEP 602 , Python adopts a stable 
annual release cadence
BPO 38379 , garbage collection does not 
block on resurrected objects;
BPO 38692 , os.pidfd_open added that allows 
process management without races and signals;
BPO 39926 , Unicode support updated to 
version 13.0.0
BPO 1635741 , when Python is initialized 
multiple times in the same process, it does not leak memory anymore
A number of Python builtins (range, tuple, set, frozenset, list) are now sped 
up using PEP 570  vectorcall
A number of standard library modules (audioop, ast, grp, _hashlib, pwd, 
_posixsubprocess, random, select, struct, termios, zlib) are now using the 
stable ABI defined by PEP 384 .
(Hey, fellow core developer, if a feature you find important is missing from 
this list, let Łukasz know .)
The next pre-release, the last alpha release of Python 3.9, will be 3.9.0a6. It 
is currently scheduled for 2020-04-22. Until then, stay safe!

Your friendly release team,
Ned Deily @nad 
Steve Dower @steve.dower 
Łukasz Langa @ambv 
-- 
https://mail.python.org/mailman/listinfo/python-list

[issue36759] astimezone() fails on Windows for pre-epoch times

[Jonathan Hsu]

Jonathan Hsu  added the comment:

I'd like to take on this issue if no one else is working on it.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38972] [venv] Link to instructions to change PowerShell execution policy for venv activation

[Brett Cannon]

Brett Cannon  added the comment:

All sound reasonable, Derek.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36759] astimezone() fails on Windows for pre-epoch times

[Jonathan Hsu]

Change by Jonathan Hsu :


--
nosy: +Jonathan Hsu

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1635741] Py_Finalize() doesn't clear all Python objects at exit

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +18489
pull_request: https://github.com/python/cpython/pull/19128

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40050] test_importlib leaked [6303, 6299, 6303] references

[STINNER Victor]

Change by STINNER Victor :


--
keywords: +patch
pull_requests: +18490
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/19128

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1635741] Py_Finalize() doesn't clear all Python objects at exit

[STINNER Victor]

STINNER Victor  added the comment:

> The last merged pull request, GH-GH-19084, causes refleaks in importlib 
> tests. Stable buildbots are failing, I can reproduce on macOS Catalina.

I expect that the bug is non-trivial, so I prefer to open a separated issue: 
bpo-40050 "test_importlib leaked [6303, 6299, 6303] references".

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40050] test_importlib leaked [6303, 6299, 6303] references

[STINNER Victor]

New submission from STINNER Victor :

https://buildbot.python.org/all/#/builders/206/builds/119

test_importlib leaked [6303, 6299, 6303] references, sum=18905
test_importlib leaked [2022, 2020, 2022] memory blocks, sum=6064

Issue reported at: https://bugs.python.org/issue1635741#msg364845

It seems like the regression was introduced by the following change:

commit 8334f30a74abcf7e469b901afc307887aa85a888 (HEAD)
Author: Hai Shi 
Date:   Fri Mar 20 16:16:45 2020 +0800

bpo-1635741: Port _weakref extension module to multiphase initialization 
(PEP 489) (GH-19084)

--
components: Tests
messages: 364905
nosy: vstinner
priority: normal
severity: normal
status: open
title: test_importlib leaked [6303, 6299, 6303] references
versions: Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: `async def` breaks encapsulation?

[Greg Ewing]

On 24/03/20 7:25 am, Marco Sulla wrote:

But what if you don't need `await`?
A coroutine call `mycoro()` could act as the actual `await mycoro()`.
If you really want `mycoro()` actual behavior, you could write `async
mycoro()`.


The way async/await is implemented, it's not feasible to wait
until runtime to decide whether a given call should be treated
as an await or not. Different bytecode is generated for it,
and the behaviour of the enclosing function is quite different
right from the start, before the await is even reached.

Other implementation strategies are possible that would
enable "async" and "await" to be eliminated. But there are
many people, including Guido, who regard the need for these
as a feature rather than a bug, because they mark the
places where suspensions can occur. So you would have to
convince those people that these keywords *should* be
eliminated, before thinking about how to achieve it.

--
Greg

--
https://mail.python.org/mailman/listinfo/python-list

[issue40014] os.getgrouplist() can fail on macOS if the user has more than 17 groups

[STINNER Victor]

STINNER Victor  added the comment:

> Moreover, on Linux, getgrouplist() can also fail with -1 if the group list is 
> too small. In that case, ngroups is updated to the number of groups and so 
> can be used to enlarge the list!

I tested by manually initializing ngroups to a value way lower than MAX_GROUPS 
(65536 on my Fedora 31): os.getgrouplist() raises OSError() with a random 
errno, getgrouplist() doesn't set errno on failure.

I wrote PR 19126 to fix the issue on all platforms. The glibc implementation is 
detected and used: use updated ngroups value (only if it's larger).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36144] Dictionary union. (PEP 584)

[Curtis Bucher]

Change by Curtis Bucher :


--
pull_requests: +18488
pull_request: https://github.com/python/cpython/pull/19127

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40014] os.getgrouplist() can fail on macOS if the user has more than 17 groups

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +18487
pull_request: https://github.com/python/cpython/pull/19126

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38948] os.path.ismount() returns False for current working drive

[Jonathan Hsu]

Change by Jonathan Hsu :


--
nosy: +Jonathan Hsu

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19462] Add remove_argument() method to argparse.ArgumentParser

[paul j3]

paul j3  added the comment:

I think it can be closed.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36054] On Linux, os.count() should read cgroup cpu.shares and cpu.cfs (CPU count inside docker container)

[Christian Heimes]

Christian Heimes  added the comment:

I suggest that your provide a low solution that returns general information 
from cgroup v1 and unified cgroup v2 rather than a solution that focuses on CPU 
only. Then you can provide a high level interface that returns effective CPU 
cores.

cgroup v2 (unified hierarchy) has been around for 6 years and slowly gains 
traction as container platforms start to support them.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36054] On Linux, os.count() should read cgroup cpu.shares and cpu.cfs (CPU count inside docker container)

[Keir Lawson]

Change by Keir Lawson :


--
nosy:  -keirlawson

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36054] On Linux, os.count() should read cgroup cpu.shares and cpu.cfs (CPU count inside docker container)

[STINNER Victor]

STINNER Victor  added the comment:

I'm not sure that it's a good idea to change os.cpucount(). I suggest to add a 
new function instead.

os.cpu_count() iss documented as:
"Return the number of CPUs in the system."
https://docs.python.org/dev/library/os.html#os.cpu_count

By the way, the documentation adds:

"This number is not equivalent to the number of CPUs the current process can 
use. The number of usable CPUs can be obtained with 
len(os.sched_getaffinity(0))"

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: `async def` breaks encapsulation?

[Marco Sulla]

On Tue, 17 Mar 2020 at 08:36, Greg Ewing  wrote:
>
> On 4/03/20 12:52 pm, Marco Sulla wrote:
> > Why can't an asynchronous coroutine be simply a coroutine that has an
> > `async` or an `await` in its code, without `async` in the signature?
>
> That wouldn't help as much as you seem to think. You still need
> to use 'await' whenever you call a coroutine, so switching a
> function between coroutine and non-coroutine would still have
> just as much of a ripple effect on the rest of your code.

Good point. But what if you don't need `await`?
A coroutine call `mycoro()` could act as the actual `await mycoro()`.
If you really want `mycoro()` actual behavior, you could write `async
mycoro()`.
-- 
https://mail.python.org/mailman/listinfo/python-list

[issue36144] Dictionary union. (PEP 584)

[Guido van Rossum]

Guido van Rossum  added the comment:


New changeset 25e580a73c163f472fdeb5489bebef85da21655c by Curtis Bucher in 
branch 'master':
bpo-36144: Add union operators to WeakKeyDictionary (#19106)
https://github.com/python/cpython/commit/25e580a73c163f472fdeb5489bebef85da21655c


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32592] Drop support of Windows Vista and 7 in Python 3.9

[C.A.M. Gerlach]

C.A.M. Gerlach  added the comment:

Thanks, will do as requested. I was planning on knocking it out yesterday, but 
some unexpected things came up so I might have to push it all the way to next 
weekend, but I'll get it done.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36054] On Linux, os.count() should read cgroup cpu.shares and cpu.cfs (CPU count inside docker container)

[Manjusaka]

Manjusaka  added the comment:

I will make a PR in this week

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40036] Deleting duplicates in itertoolsmodule.c

[Raymond Hettinger]

Raymond Hettinger  added the comment:

Thanks for the patch.  I've applied it to the 3.9.   Don't see any need to 
backport this because it is just a minor code cleanup.

--
resolution:  -> fixed
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40036] Deleting duplicates in itertoolsmodule.c

[Raymond Hettinger]

New submission from Raymond Hettinger :


New changeset 8dd1792c680caaf94a00cead82b238238f419172 by AlphaHot in branch 
'master':
bpo-40036: Deleting duplicates in itertoolsmodule.c (GH-18958)
https://github.com/python/cpython/commit/8dd1792c680caaf94a00cead82b238238f419172


--
nosy: +rhettinger

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19462] Add remove_argument() method to argparse.ArgumentParser

[Raymond Hettinger]

Raymond Hettinger  added the comment:

> In order to migrate from optparse to argparse we need to have 
> an ability to substitute anguments, e.g. remove and then create.

It seems to me that the original use case is obsolete.

Paul, do you think this issue should be closed?

--
nosy: +rhettinger

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36054] On Linux, os.count() should read cgroup cpu.shares and cpu.cfs (CPU count inside docker container)

[Manjusaka]

Manjusaka  added the comment:

Hello Mike, thanks for your code. 

I think it's a good way 

I think if  cpu.quota and cpu.shares are -1, just return the original value in 
os.cpu_count() is OK

--
nosy: +vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40045] Make "dunder" method documentation easier to locate

[Kyle Stanley]

Kyle Stanley  added the comment:

> I vote for #2.   It improves findability while also recognizing that "dunder" 
> isn't a formal term and lies somewhere between slang and jargon.

Yeah, I suppose it is a bit more of a slang or jargon term rather than 
something with a formal definition, so (2) is fine with me. Thanks for the 
feedback. 

I'll leave the issue open for a new contributor as it would be an easy PR to 
work on. But, if it doesn't receive any attention within a few months or so, 
I'll likely just add it myself since it would be useful to me for easily 
locating the section in the future.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19462] Add remove_argument() method to argparse.ArgumentParser

[Alexander Hirner]

Alexander Hirner  added the comment:

@paul j3
FWIW, popping optionals from a cache that is maintained in addition to 
self._actions, makes special conflict handling unnecessary.

i.e.:
for option_string in a.option_strings:
parser._option_string_actions.pop(option_string)

--
nosy: +cybertreiber

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: [Twisted-Python] Twisted 20.3.0 Release Announcement

[Amber Brown (hawkowl)]

On 21/3/20 12:20 am, Amber Brown (hawkowl) wrote:
On behalf of Twisted Matrix Laboratories, I am honoured to announce the 
release of Twisted 20.3!


As an addendum, as discussed earlier in the year on Twisted's mailing 
list, this release is the final one to support Python 2.7. The next 
version of Twisted will be Python 3.5+ only.


- Amber
--
Python-announce-list mailing list -- python-announce-list@python.org
To unsubscribe send an email to python-announce-list-le...@python.org
https://mail.python.org/mailman3/lists/python-announce-list.python.org/

   Support the Python Software Foundation:
   http://www.python.org/psf/donations/

PyAmsterdam meetup moves online! 25/03/2020

[Nicolás Demarchi]

Hello all! We decided to host a few meetups online on the Python Amsterdam 
community.   Sharing it here because in this opportunity it's possible to join 
from anywhere in the world :-)
https://py.amsterdam/2020/03/25/virtual-pyamsterdam-from-home-stayathome.html

[VIRTUAL] Py.Amsterdam from home! #StayAtHome
=

Are you dealing with isolation due to COVID-19? Are you missing your
friends from the Python community? PyAmsterdam is also moving online
to help you!

This is going to be our first online meetup and we have two amazing
speakers!

Join the meetup using this Zoom meeting link [1]

About Optiver
-

Optiver [2] hosted our first meetup in 2019 and a year later they are
sponsoring our first online meetup. Thank you! Optiver is a leading
technology-based trading firm with over one thousand people globally,
united in our mission to improve the market. We provide liquidity
using our capital, at our own risk, trading in tens of thousands of
financial instruments on more than 50 exchanges around the world.
Engineering is at the core of our trading strategies. We build and
maintain low latency trading systems that respond to market events in
nanoseconds. Crucial to our success is the precision, reliability and
speed of our systems. Our engineers continually refine, store, and
analyse hundreds of terabytes of data, enabling us to automatically
price large, diverse sets of financial instruments with extremely low
error tolerance. To do this we use sophisticated technology and large-
scale systems. Our infrastructure is a combination of 10 000 highly
customised components and 600 different applications running on 1500
servers, distributed globally. Check their current open positions [3]:

Schedule


+---+-+
| 19:00 | Intro 
  |
+---+-+
| 19:15 | How to be Pythonic? Design a Query Language in Python (Cheuk Ting Ho) 
  |
+---+-+
| 20:00 | Building community and the impact of COVID-19 on the PSF/PyCon and 
its community work (Naomi Ceder) |
+---+-+
| 21:00 | Lightning talks   
  |
+---+-+
| 21:30 | Closing (if no further interest)  
  |
+---+-+

How to be Pythonic? Design a Query Language in Python
-

About Cheuk Ting Ho [4]:

After having a career in data science, Cheuk now brings her knowledge
in data and passion for the tech community into TerminusDB as the
developer relations lead. Cheuk constantly contributes to the open-
source community by giving AI and deep learning workshops and organize
sprints to encourage diversity contributions. You can also find her at
twitter via @cheukting_ho [7]

Abstract


We created Python API calls that let you can make queries and
manipulate data in our graph database. We thought about what will be
best for Pythonistas? What will be the most Pythonic way to do it? (Is
it a thing?) Here’s our journey in making WOQLpy and we want to make
it useful to you.

Building community and the impact of COVID-19 on the PSF/PyCon and its 
community work
-

About Naomi Ceder [5]:

Naomi Ceder earned a PhD in Classics several decades ago but switched
from ancient human languages to computer languages sometime in the
last century. Since 2001, she has been learning, teaching, writing
about, and using Python.

An elected fellow of the Python Software Foundation, Naomi currently
serves as chair of its board of directors. She also speaks
internationally about the Python community, and on inclusion and
diversity in technology in general.

By day she leads a team of Python programmers for Dick Blick Art
Materials, and in her spare time, she enjoys sketching, knitting, and
deep philosophical conversations with her dog. You can follow her at
twitter @NaomiCeder [6]

Abstract


The past few weeks have been difficult for tech communities in general
and for the Python community. I'll talk a little about the importance
of building communities and give some background on how the PSF is
dealing with the current situation, 

[issue40047] itertools.tee does not release resources during iteration?

[Raymond Hettinger]

Raymond Hettinger  added the comment:

I've just reviewed the code.  The observed behavior is an intrinsic part of the 
design and was an expected behavior (noted in the comments).  I don't see a 
non-invasive way of making the memory releases more aggressive.

--
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed
versions: +Python 3.9 -Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39533] Use `statx(2)` system call on Linux for extended `os.stat` information

[ntninja]

Change by ntninja :


--
keywords: +patch
pull_requests: +18486
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/19125

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39853] Segmentation fault with urllib.request.urlopen and threads

[Manjusaka]

Manjusaka  added the comment:

Hello Victor

I have tried on both MacOS and Ubuntu 18.04 from 3.8.2 to the newest code in 
master, and can't reproduce this problem

macOS-10.15.4-x86_64-i386-64bit
Python 3.9.0a4+ (heads/master:9a81ab107a, Mar 24 2020, 02:06:30)
[Clang 11.0.3 (clang-1103.0.32.26)]

Linux-4.15.0-1060-aws-x86_64-with-glibc2.27
Python 3.8.2 (default, Mar 23 2020, 19:21:16)
[GCC 8.3.0]

Linux-4.15.0-1060-aws-x86_64-with-glibc2.27
Python 3.9.0a4 (default, Mar 23 2020, 19:25:43)
[GCC 8.3.0]

I think it may be caused by the kernel version and the glibc version, I don't 
have enough evidence to prove it.

I will try the Python 3.7.6 to the newest version in Ubuntu 19.10 with Kernel 
5.3.0-29 and GCC 8.3.0

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40014] os.getgrouplist() can fail on macOS if the user has more than 17 groups

[miss-islington]

miss-islington  added the comment:


New changeset 1cdc61c7673f71f2cef57715e482c84efda6d9e0 by Miss Islington (bot) 
in branch '3.7':
bpo-40014: Fix os.getgrouplist() on macOS (GH-19118)
https://github.com/python/cpython/commit/1cdc61c7673f71f2cef57715e482c84efda6d9e0


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40014] os.getgrouplist() can fail on macOS if the user has more than 17 groups

[miss-islington]

miss-islington  added the comment:


New changeset 21bee0bd71e1ad270274499f9f58194ebb52e236 by Miss Islington (bot) 
in branch '3.8':
bpo-40014: Fix os.getgrouplist() on macOS (GH-19118)
https://github.com/python/cpython/commit/21bee0bd71e1ad270274499f9f58194ebb52e236


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36144] Dictionary union. (PEP 584)

[Guido van Rossum]

Guido van Rossum  added the comment:


New changeset f393b2c588559162dc2e77f8079a42e48558870a by Curtis Bucher in 
branch 'master':
bpo-36144: Add PEP 584 operators to collections.ChainMap (#18832)
https://github.com/python/cpython/commit/f393b2c588559162dc2e77f8079a42e48558870a


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40014] os.getgrouplist() can fail on macOS if the user has more than 17 groups

[miss-islington]

Change by miss-islington :


--
nosy: +miss-islington
nosy_count: 6.0 -> 7.0
pull_requests: +18484
pull_request: https://github.com/python/cpython/pull/19123

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40047] itertools.tee does not release resources during iteration?

[Raymond Hettinger]

Raymond Hettinger  added the comment:

Whats going on is that the implementation tracks blocks of 56 references at a 
time, freeing entire blocks at time.   So the memory release does occur 
periodically, but not as aggressively as it could. This code shows what is 
going on:

# Show the timing of when objects are decreffed

from itertools import tee

class Notify:
def __init__(self, x):
self.x = x
def __del__(self):
print('' % self.x)
def __repr__(self):
return 'Notify(%r)' % self.x

it = map(Notify, range(100))
p, q = tee(it)
for i in range(100):
next(p)
next(q)
print(i)
print('Deleting it')
del it
print('Deleting p')
del p
print('Deleting q')
del q

I'll look to see if the objects can be freed sooner without doing major brain 
surgery to the existing code.

--
assignee:  -> rhettinger
nosy: +rhettinger

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40014] os.getgrouplist() can fail on macOS if the user has more than 17 groups

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 8ec7370c89aa522602eb9604086ce9f09770953d by Victor Stinner in 
branch 'master':
bpo-40014: Fix os.getgrouplist() on macOS (GH-19118)
https://github.com/python/cpython/commit/8ec7370c89aa522602eb9604086ce9f09770953d


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40014] os.getgrouplist() can fail on macOS if the user has more than 17 groups

[miss-islington]

Change by miss-islington :


--
pull_requests: +18485
pull_request: https://github.com/python/cpython/pull/19124

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40014] os.getgrouplist() can fail on macOS if the user has more than 17 groups

[STINNER Victor]

STINNER Victor  added the comment:

test.pythoninfo of the macOS job on the PR:

os.getgrouplist: 20, 0, 12, 61, 79, 80, 81, 98, 264, 399, 701, 33, 100, 204, 
250, 395, 398, 400
os.getgroups: 20, 0, 12, 61, 79, 80, 81, 98, 264, 399, 701, 33, 100, 204, 250, 
395, 398, 400

So yeah, there are 18 groups ;-)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1635741] Py_Finalize() doesn't clear all Python objects at exit

[STINNER Victor]

STINNER Victor  added the comment:


New changeset bd409bb5b78e7ccac5fcda9ab4cec770552f3090 by Paulo Henrique Silva 
in branch 'master':
bpo-1635741: Port time module to multiphase initialization (PEP 489) (GH-19107)
https://github.com/python/cpython/commit/bd409bb5b78e7ccac5fcda9ab4cec770552f3090


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39853] Segmentation fault with urllib.request.urlopen and threads

[STINNER Victor]

STINNER Victor  added the comment:

Try to get the gdb traceback (C call stack).

You may also try: python3 -X faulthandler urllib_segfault.py.

But I expect a crash in Python finalization, where there is no Python frame and 
so empty traceback.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39853] Segmentation fault with urllib.request.urlopen and threads

[STINNER Victor]

STINNER Victor  added the comment:

Python 3.7.3 is outdated. Can you please try on newer Python version?


I failed to reproduce the crash on Fedora 31. I tested:

vstinner@apu$ python3.7 -VV
Python 3.7.6 (default, Jan 30 2020, 09:44:41) 
[GCC 9.2.1 20190827 (Red Hat 9.2.1-1)]

vstinner@apu$ python3.8 -VV
Python 3.8.2 (default, Feb 26 2020, 00:00:00) 
[GCC 9.2.1 20190827 (Red Hat 9.2.1-1)]

vstinner@apu$ python3.9 -VV
Python 3.9.0a4 (default, Feb 27 2020, 00:00:00) 
[GCC 9.2.1 20190827 (Red Hat 9.2.1-1)]

vstinner@apu$ python3-debug -VV
Python 3.7.6 (default, Jan 30 2020, 09:04:19) 
[GCC 9.2.1 20190827 (Red Hat 9.2.1-1)]

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: How to copy paragraphs (with number formatting) and images from Words (.docx) and paste into Excel (.xlsx) using Python

[Beverly Pope]

On Mar 22, 2020, at 11:12 PM, A S  wrote:
> 
> On Monday, 23 March 2020 01:58:38 UTC+8, Beverly Pope  wrote:
>>> On Mar 22, 2020, at 9:47 AM, A S >> > wrote:
>>> 
>>> I can't seem to paste pictures into this discussion so please see both my 
>>> current and desired Excel output here:
>>> 
>>> https://stackoverflow.com/questions/60800494/how-to-copy-paragraphs-with-number-formatting-and-images-from-words-docx-an
>>>  
>>> >>  
>>> >
>> Did you try using the 2 part answer on the stackoverflow webpage?
>> 
>> Bev in TX
> 
> I'm able to get the paragraphs copied correctly now! But i'm trying to figure 
> out if there's a way to copy and paste the images into the Excel, along with 
> the paragraphs as well. Do you have an idea? :)

I’m glad to hear that solution worked for you.  With that said, I only went to 
stackoverflow out of curiosity and happened ro see the posted solution. I 
probably know less about using Python to copy data from Word to Excel than you 
do, given that yesterday was the first time that I had heard about it.  

I did read that MS docx files are zip files, which can be unzipped in Python.
https://gist.github.com/another-junior-dev/990a4e622868627cb93be3d8fa2eff04 

That could provide access to the pictures contained in the document, but it 
doesn’t explain how to determine where you want to place the pictures with 
relation to text in your Excel spreadsheet.  If you could determine that, then 
you could use XlsxWriter module’s worksheet.insert_image() to insert the image. 
 See n”the Worksheet Class” in the XlsxWriter docs:
https://xlsxwriter.readthedocs.io 
The rest is beyond the realm of my knowledge.

If it were me, I would go back to stackoveflow and open a new question, as this 
is different than your original post.

Bev in TX
-- 
https://mail.python.org/mailman/listinfo/python-list

[issue40014] os.getgrouplist() can fail on macOS if the user has more than 17 groups

[STINNER Victor]

STINNER Victor  added the comment:

> Most likely this hasn't been an issue before is that 10.15 creates more 
> groups for system services than the earlier versions did.

Aha, interesting. I updated my PR to remove mentions of the macOS version 
number. I just wrote "on macOS". Yeah, I wouldn't be surprised that the 
function always failed.

--

I'm not sure that the Linux implementation is correct. It rely on errno, 
whereas Linux manual page of getgrouplist() doesn't say anything about errno 
being set on error.

Moreover, on Linux, getgrouplist() can also fail with -1 if the group list is 
too small. In that case, ngroups is updated to the number of groups and so can 
be used to enlarge the list!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20526] python: Modules/gcmodule.c:379: visit_decref: Assertion `((gc)->gc.gc_refs >> (1)) != 0' failed.

[STINNER Victor]

STINNER Victor  added the comment:

> (This issue previously had "release blocker" priority before it was closed.  
> Now that it is re-opened does it still need that? I'm assuming not.)

Techicanilly, it's a 3.9 regression but I don't think that it should be hold 
3.9 release. So no, it's not a blocker issue.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20526] python: Modules/gcmodule.c:379: visit_decref: Assertion `((gc)->gc.gc_refs >> (1)) != 0' failed.

[Ned Deily]

Ned Deily  added the comment:

(This issue previously had "release blocker" priority before it was closed.  
Now that it is re-opened does it still need that? I'm assuming not.)

--
nosy: +ned.deily
priority: release blocker -> high
versions: +Python 3.9 -Python 3.4

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: Like c enumeration in python3

[Paulo da Silva]

Thank you very much for all your responses!
Now I have too much ideas :-)
I'm saving your answers and I'll see what is more
appropriate/comfortable in my case.

Best regards.
Paulo
-- 
https://mail.python.org/mailman/listinfo/python-list

[issue40045] Make "dunder" method documentation easier to locate

[Raymond Hettinger]

Raymond Hettinger  added the comment:

I vote for #2.   It improves findability while also recognizing that "dunder" 
isn't a formal term and lies somewhere between slang and jargon.

--
nosy: +rhettinger

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1635741] Py_Finalize() doesn't clear all Python objects at exit

[Paulo Henrique Silva]

Change by Paulo Henrique Silva :


--
pull_requests: +18483
pull_request: https://github.com/python/cpython/pull/19122

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39853] Segmentation fault with urllib.request.urlopen and threads

[Manjusaka]

Manjusaka  added the comment:

use the same GCC 8.3.0 to recompile the Python 3.7.3 still no fault

maybe we need the core dump to figure it out.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39148] DatagramProtocol + IPv6 does not work with ProactorEventLoop

[Kjell Braden]

Change by Kjell Braden :


--
keywords: +patch
pull_requests: +18482
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/19121

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37319] Deprecate using random.randrange() with non-integers

[Raymond Hettinger]

Raymond Hettinger  added the comment:

Marking this as rejected.  Nothing good would come from obfuscating the docs to 
note a minor implementation detail.

--
resolution:  -> rejected
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40033] Just defined class missing from scope

[Raymond Hettinger]

Raymond Hettinger  added the comment:

> I assumed that Level1A should already be in scope while defining 
> the insides of Level1B. But it isn't.

That analysis is correct.  Class namespaces don't nest like functions do.  The 
"enclosing" namespace is globals, not the surrounding class.

This isn't a bug, but it is understandable why you might assume otherwise.

--
nosy: +rhettinger
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39853] Segmentation fault with urllib.request.urlopen and threads

[Manjusaka]

Manjusaka  added the comment:

I have tried it on Python 3.7.3 Ubuntu 18.04 

Linux-4.15.0-1060-aws-x86_64-with-debian-buster-sid
Python 3.7.3 (default, Mar 23 2020, 18:15:26)
[GCC 7.5.0]

there is no segmentation fault 

would you mind sharing the core dump to help us find more detail about the 
crash?

--
nosy: +Manjusaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37319] Deprecate using random.randrange() with non-integers

[Tim Peters]

Tim Peters  added the comment:

It's neither "bug" nor "feature" - it's an inherited quirk CPython maintains to 
avoid annoying users for no good reason.

If you absolutely have to "do something", how about adding a mere footnote?  
The docs already imply the args have the same restrictions as for `range()`.  A 
footnote could add that, to preserve historical compatibility, CPython also 
accepts arguments `x` for which `int(x) == x`, and uses `int(x)` in such cases 
- but portable code shouldn't rely on that.

But nobody cares, so putting that in the main part of the docs would also annoy 
99.99% of users for no good reason ;-)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1635741] Py_Finalize() doesn't clear all Python objects at exit

[hai shi]

hai shi  added the comment:

> The last merged pull request, GH-GH-19084, causes refleaks in importlib 
> tests. Stable buildbots are failing, I can reproduce on macOS Catalina.

thanks, Łukasz.
I catched this problem in my vm of centos too. I don't the broken reason 
temporarily.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40014] os.getgrouplist() can fail on macOS if the user has more than 17 groups

[Ned Deily]

Ned Deily  added the comment:

Thanks for the PR, Victor, it does address the problem.  I went back and 
explicitly tested on macOS 10.14 and 10.13 by creating some more groups and 
adding the user to them and the same failure occurred there so this is not just 
a 10.15 issue.  Most likely this hasn't been an issue before is that 10.15 
creates more groups for system services than the earlier versions did.  So the 
PR should be updated to remove the 10.15 (and 10.5!) references.

--
title: os.getgrouplist() fails on macOS 10.15 (Catalina) if the user has more 
than 17 groups -> os.getgrouplist() can fail on macOS if the user has more than 
17 groups

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34591] smtplib mixes RFC821 and RFC822 addresses

[Manjusaka]

Manjusaka  added the comment:

> This is a problem because not all RFC822 addresses are valid RFC821 addresses.

Do you mean that we would add a verification before we send the command?

--
nosy: +Manjusaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40012] Avoid Python 2 documentation to appear in Web search results

[Terry Davis]

Terry Davis  added the comment:

It seems like using "noindex" to tell search engines not to look at Python 2 
docs at all would have the side-effect of breaking Python 2 documentation 
searches.
I have gotten into the habit of searching for "python 3" explicitly, which I 
think is a reasonable work-around. Especially since python 2 and 3 could be 
considered different languages.

--
nosy: +Terry Davis

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: Confusing textwrap parameters, and request for RE help

[Dieter Maurer]

Chris Angelico wrote at 2020-3-23 06:00 +1100:
>When using textwrap.fill() or friends, setting break_long_words=False
>without also setting break_on_hyphens=False has the very strange
>behaviour that a long hyphenated word will still be wrapped.

Having worked with `TeX`, I am familiar that hyphens indicate
word locations favorable to line breaks. Therefore, I understand
that one wants a way to control whether or not breaks are allowed
at hyphens, even if words should not be broken normally.

Apparently, "break_on_hyphens" adds hyphen as a word delimiter.
With this notion, the behavior can be understood.
-- 
https://mail.python.org/mailman/listinfo/python-list

[issue20526] python: Modules/gcmodule.c:379: visit_decref: Assertion `((gc)->gc.gc_refs >> (1)) != 0' failed.

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +18481
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/19120

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40039] [CVE-2020-10796] Python multiprocessing Remote Code Execution vulnerability

[STINNER Victor]

STINNER Victor  added the comment:

Lib/multiprocessing/connection.py uses a challenge to authenticate the client. 
How do you connect to the server?

Yes, it's known that pickle is not safe, there is a big red warning at the top 
of the doc:
https://docs.python.org/dev/library/pickle.html

But please elaborate your attack scenario. How do you execute arbitrary code on 
a server? How do you inject code?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36759] astimezone() fails on Windows for pre-epoch times

[SilentGhost]

SilentGhost  added the comment:

Yes, I was also able to verify this issue on 3.8.2 on win10. Argument to 
astimezone is not required, and this happens for both naïve and aware objects.

--
nosy: +belopolsky, p-ganssle
resolution: duplicate -> 
stage: resolved -> test needed
status: closed -> open
superseder: [Windows] datetime.fromtimestamp(t) when 0 <= t <= 86399 fails on 
Python 3.6 -> 
title: datetime: astimezone() results in OSError: [Errno 22] Invalid argument 
-> astimezone() fails on Windows for pre-epoch times
versions: +Python 3.8, Python 3.9 -Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40048] _PyEval_EvalFrameDefault() doesn't reset tstate->frame if _PyCode_InitOpcache() fails

[STINNER Victor]

STINNER Victor  added the comment:

Note: I found this issue with Pablo Galindo Galgado while investigating 
bpo-20526.

--
nosy: +inada.naoki, pablogsal

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40014] os.getgrouplist() fails on macOS 10.15 (Catalina) if the user has more than 17 groups

[STINNER Victor]

STINNER Victor  added the comment:

Python 3.7 and 3.8 are also affected on macOS 10.15 (Catalina).

--
versions: +Python 3.7, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: Like c enumeration in python3 [ERRATA]

[Terry Reedy]

On 3/22/2020 11:37 PM, DL Neil via Python-list wrote:

On 23/03/20 3:32 PM, Paulo da Silva wrote:

Às 02:18 de 23/03/20, Paulo da Silva escreveu:

Hi!


Olá,



Suppose a class C.
I want something like this:

class C:
KA=0
KB=1

KC=2

...
Kn=n

def __init__ ...
    ...


These constants come from an enum in a .h (header of C file).
They are many and may change from time to time.
Is there a way to somehow define them from inside __init__ giving for
example a list of names as strings?
There is an additional problem: C is not recognized inside __init__!



Please read the PSL docs carefully, because Python's enums seem to 
differ from those in other languages - sometimes in significant ways, 
and sometimes in a subtle manner!



I have been asking similar questions recently - particularly wanting to 
(also) use the "strings", because of the effort of upgrading many 
modules of code at the same time (the manual, or was it the PEP(?) 
refers to the need to upgrade the PSL to use enums, and to the effort 
that might cost - I suggest there has been little/v.slow take-up, but 
then keeping the PSL updated is a subject of a great many conversations, 
elsewhere).


Python's enums cannot be changed. Once set, C.KA cannot be changed to 1 


I am not sure what you are saying.  The OP's class is not an enum in the 
enum module sense.  Class attributues are usually mutable.


>>> class C:
K = 0


>>> C.K = 1
>>> C.K
1


--
Terry Jan Reedy


--
https://mail.python.org/mailman/listinfo/python-list

[issue40024] Add _PyModule_AddType private helper function

[Dong-hee Na]

Change by Dong-hee Na :


--
pull_requests: +18480
pull_request: https://github.com/python/cpython/pull/19119

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38972] [venv] Link to instructions to change PowerShell execution policy for venv activation

[Derek Keeler]

Derek Keeler  added the comment:

Where would we like to put this information?

My proposal would be to add this in a combination of places:

1. Add a further note near the symlinks on Windows comment in this document: 
`Doc\using\venv-create.inc`

2. Add a blurb in the Activate.ps1 script itself, so that any user can see it 
very near where the problem might occur. `Lib\venv\scripts\common\Activate.ps1`

I'd probably put something like the following in there:
---

.. note::
   On Microsoft Windows, it may be required to enable the ``Activate.ps1``
   script by setting the execution policy for the user. You can do this by
   issuing the following Powershell command from a Powershell command window:

   PS C:\Users\user> Set-ExecutionPolicy -ExecutionPolicy RemoteSigned -Scope 
CurrentUser

   See `About Execution Policies 
   
`_
   for more information.

---

(This would exist as a ".Notes" section in the Activate.ps1 script)


What do folks think?

--
nosy: +d3r3kk

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40014] os.getgrouplist() fails on macOS 10.15 (Catalina) if the user has more than 17 groups

[STINNER Victor]

Change by STINNER Victor :


--
title: os.getgrouplist() can fail on macOS -> os.getgrouplist() fails on macOS 
10.15 (Catalina) if the user has more than 17 groups

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40014] os.getgrouplist() can fail on macOS

[STINNER Victor]

STINNER Victor  added the comment:

tl; dr os.getgrouplist() is limited to 17 groups and fails with a random 
OSError if the request user has more groups. PR 19118 fix the issue.

--

On macOS-10.15.1-x86_64-i386-64bit (python -m platform), os.getgrouplist() 
fails with my user name and group identifier:

>>> os.getgrouplist('haypo', 20)
Traceback (most recent call last):
  File "", line 1, in 
OSError: [Errno 0] Error

Full example:

macbook:master haypo$ ./python.exe 
Python 3.9.0a4+ (heads/master:da2914d, Mar 20 2020, 09:45:36) 
[Clang 11.0.0 (clang-1100.0.33.8)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import os, pwd
>>> entry=pwd.getpwuid(os.getuid())

>>> os.getgrouplist(entry.pw_name, entry.pw_gid)
Traceback (most recent call last):
  File "", line 1, in 
InterruptedError: [Errno 4] Interrupted system call

>>> os.getgrouplist(entry.pw_name, entry.pw_gid)
Traceback (most recent call last):
  File "", line 1, in 
OSError: [Errno 0] Error

>>> entry
pwd.struct_passwd(pw_name='haypo', pw_passwd='', pw_uid=502, pw_gid=20, 
pw_gecos='Victor Stinner', pw_dir='/Users/haypo', pw_shell='/bin/bash')


My user has the following groups:

macbook:master haypo$ id
uid=502(haypo) gid=20(staff) 
groups=20(staff),702(com.apple.sharepoint.group.2),12(everyone),61(localaccounts),79(_appserverusr),80(admin),81(_appserveradm),98(_lpadmin),701(com.apple.sharepoint.group.1),33(_appstore),100(_lpoperator),204(_developer),250(_analyticsusers),395(com.apple.access_ftp),398(com.apple.access_screensharing),399(com.apple.access_ssh),400(com.apple.access_remote_ae),703(com.apple.sharepoint.group.3)

My user has 18 groups.

MAX_GROUPS=16, Python uses 1 + MAX_GROUPS.

getgrouplist() manual page says:

RETURN VALUES
 The getgrouplist() function returns 0 on success.  If the size of the 
group list is too small to hold
 all the user's groups, getgrouplist() returns -1 to indicate failure.  In 
this case, the group array
 will be filled with as many groups as will fit.


In short, getgrouplist() doesn't set errno on failure.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40014] os.getgrouplist() can fail on macOS

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +18479
pull_request: https://github.com/python/cpython/pull/19118

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39148] DatagramProtocol + IPv6 does not work with ProactorEventLoop

[Steve Dower]

Steve Dower  added the comment:

Main thing we need is a PR (straightforward) and confirming that the relevant 
tests all run. We can use the buildbot fleet to make sure that all supported 
platforms have the necessary APIs, but it wouldn't surprise me if there's some 
weird edge cases here.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40014] os.getgrouplist() can fail on macOS

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +18478
pull_request: https://github.com/python/cpython/pull/19117

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40049] tarfile cannot extract from stdin

[Danijel]

New submission from Danijel :

Hi,

I have the following code:

```
import tarfile
import sys

tar = tarfile.open(fileobj=sys.stdin.buffer, mode='r|*')
tar.extractall("tarout")
tar.close()
```

then doing the following on a debian 10 system:

```
$ python -m tarfile -c git.tar /usr/share/doc/git
$ python -V
Python 3.8.1
$ cat git.tar | python foo.py
$ cat git.tar | python foo.py
Traceback (most recent call last):
  File "foo.py", line 5, in 
tar.extractall("tarout")
  File "/home/danielt/miniconda3/lib/python3.8/tarfile.py", line 2026, in 
extractall
self.extract(tarinfo, path, set_attrs=not tarinfo.isdir(),
  File "/home/danielt/miniconda3/lib/python3.8/tarfile.py", line 2067, in 
extract
self._extract_member(tarinfo, os.path.join(path, tarinfo.name),
  File "/home/danielt/miniconda3/lib/python3.8/tarfile.py", line 2139, in 
_extract_member
self.makefile(tarinfo, targetpath)
  File "/home/danielt/miniconda3/lib/python3.8/tarfile.py", line 2178, in 
makefile
source.seek(tarinfo.offset_data)
  File "/home/danielt/miniconda3/lib/python3.8/tarfile.py", line 513, in seek
raise StreamError("seeking backwards is not allowed")
tarfile.StreamError: seeking backwards is not allowed
```

The second extraction trys to seek, although the mode is 'r|*'.


For reference if I remove ".buffer" from the code above, I can run
it with python2 without problems:

```
$ cat foo2.py
import tarfile
import sys

tar = tarfile.open(fileobj=sys.stdin, mode='r|*')
tar.extractall("tarout")
tar.close()

$ cat git.tar | python2 foo2.py
$ cat git.tar | python2 foo2.py
$ cat git.tar | python2 foo2.py
$ cat git.tar | python2 foo2.py
$ cat git.tar | python2 foo2.py
```

--
components: Library (Lib)
messages: 364860
nosy: dtamuc
priority: normal
severity: normal
status: open
title: tarfile cannot extract from stdin
versions: Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40046] Increase test coverage of the random module

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

What is wrong with adding more tests?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40033] Just defined class missing from scope

[Damian Yurzola]

Change by Damian Yurzola :


--
versions: +Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40046] Increase test coverage of the random module

[Raymond Hettinger]

Raymond Hettinger  added the comment:

I'm against this doc change.  While factually true, it is irrelevant and 
distracting to most users.  We do not need to test and document every 
implementation detail, nor should we.

--
resolution:  -> rejected
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37319] Deprecate using random.randrange() with non-integers

[Raymond Hettinger]

Raymond Hettinger  added the comment:

How about we just leave it alone.  AFAICT nothing is broken -- no user is 
having any issues.  We really don't need to go through every tool in the 
standard library that uses int() and document that it accepts any type that 
defines __int__, nor do we need to test those cases.  I understand that you are 
bugged by this but it isn't a real problem.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40018] test_ssl fails with OpenSSL 1.1.1e

[Kubilay Kocak]

Kubilay Kocak  added the comment:

Confirming on FreeBSD CURRENT buildbot worker which had its openssl updated to 
1.1.1e a number of days ago. I reverted back to 1.1.1d

Full log of the failing build/tests with 1.1.1e attached

--
nosy: +koobs
Added file: 
https://bugs.python.org/file48997/amd64.freebsd.shared.3.x.build.450.test.stdio.txt

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38243] [security][CVE-2019-16935] A reflected XSS in python/Lib/DocXMLRPCServer.py

[STINNER Victor]

STINNER Victor  added the comment:

Charalampos Strataris's advice: If you backport the security fix and 
test_docxmlrpc starts to hang randomly, you should also backport bpo-27614 fix. 
For example, it's the commit 3911d8333c5b6f9374fa11ab7c912f1471580f0f for 
Python 2.7. We had the issue on RHEL 7.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40018] test_ssl fails with OpenSSL 1.1.1e

[STINNER Victor]

STINNER Victor  added the comment:

I closed bpo-39787 "test_ssl and test_urllib2_localnet failing with new 
OpenSSL" as a duplicate of this issue. Copy of the message:

"""
test_ssl and test_urllib2_localnet are failing when Python is built against 
top-of-tree OpenSSL. I'm attaching the output of: `regrtest.py test_ssl 
test_urllib2_localnet -W`

The output is from a powerpc64le machine with Python 3.8.2+ (1bbb81b251bc) and 
OpenSSL master (db943f43a60d1b).

A git bisect showed the problems started with the following OpenSSL commit:

commit db943f43a60d1b5b1277e4b5317e8f288e7a0a3a
Author: Matt Caswell 
Date:   Fri Jan 17 17:39:19 2020 +

Detect EOF while reading in libssl

If we hit an EOF while reading in libssl then we will report an error
back to the application (SSL_ERROR_SYSCALL) but errno will be 0. We add
an error to the stack (which means we instead return SSL_ERROR_SSL) and
therefore give a hint as to what went wrong.

Contains a partial fix for #10880

Reviewed-by: Tomas Mraz 
Reviewed-by: Dmitry Belyavskiy 
(Merged from https://github.com/openssl/openssl/pull/10882)

This also looks similar to: https://bugs.python.org/issue28689
"""

There is a file attached which contains test failures like these ones:

test_ciphers (test.test_ssl.SimpleBackgroundTests) ...  server:  new connection 
from ('127.0.0.1', 39550)
 server: connection cipher is now ('TLS_AES_256_GCM_SHA384', 'TLSv1.3', 256)
 server: selected protocol is now None
Test server failure:
Traceback (most recent call last):
   File 
"/home/mscastanho/AT/next/opt/at-next-14.0-0-alpha/lib64/python3.8/test/test_ssl.py",
 line 2388, in run
msg = self.read()
   File 
"/home/mscastanho/AT/next/opt/at-next-14.0-0-alpha/lib64/python3.8/test/test_ssl.py",
 line 2365, in read
return self.sslconn.read()
   File 
"/home/mscastanho/AT/next/opt/at-next-14.0-0-alpha/lib64/python3.8/ssl.py", 
line 1101, in read
return self._sslobj.read(len)
 OSError: [Errno 0] Error
ERROR

==
ERROR: test_connect_capath (test.test_ssl.SimpleBackgroundTests)
--
Traceback (most recent call last):
  File 
"/home/mscastanho/AT/next/opt/at-next-14.0-0-alpha/lib64/python3.8/test/test_ssl.py",
 line 2038, in test_connect_capath
s.connect(self.server_addr)
  File 
"/home/mscastanho/AT/next/opt/at-next-14.0-0-alpha/lib64/python3.8/ssl.py", 
line 1342, in connect
self._real_connect(addr, False)
  File 
"/home/mscastanho/AT/next/opt/at-next-14.0-0-alpha/lib64/python3.8/ssl.py", 
line 1333, in _real_connect
self.do_handshake()
  File 
"/home/mscastanho/AT/next/opt/at-next-14.0-0-alpha/lib64/python3.8/ssl.py", 
line 1309, in do_handshake
self._sslobj.do_handshake()
ConnectionResetError: [Errno 104] Connection reset by peer



==
ERROR: test_https (test.test_urllib2_localnet.TestUrlopen)
--
Traceback (most recent call last):
  File 
"/home/mscastanho/AT/next/opt/at-next-14.0-0-alpha/lib64/python3.8/test/test_urllib2_localnet.py",
 line 560, in test_https
data = self.urlopen("https://localhost:%s/bizarre; % handler.port, 
context=context)
  File 
"/home/mscastanho/AT/next/opt/at-next-14.0-0-alpha/lib64/python3.8/test/test_urllib2_localnet.py",
 line 464, in urlopen
l.extend(f.readlines(200))
  File 
"/home/mscastanho/AT/next/opt/at-next-14.0-0-alpha/lib64/python3.8/http/client.py",
 line 655, in readline
result = self.fp.readline(limit)
  File 
"/home/mscastanho/AT/next/opt/at-next-14.0-0-alpha/lib64/python3.8/socket.py", 
line 669, in readinto
return self._sock.recv_into(b)
  File 
"/home/mscastanho/AT/next/opt/at-next-14.0-0-alpha/lib64/python3.8/ssl.py", 
line 1241, in recv_into
return self.read(nbytes, buffer)
  File 
"/home/mscastanho/AT/next/opt/at-next-14.0-0-alpha/lib64/python3.8/ssl.py", 
line 1099, in read
return self._sslobj.read(len, buffer)
ssl.SSLError: [SSL: KRB5_S_TKT_NYV] unexpected eof while reading (_ssl.c:2607)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39787] test_ssl and test_urllib2_localnet failing with new OpenSSL

[STINNER Victor]

STINNER Victor  added the comment:

Duplicate of bpo-40018.

--
nosy: +vstinner
resolution:  -> duplicate
stage:  -> resolved
status: open -> closed
superseder:  -> test_ssl fails with OpenSSL 1.1.1e

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40018] test_ssl fails with OpenSSL 1.1.1e

[Matheus Castanho]

Matheus Castanho  added the comment:

Hi, I believe this is the exact same problem as reported by 
https://bugs.python.org/issue39787

Both issues could be closed by the same fix =)

--
nosy: +mscastanho

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20526] python: Modules/gcmodule.c:379: visit_decref: Assertion `((gc)->gc.gc_refs >> (1)) != 0' failed.

[STINNER Victor]

STINNER Victor  added the comment:

Right now, threading_shutdown_interrupted.py does crash on the master branch 
(commit 05e4a296ecc127641160a04f39cc02c0f66a8c27). I reintroduced the bug with:

commit 9ad58acbe8b90b4d0f2d2e139e38bb5aa32b7fb6
Author: Victor Stinner 
Date:   Mon Mar 9 23:37:49 2020 +0100

bpo-19466: Py_Finalize() clears daemon threads earlier (GH-18848)

Clear the frames of daemon threads earlier during the Python shutdown to
call objects destructors. So "unclosed file" resource warnings are now
emitted for daemon threads in a more reliable way.

I investigated this bug with Pablo Galindo Salgado. When the bug triggers, 
there are 2 strong references to a Frame object:

* Traceback object
* Generator object

The bug occurs in "except ValueError as exc:" of EvilThread.coroutine().

The exception contains a traceback object which has a strong reference to the 
frame.

The frame object contains "exc" variable (the exception) which contains a 
reference to the tracecback (exc.__traceback__) which contains a refererence to 
the frame (tb.tb_frame): there is a reference cycle. (It's not really an issue 
by itself).

There are 2 strong references to the Frame object.

Py_FinalizeEx() calls _PyThreadState_DeleteExcept() which doesn't clear the 
reference cycle. But PyThreadState_Clear() calls Py_CLEAR(tstate->frame) which 
reduces the Frame reference counter from 2 to 1, even if there are still 2 
strong references to it (traceback and generator).

tstate->frame is only set at 3 places:

* new_threadstate(): when a Python thread state is created (it's initialized to 
NULL)
* _PyEval_EvalFrameDefault() entry: set to frame
* _PyEval_EvalFrameDefault() exit: set to frame->f_back (which can be NULL

By the way, _PyEval_EvalFrameDefault() has a bug: tstate->frame is not reset if 
_PyCode_InitOpcache() fails: bpo-40048.

Py_FinalizeEx() calls _PyRuntimeState_SetFinalizing(runtime, tstate) and so 
threads which tries to take the GIL must exit immediately. Extract of 
take_gil():
---
static inline int
tstate_must_exit(PyThreadState *tstate)
{
PyThreadState *finalizing = _PyRuntimeState_GetFinalizing(&_PyRuntime);
return (finalizing != NULL && finalizing != tstate);
}

static void
take_gil(PyThreadState *tstate)
{
...

if (tstate_must_exit(tstate)) {
PyThread_exit_thread();
}
...
}
---

The thread is exited in the middle of _PyEval_EvalFrameDefault() by take_gil() 
which calls PyThread_exit_thread() because the thread must exit (because the 
main thread destroyed its Python thread state and so accessing it would crash 
the thread).

_PyThreadState_DeleteExcept() calls PyThreadState_Clear() which calls 
Py_CLEAR(tstate->frame): this case only occurs because the thread exited in the 
middle of _PyEval_EvalFrameDefault(). In the normal case, 
_PyEval_EvalFrameDefault() always resets tstate->frame to its previous value.

Py_CLEAR(tstate->frame) is wrong: it's a *borrowed* reference.

Pablo and me agree with PyThreadState_Clear() must not call 
Py_CLEAR(tstate->frame): it's borrowed reference, not a strong reference.

I'm working on a PR to fix the issue.

--
resolution: wont fix -> 
status: closed -> open

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com