[issue39075] types.SimpleNamespace should preserve attribute ordering (?)

[Zackery Spytz]

Change by Zackery Spytz :


--
keywords: +patch
nosy: +ZackerySpytz
nosy_count: 6.0 -> 7.0
pull_requests: +18787
stage: needs patch -> patch review
pull_request: https://github.com/python/cpython/pull/19430

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40214] test_ctypes.test_load_dll_with_flags Windows failure

[Karthikeyan Singaravelan]

Karthikeyan Singaravelan  added the comment:

Is this going to be backported? It seems backports also use the same build and 
have this error. Sample 3.8 backport build that seems to be related to this 
issue : 
https://dev.azure.com/Python/cpython/_build/results?buildId=60753=logs=d554cd63-f8f4-5b2d-871b-33e4ea76e915=5a14d0eb-dbd4-5b80-f5d0-7909f950a1cc=1570

--
nosy: +xtreak

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40214] test_ctypes.test_load_dll_with_flags Windows failure

[Kyle Stanley]

Kyle Stanley  added the comment:

Steve Dower wrote:
> (FWIW, we build and test a release build, not a debug build, which is why 
> we're not looking for sqlite3_d.dll... but perhaps we should be using a debug 
> build? That might be slower, but the extra validation is probably 
> worthwhile...)

In general, I think a debug build instead of a release would make more sense 
for the PR tests. Do you have a general estimate or rough idea as to how much 
slower it would be in comparison?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40220] Modules/gcmodule.c:434: update_refs: Assertion "gc_get_refs(gc) != 0" failed

[Gregory P. Smith]

Gregory P. Smith  added the comment:

bad buildbot hardware.  i've taken that bot offline.

--
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed
type:  -> crash

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40091] Crash in logging._after_at_fork_child_reinit_locks()

[melwitt]

melwitt  added the comment:

Thank you for explaining those details and pointing me in the right direction 
about the proper way to solve the problem in oslo.db.

We still have to support python 3.6, so for now we will still need to do 
something different (in nova) to handle this (clear our cache of oslo.db 
_TransactionContextManager during oslo.service start(), is what I have 
proposed).

For those running python 3.7 and newer, I will try to fix the situation more 
generally in oslo.db with os.register_at_fork() using the examples you gave.

Thank you again for your help, I have learned a lot.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40220] Modules/gcmodule.c:434: update_refs: Assertion "gc_get_refs(gc) != 0" failed

[Gregory P. Smith]

Gregory P. Smith  added the comment:

While I did a general apt update on that worker, I suspect I may have knocked 
the heatsink loose over the weekend while dealing with an upcoming bot sitting 
behind it on the shelf.  That'd explain why it is alternately crashing and 
making mystery results.  I'll figure it out.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40170] [C API] Make PyTypeObject structure an opaque structure in the public C API

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 307b9d0144e719b016a47fcc43397c070615e01e by Victor Stinner in 
branch 'master':
bpo-40170: Remove PyIndex_Check() macro (GH-19428)
https://github.com/python/cpython/commit/307b9d0144e719b016a47fcc43397c070615e01e


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40217] The garbage collector doesn't take in account that objects of heap allocated types hold a strong reference to their type

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

Sorry when I said:

> If we do the same experiment after PR19314:

I meant:

If we do the same experiment after PR 19414:

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40170] [C API] Make PyTypeObject structure an opaque structure in the public C API

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +18786
pull_request: https://github.com/python/cpython/pull/19428

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40170] [C API] Make PyTypeObject structure an opaque structure in the public C API

[STINNER Victor]

STINNER Victor  added the comment:

Hum, once most changes will land, maybe it would be worth it to document them 
at:
https://docs.python.org/dev/whatsnew/3.9.html#build-and-c-api-changes

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40217] The garbage collector doesn't take in account that objects of heap allocated types hold a strong reference to their type

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

I have made some investigation, and I think a form of this bug was there from a 
long time and does not really relate to heap types.
For instance consider this code on Python3.7 (so commit 
364f0b0f19cc3f0d5e63f571ec9163cf41c62958 is not present). 

If you consider the more simple heap type:

>>> class A:
......
... 
>>> A().__class__


As the class instance refers to its type, it must visit it in the gc and we can 
see that indeed, that is the case:

>>> import gc
>>> gc.get_referents(A())
[]

But for instance, consider ast.AST, which in 3.7 is a static type created with 
PyType_Ready:


>>> import ast
>>> x = ast.AST()
>>> x.__class__


we can see that the object refers to its type as the other one butoh no, 
the gc does not know about that link:

>>> import gc
>>> gc.get_referents(x)
[]

This is because its traverse function does not visit the type:

static int
ast_traverse(AST_object *self, visitproc visit, void *arg)
{
 Py_VISIT(self->dict);
 return 0;
}


This is not a problem if the type is *static* because __although is technically 
an error__ because the GC cannot resolve
cycles that go through the type, as the type is eternal those cycles will never 
be collected. 

The problem appears when the type is not eternal and can be destroyed. For 
instance, to understand this consider a function
in the _testcapi that creates a heap type using PyType_FromSpec:

>>> import gc, _testcapi
>>> import weakref
>>> import sys

# Create a new heap type with custom traverse function
>>> x = _testcapi.construct_new_gc_type()
>>> sys.getrefcount(x)
5
>>> new_obj = x()
>>> sys.getrefcount(x)
6
# We know that now there should be a link between new_obj and x because one 
points to the other
>>> x in gc.get_referents(new_obj)
False
# Oh no, there is no link
>>> class A:
...def __del__(self):
...   print("Ouch")
... 
>>> x_w = weakref.ref(x)
# Create a reference cycle  a -> new_obj -> x -> a
>>> a = A()
>>> 
>>> x.a = a
>>> a.y = new_obj
>>> a.x = x
>>> gc.collect()
0
>>> del x,a
>>> gc.collect()
0
>>> sys.getrefcount(x_w())
6
>>> del new_obj
# At this point all variables are gone and the collector should clean everything
>>> gc.collect()
0
# Oh, no! The type is still alive
>>> sys.getrefcount(x_w())
6


If we do the same experiment after PR19314:

>>> import sys, gc, _testcapi
>>> import weakref
>>> x = _testcapi.construct_new_gc_type()
>>> sys.getrefcount(x)
5
>>> new_obj = x()
>>> sys.getrefcount(x)
6
# We know that now there should be a link between new_obj and x because one 
points to the other
>>> x in gc.get_referents(new_obj)
True
# Good!
>>> class A:
...   def __del__(self):
...  print("Ouch")
... 

>>> x_w = weakref.ref(x)
# Create a reference cycle  a -> new_obj -> x -> a
>>> a = A()
>>> x.a = a
>>> a.y = new_obj
>>> a.x = x
>>> gc.collect()
Traversed!
Traversed!
36
>>> del x,a
>>> gc.collect()
Traversed!
Traversed!
0
>>> sys.getrefcount(x_w())
6
>>> del new_obj
# At this point all variables are gone and the collector should clean everything
>>> gc.collect()
Ouch
8
# Nice, the collector cleaned the cycle


---

So the conclusion:

* This bug affects all types but is only really relevant for types that are not 
eternal (because eternal types are already "leaked").
* The only real problem and leaks will appear for heap types that are not 
eternal with custom traverse functions.
* After commit 364f0b0f19cc3f0d5e63f571ec9163cf41c62958 now all heap types that 
are not eternal, for instance 
  the ones created with PyType_FromSpec, need to traverse the type because they 
own it. Falining to do this can create leaks in the GC.
* The *correct* thing to do is modify the tp_traverse of all non-eternal heap 
types, but sadly the only way to do this in a backwards-compatible
  without modifying all user functions is injecting automatically the behaviour 
as PR 19414 is doing.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39481] Implement PEP 585 (Type Hinting Generics In Standard Collections)

[Ethan Smith]

Change by Ethan Smith :


--
pull_requests: +18785
pull_request: https://github.com/python/cpython/pull/19427

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40170] [C API] Make PyTypeObject structure an opaque structure in the public C API

[STINNER Victor]

STINNER Victor  added the comment:


New changeset a15e260b708a98edaba86a2aa663c3f6b2abc964 by Victor Stinner in 
branch 'master':
bpo-40170: Add _PyIndex_Check() internal function (GH-19426)
https://github.com/python/cpython/commit/a15e260b708a98edaba86a2aa663c3f6b2abc964


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40170] [C API] Make PyTypeObject structure an opaque structure in the public C API

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 45ec5b99aefa54552947049086e87ec01bc2fc9a by Victor Stinner in 
branch 'master':
bpo-40170: PyType_HasFeature() now always calls PyType_GetFlags() (GH-19378)
https://github.com/python/cpython/commit/45ec5b99aefa54552947049086e87ec01bc2fc9a


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40170] [C API] Make PyTypeObject structure an opaque structure in the public C API

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +18784
pull_request: https://github.com/python/cpython/pull/19426

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19468] Relax the type restriction on reloaded modules

[Furkan Onder]

Change by Furkan Onder :


--
keywords: +patch
nosy: +furkanonder
nosy_count: 3.0 -> 4.0
pull_requests: +18783
stage: needs patch -> patch review
pull_request: https://github.com/python/cpython/pull/19424

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39481] Implement PEP 585 (Type Hinting Generics In Standard Collections)

[Ethan Smith]

Change by Ethan Smith :


--
pull_requests: +18782
pull_request: https://github.com/python/cpython/pull/19425

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39481] Implement PEP 585 (Type Hinting Generics In Standard Collections)

[Batuhan Taskaya]

Change by Batuhan Taskaya :


--
pull_requests: +18781
pull_request: https://github.com/python/cpython/pull/19423

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40170] [C API] Make PyTypeObject structure an opaque structure in the public C API

[STINNER Victor]

STINNER Victor  added the comment:


New changeset ef5c615f5ae72c4f6979159c94da46afefbfab9a by Victor Stinner in 
branch 'master':
bpo-40170: Convert PyObject_CheckBuffer() macro to a function (GH-19376)
https://github.com/python/cpython/commit/ef5c615f5ae72c4f6979159c94da46afefbfab9a


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40091] Crash in logging._after_at_fork_child_reinit_locks()

[STINNER Victor]

STINNER Victor  added the comment:

> because I encountered a problem where a standard library lock was held by a 
> parent process at the time that child processes were forked, so the child 
> processes got stuck behind the inherited held locks.

Which lock from which module? You wrote details at:
https://github.com/python/cpython/pull/19195#issuecomment-609583084

According to your comment #28 at https://bugs.launchpad.net/nova/+bug/1844929 
the lock involved in the issue comes from _TransactionFactory of oslo.db:
https://github.com/openstack/oslo.db/blob/b903d4e1ee07ef2ec454daa5b8418b3039e02774/oslo_db/sqlalchemy/enginefacade.py#L189

So it's a bug in oslo.db, not in Python. Python doesn't provide any machinery 
to automatically reinitialize all locks created in Python at fork in the child 
process. os.register_at_fork() must be used explicitly.


> But, if I'm understanding correctly, this issue is fixing something in python 
> logging specifically and not all standard library locks in general.

This issue is specific to logging.


> My question is, will there be a way to reinit standard library locks in 
> general using _at_fork_reinit()? That is, should we expect a future fix in 
> python to do this or is the recommendation to continue to ensure the 
> application reinits locks during process start if we know the process could 
> be a child?

Each module has to setup an os.register_at_fork() callback to reinitialize its 
locks. It's done by threading and logging modules for example. The 
multiprocessing has its own util.register_after_fork() machinery (see 
bpo-40221)..

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40153] json dump with repeated key

[Facundo Batista]

Facundo Batista  added the comment:

It's a theoretical issue, I didn't hit it myself.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40221] Use new _at_fork_reinit() lock method in multiprocessing

[STINNER Victor]

New submission from STINNER Victor :

Currently, _ResourceSharer._after_fork() of multiprocessing.resource_sharer 
creates new locks and leak old locks on purpose. This method can benefit of the 
newly added _at_fork_reinit() method added by bpo-40089.

Queue._after_fork() could also call self._notempty._at_fork_reinit().

Also: ForkAwareThreadLock could reinitializes its lock.

--
components: Library (Lib)
messages: 365957
nosy: vstinner
priority: normal
severity: normal
status: open
title: Use new _at_fork_reinit() lock method in multiprocessing
versions: Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39481] Implement PEP 585 (Type Hinting Generics In Standard Collections)

[Ethan Smith]

Change by Ethan Smith :


--
pull_requests: +18780
pull_request: https://github.com/python/cpython/pull/19422

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39481] Implement PEP 585 (Type Hinting Generics In Standard Collections)

[Ethan Smith]

Change by Ethan Smith :


--
pull_requests: +18779
pull_request: https://github.com/python/cpython/pull/19421

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40170] [C API] Make PyTypeObject structure an opaque structure in the public C API

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 9205520d8c43488696d66cbdd9aefbb21871c508 by Victor Stinner in 
branch 'master':
bpo-40170: PyObject_NEW() becomes an alias to PyObject_New() (GH-19379)
https://github.com/python/cpython/commit/9205520d8c43488696d66cbdd9aefbb21871c508


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40214] test_ctypes.test_load_dll_with_flags Windows failure

[Steve Dower]

Steve Dower  added the comment:

Found it (and it's kind-of us):

Checking C:\Program Files\Amazon\AWSCLIV2\
*** Found at C:\Program Files\Amazon\AWSCLIV2\sqlite3.dll
*** Found at C:\Program Files\Amazon\AWSCLIV2\_sqlite3.pyd

But I'm not sure why that is getting loaded earlier than the current directory. 
Is that the behaviour we went for here?

(FWIW, we build and test a release build, not a debug build, which is why we're 
not looking for sqlite3_d.dll... but perhaps we should be using a debug build? 
That might be slower, but the extra validation is probably worthwhile...)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39481] Implement PEP 585 (Type Hinting Generics In Standard Collections)

[Batuhan Taskaya]

Change by Batuhan Taskaya :


--
pull_requests: +18778
pull_request: https://github.com/python/cpython/pull/19418

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40214] test_ctypes.test_load_dll_with_flags Windows failure

[Steve Dower]

Steve Dower  added the comment:

It might also be a "sqlite3_d.dll". Updated build is 
https://dev.azure.com/Python/cpython/_build/results?buildId=60787=results

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40214] test_ctypes.test_load_dll_with_flags Windows failure

[Steve Dower]

Steve Dower  added the comment:

It's probably the sqlite3.dll dependency that's failing, not the _sqlite3.dll 
(originally _sqlite3.pyd) one.

The test is verifying that dependent DLLs are loaded correctly. So I assume in 
this case the _sqlite3.dll is being loaded, but it's finding the wrong 
sqlite3.dll.

Since I could, I updated Zach's PR to check for other sqlite3.dll files. The 
new build should be at 
https://dev.azure.com/Python/cpython/_build/results?buildId=60785=logs=c83831cd-3752-5cc7-2f01-8276919eb334
 in a few minutes.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40091] Crash in logging._after_at_fork_child_reinit_locks()

[melwitt]

melwitt  added the comment:

Hi, I've been following the work related to:

https://bugs.python.org/issue6721
https://bugs.python.org/issue40089

because I encountered a problem where a standard library lock was held by a 
parent process at the time that child processes were forked, so the child 
processes got stuck behind the inherited held locks.

But, if I'm understanding correctly, this issue is fixing something in python 
logging specifically and not all standard library locks in general.

My question is, will there be a way to reinit standard library locks in general 
using _at_fork_reinit()? That is, should we expect a future fix in python to do 
this or is the recommendation to continue to ensure the application reinits 
locks during process start if we know the process could be a child?

Thanks for your advice.

--
nosy: +melwitt

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39481] Implement PEP 585 (Type Hinting Generics In Standard Collections)

[Ethan Smith]

Change by Ethan Smith :


--
pull_requests: +18777
pull_request: https://github.com/python/cpython/pull/19417

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39983] test.regrtest: test marked as failed (env changed), but no warning: test_multiprocessing_forkserver

[STINNER Victor]

STINNER Victor  added the comment:

Another example with test_asyncio on s390x RHEL7 3.x:
https://buildbot.python.org/all/#/builders/320/builds/410

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40220] Modules/gcmodule.c:434: update_refs: Assertion "gc_get_refs(gc) != 0" failed

[STINNER Victor]

STINNER Victor  added the comment:

Gregory: is this worker sick? Does it always rebuild from scratch?

--
nosy: +gregory.p.smith

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39481] Implement PEP 585 (Type Hinting Generics In Standard Collections)

[Ethan Smith]

Change by Ethan Smith :


--
nosy: +ethan smith

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39481] Implement PEP 585 (Type Hinting Generics In Standard Collections)

[Guido van Rossum]

Guido van Rossum  added the comment:


New changeset f9dd51e7db27d04e0b716d41a2804d5acbf145d1 by Batuhan Taşkaya in 
branch 'master':
bpo-39481: Make os.DirEntry generic (GH-19415)
https://github.com/python/cpython/commit/f9dd51e7db27d04e0b716d41a2804d5acbf145d1


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40089] Add _at_fork_reinit() method to locks

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 6318e45bda6c37d5497f33a6039cdb65aa494c93 by Miss Islington (bot) 
in branch '3.8':
bpo-40089: Fix threading._after_fork() (GH-19191) (GH-19194)
https://github.com/python/cpython/commit/6318e45bda6c37d5497f33a6039cdb65aa494c93


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40089] Add _at_fork_reinit() method to locks

[STINNER Victor]

STINNER Victor  added the comment:


New changeset a514ccb3ea6f01fef850d9465b82a1670d5ace44 by Miss Islington (bot) 
in branch '3.7':
bpo-40089: Fix threading._after_fork() (GH-19191) (GH-19193)
https://github.com/python/cpython/commit/a514ccb3ea6f01fef850d9465b82a1670d5ace44


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40091] Crash in logging._after_at_fork_child_reinit_locks()

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +18776
pull_request: https://github.com/python/cpython/pull/19416

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40220] Modules/gcmodule.c:434: update_refs: Assertion "gc_get_refs(gc) != 0" failed

[STINNER Victor]

STINNER Victor  added the comment:

Other errors in the same build:

0:14:10 load avg: 7.70 [392/420/2] test_http_cookiejar crashed (Exit code -6) 
-- running: test_buffer (1 min 9 sec), test_pickle (57.5 sec), test_asyncio (3 
min 47 sec), test_tarfile (36.8 sec), test_unparse (46.4 sec)
./Include/object.h:492: _Py_NegativeRefcount: Assertion failed: object has 
negative ref count
Enable tracemalloc to get the memory block allocation traceback

object address  : 0xb53dddc0
object refcount : -1
object type : 0x772950
object type name: str
object repr : 

Fatal Python error: _PyObject_AssertFailed: _PyObject_AssertFailed
Python runtime state: initialized

Current thread 0xb6f1b010 (most recent call first):
  File 
"/ssd/buildbot/buildarea/3.x.gps-ubuntu-exynos5-armv7l/build/Lib/test/libregrtest/runtest.py",
 line 193 in runtest
  File 
"/ssd/buildbot/buildarea/3.x.gps-ubuntu-exynos5-armv7l/build/Lib/test/libregrtest/runtest_mp.py",
 line 80 in run_tests_worker
  File 
"/ssd/buildbot/buildarea/3.x.gps-ubuntu-exynos5-armv7l/build/Lib/test/libregrtest/main.py",
 line 654 in _main
  File 
"/ssd/buildbot/buildarea/3.x.gps-ubuntu-exynos5-armv7l/build/Lib/test/libregrtest/main.py",
 line 634 in main
  File 
"/ssd/buildbot/buildarea/3.x.gps-ubuntu-exynos5-armv7l/build/Lib/test/libregrtest/main.py",
 line 712 in main
  File 
"/ssd/buildbot/buildarea/3.x.gps-ubuntu-exynos5-armv7l/build/Lib/test/regrtest.py",
 line 43 in _main
  File 
"/ssd/buildbot/buildarea/3.x.gps-ubuntu-exynos5-armv7l/build/Lib/test/regrtest.py",
 line 47 in 
  File 
"/ssd/buildbot/buildarea/3.x.gps-ubuntu-exynos5-armv7l/build/Lib/runpy.py", 
line 87 in _run_code
  File 
"/ssd/buildbot/buildarea/3.x.gps-ubuntu-exynos5-armv7l/build/Lib/runpy.py", 
line 197 in _run_module_as_main

(...)

0:14:26 load avg: 7.84 [402/420/3] test_unicode crashed (Exit code -6) -- 
running: test_buffer (1 min 25 sec), test_pickle (1 min 13 sec), test_asyncio 
(4 min 3 sec), test_unparse (1 min 2 sec)
Objects/unicodeobject.c:559: _PyUnicode_CheckConsistency: Assertion failed: 
compact->wstr_length == ascii->length
Enable tracemalloc to get the memory block allocation traceback

object address  : 0xb12af060
object refcount : 1
object type : 0x76d950
object type name: str
object repr : Objects/unicodeobject.c:559: _PyUnicode_CheckConsistency: 
Assertion failed: compact->wstr_length == ascii->length
Enable tracemalloc to get the memory block allocation traceback

object address  : 0xb12af060
object refcount : 1
object type : 0x76d950
object type name: str
object repr : Objects/unicodeobject.c:559: _PyUnicode_CheckConsistency: 
Assertion failed: compact->wstr_length == ascii->length
Enable tracemalloc to get the memory block allocation traceback

(... same error dozens of times ...)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40089] Add _at_fork_reinit() method to locks

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 87255be6964979b5abdc4b9dcf81cdcfdad6e753 by Victor Stinner in 
branch 'master':
bpo-40089: Add _at_fork_reinit() method to locks (GH-19195)
https://github.com/python/cpython/commit/87255be6964979b5abdc4b9dcf81cdcfdad6e753


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40220] Modules/gcmodule.c:434: update_refs: Assertion "gc_get_refs(gc) != 0" failed

[STINNER Victor]

New submission from STINNER Victor :

ARMv7 Debian buster 3.x:
https://buildbot.python.org/all/#/builders/168/builds/688

0:12:12 load avg: 9.96 [332/420] test_multiprocessing_spawn passed (8 min) -- 
running: test_zipfile (1 min 11 sec), test_largefile (1 min 25 sec), 
test_weakref (1 min 17 sec), test_asyncio (1 min 49 sec), 
test_concurrent_futures (4 min 16 sec)
./Include/object.h:492: _Py_NegativeRefcount: Assertion failed: object has 
negative ref count
Enable tracemalloc to get the memory block allocation traceback

object address  : 0xb5ef05a0
object refcount : -1
object type : 0x6f9c6c
object type name: method_descriptor
object repr : 

Fatal Python error: _PyObject_AssertFailed: _PyObject_AssertFailed
Python runtime state: finalizing (tstate=0x1e11c50)

Current thread 0xb6f44010 (most recent call first):

/ssd/buildbot/buildarea/3.x.gps-ubuntu-exynos5-armv7l/build/Lib/multiprocessing/resource_tracker.py:216:
 UserWarning: resource_tracker: There appear to be 1 leaked shared_memory 
objects to clean up at shutdown
  warnings.warn('resource_tracker: There appear to be %d '
/ssd/buildbot/buildarea/3.x.gps-ubuntu-exynos5-armv7l/build/Lib/multiprocessing/resource_tracker.py:229:
 UserWarning: resource_tracker: '/psm_eec4c151': [Errno 2] No such file or 
directory: '/psm_eec4c151'
  warnings.warn('resource_tracker: %r: %s' % (name, e))

(...)

0:13:03 load avg: 8.51 [355/420/1] test_concurrent_futures failed (5 min 5 sec) 
-- running: test_asyncio (2 min 40 sec), test_compileall (33.7 sec)
Modules/gcmodule.c:434: update_refs: Assertion "gc_get_refs(gc) != 0" failed
Enable tracemalloc to get the memory block allocation traceback

object address  : 0xb63581c0
object refcount : 0
object type : 0x69d1d0
object type name: tuple
object repr : 

Fatal Python error: _PyObject_AssertFailed: _PyObject_AssertFailed
Python runtime state: initialized

Current thread 0xb6fb3010 (most recent call first):
  File 
"/ssd/buildbot/buildarea/3.x.gps-ubuntu-exynos5-armv7l/build/Lib/typing.py", 
line 762 in __setattr__
  File 
"/ssd/buildbot/buildarea/3.x.gps-ubuntu-exynos5-armv7l/build/Lib/typing.py", 
line 658 in __init__
  File 
"/ssd/buildbot/buildarea/3.x.gps-ubuntu-exynos5-armv7l/build/Lib/typing.py", 
line 1549 in _alias
  File 
"/ssd/buildbot/buildarea/3.x.gps-ubuntu-exynos5-armv7l/build/Lib/typing.py", 
line 1607 in 
  File "", line 228 in _call_with_frames_removed
  File "", line 790 in exec_module
  File "", line 680 in _load_unlocked
  File "", line 986 in _find_and_load_unlocked
  File "", line 1007 in _find_and_load
  File 
"/ssd/buildbot/buildarea/3.x.gps-ubuntu-exynos5-armv7l/build/Lib/asyncio/staggered.py",
 line 6 in 
  File "", line 228 in _call_with_frames_removed
  File "", line 790 in exec_module
  File "", line 680 in _load_unlocked
  File "", line 986 in _find_and_load_unlocked
  File "", line 1007 in _find_and_load
  File "", line 228 in _call_with_frames_removed
  File "", line 1058 in _handle_fromlist
  File 
"/ssd/buildbot/buildarea/3.x.gps-ubuntu-exynos5-armv7l/build/Lib/asyncio/base_events.py",
 line 45 in 
  File "", line 228 in _call_with_frames_removed
  File "", line 790 in exec_module
  File "", line 680 in _load_unlocked
  File "", line 986 in _find_and_load_unlocked
  File "", line 1007 in _find_and_load
  File 
"/ssd/buildbot/buildarea/3.x.gps-ubuntu-exynos5-armv7l/build/Lib/asyncio/__init__.py",
 line 8 in 
  File "", line 228 in _call_with_frames_removed
  File "", line 790 in exec_module
  File "", line 680 in _load_unlocked
  File "", line 986 in _find_and_load_unlocked
  File "", line 1007 in _find_and_load
  File "", line 228 in _call_with_frames_removed
  File "", line 972 in _find_and_load_unlocked
  File "", line 1007 in _find_and_load
  File 
"/ssd/buildbot/buildarea/3.x.gps-ubuntu-exynos5-armv7l/build/Lib/test/support/__init__.py",
 line 6 in 
  File "", line 228 in _call_with_frames_removed
  File "", line 790 in exec_module
  File "", line 680 in _load_unlocked
  File "", line 986 in _find_and_load_unlocked
  File "", line 1007 in _find_and_load
  File "", line 228 in _call_with_frames_removed
  File "", line 1058 in _handle_fromlist
  File 
"/ssd/buildbot/buildarea/3.x.gps-ubuntu-exynos5-armv7l/build/Lib/test/libregrtest/cmdline.py",
 line 4 in 
  File "", line 228 in _call_with_frames_removed
  File "", line 790 in exec_module
  File "", line 680 in _load_unlocked
  File "", line 986 in _find_and_load_unlocked
  File "", line 1007 in _find_and_load
  File 
"/ssd/buildbot/buildarea/3.x.gps-ubuntu-exynos5-armv7l/build/Lib/test/libregrtest/__init__.py",
 line 1 in 
  File "", line 228 in _call_with_frames_removed
  File "", line 790 in exec_module
  File "", line 680 in _load_unlocked
  File "", line 986 in _find_and_load_unlocked
  File "", line 1007 in _find_and_load
  File 
"/ssd/buildbot/buildarea/3.x.gps-ubuntu-exynos5-armv7l/build/Lib/test/regrtest.py",
 line 11 in 
  File 
"/ssd/buildbot/buildarea/3.x.gps-ubuntu-exynos5-armv7l/build/Lib/runpy.py", 

[issue39481] Implement PEP 585 (Type Hinting Generics In Standard Collections)

[Guido van Rossum]

Guido van Rossum  added the comment:

Ethan Smith produced a list of types that are Generic in typeshed but not in 
the stdlib. So these could be added.

https://github.com/gvanrossum/cpython/pull/1#issuecomment-582781121

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39481] Implement PEP 585 (Type Hinting Generics In Standard Collections)

[Batuhan Taskaya]

Change by Batuhan Taskaya :


--
pull_requests: +18775
pull_request: https://github.com/python/cpython/pull/19415

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39019] Missing class getitems in standard library classes

[Batuhan Taskaya]

Batuhan Taskaya  added the comment:

> Hold on, os.DirEntry[str] still doesn't work.

That is what I asked on the issue 39481. I couldn't find anything about its 
cover on PEP 585.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39019] Missing class getitems in standard library classes

[Guido van Rossum]

Guido van Rossum  added the comment:

Hold on, os.DirEntry[str] still doesn't work.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40219] ttk LabeledScale: label covered by hidden element

[Stephen Bell]

Change by Stephen Bell :


--
type:  -> behavior

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40219] ttk LabeledScale: label covered by hidden element

[Stephen Bell]

New submission from Stephen Bell :

The LabeledScale in tkinter.ttk seems to have some kind of hidden element that 
covers the LabeledScale's label when the value is set to mid-scale. Tested on 
Windows 10, Python 3.6

See below code to reproduce:

import tkinter
from tkinter import ttk

master = tkinter.Tk()
_out1Value = tkinter.IntVar(master)
out1Slider = ttk.LabeledScale(master, from_=-100, to=100, variable=_out1Value, 
compound="bottom")
_out1Value.set(0)

# uncomment to "fix"
# out1Slider.label.lift()

out1Slider.pack()

master.mainloop()

--
components: Tkinter
messages: 365940
nosy: Stephen Bell
priority: normal
severity: normal
status: open
title: ttk LabeledScale: label covered by hidden element
versions: Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40153] json dump with repeated key

[Vedran Čačić]

Vedran Čačić  added the comment:

JSON is JavaScript Object Notation, that is, a notation for JS Objects. Python 
dicts are much more general than that (not only in keys, but in values too: 
JSON keys must be strings, and values must be Strings, Numbers, Booleans, 
Arrays, (JS) Objects, or nulls -- both are restricted compared to Python 
dicts). There is really no reason to expect full embedding.

--
nosy: +veky

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40149] test_threading leaked [38, 38, 38] references, sum=114

[STINNER Victor]

STINNER Victor  added the comment:

> bpo-40149: Implement traverse in _abc._abc_data (GH-19412)

Pablo told me that this change is not correct: the overriden traverse function 
must call PyType_Type.tp_traverse (parent method).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39481] Implement PEP 585 (Type Hinting Generics In Standard Collections)

[Serhiy Storchaka]

Change by Serhiy Storchaka :


--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39019] Missing class getitems in standard library classes

[Batuhan Taskaya]

Change by Batuhan Taskaya :


--
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39019] Missing class getitems in standard library classes

[Batuhan Taskaya]

Batuhan Taskaya  added the comment:

PEP 585 is landed, closing the issue (and linked PRs)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39481] Implement PEP 585 (Type Hinting Generics In Standard Collections)

[Batuhan Taskaya]

Batuhan Taskaya  added the comment:

@gvanrossum is new types going to support generic alias protocol or this subset 
will be kept? Like typeshed uses os.DirEntry as a generic, but in reality it is 
not. (https://github.com/python/cpython/pull/17561)

--
nosy: +BTaskaya

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40218] sys.executable is a non existing path if python is executed from gdb

[Volker Weißmann]

Volker Weißmann  added the comment:

The gdb issue is here: https://sourceware.org/bugzilla/show_bug.cgi?id=25800

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40218] sys.executable is a non existing path if python is executed from gdb

[Volker Weißmann]

New submission from Volker Weißmann :

Note: I'm not sure if this is a bug in python or in gdb. I will also submit a 
bug report to gdb and post a link here.

Pythons documentation says that sys.executable is always either None, empty 
string or a path to the python interpreter. Using gdb and python, we can 
produce situations where this is not true.

Simple but unrealistic way to reproduce this bug:
Install gdb with python support. E.g using
$ pacman -S gdb
Remove all the python binaries:
$ rm /usr/bin/python
$ rm /usr/bin/python3
$ rm /usr/bin/python3.8
Run $gdb -x gdbinit$ with the contents of gdbinit being:
python
import sys
import os
print(sys.executable)
print(os.path.exists(sys.executable))
end

Result:
/usr/bin/python
False

Here, sys.executable is /usr/bin/python, but there is no python executable in 
/usr/bin/python, because we just deleted it.

Complicated but realistic way to reproduce this bug:
Build gdb with
../configure --with-python=python2
and run gdb with the gdbinit being:
python
import sys
print(sys.executable)
print(sys.version)
end
Result:
/usr/bin/python
2.7.17 (default, Mar 21 2020, 00:47:07) 
[GCC 9.3.0]
Here it says that the python2 executable lies in "/usr/bin/python", even if 
there is no python2 executable in /usr/bin/python.

--
messages: 365934
nosy: Volker Weißmann
priority: normal
severity: normal
status: open
title: sys.executable is a non existing path if python is executed from gdb

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40217] The garbage collector doesn't take in account that objects of heap allocated types hold a strong reference to their type

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

In https://github.com/python/cpython/pull/19414 I have attached a proof of 
concept of a wrapper for tp_traverse that only affects types created by 
PyType_FromSpec that will call Py_VISIT(Py_TYPE(self)) and then the user 
function.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40217] The garbage collector doesn't take in account that objects of heap allocated types hold a strong reference to their type

[Pablo Galindo Salgado]

Change by Pablo Galindo Salgado :


--
keywords: +patch
pull_requests: +18774
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/19414

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40217] The garbage collector doesn't take in account that objects of heap allocated types hold a strong reference to their type

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

> Now we changed rules. A strong reference is created implicitly. Who is 
> responsible to manage a strong reference? Whose who created it, ant it is the 
> interpreter, not the user.

Even if we decide that the patch that introduced the new rules should not be 
reverted, then what we should do is wrap the tp_traverse of the user in 
something that also calls Py_VISIT(Py_TYPE(self)) so basically the tp_traverse 
of the type created by PyType_FromSpec will do

static int
PyType_FromSpec_tp_traverse(_abc_data *self, visitproc visit, void *arg)
{
Py_VISIT(Py_TYPE(self))
return self->user_provided_tp_traverse(self, visit, arg);
}

That way, we can still reason about what the tp_traverse should do, we don't 
break more rules and we don't need to make maintaining the GC even more 
difficult.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40217] The garbage collector doesn't take in account that objects of heap allocated types hold a strong reference to their type

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

The problem is that we suddenly changed rules. It was not required that the 
object's type should be visited in tp_visit. It was incorrect to visit it, 
because object did not have strong reference to its type. User never created 
it, and it was not created implicitly.

Now we changed rules. A strong reference is created implicitly. Who is 
responsible to manage a strong reference? Whose who created it, ant it is the 
interpreter, not the user. User does not know anything about it. If we pass the 
responsibility for the strong reference to the type on the user, we makes all 
user code incorrect, and we add a burden of fixing it and maintaining 
compatibility with incompatible Python versions on the user. I think it is very 
bad.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40217] The garbage collector doesn't take in account that objects of heap allocated types hold a strong reference to their type

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

> We cannot change all user code, so we should change the interpreter code so 
> that it will work correctly with existing user code.

If we made a change that make all user code suddenly incorrect, that change 
should be reverted. The GC has clear rules about what tp_traverse should and 
should not do, and we should not violate those rules and make special cases in 
the gc just because we forced some classes to be incorrect. This will make much 
more difficult to reason about GC bugs, the tp_traverse implementation of 
classes and much difficult to maintain the GC itself.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40217] The garbage collector doesn't take in account that objects of heap allocated types hold a strong reference to their type

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

Recently many static allocated types were converted to heap allocated types 
(using PyType_FromSpec). Now you need to add Py_VISIT(Py_TYPE(self)) in all 
corresponding tp_visit implementations.

And since even official example for heap allocated types do not contain it (and 
it was not needed before 3.9), I am sure that all existing code do not contain 
it.

We cannot change all user code, so we should change the interpreter code so 
that it will work correctly with existing user code.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40217] The garbage collector doesn't take in account that objects of heap allocated types hold a strong reference to their type

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

> Hummm, I think we may just be missing a Py_VISIT(Py_TYPE(self))here:

Checking it more closely, that is incorrect, so we are not missing that 
visitation :(

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40217] The garbage collector doesn't take in account that objects of heap allocated types hold a strong reference to their type

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

Hummm, I think we may just be missing a Py_VISIT(Py_TYPE(self))here:

https://github.com/python/cpython/blob/master/Objects/typeobject.c#L3562

A class owns references to its type (type) or another metaclass

Tim, could you confirm that that is the case?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40217] The garbage collector doesn't take in account that objects of heap allocated types hold a strong reference to their type

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

Also, as I mentioned, you don't need to modify all objects tp_traverse, only 
it's type.tp_traverse slot. For instance, all python objects know how to 
traverse stuff because they share the same tp_traverse:

https://github.com/python/cpython/blob/master/Objects/typeobject.c#L1082

So unless I am missing something, if you want to affect all heap types you just 
need to modify one tp_traverse in one place: the superclass.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40217] The garbage collector doesn't take in account that objects of heap allocated types hold a strong reference to their type

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

Interesting, this issue may be related to issue24379. The problem with the 
proposed implementation of subscript was that it created a reference loop, and 
not all links in this loop were visible by GC.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40217] The garbage collector doesn't take in account that objects of heap allocated types hold a strong reference to their type

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

>>> Since tp_visit is GC specific thing, I think it wold be more convenient 
>>> make a special case for object types.

I don't think that follows: The gc defers the logic of what and what should not 
be visited to the tp_traverse implementation of every object. The GC must be 
agnostic of the types it receives and heap types here are not special.

--

Also, if we make an exception in the GC for this special case, all tp_traverse 
of all those functions will be incorrect. Someone reading those tp_traverse can 
say "Oh, why these functions do not visit the type if they own s reference to 
it, this looks incorrect" and then we need to explain that that is an exception 
in the GC just because we were lazy to implement them correctly.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40217] The garbage collector doesn't take in account that objects of heap allocated types hold a strong reference to their type

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

It would be inconvenient to require adding Py_VISIT(Py_TYPE(self)) in all 
tp_visit implementations of heap allocated types (including third-party 
extensions). Since tp_visit is GC specific thing, I think it wold be more 
convenient make a special case for object types.

--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40217] The garbage collector doesn't take in account that objects of heap allocated types hold a strong reference to their type

[Tim Peters]

Tim Peters  added the comment:

If object A owns a strong reference to object B, and A participates in cyclic 
gc, and B may be part of a cycle, then it's necessary and sufficient that A's 
type's tp_traverse implementation invoke Py_VISIT() passing A's pointer to B.

It would be a Really Bad Idea to add special cases to the gc module to spare 
some specific type(s) from following that (currently) utterly uniform rule.

--
nosy: +tim.peters

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40217] The garbage collector doesn't take in account that objects of heap allocated types hold a strong reference to their type

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

Also, heap types (created with type_new()) are already taking into account the 
type when visiting references:

https://github.com/python/cpython/blob/master/Objects/typeobject.c#L

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23224] bz2/lzma: Compressor/Decompressor objects are only initialized in __init__

[Dong-hee Na]

Dong-hee Na  added the comment:

The issue is not solved yet.
@ZackerySpytz would you like to finalize this issue?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23224] bz2/lzma: Compressor/Decompressor objects are only initialized in __init__

[Dong-hee Na]

Change by Dong-hee Na :


--
nosy: +corona10

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23224] bz2/lzma: Compressor/Decompressor objects are only initialized in __init__

[Dong-hee Na]

Change by Dong-hee Na :


--
versions: +Python 3.9 -Python 2.7, Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40214] test_ctypes.test_load_dll_with_flags Windows failure

[Zachary Ware]

Zachary Ware  added the comment:

Assuming I implemented my checks correctly (see PR19413), I think I've just 
debunked both of our leading theories.

Results here: 
https://dev.azure.com/Python/cpython/_build/results?buildId=60764=logs=d554cd63-f8f4-5b2d-871b-33e4ea76e915=5a14d0eb-dbd4-5b80-f5d0-7909f950a1cc

--
stage: patch review -> needs patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40149] test_threading leaked [38, 38, 38] references, sum=114

[hai shi]

Change by hai shi :


--
nosy: +shihai1991

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40217] The garbage collector doesn't take in account that objects of heap allocated types hold a strong reference to their type

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

> I propose to modify the GC to take bpo-35810 in account.

What? The GC is agnostic of what it receives. It works with objects in general 
that implement the gc support, but it does not care about what those objects 
are. The only specal case are weakreferences and because those have inherit GC 
semantics.

I am not sure about what you are proposing, could you elaborate?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40112] AIX: xlc - default path changed and no longer recognized

[STINNER Victor]

STINNER Victor  added the comment:

The CI failed on my backport to 3.8: PR 19408.

The CI failed because of bpo-40204.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39481] Implement PEP 585 (Type Hinting Generics In Standard Collections)

[Guido van Rossum]

Guido van Rossum  added the comment:

The base implementation has landed. We still need docs, and I'm sure that the 
alpha and beta release cycle will find small things that need to be improved.

Perhaps the next priority is an update for Doc/whatsnew/3.9.rst.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40149] test_threading leaked [38, 38, 38] references, sum=114

[Dong-hee Na]

Dong-hee Na  added the comment:

Wow Thank you for the summary :)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40149] test_threading leaked [38, 38, 38] references, sum=114

[STINNER Victor]

STINNER Victor  added the comment:

In _abcmodule_exec(), when _abc_data type is created, it's created with 
refcnt=3:
* 1 strong reference (normal)
* +1 ref from tp_dict['__new__'] slot
* +1 ref from tp_mro

type_traverse() visits tp_dict and tp_mro, so it's fine.

In Py_EndInterpreter(), PyInterpreterState_Clear() clears os.register_atfork() 
callbacks which were one of the last references to _abc_data type. The first 
call to _PyGC_CollectNoFail() destroys _abc_data *instances* but not the 
_abc_data type.

The following patch works around the issue:

diff --git a/Modules/_abc.c b/Modules/_abc.c
index 1efc98bf72..410dbcf96a 100644
--- a/Modules/_abc.c
+++ b/Modules/_abc.c
@@ -54,6 +54,7 @@ typedef struct {
 static int
 abc_data_traverse(_abc_data *self, visitproc visit, void *arg)
 {
+Py_VISIT(Py_TYPE(self));
 Py_VISIT(self->_abc_registry);
 Py_VISIT(self->_abc_cache);
 Py_VISIT(self->_abc_negative_cache);


$ ./python -m test -R 3:3 test_threading -m 
test.test_threading.SubinterpThreadingTests.test_threads_join_2
(...)
Tests result: SUCCESS


I'm not sure why Py_VISIT(Py_TYPE(self)) is needed. Maybe it's a regression 
caused by commit 364f0b0f19cc3f0d5e63f571ec9163cf41c62958 of bpo-35810.

It sems like the GC doesn't take in account that instances of types allocated 
on the heap (if type->tp_flags has the Py_TPFLAGS_HEAPTYPE flag) hold a strong 
refeference to the type (PyObject.ob_type).

I created bpo-40217: "The garbage collector doesn't take in account that 
objects of heap allocated types hold a strong reference to their type".

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39481] Implement PEP 585 (Type Hinting Generics In Standard Collections)

[Guido van Rossum]

Guido van Rossum  added the comment:


New changeset 48b069a003ba6c684a9ba78493fbbec5e89f10b8 by Guido van Rossum in 
branch 'master':
bpo-39481: Implementation for PEP 585 (#18239)
https://github.com/python/cpython/commit/48b069a003ba6c684a9ba78493fbbec5e89f10b8


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40217] The garbage collector doesn't take in account that objects of heap allocated types hold a strong reference to their type

[hai shi]

Change by hai shi :


--
nosy: +shihai1991

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40149] test_threading leaked [38, 38, 38] references, sum=114

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 9cc3ebd7e04cb645ac7b2f372eaafa7464e16b9c by Victor Stinner in 
branch 'master':
bpo-40149: Implement traverse in _abc._abc_data (GH-19412)
https://github.com/python/cpython/commit/9cc3ebd7e04cb645ac7b2f372eaafa7464e16b9c


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40214] test_ctypes.test_load_dll_with_flags Windows failure

[Zachary Ware]

Change by Zachary Ware :


--
pull_requests: +18773
stage: needs patch -> patch review
pull_request: https://github.com/python/cpython/pull/19413

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40217] The garbage collector doesn't take in account that objects of heap allocated types hold a strong reference to their type

[Dong-hee Na]

Change by Dong-hee Na :


--
nosy: +corona10

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40217] The garbage collector doesn't take in account that objects of heap allocated types hold a strong reference to their type

[STINNER Victor]

New submission from STINNER Victor :

The bpo-35810 modified the object allocate to hold a *strong* reference to the 
type in PyObject.ob_type, whereas PyObject.ob_type is a *borrowed* references 
if the type is statically allocated.

commit 364f0b0f19cc3f0d5e63f571ec9163cf41c62958
Author: Eddie Elizondo 
Date:   Wed Mar 27 07:52:18 2019 -0400

bpo-35810: Incref heap-allocated types in PyObject_Init (GH-11661)

* Incref heap-allocated types in PyObject_Init
* Add documentation and porting notes to What's New


The problem is now in some corner cases, the GC fails to visit all referrer of 
a type and so considers that the type is still alive.

bpo-40149 is a concrete example of such bug.

I propose to modify the GC to take bpo-35810 in account.

... or maybe I just misunderstood bpo-40149 bug :-)

--
components: Interpreter Core
messages: 365911
nosy: pablogsal, vstinner
priority: normal
severity: normal
status: open
title: The garbage collector doesn't take in account that objects of heap 
allocated types hold a strong reference to their type
versions: Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40149] test_threading leaked [38, 38, 38] references, sum=114

[STINNER Victor]

Change by STINNER Victor :


--
keywords: +patch
pull_requests: +18772
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/19412

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40214] test_ctypes.test_load_dll_with_flags Windows failure

[Eryk Sun]

Eryk Sun  added the comment:

> Maybe something else was installed that put an incompatible 
> _sqlite3.dll on PATH? Thereby proving the inherent risk of 
> using unsafe DLL load settings 

should_pass() sets the working directory of the test process to the `tmp` 
directory. The loader normally checks the working directory before PATH. But 
maybe the system itself is configured to disallow loading DLLs from the working 
directory. There's a registry setting for that, but it's little known and 
rarely used because it's disruptive in general.

--
nosy: +eryksun

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40216] Support --runstatedir in configure

[Batuhan Taskaya]

Change by Batuhan Taskaya :


--
keywords: +patch
pull_requests: +18771
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/19411

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40216] Support --runstatedir in configure

[Batuhan Taskaya]

New submission from Batuhan Taskaya :

AC allows to set runstatedir but looks like python's configure is a bit 
outdated, it requires to be regenerated.

--
components: Build
messages: 365909
nosy: BTaskaya
priority: normal
severity: normal
status: open
title: Support --runstatedir in configure
versions: Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39689] struct and memoryview tests rely on undefined behavior (as revealed by clang 9)

[Petr Viktorin]

Change by Petr Viktorin :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40060] socket.TCP_NOTSENT_LOWAT is missing in official macOS builds

[Ma Lin]

Ma Lin  added the comment:

It seems that people usually use the socket module like this, I think it's safe 
to respect this habit:

if hasattr(socket, "FLAG_NAME"):
do_something

If use PR19402, your program will have problem on the older version system, not 
only "don't break existing code".

So I think delete-at-runtime is a suitable way.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37207] Use PEP 590 vectorcall to speed up calls to range(), list() and dict()

[Petr Viktorin]

Petr Viktorin  added the comment:

As discussed briefly in Mark's PR, benchmarks like this are now slower:

ret = dict(**{'a': 2, 'b': 4, 'c': 6, 'd': 8})


Python 3.8: Mean +- std dev: 281 ns +- 9 ns
master: Mean +- std dev: 456 ns +- 14 ns

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37388] unknown error handlers should be reported early

[STINNER Victor]

STINNER Victor  added the comment:


New changeset d8acf0d9aae71d1897e8f91989bd8bfb4a9ef9c6 by Victor Stinner in 
branch 'master':
bpo-37388: Don't check encoding/errors during finalization (GH-19409)
https://github.com/python/cpython/commit/d8acf0d9aae71d1897e8f91989bd8bfb4a9ef9c6


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37388] unknown error handlers should be reported early

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +18770
pull_request: https://github.com/python/cpython/pull/19409

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5654] Add C hook in PyDict_SetItem for debuggers

[STINNER Victor]

STINNER Victor  added the comment:

No activity for 9 years, I close the issue.

--
resolution:  -> rejected
stage: needs patch -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40112] AIX: xlc - default path changed and no longer recognized

[miss-islington]

Change by miss-islington :


--
nosy: +miss-islington
nosy_count: 4.0 -> 5.0
pull_requests: +18769
pull_request: https://github.com/python/cpython/pull/19408

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26628] Undefined behavior calling C functions with ctypes.Union arguments

[Mats Wichmann]

Mats Wichmann  added the comment:

For readers who got here via a search after hitting the new traceback, note the 
fix in bpo-16575 was reverted. It's still a duplicate issue, so follow the 
progress there.

--
nosy: +mwichmann

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5654] Add C hook in PyDict_SetItem for debuggers

[Piotr Dobrogost]

Change by Piotr Dobrogost :


--
nosy: +piotr.dobrogost

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40204] Docs build error with Sphinx 3.0 due to invalid C declaration

[Maciej Olko]

Change by Maciej Olko :


--
nosy: +Maciej Olko

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40206] Multiplying 4.6*100 will result in 459.99999999999994

[Stefan Krah]

Stefan Krah  added the comment:

You can also set the decimal.FloatOperation trap to avoid accidental
errors:

>>> from decimal import *
>>> c = getcontext()
>>> Decimal(4.6) * 100
Decimal('459.9644728632120')

>>> c.traps[FloatOperation] = True
>>> Decimal(4.6) * 100
Traceback (most recent call last):
  File "", line 1, in 
decimal.FloatOperation: []

>>> Decimal("4.6") * 100
Decimal('460.0')
>>>

--
nosy: +skrah

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com