New submission from Daniel Barkalow <barka...@iabervon.org>:

Nothing in the documentation gives you enough information to find what you're 
doing wrong if you have the following bug for an object tracked by the GC: (a) 
you store a borrowed reference to an object whose lifetime is always longer 
that the buggy object (and never decrement it); (b) you visit that object in 
your tp_traverse.

Despite the bug, everything works nearly all of the time. However, when it 
fails, the effect makes no sense from a documentation-only understanding of the 
garbage collection (unless Python is built with --with-assertions, perhaps). 
The only sequence in which it fails is:

* All of the objects that will reference the victim get moved into an older 
generation.
* The victim gets exactly as many buggy references from objects in its own 
generation as there are good references, and no good references from objects in 
its own generation.
* Garbage collection runs for only the younger generation.

At this point, the victim is marked as garbage despite the fact that there was 
a good object referencing it at all times.

Reading the Python source, it becomes clear that the garbage collector handles 
references from older generations not by traversing the objects in older 
generations, but by comparing the count of references from other young objects 
to the usual reference count. That is, visiting an object you don't hold a 
reference to may cause that object to get collected, rather than protecting it, 
even if there are other reasons not to collect it.

The best fix would probably be a warning in the documentation for tp_traverse 
that visiting an object you do not hold a strong reference to can cause 
inexplicable effects, because the intuitive guess would be that it could only 
cause memory leaks.

It would probably also be worth mentioning in the documentation of the garbage 
collector something about its algorithm, so people have a hint that references 
from older objects aren't necessarily sufficient to overcome buggy use of the C 
API.

----------
assignee: docs@python
components: C API, Documentation
messages: 372445
nosy: docs@python, iabervon
priority: normal
severity: normal
status: open
title: Insufficient description of cyclic garbage collector for C API
versions: Python 3.7, Python 3.8

_______________________________________
Python tracker <rep...@bugs.python.org>
<https://bugs.python.org/issue41133>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Reply via email to