New submission from Allan Feldman <[email protected]>:
Our team is making use of a weakref.WeakValueDictionary() that is accessed
through the finalizer of a class. We observed that in Python 3 occasionally
values that are directly referenced by an object being finalized were missing
from the WeakValueDictionary.
Example:
import weakref
cache = weakref.WeakValueDictionary()
class Foo(object):
pass
class Bar(object):
def __init__(self, foo):
self.foo = foo
cache['foo'] = foo
def __del__(self):
del cache['foo']
bar = Bar(Foo())
del bar
Upon further investigation, we realized that this had to do with the weakref
callback within WeakValueDictionary being called (removing the key from the
dict) before the finalizer for Foo was called.
Reproduction:
import gc
import weakref
cache = weakref.WeakValueDictionary()
class Foo(object):
pass
class Bar(object):
def __init__(self, foo):
# Force a reference cycle to run del only on gc.collect
self._self = self
self.foo = foo
cache["foo"] = foo
def __del__(self):
# foo is missing from the cache because the weakref callback has
# already run. KeyError is raised.
del cache["foo"]
bar = Bar(Foo())
del bar
gc.collect()
Expected behavior:
The weakref callback should only be called when the object is known to be
deleted (after the finalizer runs). Running weakref callbacks before then means
that the weakref callback can run on objects being ressurected by the finalizer.
Example:
import gc
import weakref
class ForeverObject(object):
def __init__(self, circular):
# Introduce a circular reference so that gc must collect the object
if circular:
self._self = self
def __del__(self):
global o
o = self
def callback(wr):
print("callback running", wr)
for circular in (True, False):
print("------- Circular reference:", circular, "-------")
o = ForeverObject(circular)
wr = weakref.ref(o, callback)
del o
gc.collect()
print("--------------")
Note: Python 2.7 appears to have the opposite behavior - weakref callbacks are
only invoked when dealloc occurs outside of gc. The Python 2.7 behavior hasn't
yet been investigated.
If the expected behavior above is confirmed, I would be happy to submit a patch
for this issue!
----------
components: Interpreter Core
messages: 366675
nosy: a-feld
priority: normal
severity: normal
status: open
title: Weakref callbacks running before finalizers in GC collection
type: behavior
versions: Python 2.7, Python 3.5, Python 3.6, Python 3.7, Python 3.8, Python 3.9
_______________________________________
Python tracker <[email protected]>
<https://bugs.python.org/issue40312>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
