New submission from Allan Feldman <allan.d.feld...@gmail.com>:
Our team is making use of a weakref.WeakValueDictionary() that is accessed through the finalizer of a class. We observed that in Python 3 occasionally values that are directly referenced by an object being finalized were missing from the WeakValueDictionary. Example: import weakref cache = weakref.WeakValueDictionary() class Foo(object): pass class Bar(object): def __init__(self, foo): self.foo = foo cache['foo'] = foo def __del__(self): del cache['foo'] bar = Bar(Foo()) del bar Upon further investigation, we realized that this had to do with the weakref callback within WeakValueDictionary being called (removing the key from the dict) before the finalizer for Foo was called. Reproduction: import gc import weakref cache = weakref.WeakValueDictionary() class Foo(object): pass class Bar(object): def __init__(self, foo): # Force a reference cycle to run del only on gc.collect self._self = self self.foo = foo cache["foo"] = foo def __del__(self): # foo is missing from the cache because the weakref callback has # already run. KeyError is raised. del cache["foo"] bar = Bar(Foo()) del bar gc.collect() Expected behavior: The weakref callback should only be called when the object is known to be deleted (after the finalizer runs). Running weakref callbacks before then means that the weakref callback can run on objects being ressurected by the finalizer. Example: import gc import weakref class ForeverObject(object): def __init__(self, circular): # Introduce a circular reference so that gc must collect the object if circular: self._self = self def __del__(self): global o o = self def callback(wr): print("callback running", wr) for circular in (True, False): print("------- Circular reference:", circular, "-------") o = ForeverObject(circular) wr = weakref.ref(o, callback) del o gc.collect() print("--------------") Note: Python 2.7 appears to have the opposite behavior - weakref callbacks are only invoked when dealloc occurs outside of gc. The Python 2.7 behavior hasn't yet been investigated. If the expected behavior above is confirmed, I would be happy to submit a patch for this issue! ---------- components: Interpreter Core messages: 366675 nosy: a-feld priority: normal severity: normal status: open title: Weakref callbacks running before finalizers in GC collection type: behavior versions: Python 2.7, Python 3.5, Python 3.6, Python 3.7, Python 3.8, Python 3.9 _______________________________________ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue40312> _______________________________________ _______________________________________________ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com