New submission from Konrad Schwarz <[email protected]>:
I am seeing the following non-deterministic behavior:
My code processes DeviceTree, a tree-based specification format for hardware
descriptions, that includes cross-references ("phandles"). For all intents and
purposes, this format is similar to XML; phandles are analog to ID/IDREFS.
To prevent reference cycles and avoid the need for garbage collection, my code
uses weakref.proxy for parent pointers and weakref.ref for cross-references.
My goal is to provide a "projection" operation on a DeviceTree: creating
derived DeviceTrees that model subsets of the hardware (this is to partition
the hardware into multiple independent sub-machines). The projection is
specified by newly introduced nodes and attributes (aka properties) in the
tree; phandles are used to indicate which part belongs to which partition.
Python weak references provide a callback interface to indicate the demise of
their referents and my code uses that to prune the tree: e.g., if a node
modeling a partition is deleted, nodes that reference that node (i.e., indicate
they belong to that partition) are deleted in the corresponding weakref
callback. So technically, the code implicitly uses the interpreters list of
weak referrers (__weakref__) to find and execute code on them when the
referent's state changes.
This works exactly as envisioned when single-stepping in PDB.
When running at full speed however, I see that weak reference callbacks are
being triggered after the corresponding weak reference has been deleted with
del (the weak reference is a value of a Python dict holding a node's
attributes.)
I suspect that this is because of some batching or deferred processing in the
Python interpreter.
Ultimately, this is a violation of the semantics and must be classified as a
bug.
However, in my case, it would suffice to have a "memory barrier" type of
operation that flushes the queue of deferred deletions before continuing.
Something like that must exist, because single stepping in PDB is successful.
Initial tests of calling the garbage collector to this end were inconclusive,
unfortunately.
----------
components: Interpreter Core
messages: 387999
nosy: konrad.schwarz
priority: normal
severity: normal
status: open
title: imprecise handling of weakref callbacks
type: behavior
versions: Python 3.7
_______________________________________
Python tracker <[email protected]>
<https://bugs.python.org/issue43383>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
