#14471: Exception KeyError: in WeakValueDictionary remove ignored
---------------------------+------------------------------------------------
Reporter: vbraun | Owner: rlm
Type: defect | Status: new
Priority: major | Milestone: sage-5.10
Component: memleak | Resolution:
Keywords: | Work issues:
Report Upstream: N/A | Reviewers:
Authors: | Merged in:
Dependencies: | Stopgaps:
---------------------------+------------------------------------------------
Comment (by nbruin):
`WeakValueDict` is a standard python library class, not one of our own
brew. So once we can put the finger on the bug we should likely report
upstream. See also #13394, although I doubt the issue pointed out there is
what is at play here.
My first guess would be that the cache at play is a "cached_method" or a
"cached_function" cache, probably the one from `UniqueRepresentation`, but
I'm not entirely sure. The entries in:
{{{
super(MatrixSpace,sage.matrix.matrix_space.MatrixSpace).__classcall__.get_cache().keys()
}}}
aren't quite of the shape you're finding.
A quick analysis; This is `WeakValueDictionary.__init__`:
{{{
def __init__(self, *args, **kw):
def remove(wr, selfref=ref(self)):
self = selfref()
if self is not None:
del self.data[wr.key]
self._remove = remove
UserDict.UserDict.__init__(self, *args, **kw)
}}}
As you can see, `remove` only stores a link to the dictionary via a weak
reference, so that the dictionary can get deallocated before the callback
(and the callback finds that the dict doesn't exist anymore and simply
does nothing). So that's correct.
We also see that `remove` ''only'' removes the key. It doesn't check
whether the value is still what it's supposed to be and it also doesn't
check if the key is still there in the first place. That would be safe if
there's a guarantee that as soon as a value gets removed from the
dictionary, the `KeyedRef` gets removed as well (in which case the
callback dies without getting executed).
The bug we're seeing indicates that we end up with:
- `weakvaluedict[key]=value`
- `key:KeyedRef(value)` pair gets removed from dict - this ''should''
more or less atomically destroy the `KeyedRef(value)`.
- somehow, the callback from the `KeyedRef(value)` stored gets called
before it gets destroyed.
Since the `KeyedRef(value)` should really only be referenced from the
`weakvaluedict`, it's a bit hard to let it survive: As soon it gets
decreffed it should get deallocated.
My guess is that this happens IN cyclic garbage removal, where callbacks
do get separated from deletions. Python tries hard to avoid calling such
callbacks unnecessarily or in situations where it's unsafe to do so, but
perhaps cleaning up extremely complicated cyclic structures (which we do
create) could fool the detection.
--
Ticket URL: <http://trac.sagemath.org/sage_trac/ticket/14471#comment:4>
Sage <http://www.sagemath.org>
Sage: Creating a Viable Open Source Alternative to Magma, Maple, Mathematica,
and MATLAB
--
You received this message because you are subscribed to the Google Groups
"sage-trac" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
Visit this group at http://groups.google.com/group/sage-trac?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.
