#715: Parents probably not reclaimed due to too much caching
-------------------------------------------------------------------+--------
Reporter: robertwb |
Owner: somebody
Type: defect |
Status: needs_review
Priority: major |
Milestone: sage-5.4
Component: coercion |
Resolution:
Keywords: weak cache coercion Cernay2012 | Work
issues:
Report Upstream: N/A |
Reviewers: Jean-Pierre Flori, Simon King, Nils Bruin
Authors: Simon King, Jean-Pierre Flori | Merged
in:
Dependencies: #9138, #11900, #11599, to be merged with #11521 |
Stopgaps:
-------------------------------------------------------------------+--------
Comment (by SimonKing):
Replying to [comment:290 nbruin]:
> - if you store all key triples fed into TripleDict (setting strong
refs), you find 220 keys before the tests run (i.e., just due to sage
startup) and 351 after (and no segfault of course). A set of the 151 new
entries:
> ...
> Quite some entries involving "root systems" etc., so it's not so far
fetched to think that a bad deletion of something involving the
`WeylGroup` causes the memory corruption.
I tried to view it from the opposite direction: When feeding a value into
a `TripleDict`, I stored the string representation of the value in a
dictionary, indexed by the memory address of the value. And when
`TripleDictEraser` was removing an item of a `TripleDict`, I wrote the
string representation of the current value being deleted and its original
string representation into a file.
Result: When running the tests of cachefunc.pyx, it happens 122 times that
`TripleDictEraser` is called. It is called on precisely two kinds of
values:
1. The value could be an action. If this is the case, then the underlying
set of the action is already garbage collected, at the time when the
action is removed from the `TripleDict`.
2. The value could be a weak reference to a set of homomorphisms. If this
is the case, then the weak reference is already dead, at the time when it
is removed from the `TripleDict`.
Here is the change that I applied:
{{{
#!diff
diff --git a/sage/structure/coerce_dict.pyx
b/sage/structure/coerce_dict.pyx
--- a/sage/structure/coerce_dict.pyx
+++ b/sage/structure/coerce_dict.pyx
@@ -33,7 +33,7 @@
include "../ext/python_list.pxi"
from weakref import KeyedRef
-
+tmp_dict = {}
############################################
# The following code is responsible for
# removing dead references from the cache
@@ -120,14 +120,21 @@
cdef size_t h = (k1 + 13*k2 ^ 503*k3)
cdef list bucket = <object>PyList_GET_ITEM(self.D.buckets, h %
PyList_GET_SIZE(self.D.buckets))
cdef int i
+ f = file('/Users/SimonKing/SAGE/work/tmp','a')
for i from 0 <= i < PyList_GET_SIZE(bucket) by 4:
if <size_t><object>PyList_GET_ITEM(bucket, i)==k1 and \
<size_t><object>PyList_GET_ITEM(bucket, i+1)==k2 and \
<size_t><object>PyList_GET_ITEM(bucket, i+2)==k3:
+ try:
+ f.write('%s: '%repr(bucket[i+3]))
+ except BaseException,msg:
+ f.write('%s: '%repr(msg))
+ f.write( '%s\n'%tmp_dict[id(bucket[i+3])])
del bucket[i:i+4]
self.D._size -= 1
break
try:
+ f.close()
self.D._refcache.__delitem__((k1,k2,k3))
except KeyError:
pass
@@ -451,6 +458,10 @@
self.set(k1, k2, k3, value)
cdef set(self, object k1, object k2, object k3, value):
+ if getattr(value,'__module__',None)=='weakref':
+ tmp_dict[id(value)] = repr(value())
+ else:
+ tmp_dict[id(value)] = repr(value)
if self.threshold and self._size > len(self.buckets) *
self.threshold:
self.resize()
cdef size_t h1 = <size_t><void *>k1
}}}
Unfortunately, with that change, the signal 11 is gone. After all, it is a
Heisenbug...
The question is: What do we learn from these data?
If the underlying set of an action has already been deleted when deleting
the action, then of course it would be a problem is some `__dealloc__`
method would try to do something with the underlying set.
--
Ticket URL: <http://trac.sagemath.org/sage_trac/ticket/715#comment:293>
Sage <http://www.sagemath.org>
Sage: Creating a Viable Open Source Alternative to Magma, Maple, Mathematica,
and MATLAB
--
You received this message because you are subscribed to the Google Groups
"sage-trac" group.
To post to this group, send email to [email protected].
To unsubscribe from this group, send email to
[email protected].
For more options, visit this group at
http://groups.google.com/group/sage-trac?hl=en.
