#12313: Fix yet another memory leak caused by caching of coercion data
-----------------------------------------------------------------------------------------------------------+
Reporter: SimonKing
| Owner: rlm
Type: defect
| Status: needs_review
Priority: major
| Milestone: sage-5.0
Component: memleak
| Keywords: coercion weak dictionary
Work_issues: Find out why some attribute of a parent is deleted before the
parent's elements are deleted | Upstream: N/A
Reviewer:
| Author: Simon King
Merged:
| Dependencies: #715
-----------------------------------------------------------------------------------------------------------+
Changes (by SimonKing):
* status: needs_work => needs_review
Old description:
> The following could not be fixed in #715:
> {{{
> sage: K = GF(1<<55,'t')
> sage: for i in range(50):
> ....: a = K.random_element()
> ....: E = EllipticCurve(j=a)
> ....: b = K.has_coerce_map_from(E)
> ....:
> sage: import gc
> sage: gc.collect()
> 0
> }}}
>
> The problem is that any parent has a dictionary that stores any coerce
> map (and a different dictionary for conversion maps) ending at this
> parent. The keys are given by the domains of the maps. So, in the example
> above, the field `K` has an attribute that is a dictionary whose keys are
> the different elliptic curves.
>
> In coercion, it is usually best to compare parents not by equality but by
> identity. Therefore, I suggest to implement a container called `MonoDict`
> that works similar to the new weak `TripleDict` (see #715), but takes a
> single item as a key.
>
> First tests show that one can actually gain a lot of speed: `MonoDict`
> can access its items much faster than a usual dictionary.
New description:
The following could not be fixed in #715:
{{{
sage: K = GF(1<<55,'t')
sage: for i in range(50):
....: a = K.random_element()
....: E = EllipticCurve(j=a)
....: b = K.has_coerce_map_from(E)
....:
sage: import gc
sage: gc.collect()
0
}}}
The problem is that any parent has a dictionary that stores any coerce map
(and a different dictionary for conversion maps) ending at this parent.
The keys are given by the domains of the maps. So, in the example above,
the field `K` has an attribute that is a dictionary whose keys are the
different elliptic curves.
In coercion, it is usually best to compare parents not by equality but by
identity. Therefore, I suggest to implement a container called `MonoDict`
that works similar to the new weak `TripleDict` (see #715), but takes a
single item as a key.
First tests show that one can actually gain a lot of speed: `MonoDict` can
access its items much faster than a usual dictionary.
__Apply__
* [attachment:trac12313_mono_dict.patch]
* [attachment:trac12313_polynomial_template_dealloc.patch]
--
Comment:
I have attached a second patch, that catches the attribute error that may
occur during deallocation.
Here is why I think the problem arose in the first place. The error occurs
in a `__dealloc__` method of a polynomial. The `__dealloc__` method is, as
much as I know, called ''after'' deletion of the python attributes. In
particular, since `self._parent` can be garbage collected with my patches
(without #715, #11521 and #12313, it would stay in memory forever),
`self._parent._modulus` is deleted before calling the polynomial's
`__dealloc__`. Hence, the error.
I guess, the cleanest solution would be to keep a direct pointer to
`self._parent._modulus` as an attribute of `self`. But that would create
additional memory consumption. And since, by the example below, no memory
leak is created with my second patch, I think the solution of simply
catching the attribute error is good enough.
With the second patch, the above example works without a warning.
Moreover, it seems to come without a speed penalty:
{{{
sage: K.<z> = GF(4)
sage: def test(K):
....: for n in xrange(10000):
....: P.<x> = K[]
....: del P
....: del x
sage: %timeit test(K)
5 loops, best of 3: 376 ms per loop
}}}
with the second patch, but 427 without the second patch.
I also tested whether the second patch introduces a hidden memory leak
(after all, when the attribute error occurs for P, the deallocation of x
is interrupted): In the above example, `test(K)` does not seem to leak.
{{{
sage: test(K)
sage: get_memory_usage()
825.640625
sage: test(K)
sage: get_memory_usage()
825.640625
}}}
Needs review, then!
Apply trac12313_mono_dict.patch
trac12313_polynomial_template_dealloc.patch
--
Ticket URL: <http://trac.sagemath.org/sage_trac/ticket/12313#comment:16>
Sage <http://www.sagemath.org>
Sage: Creating a Viable Open Source Alternative to Magma, Maple, Mathematica,
and MATLAB
--
You received this message because you are subscribed to the Google Groups
"sage-trac" group.
To post to this group, send email to [email protected].
To unsubscribe from this group, send email to
[email protected].
For more options, visit this group at
http://groups.google.com/group/sage-trac?hl=en.
