#12313: Fix yet another memory leak caused by caching of coercion data
--------------------------------------------------+-------------------------
Reporter: SimonKing | Owner:
Type: defect | Status:
needs_review
Priority: major | Milestone: sage-5.3
Component: memleak | Resolution:
Keywords: coercion weak dictionary | Work issues:
Report Upstream: N/A | Reviewers: Simon King,
Jean-Pierre Flori, John Perry
Authors: Simon King, Jean-Pierre Flori | Merged in:
Dependencies: #11521, #11599, #12969, #12215 | Stopgaps:
--------------------------------------------------+-------------------------
Comment (by SimonKing):
Here are some timings. They are quite conclusive: `MonoDict` is very fast,
even in the "validate weak references before giving an answer" version!
First, something that is quite realistic. We have many polynomial rings
over different finite base fields - something that often occurs in
elliptic curve computations. Then, we create a `WeakKeyDictionary` and a
`MonoDict` keyed with these polynomial rings, and then test containment
with a ring that is in the dictionary and one that isn't:
{{{
sage: L = []
sage: for p in prime_range(10000):
....: L.append(GF(p)['x','y'])
....:
sage: import weakref
sage: from sage.structure.coerce_dict import MonoDict
sage: M = MonoDict(53)
sage: for i,K in enumerate(L):
....: M[K] = i
....:
sage: W = weakref.WeakKeyDictionary()
sage: for i,K in enumerate(L):
....: W[K] = i
....:
sage: K = GF(97)['x','y']
sage: K2 = GF(next_prime(p))['x','y']
sage: K in W
True
sage: K in M
True
sage: K2 in W
False
sage: K2 in M
False
sage: %timeit K in W
625 loops, best of 3: 55.9 µs per loop
sage: %timeit K in M
625 loops, best of 3: 533 ns per loop
sage: %timeit K2 in W
625 loops, best of 3: 2.41 µs per loop
sage: %timeit K2 in M
625 loops, best of 3: 1.56 µs per loop
sage: %timeit a = W[K]
625 loops, best of 3: 54.9 µs per loop
sage: %timeit a = M[K]
625 loops, best of 3: 1.08 µs per loop
}}}
Hence, in this realistic example, `MonoDict` is a lot faster than
`WeakKeyDictionary` -- even though I am talking about the recent version
of `MonoDict` where the weak references are validated.
The polynomial rings have a decently fast hash, but comparison is slow:
{{{
sage: %timeit hash(K)
625 loops, best of 3: 649 ns per loop
sage: %timeit hash(K2)
625 loops, best of 3: 678 ns per loop
sage: %timeit K == K
625 loops, best of 3: 53.2 µs per loop
sage: %timeit K == K2
625 loops, best of 3: 65.7 µs per loop
}}}
So, let us see what happens if we do "the same" with
`UniqueRepresentation`:
{{{
sage: class A(UniqueRepresentation):
....: def __init__(self,p):
....: self.p = p
....:
sage: L = []
sage: for p in prime_range(10000):
....: L.append(A(p))
....:
sage: K = A(97)
sage: K2 = A(next_prime(p))
sage: import weakref
sage: W = weakref.WeakKeyDictionary()
sage: from sage.structure.coerce_dict import MonoDict
sage: M = MonoDict(53)
sage: for i,K in enumerate(L):
....: W[K] = i
....: M[K] = i
....:
sage: K in M
True
sage: K in W
True
sage: K2 in M
False
sage: K2 in W
False
sage: %timeit K in M
625 loops, best of 3: 552 ns per loop
sage: %timeit K in W
625 loops, best of 3: 3.67 µs per loop
sage: %timeit K2 in M
625 loops, best of 3: 1.6 µs per loop
sage: %timeit K2 in W
625 loops, best of 3: 2.72 µs per loop
sage: %timeit M[K]
625 loops, best of 3: 1.52 µs per loop
sage: %timeit W[K]
625 loops, best of 3: 3.22 µs per loop
sage: %timeit hash(K)
625 loops, best of 3: 944 ns per loop
sage: %timeit K == K
625 loops, best of 3: 910 ns per loop
sage: %timeit K == K2
625 loops, best of 3: 918 ns per loop
}}}
Hence, comparison is now a lot faster, and this is noticeable in
`WeakKeyDictionary`.
However: `MonoDict` still outperforms `WeakKeyDictionary`. I am slightly
surprised - I didn't expect this would happen, after introducing the
validation of weak references. Note that testing `K in M` is even faster
than a single comparison `K==K`!
--
Ticket URL: <http://trac.sagemath.org/sage_trac/ticket/12313#comment:159>
Sage <http://www.sagemath.org>
Sage: Creating a Viable Open Source Alternative to Magma, Maple, Mathematica,
and MATLAB
--
You received this message because you are subscribed to the Google Groups
"sage-trac" group.
To post to this group, send email to [email protected].
To unsubscribe from this group, send email to
[email protected].
For more options, visit this group at
http://groups.google.com/group/sage-trac?hl=en.
