#11521: Use weak references to cache homsets
--------------------------------------------------+-------------------------
Reporter: jpflori | Owner: robertwb
Type: defect | Status: needs_work
Priority: major | Milestone: sage-5.4
Component: coercion | Resolution:
Keywords: sd35 | Work issues:
Report Upstream: N/A | Reviewers: Jean-Pierre
Flori, Nils Bruin
Authors: Simon King | Merged in:
Dependencies: #12969; to be merged with #715 | Stopgaps:
--------------------------------------------------+-------------------------
Changes (by nbruin):
* status: positive_review => needs_work
Comment:
Aw, this weakref caching stuff is very headache-inducing. My sincere
apologies. I'm afraid I have to withdraw my positive review due to a
previous failure in my logic. As I see it now, the current patch reduces
the caching of coercion maps to near uselessness. The fact that it DOES
seem to work to some extent might give a lead to why other things are
going wrong. Let me try:
Ignoring the weakref on H for now, the coercion cache here consists of a
TripleDict, stored on Y, indexing
{{{
(X,Y,C) : H
}}}
or
{{{
(X,Y,C): None
}}}
Here `H` is a map from `X` to `Y` wrt to the category `C`, so it stores
strong references to `X`,`Y`,`C`. That means that as long as `H` is alive,
no component of the key will die, so the fact that this is stored in a
`TripleDict` is irrelevant: The value `H` would keep the key alive anyway.
The only exception is the value `None`. There the `TripleDict` does
exactly what it should.
This cycle problem is fixed by weakreffing `H`. But the whole point of the
cache is to keep `H` alive as long as all of `X`, `Y` and `C` exist, so
that we can look it up. Since the normal use for coercion would be to
discover the coercion map and forget about it as soon as it has been
applied, one would expect the cache to be empty all the time!
I don't immediately see a canonical way to solve this problem. Here are
some ideas that may mitigate it:
We're caching coercions on the codomain, which I think is the right place
for the following reason. Long-lived parents such as `ZZ` tend to have a
lot of coercions ''to'' other parents, but very few ''from''. As observed
above, we are caching the absense of coercions, but those don't involve an
`H` that might keep the codomain alive. So, we could cache the absense of
coercions in a `TripleDict`. In fact, we could make that a `TripleSet` if
we really want.
The PRESENCE of a coercion should probably be cached with a strong
reference to the coercion. That's the point of caching the thing!
This should be keyed on `(X,C)` (the `Y` is implied by the cache we're
looking in). Storing the key weakly has no effect with the present design
of coercion.
That means that discovering those impose a memory cost on Y. However,
usually there tend to not be too many of those (is that true in the
p-adics with all those different precisions floating around? Still, that
should be managable relative to creating all finite fields up to size
10^9^ and storing strong references on `ZZ` to them to record the absence
of a coercion from them, which I think was the problem in the example that
started this ticket)
For permanent objects with coercions ''from'' lots of objects (like the
symbolic ring, real/complex fields, [number fields with declared
embeddings], `QQbar` [number fields again]) we have a big problem.
One solution would be to turn off caching for those (which needs
infrastructure ... I guess a flag
`Y.no_coercion_caching_on_me_please=True`, which needs checking any time
you're about to cache something. Or, if that is too drastic, an optional
parent method
`Y.is_this_a_coercion_cachable_domain(X)`
or`Y.is_this_a_coercion_cachable_domain_type(type(X))`
Another would be to warn people that mixing parents makes the "larger"
parent remember the "smaller" one, so if you let lots of "smaller" parent
interact with a "larger" one, you'll be using memory for the lifetime of
that "larger" one.
The "proper" solutions below would force coercion maps to be different
from normal homomorphisms, fit for users. That's because if a user keeps a
reference `h` to an element of `Hom(X,Y)`, he/she rightfully expects that
to keep `X` and `Y` alive.
we'd have to weakly key the cache, but still strongly store the coercion
map.
The "proper" (but possibly too expensive) option is to not store `X` on
coercion maps. If we have `H=Y.coercion_map_from(X)` and `H` is used
correctly, then `H(x)` can get access to X via `x.parent()` anyway, so `H`
doesn't strictly need a reference to `X`.
So the object we store in the cache could be a map-like object `cH` that
doesn't store its domain. Upon calling `Y.coercion_map_from(X)` we could
wrap `cH` into a proper `Hom(X,Y)` member that does have a strong
reference to X. This might be too expensive for something so fundamental
as coercion.
'''TL;DR''': Do not store a `KeyedRef` to `H` but store `H` itself in the
cache. We won't solve the leak completely, but at least we're not leaking
on the ''absence'' of coercions as we were before.
--
Ticket URL: <http://trac.sagemath.org/sage_trac/ticket/11521#comment:152>
Sage <http://www.sagemath.org>
Sage: Creating a Viable Open Source Alternative to Magma, Maple, Mathematica,
and MATLAB
--
You received this message because you are subscribed to the Google Groups
"sage-trac" group.
To post to this group, send email to [email protected].
To unsubscribe from this group, send email to
[email protected].
For more options, visit this group at
http://groups.google.com/group/sage-trac?hl=en.
