On Feb 4, 3:31 pm, Robert Bradshaw <rober...@math.washington.edu> wrote: > This is a bit of a tangent, but perhaps _convert_from_hash could be a > weak-ref dict.
It is a "MonoDict" (weak key ref dict looked up by id) after #12313 > Binary operations can be used for actions more complicated than > coercion-to-a-common-parent followed by arithmetic. But an action > implementing these three cases need not hold a reference to the Parent > (if it's passed the elements) which could cover a large number of > cases. Are you suggesting something specific? > Ideally, an action (A, B) -> C, it should be reclaimed iff either A or > B cease to be externally referable and it would be nice to keep around > if C exists or it's been used recently. I think weakrefs could be > somehow used to make this happen. Well, since #715 the "TripleDict" used to store these things is weakly keyed. The problem is that the thing stored in the TripleDict carries strong references to the key components, so the weak reffing is defeated! At least in cases where None is stored, we get some benefit. > This also requires two (or three) lookups instead of just one even in > the fast path. Indeed, except that the fastest path (both operands have the same parent) can still be special-cased for most applications. But do you have an alternative? we have to keep a strong reference to the coercion morphisms somewhere, because the user isn't interested in them per se. That just leaves one of the parents; hopefully the shortest lived one. If our heuristic for selecting between the two parents is fast an reliable enough, we could just do one lookup on that one parent. But that becomes complicated if we're really supposed to end up on a common cover parent. However, that seems like a case to me that doesn't need to be superfast. Given that the lookup speeds on MonoDict, where the actual coercion is supposed to be found can be within a factor 2 of a python "dict" lookup of a trivially hashed object, I think this extra lookup won't be too much of an issue (given that identical parents should be special cased anyway). It's on par with what happens regularly for method resolution in python anyway. > For this (and elsewhere) I think it may be worth having a circular > queue of hard references to the last 10 (?) parents to be reclaimed, > which would strike a balance between constantly re-creating things and > avoiding unbounded memory increases. You don't get a hook for that. "__del__" methods would give an avenue for it, but that disqualifies the use of cyclic garbage collection and we're getting a LOT of benefit from that at the moment (plenty of circularities). By the time you get to hear that a parent is slated for destruction there's no going back. What we could do have "most recently used" (MRU) caches, perhaps even for coercion morphism lookups if we end up having to go with multiple stage lookups and those end up being too slow. I would not be a fan of that, though, because I can already see the surprises when performance suddenly takes a dive when you loop over one prime more ... (when you discover you were depending on a MRU cache to keep your parent alive) > > If we could get a cdeffed > > interface to weakref objects we could probably gain a little more. > > I don't think that should be too hard, in fact I've thought about > doing this ever since I saw that weak dereference Python call in the > critical path. YES! indeed, if the weakref handling in TripleDict and MonoDict would just use the advertised weakref interface routines: http://docs.python.org/2/c-api/weakref.html we'd already save the method lookups we have there now, which are probably the most expensive part. Slight complication: We rely on KeyedRef, which is a Python class. However, that's a very simple class that we could probably cythonize quite easily. We can do that if we ever find that's necessary. Note: the time-critical thing for us is to access the reference of a weakref. We currently do that via a normal method lookup, but using PyWeakref_GetObject would be WAY faster. Question: If we have a KeyedRef object (which inherits from "ref"). Would it be safe to just call PyWeakref_GetObject on it? -- You received this message because you are subscribed to the Google Groups "sage-devel" group. To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+unsubscr...@googlegroups.com. To post to this group, send email to sage-devel@googlegroups.com. Visit this group at http://groups.google.com/group/sage-devel?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
