#13400: Use strong caches diligently
-------------------------------+--------------------------------------------
Reporter: nbruin | Owner: robertwb
Type: enhancement | Status: new
Priority: major | Milestone: sage-wishlist
Component: coercion | Resolution:
Keywords: | Work issues:
Report Upstream: N/A | Reviewers:
Authors: | Merged in:
Dependencies: | Stopgaps:
-------------------------------+--------------------------------------------
Comment (by SimonKing):
Replying to [comment:3 nbruin]:
> Since the profile above seems that an insane number of calls to
`is_subcategory` is to blame for at least half a second, I took a look at
it. As expected, it's recursive!
That shouldn't be the problem, because ...
> I'm assuming you'll have to try *really* hard to get the category system
to produce an arbitrarily long list of categories and that the
`is_subcategory` relation doesn't change for the lifetime of a category.
Then making this routine caching would help a lot and only cost a fixed
amount of memory.
... it ''is'' cached, mainly.
Do we talk about the same version of sage? Namely, with #11943, the set of
super categories is created once and for all, stored as an attribute, and
then testing subcategories just amounts to testing set containment. And
before merging #11943, I think `is_subcategory` has even been a
cached_method, by #11900 (but I could be mistaken here, I didn't verify
it), so that the work also just amounted to a dictionary lookup.
Before looking into the set of supercategories, `self.is_subcategory(c)`
calls `c._subcategory_hook_(self)`. The idea is that
`c._subcategory_hook_` is a quick (!!) test that allows to avoid the
creation of the whole set of supercategories (which can be a bottle neck)
in many cases. See #11943 for examples where that helps.
Since `_subcategory_hook_` is not mentioned in your profiling, I assume
that it isn't problematic. And then, `self.is_subcategory` can hardly be
problematic either.
Actually I am pretty much sure that the categories involved in that
computation are all different (namely, algebras over different base rings
or so). If they are all different, then what could be done about it? One
could improve the `_subcategory_hook_`, or (if creation of parent_class is
the problem) use #11935 (whose patch currently needs to be rebased).
> Indeed, when you do that, you quickly get NO PRINT STATEMENT AT ALL
anymore, so it seems that at least my first assumption is about correct.
The question is: Where did you put the print statement? Namely, the actual
work is done in `_subcategory_hook_` and in the lazy attribute
`_set_of_supercategories`. Can you include the print statement there,
rather than in is_subcategory?
> The next big one seems to be `matrix_space.py:230(__init__)` which is
costing us `0.572` cumulative and seems absent on plain 5.3b2. That's
correct, because all these matrix spaces that are used for multimodular
computation are now destroyed an reconstructed everytime, whereas before
they remained in permanent cache.
My impression is that this is more of a problem then `is_subcategory`. And
it illustrates what I meant when I changed the title of this ticket: The
matrix spaces should be strongly cached ''during echelon computation'',
because they will be used repeatedly.
The question is whether we look into the code and do that manually. Say,
by creating a list of all matrix spaces occurring in the computation,
where the list either resides on module level (if the matrix spaces are
used in different functions/methods) or locally (if there is just a single
function creating these matrix spaces).
Or perhaps it is easier to create a context that prevents matrix spaces
(or ''all'' weak values) to be temporarily strongly cached.
--
Ticket URL: <http://trac.sagemath.org/sage_trac/ticket/13400#comment:4>
Sage <http://www.sagemath.org>
Sage: Creating a Viable Open Source Alternative to Magma, Maple, Mathematica,
and MATLAB
--
You received this message because you are subscribed to the Google Groups
"sage-trac" group.
To post to this group, send email to [email protected].
To unsubscribe from this group, send email to
[email protected].
For more options, visit this group at
http://groups.google.com/group/sage-trac?hl=en.
