#13400: Use strong caches diligently
-------------------------------+--------------------------------------------
Reporter: nbruin | Owner: robertwb
Type: enhancement | Status: new
Priority: major | Milestone: sage-wishlist
Component: coercion | Resolution:
Keywords: | Work issues:
Report Upstream: N/A | Reviewers:
Authors: | Merged in:
Dependencies: | Stopgaps:
-------------------------------+--------------------------------------------
Comment (by nbruin):
Since the profile above seems that an insane number of calls to
`is_subcategory` is to blame for at least half a second, I took a look at
it. As expected, it's recursive! And if you put a little print statement
{{{
print "is_subcategory(self=%s,C=%s)"%(self,C)
}}}
it's pretty scary to see how often the system goes off on a investigation
of completely trivial questions. For instance, just the line
{{{
N=20; matrix(QQ,N,N,srange(N*N)).echelon_form()
}}}
triggers 140 lines like
{{{
is_subcategory(self=Category of subquotients of monoids,C=Category of
commutative rings)
is_subcategory(self=Category of subquotients of semigroups,C=Category of
commutative rings)
is_subcategory(self=Category of subquotients of magmas,C=Category of
commutative rings)
is_subcategory(self=Category of subquotients of sets,C=Category of
commutative rings)
...
is_subcategory(self=Category of subquotients of semigroups,C=Category of
fields)
is_subcategory(self=Category of subquotients of magmas,C=Category of
fields)
is_subcategory(self=Category of subquotients of sets,C=Category of fields)
is_subcategory(self=Category of quotients of sets,C=Category of fields)
is_subcategory(self=Category of subquotients of sets,C=Category of fields)
}}}
I'm assuming you'll have to try *really* hard to get the category system
to produce an arbitrarily long list of categories and that the
`is_subcategory` relation doesn't change for the lifetime of a category.
Then making this routine caching would help a lot and only cost a fixed
amount of memory. Indeed, when you do that, you quickly get NO PRINT
STATEMENT AT ALL anymore, so it seems that at least my first assumption is
about correct. And it DOES make a big difference. For the code above, we
now get:
5.3b2 + tickets + caching is_subcategory:
{{{
1236719 function calls (1202726 primitive calls) in 4.832 seconds
Ordered by: internal time
ncalls tottime percall cumtime percall filename:lineno(function)
1000 2.036 0.002 4.431 0.004 {method 'echelon_form' of
'sage.matrix.matrix_rational_dense.Matrix_rational_dense' objects}
4993 0.327 0.000 0.572 0.000
matrix_space.py:230(__init__)
1981 0.242 0.000 0.308 0.000
quotient_ring.py:322(__init__)
5000 0.184 0.000 0.903 0.000 matrix_space.py:1180(matrix)
6000 0.180 0.000 0.182 0.000
dynamic_class.py:259(dynamic_class_internal)
1981 0.103 0.000 0.120 0.000 {method 'ideal' of
'sage.rings.ring.Ring' objects}
16143 0.090 0.000 0.098 0.000 {method 'is_prime' of
'sage.rings.integer.Integer' objects}
1000 0.085 0.000 0.085 0.000 {method 'range' of
'sage.rings.integer_ring.IntegerRing_class' objects}
5943 0.082 0.000 0.251 0.000 category.py:112(_join)
17960/5967 0.074 0.000 0.558 0.000
lazy_attribute.py:506(__get__)
1 0.069 0.069 4.834 4.834 <ipython console>:1(test1)
14162 0.062 0.000 0.189 0.000 arith.py:401(is_prime)
10000 0.061 0.000 0.695 0.000
matrix_space.py:154(__classcall__)
127261 0.054 0.000 0.054 0.000 {isinstance}
42822 0.050 0.000 0.064 0.000 weakref.py:55(__getitem__)
2974 0.048 0.000 0.245 0.000 {sage.rings.ring.is_Field}
2000 0.045 0.000 0.215 0.000
arith.py:1051(previous_prime)
1993 0.041 0.000 0.148 0.000
matrix_space.py:1133(zero_matrix)
1981 0.040 0.000 0.614 0.000
integer_mod_ring.py:191(__init__)
6000/1000 0.035 0.000 0.489 0.000
category.py:1023(parent_class)
7000 0.035 0.000 0.045 0.000
category_types.py:261(__init__)
3974 0.034 0.000 0.067 0.000
sageinspect.py:917(sage_getargspec)
4993 0.033 0.000 0.059 0.000
matrix_space.py:904(_get_matrix_class)
18936 0.032 0.000 0.074 0.000 weakref.py:79(__setitem__)
2000 0.032 0.000 0.052 0.000 sequence.py:86(Sequence)
12000 0.028 0.000 0.057 0.000 misc.py:179(cputime)
53487 0.027 0.000 0.053 0.000 category.py:148(<genexpr>)
12993/9993 0.026 0.000 0.635 0.000
unique_representation.py:452(__classcall__)
993 0.025 0.000 0.079 0.000 homset.py:80(Hom)
14162 0.025 0.000 0.030 0.000 all.py:1(arithmetic)
24894 0.025 0.000 0.025 0.000 weakref.py:228(__init__)
47544 0.024 0.000 0.046 0.000 category.py:150(<genexpr>)
993 0.022 0.000 0.046 0.000 homset.py:353(__init__)
12000 0.021 0.000 0.021 0.000 {resource.getrusage}
1000 0.019 0.000 0.156 0.000 misc.py:1051(srange)
12993/9993 0.018 0.000 0.614 0.000
{sage.misc.classcall_metaclass.typecall}
37639 0.018 0.000 0.043 0.000
category.py:1102(is_subcategory)
26875 0.017 0.000 0.027 0.000 weakref.py:223(__new__)
1000 0.017 0.000 0.177 0.000 constructor.py:34(matrix)
}}}
That shaves about `0.7` seconds off!
The next big one seems to be `matrix_space.py:230(__init__)` which is
costing us `0.572` cumulative and seems absent on plain 5.3b2. That's
correct, because all these matrix spaces that are used for multimodular
computation are now destroyed an reconstructed everytime, whereas before
they remained in permanent cache. Setting `gc.disable()` helps a bit:
{{{
1186440 function calls (1152569 primitive calls) in 4.464 seconds
...
3996 0.132 0.000 0.367 0.000
matrix_space.py:230(__init__)
...
}}}
so apparently some spaces get stuck in cycles, but not all.
--
Ticket URL: <http://trac.sagemath.org/sage_trac/ticket/13400#comment:3>
Sage <http://www.sagemath.org>
Sage: Creating a Viable Open Source Alternative to Magma, Maple, Mathematica,
and MATLAB
--
You received this message because you are subscribed to the Google Groups
"sage-trac" group.
To post to this group, send email to [email protected].
To unsubscribe from this group, send email to
[email protected].
For more options, visit this group at
http://groups.google.com/group/sage-trac?hl=en.
