Ondrej, I'm not sure why you don't see performance increase with increased cache. The following shows that the benchmarks do run faster with a large cache. Interestingly the memory doesn't seem to change (but I'm not sure I understand how they measure mem usage). Notice that the jacobian wrt to symbols is not affected by the cache size but the jacobians wrt to the functions is, as is substitution. It seems that .diff() calls subs a lot (see https://github.com/sympy/sympy/issues/9701), so maybe the slow down is all due to the caching used for subs.
moorepants@moorepants-2170p:sympy_benchmarks_bjodah(larger-expr)$ asv --config asv.conf.conda.json run · Fetching recent changes. · Creating environments · Discovering benchmarks ·· Uninstalling from py2.7-fastcache-mpmath. ·· Building for py2.7-fastcache-mpmath ·· Installing into py2.7-fastcache-mpmath.. · Running 9 total benchmarks (1 commits * 1 environments * 9 benchmarks) [ 0.00%] · For sympy commit hash 488f3c20: [ 0.00%] ·· Building for py2.7-fastcache-mpmath... [ 0.00%] ·· Benchmarking py2.7-fastcache-mpmath [ 11.11%] ··· Running integrate.TimeIntegration01.time_doit 365.73ms [ 22.22%] ··· Running integrate.TimeIntegration01.time_doit_meijerg 107.41ms [ 33.33%] ··· Running large_exprs.TimeLargeExpressionOperations.peakmem_jacobian_wrt_functions 37M [ 44.44%] ··· Running large_exprs.TimeLargeExpressionOperations.peakmem_jacobian_wrt_symbols 37M [ 55.56%] ··· Running large_exprs.TimeLargeExpressionOperations.peakmem_subs 37M [ 66.67%] ··· Running large_exprs.TimeLargeExpressionOperations.time_jacobian_wrt_functions 1.75s [ 77.78%] ··· Running large_exprs.TimeLargeExpressionOperations.time_jacobian_wrt_symbols 35.49ms [ 88.89%] ··· Running large_exprs.TimeLargeExpressionOperations.time_manual_jacobian_wrt_functions 1.79s [100.00%] ··· Running large_exprs.TimeLargeExpressionOperations.time_subs 1.66s moorepants@moorepants-2170p:sympy_benchmarks_bjodah(larger-expr)$ SYMPY_CACHE_SIZE=10000 asv --config asv.conf.conda.json run · Fetching recent changes. · Creating environments · Discovering benchmarks ·· Uninstalling from py2.7-fastcache-mpmath. ·· Building for py2.7-fastcache-mpmath ·· Installing into py2.7-fastcache-mpmath.. · Running 9 total benchmarks (1 commits * 1 environments * 9 benchmarks) [ 0.00%] · For sympy commit hash 488f3c20: [ 0.00%] ·· Building for py2.7-fastcache-mpmath... [ 0.00%] ·· Benchmarking py2.7-fastcache-mpmath [ 11.11%] ··· Running integrate.TimeIntegration01.time_doit 361.81ms [ 22.22%] ··· Running integrate.TimeIntegration01.time_doit_meijerg 104.40ms [ 33.33%] ··· Running large_exprs.TimeLargeExpressionOperations.peakmem_jacobian_wrt_functions 37M [ 44.44%] ··· Running large_exprs.TimeLargeExpressionOperations.peakmem_jacobian_wrt_symbols 37M [ 55.56%] ··· Running large_exprs.TimeLargeExpressionOperations.peakmem_subs 37M [ 66.67%] ··· Running large_exprs.TimeLargeExpressionOperations.time_jacobian_wrt_functions 56.62ms [ 77.78%] ··· Running large_exprs.TimeLargeExpressionOperations.time_jacobian_wrt_symbols 33.04ms [ 88.89%] ··· Running large_exprs.TimeLargeExpressionOperations.time_manual_jacobian_wrt_functions 55.25ms [100.00%] ··· Running large_exprs.TimeLargeExpressionOperations.time_subs 13.89ms Jason moorepants.info +01 530-601-9791 On Mon, Jul 20, 2015 at 6:57 PM, Ondřej Čertík <ondrej.cer...@gmail.com> wrote: > On Mon, Jul 20, 2015 at 7:48 PM, Ondřej Čertík <ondrej.cer...@gmail.com> > wrote: > > On Sun, Jul 19, 2015 at 4:57 PM, Jason Moore <moorepa...@gmail.com> > wrote: > >> I just tried this out with jacobian() and subs() over the commits since > >> 0.7.3 to master. It's showing me that the new caching is the killer > >> slowdown: > >> > >> https://github.com/sympy/sympy/commit/a63005e4 > >> > >> I've submitted a PR to Björn's repo: > >> https://github.com/bjodah/sympy_benchmarks_bjodah/pull/1/files > >> > >> I'm assuming it uses fastcache because I have it installed, but maybe > not. > >> I'm not sure how to control for dependencies yet. > > > > I modified your benchmark to first do the substitution for symbols, > > only then do the Jacobian: > > > > > https://github.com/sympy/symengine/commit/2ababdb89e6c6db04d4a5df3e30185b824de9c0c > > > > Here are the results: > > > > certik@redhawk:~/repos/symengine/benchmarks(py)$ python kane2.py > > Setup > > Converting to SymEngine... > > SymPy Jacobian: > > Total time: 0.155253887177 s > > SymEngine Jacobian: > > Total time: 0.00277400016785 s > > > > Speedup: 55.97x > > > > > > I played with the cache size, it makes no difference. Your original > > benchmark ran about 1s on my machine, with SYMPY_CACHE_SIZE=5000 it > > took about 0.9s. So you can speedup SymPy itself just by using symbols > > instead of functions by a factor of 5.8x. Btw, I tested that the > > result that I get from symengine is exactly the same as in sympy > > ( > https://github.com/sympy/symengine/commit/2ababdb89e6c6db04d4a5df3e30185b824de9c0c#diff-e423f8edd8641c702513b2ea9c10eaecR32 > ). > > > > > > How did you generate the benchmark? Can you generate a bit larger > > matrix? Once the symengine benchmark runs in about 1s, then I can try > > to speed it up (because I would see the result in the timing more > > easily). > > I actually found a much larger benchmark, where just a simple > differentiation of a single entry (after substituting functionsf or > symbols) takes 60s for sympy. > > > https://github.com/sympy/symengine/commit/31378da77e5463870b9a22fda7a17deb68068fa8 > > Here is where I am saving the matrix: > > > https://github.com/sympy/symengine/commit/31378da77e5463870b9a22fda7a17deb68068fa8#diff-39829995ea91876984e13e3b828d740aR252 > > Does this make sense? Is that a good benchmark? I love the size, I > just want to make sure I am saving the right thing. Is that the full > bicycle? > > Ondrej > > -- > You received this message because you are subscribed to the Google Groups > "sympy" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to sympy+unsubscr...@googlegroups.com. > To post to this group, send email to sympy@googlegroups.com. > Visit this group at http://groups.google.com/group/sympy. > To view this discussion on the web visit > https://groups.google.com/d/msgid/sympy/CADDwiVDZ_JSgC-C_j5J0zmSOXS9TAQsGFv_XcmbBvx3ijPhPrg%40mail.gmail.com > . > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "sympy" group. To unsubscribe from this group and stop receiving emails from it, send an email to sympy+unsubscr...@googlegroups.com. To post to this group, send email to sympy@googlegroups.com. Visit this group at http://groups.google.com/group/sympy. To view this discussion on the web visit https://groups.google.com/d/msgid/sympy/CAP7f1AhavPNhXzdhUDUVH1ihgDgbc4FbdmRPBHrMW9HQ4ijZDA%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
