I just tried this out with jacobian() and subs() over the commits since 0.7.3 to master. It's showing me that the new caching is the killer slowdown:
https://github.com/sympy/sympy/commit/a63005e4 I've submitted a PR to Björn's repo: https://github.com/bjodah/sympy_benchmarks_bjodah/pull/1/files I'm assuming it uses fastcache because I have it installed, but maybe not. I'm not sure how to control for dependencies yet. Jason moorepants.info +01 530-601-9791 On Sun, Jul 19, 2015 at 10:39 AM, Ondřej Čertík <ondrej.cer...@gmail.com> wrote: > On Sun, Jul 19, 2015 at 5:16 AM, Björn Dahlgren <bjo...@gmail.com> wrote: > > Hi all, > > > > On Tuesday, 14 July 2015 02:49:57 UTC+2, Aaron Meurer wrote: > >> > >> - Get a benchmark machine and run airspeed velocity on it. We need to > >> catch performance regressions. The benchmark suite can be anything, > although > >> obviously well-made benchmarks are better. > > > > > > I found a few hours and threw this together (motivated by having noticed > > some things I have been doing with sympy has been getting slower): > > > > https://github.com/bjodah/sympy_benchmarks_bjodah > > > > then, on a machine with asv 0.1.1, I ran: > > > > $ asv run ALL -s 100 > > > > which essensitally takes 100 equally spaced commits from the whle history > > and run the benchmarks one those. > > You can view the results here: > > > > http://hera.physchem.kth.se/~sympy_asv > > > > some commits failed, maybe I should be targetting the merge commits but I > > haven't > > figured out how to do that yet. > > > > Furthermore: the comparison isn't quite fair since I belive the later > > versions return more correct results (Piecewise solutions), > > I'm not sure how to deal with this fairly, maybe some kind of assertion > > should be made (like a tests) at the end of the individual benchmark, > > a passing assertion means it should be included, a failing one that it > > should be excluded. But then this > > assertion shouldn't be included in the timing. There could be something > in > > asv for this but I haven't figured it out yet. > > > > I am stil a bit new to using asv so I haven't quite figured out best > > practives here. > > It would be nice to have a dedicated machine (or maybe several), maybe a > > Raspberry Pi 2 which > > would make it easy for people to get the exact same hardware for > > comparisons? > > > > Anyways, thought I would share. Feel free to fork/send pull requests, > I'll > > happily run them on that machine if there is an > > interest. > > Björn, thanks a lot for doing this! I need to setup something similar > for SymEngine, I am currently benchmarking every commit that I think > might slow things down by hand. > > > I agree with everything that Aaron said. SymEngine is only fixing the > "the speed as a side-effect of being in pure Python", i.e. with C++ we > are much closer to the metal, so for low level representations of the > symbolic expressions we have more freedom to nail the speed, but > algorithmically it's the same issue as with SymPy and everything that > Aaron said applies to SymEngine as well. > > We can buy a dedicated machine via NumFOCUS, SymPy should have enough > money there, but I don't have time personally to make it happen. If > anybody is interested, let me or Aaron now. > > Ondrej > > -- > You received this message because you are subscribed to the Google Groups > "sympy" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to sympy+unsubscr...@googlegroups.com. > To post to this group, send email to sympy@googlegroups.com. > Visit this group at http://groups.google.com/group/sympy. > To view this discussion on the web visit > https://groups.google.com/d/msgid/sympy/CADDwiVBBSj_BRVzPGubknQYmAvEaGLYhH%2BK4B7dXR90szteXNg%40mail.gmail.com > . > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "sympy" group. To unsubscribe from this group and stop receiving emails from it, send an email to sympy+unsubscr...@googlegroups.com. To post to this group, send email to sympy@googlegroups.com. Visit this group at http://groups.google.com/group/sympy. To view this discussion on the web visit https://groups.google.com/d/msgid/sympy/CAP7f1Aggf6HP_ka_3b73QwVGv67ShwRqtqgoRnCULVDmtbxTMg%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
