Can you post your timings results? I am doing your benchmark with SymEngine and Sage for comparison.
On Sun, Jul 19, 2015 at 4:57 PM, Jason Moore <moorepa...@gmail.com> wrote: > I just tried this out with jacobian() and subs() over the commits since > 0.7.3 to master. It's showing me that the new caching is the killer > slowdown: > > https://github.com/sympy/sympy/commit/a63005e4 > > I've submitted a PR to Björn's repo: > https://github.com/bjodah/sympy_benchmarks_bjodah/pull/1/files > > I'm assuming it uses fastcache because I have it installed, but maybe not. > I'm not sure how to control for dependencies yet. > > > Jason > moorepants.info > +01 530-601-9791 > > On Sun, Jul 19, 2015 at 10:39 AM, Ondřej Čertík <ondrej.cer...@gmail.com> > wrote: >> >> On Sun, Jul 19, 2015 at 5:16 AM, Björn Dahlgren <bjo...@gmail.com> wrote: >> > Hi all, >> > >> > On Tuesday, 14 July 2015 02:49:57 UTC+2, Aaron Meurer wrote: >> >> >> >> - Get a benchmark machine and run airspeed velocity on it. We need to >> >> catch performance regressions. The benchmark suite can be anything, >> >> although >> >> obviously well-made benchmarks are better. >> > >> > >> > I found a few hours and threw this together (motivated by having noticed >> > some things I have been doing with sympy has been getting slower): >> > >> > https://github.com/bjodah/sympy_benchmarks_bjodah >> > >> > then, on a machine with asv 0.1.1, I ran: >> > >> > $ asv run ALL -s 100 >> > >> > which essensitally takes 100 equally spaced commits from the whle >> > history >> > and run the benchmarks one those. >> > You can view the results here: >> > >> > http://hera.physchem.kth.se/~sympy_asv >> > >> > some commits failed, maybe I should be targetting the merge commits but >> > I >> > haven't >> > figured out how to do that yet. >> > >> > Furthermore: the comparison isn't quite fair since I belive the later >> > versions return more correct results (Piecewise solutions), >> > I'm not sure how to deal with this fairly, maybe some kind of assertion >> > should be made (like a tests) at the end of the individual benchmark, >> > a passing assertion means it should be included, a failing one that it >> > should be excluded. But then this >> > assertion shouldn't be included in the timing. There could be something >> > in >> > asv for this but I haven't figured it out yet. >> > >> > I am stil a bit new to using asv so I haven't quite figured out best >> > practives here. >> > It would be nice to have a dedicated machine (or maybe several), maybe a >> > Raspberry Pi 2 which >> > would make it easy for people to get the exact same hardware for >> > comparisons? >> > >> > Anyways, thought I would share. Feel free to fork/send pull requests, >> > I'll >> > happily run them on that machine if there is an >> > interest. >> >> Björn, thanks a lot for doing this! I need to setup something similar >> for SymEngine, I am currently benchmarking every commit that I think >> might slow things down by hand. >> >> >> I agree with everything that Aaron said. SymEngine is only fixing the >> "the speed as a side-effect of being in pure Python", i.e. with C++ we >> are much closer to the metal, so for low level representations of the >> symbolic expressions we have more freedom to nail the speed, but >> algorithmically it's the same issue as with SymPy and everything that >> Aaron said applies to SymEngine as well. >> >> We can buy a dedicated machine via NumFOCUS, SymPy should have enough >> money there, but I don't have time personally to make it happen. If >> anybody is interested, let me or Aaron now. >> >> Ondrej >> >> -- >> You received this message because you are subscribed to the Google Groups >> "sympy" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to sympy+unsubscr...@googlegroups.com. >> To post to this group, send email to sympy@googlegroups.com. >> Visit this group at http://groups.google.com/group/sympy. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/sympy/CADDwiVBBSj_BRVzPGubknQYmAvEaGLYhH%2BK4B7dXR90szteXNg%40mail.gmail.com. >> For more options, visit https://groups.google.com/d/optout. > > > -- > You received this message because you are subscribed to the Google Groups > "sympy" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to sympy+unsubscr...@googlegroups.com. > To post to this group, send email to sympy@googlegroups.com. > Visit this group at http://groups.google.com/group/sympy. > To view this discussion on the web visit > https://groups.google.com/d/msgid/sympy/CAP7f1Aggf6HP_ka_3b73QwVGv67ShwRqtqgoRnCULVDmtbxTMg%40mail.gmail.com. > > For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups "sympy" group. To unsubscribe from this group and stop receiving emails from it, send an email to sympy+unsubscr...@googlegroups.com. To post to this group, send email to sympy@googlegroups.com. Visit this group at http://groups.google.com/group/sympy. To view this discussion on the web visit https://groups.google.com/d/msgid/sympy/CADDwiVBECZQHgY9KfpGCmymq%3DbGFWCr4LNxq6HY8UbakL6qWZg%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
