I'm rerunning with the dependencies installed (I think). I'll post results once it is done.
Jason moorepants.info +01 530-601-9791 On Sun, Jul 19, 2015 at 4:16 PM, Ondřej Čertík <[email protected]> wrote: > Can you post your timings results? I am doing your benchmark with > SymEngine and Sage for comparison. > > On Sun, Jul 19, 2015 at 4:57 PM, Jason Moore <[email protected]> wrote: > > I just tried this out with jacobian() and subs() over the commits since > > 0.7.3 to master. It's showing me that the new caching is the killer > > slowdown: > > > > https://github.com/sympy/sympy/commit/a63005e4 > > > > I've submitted a PR to Björn's repo: > > https://github.com/bjodah/sympy_benchmarks_bjodah/pull/1/files > > > > I'm assuming it uses fastcache because I have it installed, but maybe > not. > > I'm not sure how to control for dependencies yet. > > > > > > Jason > > moorepants.info > > +01 530-601-9791 > > > > On Sun, Jul 19, 2015 at 10:39 AM, Ondřej Čertík <[email protected] > > > > wrote: > >> > >> On Sun, Jul 19, 2015 at 5:16 AM, Björn Dahlgren <[email protected]> > wrote: > >> > Hi all, > >> > > >> > On Tuesday, 14 July 2015 02:49:57 UTC+2, Aaron Meurer wrote: > >> >> > >> >> - Get a benchmark machine and run airspeed velocity on it. We need to > >> >> catch performance regressions. The benchmark suite can be anything, > >> >> although > >> >> obviously well-made benchmarks are better. > >> > > >> > > >> > I found a few hours and threw this together (motivated by having > noticed > >> > some things I have been doing with sympy has been getting slower): > >> > > >> > https://github.com/bjodah/sympy_benchmarks_bjodah > >> > > >> > then, on a machine with asv 0.1.1, I ran: > >> > > >> > $ asv run ALL -s 100 > >> > > >> > which essensitally takes 100 equally spaced commits from the whle > >> > history > >> > and run the benchmarks one those. > >> > You can view the results here: > >> > > >> > http://hera.physchem.kth.se/~sympy_asv > >> > > >> > some commits failed, maybe I should be targetting the merge commits > but > >> > I > >> > haven't > >> > figured out how to do that yet. > >> > > >> > Furthermore: the comparison isn't quite fair since I belive the later > >> > versions return more correct results (Piecewise solutions), > >> > I'm not sure how to deal with this fairly, maybe some kind of > assertion > >> > should be made (like a tests) at the end of the individual benchmark, > >> > a passing assertion means it should be included, a failing one that it > >> > should be excluded. But then this > >> > assertion shouldn't be included in the timing. There could be > something > >> > in > >> > asv for this but I haven't figured it out yet. > >> > > >> > I am stil a bit new to using asv so I haven't quite figured out best > >> > practives here. > >> > It would be nice to have a dedicated machine (or maybe several), > maybe a > >> > Raspberry Pi 2 which > >> > would make it easy for people to get the exact same hardware for > >> > comparisons? > >> > > >> > Anyways, thought I would share. Feel free to fork/send pull requests, > >> > I'll > >> > happily run them on that machine if there is an > >> > interest. > >> > >> Björn, thanks a lot for doing this! I need to setup something similar > >> for SymEngine, I am currently benchmarking every commit that I think > >> might slow things down by hand. > >> > >> > >> I agree with everything that Aaron said. SymEngine is only fixing the > >> "the speed as a side-effect of being in pure Python", i.e. with C++ we > >> are much closer to the metal, so for low level representations of the > >> symbolic expressions we have more freedom to nail the speed, but > >> algorithmically it's the same issue as with SymPy and everything that > >> Aaron said applies to SymEngine as well. > >> > >> We can buy a dedicated machine via NumFOCUS, SymPy should have enough > >> money there, but I don't have time personally to make it happen. If > >> anybody is interested, let me or Aaron now. > >> > >> Ondrej > >> > >> -- > >> You received this message because you are subscribed to the Google > Groups > >> "sympy" group. > >> To unsubscribe from this group and stop receiving emails from it, send > an > >> email to [email protected]. > >> To post to this group, send email to [email protected]. > >> Visit this group at http://groups.google.com/group/sympy. > >> To view this discussion on the web visit > >> > https://groups.google.com/d/msgid/sympy/CADDwiVBBSj_BRVzPGubknQYmAvEaGLYhH%2BK4B7dXR90szteXNg%40mail.gmail.com > . > >> For more options, visit https://groups.google.com/d/optout. > > > > > > -- > > You received this message because you are subscribed to the Google Groups > > "sympy" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [email protected]. > > To post to this group, send email to [email protected]. > > Visit this group at http://groups.google.com/group/sympy. > > To view this discussion on the web visit > > > https://groups.google.com/d/msgid/sympy/CAP7f1Aggf6HP_ka_3b73QwVGv67ShwRqtqgoRnCULVDmtbxTMg%40mail.gmail.com > . > > > > For more options, visit https://groups.google.com/d/optout. > > -- > You received this message because you are subscribed to the Google Groups > "sympy" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To post to this group, send email to [email protected]. > Visit this group at http://groups.google.com/group/sympy. > To view this discussion on the web visit > https://groups.google.com/d/msgid/sympy/CADDwiVBECZQHgY9KfpGCmymq%3DbGFWCr4LNxq6HY8UbakL6qWZg%40mail.gmail.com > . > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "sympy" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/sympy. To view this discussion on the web visit https://groups.google.com/d/msgid/sympy/CAP7f1AgmjuphE09C0VVkdWSfk3VOvOi2TAqxvBMCrM3qyiLeDw%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
