> On 5 Jun 2017, at 19:30, Zoffix Znet (via RT) <perl6-bugs-follo...@perl.org> > wrote: > > # New Ticket Created by Zoffix Znet > # Please include the string: [perl #131515] > # in the subject line of all future correspondence about this issue. > # <URL: https://rt.perl.org/Ticket/Display.html?id=131515 > > > > Recently, we have had about ~2.7% overall performance loss, when running > stresstest/spectest that a couple of devs noticed. > Tux's bench also showed a slowdown. > > Based on stresstest timings I post ( > https://irclog.perlgeek.de/perl6-dev/search/?nick=&q=ZOFVM ), the slowdown > happened in > some commits made between 2017-05-31 and 2017-06-01. I tried narrowing down > what brought the slowdown, but couldn't quite > pinpoint it, because there's some randomness in timings of some tests. Here's > my results of test times: > > SHA Stresstest (s) Spectest (s) > aebd0fa 123 > 01d948d 122 > 2500e50 121 / 122 86 > 79b8ab9 119 / 118 83 > b879060 117 / 119 82 / 82 > 3f5cc5a 117 > 3488914 118 > fb7dd8a 119 > > As you can see, the middle commits is where the slowage appears to occur, > with 2500e50 being the most likely culprit, > even though, the code itself isn't something that would be even remotely hot. > > I went and rewrote it in NQP (attached) to see if it drops the numbers at > all, and... it didn't really. > I got a 120s stresstest and a 85s spectest. > > I gave up with this. > > I think we need some extensive bench without any random sleeps or delays in > it and comprehensive enough to test more than > just a few text methods. Perhaps, something that can be cooked up with > https://map.perl6.party/ > > And with such a bench in hand, it might become easier to hunt commits that > slow stuff down considerably.<exception.diff>
Generally, we don’t care about performance once we have an unrecoverable exception that needs to be reported. If the stresstest regression is caused by trying to do Levenstein on method names (which can be a lot on some objects), then maybe the performance of spectest/stresstest can be accounted for by *just* the tests that throw Method Not Found errors?
