Yeah, history is now not very meaningful. While we are at it, I think we should complete the job and define a standard set of benchmarks together with the python.org and unladen guys. So that we can keep the data this time :-)
How were the chats about that developing, fijal? 2010/3/7 Maciej Fijalkowski <[email protected]>: > Hey. > > Yes, sorry about not saying that upfront. I've removed --fast from > run, so our numbers looks better (because warmup is more spread over). > It makes sense to remove history IMO. > > However, I plan to work a bit more on that, to completely remove > warmup time from some benchmark and to include warmup for each run for > others (like richards, html5lib), so that would be yet another removal > of history. > > Right now the "average" is not that meaningful (and stddev even less, > although it's not displayed on speed), since it averages warmup across > all the run. I would like to have warmup either completely included > for each run (for benchmarks that it makes sense) or completely > separated and reported as some other variable. > > Cheers, > fijal > > On Sat, Mar 6, 2010 at 2:27 PM, Armin Rigo <[email protected]> wrote: >> Hi all, >> >> Fijal may comment on this more, but note that he tweaked the number of >> runs of the benchmarks displayed by speed.pypy.org, which explains the >> jumps we see in some of them. The positive-for-us jumps, I should add >> :-) >> >> I think that we should remove the history up to today, as it makes >> little sense to compare... >> >> >> A bientot, >> >> Armin. >> _______________________________________________ >> [email protected] >> http://codespeak.net/mailman/listinfo/pypy-dev >> > _______________________________________________ > [email protected] > http://codespeak.net/mailman/listinfo/pypy-dev _______________________________________________ [email protected] http://codespeak.net/mailman/listinfo/pypy-dev
