On Sun, 2009-12-06 at 01:33 -0800, Jonathan Leto wrote: > This one is all over the place and trunk is not looking very pretty. > > http://gist.github.com/250139
While I would love for HEAD to be always the fastest, if you mentally graph this you can see that trunk is in the main grouping of total times in the 32 to 40 second range. It's the outliers in the 55 to 74 second range that really aren't looking good. To me there are two optimization goals: 1. Make sure HEAD is in the same grouping as the fastest time, for all benchmarks we can find. (Don't ask me for an exact rule for grouping -- but if you need a programmatic rule of thumb for a meta-bench tool, perhaps cut off at 30% or even 50% worse than the best time.) 2. Make sure HEAD has the fastest time on all of our benchmarks. Right now I think we should be focusing on #1, both because #2 is too far ahead of where we are now, and because violations of #1 will tend to tell us when we've found a particular use case that needs more immediate attention. -'f _______________________________________________ http://lists.parrot.org/mailman/listinfo/parrot-dev
