Re: our benchmark-suite

2012-05-19 Thread Neil Jerram
Andy Wingo wi...@pobox.com writes: Neat :) (Do you pngcrush these? They seem a little slow to serve.) I just tried running pngcrush on all the .pngs, and didn't get more than 6-8% reduction. So unfortunately it doesn't look like that would help much. Thanks for the idea though! Neil

Re: our benchmark-suite

2012-05-16 Thread Andy Wingo
Howdy! On Wed 25 Apr 2012 22:39, l...@gnu.org (Ludovic Courtès) writes: So, those are the problems: benchmarks running for inappropriate, inconsistent durations; I don’t really see such a problem. It doesn’t matter to me if ‘arithmetic.bm’ takes 2mn while ‘vlists.bm’ takes 40s, since I’m

Re: our benchmark-suite

2012-05-16 Thread Ludovic Courtès
Hi! Andy Wingo wi...@pobox.com skribis: On Wed 25 Apr 2012 22:39, l...@gnu.org (Ludovic Courtès) writes: So, those are the problems: benchmarks running for inappropriate, inconsistent durations; I don’t really see such a problem. It doesn’t matter to me if ‘arithmetic.bm’ takes 2mn while

Re: our benchmark-suite

2012-05-15 Thread Andy Wingo
Heya Neil, On Fri 04 May 2012 23:43, Neil Jerram n...@ossau.homelinux.net writes: It turns out I'm already scaling by iteration count - in fact since November 2009. :-) Excellent, so we can scale iteration counts in Guile's git with impunity :) It would be nice for the graphs for individual

Re: our benchmark-suite

2012-05-07 Thread Ludovic Courtès
Hi Neil! Neil Jerram n...@ossau.homelinux.net skribis: Still, I wanted to do something new, so I've added further graphs showing just the last 50 measurements for each benchmark (whereas the existing graphs showed all measurements since my data collection began). The generation of those is

Re: our benchmark-suite

2012-05-04 Thread Neil Jerram
For http://ossau.homelinux.net/~neil I do still have all of the raw data including iteration counts, so I could easily implement dividing by the iteration count, and hence allow for future iteration count changes. Is there any downside from doing that? (I don't think so.) No, I guess. And

Re: our benchmark-suite

2012-05-02 Thread Ludovic Courtès
Hi, Neil Jerram n...@ossau.homelinux.net skribis: l...@gnu.org (Ludovic Courtès) writes: My proposal is to rebase the iteration count in 0-reference.bm to run for 0.5s on some modern machine, and adjust all benchmarks to match, removing those benchmarks that do not measure anything useful.

Re: our benchmark-suite

2012-04-28 Thread Neil Jerram
l...@gnu.org (Ludovic Courtès) writes: My proposal is to rebase the iteration count in 0-reference.bm to run for 0.5s on some modern machine, and adjust all benchmarks to match, removing those benchmarks that do not measure anything useful. Sounds good. However, adjusting iteration counts

Re: our benchmark-suite

2012-04-24 Thread Andy Wingo
: Hi, I was going to try to optimize vhash-assoc, but I wanted a good benchmark first, so I started to look at our benchmark suite. We have some issues to deal with. For those of you who are not familiar with the benchmark suite, we have a bunch of benchmarks in benchmark-suite/benchmarks

our benchmark-suite

2012-04-23 Thread Andy Wingo
Hi, I was going to try to optimize vhash-assoc, but I wanted a good benchmark first, so I started to look at our benchmark suite. We have some issues to deal with. For those of you who are not familiar with the benchmark suite, we have a bunch of benchmarks in benchmark-suite/benchmarks/: those