> It is warming up as the given number of seconds with -w flag before > every benchmark test. > > There are still differences like 100%.. Also, 1 sec warmup means > (test count)*(font count) 70 secs for the results.
Mhmm, I'm not sure whether a warmup *time span* makes sense. I would rather have thought that every test would get a certain number of warmup *loops*. For example, '--warmup 100' means that for a value of N=50000, the first 100 loops of each test are not taken into account for timing so that effects of the various processor and memory caches, the operating system's memory page swapping, etc., etc., doesn't have too much influence. This should be just a very small fraction of time, not 70s. > I am thinking of what else can be done and waiting for your test. Just looking at your most recent HTML page I see some peculiarities. * What exactly means 'Baseline (ms)'? Is the shown number the time for one loop? For all loops together? Please clarify and mention this on the HTML page. * There seems to be a fundamental math problem in calculating the percentage numbers. For example, looking at the 'TOTAL' field, the percental difference between 2.788 and 2.740 is not -6.1% but -1.7%! What am I missing? * Looking at the 'Load_Advances (Unscaled)' row, I think that 100% difference between 0.001 and 0.002 doesn't make any sense. How do you compute the percentage? Is this based on the cumulative time of all loops? If so, and you really get such small numbers, there must be some fine-tuning for high-speed tests (for example, increasing N for this particular test by a factor of 10, say) to get meaningful timing values. Werner