2008/7/13 Greg Troxel <[EMAIL PROTECTED]>:
> My immediate reaction is that test suites aren't good benchmarks because
> we will often want to add to test suites, while changing the benchmark
> invalidates previous data so we will not want to change the benchmark.
Yes, but...
> Now, if you mean to use the test suite as a collection of
> micro-benchmarks, so that we just have a rule that individual tests
> aren't modified without serious cause, but new ones can be added, then
> that makes senes.
Yes, this is what I meant. And I also think there will be enough test
cases not changing, over the long term, that we won't have to worry in
practice about changing a few here and there.
I've taken a better look now at the existing benchmark-suite - which
doesn't contain very many benchmarks, but does provide a sensible
discussion of timings, and useful library functions. I don't think it
will be very hard to somehow incorporate the test-suite into the set
of benchmarks.
Regards,
Neil
