Francesc Alted wrote: > A Thursday 10 September 2009 13:45:10 Dag Sverre Seljebotn escrigué: > > Do you see any issues with this approach: Add a flag timeit to provide > > > two modes: > > > > > > a) Do an initial run which is always not included in timings (in fact, > > > as it gets "min" and not "mean", I think this is the current behaviour) > > Yup, you are right, it is 'min'. In fact, this is why timeit normally > 'forgets' about data transmission times (with a 'mean' the effect is > very similar anyways). > > > b) Do something else between every run which should clear out the cache > > > (like, just do another big dummy calculation). > > Yeah. In fact, you can simulate this behaviour by running two instances > of timeit: one with your code + big dummy calculation, and the other > with just the big dummy calculation. Subtract both numbers and you will > have a better guess for non-cached calculations. > > > > > > (Also a guard in timeit against CPU frequency scaling errors would be > > > great :-) Like simply outputting a warning if frequency scaling is > > > detected). > > Sorry, I don't get this one.
I had some trouble getting reliable benchmarks on my own computer until I realised that the power-saving capabilities of my CPU down-throttled the clock speed when it was not in use. Thus if I did two calls to timeit right after one another, the second would always report lower runtime, because the first one started at a lower clock speed. Changing a BIOS setting solved this, but it might be a gotcha which e.g. timeit and IPython could report (they could just inspect the CPU information and emit a warning -- or, do something to throttle up the CPU to full speed first). -- Dag Sverre _______________________________________________ NumPy-Discussion mailing list [email protected] http://mail.scipy.org/mailman/listinfo/numpy-discussion
