On Wed, Dec 7, 2011 at 03:19, Peter Geoghegan <pe...@2ndquadrant.com> wrote: > The results are...taking the median value of each set of runs as > representative, my patch appears to run marginally faster than head. > Of course, there is no reason to believe that it should, and I'm > certain that the difference can be explained by noise, even though > I've naturally strived to minimise noise.
You should use the t-test to distinguish whether two data sets show a consistent difference or whether it's just noise. Excel/OpenOffice have the TTEST() macro for this purpose. For statistics doofuses like me, just pick mode=2 and type=3 as that's the most conservative. If the TTEST result is less than 0.05 then you have 95% certainty that the two dataset are consistently different. If not, you need more consistent data. More information here: http://www.gifted.uconn.edu/siegle/research/t-test/t-test.html Regards, Marti -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers