On Tue, May 10, 2016 at 9:02 AM, Tom Lane <t...@sss.pgh.pa.us> wrote: > Kevin Grittner <kgri...@gmail.com> writes: >> There were 75 samples each of "disabled" and "reverted" in the >> spreadsheet. Averaging them all, I see this: > >> reverted: 290,660 TPS >> disabled: 292,014 TPS > >> That's a 0.46% overall increase in performance with the patch, >> disabled, compared to reverting it. I'm surprised that you >> consider that to be a "clearly measurable difference". I mean, it >> was measured and it is a difference, but it seems to be well within >> the noise. Even though it is based on 150 samples, I'm not sure we >> should consider it statistically significant. > > You don't have to guess about that --- compare it to the standard > deviation within each group.
My statistics skills are rusty, but I thought that just gives you an effect size, not any idea of whether the effect is statistically significant. Does anyone with sharper skills in this area than I want to opine on whether there is a statistically significant difference between the numbers on "master-default-disabled" lines and "master-revert" lines in the old_snap.ods file attached to an earlier post on this thread? -- Kevin Grittner EDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company
