On Fri, Mar 11, 2016 at 7:08 AM, Dilip Kumar <dilipbal...@gmail.com> wrote:
> > On Thu, Mar 10, 2016 at 8:26 PM, Alexander Korotkov < > a.korot...@postgrespro.ru> wrote: > >> I don't think we can rely on median that much if we have only 3 runs. >> For 3 runs we can only apply Kornfeld method which claims that confidence >> interval should be between lower and upper values. >> Since confidence intervals for master and patched versions are >> overlapping we can't conclude that expected TPS numbers are different. >> Dilip, could you do more runs? 10, for example. Using such statistics we >> would be able to conclude something. >> > > Here is the reading for 10 runs.... > > > Median Result > ----------------------- > > Client Base Patch > ------------------------------------------- > 1 19873 19739 > 2 38658 38276 > 4 68812 62075 > > Full Results of 10 runs... > > Base > ------------- > Runs 1 Client 2 Client 4 Client > ----------------------------------------------------- > 1 19442 34866 49023 > 2 19605 35139 51695 > 3 19726 37104 53899 > 4 19835 38439 55708 > 5 19866 38638 67473 > 6 19880 38679 70152 > 7 19973 38720 70920 > 8 20048 38737 71342 > 9 20057 38815 71403 > 10 20344 41423 77953 > ----------------------------------------------------- > > > Patch > ----------- > Runs 1 Client 2 Client 4 Client > ------------------------------------------------------ > 1 18881 30161 54928 > 2 19415 32031 59741 > 3 19564 35022 61060 > 4 19627 36712 61839 > 5 19670 37659 62011 > 6 19808 38894 62139 > 7 19857 39081 62983 > 8 19910 39923 75358 > 9 20169 39937 77481 > 10 20181 40003 78462 > ------------------------------------------------------ > I've drawn graphs for these measurements. The variation doesn't look random here. TPS is going higher from measurement to measurement. I bet you did measurements sequentially. I think we should do more measurements until TPS stop growing and beyond to see accumulate average statistics. Could you please do the same tests but 50 runs. ------ Alexander Korotkov Postgres Professional: http://www.postgrespro.com The Russian Postgres Company
-- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
