On Thu, Mar 10, 2016 at 8:26 PM, Alexander Korotkov <
[email protected]> wrote:
> I don't think we can rely on median that much if we have only 3 runs.
> For 3 runs we can only apply Kornfeld method which claims that confidence
> interval should be between lower and upper values.
> Since confidence intervals for master and patched versions are overlapping
> we can't conclude that expected TPS numbers are different.
> Dilip, could you do more runs? 10, for example. Using such statistics we
> would be able to conclude something.
>
Here is the reading for 10 runs....
Median Result
-----------------------
Client Base Patch
-------------------------------------------
1 19873 19739
2 38658 38276
4 68812 62075
Full Results of 10 runs...
Base
-------------
Runs 1 Client 2 Client 4 Client
-----------------------------------------------------
1 19442 34866 49023
2 19605 35139 51695
3 19726 37104 53899
4 19835 38439 55708
5 19866 38638 67473
6 19880 38679 70152
7 19973 38720 70920
8 20048 38737 71342
9 20057 38815 71403
10 20344 41423 77953
-----------------------------------------------------
Patch
-----------
Runs 1 Client 2 Client 4 Client
------------------------------------------------------
1 18881 30161 54928
2 19415 32031 59741
3 19564 35022 61060
4 19627 36712 61839
5 19670 37659 62011
6 19808 38894 62139
7 19857 39081 62983
8 19910 39923 75358
9 20169 39937 77481
10 20181 40003 78462
------------------------------------------------------
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
