Just to have some fun - Below is the result of running a build of AB
against the same apache 1.3.0 (stock)
It is a simple loop - checkout against a tag; cd apache-1.3/src && cp
Configuration.tmp && Configuration && ./Configure && make && cd support &&
make) and then run 20 times ./ab -c 30 -n 10000 and take the last 25 runs.
By the way - not a -single- tag failed to build on FreeBSD 5-Current and
not a single tag needed a differnet config ! A record hard to beat by
commercial software !
For each is displayed to total avaerage time/connection; the 98%
confidence interval (all in mSec) and the averegate for that version
of AB.
Note that typically (with eceptiosn early on) each version of AB produces
run within the confidence interval - and that successive versions their
confidence intertvals do generally not overlap (except early in the
development) in the more recent series.
So though this shows that comparing versions of AB from different versions
is generally waranted (same mean, same SD) - for recent versions of
1.3 - and that comparisions between version no's is not a good idea.
BUT at the same time - that has not always been the case - and that is is
kind of a miracle (or a tribute to release management/fact that AB hardly
ever changed) that comparison is possible. Also if you include 1.3c and
1.3a then you can conclude that these ranges match - thus from these
measurements you would conclude that the code did not change substantially
(it may of course do for keep alive or something).
Dw.
Tag checout #define in ab mean/SD aver CI
Tag: APACHE_1_3_24 AB: 1.3d 66 +-3 66.5 +-3
Tag: APACHE_1_3_23 AB: 1.3d 67 +-3
Tag: APACHE_1_3_22 AB: 1.3d 67 +-3
Tag: APACHE_1_3_21 AB: 1.3d 65 +-3
Tag: APACHE_1_3_20 AB: 1.3c 63 +-2 61.8 +-2
Tag: APACHE_1_3_19 AB: 1.3c 60 +-2
Tag: APACHE_1_3_18 AB: 1.3c 61 +-2
Tag: APACHE_1_3_17 AB: 1.3c 61 +-2
Tag: APACHE_1_3_16 AB: 1.3c 63 +-2
Tag: APACHE_1_3_15 AB: 1.3c 62 +-2
Tag: APACHE_1_3_14 AB: 1.3c 63 +-2
Tag: APACHE_1_3_13 AB: 1.3c 62 +-2
Tag: APACHE_1_3_12 AB: 1.3c 62 +-2
Tag: APACHE_1_3_11 AB: 1.3c 60 +-2
Tag: APACHE_1_3_10 AB: 1.3c 63 +-2
Tag: APACHE_1_3_9 AB: 1.3a 60 +-1 60.4 +-1
Tag: APACHE_1_3_8 AB: 1.3a 61 +-1
Tag: APACHE_1_3_7 AB: 1.3a 61 +-1
Tag: APACHE_1_3_6 AB: 1.3 64 +-2 65.2 +-2
Tag: APACHE_1_3_5 AB: 1.3 66 +-3
Tag: APACHE_1_3_4 AB: 1.2 61 +-3 62.9 +-3
Tag: APACHE_1_3_3 AB: 1.2 61 +-4
Tag: APACHE_1_3_2 AB: 1.2 64 +-3
Tag: APACHE_1_3_1 AB: 1.1 60 +-9 65.8 +-10
Tag: APACHE_1_3_0 AB: 1.1 67 +-6
Tag: APACHE_1_3b7 AB: 1.1 64 +-9
Tag: APACHE_1_3b6 AB: 1.1 79 +-8
Dw
--
Dirk-Willem van Gulik
