Hi all,
yesterday's and today's benchmarks look particularly good (listed
below). I haven't paid attention to commits, but it seems some pretty
good perf commits have happened recently (except for 'svn delete').
'svn delete' is still 10 times slower on trunk for 100x1 (in a deep
working copy of 100 nested dirs and a file each), and 20 times slower
for 5x5 (tree of 5 levels and 5 subtrees per node).
In all the other cases and all the other svn commands, trunk seems
either faster, *much* faster, or roughly the same speed.
There are a few ratings showing a slower trunk, but these account mostly
for delays in hundredth seconds range. Compared to the timings I saw 6
months ago, this looks really really really good!! :D
I just verified these results on the hardware I originally tested on
(the benchmarks you are seeing these days are run on our ASF VM). On my
"old" laptop, the ratings show less speed gain for trunk than on the VM
(mostly no speed gain, really), but results confirm the same tendency:
the speed factors, except for 'svn delete', are mostly close to 1 -- and
used to be a lot higher.
Again, an exception: on my old hardware, 'svn merge' still slows down on
trunk. But Remarkably, 'merge' used to be around factors 2.6 and higher
(= trunk much slower), which has gone down to only around 1.4.
On all hardware, 'svn delete', in situations with multiple dir levels,
has gone Evil since the benchmarks a few months ago.
So, to summarize, runtime wise we are on par with 1.6.x with exceptions:
- Someone broke/had to break 'svn delete'-with-multiple-dir-levels perf.
Trunk can take 10 and 20 times longer here, which seems a bit off.
- On some hardware, 'svn merge' is slower on trunk, but...
...it's not as bad as a few months ago and
...on our VM, 'merge' is actually not slower but faster on trunk and
...remember that this is measuring mostly libsvn_wc overhead because
benchmarks use ra_local only, and my guess is that merge over http slows
down such that this should become hardly noticeable.
- I also see that trunk seems to be able to better put high-performance
hardware to use than 1.6 did, since on our VM I see noticeable perf
improvements, which doesn't show on "normal" hardware.
- Almost forgot to mention: 'svn status' is slower on trunk (5x5 case).
Though this accounts for only + half-a-second on average, IMHO it would
be nice for 'svn status' to be as fast as possible.
All in all, once we clarified why 'svn delete' is as slow (and possibly
choose not to worry about it), I'd suggest that trunk's performance is
release worthy when compared to 1.6.x.
The best numbers are the "TOTAL RUN" ones, where the whole test run is
summed up; with the numbers shown here, we can statistically prove that,
hold your breath,
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
--- Trunk Is Faster Than 1.6.x (*) ---
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
(*) <fineprintfineprint>
Yay!
Hmm, stsp, wouldn't that also make a good T-shirt slogan? ;)
~Neels
Results for dir levels: 5 spread: 5
5x5 with 1.6: ~444 seconds
trunk: ~418 seconds
COMPARE 5x5_1.6 to 5x5_trunk
1.23|+0.45 means factor=1.23, difference in seconds = 0.45
factor < 1 or difference < 0 means '5x5_trunk' is faster than
'5x5_1.6'
min max avg operation
0.88|-47.746 0.95|-25.249 0.94|-25.663 TOTAL RUN
1.29| +0.002 0.48| -4.468 0.86| -0.006 add
1.73| +0.010 0.54| -6.573 0.55| -2.626 checkout
0.64| -2.009 0.51|-28.064 0.67| -6.072 commit
0.65| -0.014 0.09| -0.621 0.22| -0.157 copy
40.04|+92.021 10.96|+116.441 19.45|+102.098 delete
1.09| +0.678 0.46|-17.970 0.74| -3.778 merge
1.38| +0.003 0.60| -0.090 1.53| +0.006 mkdir
1.23| +0.002 0.06| -3.518 0.84| -0.003 prop mod
1.18| +0.002 0.03| -2.268 0.32| -0.032 propdel
1.21| +0.002 0.07| -1.481 1.38| +0.004 proplist
1.24| +0.002 0.08| -5.684 0.86| -0.003 propset
1.18| +0.002 1.89| +0.031 1.50| +0.006 resolve
3.02| +0.615 1.16| +0.279 2.24| +0.836 resolved
1.39| +0.070 0.74| -1.075 1.78| +0.420 status
0.52| -4.281 0.35|-14.527 0.40| -8.079 switch
1.61| +0.092 0.24|-27.006 0.40| -3.899 update
Results for dir levels: 100 spread: 1
100x1 with 1.6: ~78 seconds
trunk: ~28 seconds (!)
COMPARE 100x1_1.6 to 100x1_trunk
1.23|+0.45 means factor=1.23, difference in seconds = 0.45
factor < 1 or difference < 0 means '100x1_trunk' is faster than
'100x1_1.6'
min max avg operation
0.35|-47.448 0.36|-53.207 0.36|-49.989 TOTAL RUN
1.26| +0.002 0.42| -0.435 0.76| -0.008 add
1.73| +0.009 0.06| -7.794 0.07| -3.062 checkout
0.44| -0.408 0.85| -0.400 0.81| -0.224 commit
0.87| -0.003 0.18| -0.134 0.31| -0.051 copy
9.86| +2.260 9.09| +3.024 8.88| +2.428 delete
0.96| -0.030 2.23| +1.534 1.12| +0.114 merge
1.35| +0.003 1.66| +0.027 1.38| +0.005 mkdir
1.25| +0.002 0.33| -0.093 1.32| +0.004 prop mod
1.22| +0.002 0.12| -0.254 0.59| -0.011 propdel
1.21| +0.002 0.96| -0.002 1.39| +0.004 proplist
1.25| +0.002 0.23| -0.179 1.19| +0.002 propset
1.16| +0.001 0.78| -0.003 1.10| +0.001 resolve
1.27| +0.022 1.12| +0.020 1.17| +0.021 resolved
0.96| -0.003 3.14| +0.453 1.43| +0.053 status
0.10| -4.235 0.09| -5.890 0.09| -5.098 switch
0.86| -0.010 0.09| -7.623 0.11| -2.753 update
Results for dir levels: 1 spread: 100
1x100 with 1.6: ~8 seconds
trunk: ~6.5 seconds
COMPARE 1x100_1.6 to 1x100_trunk
1.23|+0.45 means factor=1.23, difference in seconds = 0.45
factor < 1 or difference < 0 means '1x100_trunk' is faster than
'1x100_1.6'
min max avg operation
1.14| +0.670 0.48| -8.540 0.80| -1.579 TOTAL RUN
0.80| -0.009 0.42| -0.055 0.68| -0.018 add
1.67| +0.009 0.49| -0.165 1.08| +0.006 checkout
1.14| +0.010 0.45| -1.036 0.96| -0.012 commit
1.00| +0.000 0.83| -0.004 0.97| -0.001 copy
1.05| +0.004 0.03| -4.393 0.09| -1.008 delete
0.82| -0.032 0.28| -0.627 0.66| -0.098 merge
1.12| +0.001 0.35| -0.144 0.83| -0.002 prop mod
1.09| +0.001 0.17| -0.176 0.66| -0.006 propdel
1.14| +0.001 1.82| +0.020 1.20| +0.002 proplist
1.11| +0.001 0.13| -0.733 0.85| -0.002 propset
1.14| +0.001 5.39| +0.044 1.47| +0.004 resolve
2.31| +0.014 2.92| +0.034 2.22| +0.017 resolved
1.18| +0.002 4.41| +0.070 1.39| +0.006 status
0.88| -0.011 0.46| -0.105 0.76| -0.028 switch
1.35| +0.004 0.49| -0.182 0.86| -0.009 update
