On Thu, 2011-04-14 at 23:22 +0200, Johan Corveleyn wrote: > Hi Neels, > > Great work, and good to see those numbers. But ... hold on a minute > before jumping to conclusions. IIUC you only tested one type of > platform: *nix with local disk (on a (powerful) VM, and on a (slightly > less powerful) "real machine"). It gives a good indication, but it's > definitely not enough to declare trunk release worthy > performance-wise.
I'm aware of that and am usually very careful to place "I'd say" and "IMHO" all around my humble "conclusions" ;) As-is, my scripts definitely won't run on Windows. The core is python, but there's a thin layer of bash around it that calls the python with the desired settings and filenames, so the bash stuff needs to be looked at and translated to BAT ...or python probably. It could work on OSX without changes, AFAIK. Frankly, I won't personally go into testing on Windows or a WC on an NFS mount, because, *IMHO*, both Win and WC-on-NFS are misguided, and I personally can't be bothered. This is where people using those things and whose jobs depend on svn 1.7 doing well on them should get involved, as far as I'm concerned. Johan is right to highlight the limitations. And I can add a large portion of limitations inherent to benchmarks like these (hardware particulars, time measurement inaccuracies, simulation validity, statistics, interpretation, selective perception, chance,... ). Only the actual release 1.7.0 will bring all the real issues to the surface. And yet, the conclusion I find from my past and present test runs is: the performance I have seen in my setup a while back has definitely been a show-stopper. And now I don't see those particular show-stoppers anymore. So, yes, I should probably be more clear instead of saying "release-worthy" -- I meant it rather negatively defined, as in: my personal perf show-stoppers have vanished. (Thanks for poking) So, does anyone do Windows? Do you speak bash, python and BAT? Want to know about perf on your setup? I'll gladly send you my scripts and explain anything you want to know about it. Oh, and about the time factors getting much better on the VM -- that's not just because the VM is faster than my home machines, since then 1.6.x would also be faster on the VM, resulting in similar time factors as on other hardware. I don't know in detail why we're seeing this, but it must be something like, maybe SQlite has some magic that enhances performance when -- I don't know -- the CPU bus is very wide, or disk cache is fast, or something like that, boosting fast hardware even more. It must be something that svn 1.6 can't benefit from. Enough talk :) ~Neels > > For all we know there could be a massive regression on Windows or > MacOS, or on NTFS specifically (like the locking in WC-1, which was > orders of magnitude slower on Windows/NTFS than on Linux), or, as > already hinted a few times by Philip, on network-mounted filesystems: > NFS, CIFS, ... > > So, to get a better picture, your test-suite should be run on a > variety of platforms (others can help, I guess, like with Mark's > perf-tests). Can your test-suite be run "as is" by others on Windows > for example? Maybe you could already run it yourself on an NFS-mounted > volume? > > I think we need to at least test [ *nix | Windows | MacOS ] x [ local > disk | remote disk ] (ok maybe *nix is too coarse-grained, as is local > disk etc..., but you get the picture). > > Cheers,
