Hi Neels, I have a brand new (2 week old) MacBook Pro with 4 GB RAM. I am more than happy to run your scripts if you like.
On the same machine I have 1.6.16 (and trunk r1092145) installed - so I can do a comparison run of both. And if you think it is worthy of another comparison run, I have a 3 year old MacBook (non-Pro in flavour) that could also have the same runs. Beau. On 15/04/2011, at 9:25 AM, Neels Hofmeyr wrote: > On Thu, 2011-04-14 at 23:22 +0200, Johan Corveleyn wrote: >> Hi Neels, >> >> Great work, and good to see those numbers. But ... hold on a minute >> before jumping to conclusions. IIUC you only tested one type of >> platform: *nix with local disk (on a (powerful) VM, and on a (slightly >> less powerful) "real machine"). It gives a good indication, but it's >> definitely not enough to declare trunk release worthy >> performance-wise. > > I'm aware of that and am usually very careful to place "I'd say" and > "IMHO" all around my humble "conclusions" ;) > > As-is, my scripts definitely won't run on Windows. The core is python, > but there's a thin layer of bash around it that calls the python with > the desired settings and filenames, so the bash stuff needs to be looked > at and translated to BAT ...or python probably. > It could work on OSX without changes, AFAIK. > Frankly, I won't personally go into testing on Windows or a WC on an NFS > mount, because, *IMHO*, both Win and WC-on-NFS are misguided, and I > personally can't be bothered. This is where people using those things > and whose jobs depend on svn 1.7 doing well on them should get involved, > as far as I'm concerned. > > Johan is right to highlight the limitations. And I can add a large > portion of limitations inherent to benchmarks like these (hardware > particulars, time measurement inaccuracies, simulation validity, > statistics, interpretation, selective perception, chance,... ). Only the > actual release 1.7.0 will bring all the real issues to the surface. > > And yet, the conclusion I find from my past and present test runs is: > the performance I have seen in my setup a while back has definitely been > a show-stopper. And now I don't see those particular show-stoppers > anymore. So, yes, I should probably be more clear instead of saying > "release-worthy" -- I meant it rather negatively defined, as in: my > personal perf show-stoppers have vanished. (Thanks for poking) > > So, does anyone do Windows? Do you speak bash, python and BAT? Want to > know about perf on your setup? I'll gladly send you my scripts and > explain anything you want to know about it. > > Oh, and about the time factors getting much better on the VM -- that's > not just because the VM is faster than my home machines, since then > 1.6.x would also be faster on the VM, resulting in similar time factors > as on other hardware. I don't know in detail why we're seeing this, but > it must be something like, maybe SQlite has some magic that enhances > performance when -- I don't know -- the CPU bus is very wide, or disk > cache is fast, or something like that, boosting fast hardware even more. > It must be something that svn 1.6 can't benefit from. > > Enough talk :) > > ~Neels > > > >> >> For all we know there could be a massive regression on Windows or >> MacOS, or on NTFS specifically (like the locking in WC-1, which was >> orders of magnitude slower on Windows/NTFS than on Linux), or, as >> already hinted a few times by Philip, on network-mounted filesystems: >> NFS, CIFS, ... >> >> So, to get a better picture, your test-suite should be run on a >> variety of platforms (others can help, I guess, like with Mark's >> perf-tests). Can your test-suite be run "as is" by others on Windows >> for example? Maybe you could already run it yourself on an NFS-mounted >> volume? >> >> I think we need to at least test [ *nix | Windows | MacOS ] x [ local >> disk | remote disk ] (ok maybe *nix is too coarse-grained, as is local >> disk etc..., but you get the picture). >> >> Cheers, > >
