On Fri, Feb 6, 2009 at 4:26 AM, Stian Soiland-Reyes <[email protected]> wrote: > > > On Feb 5, 1:06 pm, Ben Collins-Sussman <[email protected]> wrote: >> 1. The svn HTTP protocol is horribly inefficient, due to huge numbers >> of extra requests that are only done to be 'dav compliant'. My >> current 20% project is to rewrite the protocol to be streamlined and >> as small as possible. We've given up on DAV, as it buys us almost >> nothing. I'm doing this work in the public svn codebase, and it >> should be released to the public by summer. It should result in >> overall HTTP speedup for anyone using 1.6 client and servers. > > This sounds very interesting, specially since it's done as part of the > main SVN development. Would this still be based on http? Some clever > async REST-like protocol perhaps..?
It's a variation on the existing webdav protocol, but with all of the fat cut out of it -- no extra network requests which dav requires to be 'formally correct'. For details, see http://svn.collab.net/repos/svn/trunk/notes/http-protocol-v2.txt . We're about 80% finished. > > >> 2. The svn client has two optional HTTP client libraries it can use: >> neon or serf. Neon is our legacy library, and does *all* HTTP >> requests synchronously. But Serf opens 4 connections to the server >> (just like a web browser does) and parallelizes requests as much as >> possible. It's also capable of 'pipelining' requests on a single >> connection. In theory, it should be faster than neon, and you can >> try it out by setting 'http-library=serf' in your >> ~/.subversion/servers file. > > I tried switching, without getting notably speed differences. However, > your good explanation of your architecture did got me thinking that I > can do parallel commits. > > So with a few for-loops I've now managed to commit all my thousands of > files and directories. :-) > > I had to trick svn a bit to avoid the svn client locking the parent > directories.. > > svn commit --depth empty * > for dir in */. ; do svn commit -m "" $a/* & done > svn commit -m "" . > > and they should not block each other in the for loop. > > > I did run into this now and then, though.. > > svn: Commit failed (details follow): > svn: Reference to non-existent node '..' in filesystem 'taverna' > > > Is this a server-side problem? On concurrent commits the error > disappeared. Hm, that's odd... I'll have to look into it. > > >> 3. The native svn protocol spoken to the 'svnserve' server is MUCH >> faster than HTTP. Mainly because it's stateful, rather than >> stateless. And yes, this protocol definitely pipelines commit >> requests. If you want speed, there's absolutely nothing faster than >> using the svn:// protocol against an svnserve server. > > But I guess Google Code is not planning to support this in the near > future as it's not HTTP-based and probably not as easy to make > distributed..? Correct... google's whole infrastructure is designed around HTTP. --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Hosting at Google Code" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/google-code-hosting?hl=en -~----------~----~----~----~------~----~------~--~---
