Hi Dmitry, I just read a note from myself asking me to follow-up on the HTTP pipelining work. Since we're coming up to a release, I thought I'd better do it now :-)
Could I ask what is the state of HTTP pipelining in Darcs? Are there any outstanding issues you think should still be fixed? If I understand correctly, all your recent pipelining patches were applied: Sun Apr 18 23:10:13 BST 2010 Dmitry Kurochkin <dmitry.kuroch...@gmail.com> * Simplify libcurl pipelining configuration. Fri Apr 16 12:21:05 BST 2010 Dmitry Kurochkin <dmitry.kuroch...@gmail.com> * Pass -DCURL_PIPELINING to C compiler when HTTP pipelining is enabled. Sun Apr 18 16:03:02 BST 2010 Dmitry Kurochkin <dmitry.kuroch...@gmail.com> * Darcs.Repository: use pipelining when copying patches. In 2010-04-16 (GSoC Project: Improving darcs' network performance), you said: > 1. I was able to reproduce the "no running handles" bug reported by > many people recently using HTTP server at localhost. [snip] That's http://bugs.darcs.net/issue1770 fixed by your 'Simplify libcurl pipelining configuration' patch > 2. Pipelining works only if darcs makes several parallel requests. I > added few debug prints to understand why no pipelining is done for > http://darcs.net repo. copyFileUsingCache tries to download patches to > /home/darcs-unstable/darcs/_darcs/pristine.hashed/ directory. There is > no such directory on my system, so createDirectoryIfMissing fails. And > speculateFileOrUrl is never called. > 3. If I create this directory on my system, some pipelining starts to > work. After several requests that reuse one connection, curl starts to > open parallel connections to darcs.net. This seems a bug in curl to > me, but it needs more investigation. Later darcs hangs in waitUrl. > This is fixed in another patch that I would send shortly. In a reply you added: > Turns out that when built with cabal -DCURL_PIPELINING flag is never > passed to C compiler. Fixed by patch208. Looks like in cabal builds > pipelining was always broken because of this. ...and patch268 was applied. But does this address all of #2 and #3 above or is there more work to do? > Summary: pipelining is tricky. There are problems on darcs side (both > in network code and patch fetching code in general) and, perhaps, curl > side. So it can be not trivial to investigate and fix. But I think > this is one of the things that should be done to improve darcs network > performance. AFAIK there were never a proper pipelining benchmarks > done, but during my first tests with libwww and darcs-1 I got very > nice results. Maybe a summary of what work (if any) needs doing? Also benchmarking the http stuff would be great. We have this darcs-benchmark tool now... not sure if we can apply it effectively for this. > I believe that pipelining combined with patch packs will resolve all > darcs network performance problems. Smart server, IMHO, is not the way > to go - too complex (from both user and development POV) for little > benefit (compared to patch packs and proper pipelining). Oh, by the way... Alexey's packing work is going to be applied quite soon. Exciting times! -- Eric Kow <http://www.nltg.brighton.ac.uk/home/Eric.Kow> PGP Key ID: 08AC04F9
signature.asc
Description: Digital signature
_______________________________________________ darcs-users mailing list darcs-users@darcs.net http://lists.osuosl.org/mailman/listinfo/darcs-users
