On 28/04/07, Sam Lawrance <[EMAIL PROTECTED]> wrote:
Yes, single file at a time would suck over a network. HTTP/1.1 is a good suggestion, but you need to use pipelining. Demonstrated use (at least in my mind :-) is FreeBSD's portsnap, which is becoming the preferred method to transfer hundreds or thousands of small patch files at a time to each user. The code is here: http:// www.daemonology.net/phttpget/
Thanks for the pointer - I though about this kind of solution too. It might work but: 1. I should be able to make the server smarter in picking up the right list of files - the client doesn't care much *which* of the millions of the files on the server it gets) so no need to request each file individually. It should be enough for my client to say something like "just bring 4756 files, that's what I need to top-up my local queue right now" and the server will send the right files, not sending the same file to multiple clients. 2. Depending on the server's side, this particular program could get into a deadlock since it relays on the server having large enough buffers to send files and receive requests before the client finishes sending all the file names (even though he's careful to switch to non-blocking mode when sending requests - he still doesn't clear up the downstream buffer). Cheers, --Amos -- SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/ Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html
