Re: rsync and debian -- summary of issues
On Thursday, April 11, 2002 06:15:43 PM +1000 Martin Pool [EMAIL PROTECTED] wrote: +-- | There seems to be a thread about rsync and Debian packages every | couple of months. I've written up a document which tries to cover all | of the questions and debates. It's pretty informal, but hopefully | will be useful. | | http://rsync.samba.org/rsync-and-debian/ | | I'd appreciate comments. +-X8 There is a patch that creates a file list that can be used later and hence avoids the file walk which addresses the server load that you complain about but IIRC it was intended to be used in server-push rather than client-pull, still it could be a start. /Michael -- This space intentionally left non-blank. -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.tuxedo.org/~esr/faqs/smart-questions.html
Re: rsync and debian -- summary of issues
On Fri, Apr 12, 2002 at 01:28:01PM +1000, Martin Pool wrote: On 12 Apr 2002, Brian May [EMAIL PROTECTED] wrote: I think some more details is required regarding rproxy. [...] AFAIK, it solves all the problems regarding server load discussed in rsync, doesn't it??? Why did you think that? rproxy addresses a rather different problem to rsync: for example, it transfers only one file at a time, not whole directories. No, rproxy does not have a magic way to do the delta computation in zero time. Compared to rsync, rproxy has the advantage of cleaner code (imho), but the disadvantage that no optimization work has been done. The big problem with rproxy is it's implemented in perl (perl: crypto for algorithms). librsync on the other hand is a nice piece of work and _should_ be used for a re-implementation of rproxy and/or rsync. I have recently got sponsorship to add a python front-end to librsync as an extension to my pysync code, which I should have done by end of next week. After this I will probably do some work on adding rproxy http extensions into something like medusa or twisted. If rproxy includes the signature in a HEAD responce, and supports ranged requests, delta calculation can be moved to the client with no changes to rproxy as follows; 1) client sends HEAD request to get upstream signature. 2) client does reverse-delta calculation. 3) client applies reverse-delta using ranged-requests to fetch required parts from upstream. You touched on this in your page, but not in relation to rproxy. I believe the client-side delta stuff you mentioned was not using rproxy http extensions at all, just adding some sort of cgi that returns signatures for objects on the server. -- -- ABO: finger [EMAIL PROTECTED] for more info, including pgp key -- -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.tuxedo.org/~esr/faqs/smart-questions.html
Re: rsync and debian -- summary of issues
On Fri, Apr 12, 2002 at 01:28:01PM +1000, Martin Pool wrote: Why did you think that? rproxy addresses a rather different problem to rsync: for example, it transfers only one file at a time, not whole directories. No, rproxy does not have a magic way to do the delta computation in zero time. Compared to rsync, rproxy has the advantage of cleaner code (imho), but the disadvantage that no optimization work has been done. However, that is exactly what applications like apt-get require: transfer of one file at a time... And consider the benifits, if it only did this on Packages and Sources, but nothing else... If implemented in squid, then the results could be cached at intermediate squid caches, too. Something that is impossible with rsync. I always thought though that rproxy, considering it was aimed at being included in public web servers, would minimize the load on the server. Perhaps I am mistaken? -- Brian May [EMAIL PROTECTED] -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.tuxedo.org/~esr/faqs/smart-questions.html
rsync and debian -- summary of issues
There seems to be a thread about rsync and Debian packages every couple of months. I've written up a document which tries to cover all of the questions and debates. It's pretty informal, but hopefully will be useful. http://rsync.samba.org/rsync-and-debian/ I'd appreciate comments. -- Martin -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.tuxedo.org/~esr/faqs/smart-questions.html
Re: rsync and debian -- summary of issues
On Thu, 11 Apr 2002, Martin Pool wrote: There seems to be a thread about rsync and Debian packages every couple of months. I've written up a document which tries to cover all of the questions and debates. It's pretty informal, but hopefully will be useful. http://rsync.samba.org/rsync-and-debian/ I'd appreciate comments. Seems good. It'd be nice if there were links to details about how the reverse rsync algorythm in used, tho. Also, I've had this idea, to use rdiff to generate the checksums, on the server. If rdiff supported a reverse rsync algo(it currently doesn't), then it would make integration with other tools simplistic(thing a wrapper around wget and rdiff). -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.tuxedo.org/~esr/faqs/smart-questions.html
Re: rsync and debian -- summary of issues
On Fri, 12 Apr 2002, Brian May wrote: I think some more details is required regarding rproxy. Why is nobody actively developing it? AFAIK, it solves all the problems regarding server load discussed in rsync, doesn't it??? No. I tested it out, and it still hits the server hard. -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.tuxedo.org/~esr/faqs/smart-questions.html
