Python implementation of rsync algorithm, was Re: rdiff - does it exist?

2000-12-07 Thread Donovan Baarda
On Wed, Dec 06, 2000 at 11:43:18AM +1100, Donovan Baarda wrote: G'day, Quoting Martin Pool [EMAIL PROTECTED]: [...] I'm changing libhsync to have an API much like zlib/bzlib, in the hope that this will make network app integration simpler. [...] I am 90% of the way through an "ex

pysync: Python implementation of rsync algorithm.

2000-12-10 Thread Donovan Baarda
G'day again, The first version of pysync turned out to be a bit buggy... release early and release often is my excuse :-) Actualy, I tripped up over an obsure bug in Python's zlib to do with decompression of sync flushes. The workaround is to decompress in smaller chunks. This makes the code

Re: 60 byte file hangs solaris 8 rcp

2001-01-15 Thread Donovan Baarda
On Mon, Jan 15, 2001 at 10:00:31AM -0600, Dave Dykstra wrote: We have run across a Solaris 8 system that will hang the TCP connection whenever we try to rcp in the attached 60 byte file from any other system. The system administrator is not very responsive so I don't know if he has contacted

Re: Q (feat. req.) rsync: using UDP, multicast

2001-02-14 Thread Donovan Baarda
Quoting Martin Pool [EMAIL PROTECTED]: On 14 Feb 2001, "Ph. Marek" [EMAIL PROTECTED] wrote: Hello once more, [...] I know that's not easy achievable. But I think it could suffice to say "push the changes made from backup/filexyz to current/filexyz on IP 224.x.x.x:y" and everywhere else

[Module] pysync 1.2

2001-03-01 Thread Donovan Baarda
, resulting in heaps of missed matches. This bug caused seriously bad performance and very large deltas. URL: http://freshmeat.net/projects/pysync/ Download: ftp://minkirri.apana.org.au/pub/python/pysync/pysync-1.2.tar.bz2 License: LGPL Categories: Encryption/Encoding Donovan Baarda

Re: Rsync and Java

2001-03-11 Thread Donovan Baarda
On Sun, Mar 11, 2001 at 08:52:25PM -0500, Andre John Mas wrote: Martin Pool wrote: On 11 Mar 2001, Andre John Mas [EMAIL PROTECTED] wrote: I am looking at possibly doing an rsync port to Java, depending on time constraints, and I would like to know whether anyone has already set up

Re: rsync and debian -- summary of issues

2002-04-12 Thread Donovan Baarda
On Fri, Apr 12, 2002 at 01:28:01PM +1000, Martin Pool wrote: On 12 Apr 2002, Brian May [EMAIL PROTECTED] wrote: I think some more details is required regarding rproxy. [...] AFAIK, it solves all the problems regarding server load discussed in rsync, doesn't it??? Why did you think

Re: Can rsync update files in place?

2002-04-16 Thread Donovan Baarda
On Tue, Apr 16, 2002 at 09:43:12PM +0200, Paul Slootman wrote: On Tue 16 Apr 2002, Martin Pool wrote: On 16 Apr 2002, Paul Slootman [EMAIL PROTECTED] wrote: Is there a way to get rsync to not create a new file while transferring and then rename it, but to instead update the existing

Re: Patch: update popt/ to 1.5.1

2002-04-21 Thread Donovan Baarda
On Sat, Apr 20, 2002 at 12:23:16PM -0700, Jos Backus wrote: This patch updates the files under popt/ to the latest vendor drop. The only change is the inclusion of a FreeBSD-specific patch to popt.c. This is needed in case somebody decides to build rsync on that platform without using the

Re: [rproxy-devel] rdiff deltas not very good compared to pysync, why?

2002-04-25 Thread Donovan Baarda
On Wed, Apr 24, 2002 at 09:46:09PM -0400, Shirish H. Phatak wrote: Hi Martin, I can definitely help in managing the librsync package, especially since I will be maintaining our local tree anyway. As of now, I don't have the resources to get the Windows stuff tested; however, I can

rsync md4sum code.

2002-04-27 Thread Donovan Baarda
G'day, I've been working on a Python interface to librsync and have noticed that it uses md4sum code borrowed from Andrew Tridgell and Martin Pool that comes via rsync and was originally written for samba. Is there anything special about this code compared to the RSA md4sum code that can be

Re: rsync md4sum code.

2002-04-27 Thread Donovan Baarda
On Sat, Apr 27, 2002 at 03:32:47PM -0700, Martin Pool wrote: On 27 Apr 2002, Donovan Baarda [EMAIL PROTECTED] wrote: G'day, I've been working on a Python interface to librsync and have noticed that it uses md4sum code borrowed from Andrew Tridgell and Martin Pool that comes via rsync

Re: rsync md4sum code.

2002-04-28 Thread Donovan Baarda
On Sun, Apr 28, 2002 at 01:06:10PM +1000, Donovan Baarda wrote: On Sat, Apr 27, 2002 at 03:32:47PM -0700, Martin Pool wrote: On 27 Apr 2002, Donovan Baarda [EMAIL PROTECTED] wrote: G'day, I've been working on a Python interface to librsync and have noticed that it uses md4sum code

Re: [rproxy-devel] Re: rsync md4sum code.

2002-04-28 Thread Donovan Baarda
On Sun, Apr 28, 2002 at 11:48:14PM +1000, Donovan Baarda wrote: [...] Ok, preliminary results are in. Because I'm primarily interested in Python interface stuff at the moment, I chose to make my benchmark by hacking together swig wrappers for the libmd md4c.c and the librsync mdfour.c

Re: timeout error in rsync-2.5.5

2002-05-03 Thread Donovan Baarda
On Fri, May 03, 2002 at 04:41:17PM -0500, Dave Dykstra wrote: You shouldn't need to have such a long timeout. The timeout is not over the whole length of the run, only the time since the last data was transferred. It's a mystery to me why it quits after 66 minutes rather than 5 hours, but

Re: Improving the rsync protocol (RE: Rsync dies)

2002-05-20 Thread Donovan Baarda
On Mon, May 20, 2002 at 09:35:04PM +1000, Martin Pool wrote: On 17 May 2002, Wayne Davison [EMAIL PROTECTED] wrote: On Fri, 17 May 2002, Allen, John L. wrote: [...] I've been thinking about this too. I think the top-level question is Start from scratch with a new protocol, or try to

Re: Compressed backup

2002-06-01 Thread Donovan Baarda
On Fri, May 31, 2002 at 05:25:15PM -0700, jw schultz wrote: On Fri, May 31, 2002 at 11:45:43AM +1000, Donovan Baarda wrote: On Thu, May 30, 2002 at 03:35:05PM -0700, jw schultz wrote: [...] I would guess that the number of changes meeting this criteria would be almost non-existant. I

Re: Compressed backup

2002-06-01 Thread Donovan Baarda
On Sat, Jun 01, 2002 at 04:57:15AM -0700, jw schultz wrote: On Sat, Jun 01, 2002 at 08:51:26PM +1000, Donovan Baarda wrote: On Fri, May 31, 2002 at 05:25:15PM -0700, jw schultz wrote: [...] When i said content-aware compressor what i meant was that the compressor would actually analize

Re: Compressed backup

2002-06-01 Thread Donovan Baarda
On Sat, Jun 01, 2002 at 05:18:42PM -0700, jw schultz wrote: On Sat, Jun 01, 2002 at 11:46:37PM +1000, Donovan Baarda wrote: On Sat, Jun 01, 2002 at 04:57:15AM -0700, jw schultz wrote: On Sat, Jun 01, 2002 at 08:51:26PM +1000, Donovan Baarda wrote: On Fri, May 31, 2002 at 05:25:15PM

Re: Compressed backup

2002-06-04 Thread Donovan Baarda
On Tue, Jun 04, 2002 at 05:43:17PM +1000, Kevin Easton wrote: [...] If you'll indulge me, I'll just restate the problem (as I see it, anyway) before chiming in with my idea... [snip big discription of why gzip-rsyncable actually does work] Thanks for the discription of how gzip-rsyncable

Re: Strong encryption

2002-06-05 Thread Donovan Baarda
On Wed, Jun 05, 2002 at 03:33:23AM -0700, 'jw schultz' wrote: On Wed, Jun 05, 2002 at 12:21:18PM +0200, C.Zimmermann wrote: If you want them stored on the destination encrypted you Yes, that?s it. The owner of the source files will be sure, that no one else can read his files on

Re: Release 3 of rzync new-protocol test

2002-06-21 Thread Donovan Baarda
On Fri, Jun 21, 2002 at 03:46:39AM -0700, Wayne Davison wrote: The count of transferred bytes in the latest protocol is now below what rsync sends for many commands -- both a start-from-scratch update or a fully-up-to-date update are usually smaller, for instance. This is mainly because my

Re: Block size optimization - let rsync find the optimal blocksize by itself.

2002-06-30 Thread Donovan Baarda
On Sun, Jun 30, 2002 at 06:23:10PM +0200, Olivier Lachambre wrote: [...] Well, the first comment: during my work, I wanted to verify that the theorical optimal block size sqrt(24*n/Q) given by Andrew Tridgell in his PHd Thesis was actually the good one, and when doing the tests on randomly

Re: Block size optimization - let rsync find the optimal blocksize by itself.

2002-07-02 Thread Donovan Baarda
On Tue, Jul 02, 2002 at 11:06:30AM +0200, Olivier Lachambre wrote: At 09:19 01/07/2002 +1000, you wrote: [...] This relies on optimal block size being related for a set of files. I'm not sure that this heuristic actually applies, and I don't know how much benefit this would buy for all the

Re: timestamp on symlink

2002-07-28 Thread Donovan Baarda
On Mon, Jul 29, 2002 at 12:40:56PM +1000, Martin Pool wrote: On 28 Jul 2002, Michael Wang [EMAIL PROTECTED] wrote: rsync does not sync the timestamp on symlink (Solaris 8). It is probablly due to the limitation of Unix implementation of symlink, but I would like to know why rsync/Unix

Re: new rsync release needed soon?

2002-07-31 Thread Donovan Baarda
On Wed, Jul 31, 2002 at 12:29:31PM -0600, Robert Weber wrote: On the subject of needed patches, I just recently completed a patch for librsync that fixed the mdfour code to have uint_64 or 2 uint_32's for size. Without this, the checksums on files 512Megs are incorrect. The code should drop

Re: new rsync release needed soon?

2002-07-31 Thread Donovan Baarda
On Wed, Jul 31, 2002 at 04:28:55PM -0700, Wayne Davison wrote: On Wed, 31 Jul 2002, Robert Weber wrote: On the subject of needed patches, I just recently completed a patch for librsync that fixed the mdfour code to have uint_64 or 2 uint_32's for size. Without this, the checksums on files

Re: MD4 bug in rsync for lengths = 64 * n

2002-09-01 Thread Donovan Baarda
On Thu, Aug 29, 2002 at 03:17:34PM -0500, Dave Dykstra wrote: On Sun, Aug 04, 2002 at 01:19:43PM -0700, Craig Barratt wrote: [...] However, as I'm sure is well-known, the Adler crc32 and MD4 computed by librsync don't match those in rsync 2.5.5. I do not recall hearing anybody mention

Re: MD4 bug in rsync for lengths = 64 * n

2002-09-03 Thread Donovan Baarda
and fixed 512M support. * * 2002-06-27: Donovan Baarda [EMAIL PROTECTED] * further optimisations and cleanups. */ #include config.h #include stdlib.h #include string.h #include stdio.h #include rsync.h #include trace.h #include types.h #include mdfour.h #define F(X,Y,Z) (((X)(Y)) | ((~(X

Re: MD4 bug in rsync for lengths = 64 * n

2002-09-03 Thread Donovan Baarda
Quoting Dave Dykstra [EMAIL PROTECTED]: Donovan or Craig, can you please post it as a patch to the current rsync source (including an increase in the protocol version and checks to do the old behavior when talking to an old version client). I'm not the person to do this, as I don't know

Re: Problem with checksum failing on large files

2002-10-14 Thread Donovan Baarda
On Mon, Oct 14, 2002 at 12:36:36AM -0700, Craig Barratt wrote: craig My theory is that this is expected behavior given the check sum size. [...] But it just occurred to me that checksum collisions is only part of the story. Things are really a lot worse. Let's assume that the block

Re: Problem with checksum failing on large files

2002-10-14 Thread Donovan Baarda
On Mon, Oct 14, 2002 at 06:22:36AM -0700, jw schultz wrote: On Mon, Oct 14, 2002 at 10:45:44PM +1000, Donovan Baarda wrote: [...] Does the first pass signature block checksum really only use 2 bytes of the md4sum? That seems pretty damn small to me. For 100M~1G you need at least 56bits

Re: Problem with checksum failing on large files

2002-10-14 Thread Donovan Baarda
On Mon, Oct 14, 2002 at 04:50:27PM -0700, jw schultz wrote: On Tue, Oct 15, 2002 at 02:25:00AM +1000, Donovan Baarda wrote: On Mon, Oct 14, 2002 at 06:22:36AM -0700, jw schultz wrote: On Mon, Oct 14, 2002 at 10:45:44PM +1000, Donovan Baarda wrote: [...] Does the first pass signature

Re: Problem with checksum failing on large files

2002-10-11 Thread Donovan Baarda
On Fri, Oct 11, 2002 at 03:26:45PM -0700, Terry Reed wrote: -Original Message- From: Derek Simkowiak [mailto:dereks;itsite.com] Sent: Friday, October 11, 2002 1:51 PM To: Terry Reed Cc: '[EMAIL PROTECTED]' Subject: Re: Problem with checksum failing on large files

Re: Problem with checksum failing on large files

2002-10-12 Thread Donovan Baarda
On Sat, Oct 12, 2002 at 11:13:50AM -0700, Derek Simkowiak wrote: My theory is that this is expected behavior given the check sum size. Craig, Excellent analysis! I was a bit concerned about his maths at first, but I did it myself from scratch using a different aproach and got the

Re: Problem with checksum failing on large files

2002-10-12 Thread Donovan Baarda
On Sat, Oct 12, 2002 at 07:29:36PM -0700, jw schultz wrote: On Sat, Oct 12, 2002 at 11:13:50AM -0700, Derek Simkowiak wrote: My theory is that this is expected behavior given the check sum size. Craig, Excellent analysis! Assuming your hypothesis is correct, I like the

Re: Optimizations and other questions for rsync

2002-10-17 Thread Donovan Baarda
On Wed, Oct 16, 2002 at 04:56:52PM -0700, jw schultz wrote: On Wed, Oct 16, 2002 at 05:00:02PM -0400, Farid Shenassa wrote: 1. is there any computational or disk IO difference between the rsync client and server (the one that does just the checksum on the block, vs the one that does rolling

Re: offline rsync

2002-11-06 Thread Donovan Baarda
On Wed, Nov 06, 2002 at 10:14:28AM -0600, Franco Bagnoli wrote: hello, I'm new to this list. here is my question: I would like to synchronize two computers (say the home one and the job one) using zip drives or similar (cdroms, etc), since modem lines are quite slow and expensive (in

Re: offline rsync

2002-11-07 Thread Donovan Baarda
On Thu, Nov 07, 2002 at 08:09:42PM +0100, Franco Bagnoli wrote: On Thu, 7 Nov 2002, Donovan Baarda wrote: rdiff, part of librsync. There is also pysync, but it's much slower (but easier to understand/modify). I know I'm sloppy, but the documentation with rdiff is rather poor

Re: weak checksum question

2002-11-12 Thread Donovan Baarda
On Mon, Nov 11, 2002 at 08:06:40PM -0500, Jeff Abrahamson wrote: The weak checksum in checksum.c (see snippet below) differs substantially from the one discussed in Andrew Tridgell's doctoral thesis on rsync and elsewhere that I've been able to find. I didn't find discussion of the change in

Re: FTP at rsync.samba.org down?

2002-11-25 Thread Donovan Baarda
On Mon, Nov 25, 2002 at 11:44:10AM +0100, Michael Schmidt wrote: Hello, just being fascinated by rsync I wanted to look at the distribution files at ftp://rsync.samba.org, but it was not possible to get a ftp connection to that address. Is the ftp service down there? Thanks in

Re: rsync protocol description

2002-12-17 Thread Donovan Baarda
On Wed, 2002-12-18 at 02:11, Christoph Bartelmus wrote: Hello, I'm currently evaluating the possibility of implementing a rsync client in a project for my company. The platform used is currently not supported and implementing the client from scratch currently seems to be the most feasible

Re: rsync to 2000/NT servers?

2002-12-17 Thread Donovan Baarda
On Tue, 2002-12-17 at 21:13, John Morgan Salomon wrote: Dr. Poo wrote: Now, can you think of a way to sync the win 2000 OS? (the WHOLE flippin' system) so that if it were to go down one could restore the full installation (bootstraps, bootloader, ect!!?) by means of the rsync'ed

Re: rsync protocol description

2002-12-18 Thread Donovan Baarda
On Wed, 2002-12-18 at 19:03, Christoph Bartelmus wrote: Donovan Baarda wrote: For that I'd be very much interested in a description of the protocol that rsync talks on port 873. Is such a description available somewhere? I don't want to have to deduce the protocol from the source code

Re: Proposal that we now create two branches - 2_5 and head

2003-01-29 Thread Donovan Baarda
On Wed, 2003-01-29 at 18:22, Craig Barratt wrote: I have several patches that I'm planning to check in soon (I'm waiting to see if we have any post-release tweaking to and/or branching to do). This list is off the top of my head, but I think it is complete: And I have several things I

RE: Proposal that we now create two branches - 2_5 and head

2003-01-29 Thread Donovan Baarda
On Thu, 2003-01-30 at 07:40, Green, Paul wrote: jw schultz [mailto:[EMAIL PROTECTED]] wrote: [general discussion of forthcoming patches removed] All well and good. But the question before this thread is are the changes big and disruptive enough to make a second branch for the event of

Re: rsync 1tb+ each day

2003-02-05 Thread Donovan Baarda
On Thu, 2003-02-06 at 10:01, Eric Whiting wrote: Kenny Gorman wrote: [...] I tested with a small 256M datafile. rsync -av is showing me about 200kBytes of changes in the datafile between each snapshot. (about 1/1000th of the file has actually changed between the hot backups) Rsync reports the

Re: rsync vs. rcp

2003-02-19 Thread Donovan Baarda
On Thu, 2003-02-20 at 05:55, va_public wrote: I got used to rsync's -v --progress option so much that I used it instead of rcp even to simply copy files across the network. I dont like software that doesnt talk to me! :-) I like the percentage bar that --progress gives! To my surprise,

Re: rsync vs. rcp

2003-02-19 Thread Donovan Baarda
On Thu, 2003-02-20 at 08:55, James Knowles wrote: RSYNC DOES NOT WORK WITH 1GB+ FILES... unless you have a sufficiently large block size. According to the archives, block size doesn't fix anything. At any rate, I'm highly disappointed that rsync is relying on statistical good fortune.

Re: rsync vs. rcp

2003-02-19 Thread Donovan Baarda
On Thu, 2003-02-20 at 08:53, va_public wrote: --- In [EMAIL PROTECTED], Donovan Baarda [EMAIL PROTECTED] wrote: On Thu, 2003-02-20 at 05:55, va_public wrote: RSYNC DOES NOT WORK WITH 1GB+ FILES... unless you have a sufficiently large block size. See the following; http

Re: rsync vs. rcp

2003-02-19 Thread Donovan Baarda
On Thu, 2003-02-20 at 11:36, Craig Barratt wrote: RSYNC DOES NOT WORK WITH 1GB+ FILES... unless you have a sufficiently large block size. See the following; http://www.mail-archive.com/rsync@lists.samba.org/msg05219.html Let's be careful here. Rsync *does* work on 1GB+ files. What

Re: rsync vs. rcp

2003-02-19 Thread Donovan Baarda
On Thu, 2003-02-20 at 13:20, jw schultz wrote: On Thu, Feb 20, 2003 at 01:03:16PM +1100, Donovan Baarda wrote: On Thu, 2003-02-20 at 11:36, Craig Barratt wrote: RSYNC DOES NOT WORK WITH 1GB+ FILES... unless you have a sufficiently large block size. See the following; [...] However

Re: [Apt-rpm] I: [PATCH] 0.5.4cnc9: rsync method support

2003-02-21 Thread Donovan Baarda
On Fri, 2003-02-21 at 18:42, Sviatoslav Sviridov wrote: [...] And other thing I want to discuss: now rsync designed as standalone program, but it will be good if library with protocol implementation created, and build rsync program based on this library. Hmmm, sounds like

Re: [RFC] dynamic checksum size

2003-03-22 Thread Donovan Baarda
On Sun, 2003-03-23 at 07:40, jw schultz wrote: [...] The two attached patches implement per-file dynamic checksum lengths. Nice one. Here is a quick table showing the file sizes at which transition occurs, the sum2 length and the jump in size of the block sum table transmitted:

Re: [RFC] dynamic checksum size

2003-03-23 Thread Donovan Baarda
On Mon, Mar 24, 2003 at 12:54:26AM +1100, Donovan Baarda wrote: On Sun, Mar 23, 2003 at 03:46:34AM -0800, jw schultz wrote: On Sun, Mar 23, 2003 at 05:45:47PM +1100, Donovan Baarda wrote: On Sun, 2003-03-23 at 07:40, jw schultz wrote: [...] The block_size heuristic is pretty arbitary

Re: [RFC] dynamic checksum size

2003-03-23 Thread Donovan Baarda
On Mon, 2003-03-24 at 03:36, jw schultz wrote: On Mon, Mar 24, 2003 at 12:54:26AM +1100, Donovan Baarda wrote: On Sun, Mar 23, 2003 at 03:46:34AM -0800, jw schultz wrote: [...] CHUNK_SIZE is used in mapping the file into memory. I'm not absolutely sure (haven't spelunked that part

Re: [RFC] dynamic checksum size

2003-03-23 Thread Donovan Baarda
On Mon, 2003-03-24 at 16:33, jw schultz wrote: On Mon, Mar 24, 2003 at 10:56:42AM +1100, Donovan Baarda wrote: On Mon, 2003-03-24 at 03:36, jw schultz wrote: [..] hadn't a decent fast integer sqrt function at hand. I don't really care to start bringing in the math lib just

Re: MD4 checksum fix

2003-03-31 Thread Donovan Baarda
. -- Donovan Baardahttp://minkirri.apana.org.au/~abo/ -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.tuxedo.org/~esr/faqs/smart

Re: MD4 checksum fix

2003-03-31 Thread Donovan Baarda
On Tue, 2003-04-01 at 13:34, jw schultz wrote: On Tue, Apr 01, 2003 at 12:41:39PM +1000, Donovan Baarda wrote: On Tue, 2003-04-01 at 09:35, jw schultz wrote: [...] Comparative complexity is a matter of perspective. I think it is a bit more deterministic. It has been a few years I

Re: MD4 checksum fix

2003-04-05 Thread Donovan Baarda
. It looks like it will work OK, but it's kinda ugly in that starts embedding version stuff into the mdfour implementation. Still... its better than the nothing I've produced :-) -- Donovan Baardahttp

Re: MD4 checksum fix

2003-04-05 Thread Donovan Baarda
in a replacement implementation or link against a standard md4 library implementation. -- Donovan Baardahttp://minkirri.apana.org.au/~abo

Re: state of the rsync nation? (revisited 6/2003 from 11/2000)

2003-06-07 Thread Donovan Baarda
does). It would be nice if librsync and/or xdelta could become _the_ delta library. -- Donovan Baardahttp://minkirri.apana.org.au/~abo

Re: state of the rsync nation? (revisited 6/2003 from 11/2000)

2003-06-10 Thread Donovan Baarda
On Tue, 2003-06-10 at 14:25, Martin Pool wrote: On 8 Jun 2003, Donovan Baarda [EMAIL PROTECTED] wrote: The next big thing in delta calculation is probably going to be the vcdiff encoding format, which should allow a common delta format for various applications and supports self

Re: [librsync-devel] Re: state of the rsync nation? (revisited6/2003 from 11/2000)

2003-06-10 Thread Donovan Baarda
On Wed, 2003-06-11 at 13:59, Martin Pool wrote: On 11 Jun 2003, Donovan Baarda [EMAIL PROTECTED] wrote: The vcdiff standard is available as RFC3284, and Josh is listed as one of the authors. Yes, I've just been reading that. I seem to remember that it was around as an Internet-Draft

Re: [librsync-devel] Re: state of the rsync nation? (revisited6/2003 from 11/2000)

2003-06-11 Thread Donovan Baarda
On Wed, 2003-06-11 at 16:13, Martin Pool wrote: On 11 Jun 2003, Donovan Baarda [EMAIL PROTECTED] wrote: On Wed, 2003-06-11 at 13:59, Martin Pool wrote: [...] On 11 Jun 2003, Donovan Baarda [EMAIL PROTECTED] wrote: I forget if I saw this in Tridge's thesis, but I definitely noticed

Re: questions about librsync

2003-06-11 Thread Donovan Baarda
on SourceForge; http://sourceforge.net/mail/?group_id=56125 Next, make sure you are using the CVS version of librsync from the librsync project on SourceForge. If you are still having problems, I'd be interested to know... we are trying to get librsync ready for a 0.9.6 release. -- Donovan

Re: New wildmatch code in CVS

2003-07-09 Thread Donovan Baarda
Quoting Wayne Davison [EMAIL PROTECTED]: If you've been watching CVS, you may have noticed that I checked in some new files named wildmatch.c and wildmatch.h. This code implements the shell-style wildcard matching with rsync's extension that ** matches [...] build farm. One thing I

Re: Rolling Checksum Algorithms

2003-07-20 Thread Donovan Baarda
checksum, but I think it is closer to something I've seen called a Fletcher checksum. Pysync has a Python implementation of a rolling adler32 checksum, and has comments about the variations in rsync, librsync, and xdelta. -- Donovan Baarda

Re: Large files and symlinks

2003-07-30 Thread Donovan Baarda
useful behaviour for --partial would be to concatinate the partial download to the end of the old basis, rather than replace it... this leaves you with a much more useful partial result to resume from. Of course this behaviour could be _very_ confusing to people... :-) -- Donovan Baarda [EMAIL

Re: Large files and symlinks

2003-07-30 Thread Donovan Baarda
On Thu, 2003-07-31 at 10:01, jw schultz wrote: On Thu, Jul 31, 2003 at 09:22:51AM +1000, Donovan Baarda wrote: On Thu, 2003-07-31 at 06:53, jw schultz wrote: [...] In many cases invoking --partial is worse than not. If you are rsyncing a 4GB file and transfer is interrupted after

Re: [librsync-devel] Re: state of the rsync nation?(revisited 6/2003 from 11/2000)

2003-08-03 Thread Donovan Baarda
On Mon, 2003-08-04 at 11:05, Martin Langhoff wrote: Donovan Baarda wrote: For the record, librsync version 0.9.6 is _almost_ ready in CVS. A bug has been detected but not isolated yet for large files (2G+). If it's not squashed soon I'm tempted to release anyway with a KNOWN_BUGS

Re: rsync on OpenVMS

2003-10-13 Thread Donovan Baarda
when funding is not available :-) -- Donovan Baarda [EMAIL PROTECTED] http://minkirri.apana.org.au/~abo/ -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

Pysync 2.24 release, was Re: rsync on OpenVMS

2003-10-17 Thread Donovan Baarda
On Tue, 2003-10-14 at 11:01, Donovan Baarda wrote: On Mon, 2003-10-13 at 13:00, John E. Malmberg wrote: jw schultz wrote: On Sun, Oct 12, 2003 at 12:38:40AM -0400, John E. Malmberg wrote: [...] I have not heard of unison. I have heard that pysync was successful in a limited test

Re: Synching text files.

2003-10-22 Thread Donovan Baarda
files is MSDOS CR/LF vs Unix LF line termination. This effectively makes a change every line, and unless you have lines longer than the block size you are using, rsync will not be able to find a single match. -- Donovan Baarda [EMAIL PROTECTED] http://minkirri.apana.org.au/~abo/ -- To unsubscribe

Re: how rsync works

2003-11-26 Thread Donovan Baarda
. At the time I read it I thought; good, something I can point people at when they ask me. I think it was a good resource. -- Donovan Baarda [EMAIL PROTECTED] http://minkirri.apana.org.au/~abo/ -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting

Re: File rename detection?

2004-01-28 Thread Donovan Baarda
implementing it is another story :-) -- Donovan Baarda [EMAIL PROTECTED] http://minkirri.apana.org.au/~abo/ -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

Re: [patch] Add `--link-by-hash' option.

2004-02-09 Thread Donovan Baarda
-5 +21 receiver.c -0 +6rsync.c -0 +7rsync.h If this does everything I think it does, then it's a surprisingly small amount of changes for what it does. __ -- Donovan Baarda [EMAIL PROTECTED] http

Re: MD4 checksum_seed

2004-03-14 Thread Donovan Baarda
worth you could count it as x32 bits worth of checksum, not the full 32 bits. -- Donovan Baarda [EMAIL PROTECTED] http://minkirri.apana.org.au/~abo/ -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart

Re: MD4 checksum_seed

2004-03-15 Thread Donovan Baarda
On Tue, 2004-03-16 at 10:44, Eran Tromer wrote: Hi, On 2004/03/15 03:49, Donovan Baarda wrote: [...] Just to be sure, I wrote a quickdirty collision search code for the rsync and librsync hashes. I used a generic collision-finding algorithm (namely Gabriel Nivasch's multistack variant

Re: MD4 checksum_seed

2004-03-16 Thread Donovan Baarda
G'day, From: Eran Tromer [EMAIL PROTECTED] Donovan Baarda wrote: [...] I thought that without using some sort of known vulnerability the formula would be; avg number of random attempts = number of possible sum values / 2 Uhm, no -- just 2^(64/2)=2^32 random blocks, by the Birthday

Re: sort of like a diff patch file

2004-03-28 Thread Donovan Baarda
G'day, On Mon, 2004-03-29 at 13:37, Steve W. Ingram wrote: Hi there, I was wondering if there was anyway to use rsync to effectively create a 'diff' file? is this a FAQ yet? A) rdiff. -- Donovan Baarda [EMAIL PROTECTED] http://minkirri.apana.org.au/~abo/ -- To unsubscribe or change

Re: sort of like a diff patch file

2004-03-28 Thread Donovan Baarda
you diff against, allowing you do calculate diffs without local access to the original you are diffing against. This can be handy for many applications. We've lost the original application enquired about... but certainly xdelta should be better if you have local access to both files. -- Donovan

librsync and rsync vulnerability to maliciously crafted data. was Re: MD4 checksum_seed

2004-04-04 Thread Donovan Baarda
Baarda [EMAIL PROTECTED] From: Eran Tromer [EMAIL PROTECTED] Donovan Baarda wrote: [...] I thought that without using some sort of known vulnerability the formula would be; avg number of random attempts = number of possible sum values / 2 Uhm, no -- just 2^(64/2)=2^32 random

Re: [librsync-devel] librsync and rsync vulnerability to maliciously crafted data. was Re: MD4 checksum_seed

2004-04-07 Thread Donovan Baarda
On Thu, 2004-04-08 at 12:36, Martin Pool wrote: On 5 Apr 2004, Donovan Baarda [EMAIL PROTECTED] wrote: librsync needs a whole file checksum. Without it, it silently fails for case 1), 3), and 4). Yes, a whole-file checksum should be used with it. Presumably something stronger than md4

Re: librsync and rsync vulnerability to maliciously crafted data. was Re: MD4 checksum_seed

2004-04-08 Thread Donovan Baarda
. Donovan Baardahttp://minkirri.apana.org.au/~abo/ -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http

Re: [librsync-devel] librsync and rsync vulnerability to maliciously crafted data. was Re: MD4 checksum_seed

2004-04-08 Thread Donovan Baarda
that the hash is done once per sample input, not once per compare. Sure, you still only need to try 2^(n/2) blocks, but you need to calculate 2^n hashes, and that's what really hurts. Donovan Baardahttp

Re: [librsync-devel] librsync and rsync vulnerability to maliciously crafted data. was Re: MD4 checksum_seed

2004-04-11 Thread Donovan Baarda
G'day, From: Wayne Davison [EMAIL PROTECTED] On Thu, Apr 08, 2004 at 03:50:48PM +1000, Donovan Baarda wrote: I think I've just realised what you were getting at; if the checksum_seed is based on something like the whole file md4sum, it becomes repeatable, but unpredictable. Not so. Copy

Re: gzip-rsyncable.diff

2004-05-03 Thread Donovan Baarda
. Donovan Baardahttp://minkirri.apana.org.au/~abo/ -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

Re: batch-mode fixes [was: [PATCH] fix read-batch SEGFAULT]

2004-05-18 Thread Donovan Baarda
. xdelta does the same thing, except it doesn't use signatures. It is much more like a efficient generic binary diff/patch tool. This makes it unsuitable for some applications, but its deltas are much more efficient. -- Donovan Baarda [EMAIL PROTECTED] http://minkirri.apana.org.au/~abo

Re: Rsync for program loading on embedded platforms

2004-06-01 Thread Donovan Baarda
analysed and the latest rsync incorporates dynamic checksum and block sizing based on that discussion. librsync doesn't have it yet, but it could easily be added. I'd be interested to hear of anyone using or contemplating the rsync algorithm on embedded systems. -- Donovan Baarda [EMAIL PROTECTED] http

Re: Rsync for program loading on embedded platforms

2004-06-02 Thread Donovan Baarda
to minimize the download-test loop by doing block-wise checksums... most of the time it didn't save you much, and rsync would have been a distinct improvement. Donovan Baardahttp://minkirri.apana.org.au/~abo

Re: Rsync for program loading on embedded platforms

2004-06-16 Thread Donovan Baarda
... it would be easy to implement and test all these variants using it. -- Donovan Baarda [EMAIL PROTECTED] http://minkirri.apana.org.au/~abo/ -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart

Re: [librsync-users] MD4 second-preimage attack

2006-03-01 Thread Donovan Baarda
would not need to include the seed. However, it is a no-overhead addition that should help. -- Donovan Baarda [EMAIL PROTECTED] http://minkirri.apana.org.au/~abo/ -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs