Patching for --omit-symlink-times?
I was wondering if anyone would have any input on a patch to add another command line feature, --omit-symlink-times. We recently ran into a problem with rsync (at least with the way we use it): - We need to preserve file modification times between the source and destination (--preserve-times) - The user performing the rsync may not always be the owner of the files on the destination (but is in the same group, and all files have group write permissions) The problem arrises when using symlinks. Files are always recreated (as the user performing the rsync) on the destination when the source file has a later mtime, regardless of changes. Symlinks however, are NOT recreated when the mtime differs; instead, the mtime is simply adjusted on the destination. This is problematic because a user who does not own a file or symlink can not adjust the mtime of the file to a time that isn't time(NULL) (no), see utimes(2). Thus rsync fails with failed to set times on The rest of the rsync completes, but our scripts rely on trusting the return codes. I assume this is the same reason why the --omit-dir-times option exists: because it is not trivial (or necessary) to recreate a directory to update the mtime. So, my solution was to create a patch to add a --omit-symlink-times option, which simply skips trying to preserve the mtime on symlinks. It works exactly like the --omit-dir-times option. My question is: - Is there another way around this (such as forcing rsync to recreate symlinks that haven't changed)? - Would you be interested in applying the patch to the rsync source tree? Thanks! Steve Polyack -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
RFE: create batch without patching
Hi, Currently, rsync --write-batch creates the batch fileset and also updates the destination. I suggest adding the ability to disable the latter. Motivation: I wish to sync two large but similar directories residing in different machines. There is a slow (dial-up) network connection between the machines, but I have physical access to both. Thus, I would like to create a batch fileset by transmitting only file lists and checksums over the slow network, and then transfer the batch fileset on CD-R. The only essential difficulty I see is that you can't verify the MD5 sum until you do the actual patching, so such a blind batch fileset may be faulty. In the above scenario that's not a serious problem -- after patching from the batch fileset on the CD-R, I'll run rsync in normal mode over the network to fix whatever's broken (or just rely on a large csum_length, once the MD4 hash checksum_seed issue I explained yesterday is addressed). Other plausible motivating scenarios: * Using a patch fileset for an incremental backup that lets you to revert to the base copy * Estimating the bandwidth/time an rsync operation will take before actually carrying it out * Preparing several filesets for different scenarios and applying the appropriate one later Eran -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Patching?
On Sun, Sep 07, 2003 at 03:34:32PM -0700, jw schultz wrote: I'm thinking the protocol version MIN would be good to add, it might not help for this cycle but would in future. This also ties in with something i suggested earlier for the sake of testing, a --protocol option to explicitly force a downgrade. I agree with both ideas, and have worked up a patch (though the new option is, as yet, undocumented): http://www.blorf.net/rsync-protocol.patch I'll go ahead and check this in soon, since I can't see any downside to doing this. Note that this change should help out with this cycle, since this change allows someone to just bump down the PROTOCOL_VERSION in rsync.h and the code will properly skip all the new-protocol extensions. It will also give us a work-around for talking to a pre-release server that someone might have left lying around, unsupported. E.g., if the remote system says it supports protocol 27, but it's a CVS version that doesn't really support all of 27, you can use the new --protocol=26 option to force the two to talk using an earlier protocol version. ..wayne.. -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Patching?
On Sat, Sep 06, 2003 at 04:01:15PM -0700, jw schultz wrote: What you should not do is run a public rsync server on CVS having a protocol bump (current HEAD) unless you patch it to downgrade the protocol version. That's a good idea, but keep in mind that the patch would involve more than just tweaking PROTOCOL_VERSION from 27 to a 26 in rsync.h. The code assumes that it can handle P.V. 27, so if the remote side says that it supports P.V. 27, the local side would interact using the new csum code, even if it told the remote side that it was only P.V. 26 (which would totally confuse the remote side). Thus, all the remote_version checks for 27 would have to be disabled as well in such a downgraded version of rsync. ..wayne.. -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Patching?
On Sun, Sep 07, 2003 at 12:43:01PM -0700, Wayne Davison wrote: On Sat, Sep 06, 2003 at 04:01:15PM -0700, jw schultz wrote: What you should not do is run a public rsync server on CVS having a protocol bump (current HEAD) unless you patch it to downgrade the protocol version. That's a good idea, but keep in mind that the patch would involve more than just tweaking PROTOCOL_VERSION from 27 to a 26 in rsync.h. The code assumes that it can handle P.V. 27, so if the remote side says that it supports P.V. 27, the local side would interact using the new csum code, even if it told the remote side that it was only P.V. 26 (which would totally confuse the remote side). Thus, all the remote_version checks for 27 would have to be disabled as well in such a downgraded version of rsync. You're right. Fooey! I was assuming we did a remote_version = MIN(remote_version, PROTOCOL_VERSION) but we don't. Except during a protocol bump we effectively do that because a given rsync wouldn't know to send a higher protocol than it reports. I'm thinking the protocol version MIN would be good to add, it might not help for this cycle but would in future. This also ties in with something i suggested earlier for the sake of testing, a --protocol option to explicitly force a downgrade. -- J.W. SchultzPegasystems Technologies email address: [EMAIL PROTECTED] Remember Cernan and Schmitt -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Patching?
On Fri, Sep 05, 2003 at 05:39:09PM -0500, Max Kipness wrote: Hello, Hopefully this isn't too stupid of a question, but how do you patch rsync with the --files-from patch? Don't. Just get CVS head which includes the --files-from option as well as a number of bug fixes and other enhancements. -- J.W. SchultzPegasystems Technologies email address: [EMAIL PROTECTED] Remember Cernan and Schmitt -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Some C help patching sender.c (from:plain source - encrypteddestination: rsync + gpg)
Hi, I am exploring extending Kyle Jones' patch (which implements post-transfer filter in receiver.c) to pre-filter or otherwise pre-process the file before it is sent. Kyle Jones patch can be found here http://groups.google.com/groups?hl=enlr=ie=UTF-8selm=b6f55s%24256q%241%40FreeBSD.csie.NCTU.edu.twrnum=6 and the rationale for what I am trying to do here http://marc.theaimsgroup.com/?l=rsyncm=105660948415347w=2 Down to what I am asking help with: I am _not_ a C hacker at all, my strenghts are Perl -- if anywhere at all. So I would appreciate your advise before I shoot myself in the foot. Thanks for your patience. I have studied the sources and gotten down to sender.c (cvs 1.17), in particular the line where it calls do_open() and I am intending to create do_popen as an alternative to do_open() (in syscall.c). I have seen Kyle's approach of handling the forking directly, and I know I cannot handle that level of complexity. Using something like popen (I am using http://www.gnu.org/manual/glibc-2.2.3/html_chapter/libc_15.html as reference) I can probably get away with. One of my concerns is that even though we are setting --whole-file, when I follow the program logic and the output of test runs with -vvv it is still computing CRCs and (at least seems to be) preparing deltas. Hmmm, re-reading it now it seems that the do_open() I am thinking of is in an if(!read_batch) block and the CRC/deltas are under if(read_batch). Is read_batch always true when using --whole-files? Seems to be. Second concern: where is the reading and transmission of th file happening? If the do_open is in line 183, where's the read/transmit, map_file() line 202 puts the data uin buf, and then, what? You see, write_int doesn't use buf. And where is the handle closed? Thanks for any tips here... Ahhh, life. martin http://nzl.com.ar/ -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Some C help patching sender.c (from:plain source - encrypteddestination: rsync + gpg)
On Mon, Jul 07, 2003 at 11:42:04PM +1200, Martin Langhoff wrote: Hi, I am exploring extending Kyle Jones' patch (which implements post-transfer filter in receiver.c) to pre-filter or otherwise pre-process the file before it is sent. Kyle Jones patch can be found here http://groups.google.com/groups?hl=enlr=ie=UTF-8selm=b6f55s%24256q%241%40FreeBSD.csie.NCTU.edu.twrnum=6 and the rationale for what I am trying to do here http://marc.theaimsgroup.com/?l=rsyncm=105660948415347w=2 Down to what I am asking help with: I am _not_ a C hacker at all, my strenghts are Perl -- if anywhere at all. So I would appreciate your advise before I shoot myself in the foot. My _advice_ is to miss. Thanks for your patience. I have studied the sources and gotten down to sender.c (cvs 1.17), in particular the line where it calls do_open() and I am intending to create do_popen as an alternative to do_open() (in syscall.c). I have seen Kyle's approach of handling the forking directly, and I know I cannot handle that level of complexity. Using something like popen (I am using http://www.gnu.org/manual/glibc-2.2.3/html_chapter/libc_15.html as reference) I can probably get away with. Fine, as long as it is for your own use only. One of my concerns is that even though we are setting --whole-file, when I follow the program logic and the output of test runs with -vvv it is still computing CRCs and (at least seems to be) preparing deltas. Correct. As can be found by a grep for whole_file, whole-file works by operating on an empty blocksum array. Besides we want the CRCs calculated as an extra level of integrity checks. Hmmm, re-reading it now it seems that the do_open() I am thinking of is in an if(!read_batch) block and the CRC/deltas are under if(read_batch). Is read_batch always true when using --whole-files? Seems to be. No. Read_batch is only true when set using the command-line option. Second concern: where is the reading and transmission of th file happening? If the do_open is in line 183, where's the read/transmit, map_file() line 202 puts the data uin buf, and then, what? You see, write_int doesn't use buf. And where is the handle closed? Look further down, buf is transmitted by match_sums() and fd is closed shortly thereafter. -- J.W. SchultzPegasystems Technologies email address: [EMAIL PROTECTED] Remember Cernan and Schmitt -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Some C help patching sender.c (from:plain source - encrypteddestination: rsync + gpg)
On Tue, Jul 08, 2003 at 07:45:00AM +1200, Martin Langhoff wrote: I have seen Kyle's approach of handling the forking directly, and I know I cannot handle that level of complexity. Using something like popen (I am using http://www.gnu.org/manual/glibc-2.2.3/html_chapter/libc_15.html as reference) I can probably get away with. Fine, as long as it is for your own use only. I don't want to sound more stupid that I naturally am, but could you elaborate? Is this because you generally don't consider this patch interesting for the main trunk or because there are problems with popen() that I should know about? And, if there are problems with popen, what are those? Not a case of known problems but one of caution regarding portability. Given that this is is unlikely to be in mainline for a long while such caution may be excessive. One of my concerns is that even though we are setting --whole-file, when I follow the program logic and the output of test runs with -vvv it is still computing CRCs and (at least seems to be) preparing deltas. Correct. As can be found by a grep for whole_file, whole-file works by operating on an empty blocksum array. Besides we want the CRCs calculated as an extra level of integrity checks. Ok. Then I'll probably want to fudge the CRC's, although probably Kyle's patch is already taking care of that. Just let match_sums do its work as always. Besides, depending on what external processing you do, whole-file might not be optimal. (...) Second concern: where is the reading and transmission of th file happening? If the do_open is in line 183, where's the read/transmit, map_file() line 202 puts the data uin buf, and then, what? You see, write_int doesn't use buf. And where is the handle closed? Look further down, buf is transmitted by match_sums() and fd is closed shortly thereafter. Yay! match_sums() is actually doing the transmissions. Hmmm. Let's see what I can glean from that. Thanks! martin -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html -- J.W. SchultzPegasystems Technologies email address: [EMAIL PROTECTED] Remember Cernan and Schmitt -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
