RE: Option --numeric-ids not the default?
Hi, > I've browsed the history of the list, but can't seem to find > an answer to something that I find quite surprising - why > isn't --numeric-ids the default when rsync is told to > preserve permissions? because all archive-tools do preserve the rights linked to the username and not linked to the uid. e.g. for TAR you need --numeric-owner, if you don't want to change the uid. I find this behaviour quite useful, if you have more than one box, maybe different *nix distribuitions, but files owned by user X belong to user X on the other box, even or especially when they don't have the same UID there. Preserving uids with --numeric-ids seems to me more a special case - used with rsnapshot, for example, or for using rsync when booting from a rescure-linux-cd like knoppix, or similar, that use different usernames/ids. regards. Werner Maier. -- Werner Maier, Dipl.-Ing. Univ. Friedrich-Bergius-Ring 15 fidion GmbH97076 Würzburg -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
RE: rsync is concatenating the files together
Wayne, I will try some system call tracing, but as I said in my initial post, I am using NFS, so there shouldn't be any transport cleanliness issues. : Re: rsync is concatenating the files together > > On Thu, Nov 11, 2004 at 05:38:56PM -0600, Werner wrote: > > I'm wondering perhaps if this is more a Solaris 10b69 bug than an > > rsync bug? > > I would imagine so. I can only suggest (1) try a different transport > (perhaps the remote shell you're using is not binary clean, and is thus > eating the characters that would differentiate the files from one > another in the data stream); (2) specify a much higher level of > verbosity (so that rsync tells you what it thinks it is doing at every > step); and/or (3) run rsync using a system-call-tracing program (which > should help you to determine if the system calls are messing up on the > sending or receiving side -- see http://rsync.samba.org/issues.html). > > My bet is that the problem is that transport is losing data. > > ..wayne.. -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
rsync is concatenating the files together
I am seeing a really odd problem with Rsync 2.6.2 and 2.6.3. I am trying to rsync a directory containing some 2gb files from a Solaris 9 NFS server to a Solaris 10b69 NFS client. Rsync is running on the Solaris 10 client. Here is the source directory as seen from the client: Source directory: -rwxr--r-- 1 root priv 1270666114 Jan 3 2004 FLY.GHO* -rwxr--r-- 1 root priv 1595131867 Nov 11 2003 LB.GHO* -rwxr--r-- 1 root priv 263738235 Jan 3 2004 PHANT001.GHS* -rwxr--r-- 1 root priv 2147482739 Jan 3 2004 PHANTXP.GHO* -rwxrw-r-- 1 root priv 1489966355 Dec 24 2003 ROVER.GHO* -rwxr--r-- 1 root priv 2147482481 Jan 3 2004 TOSH.GHO* -rwxr--r-- 1 root priv 2147480582 Jan 3 2004 TOSH0001.GHS* -rwxr--r-- 1 root priv 2070930486 Jan 3 2004 TOSH0002.GHS* Using 2.6.2 with -a -H -S -v drwxr-xr-x 2 root other512 Nov 11 16:58 ./ drwxr-xr-x 16 root root 512 Nov 10 20:53 ../ -rwx-- 1 root root 6366602929 Nov 11 17:06 .TOSH.GHO.lRaGei* -rwxr--r-- 1 root priv 1270666114 Jan 3 2004 FLY.GHO* -rwxr--r-- 1 root priv 1595131867 Nov 11 2003 LB.GHO* -rwxr--r-- 1 root priv 263738235 Jan 3 2004 PHANT001.GHS* -rwxr--r-- 1 root priv 2147482739 Jan 3 2004 PHANTXP.GHO* -rwxrw-r-- 1 root priv 1489966355 Dec 24 2003 ROVER.GHO* Look at the size of the .TOSH.GHO.lraGei file! It's a concatenation of the three TOSH files from the source tree! At this point, rsync hangs when it reaches the end of the TOSH0002.GHS file. Using 2.6.3 with -a -H -S --inplace -v -rwxr--r-- 1 root priv 1270666114 Jan 3 2004 FLY.GHO* -rwxr--r-- 1 root priv 1595131867 Nov 11 2003 LB.GHO* -rwxr--r-- 1 root priv 263738235 Jan 3 2004 PHANT001.GHS* -rwxr--r-- 1 root priv 2147482739 Jan 3 2004 PHANTXP.GHO* -rwxrw-r-- 1 root priv 1489966355 Dec 24 2003 ROVER.GHO* -rwxr--r-- 1 root priv 2147482481 Jan 3 2004 TOSH.GHO* -- 1 root root 4218739196 Nov 11 17:20 TOSH0001.GHS Now I tried 2.6.3 with the --inplace option. Interestingly enough, it copied the TOSH.GHO file over, then concatenated TOSH0001.GHS and THS0002.GHS into TOSH0001.GHS on the receiving file system. At this point, rsync hung again. The problem seems to be somewhat random, as it happened several times with the PHANT* files, but it eventually copied them properly, after several attempts. I'm wondering perhaps if this is more a Solaris 10b69 bug than an rsync bug? Any thoughts on this? -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: 'Invalid cross-device link' message on sparc
> If you want to use --link-dest, you will have to point to a place on the > same filesystem containing the stuff you're linking. > --link-dest=DIR create hardlinks to DIR for unchanged files Yes, I know. But what I was expecting was the behaviour I get on i386: rsync should recognise that it can't use/recycle the file because it is on a different filesystem and make a new copy. Source and destination should be as identical as possible, no matter how 'useful' the data in the directory specified with '--link-dest' is. Or at least I would expect an error message. As I mentioned before, I am worried that the error code of the link system call is only printed in verbose mode. I could imagine a dozen of horrible things that could happen while rsync is running and it would be important that the user gets a notice that something went wrong. Werner Augustin -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: 'Invalid cross-device link' message on sparc
Werner Augustin <[EMAIL PROTECTED]> writes: > and when I try: > > e/ dest > building file list ... done > created directory dest > ./ > dir/ > link /tmp/rsync/unusable_link-dest/dir/foo => dir/foo : Invalid cross-device link > > wrote 106 bytes read 20 bytes 252.00 bytes/sec > total size is 9 speedup is 0.07 Oops, something went wrong when I copied this, actually I tried: [EMAIL PROTECTED]:/tmp/rsync% /usr/local/rsync-20040311/bin/rsync -a -v --link-dest=/tmp/rsync/unusable_link-dest source/ dest building file list ... done created directory dest ./ dir/ link /tmp/rsync/unusable_link-dest/dir/foo => dir/foo : Invalid cross-device link wrote 106 bytes read 20 bytes 252.00 bytes/sec total size is 9 speedup is 0.07 bye, Werner Augustin -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
'Invalid cross-device link' message on sparc
Hi, I've got problems with a symlink to another device in a directory used with '--link-dest'. I've got something like [EMAIL PROTECTED]:/tmp/rsync% ls -alR .: total 16 drwxr-xr-x4 augustin augustin 4096 Mar 15 12:07 ./ drwxrwxrwt4 root root 4096 Mar 15 10:56 ../ drwxr-xr-x3 augustin augustin 4096 Mar 11 15:05 source/ drwxr-xr-x2 augustin augustin 4096 Mar 11 15:06 unusable_link-dest/ ./source: total 12 drwxr-xr-x3 augustin augustin 4096 Mar 11 15:05 ./ drwxr-xr-x4 augustin augustin 4096 Mar 15 12:07 ../ drwxr-xr-x2 augustin augustin 4096 Mar 11 15:05 dir/ ./source/dir: total 12 drwxr-xr-x2 augustin augustin 4096 Mar 11 15:05 ./ drwxr-xr-x3 augustin augustin 4096 Mar 11 15:05 ../ -rw-r--r--1 augustin augustin9 Mar 11 12:17 foo ./unusable_link-dest: total 8 drwxr-xr-x2 augustin augustin 4096 Mar 11 15:06 ./ drwxr-xr-x4 augustin augustin 4096 Mar 15 12:07 ../ lrwxrwxrwx1 augustin augustin 19 Mar 11 15:06 dir -> /home2/augustin/dir/ and [EMAIL PROTECTED]:/tmp/rsync% ls -alR /home2/augustin/dir /home2/augustin/dir: total 20 drwxr-xr-x2 augustin augustin 4096 Mar 11 12:07 ./ drwxr-xr-x 229 augustin liinstud12288 Mar 15 12:06 ../ -rw-r--r--1 augustin augustin9 Mar 11 12:17 foo and when I try: e/ dest building file list ... done created directory dest ./ dir/ link /tmp/rsync/unusable_link-dest/dir/foo => dir/foo : Invalid cross-device link wrote 106 bytes read 20 bytes 252.00 bytes/sec total size is 9 speedup is 0.07 I get: [EMAIL PROTECTED]:/tmp/rsync% ls -alR dest dest: total 12 drwxr-xr-x3 augustin augustin 4096 Mar 11 15:05 ./ drwxr-xr-x5 augustin augustin 4096 Mar 15 12:10 ../ drwxr-xr-x2 augustin augustin 4096 Mar 11 15:05 dir/ dest/dir: total 8 drwxr-xr-x2 augustin augustin 4096 Mar 11 15:05 ./ drwxr-xr-x3 augustin augustin 4096 Mar 11 15:05 ../ I've tested this with rsync-HEAD-20040311-1010GMT.tar.gz (no special configure options) using Solaris 2.7 and Debian Woody on a UltraSparc I box. Everything is fine using Debian on i386. Perhaps something with 32/64 bit? Linux-Kernel is 64bit the libraries used are 32bit: [EMAIL PROTECTED]:/tmp/rsync% ldd /usr/local/rsync-20040311/bin/rsync libpopt.so.0 => /lib/libpopt.so.0 (0x7002c000) libresolv.so.2 => /lib/libresolv.so.2 (0x70044000) libc.so.6 => /lib/libc.so.6 (0x70064000) /lib/ld-linux.so.2 => /lib/ld-linux.so.2 (0x7000) /tmp is on a SCSI hard disk. Any hints, any ideas what might go wrong? Btw. I had a look at the code in generator.c. Wouldn't it be a good idea to report a critical error and not only a message in verbose mode if do_link fails? bye, Werner Augustin -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: [PATCH] Infinite recursion in rsync --server
Hi, --Aaron Hopkins <[EMAIL PROTECTED]> Samstag, 6. Juli 2002 16:20 -0700: > With rsync --server, sometimes when the remote side control-C's a transfer, > rsync will immediately start consuming all available RAM. > [...] > This continues until the server kills the out-of-control process due to lack > of memory, obviously making everything else on the server unhappy for a > while in the process. > > rwrite () knows about this possible recursion, but only handles it properly > with "am_daemon". > > This bug is present in at least 2.5.4 and 2.5.5. The simple attached patch > fixes this, and patches cleanly against either. > > If for some reason the MIME gets eaten, you can also pick it up from: > > https://www.die.net/tmp/50dff5d6fd24afd8e731eb2fa3f86556/rsync-2.5.4-oom.patch > [...] patch worked fine and did resolve my problem. thanks. Please Excuse: I did search BUGs and TODO, but I missed searching the archive before posting ;-) best regards. Werner Maier. -- Werner Maier, Dipl.-Ing. Univ.E-Mail: [EMAIL PROTECTED] fidion GmbH Berner Straße 2, 97084 Würzburg -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.tuxedo.org/~esr/faqs/smart-questions.html
rsync 2.5.5 consuming memory and spirals server down
Hi there, I've a problem with rsync - see below. (by the way: I'm using rsync since 2 years now, good work!) my environment: - linux, 2.4, Dual-PIII-1.2GHz, 1 GB RAM. more of these machines. - rsync 2.5.5 - openssh 3.3 the machines are on a lan, so bandwith is not a problem. I sync large directories (about 6 GB growing) between the machines, no problem - memory consumtion of rsync is normal if not pressed cntrl-c, see below. for rsync I use rsync -cvrlpgo \ --delete \ --exclude=... \ [more of them] -e ssh \ server:/path /path machines are configured to do rsa-pubkey authentication on ssh, so this can be done by a cron job. but: if I try it manually and stop rsync using cntrl-C, while it prints "receiving file list..." the rsync server process on the other machine immediately grabbs all memory in seconds (about 100 Meg per Second), spiraling the machine down if 2 Gig swap and 1 Gig RAM are used. But the best (Coders will be happy): it comes EVERY time I use cntrl-C, even if it's a very small directory to sync. any hints? (maybe setting ulimit for this ssh-connection could help, I haven't tried it yet - just discovered the "bug/strange behaviour" in my environment quite now and have been able to reproduce it). If You need additional info, I'll try to provide it. best regards. Werner Maier. -- Werner Maier, Dipl.-Ing. Univ.E-Mail: [EMAIL PROTECTED] fidion GmbH Berner Straße 2, 97084 Würzburg -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.tuxedo.org/~esr/faqs/smart-questions.html
