Numeric ids
HI, I had read from the rsync man page on transferring files using the user's numeric id and gid. I have a user which resides on 2 remote sites, having the same user names, but different uid and gid on both sites. I'm transferring the files as a root user. site A site B --- usr_A usr_A uid = 100 uid = 200 file1(usr_A) However from the diagram above, I would like transfer file1 from site A to site B to usr_A, and file1 does not exists for usr_A in site B. However, when copying file1 from A to B, I would like to have usr_A's numeric id in site A to be copied to usr_A's numeric id in site B. I've tried using the following options, but it did preserve usr_A's uid when file1 reaches site B: rsync -avzo --numeric-ids siteA:/tmp/file1 siteB:/tmp Is there a way which I can preserve the uid of file1 when it's been transferred to site B, hence automatically updating usr_A's uid in site B to 100? Thanks. CJ -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.tuxedo.org/~esr/faqs/smart-questions.html
Re: Improving the rsync protocol (RE: Rsync dies)
On Mon, May 20, 2002 at 09:35:04PM +1000, Martin Pool wrote: On 17 May 2002, Wayne Davison [EMAIL PROTECTED] wrote: On Fri, 17 May 2002, Allen, John L. wrote: [...] I've been thinking about this too. I think the top-level question is Start from scratch with a new protocol, or try to work within the current one? tough question... to avoid backwards breakage and yet implement something significantly better you would probably have to make two rsyncs in one executable; the new protocol, and the old one for a compatible fallback talking to old versions. After enough time had passed and all old rsync implementations had been purged, the old code could be dropped, leaving a nice clean small solution. I tend to think that once a delta compressing http extension gets mainstream acceptance, rsync will fade away _unless_ it offers significantly better performance by avoiding http overheads (which is why ftp lives on, despite being a bastard protocol from hell). algorithm or codebase, or need to evolve the current one. I think the nature of the current protocol is that it will be hard to make really fundamental improvements without rewriting it. [...] My feelings too. I wrote librsync. There is some documentation and I can add more if there's anything undocumented. I really want to do some work on librsync. I recently did some work writing a Python swig wrapper for it and identified several areas where it could be improved. I already have a more modular implementation of the rolling checksum that is 2~3x faster that I want to integrate with it. I haven't looked at pysync as much as it deserves, but it could be a good foundation. Pysync now includes a Python extension module to librsync. I have also implemented a Python wrapper around the rolling checksum mentioned above that makes the Python version run nearly 2x as fast as the pure Python adler32 version. This will be released in the next release. I don't think Python is viable for a final rsync solution, even using librsync as an extension module. The Python interpreter is an un-necisary overhead for a lean-mean-data-transfer-machine. However, Python is _perfect_ as a means of experimenting with protocols and algorithms. Pysync was written with this in mind, and in ~400 lines of heavily commented Python implements both the rsync and xdelta algorithms. The next release will add inverse-rsync (for client-side delta calculation). Note that pysync _does_ implement the context-compression used by rsync that is not included in librsync yet. -- -- ABO: finger [EMAIL PROTECTED] for more info, including pgp key -- -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.tuxedo.org/~esr/faqs/smart-questions.html
Re: Improving the rsync protocol (RE: Rsync dies)
On Mon, May 20, 2002 at 10:58:33PM +1000, Donovan Baarda wrote: | On Mon, May 20, 2002 at 09:35:04PM +1000, Martin Pool wrote: | On 17 May 2002, Wayne Davison [EMAIL PROTECTED] wrote: | On Fri, 17 May 2002, Allen, John L. wrote: | [...] | I've been thinking about this too. I think the top-level question is | |Start from scratch with a new protocol, or try to work within the |current one? | | tough question... to avoid backwards breakage and yet implement something | significantly better you would probably have to make two rsyncs in one | executable; the new protocol, and the old one for a compatible fallback | talking to old versions. After enough time had passed and all old rsync | implementations had been purged, the old code could be dropped, leaving a | nice clean small solution. | | I tend to think that once a delta compressing http extension gets mainstream | acceptance, rsync will fade away _unless_ it offers significantly better | performance by avoiding http overheads (which is why ftp lives on, despite | being a bastard protocol from hell). I don't see how HTTP can hope to displace RSYNC. Both have flaws, but HTTP has way more. Work is chopping away at them, but the results are often not very good. Persistent connections, for example, are a bastardization that just isn't handled effectively (for example it almost always breaks with dynamic content such as CGI). IMHO, it is HTTP that needs to be rewritten entirely from scratch. Also, I'm really not in favor of so many of these one size fits all kinds of solutions. An example is XML. It's a format. But as we try to overload it more and more as a solution to all problems, we just end up squishing it down to being a yet-another-layer in an ever growing stack, which only ends up with more and more needless layers of software. As for RSYNC, it has a basic purpose. As long as it stays focused, then it can serve that purpose well. Of course I have ideas to make it better, but I certainly don't want to see it go away even if none of those are ever adopted (it's too useful the way it is). -- - | Phil Howard - KA9WGN | Dallas | http://linuxhomepage.com/ | | [EMAIL PROTECTED] | Texas, USA | http://phil.ipal.org/ | - -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.tuxedo.org/~esr/faqs/smart-questions.html
exit code 23 - inappropriate error for copying symlinks?
Hi, I've just created a small directory as an example, it holds a file and two symlinks that don't point to anything, and another symlink going nowhere in a subdirectory. rsync (2.5.5) is happy to copy these but produces an error. I can't see why it considers this an error? It's not even aborting after it first encounters this 'error'. $ ls -lR /var/tmp/rsync.test /var/tmp/rsync.test: total 8 drwxr-xr-x 2 kjw group512 May 20 15:45 deeper -rw-r--r-- 1 kjw group 6 May 20 15:29 hello.world lrwxrwxrwx 1 kjw group 9 May 20 15:30 symbolic.link - inky-void lrwxrwxrwx 1 kjw group 11 May 20 15:38 symbolic.link.2 - inky-void.2 /var/tmp/rsync.test/deeper: total 2 lrwxrwxrwx 1 kjw group 11 May 20 15:45 symbolic.link.3 - inky-void.3 $ rsync -vaz /var/tmp/rsync.test otherhost:/var/tmp building file list ... done rsync.test/ rsync.test/deeper/ stat(rsync.test/deeper/symbolic.link.3) : No such file or directory rsync.test/deeper/symbolic.link.3 - inky-void.3 rsync.test/hello.world stat(rsync.test/symbolic.link) : No such file or directory rsync.test/symbolic.link - inky-void stat(rsync.test/symbolic.link.2) : No such file or directory rsync.test/symbolic.link.2 - inky-void.2 rsync.test/ rsync.test/deeper/ wrote 278 bytes read 36 bytes 209.33 bytes/sec total size is 37 speedup is 0.12 rsync error: partial transfer (code 23) at main.c(578) $ echo $? 23 $ rsh otherhost ls -lR /var/tmp/rsync.test /var/tmp/rsync.test: total 8 drwxr-xr-x 2 kjw group512 May 20 15:45 deeper -rw-r--r-- 1 kjw group 6 May 20 15:29 hello.world lrwxrwxrwx 1 kjw group 9 May 20 15:48 symbolic.link - inky-void lrwxrwxrwx 1 kjw group 11 May 20 15:48 symbolic.link.2 - inky-void.2 /var/tmp/rsync.test/deeper: total 2 lrwxrwxrwx 1 kjw group 11 May 20 15:48 symbolic.link.3 - inky-void.3 regards |evin -- Kevin J Walters Morgan Stanley [EMAIL PROTECTED], [EMAIL PROTECTED] 20 Cabot Square Tel: 020 7425 7886 Canary Wharf Fax: 020 7677 8504 London E14 4QW -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.tuxedo.org/~esr/faqs/smart-questions.html
Re: exit code 23 - inappropriate error for copying symlinks?
On Mon, May 20, 2002 at 10:59:41AM -0400, Crisler, Jon wrote: | Error 23 is an incomplete copy. As I understand it, there is an | inconsistancy between the first check for differences, and the start of the | copy process. | | I run into this all the time, but for my implemention it is a legitimate | error. If the receiver creates the symlink, isn't that complete? If so, why would there need to be an error just because the symlink happens to be dangling/lame? If the source tree has a dangling symlink, IMHO, it should be copied and made dangling in the destination, without that being treated as an error (there are tools to scan file trees for dangling links). You wouldn't think of treating a device node file as an error just because the device it identifies doesn't happen to be configured in the system, would you? OTOH, if the symlink is valid at the source, but points to something that cannot be copyed by rsync (outside the tree or excluded), and so it would become dangling at the destination, then I can see treating that as an error in some way. These semantics are often difficult to deal with, and so options that describe what the user prefers to happen seem to be the solution. One thing I'd like to see is symlinks that work their way through points outside of what rsync is copying and back in (such as absolute links, but also including relative links going outside then back in), be translated so they point to the same object using the same means on the destination system. For example an absolute link would become an absolute link on the destination, but changed to reflect the change in filesystem point in the destination (if actually changed). Relative would be more complicated. Clearly an option is needed to enable this as it can potentiall break established things not expecting this. -- - | Phil Howard - KA9WGN | Dallas | http://linuxhomepage.com/ | | [EMAIL PROTECTED] | Texas, USA | http://phil.ipal.org/ | - -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.tuxedo.org/~esr/faqs/smart-questions.html
Rsync and files larger than 2 GB in size on HP-UX 64bit
Hello, I read the suggestion posted on this list about adding -D_LARGEFILE64_SOURCE to build rsync, but it does not work correctly on my platform which is an: 9000/800/N4000-55 with 64bit hp-ux using HP's C compiler (B3901BA) i solved the problem by building rsync as a 64 bit application: CFLAGS='+O3 +DD64' ./configure --prefix=/opt/rsync could you please add this flag to configure if getconf KERNEL_BITS returns 64 regards, L. P.S. i am not on this list so please flame me privately -- Luca Berra -- [EMAIL PROTECTED] Communication Media Services S.r.l. /\ \ / ASCII RIBBON CAMPAIGN XAGAINST HTML MAIL / \ -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.tuxedo.org/~esr/faqs/smart-questions.html
shell title
UHm, I don't know nothing about the matter but some program have the ability to change shell's caption/title... could be cool an rsync option to do that (easier to check the transfer from a minimized window). Also, if the total file number also transmits the total byte size, a gran total % could be useful as a global progess meter. -- Lapo 'Raist' Luchini [EMAIL PROTECTED] (PGP X.509 keys available) http://www.lapo.it (ICQ UIN: 529796) -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.tuxedo.org/~esr/faqs/smart-questions.html
rsync backup hangs at boot
rsync seems to work fine copying from the running system. Files match as far as a random check. I've been hoping to simplify life by rsyncing from the running system to have a bootable backup.Most conversations on the list suggest this is easy, and I've gotten it to happen, and now I've forgotten to do some detail ( /etc/fstab is corrected to refer to the device name of the drive that is booted by grub) here is my output and rsync script: from dmesg: = everything was fine until autodetecting RAID arrays autorun ... ... autorun DONE. NET4: Linux TCP/IP 1.0 for NET4.0 IP Protocols: ICMP, UDP, TCP, IGMP IP: routing cache hash table of 1024 buckets, 8Kbytes TCP: Hash tables configured (established 8192 bind 8192) NET4: Unix domain sockets 1.0/SMP for Linux NET4.0. ds: no socket drivers loaded! VFS: Mounted root (ext2 filesystem) readonly. Freeing unused kernel memory: 236k freed Warning: unable to open an initial console hub.c: USB new device connect on bus2/2, assigned device number 2 hub.c: USB hub found hub.c: 4 ports detected script that runs rsync to a partition or disk mounted on /safety #!/bin/sh if [ -d /safety/proc ] then echo else mkdir /safety/proc fi /opk/bin/rsync -acHv --delete \ --exclude /download \ --exclude /proc/ \ --exclude /var/spool/cups/ \ --exclude /seconddisk \ --exclude /safety \ --exclude /safety2 \ --exclude /etc/fstab \ --exclude /etc/mtab \ / /safety if [ -f /etc/fstab ] then echo else echo No /etc/fstab cp /etc/fstab /safety/etc fi if [ -f /etc/mtab ] then echo else echo No /etc/mtab cp /etc/mtab /safety/etc fi -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.tuxedo.org/~esr/faqs/smart-questions.html
Fwd: rsync backup hangs at boot
found that rsync 2.50 packaged as openPKG rsync-2.5.0-1.0.1.src.rpm had this problem when compiled with redhat gcc 2.96 -- Forwarded Message -- Subject: rsync backup hangs at boot Date: Mon, 20 May 2002 12:10:55 -0500 From: John Griessen [EMAIL PROTECTED] To: [EMAIL PROTECTED] rsync seems to work fine copying from the running system. Files match as far as a random check. I've been hoping to simplify life by rsyncing from the running system to have a bootable backup.Most conversations on the list suggest this is easy, and I've gotten it to happen, and now I've forgotten to do some detail ( /etc/fstab is corrected to refer to the device name of the drive that is booted by grub) here is my output and rsync script: from dmesg: = everything was fine until autodetecting RAID arrays autorun ... ... autorun DONE. NET4: Linux TCP/IP 1.0 for NET4.0 IP Protocols: ICMP, UDP, TCP, IGMP IP: routing cache hash table of 1024 buckets, 8Kbytes TCP: Hash tables configured (established 8192 bind 8192) NET4: Unix domain sockets 1.0/SMP for Linux NET4.0. ds: no socket drivers loaded! VFS: Mounted root (ext2 filesystem) readonly. Freeing unused kernel memory: 236k freed Warning: unable to open an initial console hub.c: USB new device connect on bus2/2, assigned device number 2 hub.c: USB hub found hub.c: 4 ports detected s -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.tuxedo.org/~esr/faqs/smart-questions.html
Re: Improving the rsync protocol (RE: Rsync dies)
On 20 May 2002, Phil Howard [EMAIL PROTECTED] wrote: On Fri, May 17, 2002 at 01:42:31PM -0700, Wayne Davison wrote: | On Fri, 17 May 2002, Allen, John L. wrote: | In my humble opinion, this problem with rsync growing a huge memory | footprint when large numbers of files are involved should be #1 on | the list of things to fix. | | I have certainly been interested in working on this issue. I think it | might be time to implement a new algorithm, one that would let us | correct a number of flaws that have shown up in the current approach. OTOH, this mode of operation also needs to be retained. While I certainly would love to have an rsync that can keep millions of files in sync all at once, I also have need for an rsync that can readily detect files being moved around. There are obvious difficulties in combining those needs, so it should be a deployment issue to decide what to use. I think detecting files moved across directories is an example of something that would be a bit complicated and error-prone to insert into the current protocol. One may of handling them would be to scan the entire destination tree getting the stat() info for each file. We can then look for files already on the destination that have the same size as one that we have (above a certain minimum), and check whether they are in fact identical. Doing so requires an upfront scan of the entire destination tree, and for the client to hold all this information in memory. So for some cases at least it will be undesirable. Probably the right thing is an option like --renames=global/directory/none. I think the general point is that we don't really know exactly what the ideal implementation will be, and preferably the protocol should not restrict too tightly our ability to change or improve it in future. -- Martin -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.tuxedo.org/~esr/faqs/smart-questions.html
