Re: symbolic links
I'm not certain what result you want, but rsync will do your bidding. --copy-links makes it grab the file the link points to, instead of creating the link, --copy-unsafe-links does the same, including for links that point OUTSIDE of the directory tree you were looking at. --safe-links prevents the creation of these broken links you're getting, in that it pretends that links that point outside of your directory tree don't exist... they simply don't appear in the mirror. Without knowing more about your specific setup, I would guess that your broken links appear because they are absolute-path links (start with /), and you're rsyncing into a different absolute path. Example: source dir = /site, destination dir=/pub/site. symlink in /site/html named images points to /site/images. you rsync this over to /pub/site, and it looks for /site/images, which doesn't exist. If the tree needs to be portable, you should use relative links. /site/html would point to ../images, then it doesn't matter where it lives, as long as the rest of the tree comes along with it. I hope one of these scenarios solves your problem. Tim Conway [EMAIL PROTECTED] 303.682.4900 Philips Semiconductor - Colorado TC 1880 Industrial Circle Suite D Longmont, CO 80501 [EMAIL PROTECTED]@[EMAIL PROTECTED] on 12/13/2000 08:33:06 AM Please respond to [EMAIL PROTECTED]@SMTP Sent by:[EMAIL PROTECTED] To: [EMAIL PROTECTED]@SMTP cc: Subject:symbolic links Classification: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 hi all I tried to do a mirror with rsync, but the structure of directory that I want to download has some symbolic links,which they are maintained. The files into these directories aren't downloaded. Is there another way to download these files or directories ? Best regards, Epi -BEGIN PGP SIGNATURE- Version: PGP 6.5i iQA/AwUBOjeHbBrGIiXj/HKVEQLivQCfeaIr+/mjuPiPkW97PFBobqQWJIQAoJ2V NgeXhe+Hz/zol7f3JHhr7yyz =AV9K -END PGP SIGNATURE-
Re: Install rsync
check your /etc/rsyncd.conf. comment out the "pid file" parameter. I assume that will fix it. then, troubleshoot that line, IF you need to specify it. as you're running from inetd, you don't really need a quick pid lookup to kill the rsyncd with for refreshes and such. Tim Conway [EMAIL PROTECTED] 303.682.4900 Philips Semiconductor - Colorado TC 1880 Industrial Circle Suite D Longmont, CO 80501 [EMAIL PROTECTED]@[EMAIL PROTECTED] on 12/17/2000 05:36:48 PM Sent by:[EMAIL PROTECTED] To: [EMAIL PROTECTED]@SMTP cc: Subject:Install rsync Classification: I have a problem whehe I execute "rsync --daemon": messages file says: Dec 16 12:21:22 rsyncd[734]: Unknown Parameter encountered: "pid" Dec 16 12:21:22 rsyncd[734]: IGNORING unknown parameter "pid" Dec 16 12:21:22 rsyncd[734]: rsyncd version 2.4.6 starting Dec 16 12:21:22 rsyncd[734]: bind failed on port 873 Dec 16 12:21:22 rsyncd[734]: transfer interrupted (code 10) at socket.c(233) I have put into /etc/services "rsync 873/tcp" and into /etc/inetd.conf "rsync stream tcp nowait root/usr/bin/rsync rsyncd --daemon -config=/etc/rsyncd.conf" And even when I do "rsync localhost::", error message is "failed to connect to localhost - Connection refused ". Can anyone help me ??? Thank you very much !!!
Re: keeping updating rsync
I wouldn't recommend that, unless you are maintaining a VERY small directory tree, across a fast pipe. It takes a while to determine what, if anything, needs to be sent by rsync. You'll end up with either a continuous reinvocation of rsync, if you wait for one to finish before starting the next, or, if you invoke them asynchronously, you'll end up in the far worse situation of having them cascading, stacking up on each other, independently determining changes to be made, sending redundant changes, slowing down finishing while new ones come along every 5 seconds (or whatever fixed interval). Have you considered writing a content promotion tool? instead of putting things into the tree with cp or mv, you invoke this tool, which does the actual file move(s) and invokes discrete rsyncs for each file. It'd fire off a recursive rsyncs for directories, and non-recursive for files. if you're having changes made in place, you'd have to determine a policy whereby users (or processes) are responsible for kicking off their own rsyncs to propogate changes, or are required to use tools which do so. you might still want to run a full recursive rsync of the directory tree at regular intervals, to preen the filesystem. Tim Conway [EMAIL PROTECTED] 303.682.4900 Philips Semiconductor - Colorado TC 1880 Industrial Circle Suite D Longmont, CO 80501 -- Forwarded by Tim Conway/LMT/SC/PHILIPS on 12/18/2000 06:39 PM --- [EMAIL PROTECTED]@[EMAIL PROTECTED] on 12/17/2000 06:30:20 PM Sent by:[EMAIL PROTECTED] To: [EMAIL PROTECTED]@SMTP cc: [EMAIL PROTECTED]@SMTP Subject:Re: keeping updating rsync Classification: If you run it fairly often, it won't have much data to move. It's not going to be synchronous for all practical purposes, but you could conceivably run in over and over with a 5 second delay, and it'd be pretty close to synchronous. What are you trying to do? -- Sean BerryRentals.com IT (650) 622 2641 work[EMAIL PROTECTED] (650) 281 6610 mobileMy opinions are not necessarily (650) 326 6420 home those of my employer. On Sun, 17 Dec 2000, Paco MartXnez wrote: Is there anyway to keep update files from a host A to host B inmediatly??? How can I configure rsync to update speedly, like -RAID-?? Is it necessary to configure daemon??
Re: losing leading / when copying symbolic links
Excellent. Basic system administration: relative links are (almost) always preferable to absolute links... think nfs. Tim Conway [EMAIL PROTECTED] 303.682.4917 Philips Semiconductor - Colorado TC 1880 Industrial Circle Suite D Longmont, CO 80501 [EMAIL PROTECTED]@[EMAIL PROTECTED] on 02/08/2001 08:50:11 AM Sent by:[EMAIL PROTECTED] To: Tim Conway/LMT/SC/PHILIPS@AMEC cc: [EMAIL PROTECTED]@SMTP [EMAIL PROTECTED]@SMTP Subject:Re: losing leading / when copying symbolic links Classification: On Thu, Feb 08, 2001 at 09:35:17AM -0600, [EMAIL PROTECTED] wrote: It may be that you can fix your problem without modifying rsync or getting root access to do chroot, simply by making your symlinks relative in the source. If /module/dir/link points to /othermodule/otherdir, perhaps you could point it to ../../othermodule/otherdir, for instance. Good suggestion although actually a relative path that moves out of the module (total number '..' pieces more than the depth of the path) is also disallowed. Only relative paths within a module are allowed. The example he originally gave was a complete path to another file in the same directory so he could just remove the whole directory component of the path. - Dave Dykstra
Re: how to ssh in cron
The easiest way is to generate keys with no passphrase. it's still secure, as long as nobody can get your private key. If you insist on adding a password, you'll have to keep it in plaintext in a file and reference it with --password-file, or if you want to add a layer of indirection to make it harder to steal the pass, come ujp with a routine that ends up putting the plain text of the password into the environmental variable$RSYNC_PASSWORD. Here's a quick rot13: to save the pass tr abcdefghijklmnopqrstuvwxyz nopqrstuvwxyzabcdefghijklm encryptedpasswordfile type the password and ^d to get it back out RSYNC_PASSWORD=`cat encryptedpasswordfile |tr nopqrstuvwxyzabcdefghijklm abcdefghijklmnopqrstuvwxyz` Of course, if somebody reads your script, they've got the way to read the password file, if they can get it. you might as well use passphraseless ssh keys (ssh-keygen) Tim Conway [EMAIL PROTECTED] 303.682.4917 Philips Semiconductor - Colorado TC 1880 Industrial Circle Suite D Longmont, CO 80501 [EMAIL PROTECTED]@[EMAIL PROTECTED] on 02/12/2001 09:14:26 AM Sent by:[EMAIL PROTECTED] To: [EMAIL PROTECTED]@SMTP cc: Subject:how to ssh in cron Classification: How do I call rsync with -e ssh in a shell script run from cron. SSH requires a password. How do I pass the password to it from the script? V/r Jay
Re: Help: how to get the module list
when you're not going to an rshd server, the concept of modules does not exist. you're reaching through a remote shell of some sort (rsh, remsh, ssh, whatever), and executing the rsh command on that end. you'll have to modify your routines to use the new type of information. good luck. Tim Conway [EMAIL PROTECTED] 303.682.4917 Philips Semiconductor - Colorado TC 1880 Industrial Circle Suite D Longmont, CO 80501
Re: rsync and packet filters
You're fine. As long as port 22 is open, you're good to go. All traffic will be encapsulated inside the ssh connection, which is from some non-privileged port on the calling machine and port 22 on the server with sshd. Tim Conway [EMAIL PROTECTED] 303.682.4917 Philips Semiconductor - Colorado TC 1880 Industrial Circle Suite D Longmont, CO 80501 [EMAIL PROTECTED]@[EMAIL PROTECTED] on 02/20/2001 01:58:32 PM Sent by:[EMAIL PROTECTED] To: [EMAIL PROTECTED]@SMTP cc: Subject:rsync and packet filters Classification: I'm having a problem with rsync while running on a machine that uses packet filters (FreeBSD)... I presume this is because I need to permit out/in the primary port that rsync uses -- but I'm not sure if it routes all traffic through that port number or if it then uses unprivileged ports after initial connection. What port(s) should be opened? This is being used as: rsync -avz -e ssh some.machine.com:/usr/local/apache/ /local/machine Thanks.
Re: rsync and packet filters
They're in roots path when you're logged in. Remote command execution is subtly different from remote login. I suspect that if you ssh some.machine.com rsync --help, you'll find that rsync isn't found from a non-interactive shell. Commonly, remote command execution acts as a login shell if invoked interactively, but with only the system path, if invoked as a remote command. That means, it doesn't run your .login, .profile, whatever your system uses. When you ssh in, then do "which rsync", you initialized a shell to get to a shell prompt, so your initializations have happenned. try the flag --rsync-path=wherever it is on the other end to specify the actual location. Tim Conway [EMAIL PROTECTED] 303.682.4917 Philips Semiconductor - Colorado TC 1880 Industrial Circle Suite D Longmont, CO 80501 [EMAIL PROTECTED]@[EMAIL PROTECTED] on 02/20/2001 03:11:32 PM Sent by:[EMAIL PROTECTED] To: Tim Conway/LMT/SC/PHILIPS@AMEC cc: [EMAIL PROTECTED]@SMTP Subject:Re: rsync and packet filters Classification: But I'm getting this error: rsync -avz -e ssh some.machine.com:/usr/local/apache/ /local5/machine sh: rsync: not found unexpected EOF in read_timeout Same happens on another machine, which is running packet filters. But the odd thing is that rsync and ssh are all in root's $PATH, as evidenced by 'which'. The client machine "some.machine.com" is not running packet filters. _F At 03:59 PM 2/20/2001 -0600, [EMAIL PROTECTED] wrote: You're fine. As long as port 22 is open, you're good to go. All traffic will be encapsulated inside the ssh connection, which is from some non-privileged port on the calling machine and port 22 on the server with sshd. Tim Conway [EMAIL PROTECTED] 303.682.4917 Philips Semiconductor - Colorado TC 1880 Industrial Circle Suite D Longmont, CO 80501 [EMAIL PROTECTED]@[EMAIL PROTECTED] on 02/20/2001 01:58:32 PM Sent by:[EMAIL PROTECTED] To: [EMAIL PROTECTED]@SMTP cc: Subject:rsync and packet filters Classification: I'm having a problem with rsync while running on a machine that uses packet filters (FreeBSD)... I presume this is because I need to permit out/in the primary port that rsync uses -- but I'm not sure if it routes all traffic through that port number or if it then uses unprivileged ports after initial connection. What port(s) should be opened? This is being used as: rsync -avz -e ssh some.machine.com:/usr/local/apache/ /local/machine Thanks.
Re: rsync, rsh, and remote servers... (Solaris, in case anyone cares ;-)
I wouldn't have been able to figure out what you did. I would have wrapped it in a command line that redirected stdin from /dev/null. I get a lot of things i try to do in the background, that have no need for any stdin, but block on TTIN, and that's how i get them to quit whining and get to work. Tim Conway [EMAIL PROTECTED] 303.682.4917 Philips Semiconductor - Colorado TC 1880 Industrial Circle Suite D Longmont, CO 80501
Re: should rsync also be called ssync?
We're having a debate in which some are advocating changing a long-established and respected utility name, with the sole purpose of placating ignorant, inflexible-minded fools. Did anybody notice this? Tim Conway [EMAIL PROTECTED] 303.682.4917 Philips Semiconductor - Colorado TC 1880 Industrial Circle Suite D Longmont, CO 80501
Re: should rsync also be called ssync?
I spoke too harshly anyway. At the start, it looked like just an alias, and I couldn't figure out why the code needed to be changed, when we could have a script called ssync, containing (after the interpreter line) exec rsync $@ . The only reason I could see to put the change into the code would be to change what's seen in the process table, so some idiot who judges programs by their name won't kill it. I personally would rather see spare brain ticks spent on a speeding the construction of file lists. I wish I had the skills to help. Tim Conway [EMAIL PROTECTED] 303.682.4917 Philips Semiconductor - Colorado TC 1880 Industrial Circle Suite D Longmont, CO 80501 [EMAIL PROTECTED] on 03/01/2001 02:56:02 PM To: Tim Conway/LMT/SC/PHILIPS@AMEC cc: [EMAIL PROTECTED]@SMTP Subject:Re: should rsync also be called ssync? Classification: On Thu, Mar 01, 2001 at 01:54:04PM -0600, [EMAIL PROTECTED] wrote: We're having a debate in which some are advocating changing a long-established and respected utility name, with the sole purpose of placating ignorant, inflexible-minded fools. Did anybody notice this? Not changing. Just adding an alias. - Dave
Re: rsync Logging
Use the --log-format=FORMAT option. It's explained in the rsyncd.conf man page. Speaking of rsyncd.conf, rsync run as daemon logs to a configurable syslog facility. If you need to get plain rsh/ssh-transported rsync to syslog, you'll have to feed the output to logger or a similar utility. Tim Conway [EMAIL PROTECTED] 303.682.4917 Philips Semiconductor - Colorado TC 1880 Industrial Circle Suite D Longmont, CO 80501 [EMAIL PROTECTED]@[EMAIL PROTECTED] on 03/09/2001 09:42:13 AM Sent by:[EMAIL PROTECTED] To: [EMAIL PROTECTED]@SMTP cc: Subject:rsync Logging Classification: Folks I am evaluating rsync v2.4.6 (not rsyncd) for distributing configuration files to multiple mail servers. All seems to be very good except for the logging capabilities. I can see how to get a basic log entry on stdout but how can I get this sent to syslog in the appropriate format? Also, does rsync log errors in the same format, eg if it fails to transfer a file to the remote server? And lastly, when creating a log entry, it displays a literal %h and %a rather than the actual hostname and IP address of the remote server. Any help is much appreciated. Regards Brian
Re: rsync between partitions
You've got two choices. Hack the code to do the updates in-place, or copy the disk images to a filesystem, and treat them as files. Rsync creates a duplicate of the local copy, then unlinks the original and renames the new copy to the original. Obviously, that won't work on device interfaces. Actually, I've got one for you, though. Since there are no insertions, you can use dd on both ends, through rsh (or ssh). use it with skip, seek, conv=notrunc, count=1, and some appropriate block size (the block size of the filesystems, maybe?), and the sum command. get the sum on the master with a count=1, skip=whatever, then the replica. If they're different, send the master to the replica with seek and conv=notrunc. Make sure you do the dd AND the sum all on the remote, so all you pull back are the checksums. You can do this in a shell script, which would be easier than hacking rsync (use an advanced shell (bash,ksh), or perl, as since you need to do this, i reckon the numbers could get big. Tim Conway [EMAIL PROTECTED] 303.682.4917 Philips Semiconductor - Colorado TC 1880 Industrial Circle Suite D Longmont, CO 80501 [EMAIL PROTECTED]@[EMAIL PROTECTED] on 04/13/2001 10:12:08 AM Please respond to [EMAIL PROTECTED]@SMTP Sent by:[EMAIL PROTECTED] To: [EMAIL PROTECTED]@SMTP cc: Subject:Re: rsync between partitions Classification: Is there any way to rsync between raw partitions? You are better off syncing the filesystems on top, if you can. That's not a bad idea, but there may be none .. or at least it may not be consistent, and hence not mountable. I am talking about resyncing two halves of a raid1 mirror. The halves are separated by the net. If it is just raw bits of data on the raw partitions then reading the whole thing (anyway) with dd and updating the whole other is going to be just about as fast. No .. that's the point. The inconsistencies will be very small while the partitions will be very large. The inconsistency will derive from the rupture of the real time mirror and any changes subsequently effected in the half that stayed online. Blocks are zeroed as they are freed in the fs, so there's no problem with rubbish anyway. What I was hoping was that the algorithm brooked easy alteration to examine "two large files" (two block devices) block by block, and send the differing blocks across. That would be ideal. At any rate, it would be a huge improvement over the existing raid1 resync algorithm, which simply sends everything on resync. It's painful. Peter
Re: installing and configuring
That's an easy one. By default, rsync plans to be run through an auxilliary transport. If you want an /etc/rsyncd.conf, you have to put it there yourself. "man rsyncd.conf" will give you what you need to create it. It can be pretty minimal to start, and you can embellish it as you develop your way of using it. I recommend you invoke it from inetd. That way, every time you connect, it rereads the rsyncd.conf, so you can just make edits and test, without having to cycle the daemon every time. Tim Conway [EMAIL PROTECTED] 303.682.4917 Philips Semiconductor - Colorado TC 1880 Industrial Circle Suite D Longmont, CO 80501 [EMAIL PROTECTED]@[EMAIL PROTECTED] on 04/16/2001 05:04:09 PM Sent by:[EMAIL PROTECTED] To: [EMAIL PROTECTED]@SMTP cc: Subject:installing and configuring Classification: hi, I am trying to install rsync on Freebsd machine but after installing i am not able to find rsyncd.conf file in /usr/etc can some one help me out. thanks -chandra [EMAIL PROTECTED]
Re: rsync fails
I do. Either put /usr/local/bin in your path, or symlink rsync into a directory that IS in your path. Tim Conway [EMAIL PROTECTED] 303.682.4917 Philips Semiconductor - Colorado TC 1880 Industrial Circle Suite D Longmont, CO 80501 There are some who call me Tim? [EMAIL PROTECTED]@[EMAIL PROTECTED] on 05/17/2001 05:24:04 PM Sent by:[EMAIL PROTECTED] To: [EMAIL PROTECTED]@SMTP [EMAIL PROTECTED]@SMTP cc: Subject:Re: rsync fails Classification: I just got done dealing with this problem. It was solved for me using the --rsync-path=/usr/local/bin/rsync flag, specifying the remote location of rsync. It be great if someone knew how to avoid using this parameter. cheers, Dave - Original Message - From: Bob Foster [EMAIL PROTECTED] To: [EMAIL PROTECTED] Sent: Thursday, May 17, 2001 4:47 PM Subject: rsync fails Hi, When I rsync to one of my servers, after I enter the password, I get the following: bash: rsync: command not found unexpected EOF in read_timeout I can ssh to and from this server. I can rsync from it. Any ideas? Thanks! Bob Foster
Re: tunneling rsync through ssh
I was taking that same tack... then, I looked again. note: He's on host a, telling
it to ssh to host-b, forwarding port 873 of host-a through the ssh connection to
host-c.
He isn't sshing to the rsync host. He's sshing to a machine that can rsync to host-c.
I thought he was just being stupid, until i realized that. His need is genuine.
He can check to see if the forwarding is working simply by telnetting to port 873. He
should get back something like
telnet atlas 873
Trying 134.27.18.37...
Connected to atlas.
Escape character is '^]'.
@RSYNCD: 24
If he doesn't, try forwarding to remote port 23, 21, 25, 80, whatever... i don't know
what ports work on host-c, but he can figure it out in a few seconds, by sshing into
host-b, and trying the appropriate tests.
Heck, we know rsyncd is supposed to be there, so he can ssh to host-b, telnet host-c
873, and see if he gets the above type of message. If so, troubleshoot the
forwarding, if now, find out what port rsyncd IS on.
Tim Conway
[EMAIL PROTECTED]
303.682.4917
Philips Semiconductor - Colorado TC
1880 Industrial Circle
Suite D
Longmont, CO 80501
There are some who call me Tim?
[EMAIL PROTECTED]@[EMAIL PROTECTED] on 05/31/2001 11:00:37 AM
Sent by:[EMAIL PROTECTED]
To: [EMAIL PROTECTED]@SMTP
cc: [EMAIL PROTECTED]@SMTP
Subject:Re: tunneling rsync through ssh
Classification:
On Thu, May 31, 2001 at 04:17:36PM +0200, thomas graichen wrote:
Michael H. Warfield [EMAIL PROTECTED] wrote:
On Thu, May 31, 2001 at 01:48:56PM +0200, thomas graichen wrote:
did anyone here get rsync tunneled through ssh working so far?
just to make clear what i mean: i only have an ssh connection
to a host b (say mine is host a) and want to rsync stuff from
host c this way (because i can't do it directly due to fire-
wall limitations)
host-a ssh -L873:host-c:873 host-b
host-b
then also on host a
host-a rsync -av localhost::something .
always only results in
@ERROR: protocol startup error
Why not use rsync -av -e ssh {file specifiers} ...
Works for me. I use rsync over ssh all the time that way.
i too - but what if the other end is a public rsync server
without ssh access for you?
Then how would you establish the ssh tunnel to begin with? Seems
like we have a rabit stew situtation (first you catch your rabbit).
If you have the ability to establish an ssh connection to begin with,
you should then have the ability to run rsync from that end using ssh
as the transport. Or am I missing something here?
t
--
thomas graichen [EMAIL PROTECTED] ... perfection is reached, not
when there is no longer anything to add, but when there is no
longer anything to take away. --- antoine de saint-exupery
Mike
--
Michael H. Warfield| (770) 985-6132 | [EMAIL PROTECTED]
(The Mad Wizard) | (678) 463-0932 | http://www.wittsend.com/mhw/
NIC whois: MHW9 | An optimist believes we live in the best of all
PGP Key: 0xDF1DD471| possible worlds. A pessimist is sure of it!
Re: any news about rsync and acl's?
Maybe rsync isn't ready to do it yet, but you could cheat. Have the server side prepare a list of ACLs, from which they can be reconstructed on the reciever. I'm sure there's a more efficient way to do it, maybe with the Solaris-ACL perl package, or coding it in C, but here's a quick and dirty way to do it with shell scripting. createacllist: #!/bin/sh find $* -print |while read object do acl=`getfacl $object | sed 's/#.*//` acl=`echo $acl |tr -d ' '` acl=`echo $acl |tr ' ' ,` echo $object $acl done applyacllist: #!/bin/sh cat $1 |while read file acl do setfacl -s $acl $file done (See attached file: aclrep.tar) Have the server run createacllist from time to time. Have the client grab the acl list, and run applyacllist on it after the sync. Maybe you could save processor time by keeping the old acl list's checksum, and if it hasn't changed, don't run it. Tim Conway [EMAIL PROTECTED] 303.682.4917 Philips Semiconductor - Colorado TC 1880 Industrial Circle Suite D Longmont, CO 80501 There are some who call me Tim? Martin Pool [EMAIL PROTECTED]@lists.samba.org on 07/12/2001 05:52:13 PM Sent by: [EMAIL PROTECTED] To: Obergehrer, Walter [EMAIL PROTECTED] cc: '[EMAIL PROTECTED]' [EMAIL PROTECTED] (bcc: Tim Conway/LMT/SC/PHILIPS) Subject: Re: any news about rsync and acl's? Classification: On 12 Jul 2001, Obergehrer, Walter [EMAIL PROTECTED] wrote: Does anybody know if rsync can be used in a way to preserve the ACL's? If not, are there plans to implement it in the near future? If not, are there other tools which can handle ACL's when synchronizing? I don't know if there are any plans to support them at the moment. They are most definitely on the list for rsync 3.0, but there is no target date for that yet. We would like them to map correctly between heterogenous systems, which is not trivial. http://rsync.samba.org/cgi-bin/rsync.fom?file=48 -- Martin VA Linux Systems aclrep.tar
Re: Rsync the entire server HD through SSH
If you can't let root in, how about running it as a daemon? One end is already acting as root to initiate it, so make the other end a daemon, already waiting for the incoming connection. If you need ssh to get through a firewall or something, set up a redirect as non-root, I don't have ssh handy (i'm deep inside an intranet that I don't control), so i don't remember the syntax for redirects, but you'd redirect port 873 (or another, if you moved it) on the remote system to appear to be port 8730 or something on your local machine. Then, you just rsync -a --port=8730 localhost::whateveryoucallthemodulefor/ whereveryouwanttoputitonthelocalmachine. The security on rsyncd isn't all that great - plaintext passwords - but in this implementation, they're never going over the wire in plaintext. It's in ssh over the wire, and the plaintext is then over the loopback interface to the remote host's port. If possible, make the rsyncd on the read side. It works for writing, but i've heard it's not as solid. Tim Conway [EMAIL PROTECTED] 303.682.4917 Philips Semiconductor - Colorado TC 1880 Industrial Circle Suite D Longmont, CO 80501 There are some who call me Tim?
Re: rsync gives ld.so.1 error
ownership: you're not root, therefore can't create things belonging to other users. If you set -u and -p, it will, if you're root. of course it doesn't hapen in ufsdump. can you restore a ufsdump as non-root? (I don't know, but i'll bet that if you do, files belong to only you) Tar and cpio keep owners too. In any of those archiver cases, you're creating a file (or data on a device) containing user info, not creating the actual files. rsync is like restoring a tar. if you restore as root, ownerships are kept. if not, they're not. Tim Conway [EMAIL PROTECTED] 303.682.4917 Philips Semiconductor - Colorado TC 1880 Industrial Circle Suite D Longmont, CO 80501 There are some who call me Tim? Sudarshan Ramaswamy [EMAIL PROTECTED]@lists.samba.org on 07/20/2001 10:49:41 AM Sent by: [EMAIL PROTECTED] To: [EMAIL PROTECTED] cc: (bcc: Tim Conway/LMT/SC/PHILIPS) Subject: Re: rsync gives ld.so.1 error Classification: Hi Iam working on Solaris Platform. Yes thats what was exactly happening. Hence the only solution I could think of was compiling rsync on 2.5.1 instaed. Now rsync works fine. The only other Problem I am facing is. As rsync is in my home directory It has my account permissions. Owner , group when I rsync a directory which belongs to a different user . It does srchive it , But the owner and group is set to my account. This does not happen in ufsdump. Any help in this regard. I feel there are not sufficient DOCS on rsync. Any URL where there are adequate DOCS. Thanks for your help. -Sudarshan [EMAIL PROTECTED] wrote: On Wed, Jul 18, 2001 at 03:42:11PM -0700, Sudarshan Ramaswamy wrote: Hi I am new to rsync. I have installed rsyn2.4.6 in my home directory. I can rsh to any machine without any passwd as we have NIS . rsync works perfectly on my local machine ie if I rsync within the same machine it works ok. But when I rsync from a differnt machine or to a different machine it gives me the following error my homedir = HOMEDIR machine1:/HOMEDIR/rsync246/bin/rsync /x/*.c machine2:/y gives . building file list ... done ld.so.1: rsync: fatal: relocation error: symbol not found: stat64: referenced in rsync unexpected tag 101 I take it this is Solaris? What this means is that your rsync binary was compiled on Solaris 2.6, 7, or 8 and you're trying to run it on a Solaris 2.5.1 machine (which does have stat64 in libc). Is this what is happening? -- albert chin ([EMAIL PROTECTED])
Re: Newbie to rsync
Look again. it's not finding rsync on machine2. He's doing remote-to-remote, though he's on machine2 already. I've already replied to him in detail, concerning paths and rsync syntax/behaviour. If he fixes the path on machine2 or gives the --rsync-path= directive, then, he'll rsync to a local directory named machine3:, which probably doesn't exist, and certainly isn't what he wants. Tim Conway [EMAIL PROTECTED] 303.682.4917 Philips Semiconductor - Longmont TC 1880 Industrial Circle, Suite D Longmont, CO 80501 Available via SameTime Connect within Philips Available as n9hmg on AIM perl -e 'print pack(, 19061,29556,8289,28271,29800,25970,8304,25970,27680,26721,25451,25970), .\n ' There are some who call me Tim? Pierre Abbat [EMAIL PROTECTED]@lists.samba.org on 09/08/2001 06:22:41 AM Please respond to [EMAIL PROTECTED] Sent by: [EMAIL PROTECTED] To: [EMAIL PROTECTED] cc: (bcc: Tim Conway/LMT/SC/PHILIPS) Subject: Re: Newbie to rsync Classification: On Thursday 06 September 2001 13:53, Sudarshan Ramaswamy wrote: Hi All I have compiled rsync on a Solaris 5.5.1 machine1 . I have compiled this on a partition on the machine as root. I have done the following shared the Partition on the machine i have compiled. mounted this partition this partition on the machine2 where I need to rsync data then When I issue the command on machine2 as rsync -avz machine2:/x machine3: It gives me sh :rsync not found EOF timeout I am surorised at this cos I know that rsync is there on machine2 as a mounted partition. Any clues. Looks like it's not finding rsync on machine3. phma
RE: unexpected EOF in read_timeout
Michelle: I get the same sort of results. Solaris 7, rsync 2.4.6, with the nohang patches, or with 2.4.7pre1. If you trace the execution, you'll probably find i/o errors. I can't find out what they are, and would expect the kernel to take care of any disk or network errors, leaving me wondering what rsync is choking on. We've got all the latest patches to Solaris, so i'm pretty sure it's not a case of rsync overstressing the tcp implementation, especially as i bwlimit the transfer to 1000kbps. I've got 2328543 objects (files, directories, symlinks) in 116473553K, on NAS at each end, reached by NFS. I have to break my jobs up into smaller chunks, which makes it hard to have it delete things. Anybody else have any ideas? Tim Conway [EMAIL PROTECTED] 303.682.4917 Philips Semiconductor - Longmont TC 1880 Industrial Circle, Suite D Longmont, CO 80501 Available via SameTime Connect within Philips Available as n9hmg on AIM perl -e 'print pack(, 19061,29556,8289,28271,29800,25970,8304,25970,27680,26721,25451,25970), .\n ' There are some who call me Tim? Michelle Brownsworth [EMAIL PROTECTED]@lists.samba.org on 09/10/2001 06:34:16 PM Sent by: [EMAIL PROTECTED] To: [EMAIL PROTECTED] cc: (bcc: Tim Conway/LMT/SC/PHILIPS) Subject: RE: unexpected EOF in read_timeout Classification: Sadly, I don't seem to be having any better luck with 2.4.7pre1: su-2.05# ./rsync_nova.sh receiving file list ... done wrote 16 bytes read 6620 bytes 13272.00 bytes/sec total size is 1309032 speedup is 197.26 receiving file list ... done wrote 16 bytes read 4458 bytes 2982.67 bytes/sec total size is 6037628 speedup is 1349.49 receiving file list ... done wrote 16 bytes read 450 bytes 932.00 bytes/sec total size is 34693 speedup is 74.45 receiving file list ... done mysql/kendall/car.MYD Connection to nova closed by remote host. rsync: connection to server unexpectedly closed (26372 bytes read so far) rsync error: received SIGUSR1 or SIGINT (code 20) at rsync.c(229) rsync error: received SIGUSR1 or SIGINT (code 20) at main.c(693) .\\ichelle - Michelle Brownsworth System Administrator PrimeLogic Communications http://www.primelogic.com
Re: Problem with transfering large files.
i meant to send this to the whole list... maybe someone else has seen it and can figure out how to fix it. Tim Conway [EMAIL PROTECTED] 303.682.4917 Philips Semiconductor - Longmont TC 1880 Industrial Circle, Suite D Longmont, CO 80501 Available via SameTime Connect within Philips Available as n9hmg on AIM perl -e 'print pack(, 19061,29556,8289,28271,29800,25970,8304,25970,27680,26721,25451,25970), .\n ' There are some who call me Tim? -- Forwarded by Tim Conway/LMT/SC/PHILIPS on 09/20/2001 08:37 AM --- Tim Conway 09/20/2001 08:36 AM To: Scott Howard [EMAIL PROTECTED] @SMTP cc: Subject: Re: Problem with transfering large files. (Document link: Tim Conway) Classification: Unclassified I know that's what the docs say, but I have found that i have to set the timeout to be at least as long as the time it takes to send the largest file. If it doesn't handle any protocol data, only data data, for the timeout time, it drops. I can't trace the execution well enough to figure out why, or to fix it. I'm pretty sure that's what's going on, because if you leave timeout=0, it sets select_timeout to 60. When i ran with what should be no timeout at all, it choked on any significant file. Tim Conway [EMAIL PROTECTED] 303.682.4917 Philips Semiconductor - Longmont TC 1880 Industrial Circle, Suite D Longmont, CO 80501 Available via SameTime Connect within Philips Available as n9hmg on AIM perl -e 'print pack(, 19061,29556,8289,28271,29800,25970,8304,25970,27680,26721,25451,25970), .\n ' There are some who call me Tim? Scott Howard [EMAIL PROTECTED] on 09/20/2001 07:32:04 AM To: Tim Conway/LMT/SC/PHILIPS@AMEC cc: [EMAIL PROTECTED] [EMAIL PROTECTED] Subject: Re: Problem with transfering large files. Classification: On Mon, Sep 17, 2001 at 12:15:12PM -0600, [EMAIL PROTECTED] wrote: Easy enough... look at your parameters. You're saying that you must not transfer more than 48kb in a session... you actual mileage will be slightly smaller, because of overhead. You have limited yourself to a bandwidth of 200kbps for a duration of 2400 seconds. A single 400Mb file would take a minimum of 2048 seconds at 200kbps, all by itself. Not so. --timeout=TIMEOUT This option allows you to set a maximum IO timeout in seconds. If no data is transferred for the specified time then rsync will exit. The default is 0, which means no timeout. The 2400 second timeout means that the connection must be _idle_ for 2400 seconds. It does not mean that the session can not last for more than 2400 seconds total. Scott.
Re: destination dir doubles in size
It makes perfect sense. /mnt/backup2/systemBackup is a subdirectory of / Every time you run it, you're also backing up /mnt/backup2/systemBackup/ into itself... look. /mnt/backup2/systemBackup/mnt/backup2/systemBackup/ exists, and possibly deeper. add --exclude=/mnt/backup2/systemBackup Tim Conway [EMAIL PROTECTED] 303.682.4917 Philips Semiconductor - Longmont TC 1880 Industrial Circle, Suite D Longmont, CO 80501 Available via SameTime Connect within Philips Available as n9hmg on AIM perl -e 'print pack(, 19061,29556,8289,28271,29800,25970,8304,25970,27680,26721,25451,25970), .\n ' There are some who call me Tim? Ralph Brown [EMAIL PROTECTED]@lists.samba.org on 09/22/2001 12:52:33 PM Sent by: [EMAIL PROTECTED] To: [EMAIL PROTECTED] cc: (bcc: Tim Conway/LMT/SC/PHILIPS) Subject: destination dir doubles in size Classification: I am using Rsync to mirror my primary drive to a mounted second drive. I am using a crontab to control it: 30 3 * * * /usr/bin/rsync -rpogl /* /mnt/backup2/systemBackup/ The first run seems fine. But the subsequent running seems to double the original directory size. I notice this by running df -k. The destination size increases two fold, as if each time Rsync runs, it repeats itself, rather than skipping over unmodified files. Do i need to change a switch? FYI, i am running RH 7.1. suggestions? thanks in advance! Ralph /-\ Ralph Brown [EMAIL PROTECTED] OS X 10.0.4-10K RPM SCSI HD Apache - PHP - MySQL - WML Forget World Peace..Try Using Your Turnsignal \-/
Re: Does RSYNC work over NFS?
yes, but I recommend you add the -W option, unless it's likely that you have a lot data changing within the files and a slow pipe. Tim Conway [EMAIL PROTECTED] 303.682.4917 Philips Semiconductor - Longmont TC 1880 Industrial Circle, Suite D Longmont, CO 80501 Available via SameTime Connect within Philips Available as n9hmg on AIM perl -e 'print pack(, 19061,29556,8289,28271,29800,25970,8304,25970,27680,26721,25451,25970), .\n ' There are some who call me Tim? Karl Kopper [EMAIL PROTECTED]@lists.samba.org on 09/25/2001 02:41:16 PM Sent by: [EMAIL PROTECTED] To: [EMAIL PROTECTED] cc: (bcc: Tim Conway/LMT/SC/PHILIPS) Subject: Does RSYNC work over NFS? Classification: Does rsync work over an NFS mount? --Karl _ Get your FREE download of MSN Explorer at http://explorer.msn.com/intl.asp
RE: Does RSYNC work over NFS?
Actually, there is a possible exception to that. Certainly, in my situation, I always use -W. Ours is mostly new files or deletions... rarely partial modifications of large single files. consider, however, a slow pipe between systems, one or more mounting filesystems via nfs over a fast connection. the lan connection to the nfs is negligible versus the rsync connection from server to server. We're on one side of that threshold here, with a switched 1000bT connection from our servers to our Network Appliances filers and T1 from server-to-server (1044kbps). Of course, we use slower-performing attached storage for our organization-wide-duplicated filesystems, so there's no question in our case... it's -W... especially when we duplicate to our local redundant fileservers. However, if we were using our netapps for our purpose, -W would slow us in some situations. Tim Conway [EMAIL PROTECTED] 303.682.4917 Philips Semiconductor - Longmont TC 1880 Industrial Circle, Suite D Longmont, CO 80501 Available via SameTime Connect within Philips Available as n9hmg on AIM perl -e 'print pack(, 19061,29556,8289,28271,29800,25970,8304,25970,27680,26721,25451,25970), .\n ' There are some who call me Tim? David Bolen [EMAIL PROTECTED]@lists.samba.org on 09/25/2001 05:08:58 PM Sent by: [EMAIL PROTECTED] To: [EMAIL PROTECTED] cc: (bcc: Tim Conway/LMT/SC/PHILIPS) Subject: RE: Does RSYNC work over NFS? Classification: [EMAIL PROTECTED] [[EMAIL PROTECTED]] writes: yes, but I recommend you add the -W option, unless it's likely that you have a lot data changing within the files and a slow pipe. I wouldn't even put any caveat on it. Since rsync is going to have to read the entire file anyway just to compute the block checksums necessary for its algorithm, you're assured that the entire file will flow across the NFS link no matter how much changed. So it really only makes sense to use -W and just copy the whole thing in the first place. To be honest, the only thing rsync offers over a straight copy over NFS is the recursive comparison of timestamp/size to determine files to copy. That might still be worth it, but you're definitely not going to get any benefit in terms of any delta computation. -- David /---\ \ David Bolen\ E-mail: [EMAIL PROTECTED] / | FitLinxx, Inc.\ Phone: (203) 708-5192| / 860 Canal Street, Stamford, CT 06902 \ Fax: (203) 316-5150 \ \---/
Re: rsync errors
Ah... there's your answer. ssh is working fine, as you say, but you're invoking rsync without telling it to user ssh, so it's using rsh. If you've got this, you don't need rsyncd.conf. The syntax you're using tells rsync to use an external transport. rsync -e ssh file rmt_host:/tmp Tim Conway [EMAIL PROTECTED] 303.682.4917 Philips Semiconductor - Longmont TC 1880 Industrial Circle, Suite D Longmont, CO 80501 Available via SameTime Connect within Philips Available as n9hmg on AIM perl -e 'print pack(, 19061,29556,8289,28271,29800,25970,8304,25970,27680,26721,25451,25970), .\n ' There are some who call me Tim? Raj [EMAIL PROTECTED]@lists.samba.org on 10/18/2001 04:15:16 PM Sent by: [EMAIL PROTECTED] To: [EMAIL PROTECTED] [EMAIL PROTECTED] cc: (bcc: Tim Conway/LMT/SC/PHILIPS) Subject: Re: rsync errors Classification: I need to use rsync to copy /thirdparty/dynamo/web on host a to /thirdparty/dynamo/web on host b. Could anyone help me how to setup the rsyncd.conf file for this, do I really need to use rsync.conf. I ran ssh to the remote server, the shell is clean but when I tried the following command it gives error ssh rmt_host date (it displays date on the remote host) rsync file rmt_host:/tmp gives following error: Failed to exec remsh :permission denied unexpected EOF in read_timeout I know remsh is used for unsecured hosts using .rhosts that means some where we need to tell rsync to look for ssh not rsh but I don't know where to look. Please help. Thanks --- Raj [EMAIL PROTECTED] wrote: Hi! List, I installed rsync, configured ssh. I need help in configuring rsyncd.conf. Could anyone email me a sample of rsyncd.conf under /etc. Thanks in advance Raj = __ Do You Yahoo!? Make a great connection at Yahoo! Personals. http://personals.yahoo.com = __ Do You Yahoo!? Make a great connection at Yahoo! Personals. http://personals.yahoo.com
Re: Using rsync to mirror a hard drive.
the $(command) syntax is a ksh-specific form of backticking, which is nestable, and
therefore more flexible than backticks (I always have to do all the intermediate steps
into variables, and use variable substitution instead, for portability). You'll
also see $((arithmetic expression)), and even ((arithmetic expression with variables
expanded though not prefixed with $)) with ksh. Dave Korn seems to have thought of
everything.
Anyway, on RS/6000 systems, (and maybe some other ones), /bin is a symlink to
/usr/bin, which contains sh, which is a symlink to ksh. Under AIX, if you want a
bourne shell, you use bsh.
Now, I don't know why he would want to run it, store the whole output in a variable,
then echo the variable. that hides the output until it's completely finished.
I think what he actually wants is $? (in csh, $status). Maybe something like most
people use:
#!/bin/sh
if rsync -avv / /mirror
then
echo successful
else
echo unsuccessful
fi
Actually, mine is more along the lines of
#!/usr/bin/perl
while(system(rsync -options source destination)){
stuff to do if it failed
}
Since a shell fork with a non-zero exit is true, thus until it's false, it failed.
+WARNING TO THADDEUS+
your syntax will not do what you want.
/mirror is under /
that means the every time you rsync, you put the contents of /mirror into
/mirror/mirror, then /mirror/mirror/mirror.
You double the info every sync (+/- changes)
man rsync
note --exclude=
Tim Conway
[EMAIL PROTECTED]
303.682.4917
Philips Semiconductor - Longmont TC
1880 Industrial Circle, Suite D
Longmont, CO 80501
Available via SameTime Connect within Philips
Available as n9hmg on AIM
perl -e 'print pack(,
19061,29556,8289,28271,29800,25970,8304,25970,27680,26721,25451,25970), .\n '
There are some who call me Tim?
Dave Dykstra [EMAIL PROTECTED]@lists.samba.org on 10/22/2001 10:12:41 AM
Sent by: [EMAIL PROTECTED]
To: Thaddeus L. Olczyk [EMAIL PROTECTED]
cc: [EMAIL PROTECTED] (bcc: Tim Conway/LMT/SC/PHILIPS)
Subject: Re: Using rsync to mirror a hard drive.
Classification:
On Mon, Oct 22, 2001 at 01:18:47AM +, Thaddeus L. Olczyk wrote:
...
5) I need to know if rsync fails. It is useless to me if it fails
and then fails to notify me. The man pages fail to totally describe
return values, but I assume that they do describe success/failure.
So I wrote a script to test this out.
#!/bin/sh
export res=$(rsync -avv / /mirror)
echo $(res)
I don't understand that syntax. /bin/sh for me reports
syntax error: `(' unexpected
Under ksh I would expect that to be equivalent $(...) to be equivalent
to `...` but then what would echo `res` mean?
I then mounted a partition of 15M on /mirror, and executed the script
to get an idea of how rsync behaved when it ran out space.
rsync seems to hang in the middle of /etc .
df shows that only 75% of /mirror is filled/
Any idea why rsync hangs.
Perhaps it's a problem with your script?
6) I'm a bit confused by the notation for rsync excludes. I would like
to exclude certain things and seem to be failing. I would like to
exclude things such as ( matching by grep ) ^/proc ^/mirror
^/tmp. Of course the ^ in front indicates that I do not want to
exclude things like /home/olczyk/mirror.
Rsync excludes are not as powerful as regular expressions. However, if
you start them with a slash, they match only the beginning of a path.
So you should be able to just say /proc, /mirror, /tmp, etc.
- Dave Dykstra
Re: cannot create /padrao/.oi2.sJ5loG : No such file or directory
An analysis of your command line, (information gleaned by R(eading) T(he) F(***ing) M(anual): rsync -uve ssh ./oi2 drcpc1:/padrao/oi2 Using ssh, rsync ./oi2 into the directory /padrao/oi2 on drcpc1, updating only older files, lightly verbose. Your apparent intention is to create a subdirectory named oi2 inside the module named padrao, which is actually /usr/adm/padrao. My guess is that that isn't even what you want to do, but rather to put the CONTENTS of oi2 into the subdirectory oi2 of padrao. as you've already got ssh working, and you are writing to that system, you should forget about the rsyncd.conf entirely. writing to rsyncd doesn't work well, and leaves you open to modification of your files, with no accountability, unless you use a password, which is still eminently sniffable. The commandline you're probably looking for is rsync -uve ssh ./oi2/ drcpc1:/usr/adm/padrao/oi2. That's by the manual. I have actually had unexpected bad luck using the trailing / syntax, and had many subdirectories seem to be missing from the source, and deleted 3/4 of my installation. you might prefer this syntax: rsync -uve ssh ./oi2 drcpc1:/usr/adm/padrao. This syntax has the added advantage that if it's used in a generic synchronization script, it works whether you're sending a file or a directory, with no syntax change. If you insist on writing to the rsyncd, you'll need rsync -uv ./oi2 drcpc1::/padrao. forget the ssh, that's an external transport, like the default rsh. using the :: tells rsync to use it's internal transport, directly to a waiting (or inetd-spawned) rsyncd. Don't worry about the odd name, .oi2.sJ5loG, rsync first creates a file or directory as a hidden file with some random gibberish at the end of its filename, then moves it over. Without knowing more about your specific setup, i can't get any deeper. Please ignore this philips-required confidentiality notice. The information contained in this message is confidential and may be legally privileged. The message is intended solely for the addressee(s). If you are not the intended recipient, you are hereby notified that any use, dissemination, or reproduction is strictly prohibited and may be unlawful. If you are not the intended recipient, please contact the sender by return e-mail and destroy all copies of the original message. Tim Conway [EMAIL PROTECTED] 303.682.4917 Philips Semiconductor - Longmont TC 1880 Industrial Circle, Suite D Longmont, CO 80501 Available via SameTime Connect within Philips Available as n9hmg on AIM perl -e 'print pack(, 19061,29556,8289,28271,29800,25970,8304,25970,27680,26721,25451,25970), .\n ' There are some who call me Tim? Andreia Pio da Silva - funcTo: [EMAIL PROTECTED] [EMAIL PROTECTED] cc: (bcc: Tim Conway/LMT/SC/PHILIPS) amp.br Subject: cannot create /padrao/.oi2.sJ5loG : No such file or directory Sent by: rsync-admin@lists Classification: .samba.org 10/23/2001 09:56 AM Hi, I'm trying to use rsync. my /usr/local/etc/rsyncd.conf is: log file = /var/log/rsyncd pid file = /etc/rsyncd.pid syslog facility = local5 [padrao] comment= Arquivos para Configuracao Maq RedHat path=/usr/adm/padrao
Re: rsync recursion question
That's the way it is. If it's really a one-off change, a huge change in your structure, telnet ssh, rsh, and so forth, work really well for dropping in and deleting stuff (unless you're supplying the master, and other systems out of your control copy from you). Rsync is optimized for taking a filesystem in an unknown state, and making it identical to another filesystem in an unknown state, using network bandwidth as efficiently as possible. Once again, please ignore my stupid confidentiality notice. New notes client, required upgrade, can't delete the danged thing. The information contained in this message is confidential and may be legally privileged. The message is intended solely for the addressee(s). If you are not the intended recipient, you are hereby notified that any use, dissemination, or reproduction is strictly prohibited and may be unlawful. If you are not the intended recipient, please contact the sender by return e-mail and destroy all copies of the original message. Tim Conway [EMAIL PROTECTED] 303.682.4917 Philips Semiconductor - Longmont TC 1880 Industrial Circle, Suite D Longmont, CO 80501 Available via SameTime Connect within Philips Available as n9hmg on AIM perl -e 'print pack(, 19061,29556,8289,28271,29800,25970,8304,25970,27680,26721,25451,25970), .\n ' There are some who call me Tim? Justin Banks justinb@tricord. To: [EMAIL PROTECTED] comcc: (bcc: Tim Conway/LMT/SC/PHILIPS) Sent by:Subject: rsync recursion question rsync-admin@lists .samba.org Classification: 10/23/2001 11:37 AM Please respond to justinb Hello - I have a problem/question about rsync's recursion and directory replication. Here's the problem. Say I have a large filesystem, on the order of a TB. It's all under /mnt/bigfs on host local. In /mnt/bigfs, I have three directories, a, b, and c. I remove directory a. I want to propagate this change to the host remote, and what I'd like to do is have host remote just remove directory a recursively. The only way I can see to do that is to send directory /mnt/bigfs recursively from host sender. This doesn't work very well, though, 'cause it takes a *really* long time to walk the whole filesystem. I know the only thing that's changed is the existence of directory a - how to I propagate that change, and that change only? -justinb -- Justin Banks Tricord, Inc. [EMAIL PROTECTED] The Church says that the earth is flat, but I know that it is round, for I have seen the shadow on the moon, and I have more faith in a shadow than in the Church. --Ferdinand Magellan
Re: Connection Timeout
There you go. rsync -vn --timeout=0 [EMAIL PROTECTED]::webfiles
/local_user/frank/myjsppages, and add the password when it asks, IF it asks. did you
set up the rsyncd.conf with security? If not, it's rsync -vn --timeout=0
server.domain.com::webfiles /local_user/frank/myjsppages. The ssh stuff will only
make it fail, as you're trying to use rsync's internal transport, rather than relying
on an external one.
Now, you know, adding -n will mean it will only say what i WOULD do if not running as
a dry run (-n== --dry-run), so don't be surprised when nothing transfers. Also, as
the webfiles is a module, it's a directory. rsync won't just transfer a directory.
see example:
++
atlas{karrie}% rsync willy::
big
big1
ToolSyncModules
sjt-tools-master1
sjt-tools1
sjt-tools2
Tools
atlas{karrie}% rsync willy::big .
client: nothing to do: perhaps you need to specify some filenames or the --recursive
option?
atlas{karrie}%
++
You'll want to refer to its contents somehow, either just that top level, with /* or
simply /, or by a -r (or -r-implying) option.
rsyncd at leasts responds, (the telnet trick). You're also well on your way to
understanding... most people failing to go to a rsyncd aren't even using the correct
syntax to refer to the remote resource.
I can finally delete these stupid security disclaimers.
Tim Conway
[EMAIL PROTECTED]
303.682.4917
Philips Semiconductor - Longmont TC
1880 Industrial Circle, Suite D
Longmont, CO 80501
Available via SameTime Connect within Philips
Available as n9hmg on AIM
perl -e 'print pack(,
19061,29556,8289,28271,29800,25970,8304,25970,27680,26721,25451,25970), .\n '
There are some who call me Tim?
Jeff Sims
sims@interex To: Tim Conway/LMT/SC/PHILIPS@AMEC
.org cc:
Subject: Re: Connection Timeout
10/23/2001
03:07 PMClassification:
Tim,
The commandline is as shown below:
/opt/rsync/bin/rsync -vn --timeout=0 -e ssh
[EMAIL PROTECTED]::webfiles /local_user/frank/myjsppages
If I telnet to port 873 on the local machine or any machine on my local
network I get the following:
# telnet localhost 873
Trying...
Connected to localhost.interex.org.
Escape character is '^]'.
@RSYNCD: 24
If I telnet to port 873 from an outside network it doesn't give feedback,
just says Trying and doesn't move.
Here is the feedback from the ssh server.domain.com rsync --version
command:
ssh server.domain.com rsync --version
[EMAIL PROTECTED]'s password:
rsync version 2.4.6 protocol version 24
Written by Andrew Tridgell and Paul Mackerras
The file /etc/services has this line present:
rsync873/tcp # Rsync server
The file /etc/inetd.conf has the following line:
rsync stream tcp nowait root/opt/rsync/bin/rsync rsyncd
--daemon
I think this covered all of your inquiries and I hope this info is useful.
Thanks for all your help so far.
Jeff
*** REPLY SEPARATOR ***
On 10/23/01 at 12:35 PM [EMAIL PROTECTED] wrote:
Aaahhh... I see now. connection timeout, not io or select timeout.
What's the commandline?
if it's to a rsyncd, what happens when you telnet server.domain.com 873?
It should look something like:
tconway@atlas
/users/tconwaytelnet localhost 873
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
@RSYNCD: 25
If it's not to rsyncd, what happens when you rsh server.domain.com rsync
--version (or if you're doing ssh, ssh instead of rsh, of course)?
if that fails, is it a permission or network problem?
Please ignore the following silly confidentiality message.
The information contained in this message is confidential and may be
legally privileged. The message is intended solely for the addressee(s).
If you are not the intended recipient, you are hereby notified that any
use, dissemination, or reproduction
Re: hanging issues
There's a program to apply patches, typically called patch Try man patch. If your system doesn't have, and can't get patch, just read the patches and edit the appropriate files by hand. It's not particularly difficult. Incidentally, I want to point out that there's no good reason to cc [EMAIL PROTECTED] on a regular post. I've been seeing a lot of that lately. Also, questions like this aren't even rsync questions. I'm sure there's something like comp.os.unix.introduction, or something similar, on usenet. Tim Conway [EMAIL PROTECTED] 303.682.4917 Philips Semiconductor - Longmont TC 1880 Industrial Circle, Suite D Longmont, CO 80501 Available via SameTime Connect within Philips Available as n9hmg on AIM perl -e 'print pack(, 19061,29556,8289,28271,29800,25970,8304,25970,27680,26721,25451,25970), .\n ' There are some who call me Tim? Raj [EMAIL PROTECTED] To: [EMAIL PROTECTED] Sent by: [EMAIL PROTECTED] rsync-admin@lists cc: (bcc: Tim Conway/LMT/SC/PHILIPS) .samba.org Subject: hanging issues Classification: 10/23/2001 07:52 PM Hi List, I was going through the rsync archives to find any patches to deal with hanging issues, seems to be there are patches but there are no instructions on how to apply these patches. For instance the following email points to the links where we can download patches On Tue, 10 Jul 2001, Steve Ladendorf wrote: Is there something I'm doing wrong? What can I do to prevent rsync from hanging all the time?? If you avoid using -v, that can help rsync not to hang. However, a better fix is to apply this patch: http://www.clari.net/~wayne/rsync-nohang1.patch This fixes a very common hang problem, and makes -v safe to use again. Since you are also transferring large quantities of files, I would be very interested in knowing if rsync still hangs after applying that patch. If it does, try applying this patch: http://www.clari.net/~wayne/rsync-nohang2.patch The second patch fixes a much rarer bug, and it would be great to know if someone had run into it in simple, real-world conditions. ..wayne.. -- How do I apply these patches, could someone help me on that. Thanks in advance Raj = __ Do You Yahoo!? Make a great connection at Yahoo! Personals. http://personals.yahoo.com
Re: rsync recursion question
Now, that's a good solution. if it MUST be done over rsync, and it's not there, write it in yourself. It's amazing, sometimes, the things people want added into a generic tool, and they expect Tridge or Dave to write and maintain it. It's a tool for non-interactively maintaining directory trees, and adding in an option to do a rsh remotehost rm wouldn't be a normal function. Your situation is different. Almost nobody writes to rsyncd, especially not huge trees. Glad to see it's working well for you. Tim Conway [EMAIL PROTECTED] 303.682.4917 Philips Semiconductor - Longmont TC 1880 Industrial Circle, Suite D Longmont, CO 80501 Available via SameTime Connect within Philips Available as n9hmg on AIM perl -e 'print pack(, 19061,29556,8289,28271,29800,25970,8304,25970,27680,26721,25451,25970), .\n ' There are some who call me Tim? Justin Banks justinb@tricord. To: Tim Conway/LMT/SC/PHILIPS@AMEC comcc: [EMAIL PROTECTED] Sent by: [EMAIL PROTECTED] rsync-admin@lists[EMAIL PROTECTED] .samba.org Subject: Re: rsync recursion question Classification: 10/23/2001 08:08 PM Please respond to justinb Tim == tim conway [EMAIL PROTECTED] writes: Tim That's the way it is. If it's really a one-off change, a huge change Tim in your structure, telnet ssh, rsh, and so forth, work really well for Tim dropping in and deleting stuff (unless you're supplying the master, and Tim other systems out of your control copy from you). Rsync is opTimized Tim for taking a filesystem in an unknown state, and making it identical to Tim another filesystem in an unknown state, using network bandwidth as Tim efficiently as possible. Well, that doesn't cut it here. It seemed like all the guts were there - I mean, the functionality already exists on the receiving side, mostly, right? Anyway, I took a look, and I added an option (-d). This means that you can do rsync --delete -d /some/removed/directory foo@wherever::module:/some/removed and the remote side will remove /some/removed/directory. It will work whether it's a file or directory. Let me know if anyone wants a patch, if this is a feature that would help other folks. I can't imagine people are in my particular circumstances, where the source filesystem is on the order of a terabyte, but you never know ;) -justinb -- Justin Banks Tricord, Inc. [EMAIL PROTECTED] 'We have no intention of shipping another bloated operating system and forcing that down the throats of our Windows customers' -- Paul Maritz, Microsoft Group Vice President
Re: Feature req: multiple source trees
He's wanting rsync to interleave the directories, automatically resolving any conflicts, not put multiple directories under the destination. At least, that's what he seems to be asking. Tim Conway [EMAIL PROTECTED] 303.682.4917 Philips Semiconductor - Longmont TC 1880 Industrial Circle, Suite D Longmont, CO 80501 Available via SameTime Connect within Philips, n9hmg on AIM perl -e 'print pack(, 19061,29556,8289,28271,29800,25970,8304,25970,27680,26721,25451,25970), .\n ' There are some who call me Tim? Dave Dykstra [EMAIL PROTECTED] To: Ph. Marek [EMAIL PROTECTED] m cc: [EMAIL PROTECTED] Sent by: (bcc: Tim Conway/LMT/SC/PHILIPS) rsync-admin@lists Subject: Re: Feature req: multiple source trees .samba.org Classification: 10/24/2001 11:09 AM How is that different from the behavior now when multiple source directories are given? The rsync Usage says: Usage: rsync [OPTION]... SRC [SRC]... [USER@]HOST:DEST or rsync [OPTION]... [USER@]HOST:SRC DEST or rsync [OPTION]... SRC [SRC]... DEST or rsync [OPTION]... [USER@]HOST::SRC [DEST] or rsync [OPTION]... SRC [SRC]... [USER@]HOST::DEST or rsync [OPTION]... rsync://[USER@]HOST[:PORT]/SRC [DEST] SRC on single-colon remote HOST will be expanded by remote shell SRC on server remote HOST may contain shell wildcards or multiple sources separated by space as long as they have same top-level Multiple sources can be specified in all cases. - Dave Dykstra On Wed, Oct 24, 2001 at 11:19:01AM +0200, Ph. Marek wrote: Hi everybody, I want to share a feature request I wrote here some time ago. I'd like to give rsync more than 1 source-tree, where the sum (as described below) is rsync'd to the destination. suppose I have some machines, which have almost the same configuration, different only in small details. Then I want to rsync all machines from a master, with the details as specific to every machine. I propose the following mechanism: - the commandline is parsed as now, but instead of a single source name an array of source names is saved. - on reading the source directories the first (master) directory is read (as now). - then the rest of the source directories is read, and every file there which is already in memory is discarded. So only additional files are added to memory. and -basedir now points to the correct source directory. so given this example BASE/ a b c 1/ d e 2/ d e if I run rsync BASE --source 1 machine1:destination the trees BASE and 1 are taken and rsync'd to machine1. Another simple extension to this is to give a list of filenames (or a file with a list of filenames) as parameter, and every file in this list is not substituted but APPENDED. So I could have a base /etc/passwd and another with the user accounts in the different version directories and have them pushed together to the other machine! Here a bitfield saying in which trees the file is present would be good. If no-one else starts working on that I'll do my worst - but it can take some time. Comments, please??? Regards, Phil
Re: Both directions Mirror
point-by-point As operation to desire 1 Mirroring of both directions can be performed. If you mirror both directions, which change takes precedence? example: Host a, modifies /foo/bar. Host b does as well. you mirror both ways, now, which version do we have? 2 File deleted by hostA (hostB) It is deleted also by hostB(hostA). This is unclear. if you mean deleting the file on hostA, then rsyncing to hostB will delete the file on hostB, you're right. if you mean deleting the file on hostB gets it deleted from hostA, how do we know which to do? copy the file from A to B, or delete it from A? (see #1 above) 3 File created by hostA It can update also by hostB. see #1 above, again. There's a product called Unison. I don't use it, and can't help you set it up, but it actually does somehow try to keep this sort of stuff straight. It must keep a database of the contents of both filesystems, in order to resolve conflicts. It actually sounds like your need is for a distributed filesystem, though... maybe coda (no useful knowledge about that one, either. I know dce/dfs, but that's not free, and not widely available)? Tim Conway [EMAIL PROTECTED] 303.682.4917 Philips Semiconductor - Longmont TC 1880 Industrial Circle, Suite D Longmont, CO 80501 Available via SameTime Connect within Philips, n9hmg on AIM perl -e 'print pack(, 19061,29556,8289,28271,29800,25970,8304,25970,27680,26721,25451,25970), .\n ' There are some who call me Tim?
Re: create directories
rsync -avp host::module/ . - Simple commandline - here's what it says: recursively copy the contents of host::module into the current directory, copy symbolic links, copy permissions, copy timestamps, copy group ownership, copy user ownership, copy device files. be verbose, and copy permissions. the copy permissions duplication was intentional, as it's also in the options. If, in fact, you want to create ./module, you need this: rsync -avp host::module . The / at the end of the source says to use the contents of the source, not the source itself - a subtle but important difference. Tim Conway [EMAIL PROTECTED] 303.682.4917 Philips Semiconductor - Longmont TC 1880 Industrial Circle, Suite D Longmont, CO 80501 Available via SameTime Connect within Philips, n9hmg on AIM perl -e 'print pack(, 19061,29556,8289,28271,29800,25970,8304,25970,27680,26721,25451,25970), .\n ' There are some who call me Tim? Jason Helfman jhelfman@bizrate To: [EMAIL PROTECTED] .com cc: (bcc: Tim Conway/LMT/SC/PHILIPS) Sent by:Subject: create directories rsync-admin@lists .samba.org Classification: 10/25/2001 10:28 AM if i do an rsync to the current directory, how come the parent directory isn't created? I normally just get my files spewed into the current directory, eventhough I am trying to transfer a directory. Something like this rsync -avp host::module/ . -- Jason G Helfman Network Administrator BizRate.com 310.754.1264 desk 310.466.2319 cell Fingerprint: DA13 C109 072B CC12 B568 8D84 E9A2 6A7D C479 BCFB GnuPG http://www.gnupg.org Get Private! 1024D/D75E0A36
Re: one log for each module
Well, Ivan, it's not in the code, at least, not yet. How about this for a simple solution. use syslog. Choose an unused facility. make the destination a program. Perl is probably the best tool for this. filehandle named for module called. they can be generated on the fly. the process id is used to direct the wrote somany bytes lines to the right file. You could do it with a growing array of filehandles, or simply opening and closing the filehandles in append mode, as needed. I prefer the second one, as it makes it possible to manage the logs without restarting. Of course, you'll have one log program running continuously, with a line feeder called by syslog. Actually, it would probably be easier to just make the syslog destination a fifo, and read that from the perl script. I've had bad luck with fifos, but maybe perl does a better job handling them than the shell scripts i was using. Good luck. Tim Conway [EMAIL PROTECTED] 303.682.4917 Philips Semiconductor - Longmont TC 1880 Industrial Circle, Suite D Longmont, CO 80501 Available via SameTime Connect within Philips, n9hmg on AIM perl -e 'print pack(, 19061,29556,8289,28271,29800,25970,8304,25970,27680,26721,25451,25970), .\n ' There are some who call me Tim? Ivan Renedo [EMAIL PROTECTED] Sent by: [EMAIL PROTECTED] 10/26/2001 10:08 AM Please respond to ivan To:[EMAIL PROTECTED] cc:(bcc: Tim Conway/LMT/SC/PHILIPS) Subject:one log for each module Classification: There are some way to have a log for each module? If there aren't, it should be a a great functionality for maintaining the server. Thanks for the help and continue with this great application !
Re: rsync not copying owner/group on some files
I just realized that this was a local copy, and the other symptom, numeric uids showing on the ls, for users which exist, and some which show none at all, indicate a problem with your operating system. The only time i've ever seen blank uids was with a corrupt filesystem. My particular case was a failing drive. I suppose you could fake up blank usernames by creating a user named asdf^h^h^h^h , but i'm guessing that's not the case here. Tim Conway [EMAIL PROTECTED] 303.682.4917 Philips Semiconductor - Longmont TC 1880 Industrial Circle, Suite D Longmont, CO 80501 Available via SameTime Connect within Philips, n9hmg on AIM perl -e 'print pack(, 19061,29556,8289,28271,29800,25970,8304,25970,27680,26721,25451,25970), .\n ' There are some who call me Tim? Ian Kettleborough [EMAIL PROTECTED] 10/29/2001 06:15 PM To:Tim Conway/LMT/SC/PHILIPS@AMEC cc: Subject:Re: rsync not copying owner/group on some files Classification: Nope.. happens consistentlt. I even removed a directory at the destination and reran rsync. All the files were move, but the numeric id shows instead if the alpabetic group...no owner either. Ian [EMAIL PROTECTED] This is a multipart message in MIME format. --=_alternative 007AF6F687256AF4_= Content-Type: text/plain; charset=us-ascii I suspect that you're copying over a nonexistent user... the name exists on the source, but not the destination. Would preserving the numeric ids help? --numeric-ids don't map uid/gid values by user/group name Tim Conway [EMAIL PROTECTED] 303.682.4917 Philips Semiconductor - Longmont TC 1880 Industrial Circle, Suite D Longmont, CO 80501 Available via SameTime Connect within Philips, n9hmg on AIM perl -e 'print pack(, 19061,29556,8289,28271,29800,25970,8304,25970,27680,26721,25451,25970), .\n ' There are some who call me Tim? Ian Kettleborough [EMAIL PROTECTED] Sent by: [EMAIL PROTECTED] 10/29/2001 03:02 PM To: [EMAIL PROTECTED] cc: (bcc: Tim Conway/LMT/SC/PHILIPS) Subject:rsync not copying owner/group on some files Classification: I am trying to use rsync 2.4.6 to try creating a backup using another locally connected hard disk. Thing copy, but sometimes the group is eith numeric (valid). Often the owner is not even shown. This is the command I use for rsync: rsync-2.4.6/rsync -arxlHpogDt /usr/ /backup/usr/ --delete I login as root. Any help would be appreciated. -- Ian Kettleborough [EMAIL PROTECTED] --=_alternative 007AF6F687256AF4_= Content-Type: text/html; charset=us-ascii brfont size=2 face=sans-serifI suspect that you're copying over a nonexistent user... the name exists on the source, but not the destination. nbsp;Would preserving the numeric ids help?/font brfont size=2 face=sans-serifnbsp; nbsp; nbsp;--numeric-ids nbsp; nbsp; nbsp; nbsp; nbsp; don't map uid/gid values by user/group name/font brfont size=2 face=sans-serifbr br Tim Conwaybr [EMAIL PROTECTED]br 303.682.4917br Philips Semiconductor - Longmont TCbr 1880 Industrial Circle, Suite Dbr Longmont, CO 80501br Available via SameTime Connect within Philips, n9hmg on AIMbr perl -e 'print pack(, 19061,29556,8289,28271,29800,25970,8304,25970,27680,26721,25451,25970), quot;.\nquot; 'br quot;There are some who call me Tim?quot;/font br br br table width=100% tr valign=top td tdfont size=1 face=sans-serifbIan Kettleborough lt;[EMAIL PROTECTED]gt;/b/font brfont size=1 face=sans-serifSent by: [EMAIL PROTECTED]/font pfont size=1 face=sans-serif10/29/2001 03:02 PM/font br tdfont size=1 face=Arialnbsp; nbsp; nbsp; nbsp; /font brfont size=1 face=sans-serifnbsp; nbsp; nbsp; nbsp; To: nbsp; nbsp; nbsp; nbsp;[EMAIL PROTECTED]/font brfont size=1 face=sans-serifnbsp; nbsp; nbsp; nbsp; cc: nbsp; nbsp; nbsp; nbsp;(bcc: Tim Conway/LMT/SC/PHILIPS)/font brfont size=1 face=sans-serifnbsp; nbsp; nbsp; nbsp; Subject: nbsp; nbsp; nbsp; nbsp;rsync not copying owner/group on some files/font pfont size=1 face=sans-serifnbsp; nbsp; nbsp; nbsp; Classification: nbsp; nbsp; nbsp; nbsp;/font br/table br br brfont size=2 face=Courier NewI am trying to use rsync 2.4.6 to try creating a backup using anotherbr locally connected hard disk. Thing copy, but sometimes the group is eithbr numeric (valid). Often the owner is not even shown.br br This is the command I use for rsync:br br br rsync-2.4.6/rsync -arxlHpogDt /usr/ /backup/usr/ nbsp;--delete nbsp;br br I login as root.br br br Any help would be appreciated.br br br -- br br Ian Kettleboroughbr [EMAIL PROTECTED]br br br /font br br --=_alternative 007AF6F687256AF4_=-- -- Ian Kettleborough [EMAIL PROTECTED]
Re: Problem using -v within rsync transfer.
Dave: I suspect what you need will be provided by adding --stats to your commandline. That'll give you something like this at the end of the transfer: Number of files: 1 Number of files transferred: 1 Total file size: 43 bytes Total transferred file size: 43 bytes Literal data: 43 bytes Matched data: 0 bytes File list size: 30 Total bytes written: 125 Total bytes read: 36 wrote 125 bytes read 36 bytes 322.00 bytes/sec total size is 43 speedup is 0.27 It still doesn't solve the problem with verbosity, and the process STILL may hang, but less. I've had to break my transfers down into units of about 6 files each. Ugly, but it works. Unfortunately, it means --delete doesn't often help. Tim Conway [EMAIL PROTECTED] 303.682.4917 Philips Semiconductor - Longmont TC 1880 Industrial Circle, Suite D Longmont, CO 80501 Available via SameTime Connect within Philips, n9hmg on AIM perl -e 'print pack(, 19061,29556,8289,28271,29800,25970,8304,25970,27680,26721,25451,25970), .\n ' There are some who call me Tim? CRUISE,DAVID (A-FtCollins,ex1) [EMAIL PROTECTED] Sent by: [EMAIL PROTECTED] 10/30/2001 04:40 PM To:'[EMAIL PROTECTED]' [EMAIL PROTECTED] cc:(bcc: Tim Conway/LMT/SC/PHILIPS) Subject:Problem using -v within rsync transfer. Classification: Hello, We have used rsync on a mirroring application successfully for just over a year. After our upgrade to 2.4.6, however, the process started failing about 50% of the time. The nature of our failure was that the rsync processes on the remote system would die with no error message, leaving the local system processes hung. I have netstat and trace output for available if needed, but I see that others have sent this information in. Other details of the transfer include: our platforms are HPUX 10.20 (each with 1.5GB Ram), the total data size is around 20GB, a typical transfer is around 300MB (around 40,000 files). After reading several other postings, we tried removing the -v from our rsync command and our success rate improved dramatically (12 successes in a row now). What other platforms have this issue with -v? Is there a known fix, or is a fix in progress? I ask because part of our process has always included a transfer size and performance report. Without some verbosity, -v, this information is not obviously available. Thank you for any workarounds or updates, Dave Dave Cruise |4380 Ziegler Rd, MS 72 Agilent Technologies |Ft. Collins, CO 80525 IT Engineer |(970) 288-2895 GIS Americas |[EMAIL PROTECTED]
Re: Excludes in /etc/rsyncd.conf
Actually, though the file IS read top-to-bottom, that is irrelevant for the exclude (and for most (all?)) directives. there can be only one for the module. I don't know the behaviour when there are multiple, whether subsequent ones are ignored, the last one read wins, or it throws an error. the patterns all go on the single line, and are read in order. If you want to get fancy, use exclude from, and put all your patterns in a seperate file. it'll be easier than maintaining a huge space-delimited list. Tim Conway [EMAIL PROTECTED] 303.682.4917 Philips Semiconductor - Longmont TC 1880 Industrial Circle, Suite D Longmont, CO 80501 Available via SameTime Connect within Philips, n9hmg on AIM perl -e 'print pack(, 19061,29556,8289,28271,29800,25970,8304,25970,27680,26721,25451,25970), .\n ' There are some who call me Tim? Scott Russell [EMAIL PROTECTED] Sent by: [EMAIL PROTECTED] 11/01/2001 06:18 AM To: [EMAIL PROTECTED] cc: (bcc: Tim Conway/LMT/SC/PHILIPS) Subject:Re: Excludes in /etc/rsyncd.conf Classification: On Wed, Oct 31, 2001 at 01:38:15PM -0800, Jason Helfman wrote: Thought that you are able to add a simple - + with paths or file names at the bottom of a file and run an rsync and those are then recognized inclusions or exclusions from the synchronization You can do this but keep in mind that the file is read top to bottom. The first matching + / - rule is the one operated upon and the others below it are ignored. -- Regards, Scott Russell ([EMAIL PROTECTED]) Linux Technology Center, System Admin, RHCE. T/L 441-9289 / External 919-543-9289 http://bzimage.raleigh.ibm.com/webcam
unpredictable behaviour
(status) +++Memory usage+ load averages: 0.25, 0.33, 0.35 08:46:15 102 processes: 100 sleeping, 1 zombie, 1 on cpu CPU states: 94.1% idle, 0.5% user, 5.3% kernel, 0.0% iowait, 0.0% swap Memory: 3072M real, 1709M free, 644M swap in use, 5501M swap free PID USERNAME THR PRI NICE SIZE RES STATE TIMECPU COMMAND 10838 Tools 1 330 535M 176M sleep 55:19 0.00% rsync 19393 Tools 1 330 535M 1728K sleep 5:04 0.59% rsync 10837 Tools 1 330 285M 78M sleep 26:46 0.28% rsync ++ Tim Conway [EMAIL PROTECTED] 303.682.4917 Philips Semiconductor - Longmont TC 1880 Industrial Circle, Suite D Longmont, CO 80501 Available via SameTime Connect within Philips, n9hmg on AIM perl -e 'print pack(, 19061,29556,8289,28271,29800,25970,8304,25970,27680,26721,25451,25970), .\n ' There are some who call me Tim?
timeout and process cleanup bugs 2.4.7pre1+
I have written an application in perl, to run multiple rsyncs, retrying those which fail. I have started getting this error on very large modules since i moved to the latest CVS version. ++ ToolSyncProgress Succeeded Getting big/tools/vlsi_local/etc from willy at 1004767211:2001:11:3:6:0:11. ToolSyncProgress Getting big1/cadappl1/hpux from willy at 1004767211:2001:11:3:6:0:11. rsync: connection unexpectedly closed (50466734 bytes read so far) rsync error: error in rsync protocol data stream (code 12) at io.c(139) ++ The script checks exit status and retries, ordinarily. However, when i get THIS error, the rsync process seems to be killing the perl process. I think it's a bug in the part of the code that cleans up the spawned processes. Here's another example, which shows that the timeout as total transfer time but is still active. It also ends by killing its calling process. The error looks different, though, but it's on a local nfs-to-nfs, instead of rsyncd-to-nfs. ++ ToolSyncProgress Replicating big1/cadappl1/hpux from chi-tools1 to chi-tools2 at 1004913115:2001:11:4:22:31:55. io timeout after 3600 seconds - exiting rsync error: timeout in data send/receive (code 30) at io.c(75) ToolSyncProgress chinetsvr failed Replicating big1/cadappl1/hpux from /wan/chi-tools1/chi-tools1 to /wan/chi-tools2/chi-tools2 at 1004925444:2001:11:5:1:57:24 with 7680, retrying. ToolSyncProgress Replicating big1/cadappl1/hpux from chi-tools1 to chi-tools2 at 1004925444:2001:11:5:1:57:24. io timeout after 3600 seconds - exiting rsync error: timeout in data send/receive (code 30) at io.c(75) ToolSyncProgress chinetsvr failed Replicating big1/cadappl1/hpux from /wan/chi-tools1/chi-tools1 to /wan/chi-tools2/chi-tools2 at 1004937647:2001:11:5:5:20:47 with 7680, retrying. ToolSyncProgress Replicating big1/cadappl1/hpux from chi-tools1 to chi-tools2 at 1004937647:2001:11:5:5:20:47. io timeout after 3600 seconds - exiting rsync error: timeout in data send/receive (code 30) at io.c(75) ToolSyncProgress ToolSyncProgress Replicating io timeout after 3600 seconds - exiting rsync error: timeout in data send/receive (code 30) at io.c(75) ToolSyncProgress ToolSyncProgress Replicating rsync error: received SIGUSR1 or SIGINT (code 20) at main.c(741) ++ Any ideas? I can try to track the killing and timeout logic, if there's not already a fix out there that was just accidentally dropped. Tim Conway [EMAIL PROTECTED] 303.682.4917 Philips Semiconductor - Longmont TC 1880 Industrial Circle, Suite D Longmont, CO 80501 Available via SameTime Connect within Philips, n9hmg on AIM perl -e 'print pack(, 19061,29556,8289,28271,29800,25970,8304,25970,27680,26721,25451,25970), .\n ' There are some who call me Tim?
Re: Ptoblem for update 2.4.1 - 2.4.6
The bind mentioned in the second line refers not to Berkely Internet Name Domain, but to socket binding. I suspect that your old daemon is still running. ps ax |grep rsync will show it if that's it. Alternately, if you were running the daemon from inetd, you'll just need to make rsync available at the location pointed to by /etc/inetd.conf... inetd will be holding port 873, so you won't be able to start --daemon without reconfiguring inetd. One other possibility. port 873 1025, so if you're not root, you can't bind to that port. List: did i miss anything? Tim Conway [EMAIL PROTECTED] 303.682.4917 Philips Semiconductor - Longmont TC 1880 Industrial Circle, Suite D Longmont, CO 80501 Available via SameTime Connect within Philips, n9hmg on AIM perl -e 'print pack(, 19061,29556,8289,28271,29800,25970,8304,25970,27680,26721,25451,25970), .\n ' There are some who call me Tim? Eugene Frolenkov [EMAIL PROTECTED] Sent by: [EMAIL PROTECTED] 11/13/2001 02:41 AM Please respond to abyhan To: [EMAIL PROTECTED] cc: (bcc: Tim Conway/LMT/SC/PHILIPS) Subject:Ptoblem for update 2.4.1 - 2.4.6 Classification: After installing nevest version of sync 2.4.6 from 2.4.1, sync --daemon wrote to logfile: 2001/11/13 12:27:46 [12261] rsyncd version 2.4.6 starting 2001/11/13 12:27:46 [12261] bind failed on port 873 2001/11/13 12:27:46 [12261] transfer interrupted (code 10) at socket.c(233) sync 2.4.1 worked normally... My question: Where i can find WATHSNEW or CHANGES documentations any versions of rsync? Required rsync-2.4.6 newest version of bind? Sys: RedHat Knl: 2.2.14 bind: 8.2.3 -- W.B.R., Evgeny Frolenkov
Re: times difference causes write
In the example you give, yes, a time difference causes a write. You are using the -W (--whole-file) option, which directs rsync to simply send the file, in its entirety, if there is a discrepency in mtime (ctime too?), or extent. It is used for situations where file access is slow enough that trying to do an incremental update would take more time/resources than simply sending the file... primary example being nfs-mounted filesystems. If you have fast dasd/slow network, you should probably just drop the -W. Tim Conway [EMAIL PROTECTED] 303.682.4917 Philips Semiconductor - Longmont TC 1880 Industrial Circle, Suite D Longmont, CO 80501 Available via SameTime Connect within Philips, n9hmg on AIM perl -e 'print pack(, 19061,29556,8289,28271,29800,25970,8304,25970,27680,26721,25451,25970), .\n ' There are some who call me Tim? Don Mahurin [EMAIL PROTECTED] Sent by: [EMAIL PROTECTED] 11/13/2001 01:52 PM To: [EMAIL PROTECTED] cc: (bcc: Tim Conway/LMT/SC/PHILIPS) Subject:times difference causes write Classification: Using rsync-2.4.6: Is a times difference supposed to cause a write? Also -t vs -I makes no difference. Below shows the problem, I think: [dmahurin@pc16 /tmp]$ mkdir x y [dmahurin@pc16 /tmp]$ cp /bin/ls x [dmahurin@pc16 /tmp]$ ls -l x/ls -rwxr-xr-x1 dmahurin users 43024 Nov 13 12:46 x/ls [dmahurin@pc16 /tmp]$ rsync -vrtW x/ y building file list ... done ./ ls ./ wrote 43112 bytes read 36 bytes 86296.00 bytes/sec total size is 43024 speedup is 1.00 [dmahurin@pc16 /tmp]$ touch x/ls [dmahurin@pc16 /tmp]$ rsync -vrtW x/ y building file list ... done ls ./ wrote 43116 bytes read 36 bytes 86304.00 bytes/sec total size is 43024 speedup is 1.00 [dmahurin@pc16 /tmp]$ touch x/ls [dmahurin@pc16 /tmp]$ rsync -vrIW x/ y building file list ... done ls wrote 43116 bytes read 36 bytes 86304.00 bytes/sec total size is 43024 speedup is 1.00
Re: times difference causes write
-W and -C are actually kind of opposite effects... -W means never checksum... if it's different in length or time, send it. -c means ALWAYS checksum, even if time and length are identical. use neither. the default behaviour is to compare date/length, and if they are different, do the rolling checksum. This is done in chunks, then compares the sums, and sends the different blocks, with instructions on where to insert them. If the difference was simply timestamp, and in fact, the data is unchanged, the rolling checksum will sort of say never mind, and leave the file alone. Unfortunately for your situation, it builds tne new file next to the existing file, no matter what you do, then deletes the existing file and renames the new file to the old name. Every file done by rsync involves a full write. You're going to put a lot of cycles on your flash. I'm thinking rsync might not be the best solution for what you're doing. Flash is not suited to frequently-updated data. Tim Conway [EMAIL PROTECTED] 303.682.4917 Philips Semiconductor - Longmont TC 1880 Industrial Circle, Suite D Longmont, CO 80501 Available via SameTime Connect within Philips, n9hmg on AIM perl -e 'print pack(, 19061,29556,8289,28271,29800,25970,8304,25970,27680,26721,25451,25970), .\n ' There are some who call me Tim? Don Mahurin [EMAIL PROTECTED] Sent by: [EMAIL PROTECTED] 11/13/2001 02:36 PM To: Tim Conway/LMT/SC/PHILIPS@AMEC cc: [EMAIL PROTECTED] Subject:Re: times difference causes write Classification: My first problem is that I am writing to compact flash, so I want the minimal number of writes. My second problem is that the flash is of limited size, so I need some sort of patch rsync that does not keep the old file before writing the new one. My patch now just unlinks the file ahead, and implies -W. So my wish was that a time discrepancy would lead to a checksum, where the files would match. This is not the case, however, as you say. So for now, I must use -c. It's slow, but I know that I get the minimum number of writes. -don [EMAIL PROTECTED] wrote: In the example you give, yes, a time difference causes a write. You are using the -W (--whole-file) option, which directs rsync to simply send the file, in its entirety, if there is a discrepency in mtime (ctime too?), or extent. It is used for situations where file access is slow enough that trying to do an incremental update would take more time/resources than simply sending the file... primary example being nfs-mounted filesystems. If you have fast dasd/slow network, you should probably just drop the -W.
Re: using rsync to backup windows workstations
Im guessing it's in /etc/sshd_config. ssh issue, not rsync. try ssh linuxbox uname -a from a windoze machine. if that doesn't work without a password, neither will rsync. From man sshd: +++ PasswordAuthentication Specifies whether password authentication is allowed. The de- fault is ``yes''. Note that this option applies to both protocol versions 1 and 2. PermitEmptyPasswords When password authentication is allowed, it specifies whether the server allows login to accounts with empty password strings. The default is ``no''. +++ If PasswordAuthentication is yes, then you'll need to ensure that PermitEmptyPasswords is also yes, or you won't be allowed passphraseless authentication. If you're not permitted to change that, just use a simple ssh passphrase. From man rsync: +++ Some paths on the remote server may require authentication. If so then you will receive a password prompt when you con- nect. You can avoid the password prompt by setting the environment variable RSYNC_PASSWORD to the password you want to use or using the --password-file option. This may be use- ful when scripting rsync. +++ Wonderful thing, those man pages. Good luck. Tim Conway [EMAIL PROTECTED] 303.682.4917 Philips Semiconductor - Longmont TC 1880 Industrial Circle, Suite D Longmont, CO 80501 Available via SameTime Connect within Philips, n9hmg on AIM perl -e 'print pack(, 19061,29556,8289,28271,29800,25970,8304,25970,27680,26721,25451,25970), .\n ' There are some who call me Tim? Thomas Lambert [EMAIL PROTECTED] Sent by: [EMAIL PROTECTED] 11/14/2001 08:15 AM To: [EMAIL PROTECTED] cc: (bcc: Tim Conway/LMT/SC/PHILIPS) Subject:Re: using rsync to backup windows workstations Classification: I have about 12 NT servers that I want to backup 1 directory to my Linux Server. The problem I am having is trying to get it not to ask for the user's password. I am trying to use ssh. I just installed the latest cygwin on my NT server (1.3.4). Rsync is 2.4.6. What I have tried was on the NT machine, I ran ssh-keygen (no passphrase) and then added the identity.pub to the Linux box in the /home/thomas/.ssh/authorized_keys. When I run: rsync -uv -e ssh *.* thomas@linuxbox:backup/ it asks for a password. Besides asking for the password, it works fine. Only the files changed are updated and it is FAST! I saw the ssh-agent. Do I need to use it? and how? Is there something I need to do on my Linux server? I am pretty new to Linux, and have a long way to go to be an expert, but I can get around. Thanks for any help. Thomas Lambert VP DSC Associates, Inc. [EMAIL PROTECTED] - Original Message - From: Lapo Luchini [EMAIL PROTECTED] To: [EMAIL PROTECTED] Sent: Tuesday, November 13, 2001 5:23 PM Subject: Re: using rsync to backup windows workstations As officially I'm the mantainer of cygwin's rsync package I feel I must add something.. 0=) a) rsync binaries for win32/cygwin are now available from the binaries section of the rsync web pages, maintained by someone at redhat I think Maintained by me, I don't work for redhat, and sadly I have a bit too little time to maintain it as I should... b) the version attached is so old that you really shouldn't attempt to use it. Get the latest CVS version and add Wayne Davison's patches (see mailing list archives), in particular for use with Windoze systems, then compile it yourself with cygwin ...but not so little not to compile it with Wayne's patches (actual patch is the latest anti-hang available at the date of the release of cygwin's rsync-2.4.6-2 package) ;-) If there's some patch that I missed (I read this mailing list since when I began to mantain that package, but reading only headers of most messages maybe I missed something...) please point it to me and I'll release a new verison shortly =) Please note that I'm not an expert programmer of rsync, just a person that likes and uses it (and wants maybe to become such an expert, btw): in cygwin one package can't exist in the distribution if there's not a mantainer... so I hope that the effort I put in it is not wasted, even if it's maybe not enough... [I've got a bit of guild for that eheh] c) it's easy to compile under cygwin, and cygwin is easy to install, so I strongly recommend doing that That's true, it compiles out-of-the box. The main problem is that in winsocks if the socket is closed a RST is sent, not flushing the cache. This creates many errors at the end of trasnfer, especially in the daemon mode (which is not addressed by the patch I used, I bet). d
Re: Secure connection to testor refused !
You'll have to chose between the external transport via ssh and the rsync daemon. The commandline to use the rsyncd, based on your example, would be rsync -v guest@testor::c:/Temp/pix.log . I'm guessing, however, that you don't have a module named c: in your rsyncd.conf. I would hope that if you're trying to use the rsyncd, you have something like +++ [cdrive] path = c:\ +++ in your rsyncd.conf. that would mean your commandline would be rsync -v guest@testor::cdrive/Temp/pix.log . the internal transport (rsyncd connection) is signified by the double colon :: delimiter between the host and the path, which in this case is the name of the module, followed by the path to the subdirectory. the single colon means to open a tcp connection to the remote machine, execute rsync --server --whateverotherundocumentedoptionsitusesinthiscircumstance, and use that as the remote. the -e ssh means to use ssh instead of the default rsh. Tim Conway [EMAIL PROTECTED] 303.682.4917 Philips Semiconductor - Longmont TC 1880 Industrial Circle, Suite D Longmont, CO 80501 Available via SameTime Connect within Philips, n9hmg on AIM perl -e 'print pack(, 19061,29556,8289,28271,29800,25970,8304,25970,27680,26721,25451,25970), .\n ' There are some who call me Tim? Kapoor, Nishikant X [EMAIL PROTECTED] Sent by: [EMAIL PROTECTED] 11/15/2001 03:09 PM To:[EMAIL PROTECTED] cc:(bcc: Tim Conway/LMT/SC/PHILIPS) Subject:Secure connection to testor refused ! Classification: Here is a newbie question. Hope, you will not mind. I am trying to backup a file from my NT machine onto AIX machine. Here is the setup: NT machine (testor): rsync is in c:\Rsync\rsync.exe and is running as a daemon. (I created a service for Rsync first). AIX machine (admx): rsync -v rsync version 2.3.1 Copyright Andrew Tridgell and Paul Mackerras admx::nkapoor telnet testor 873 Trying... Connected to testor. Escape character is '^]'. @RSYNCD: 24 So, the rsync daemon seems fine on testor but the problem occurs when I try this: admx::nkapoor rsync -v -e ssh --rsync-path=c:/rsync/rsync.exe guest@testor:c:/Temp/pix.log . Secure connection to testor refused. unexpected EOF in read_timeout I have checked that 'guest' is a valid user on testor. Could someone please help me understand what could be the problem here ? Thanks Nishi
RE: Secure connection to testor refused !
There are ways... use the log file = directive in your rsyncd.conf, which will give you a small bit of info. If you can find a debug utility for cygwin, you can run the daemon directly, rather than from inetd, and trace it that way. From what you're showing, though, it looks more like a path error. Can we see a copy of your rsyncd.conf? Tim Conway [EMAIL PROTECTED] 303.682.4917 Philips Semiconductor - Longmont TC 1880 Industrial Circle, Suite D Longmont, CO 80501 Available via SameTime Connect within Philips, n9hmg on AIM perl -e 'print pack(, 19061,29556,8289,28271,29800,25970,8304,25970,27680,26721,25451,25970), .\n ' There are some who call me Tim? Kapoor, Nishikant X [EMAIL PROTECTED] 11/16/2001 07:06 AM To:Tim Conway/LMT/SC/PHILIPS@AMEC cc:[EMAIL PROTECTED] Subject:RE: Secure connection to testor refused ! Classification: Thanks for the help. I did go through quite a few docs but could not figure that out. Thanks again. However, after setting [cdrive], I am getting the following error: admx::nkapoor rsync -v guest@testor::cdrive/Temp/pix.log . @ERROR: chdir failed read error: Connection reset by peer Is there a log file that I could use to see what exaclty is going on ? Or a debug level that could be set to see the trail ? Thanks Nishi -Original Message- From: [EMAIL PROTECTED] [SMTP:[EMAIL PROTECTED]] Sent: Thursday, November 15, 2001 4:51 PM To: Kapoor, Nishikant X Cc: [EMAIL PROTECTED] Subject: Re: Secure connection to testor refused ! You'll have to chose between the external transport via ssh and the rsync daemon. The commandline to use the rsyncd, based on your example, would be rsync -v guest@testor::c:/Temp/pix.log . I'm guessing, however, that you don't have a module named c: in your rsyncd.conf. I would hope that if you're trying to use the rsyncd, you have something like +++ [cdrive] path = c:\ +++ in your rsyncd.conf. that would mean your commandline would be rsync -v guest@testor::cdrive/Temp/pix.log . the internal transport (rsyncd connection) is signified by the double colon :: delimiter between the host and the path, which in this case is the name of the module, followed by the path to the subdirectory. the single colon means to open a tcp connection to the remote machine, execute rsync --server --whateverotherundocumentedoptionsitusesinthiscircumstance, and use that as the remote. the -e ssh means to use ssh instead of the default rsh. Tim Conway [EMAIL PROTECTED] 303.682.4917 Philips Semiconductor - Longmont TC 1880 Industrial Circle, Suite D Longmont, CO 80501 Available via SameTime Connect within Philips, n9hmg on AIM perl -e 'print pack(, 19061,29556,8289,28271,29800,25970,8304,25970,27680,26721,25451,25970) , .\n ' There are some who call me Tim? Kapoor, Nishikant X [EMAIL PROTECTED] Sent by: [EMAIL PROTECTED] 11/15/2001 03:09 PM To:[EMAIL PROTECTED] cc:(bcc: Tim Conway/LMT/SC/PHILIPS) Subject:Secure connection to testor refused ! Classification: Here is a newbie question. Hope, you will not mind. I am trying to backup a file from my NT machine onto AIX machine. Here is the setup: NT machine (testor): rsync is in c:\Rsync\rsync.exe and is running as a daemon. (I created a service for Rsync first). AIX machine (admx): rsync -v rsync version 2.3.1 Copyright Andrew Tridgell and Paul Mackerras admx::nkapoor telnet testor 873 Trying... Connected to testor. Escape character is '^]'. @RSYNCD: 24 So, the rsync daemon seems fine on testor but the problem occurs when I try this: admx::nkapoor rsync -v -e ssh --rsync-path=c:/rsync/rsync.exe guest@testor:c:/Temp/pix.log . Secure connection to testor refused. unexpected EOF in read_timeout I have checked that 'guest' is a valid user on testor. Could someone please help me understand what could be the problem here ? Thanks Nishi
Is anybody else gettting these?
I keep getting these smarmy sermons from somebody's mail account. I include my response to it below, though from its message, it probably won't be read. Tim Conway [EMAIL PROTECTED] 303.682.4917 Philips Semiconductor - Longmont TC 1880 Industrial Circle, Suite D Longmont, CO 80501 Available via SameTime Connect within Philips, n9hmg on AIM perl -e 'print pack(, 19061,29556,8289,28271,29800,25970,8304,25970,27680,26721,25451,25970), .\n ' There are some who call me Tim? - Forwarded by Tim Conway/LMT/SC/PHILIPS on 11/19/2001 10:35 AM - Tim Conway 11/19/2001 10:34 AM To:[EMAIL PROTECTED] cc: Subject:RE: Secure connection to testor refused !Link Classification:Unclassified Please go perform an illicit act upon yourself. Your sanctimonius preaching sickens me. I do not control the format to which my messages are converted as they leave our corporate intranet, If I had my choice, I would be using pine/smtp, rather than Lotus Notes. If you don't want to see mail from me, unsubscribe from the rsync mailing lists. Tim Conway [EMAIL PROTECTED] 303.682.4917 Philips Semiconductor - Longmont TC 1880 Industrial Circle, Suite D Longmont, CO 80501 Available via SameTime Connect within Philips, n9hmg on AIM perl -e 'print pack(, 19061,29556,8289,28271,29800,25970,8304,25970,27680,26721,25451,25970), .\n ' There are some who call me Tim? [EMAIL PROTECTED] 11/19/2001 10:02 AM To:Tim Conway/LMT/SC/PHILIPS@AMEC cc: Subject:RE: Secure connection to testor refused ! Classification: Hello stranger You send a Mail to my address or to a mailing list and I don't get (or read) your mail! Your mail was a HTML Mail and HTML Mails are bad! Only with 'not only text' mails, you get viruses and/or trojan horses. STOP THIS! See this pages for more infos: http://www.betips.net/etc/evilmail.html http://www.georgedillon.com/web/html_email_is_evil.shtml http://piology.org/mail/ (german) http://www.kasper-online.de/goldmail/goldmail.htm (german) On the other side, normal HTML Mails are SPAM and I don't like this. Thanks.
Re: modules madness / ssh regardless of daemon
-e ssh is for external transport. So is the single colon syntax (defiance:auth) Use this cmdline: rsync -avz defiance::auth /var/www/auth Also, I'm guessing you want the contents of auth in /var/www/auth, in which case you want rsync -avz defiance::auth /var/www else, you'll get /var/www/auth/auth The documentation (well written, easily read, but sometimes, the app doesn't obey the doc) says you could use rsync -avz defiance::auth/ /var/www/auth as the trailing / on a source means the contents of auth, rather than auth itself, but I have been screwed by that one... it does not get all of the contents, and with --delete, has caused me disasters. rsync -avz defiance::auth/* /var/www/auth will also work, if you DON'T have any .files in auth, and is handy if you don't own /var/www (doesn't matter if you're doing it as root). Tim Conway [EMAIL PROTECTED] 303.682.4917 Philips Semiconductor - Longmont TC 1880 Industrial Circle, Suite D Longmont, CO 80501 Available via SameTime Connect within Philips, n9hmg on AIM perl -e 'print pack(, 19061,29556,8289,28271,29800,25970,8304,25970,27680,26721,25451,25970), .\n ' There are some who call me Tim? Brian Clark [EMAIL PROTECTED] Sent by: [EMAIL PROTECTED] 11/19/2001 10:24 AM Please respond to Brian Clark To:[EMAIL PROTECTED] cc:(bcc: Tim Conway/LMT/SC/PHILIPS) Subject:modules madness / ssh regardless of daemon Classification: I just installed rsync on two machines, I think I'm a complete moron, and I need a clue bat. The remote machine's /etc/rsyncd.conf (just for testing): use chroot = no log file = /var/log/rsyncd.log pid file = /var/run/rsyncd.pid lock file = /var/run/rsync.lock [auth] path = /var/www/auth comment = apache authentication files. read only = yes Then for a test, I fired up /usr/local/bin/rsync --daemon on the remote machine. On the local machine, I did: % rsync -avz -e ssh defiance:auth /var/www/auth root@defiance's password: receiving file list ... link_stat auth : No such file or directory done client: nothing to do FYI, /var/www/auth on the remote machine _does_ exist. So that obviously didn't work, but then I did this: % rsync -avz -e ssh defiance:/var/www/auth /var/www/auth root@defiance's password: receiving file list ... done created directory /var/www/auth auth/ auth/foo/ auth/bar/ auth/baz/ etc And it works. But this next part I don't understand.. I kill the rsync daemon on the remote host, then I try the above command line again on the local host and it works!? Say what?! By using ssh does it bypass the remote daemon altogether? Plus, `hosts access' doesn't work in my /etc/rsyncd.conf and using rsync as a daemon through inetd or xinetd doesn't even let me make a connection. It's like it's not reading /etc/rsyncd.conf -- I just don't get it. % rsync --version rsync version 2.4.6 protocol version 24 Written by Andrew Tridgell and Paul Mackerras -- -Brian Clark | PGP is spoken here: 0xE4D0C7C8 Please, DO NOT carbon copy me on list replies.
Re: modules madness / ssh regardless of daemon
You're fine. You can either use the rsyncd (defiance::auth), thus going to your rsyncd, or you can use -e ssh, using ssh as the remote transport. rsync -options -e ssh remotehost:/path/to/auth /path/to/auth/mirror. if you use the second version, the rsyncd.conf is irrelevant you don't have to relate a path as a named module... you just use the path... period. Tim Conway [EMAIL PROTECTED] 303.682.4917 Philips Semiconductor - Longmont TC 1880 Industrial Circle, Suite D Longmont, CO 80501 Available via SameTime Connect within Philips, n9hmg on AIM perl -e 'print pack(, 19061,29556,8289,28271,29800,25970,8304,25970,27680,26721,25451,25970), .\n ' There are some who call me Tim? Brian Clark [EMAIL PROTECTED] Sent by: [EMAIL PROTECTED] 11/19/2001 11:50 AM Please respond to Brian Clark To:[EMAIL PROTECTED] cc:(bcc: Tim Conway/LMT/SC/PHILIPS) Subject:Re: modules madness / ssh regardless of daemon Classification: Hi Tim, @ 1:40:43 PM on 11/19/2001, [EMAIL PROTECTED] wrote: tcpc -e ssh is for external transport. So is the single colon syntax tcpc (defiance:auth) Use this cmdline: rsync -avz defiance::auth OK, I should have mentioned this earlier, and I forgot. I don't have rsh on [either] system because I've been told over and over that it's insecure (or was that something else?). Anyway, I think that's before my time, or maybe not. All I've ever know are telnet, then ssh, so that's what I immediately stuck to when rsync. I'm now wondering how I'm going to mirror anything unattended without having a valid ssh login/password in a file some where. On top of that, -e ssh renders rsyncd.conf useless (hosts access, auth users, etc.). :-\ -- -Brian Clark | PGP is spoken here: 0xE4D0C7C8 Please, DO NOT carbon copy me on list replies.
Re: dir list behavior differences?
add a /, and it will show the contents. add -r, and it will show all the contents, recursively. Incidentally: I've deleted and reinstalled notes, in the hopes that the user preference for internet mail (TEXT ONLY, instead of HTML) has been honored. I set that as soon as i started, but obviously, our gateways were either messing with the mail, or this stupid new R5 client does things on its own. Tim Conway [EMAIL PROTECTED] 303.682.4917 Philips Semiconductor - Longmont TC 1880 Industrial Circle, Suite D Longmont, CO 80501 Available via SameTime Connect within Philips, n9hmg on AIM perl -e 'print pack(, 19061,29556,8289,28271,29800,25970,8304,25970,27680,26721,25451,25970), .\n ' There are some who call me Tim? Don Fike [EMAIL PROTECTED] Sent by: [EMAIL PROTECTED] 11/19/2001 01:49 PM To: [EMAIL PROTECTED] cc: (bcc: Tim Conway/LMT/SC/PHILIPS) Subject:dir list behavior differences? Classification: Hello, I have set up an anonymous rsync server that seems to work as expected. However when I try to access the server from several other unix boxes using rsync with a command like this; rsync netlib3.cs.utk.edu::netlib I get the motd and then a line that states client: nothing to do Now I would expect this to give me a directory listing. I do get the directory listing as expected when doing the command from the local box. I have tried other servers listed here; http://sunsite.dk/info/guides/rsync/rsync-mirroring03.html and I get the same results, when using some of my clients I just get the motd and the client: nothing to do error. Is this an issue with backwards compatibility between new server and older client communications? Is there a way to get the listings as expected for all clients? Thanks, Don -- Don Fike _/_/_/ _/_/_/ _/ ICL/UTK _/_/ _/ [EMAIL PROTECTED]_/_/ _/ 865-974-0531_/_/_/ _/_/_/ _/_/_/ -- We're gonna see a brave new world where they run everyone a wire and hook us all up to a grid. Yessir, a veritable age of reason -Ulysses Everett McGill-
Re: Bandwidth Limits
Try this commandline: rsync -avz -e ssh --bwlimit=64 localfiles.tar.gz user@remote:/path/to/file/arch That should average out to use only about 1/2 of your bandwidth. You will find periods where it uses all of it, and periods where it uses none, but on average, it will use only as many kbps as you permit it on the cmdline .I notice that your example didn't actually use the --bwlimit parameter. rsync doesn't do any kind of adaptive logic to determine how much bandwidth it should use... it just takes everything it can, either up to actual bandwidth limits, the bwlimit parameter, or how fast it can put together and recieve the stream. Tim Conway [EMAIL PROTECTED] 303.682.4917 Philips Semiconductor - Longmont TC 1880 Industrial Circle, Suite D Longmont, CO 80501 Available via SameTime Connect within Philips, n9hmg on AIM perl -e 'print pack(, 19061,29556,8289,28271,29800,25970,8304,25970,27680,26721,25451,25970), .\n ' There are some who call me Tim? [EMAIL PROTECTED] Sent by: [EMAIL PROTECTED] 11/19/2001 05:27 PM To: [EMAIL PROTECTED] cc: (bcc: Tim Conway/LMT/SC/PHILIPS) Subject:Bandwidth Limits Classification: Has anyone noticed that the --bwlimit doesn't really work? I have MRTG stats happening, and on a 128Kbps circuit, an rsync with the following syntax takes up the whole line. $ rsync -avz -e ssh localfiles.tar.gz user@remote:/path/to/file/arch Any ideas? It's on an OpenBSD box going to a FreeBSD box. Both are 2.4.6. -#0
RE: dir list behavior differences?
You're not crazy. I've seen similar behaviour. The trailing / does not always (or even usually) bring about the documented behaviour. I have tried to use that before, and had rsync delete most of what i already had, even though it exists on the source... it simply doesn't come through. For directories with no .files, i use dir/*. for directories with .files, i have to sync the directory itself. Fortunately, in my application, that's not a problem. It just really bit me in the butt the few times i've tried to pull over directory contents. If I were a better programmer, i'd try again to debug it, but i'm not, so i just live with it. Tim Conway [EMAIL PROTECTED] 303.682.4917 Philips Semiconductor - Longmont TC 1880 Industrial Circle, Suite D Longmont, CO 80501 Available via SameTime Connect within Philips, n9hmg on AIM perl -e 'print pack(, 19061,29556,8289,28271,29800,25970,8304,25970,27680,26721,25451,25970), .\n ' There are some who call me Tim? Don Fike [EMAIL PROTECTED] 11/20/2001 07:42 AM To: Dave Dykstra [EMAIL PROTECTED] Tim Conway/LMT/SC/PHILIPS@AMEC cc: [EMAIL PROTECTED] Subject:RE: dir list behavior differences? Classification: The clients that aren't doing a directory listing are; rsync version 2.3.1 protocol version 20 rsync version 2.2.1 protocol version 19 The client that works as desired and expected is; rsync version 2.4.6 protocol version 24 I tried adding a trailing / but that didn't encourage the older versions to do a directory listing. Instead I got the response below; skipping directory / client: nothing to do Adding a -r did, but the recursion returns too much data to be useful. I can upgrade the older versions here but my concern and desire is to support those outside my department using older clients. I am getting the impression that those using older clients are accustomed to not seeing a directory listing. If this is true than I guess I am content with my setup. Is there is another way I can encourage a directory listing, maybe on the server side. Thanks, Don [EMAIL PROTECTED] -Original Message- From: Dave Dykstra [mailto:[EMAIL PROTECTED]] Sent: Monday, November 19, 2001 5:47 PM To: Don Fike Cc: [EMAIL PROTECTED] Subject: Re: dir list behavior differences? On Mon, Nov 19, 2001 at 03:49:27PM -0500, Don Fike wrote: Hello, I have set up an anonymous rsync server that seems to work as expected. However when I try to access the server from several other unix boxes using rsync with a command like this; rsync netlib3.cs.utk.edu::netlib/ I get the motd and then a line that states client: nothing to do Now I would expect this to give me a directory listing. I do get the directory listing as expected when doing the command from the local box. I have tried other servers listed here; http://sunsite.dk/info/guides/rsync/rsync-mirroring03.html and I get the same results, when using some of my clients I just get the motd and the client: nothing to do error. Is this an issue with backwards compatibility between new server and older client communications? Is there a way to get the listings as expected for all clients? You don't mention the version numbers, but I believe the issue is yes it is a matter of client version. Can you not upgrade the older clients? - Dave Dykstra
Re: Bandwidth Limits
That was just too small a test. --bwlimit= causes rsync to sleep for an appropriate interval after each data block (not tcp packet), in order to pull the average transfer rate down to the specified limit. Your reported transfer rate was only a little above 1kbps anyway, as the test was so small, there wasn't time to even get the sendsleepsendsleep cycle going... overhead stretched the time out so much that the 128k transferred took an insignificant portion of the total runtime. Try a larger test. maybe a meg or so. averages are meaningless on single instances... average height of an adult american male is about 6 feet, right? if you find a man 4 feet tall, that doesn't mean the average is wrong. Tim Conway [EMAIL PROTECTED] 303.682.4917 Philips Semiconductor - Longmont TC 1880 Industrial Circle, Suite D Longmont, CO 80501 Available via SameTime Connect within Philips, n9hmg on AIM perl -e 'print pack(, 19061,29556,8289,28271,29800,25970,8304,25970,27680,26721,25451,25970), .\n ' There are some who call me Tim? [EMAIL PROTECTED] Sent by: [EMAIL PROTECTED] 11/19/2001 06:04 PM To: Tim Conway/LMT/SC/PHILIPS@AMEC cc: [EMAIL PROTECTED] Subject:Re: Bandwidth Limits Classification: On Mon, 2001-11-19 at 17:45:26 -0700, [EMAIL PROTECTED] wrote... ; Try this commandline: ; rsync -avz -e ssh --bwlimit=64 localfiles.tar.gz ; user@remote:/path/to/file/arch I did. This is what I getI created a 128K file.. bash-2.05$ rsync --stats -avz --bwlimit=8 -e ssh blah:~/out . user@remote's password: receiving file list ... done out Number of files: 1 Number of files transferred: 1 Total file size: 127976 bytes Total transferred file size: 127976 bytes Literal data: 127976 bytes Matched data: 0 bytes File list size: 57 Total bytes written: 32 Total bytes read: 15325 wrote 32 bytes read 15325 bytes 1228.56 bytes/sec total size is 127976 speedup is 8.33 bash-2.05$ ls -l out -rw--- 1 user luser 127976 Nov 19 19:01 out
Re: --no-detach option?
Not in daemon mode. = # ps -ef |grep rsync # rsync --daemon # ps -ef |grep rsync root 23716 1 0 07:42:58 ?0:00 rsync --daemon # rsync localhost:: tconway # Like most daemons, it forks by default. I've wanted this feature for some time, for debugging, but instead, just started it with truss, with the fork option. Tim Conway [EMAIL PROTECTED] 303.682.4917 Philips Semiconductor - Longmont TC 1880 Industrial Circle, Suite D Longmont, CO 80501 Available via SameTime Connect within Philips, n9hmg on AIM perl -e 'print pack(, 19061,29556,8289,28271,29800,25970,8304,25970,27680,26721,25451,25970), .\n ' There are some who call me Tim? Dave Dykstra [EMAIL PROTECTED] Sent by: [EMAIL PROTECTED] 11/21/2001 07:15 AM To: [EMAIL PROTECTED] cc: (bcc: Tim Conway/LMT/SC/PHILIPS) Subject:Re: --no-detach option? Classification: On Wed, Nov 21, 2001 at 12:23:29PM -, Max Bowsher wrote: I did the same, for cygwin, but I called the option --debug-daemon. --no-detach is probably a better name. Heres my patch. - Original Message - From: Jos Backus [EMAIL PROTECTED] To: [EMAIL PROTECTED] Sent: Tuesday, November 20, 2001 11:05 PM Subject: --no-detach option? How about adding a --no-detach option (to be used in combination with --daemon) to rsync so it can be run under Dan Bernstein's daemontools' supervise? If there's interest I'll provide a patch. Rsync will already not detach if stdin is not a socket. Is that sufficient? - Dave Dykstra
Re: Rsync Throughput issues
You might try putting the rsycd (server) on the source. rsync as a server is really designed for read-only. The read-write support is an afterthough, and very flaky. Martin, Tridge, Wayne: correct me if that's not true any more. Make sure you're not using --bwlimit=, unless you need to parcel your bandwidth (In my situation, I do). Don't use -H, unless you actually use multiple hard links to files, as that adds tremendous overhead on large filesystems. If the files are nfs mounted, use -W, else you will read both files completely, before you transfer anything... might as well just send it if datestamp or size differs. I doubt ssh was giving you any slowdown, unless you're cpu-limited on one or the other box. If you don't need the security, and policy permits, plain rsh transport might be a bit faster. Tim Conway [EMAIL PROTECTED] 303.682.4917 Philips Semiconductor - Longmont TC 1880 Industrial Circle, Suite D Longmont, CO 80501 Available via SameTime Connect within Philips Available as n9hmg on AIM perl -e 'print pack(, 19061,29556,8289,28271,29800,25970,8304,25970,27680,26721,25451,25970), .\n ' There are some who call me Tim? Ben Ricker [EMAIL PROTECTED]@lists.samba.org on 10/08/2001 06:50:02 PM Sent by: [EMAIL PROTECTED] To: [EMAIL PROTECTED] cc: (bcc: Tim Conway/LMT/SC/PHILIPS) Subject: Rsync Throughput issues Classification: I am using Rsync between a Redhat Linux box and an AIX RS600. We have a about 30gb of database we need to sync to a backup server. Sounds good, right? The problem is that Rsync is so slow when we do the initial dump. We have files that are 1 - 5gb. It takes around 14-20 hours to Rsync the file structure, which means our daily backup will miss that days data because we are waiting for the data to rsync. At first we were using ssh as the agent but we thought that might be slowing us down. We then tried to use rsync as a server and push the data to the module/zone of the server. Consistently, it takes roughly 2 minutes to transfer 10meg of data I thought it was a network problem so we tried ftp. A 10 mb file took 90% the time (about 20 seconds. This is on a 100bit Full Duplex switched network with both boxes on the SAME subnet. Am I missing something? Should I expect these speeds? Could there be something on the App switch throttling rsync for some reason? Any thought will help, Ben Ricker System Administrator Wellinx.com
Re: @ERROR: chroot failed
Three things: Does the system support chroot? Is the rsyncd running as root? Is the path correctly specified in the rsyncd.conf? I can't think of anything else I've run into or read of, to cause that problem. If it's not number three, try running it as root. If that fails, try adding use chroot = no to your rsyncd.conf. Tim Conway [EMAIL PROTECTED] 303.682.4917 Philips Semiconductor - Longmont TC 1880 Industrial Circle, Suite D Longmont, CO 80501 Available via SameTime Connect within Philips Available as n9hmg on AIM perl -e 'print pack(, 19061,29556,8289,28271,29800,25970,8304,25970,27680,26721,25451,25970), .\n ' There are some who call me Tim? Jason Helfman [EMAIL PROTECTED]@lists.samba.org on 10/10/2001 12:47:11 PM Sent by: [EMAIL PROTECTED] To: [EMAIL PROTECTED] cc: (bcc: Tim Conway/LMT/SC/PHILIPS) Subject: @ERROR: chroot failed Classification: Why would I be getting this on a box? I have the same permissions on many other directories. Using verbose mode shows nothing more then the error in the Subject line. Any help is appreciated. -- Jason G Helfman Network Administrator BizRate.com 310.754.1264 desk 310.466.2319 cell Fingerprint: DA13 C109 072B CC12 B568 8D84 E9A2 6A7D C479 BCFB GnuPG http://www.gnupg.org Get Private! 1024D/D75E0A36
Re: problems between different versions of rsync
I'm just guessing: the files on the machine where you're invoking the transfer belong to the user who created them, probably with his default group, and on the machine you're sending to, they belong to the user in user@host, and his default group, right? Tim Conway [EMAIL PROTECTED] 303.682.4917 Philips Semiconductor - Longmont TC 1880 Industrial Circle, Suite D Longmont, CO 80501 Available via SameTime Connect within Philips Available as n9hmg on AIM perl -e 'print pack(, 19061,29556,8289,28271,29800,25970,8304,25970,27680,26721,25451,25970), .\n ' There are some who call me Tim? Jeff Beley [EMAIL PROTECTED]@lists.samba.org on 10/17/2001 11:39:38 AM Sent by: [EMAIL PROTECTED] To: [EMAIL PROTECTED] cc: (bcc: Tim Conway/LMT/SC/PHILIPS) Subject: problems between different versions of rsync Classification: Are there issues between different versions of rsync? Specifically I have a 2.4.6 on solaris 8 pushing to a 2.3.0 on solaris 2.6 over ssh(F-Secure version). The issue that I am seeing is that when I use the following command: rsync --delete -rlvp -e ssh2 $localdir user@host:remotedir The user and group are different on both boxes(neither being root). Any help would be appreciated. --Jeff -- Jeff Beley Internet Services
Re: problems between different versions of rsync
If a user does not have authority to create files under another user id (meaning he has a null user id, meaning root, in most cases), all files he creates will belong to him. rsync is graceful enough to not even try. For basic unix skills, I'd recommend O'Reilly's Unix in a Nutshell http://www.oreilly.com/catalog/unixnut3 I've never read it, but O'Reilly seems to be the best with the technical books. Incidentally, even if you were running as root on the other end, you wouldn't preserve ownerships, as your commandline doesn't say to do so. you'd need the -o and -g options. You'd probably just want to use the -a (archive, preserve everything about it) option. It's equivalent to glopr (preserve group, symlinks, ownership, and permissions, recursively). The rsync distribution comes with some pretty good documentation on its options and functionality. On the versions: I'm not sure where the cutoffs are for your versions, but as they work at all, you're probably ok. You'll probably, however, want to upgrade the 2.3.0 system, and if you're running large jobs, you'll want to, at the least, patch the 2.4 with the rsync-nohang patches. I personally, am getting so frustrated with the 2.4 series on huge trees (currently about 130Gb in 24M files), that i'm about ready to revert to 2.3.1. I'll miss --bwlimit, but oh, well. Tim Conway [EMAIL PROTECTED] 303.682.4917 Philips Semiconductor - Longmont TC 1880 Industrial Circle, Suite D Longmont, CO 80501 Available via SameTime Connect within Philips Available as n9hmg on AIM perl -e 'print pack(, 19061,29556,8289,28271,29800,25970,8304,25970,27680,26721,25451,25970), .\n ' There are some who call me Tim? Jeff Beley [EMAIL PROTECTED] on 10/17/2001 01:21:52 PM To: Tim Conway/LMT/SC/PHILIPS@AMEC cc: Subject: Re: problems between different versions of rsync Classification: correct. On Wed, Oct 17, 2001 at 01:19:39PM -0600, [EMAIL PROTECTED] [EMAIL PROTECTED] wrote: I'm just guessing: the files on the machine where you're invoking the transfer belong to the user who created them, probably with his default group, and on the machine you're sending to, they belong to the user in user@host, and his default group, right? Tim Conway [EMAIL PROTECTED] 303.682.4917 Philips Semiconductor - Longmont TC 1880 Industrial Circle, Suite D Longmont, CO 80501 Available via SameTime Connect within Philips Available as n9hmg on AIM perl -e 'print pack(, 19061,29556,8289,28271,29800,25970,8304,25970,27680,26721,25451,25970), .\n ' There are some who call me Tim? Jeff Beley [EMAIL PROTECTED]@lists.samba.org on 10/17/2001 11:39:38 AM Sent by: [EMAIL PROTECTED] To: [EMAIL PROTECTED] cc: (bcc: Tim Conway/LMT/SC/PHILIPS) Subject: problems between different versions of rsync Classification: Are there issues between different versions of rsync? Specifically I have a 2.4.6 on solaris 8 pushing to a 2.3.0 on solaris 2.6 over ssh(F-Secure version). The issue that I am seeing is that when I use the following command: rsync --delete -rlvp -e ssh2 $localdir user@host:remotedir The user and group are different on both boxes(neither being root). Any help would be appreciated. --Jeff -- Jeff Beley Internet Services -- Jeff Beley Internet Services
Re: group ownership
Is the user you're rsyncing as a member of cdburners, or is he root? Tim Conway [EMAIL PROTECTED] 303.682.4917 Philips Semiconductor - Longmont TC 1880 Industrial Circle, Suite D Longmont, CO 80501 Available via SameTime Connect within Philips Available as n9hmg on AIM perl -e 'print pack(, 19061,29556,8289,28271,29800,25970,8304,25970,27680,26721,25451,25970), .\n ' There are some who call me Tim? Ronald D. Walkup [EMAIL PROTECTED]@lists.samba.org on 10/17/2001 01:02:14 PM Sent by: [EMAIL PROTECTED] To: rsync [EMAIL PROTECTED] cc: (bcc: Tim Conway/LMT/SC/PHILIPS) Subject: group ownership Classification: I am attempting to rsync data from a rsync server and set the permissions to a different gid on the client: my servers name is rserver01 my clients name is rclient01 here is the rysync.conf contained on rserver01: # log file log file = /var/adm/rsync_log # global options for all modules dont compress = *.gz *.tgz *.zip *.z *.rpm *.deb *.iso *.bz2 *.tbz uid = nobody gid = nobody transfer logging=true list=true [test] path=/var/tmp/rsync_test comment = rsync_test read only = true hosts allow=rclient gid = cdburners on the server, ls -al of the source directory indicates that group ownership is visitor: rserver01ls -al total 12 drwxr-xr-x 3 rwalkup visitor 512 Oct 17 13:43 . drwxrwxrwt 6 sys sys 1536 Oct 17 13:31 .. -rw-r--r-- 1 rwalkup visitor 79 Mar 13 2001 hosts drwxr-xr-x 3 rwalkup visitor 512 Mar 13 2001 inet -rw-r--r-- 1 rwalkup visitor 330 Mar 21 2001 vfstab I then execute the following syntax from the client: rclien01 rsync -avz rserver01::test /var/tmp/rsync_test after the copy has completed, I then execute: rclient01 ls -al /var/tmp/rsync_test total 12 drwxr-xr-x 3 rwalkup visitor 512 Oct 17 13:49 . drwxrwxrwt 6 sys sys 1536 Oct 17 13:49 .. -rw-r--r-- 1 rwalkup visitor 79 Mar 13 2001 hosts drwxr-xr-x 3 rwalkup visitor 512 Mar 13 2001 inet -rw-r--r-- 1 rwalkup visitor 330 Mar 21 2001 vfstab Is it not possible that the group ownerhip be changed to cdburners as the rsyncd.conf file indicates? cdburners exist on both client and server. any ideas would be greatly appreciated. Thanks, Ron
Re: include/exclude ?
rsync doesn't do the odd wildcard syntax you show there... you'll have to
do --include=an* --include=mp* --include=ERR*
Tim Conway
[EMAIL PROTECTED]
303.682.4917
Philips Semiconductor - Longmont TC
1880 Industrial Circle, Suite D
Longmont, CO 80501
Available via SameTime Connect within Philips, n9hmg on AIM
perl -e 'print pack(,
19061,29556,8289,28271,29800,25970,8304,25970,27680,26721,25451,25970),
.\n '
There are some who call me Tim?
Kapoor, Nishikant X [EMAIL PROTECTED]
Sent by: [EMAIL PROTECTED]
11/30/2001 09:09 AM
To: [EMAIL PROTECTED]
cc: (bcc: Tim Conway/LMT/SC/PHILIPS)
Subject:include/exclude ?
Classification:
All,
Could someone please help me resolve this:
[admx:test] $ ls
ERR01 ah01 ah02 an01 an02 mp01 mp02
[admx:test] $ ls {an,mp,ERR}*
ERR01 an01 an02 mp01 mp02
I want to rsync only the {an,mp,ERR}* files across using the following
command but do not see the expected results.
[admx:test] $ rsync -va --exclude=* --include={an,mp,ERR}* ./*
mm2x::AppError
building file list ... done
wrote 67 bytes read 337 bytes 269.33 bytes/sec
total size is 0 speedup is 0.00
I am running 'rsync version 2.3.1 protocol version 20' on a IBM
RS6000/F50 running AIX 4.3.1 on both sides.
Thanks
Nishi
RE: include/exclude ?
You're right. the order dependency actually lets you create very complex
include/exclude rules. for each item, each --include and --exclude is
evaluated in commandline order, until the item has either passed all the
tests, and is thus included, or has been excluded, at which point
evaluation restarts with the next item. if you --exclude=* first, every
file gets excluded right away, and never gets checked to see if it matches
--include=an* .
Group: That's a correct synopsis of the behaviour, isn't it? If not,
don't let my error stand.
Tim Conway
[EMAIL PROTECTED]
303.682.4917
Philips Semiconductor - Longmont TC
1880 Industrial Circle, Suite D
Longmont, CO 80501
Available via SameTime Connect within Philips, n9hmg on AIM
perl -e 'print pack(,
19061,29556,8289,28271,29800,25970,8304,25970,27680,26721,25451,25970),
.\n '
There are some who call me Tim?
Kapoor, Nishikant X [EMAIL PROTECTED]
11/30/2001 01:40 PM
To: Tim Conway/LMT/SC/PHILIPS@AMEC
Kapoor, Nishikant X [EMAIL PROTECTED]
cc: [EMAIL PROTECTED]
[EMAIL PROTECTED]
Subject:RE: include/exclude ?
Classification:
It is finally working but I am not sure if I understand it right. It
seems to me that the order in which 'include' and 'exclude' exist in the
command line is making all the difference. Check out the following two
commands:
*** include is BEFORE exclude - Working fine ! ***
[admx:test] $ rsync -va --include=an* --exclude=* ./* mm2x::AppError
building file list ... done
an01
an02
wrote 178 bytes read 369 bytes 1094.00 bytes/sec
total size is 0 speedup is 0.00
*** include is AFTER exclude - Not Working ! ***
[admx:test] $ rsync -va --exclude=* --include=an* ./* mm2x::AppError
building file list ... done
wrote 67 bytes read 337 bytes 269.33 bytes/sec
total size is 0 speedup is 0.00
Unless I am missing something obvious, the order is all that is
different in two statements above.
Nishi
-Original Message-
From: [EMAIL PROTECTED] [SMTP:[EMAIL PROTECTED]]
rsync doesn't do the odd wildcard syntax you show there... you'll have
to
do --include=an* --include=mp* --include=ERR*
Tim Conway
[EMAIL PROTECTED]
303.682.4917
Philips Semiconductor - Longmont TC
1880 Industrial Circle, Suite D
Longmont, CO 80501
Available via SameTime Connect within Philips, n9hmg on AIM
perl -e 'print pack(,
19061,29556,8289,28271,29800,25970,8304,25970,27680,26721,25451,25970)
,
.\n '
There are some who call me Tim?
Kapoor, Nishikant X [EMAIL PROTECTED]
All,
Could someone please help me resolve this:
[admx:test] $ ls
ERR01 ah01 ah02 an01 an02 mp01 mp02
[admx:test] $ ls {an,mp,ERR}*
ERR01 an01 an02 mp01 mp02
I want to rsync only the {an,mp,ERR}* files across using the
following
command but do not see the expected results.
[admx:test] $ rsync -va --exclude=* --include={an,mp,ERR}* ./*
mm2x::AppError
building file list ... done
wrote 67 bytes read 337 bytes 269.33 bytes/sec
total size is 0 speedup is 0.00
I am running 'rsync version 2.3.1 protocol version 20' on a IBM
RS6000/F50 running AIX 4.3.1 on both sides.
Thanks
Nishi
Re: Bug in rsyncd 2.5.0 while handling config file string values
Well, that's a bug, I'd reckon, but i want to point out that a config file
with only one line is invalid. you need to have at least a module and its
path defined. I wouldn't expect daemon mode to gracefully handle an
invalid config, though simply declaring that the file is invalid and
exiting would be nice.
Tim Conway
[EMAIL PROTECTED]
303.682.4917
Philips Semiconductor - Longmont TC
1880 Industrial Circle, Suite D
Longmont, CO 80501
Available via SameTime Connect within Philips, n9hmg on AIM
perl -e 'print pack(,
19061,29556,8289,28271,29800,25970,8304,25970,27680,26721,25451,25970),
.\n '
There are some who call me Tim?
Heikki Vatiainen [EMAIL PROTECTED]
Sent by: [EMAIL PROTECTED]
12/02/2001 06:14 AM
To: [EMAIL PROTECTED]
cc: (bcc: Tim Conway/LMT/SC/PHILIPS)
Subject:Bug in rsyncd 2.5.0 while handling config file string values
Classification:
I compiled and tried rsync 2.5.0 but could not get the server
running. loadparm.c:string_set() now calls free() which it did not do
in 2.4.6 and this free() tries to free memory that was not allocated
with malloc.
Here is a gdb run (done after adding return before fork() in
become_daemon) which shows where it crashes and what my minimal config
file for demonstrating the bug was. The config file only had one line
which was: uid = foobar
/usr/local/src/rsync/rsync-2.5.0:gdb ./rsync
GNU gdb 5.0.90-cvs (MI_OUT)
(gdb) r --daemon --config=../rsyncd.conf
Starting program: /usr/local/src/rsync/rsync-2.5.0/./rsync --daemon
--config=../rsyncd.conf
Program received signal SIGSEGV, Segmentation fault.
0x4009cb2d in free () from /lib/libc.so.6
(gdb) bt
#0 0x4009cb2d in free () from /lib/libc.so.6
#1 0x08061286 in string_set (s=0x807ace4, v=0x808b374 foobar) at
loadparm.c:400
#2 0x08061a20 in lp_do_parameter (snum=-2, parmname=0x808b370 uid,
parmvalue=0x808b374 foobar)
at loadparm.c:647
#3 0x08061b8b in do_parameter (parmname=0x808b370 uid,
parmvalue=0x808b374 foobar) at loadparm.c:678
#4 0x0806057b in Parameter (InFile=0x808b200, pfunc=0x8061b58
do_parameter, c=0) at params.c:405
#5 0x080606a4 in Parse (InFile=0x808b200, sfunc=0, pfunc=0x8061b58
do_parameter) at params.c:461
#6 0x08060848 in pm_process (FileName=0xb80c ../rsyncd.conf,
sfunc=0,
pfunc=0x8061b58 do_parameter) at params.c:540
#7 0x08061cc7 in lp_load (pszFname=0x808b1c0 ../rsyncd.conf,
globals_only=1) at loadparm.c:747
#8 0x08063095 in daemon_main () at clientserver.c:506
#9 0x0805225a in main (argc=0, argv=0x0) at main.c:817
#10 0x400476ef in __libc_start_main () from /lib/libc.so.6
(gdb) select 1
(gdb) p s
$1 = (char **) 0x807ace4
(gdb) p *s
$2 = 0x807680c nobody
(gdb) p sDefault
$3 = {name = 0x0, path = 0x0, comment = 0x0, lock_file = 0x8076813
/var/run/rsyncd.lock,
read_only = 1, list = 1, use_chroot = 1, transfer_logging = 0,
ignore_errors = 0,
uid = 0x807680c nobody, gid = 0x807680c nobody, hosts_allow = 0x0,
hosts_deny = 0x0,
auth_users = 0x0, secrets_file = 0x0, strict_modes = 1, exclude = 0x0,
exclude_from = 0x0,
include = 0x0, include_from = 0x0, log_format = 0x80767f3 %o %h [%a] %m
(%u) %f %l,
refuse_options = 0x0, dont_compress = 0x80767c0 *.gz *.tgz *.zip *.z
*.rpm *.deb *.iso *.bz2 *.tbz,
timeout = 0, max_connections = 0, ignore_nonreadable = 0}
(gdb) shell cat ../rsyncd.conf
uid = foobar
(gdb)
--
Heikki Vatiainen * [EMAIL PROTECTED]
Tampere University of Technology * Tampere, Finland
Re: rsync-2.5.1pre1 with -F option
I always get that too. I just figured i'd screwed something up.
solaris 7
Tim Conway
[EMAIL PROTECTED]
303.682.4917
Philips Semiconductor - Longmont TC
1880 Industrial Circle, Suite D
Longmont, CO 80501
Available via SameTime Connect within Philips, n9hmg on AIM
perl -e 'print pack(,
19061,29556,8289,28271,29800,25970,8304,25970,27680,26721,25451,25970),
.\n '
There are some who call me Tim?
Ayamura KIKUCHI [EMAIL PROTECTED]
Sent by: [EMAIL PROTECTED]
12/02/2001 11:09 AM
To: [EMAIL PROTECTED]
cc: (bcc: Tim Conway/LMT/SC/PHILIPS)
Subject:rsync-2.5.1pre1 with -F option
Classification:
I compiled and tried rsync 2.5.1pre1.
RSYNC with -F option dumps a core.
% gdb ./rsync
GNU gdb 5.1
(gdb) r -F
Starting program: /work/rsync-2.5.1pre1/./rsync -F
Program received signal SIGSEGV, Segmentation fault.
write_batch_argvs_file (orig_argc=-2, argc=0, argv=0x0) at batch.c:153
153if ( !strcmp(argv[i],-F) ){ /* safer to change it here
than script*/
...
-- ayamura
Re: Why does one of there work and the other doesn't
rsync already has a memory-hogging issue. Imagine having it search your entire directory tree, checksumming all files, storing and sending them all, comparing both lists looking for matching date/time/checksums to guess where you've moved files to. You'd be better off to use a wrapper the tools you move files with, keeping a replayable log, and have your mirrors retrieve and replay that log, before doing the rsync. Tim Conway [EMAIL PROTECTED] 303.682.4917 Philips Semiconductor - Longmont TC 1880 Industrial Circle, Suite D Longmont, CO 80501 Available via SameTime Connect within Philips, n9hmg on AIM perl -e 'print pack(, 19061,29556,8289,28271,29800,25970,8304,25970,27680,26721,25451,25970), .\n ' There are some who call me Tim? Phil Howard [EMAIL PROTECTED] Sent by: [EMAIL PROTECTED] 12/03/2001 09:04 AM To: [EMAIL PROTECTED] cc: (bcc: Tim Conway/LMT/SC/PHILIPS) Subject:Re: Why does one of there work and the other doesn't Classification: On Mon, Dec 03, 2001 at 12:09:16AM +1100, Martin Pool wrote: | On 30 Nov 2001, Randy Kramer [EMAIL PROTECTED] wrote: | | I am not sure which end the 100 bytes per file applies to, and I guess | that is the RAM memory footprint?. Does rsync need 100 bytes for each | file that might be transferred during a session (all files in the | specified directory(ies)), or does it need only 100 bytes as it does one | file at a time? | | At the moment that is 100B for all files to be transferred in the | whole session. This is a big limit to scalability at the moment, and | a goal of mine is to reduce it to at most holding file information | from a single directory in memory. It would still be nice to have an option to gather all files at once, but this will be of value if it also gathers all the checksums and syncronizes files moves that have happened on the source end by doing the syncronization of the moved file to the new location using the old (checksum matched) file on the destination end. Right now if a file gets moved from one location to another (especially in a different directory, which is often the case with a re-organization) things get retransferred even though most every file already exists somewhere on the destination. -- - | Phil Howard - KA9WGN | Dallas | http://linuxhomepage.com/ | | [EMAIL PROTECTED] | Texas, USA | http://phil.ipal.org/ | -
Re: Netware modify bit changed
I think that what Juan means is that when you rsync from one system where the archive bit is unset, to another place, these files, which were archive=0 on their source, are new creations on the target system, and have the archive bit set. Rsync has no provision to preserve these attributes, as there's really no unix attribute corresponding to the dos archive attribute. I'm guessing that system isn't preserved, either, and i wonder what the handling of read-only and hidden would be... is readonly equivalent to ugo-w, and is hidden ugo-r? The unix attribute for hidden is to start the filename with a dot. I would expect the dos attributes to just be ignored, and a readonly, system, hidden, readonly file from one system would arrive at another system as a non-system, readwrite, visible file, with the archive attribute set. Incidentally, since it's netware-to-netware, what about NCP? I don't know much about it, but i understand that it transfers the underlying netware information instead of going through all the filesystem layers... very fast, and i think it can go system-to-system within a netware domain. Does it still even exist post 3.12? On 5 Dec 2001, Juan J. L?pez [EMAIL PROTECTED] wrote: Dave: With archive bit I mean a MS-DOS file attribute (like read only, system or hidden). When the archive attribute of a file is set, that file is presumed to be changed after the last backup and then must be copied again. The backup aplication Tim Conway [EMAIL PROTECTED] 303.682.4917 Philips Semiconductor - Longmont TC 1880 Industrial Circle, Suite D Longmont, CO 80501 Available via SameTime Connect within Philips, n9hmg on AIM perl -e 'print pack(, 19061,29556,8289,28271,29800,25970,8304,25970,27680,26721,25451,25970), .\n ' There are some who call me Tim?
Re: bug in permissions on symlinks
The only circumstance where i could see symlink ownership being an issue would be in the case where one might need to be changed, on those systems which support that. Most i've seen delete and recreate the link, so if the person needing to own the link has write, with no sticky bit, on the containing directory,, he's good to go. Can anyone see another issue? Certainly, the whole inane follow-link behaviour of chown and chmod are big traps. I was shocked to see a chown down a users directory tree on solaris make him the owner of many system files on the system i did it from (nfs user dirs), and won't make the same mistake twice. If rsync isn't going to have predictable behaviour on chowning (chmodding too, on some systems), perhaps we should let it leave ownerships and modes on symlinks at the system default. Scenario: I have an account on a system being backed up to another system, with rsync. I link /etc/security/password (or /etc/shadow/password, or /etc/shadow) into my tree. i get backed up. I go to the other system. I edit /etc/passwd, since it now belongs to me, moving my password into root's password field ( saving the old one so i can cover my tracks). I log in as root. I do what I want. I put the password back and fix the ownership. I log back out of root. If rsync gets a lchown or sensible chown, this won't happen, if it doesn't, it could. Tim Conway [EMAIL PROTECTED] 303.682.4917 Philips Semiconductor - Longmont TC 1880 Industrial Circle, Suite D Longmont, CO 80501 Available via SameTime Connect within Philips, n9hmg on AIM perl -e 'print pack(, 19061,29556,8289,28271,29800,25970,8304,25970,27680,26721,25451,25970), .\n ' There are some who call me Tim?
Re: Problems with rsync 2.5.1pre1 and hardlinks
I have to post a correction to this: * Why would symlinks eat up disk space? Correction, of course they don't. I have to use hardlinks because every a symbolic link does, in fact, consume disk space. Unlike a hard link, (which is just another pointer to the files data, a symbolic link is a pointer to the files name, and fits into the directories data blocks, and takes up an allocation of filesystem data space, in which to store the arbitrary-length name, rather than the inode number. One effect of this is that it allows a symbolic link to point out of the filesystem its in., because a directory entry points to an inode, having no way to indicate that its in another filesystem. hard link=directory entry=a name and an inode. symbolic link=(like a)file containing the name of what it points to. I know, technically, a symlink is not a file. That distinction is maintained by the 0xF000 nybble of the mode of the object. nevertheless, a directory takes space, a file takes space, a symbolic link takes space. An additional hard link to an existing file takes only directory space, which, if it's not enough of an addition to that directories existing data to cause the filesystem driver to add another allocation to the directories data space, takes up no more disk space. A symlink, however, has the same effect in the directory, but, in addition, gets its own data space, and inode, as well. In this example, an empty file is created. it takes an inode, no space. adding another link to it takes neither space, nor an inode. adding a symbolic link to it takes up both an inode and space... another symlink, to the other hard link, does the same. adding 512 new hard links to the original file takes up no more inodes, but it does take up disk space, by causing the directory to expand to hold all those names. tconway@atlas /var/tmp/testdf . /var (/dev/dsk/c0t0d0s3 ): 1662512 blocks 250481 files /var/tmp/testls -ld drwxrwxrwx 2 tconway Vlsieng 512 Dec 12 07:59 . tconway@atlas /var/tmp/testtouch emptyfile tconway@atlas /var/tmp/testdf . /var (/dev/dsk/c0t0d0s3 ): 1662512 blocks 250480 files tconway@atlas /var/tmp/testln emptyfile emptyfile2 tconway@atlas /var/tmp/testdf . /var (/dev/dsk/c0t0d0s3 ): 1662512 blocks 250480 files tconway@atlas /var/tmp/testln -s emptyfile linktoemptyfile tconway@atlas /var/tmp/testdf . /var (/dev/dsk/c0t0d0s3 ): 1662510 blocks 250479 files tconway@atlas /var/tmp/testln -s emptyfile2 linktoemptyfile2 tconway@atlas /var/tmp/testdf . /var (/dev/dsk/c0t0d0s3 ): 1662508 blocks 250478 files tconway@atlas /var/tmp/testls -li total 4 27376 -rw-rw-rw- 2 tconway Vlsieng0 Dec 12 07:59 emptyfile 27376 -rw-rw-rw- 2 tconway Vlsieng0 Dec 12 07:59 emptyfile2 27377 lrwxrwxrwx 1 tconway Vlsieng9 Dec 12 08:00 linktoemptyfile - emptyfile 27380 lrwxrwxrwx 1 tconway Vlsieng 10 Dec 12 08:00 linktoemptyfile2 - emptyfile2 tconway@atlas /var/tmp/testls -ld drwxrwxrwx 2 tconway Vlsieng 512 Dec 12 08:00 . tconway@atlas /var/tmp/testfor a in 0 1 2 3 4 5 6 7 8 9 a b c d e f do for b in 0 1 2 3 4 5 6 7 8 9 a b c d e f do ln emptyfile $a$b done done tconway@atlas /var/tmp/testls -ld drwxrwxrwx 2 tconway Vlsieng 3584 Dec 12 08:03 . tconway@atlas /var/tmp/testdf . /var (/dev/dsk/c0t0d0s3 ): 1662502 blocks 250478 files tconway@atlas /var/tmp/test I know it's not rsync-specific, but we're mostly unix guys, and need to be correct. Tim Conway [EMAIL PROTECTED] 303.682.4917 Philips Semiconductor - Longmont TC 1880 Industrial Circle, Suite D Longmont, CO 80501 Available via SameTime Connect within Philips, n9hmg on AIM perl -e 'print pack(, 19061,29556,8289,28271,29800,25970,8304,25970,27680,26721,25451,25970), .\n ' There are some who call me Tim? [EMAIL PROTECTED] Sent by: [EMAIL PROTECTED] 12/11/2001 11:42 PM To: Dave Dykstra [EMAIL PROTECTED] [EMAIL PROTECTED] cc: (bcc: Tim Conway/LMT/SC/PHILIPS) Subject:Re: Problems with rsync 2.5.1pre1 and hardlinks Classification: Dave Dykstra schrieb am Mon, Dec 10, 2001 at 02:21:46PM -0600: [...] * * * * * Ideas: * * 1. Would it be possible to use symlinks instead of hardlinks? That * *would give you more flexibility to split things up however you * *like. * * 2. Perhaps you could break it up into ~70 copies, where each time you * *give it the first directory that contains the data and another * *one that contains one of the hardlinks. * * Both alternatives will eat up huge amounts of disk space as the numbers * above suggest. I will therefore consider plugging in more mem/swap before * trying
Re: Problems with rsync 2.5.1pre1 and hardlinks
Well, I'll be damned. I'd never run into that trick. My apologies.
Tim Conway
[EMAIL PROTECTED]
303.682.4917
Philips Semiconductor - Longmont TC
1880 Industrial Circle, Suite D
Longmont, CO 80501
Available via SameTime Connect within Philips, n9hmg on AIM
perl -e 'print pack(,
19061,29556,8289,28271,29800,25970,8304,25970,27680,26721,25451,25970),
.\n '
There are some who call me Tim?
Martin Pool [EMAIL PROTECTED]
12/12/2001 03:27 PM
To: Tim Conway/LMT/SC/PHILIPS@AMEC
cc: [EMAIL PROTECTED]
Dave Dykstra [EMAIL PROTECTED]
[EMAIL PROTECTED]
Subject:Re: Problems with rsync 2.5.1pre1 and hardlinks
Classification:
On 12 Dec 2001, [EMAIL PROTECTED] wrote:
An additional hard link to an existing file takes only directory
space, which, if it's not enough of an addition to that directories
existing data to cause the filesystem driver to add another
allocation to the directories data space, takes up no more disk
space. A symlink, however, has the same effect in the directory,
but, in addition, gets its own data space, and inode, as well.
Actually on Linux (and some others?) the content of a symlink is
normally stored *inside* the inode, in the space that would otherwise
be used for pointers to the data and indirect blocks. This only helps
if the target of the symlink as a string is short enough to fit in
this uint32[15] field, in other words 60 bytes. These are called
fast symlinks, and you can see them listed separately in the output
of e2fsck. Of course as well as not occupying blocks they have the
advantage of being read faster, and not using any time in modifying
block allocation maps, etc.
If the symlink is longer, data blocks are allocated as on classic
Unix.
From linux-2.4.16/fs/ext2/namei.c:ext2_symlink:
if (l sizeof (inode-u.ext2_i.i_data)) {
/* slow symlink */
inode-i_op =
page_symlink_inode_operations;
inode-i_mapping-a_ops = ext2_aops;
err = block_symlink(inode, symname, l);
if (err)
goto out_fail;
} else {
/* fast symlink */
inode-i_op =
ext2_fast_symlink_inode_operations;
memcpy((char*)inode-u.ext2_i.i_data,symname,l);
inode-i_size = l-1;
}
So since most ext2 filesystems have more inodes allocated than they
can use and most symlinks are short, symlinks are usually no more
likely to make you run low on space than hardlinks.
The big problem in rsync is that using -H doubles the amount of space
used to store the file list:
hlink.c:57
if (!(hlink_list =
(struct file_struct
*)malloc(sizeof(hlink_list[0])*flist-count)))
out_of_memory(init_hard_links);
There are certainly better ways to do this. To start with, hlink.c
only needs to even *think* about non-directories which have nlinks1.
--
Martin
Re: Newbe....
Joe: I'm guessing you're doing the external transport thing: 'rsync -options /path/to/whatever remotehost:/path/to/where/you/want/whatever' In that form, it relies on host-based authentication... rsh, rlogin sort of thing. if you can't 'rsh remotehost rsync --version', you can't use the rsh external transport. If you have ssh available, you can add '-e ssh', which will have rsync use ssh instead of rsh. If you've set up a rsyncd server, make sure you're using the '::' syntax, else your client will try rsh again. man rsh man hosts.equiv man rhosts (may be same as hosts.equiv... ymmv) man ssh man rsyncd.conf man rsync? Tim Conway [EMAIL PROTECTED] 303.682.4917 Philips Semiconductor - Longmont TC 1880 Industrial Circle, Suite D Longmont, CO 80501 Available via SameTime Connect within Philips, n9hmg on AIM perl -e 'print pack(, 19061,29556,8289,28271,29800,25970,8304,25970,27680,26721,25451,25970), .\n ' There are some who call me Tim? [EMAIL PROTECTED] Sent by: [EMAIL PROTECTED] 12/14/2001 06:31 AM To: [EMAIL PROTECTED] cc: (bcc: Tim Conway/LMT/SC/PHILIPS) Subject:Newbe Classification: I'm just starting out with rsync and having the following problems: I'm getting autorization errors when I try to fire up a transfer between two e250's. I would like to basically mirror users between two systems. I've used the basic format that has been stipulated in the doc's still having problems Joe Despres [EMAIL PROTECTED]
Re: () .
Anybody else getting sick of our list getting spammed? This [EMAIL PROTECTED] hit us before, I think. Tim Conway [EMAIL PROTECTED] 303.682.4917 Philips Semiconductor - Longmont TC 1880 Industrial Circle, Suite D Longmont, CO 80501 Available via SameTime Connect within Philips, n9hmg on AIM perl -e 'print pack(, 19061,29556,8289,28271,29800,25970,8304,25970,27680,26721,25451,25970), .\n ' There are some who call me Tim?
Re: Is it possible to copy only files not existing in the destination?
cd destdir find . -type f -print /tmp/excludelist rsync -a --exclude-from=/tmp/excludelist srcserver:/srcdir/ . Tim Conway [EMAIL PROTECTED] 303.682.4917 Philips Semiconductor - Longmont TC 1880 Industrial Circle, Suite D Longmont, CO 80501 Available via SameTime Connect within Philips, n9hmg on AIM perl -e 'print pack(, 19061,29556,8289,28271,29800,25970,8304,25970,27680,26721,25451,25970), .\n ' There are some who call me Tim? John Evoy [EMAIL PROTECTED] Sent by: [EMAIL PROTECTED] 12/16/2001 11:34 AM To: [EMAIL PROTECTED] cc: (bcc: Tim Conway/LMT/SC/PHILIPS) Subject:Is it possible to copy only files not existing in the destination? Classification: Hi all rsync gurus Is it possible to copy only files that do not exist in the destination but NOT to update existing files even if their date is older than source files' date. The background of the question is that we have an environment (Lotus Domino) that makes it difficult to create new files but after they are created they are easy to keep in sync. J _ MSN Photos is the easiest way to share and print your photos: http://photos.msn.com/support/worldwide.aspx
Re: return status of rsync
instead of *, which may not match anything, and thus name something that doesn't exist, try using ., which is the current directory. This isn't dos, and * has no special meaning to rsync. it is, rather, globbed by your shell, being replaced by whatever it matches, or returned as a literal *. Tim Conway [EMAIL PROTECTED] 303.682.4917 Philips Semiconductor - Longmont TC 1880 Industrial Circle, Suite D Longmont, CO 80501 Available via SameTime Connect within Philips, n9hmg on AIM perl -e 'print pack(, 19061,29556,8289,28271,29800,25970,8304,25970,27680,26721,25451,25970), .\n ' There are some who call me Tim? Kapoor, Nishikant X [EMAIL PROTECTED] Sent by: [EMAIL PROTECTED] 12/17/2001 08:29 AM To: [EMAIL PROTECTED] cc: (bcc: Tim Conway/LMT/SC/PHILIPS) Subject:return status of rsync Classification: Hello, rsync returns zero if it is successful but a non-zero (2816) if it can not find any file(s) to move across. Just curious if that should still return zero ? nkapoor rsync -av * nk01::incoming building file list ... * : No such file or directory done wrote 73 bytes read 337 bytes 820.00 bytes/sec total size is 0 speedup is 0.00 I am trying to use this return status to determine if the rsync operation was successful. But the source may or may not have any files to send. Is there a better way to do this ? Thanks Nishi
Re: RSYNC: Backup Solution thoughts...
readonly full disk. rsync yourhost::rootmodule/etc/shadow . satan -f ./shadow telnet yourhost login as user su - f*** you over. what, no telnet, only ssh? grab an identity file and ssh in. yes, it's bad... at least, exclude secure areas. Tim Conway [EMAIL PROTECTED] 303.682.4917 Philips Semiconductor - Longmont TC 1880 Industrial Circle, Suite D Longmont, CO 80501 Available via SameTime Connect within Philips, n9hmg on AIM perl -e 'print pack(, 19061,29556,8289,28271,29800,25970,8304,25970,27680,26721,25451,25970), .\n ' There are some who call me Tim? Philip Mak [EMAIL PROTECTED] Sent by: [EMAIL PROTECTED] 12/18/2001 11:51 AM To: Lachlan Cranswick [EMAIL PROTECTED] cc: [EMAIL PROTECTED] (bcc: Tim Conway/LMT/SC/PHILIPS) Subject:Re: RSYNC: Backup Solution thoughts... Classification: On Tue, 18 Dec 2001, Lachlan Cranswick wrote: My question is: How can I make server A give READ-ONLY rsync access of the entire disk to server B? Something similar the following in the /etc/rsyncd.conf file in the server A might work(?) - just put a read only = yes in the config. (plus retricting the hosts allow) I guess the path would be / path = / read only = yes uid = nobody gid = nobody Wouldn't I need to put uid = root in order to give access to the entire file system (but read only)? Is this safe?
Re: rsync doesn't accept commands
Tim: I'm ccing the list so they'll know what's going on, save duplicated effort, suggest other solutions, etc. ... private mail and all, but public forum... anyway, here's the deal i see there. you're telling rsync to use a password file, used for contacting a rsync server. Then, you use the syntax to use external transport. you need a double colon between the server and the module name. I assume, of course, that you have the server configured and running or started by inetd. I don't get an error using that wrong syntax - the --password-file option is just ignored, but i'm probably running a different version. It's possible something funny happenned in options processing. rsync uses something called popt. I don't know what that means, but it works for me. does rsync --version work? If not, you're just plain not parsing commands. perhaps you'll see something if you trace the execution. strace in linux, i believe. Tim Conway [EMAIL PROTECTED] 303.682.4917 Philips Semiconductor - Longmont TC 1880 Industrial Circle, Suite D Longmont, CO 80501 Available via SameTime Connect within Philips, n9hmg on AIM perl -e 'print pack(, 19061,29556,8289,28271,29800,25970,8304,25970,27680,26721,25451,25970), .\n ' There are some who call me Tim? Tim Shubitz [EMAIL PROTECTED] 12/18/2001 04:25 PM To: Tim Conway/LMT/SC/PHILIPS@AMEC cc: Subject:Re: rsync doesn't accept commands Classification: Tim, Here's what I was typing in on the client end: rsync -ra --password-file=/path/to/password.pass /path/to/folder/rsynced login@rsyncserver:rsyncmodule and I'd end up with rsync spitting out what would appear if you just typed rsync. I've tried putting in the full path of rsync (/usr/bin/rsync) to see if that would help in any way. No luck so far. I've tried outputting errors to a file but get the same thing. No matter what I try rsync only runs itself without acknowledging commands. This is happening on a LinuxPPC box version 2.2.15. Both ends are running the same version (2.3.1). Doesn't really matter since the software doesn't get to the point where it could talk to another rsync process. Also, it won't work locally, same problem. Thanks for the help. Have a wonderful evening. Tim which rsync make sure you're calling the one you think. it might be a wrapper script or something. failing that, can you show me an example? Tim Conway [EMAIL PROTECTED] 303.682.4917 Philips Semiconductor - Longmont TC 1880 Industrial Circle, Suite D Longmont, CO 80501 Available via SameTime Connect within Philips, n9hmg on AIM perl -e 'print pack(, 19061,29556,8289,28271,29800,25970,8304,25970,27680,26721,25451,25970), .\n ' There are some who call me Tim?
Re: RSYNC: Backup Solution thoughts...
First: yes, wide open. Second: Sure, if your network is secure. I initially didn't understand that you were going to limit access. Anyway, trusted host access is a vulnerability. You know your system and situation. Tim Conway [EMAIL PROTECTED] 303.682.4917 Philips Semiconductor - Longmont TC 1880 Industrial Circle, Suite D Longmont, CO 80501 Available via SameTime Connect within Philips, n9hmg on AIM perl -e 'print pack(, 19061,29556,8289,28271,29800,25970,8304,25970,27680,26721,25451,25970), .\n ' There are some who call me Tim? Philip Mak [EMAIL PROTECTED] 12/18/2001 04:50 PM To: Tim Conway/LMT/SC/PHILIPS@AMEC cc: [EMAIL PROTECTED] Subject:Re: RSYNC: Backup Solution thoughts... Classification: If I understand correctly, you're saying that if someone manages to gain access to my rsync server, they can read my password files and private keys, right? How would they accomplish gaining access to my rsync server, though? Only my backup server would be authorized to connect to it... I think I *do* want to back up /etc/shadow actually, because this is a full system backup that's meant to be able to be used to restore the system in case of hard drive failure. If I didn't include /etc/shadow, then everyone would lose their passwords. On Tue, 18 Dec 2001 [EMAIL PROTECTED] wrote: readonly full disk. rsync yourhost::rootmodule/etc/shadow . satan -f ./shadow telnet yourhost login as user su - f*** you over. what, no telnet, only ssh? grab an identity file and ssh in. yes, it's bad... at least, exclude secure areas.
Re: Question on encryption
You've hit on just what rsyncd is best for. It's not particularly secure, as it uses plaintext (default null) passwords, so someone working for your ISP could sniff your password. the modules (sections of the filesystem tree you expose via rsyncd) are, by default, accessed in a chroot() environment, so even if someone gets in, all they see is what they would see sniffing your transfer, though it's more convenient. By default, the connection is readonly, so the only trouble they could cause you would network and disk load from reading your modules. Tridge wrote rsync specifically to mirror publically-available directory trees. Tim Conway [EMAIL PROTECTED] 303.682.4917 Philips Semiconductor - Longmont TC 1880 Industrial Circle, Suite D Longmont, CO 80501 Available via SameTime Connect within Philips, n9hmg on AIM perl -e 'print pack(, 19061,29556,8289,28271,29800,25970,8304,25970,27680,26721,25451,25970), .\n ' There are some who call me Tim? George Sinclair [EMAIL PROTECTED] Sent by: [EMAIL PROTECTED] 12/20/2001 02:21 PM Please respond to george.sinclair To: [EMAIL PROTECTED] cc: (bcc: Tim Conway/LMT/SC/PHILIPS) Subject:Question on encryption Classification: I am not currently subscribed so please email me below. First, my only experience with rsync has been older versions (e.g. 1.7.x) which did not allow daemon mode, so please bear with me. My problem is that I would like to mirror large quantities of data on a remote machine to a local one, but I don't care about encrypting the data itself. I only care about securing the connection from the authentication point of view. Okay, I must admit that I like SSH because it protects the end user from a variety of attacks, but what I'm saying is that I don't care if people look at the data. Typically, this mirroring has been done using just the local client's rsync program in conjunction with ssh (both on the client and sshd on the remote machine): rsync --delete --rsh ssh --rsync-path /path_to_rsync/rsync -rlpt sourcedir remote_host/target_dir/ The problem, however, is that due to the large size of the data, and the slowness typically suffered under encryption, the remote machine crawls to a halt or is seriously impaired. Working with small numbers of files or infrequent mirrors, the encryption is not a problem, but it gets to be a burden when you're doing this every night on a lot of data. Some have suggested using something like 'blowfish -c' instead of the default to speed things up by perhaps a factor of 3. Anyway, here is my question: Does running rsync in daemon mode on the remote host preclude the need to use SSH from the client? If so, how secure is this versus using rsync in non daemon mode with SSH? I have considered building SSH to not use encryption, but I was thinking rsync in daemon mode might obviate the need to have to use SSH if it can still be made secure. Thanks. George Sinclair | [EMAIL PROTECTED]
Re: --owner --group without root access?
rsync makes exact copies of filesystems. It's a mirroring tool, not a backup tool. It stores the information as a filesystem, and if it's not allowed to save group and user id on the filesystem, it doesn't. Perhaps you need an archiving system? maybe doing incremental backups? That said, if you need to record who owns what, how about using the --write-batch/--read-batch (rsync+) features? rsync it once just getting the files and their permissions. Then, do it again with --write-batch, which will create a file containing instructions on how to fix the ownerships. then, to restore, you can just rsync the files back, then apply the ownerships file with --read-batch. Of course, the batch file will contain other changes made between the sync and the batch, but at a quiescent time, it should be minimal, and you'd probably want the changes anyway. Tim Conway [EMAIL PROTECTED] 303.682.4917 Philips Semiconductor - Longmont TC 1880 Industrial Circle, Suite D Longmont, CO 80501 Available via SameTime Connect within Philips, n9hmg on AIM perl -e 'print pack(, 19061,29556,8289,28271,29800,25970,8304,25970,27680,26721,25451,25970), .\n ' There are some who call me Tim? Philip Mak [EMAIL PROTECTED] Sent by: [EMAIL PROTECTED] 12/30/2001 02:24 AM To: [EMAIL PROTECTED] cc: (bcc: Tim Conway/LMT/SC/PHILIPS) Subject:--owner --group without root access? Classification: Is there a way to preserve the owner and group permissions without having root access? Well, this is not possible on the filesystem level of course, but what about storing the owner/group information in a supplementary file that can be read by rsync to later reconstruct this information? I'm using rsync to perform a server-to-server backup of a machine's hard drive. If the hard drive being backed up were to actually fail, I would want to be able to restore all the files with their exact ownership information. However, I think that needing to have root access on the backup server should not be necessary to do this...
Re: reverse delete?
#!/bin/sh for file in `rsh remote 'cd ~/Maildir;find . -type f -print'` do [ -f ~/Maildir/$file ] rm ~/Maildir/$file done Tim Conway [EMAIL PROTECTED] 303.682.4917 Philips Semiconductor - Longmont TC 1880 Industrial Circle, Suite D Longmont, CO 80501 Available via SameTime Connect within Philips, n9hmg on AIM perl -e 'print pack(, 19061,29556,8289,28271,29800,25970,8304,25970,27680,26721,25451,25970), .\n ' There are some who call me Tim? Graham Guttocks [EMAIL PROTECTED] Sent by: [EMAIL PROTECTED] 01/01/2002 01:57 PM To: [EMAIL PROTECTED] cc: (bcc: Tim Conway/LMT/SC/PHILIPS) Subject:reverse delete? Classification: Greetings, I'm looking for an option that deletes from the receiving side any files contained on the sending side. For example, If local:~/Maildir/ contains: 123.txt 456.txt And remote:~/Maildir/ contains: 123.txt 456.txt 789.txt Running rsync local:~/Maildir/ remote:~/Maildir/ with the appropriate options would leave remote:~/Maildir/ with: 789.txt and local:~/Maildir/ would remain unchanged with: 123.txt 456.txt Regards, Graham __ Do You Yahoo!? Send your FREE holiday greetings online! http://greetings.yahoo.com
Re: rsync -e ssh --bwlimit=20 ?
My results: +++ Tools@willy /users/Tools/tccrsync --bwlimit=20 sim/dist/sdonetsvr/sim.lic.010703 atlas:/tmp Tools@willy /users/Tools/tccrsync --version rsync version 2.5.1pre3 protocol version 25 Copyright (C) 1996-2001 by Andrew Tridgell and others http://rsync.samba.org/ Capabilities: 64-bit files, socketpairs, hard links, symlinks, batchfiles, no IPv6 Tools@willy /users/Tools/tccuname -a SunOS willy 5.7 Generic_106541-07 sun4u sparc SUNW,Ultra-4 Tools@willy /users/Tools/tcc +++ the second command is the crux. How about a rsync --version;rsh from.com rsync --version I'm guessing we'll see one or both with a version that doesn't support bwlimit. Tim Conway [EMAIL PROTECTED] 303.682.4917 Philips Semiconductor - Longmont TC 1880 Industrial Circle, Suite D Longmont, CO 80501 Available via SameTime Connect within Philips, n9hmg on AIM perl -e 'print pack(, 19061,29556,8289,28271,29800,25970,8304,25970,27680,26721,25451,25970), .\n ' There are some who call me Tim? J Robinson [EMAIL PROTECTED] Sent by: [EMAIL PROTECTED] 01/03/2002 03:54 PM To: [EMAIL PROTECTED] cc: (bcc: Tim Conway/LMT/SC/PHILIPS) Subject:rsync -e ssh --bwlimit=20 ? Classification: Hello Everyone: I've been trying to get rsync using --bwlimit using openssh for transport without success. Here's what I get: % rsync --bwlimit=20 from.com:/dir/file.txt /dest [EMAIL PROTECTED]'s password: rsync: unrecognized option `--bwlimit=20' unexpected EOF in read_timeout I tried specifying ssh transport using both the -e option and by setting the RSYNC_RSH evironment variable-- it made no difference. rsync works as expected without the --bwlimit option. I know --bwlimit is a fairly new feature. and I researched this as much as I could without going into the source code... anyone able to get this to work, or have any tips? Thanks jrobinson __ Do You Yahoo!? Send your FREE holiday greetings online! http://greetings.yahoo.com
Re: Error Message!!
Almost certainly, something is already using port 873 - probably a previous invocation of rsync --daemon, or you have inetd listening to it already. On a commandline invocation, rsync forks, so invoking it twice in a row without killing the first one or having the first one die, makes the second invocation fail, unless you specify different ports for each one. In this example, rsync is run from inetd, though not active at the moment. Tools@willy /cadappl/encap/packagesnetstat -an |grep 873 *.873*.*0 0 0 0 LISTEN Tools@willy /cadappl/encap/packagesps -ef |grep rsync Tools@willy /cadappl/encap/packagesgrep rsync /etc/services /etc/inetd.conf /etc/inetd.conf:rsync stream tcp nowait root/usr/bin/rsync rsyncd --daemon /etc/services:rsync 873/tcp rsyncd # rsync daemon Good luck. Tim Conway [EMAIL PROTECTED] 303.682.4917 Philips Semiconductor - Longmont TC 1880 Industrial Circle, Suite D Longmont, CO 80501 Available via SameTime Connect within Philips, n9hmg on AIM perl -e 'print pack(, 19061,29556,8289,28271,29800,25970,8304,25970,27680,26721,25451,25970), .\n ' There are some who call me Tim? David Nickel Jr. [EMAIL PROTECTED] Sent by: [EMAIL PROTECTED] 01/11/2002 08:49 AM To: Rsync Mailing List [EMAIL PROTECTED] cc: (bcc: Tim Conway/LMT/SC/PHILIPS) Subject:Error Message!! Classification: When rsync starts I get the following messages in my log file Jan 11 11:00:04 WEB1 rsyncd[31281]: rsync: bind failed on port 873 Jan 11 11:00:04 WEB1 rsyncd[31281]: rsync error: error in socket IO (code 10) at socket.c(361) Anybody know what might be causing this? David
RE: Error Message!!
Well, there you go. rsyncd is already set up on your system. You don't run rsync, inetd does. The way it's set up makes it nice for testing, too, as you just edit the conf file and test it. If this is, in fact, another installation of rsyncd, which you shouldn't mess with, you'll have to call rsync --daemon with --port=something and --config-file=something to use your alternate setup. If, on the other hand, this is your installation, but you wish to manually start rsync, you'll have to comment out (or delete) the rsync line in /etc/inetd.conf and HUP inetd. What you've go, however, is better, in my opinion. The doc you reference describes a directly-run rsyncd. If you did your install from that, you wouldn't have done the inetd.conf setup, so it looks like you've got a previous setup on your system. If it's your system now, just edit /etc/rsyncd.conf as needed. changes are instantaneous, affecting subsequent calls to the daemon. test = rsync localhost:: should list your modules. Tools@willy /users/Toolsgrep \\[ /etc/rsyncd.conf [big] [big1] [ToolSyncModules] Tools@willy /users/Toolsrsync localhost:: big big1 ToolSyncModules Tools@willy /users/Tools This way, rsyncd also doesn't take any resources until it's invoked. Tim Conway [EMAIL PROTECTED] 303.682.4917 Philips Semiconductor - Longmont TC 1880 Industrial Circle, Suite D Longmont, CO 80501 Available via SameTime Connect within Philips, n9hmg on AIM perl -e 'print pack(, 19061,29556,8289,28271,29800,25970,8304,25970,27680,26721,25451,25970), .\n ' There are some who call me Tim? David Nickel Jr. [EMAIL PROTECTED] 01/11/2002 09:41 AM To: Tim Conway/LMT/SC/PHILIPS@AMEC cc: Subject:RE: Error Message!! Classification: I am falling this article http://www.freebsddiary.org/rsync.php on installing and configuring rysnc to mirror the web directories of my two web servers. here is the output netstat -an |grep 873 tcp4 0 0 *.873 *.* LISTEN tcp46 0 0 *.873 *.*LISTEN grep 873 /etc/services rsync 873/tcp # Rsync Daemon grep rsync /etc/inetd.conf rsync stream tcp nowait root/usr/local/bin/rsync rsyncd --daemon -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]] Sent: Friday, January 11, 2002 11:32 AM To: David Nickel Jr. Subject: RE: Error Message!! That's why i included the part about indetd.conf and services. Most likely, your inetd is configured to handle port 873. The fact that you have two listeners on 873 tell me, however, that it may not be for rsync, as rsync is a TCP-only application, and only one thing can have a port/protocol listener open, so the other port 873 you see is probably something listening on UDP, else, you're seeing a listener and a connection. lsof and some versions of fuser can be used to determine what is using a port. poke around the man pages to find it. If those aren't available, send me the output of netstat -an |grep 873;grep 873 /etc/services and if there's something found in services, grep first word on the line from services /etc/inetd.conf Tim Conway [EMAIL PROTECTED] 303.682.4917 Philips Semiconductor - Longmont TC 1880 Industrial Circle, Suite D Longmont, CO 80501 Available via SameTime Connect within Philips, n9hmg on AIM perl -e 'print pack(, 19061,29556,8289,28271,29800,25970,8304,25970,27680,26721,25451,25970), .\n ' There are some who call me Tim? David Nickel Jr. [EMAIL PROTECTED] 01/11/2002 09:19 AM To: Tim Conway/LMT/SC/PHILIPS@AMEC cc: Subject:RE: Error Message!! Classification: Thanks for the info. I used netstat there are two occurances of port 873 listening... how do I kill them. When I grep ps I rsync is not running.. David -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]] Sent: Friday, January 11, 2002 11:16 AM To: David Nickel Jr. Cc: Rsync Mailing List Subject: Re: Error Message!! Almost certainly, something is already using port 873 - probably a previous invocation of rsync --daemon, or you have inetd listening to it already. On a commandline invocation, rsync forks, so invoking it twice in a row without killing the first one or having the first one die, makes the second invocation fail, unless you specify different ports for each one. In this example, rsync is run from inetd, though not active at the moment. Tools@willy /cadappl/encap/packagesnetstat -an |grep 873 *.873*.*0 0 0 0 LISTEN Tools@willy /cadappl/encap/packagesps -ef |grep rsync Tools@willy /cadappl/encap/packagesgrep rsync /etc/services /etc/inetd.conf /etc/inetd.conf:rsync stream tcp nowait root/usr/bin/rsync rsyncd --daemon /etc/services:rsync 873
Re: problem with --delete if I sync part of a dir
check this out: echo rsync -azv --stats -e ssh --delete i_* user@host:/dir/. it expands to sync -azv --stats -e ssh --delete all existing i_* files user@host:/dir/ It won't mention the ones that are gone. rsync the directory/. including i_* and excluding * (the order matters). rsync -azv --stats -e ssh --delete ./. --include=i_* --exclude=* user@host:/dir/ Tim Conway [EMAIL PROTECTED] 303.682.4917 Philips Semiconductor - Longmont TC 1880 Industrial Circle, Suite D Longmont, CO 80501 Available via SameTime Connect within Philips, n9hmg on AIM perl -e 'print pack(, 19061,29556,8289,28271,29800,25970,8304,25970,27680,26721,25451,25970), .\n ' There are some who call me Tim? Palmans Pepijn [EMAIL PROTECTED] Sent by: [EMAIL PROTECTED] 01/15/2002 02:01 AM To: [EMAIL PROTECTED] cc: (bcc: Tim Conway/LMT/SC/PHILIPS) Subject:problem with --delete if I sync part of a dir Classification: Hi, I'm using rsync for a couple of things but I am at a point where I could not find a solution for my problem. this is what I want to do: rsync only the files i_* from a directory from machine A to machine Bi and delete them from machine B if they on longer exist on machine A. so I use the command: rsync -azv --stats -e ssh --delete i_* user@host:/dir/ When using this command the files starting with i_ will get placed on the remote server or even updated when I change them but if I delete a file on machine A it will not be deleted on machine B. This is only a problem if I use i_* as my source. If i sync whole directories (recursive or not) it's no problem at all to get files deleted on the other end. Is there anything to do about this ? Regards, Pepijn Palmans Kangaroot Linux Solutions Grote Steenweg 91 2600 Berchem, Antwerpen, Belgium Email: [EMAIL PROTECTED]
Re: Where can I download the latest and greatest rsync Win32 binary?
From www.cygwin.com (did you even look there?). ftp://mirrors.rcn.net/pub/sourceware/cygwin/contrib/rsync/rsync-2.5.1-2.tar.bz2 Tim Conway [EMAIL PROTECTED] 303.682.4917 Philips Semiconductor - Longmont TC 1880 Industrial Circle, Suite D Longmont, CO 80501 Available via SameTime Connect within Philips, n9hmg on AIM perl -e 'print pack(, 19061,29556,8289,28271,29800,25970,8304,25970,27680,26721,25451,25970), .\n ' There are some who call me Tim? Carlos Gutierrez [EMAIL PROTECTED] Sent by: [EMAIL PROTECTED] 01/14/2002 11:26 AM To: [EMAIL PROTECTED] cc: (bcc: Tim Conway/LMT/SC/PHILIPS) Subject:Where can I download the latest and greatest rsync Win32 binary? Classification: Is there a site doing current builds of Win32 binaries? Also, is there any project working in a non cygwin port of rsync? Thanks!
Re: SSH Error
Ok, you're talking about ssh, and you're talking about a rsyncd. Except in rare cases, you don't use ssh to access rsyncd. I'm guessing you're doing something like rsync -e ssh remotehost:www localdestinationdirectory (-e ssh isn't needed if $RSYNC_SSH=ssh). drop the -e ssh and use TWO colons between the servername and the modulename. Tim Conway [EMAIL PROTECTED] 303.682.4917 Philips Semiconductor - Longmont TC 1880 Industrial Circle, Suite D Longmont, CO 80501 Available via SameTime Connect within Philips, n9hmg on AIM perl -e 'print pack(, 19061,29556,8289,28271,29800,25970,8304,25970,27680,26721,25451,25970), .\n ' There are some who call me Tim? David L Nickel [EMAIL PROTECTED] Sent by: [EMAIL PROTECTED] 01/15/2002 02:23 PM To: [EMAIL PROTECTED] cc: (bcc: Tim Conway/LMT/SC/PHILIPS) Subject:SSH Error Classification: I created a user on the machine I am attempting to mirror. I when I run rsync with this user I get the following errors receiving file list ... link_stat www : No such file or directory done client: nothing to do: perhaps you need to specify some filenames or the --recursive option? rsync error: partial transfer (code 23) at main.c(553) here is my conf file ## use chroot = no max connections = 5 syslog facility = local3 [www] path = /www comment = Web Directory hosts allow = *.*.1.118 read only = yes list = yes uid = root gid = wheel auth users = user ##
Re: Where can I download the latest and greatest rsync Win32 binary?
It sounded like he had a working installation already, and just needed to add rsync to it. I did mine by compiling it in (though i wonder if the one you supply might not have that annoying conection reset by peer error or whatever it was). I tried rerunning setup once, just checking the packages i wanted to add, and it left me with a minimal installation, wiping out everything i didn't check in the setup. I figured grabbing the tar.bz2 and untarring it in / would be more efficient for him. Tim Conway [EMAIL PROTECTED] 303.682.4917 Philips Semiconductor - Longmont TC 1880 Industrial Circle, Suite D Longmont, CO 80501 Available via SameTime Connect within Philips, n9hmg on AIM perl -e 'print pack(, 19061,29556,8289,28271,29800,25970,8304,25970,27680,26721,25451,25970), .\n ' There are some who call me Tim? David Starks-Browning [EMAIL PROTECTED] Sent by: [EMAIL PROTECTED] 01/15/2002 03:29 PM To: [EMAIL PROTECTED] cc: Carlos Gutierrez [EMAIL PROTECTED] (bcc: Tim Conway/LMT/SC/PHILIPS) Subject:Re: Where can I download the latest and greatest rsync Win32 binary? Classification: On Tuesday 15 Jan 02, [EMAIL PROTECTED] writes: From www.cygwin.com (did you even look there?). ftp://mirrors.rcn.net/pub/sourceware/cygwin/contrib/rsync/rsync-2.5.1-2.tar.bz2 Tim, It's best not to provide a specific cygwin package URL like that. If anyone wants to download the Cygwin version of rsync, they should just visit http://cygwin.com/ (like you said) and use the Cygwin-supplied setup.exe to install Cygwin and rsync. One will probably have to explicitly enable the rsync package installation, because I don't think it comes by default. The Cygwin setup.exe program does a few crucial things like define a mount table in the Windows Registry. If you just downloaded and unpacked the bz2 file, without installing Cygwin with setup.exe, I don't think rsync would work. In response to Carlos' 2nd question: I've been watching this list for over a year and I've never seen a non-Cygwin Windows port of rsync advertised. Since it works under Cygwin, why would one bother? Cheers, David (Cygwin FAQ maintainer) Carlos Gutierrez [EMAIL PROTECTED] Sent by: [EMAIL PROTECTED] 01/14/2002 11:26 AM To: [EMAIL PROTECTED] cc: (bcc: Tim Conway/LMT/SC/PHILIPS) Subject:Where can I download the latest and greatest rsync Win32 binary? Classification: Is there a site doing current builds of Win32 binaries? Also, is there any project working in a non cygwin port of rsync? Thanks!
Re: error in rsync protocol data stream (code 12) at io.c(139)
As you see in the response, ksh can't find rsync (ksh: rsync: not found), and this is included in the error stream of rsync (rsync: connection unexpectedly closed (0 bytes read so far)). This means rsync is trying to run rsync, which means that you're using external transport (single-colon (:) syntax). If by trying to connect to a rsync server you mean connect to a server using rsync over external transport, you just need to make it possible for your local rsync to find the rsync binary on that server. There are several ways to do that. Change the global path on the remote system to include the location where the rsync binary is kept, copy or link said binary somewhere into the path or put a wrapper script named rsync in the path which just invokes the real one with the same args (a symlink is better), or, least intrusively, use '--rsync-path='. If, in fact, you do have a rsync server running on the remote host, and you want to contact that, use double colons '::' to seperate the hostname from the module name. The rsync manpage covers these, and other rsync-related topics in exquisite detail. Tim Conway [EMAIL PROTECTED] 303.682.4917 Philips Semiconductor - Longmont TC 1880 Industrial Circle, Suite D Longmont, CO 80501 Available via SameTime Connect within Philips, n9hmg on AIM perl -e 'print pack(, 19061,29556,8289,28271,29800,25970,8304,25970,27680,26721,25451,25970), .\n ' There are some who call me Tim? Victor J. Hernandez Gomez [EMAIL PROTECTED] Sent by: [EMAIL PROTECTED] 01/22/2002 10:21 AM To: [EMAIL PROTECTED] cc: (bcc: Tim Conway/LMT/SC/PHILIPS) Subject:error in rsync protocol data stream (code 12) at io.c(139) Classification: When trying to connect to a rsync server (sparc.solaris.2.6) from a linux machine (redhat 6.2). # rsync machine.upo.es ksh: rsync: not found rsync: connection unexpectedly closed (0 bytes read so far) rsync error: error in rsync protocol data stream (code 12) at io.c(139) Both machines have 2.5.1 version of rsync code, compiled from scratch. rsh is working properly when rsync is not in scene. Thank you in advance for your help, -- Víctor Hernández Centro de Informática y Comunicaciones Universidad Pablo de Olavide, de Sevilla
Re: Incredible Information
Three copies of the spam? I'm not saying we should do anything illegal, but it would be interesting to see what would happen if members of the list who are in countries where they can get away with it were to destroy and deface http://hop.clickbank.net with messages indicating that it was done in retaliation for spamming. Mind, I'm not saying it should be done. I'm just idly speculating. :-) Tim Conway [EMAIL PROTECTED] 303.682.4917 Philips Semiconductor - Longmont TC 1880 Industrial Circle, Suite D Longmont, CO 80501 Available via SameTime Connect within Philips, n9hmg on AIM perl -e 'print pack(, 19061,29556,8289,28271,29800,25970,8304,25970,27680,26721,25451,25970), .\n ' There are some who call me Tim?
Re: suid files and bsd
rsync tries to make the files the same. If you include mode, that's that. Perhaps you could modify generator.c for your own copy, to mask off the suid and sgid bits, so the list would make them all look non-suid, source and dest, and thus not try to set them, and not care that they're not set. I'm not sure it's generator.c, but i'm sure you can find it. Tim Conway [EMAIL PROTECTED] 303.682.4917 Philips Semiconductor - Longmont TC 1880 Industrial Circle, Suite D Longmont, CO 80501 Available via SameTime Connect within Philips, n9hmg on AIM perl -e 'print pack(, 19061,29556,8289,28271,29800,25970,8304,25970,27680,26721,25451,25970), .\n ' There are some who call me Tim? Steve [EMAIL PROTECTED] Sent by: [EMAIL PROTECTED] 01/24/2002 10:27 AM To: [EMAIL PROTECTED] cc: (bcc: Tim Conway/LMT/SC/PHILIPS) Subject:suid files and bsd Classification: Currently I only have one machine backing up to another. I am backing up to a filesystem that is mounted as nosuid. The syncing process works perfectly except it fails on files that are suid. Example proc/ root/ tmp/ mknod tmp/mysql.sock : Invalid argument mknod tmp/orbit-daud/orb-12573780511489051058 : Invalid argument mknod tmp/orbit-daud/orb-139425434385535813 : Invalid argument mknod var/pwcheck/pwcheck : Invalid argument var/run/ mknod var/run/log : Invalid argument mknod var/run/ndc : Invalid argument mknod var/run/printer : Invalid argument srw-rw-rw- 1 rootwheel 0 Jan 23 11:52 log srw--- 1 rootwheel 0 Jan 23 11:53 ndc srwxrwx--- 1 rootwheel 0 Jan 23 11:53 printer I would figure that copying the file and setting the file typr would be different. Is this the rsyncd that is seeing the file system set nosuid or has anyone else run into this sort of problem? I would really rather not have suid files allowed on the backup filesystem. Regards, Steve -- Systems Administrator | D i m e n s i o n a l C o m m u n i c a t i o n s | [EMAIL PROTECTED]| DSL/V90/K56flex/V34/ISDN/Frame/T1/T3 starts @$25/mo | 303.285.INET voice | http://www.dimensional.com/ [EMAIL PROTECTED] | 888.3.DIMCOM tollfree | fast reliable internet access for home and business | 303.609.9373 emergency | Denver * Boulder * Longmont * Bailey * CO Springs |
Re: Rsync
No. Rsync is unidirectional. If you try to do it both ways, first, it's going to delete anything on hostb that isn't on hosta, then make all the files on hostb just like hosta. Then, if you try to go back the other way, all the changes on hostb are already gone. There is a product called unison, which is made to do what you're asking about. I don't know more about it than that it exists. A quick google should get you to it. Good luck. Tim Conway [EMAIL PROTECTED] 303.682.4917 Philips Semiconductor - Longmont TC 1880 Industrial Circle, Suite D Longmont, CO 80501 Available via SameTime Connect within Philips, n9hmg on AIM perl -e 'print pack(, 19061,29556,8289,28271,29800,25970,8304,25970,27680,26721,25451,25970), .\n ' There are some who call me Tim? Raj [EMAIL PROTECTED] 01/24/2002 09:21 AM To: Tim Conway/LMT/SC/PHILIPS@AMEC cc: [EMAIL PROTECTED] [EMAIL PROTECTED] Subject:Rsync Classification: Tim, Currently I have rsync setup on hosta which syncs to hostb. Rsync deletes whatever deleted from hosta and copy and sync from hosta to hostb. Does rsync catch changes made and sync hosta in case any changes are made to hostb? Please let me know. Thanks Raj = __ Do You Yahoo!? Great stuff seeking new owners in Yahoo! Auctions! http://auctions.yahoo.com
Re: suid files and bsd - correction
it's in syscall.c, not generator.c
You'll have to save the status of the lstat, modify the mode in st, and
return the lstat status. I don't know how to do it, though.
#if SUPPORT_LINKS
int do_lstat(const char *fname, STRUCT_STAT *st)
{
#if HAVE_OFF64_T
return lstat64(fname, st);
#else
return lstat(fname, st);
#endif
}
#endif
Tim Conway
[EMAIL PROTECTED]
303.682.4917
Philips Semiconductor - Longmont TC
1880 Industrial Circle, Suite D
Longmont, CO 80501
Available via SameTime Connect within Philips, n9hmg on AIM
perl -e 'print pack(,
19061,29556,8289,28271,29800,25970,8304,25970,27680,26721,25451,25970),
.\n '
There are some who call me Tim?
- Forwarded by Tim Conway/LMT/SC/PHILIPS on 01/24/2002 02:07 PM -
Tim Conway
01/24/2002 02:04 PM
To: Steve [EMAIL PROTECTED]
cc: [EMAIL PROTECTED]
[EMAIL PROTECTED]
Subject:Re: suid files and bsd
Classification: Unclassified
rsync tries to make the files the same. If you include mode, that's that.
Perhaps you could modify generator.c for your own copy, to mask off the
suid and sgid bits, so the list would make them all look non-suid, source
and dest, and thus not try to set them, and not care that they're not set.
I'm not sure it's generator.c, but i'm sure you can find it.
Tim Conway
[EMAIL PROTECTED]
303.682.4917
Philips Semiconductor - Longmont TC
1880 Industrial Circle, Suite D
Longmont, CO 80501
Available via SameTime Connect within Philips, n9hmg on AIM
perl -e 'print pack(,
19061,29556,8289,28271,29800,25970,8304,25970,27680,26721,25451,25970),
.\n '
There are some who call me Tim?
Steve [EMAIL PROTECTED]
Sent by: [EMAIL PROTECTED]
01/24/2002 10:27 AM
To: [EMAIL PROTECTED]
cc: (bcc: Tim Conway/LMT/SC/PHILIPS)
Subject:suid files and bsd
Classification:
Currently I only have one machine backing up to another.
I am
backing up to a filesystem that is mounted as nosuid. The syncing process
works perfectly except it fails on files that are suid.
Example
proc/
root/
tmp/
mknod tmp/mysql.sock : Invalid argument
mknod tmp/orbit-daud/orb-12573780511489051058 : Invalid argument
mknod tmp/orbit-daud/orb-139425434385535813 : Invalid argument
mknod var/pwcheck/pwcheck : Invalid argument
var/run/
mknod var/run/log : Invalid argument
mknod var/run/ndc : Invalid argument
mknod var/run/printer : Invalid argument
srw-rw-rw- 1 rootwheel 0 Jan 23 11:52 log
srw--- 1 rootwheel 0 Jan 23 11:53 ndc
srwxrwx--- 1 rootwheel 0 Jan 23 11:53 printer
I would figure that copying the file and setting the file typr would be
different. Is this the rsyncd that is seeing the file system set nosuid
or has anyone else run into this sort of problem? I would really rather
not have suid files allowed on the backup filesystem.
Regards,
Steve
--
Systems Administrator | D i m e n s i o n a l C o m m u n i c a t i o n
s |
[EMAIL PROTECTED]| DSL/V90/K56flex/V34/ISDN/Frame/T1/T3 starts
@$25/mo |
303.285.INET voice | http://www.dimensional.com/ [EMAIL PROTECTED]
|
888.3.DIMCOM tollfree | fast reliable internet access for home and
business |
303.609.9373 emergency | Denver * Boulder * Longmont * Bailey * CO
Springs |
Re: --compare-dest option
Here's an example. dest = directory tree full of files all interdependent. a change in one without the corresponding change in another makes the whole tree invalid. If you just rsync the directory changes over, the tree is invalid from the first change is started until the last one is done. To avoid this, you rsync to another directory in the same filesystem, using --compare-dest to point to the actual directory. Once the new copy is finished, you rename the original to a temporary name, rename the new copy to the originals name (these two operations are nearly instantaneous), then recursively delete the original directory. This way, there is no time at all where the directory is invalid, and only maybe a millisecond where it doesn't exist. Tim Conway [EMAIL PROTECTED] 303.682.4917 Philips Semiconductor - Longmont TC 1880 Industrial Circle, Suite D Longmont, CO 80501 Available via SameTime Connect within Philips, n9hmg on AIM perl -e 'print pack(, 19061,29556,8289,28271,29800,25970,8304,25970,27680,26721,25451,25970), .\n ' There are some who call me Tim? [EMAIL PROTECTED] Sent by: [EMAIL PROTECTED] 01/28/2002 06:02 AM To: [EMAIL PROTECTED] cc: (bcc: Tim Conway/LMT/SC/PHILIPS) Subject:--compare-dest option Classification: I'm translating rsync.1, and i'm having some problems with the --compare-dest option description, actually i dont understand the part which describes the usefulness of this option, it says : This is useful for doing transfers to a new destination while leaving existing files intact, and then doing a flash-cutover when all files have been successfully transferred (for example by moving directories around and removing the old directory, although this requires also doing the transfer with -I to avoid skipping files that haven´t changed). If someone could explain me this (especially the flash-cutover thing) , maybe with a little example, it would greatly help me for the translation. Thanks in advance Denis
Re: Doubt in Rsync !!
I just played back your mail in my head, and realized that you mentioned the rsync server. I read your command, from which it was plain that you were NOT trying to contact a rsync server, and gave instructions based on that. In case you were trying to contact a rsync server (rsyncd), I suggest you read the man pages for both rsync and rsyncd.conf. the rsync manpage explains how to invoke rsync to have it be a server, and the rsyncd.conf manpage explains how to set up the required configuration file. The rsync manpage also explains how to invoke rsync to CALL a server (as opposed to starting a temporary process via an external transport to act as your remote server, as your commandline showed). Direct consultation of the documentation which Tridge, Martin, Dave, and everybody else has put so much work into, can cover the broad possibilities with much less latency than an email list. Tim Conway [EMAIL PROTECTED] 303.682.4917 Philips Semiconductor - Longmont TC 1880 Industrial Circle, Suite D Longmont, CO 80501 Available via SameTime Connect within Philips, n9hmg on AIM perl -e 'print pack(, 19061,29556,8289,28271,29800,25970,8304,25970,27680,26721,25451,25970), .\n ' There are some who call me Tim? Tim Conway 02/04/2002 08:33 AM To: [EMAIL PROTECTED] cc: Lakshminarayanan Radhakrishnan [EMAIL PROTECTED] [EMAIL PROTECTED] [EMAIL PROTECTED] Subject:Re: Doubt in Rsync !! Classification: Unclassified Ok: You're using an external transport (rsh, unless you've defined RSYNC_RSH as something else(probably ssh)). First thing to check is whether you can rsh to destinationmachine. See what happens if you do rsh destinationmachine uname -a. Does this report back the information for destinationmachine, or does it give you Permission denied. If so, get rsh working. From your error, I'm certain this is the problem, as if it were a permission problem on a file or directory, you'd get an error more like this tconway@atlas /site/local/share/ToolSync/newsyncrsync newsync.log.2 atlas:/kernel mkstemp .newsync.log.2.GYaiMq failed rsync error: partial transfer (code 23) at main.c(537) tconway@atlas /site/local/share/ToolSync/newsync . Note: If this is an hpux, SCO, and maybe some other primitive systems, the name rsh is reserved for a restricted shell, and they use remsh instead as the command. Rsync already knows about this, but you need to do the right command for your test. Investigate /etc/hosts.equiv, /etc/hosts.deny, $HOME/.rhosts, your name resolution, etc.. Tim Conway [EMAIL PROTECTED] 303.682.4917 Philips Semiconductor - Longmont TC 1880 Industrial Circle, Suite D Longmont, CO 80501 Available via SameTime Connect within Philips, n9hmg on AIM perl -e 'print pack(, 19061,29556,8289,28271,29800,25970,8304,25970,27680,26721,25451,25970), .\n ' There are some who call me Tim? [EMAIL PROTECTED] Sent by: [EMAIL PROTECTED] 02/04/2002 02:14 AM To: [EMAIL PROTECTED] cc: Lakshminarayanan Radhakrishnan [EMAIL PROTECTED] (bcc: Tim Conway/LMT/SC/PHILIPS) Subject:Doubt in Rsync !! Classification: Sir, How to run the rsync server in the remote machine. I want to mirror the set of directories from one machine to another machine which are in the network. I used the following command : /usr/sbin/rsync -vv --delete --recursive --times --perms --update source directory destinationmachine:destination directory. I got the error: Permission denied. How to tackle this error ? thanks laks -R.Lakshminarayanan -Axes Technologies India Pvt. Ltd., -Chennai - 600 034. -Email : [EMAIL PROTECTED], Phone: 8253323
Re: rsync dir in _both_ directions?
There's no such option. Rember that Rsync was devised as an efficient way to mirror ftp sites. What you're looking for is unison. Some of the guys on here use it. It's at http://www.cis.upenn.edu/~bcpierce/unison;, and looks promising for your application. Tim Conway [EMAIL PROTECTED] 303.682.4917 Philips Semiconductor - Longmont TC 1880 Industrial Circle, Suite D Longmont, CO 80501 Available via SameTime Connect within Philips, n9hmg on AIM perl -e 'print pack(, 19061,29556,8289,28271,29800,25970,8304,25970,27680,26721,25451,25970), .\n ' There are some who call me Tim? Jack McKinney [EMAIL PROTECTED] Sent by: [EMAIL PROTECTED] 02/04/2002 10:58 AM To: [EMAIL PROTECTED] cc: (bcc: Tim Conway/LMT/SC/PHILIPS) Subject:rsync dir in _both_ directions? Classification: I felt certain there was an option to do this, but I cannot find it. I want to rsync a directory on machine A over to machine B, and then rsync the directory on machine B back to machine A. The idea is this: I read my email using mutt, which is set to save my email in ~/Mail in maildir format. This means that each message is in its own file, instead of being appended to an existing file. Thus, if I backup my email by rsync'ing it to another machine, only the new messages are copied over. Mailboxes need not be. Before you say that rsync only sends changed blocks, and the changed blocks are at the end if the message is appended, remember that mail readers write status into the headers of the messages. If I change the status on the first message in the box (reply to it, delete it), then the ENTIRE mailbox is re-copied. Not so in maildir format. Well, I read my email on my laptop, and it is sometimes necessary to login to a central server to read my email (firewall issues). What I do is to peridocally sync with the central server (which we'll call server) by: rsync -e ssh -rptlv Mail server:. rsync -e ssh -rptlv server:Mail . I'd like to do this in one command. Is there a switch to rsync that says to make the local and remote directories identical, even if that means transfering data in both directions? -- There is no parameter that makes it impossibleJack McKinney for you to perform still more excellently. [EMAIL PROTECTED] -Mario Cuomo, on the lack of a clock in baseball http://www.lorentz.com 1024D/FBED2DAA 4096g/3F93879F 2002 Chicago Cubs magic number: 163 attj7zff.dat Description: Binary data
Re: SIGUSR1 or SIGINT error
When i was getting these, I traced the process and its children (solaris: truss -f). I found that one of the spawned threads was experiencing an io timeout while the filelist was building. I had set no timeout, but it did it at 60 seconds every time. I found that this corresponded to a SELECT_TIMEOUT parameter, which was set to 60 if IO_TIMEOUT was 0. BY setting my timeout to 86400 (1 day), i stopped those. Of course, then, it choked farther along, but that's another story. Try setting a timeout, even if you don't want one. Make it the longest the process should ever take. Tim Conway [EMAIL PROTECTED] 303.682.4917 Philips Semiconductor - Longmont TC 1880 Industrial Circle, Suite D Longmont, CO 80501 Available via SameTime Connect within Philips, n9hmg on AIM perl -e 'print pack(, 19061,29556,8289,28271,29800,25970,8304,25970,27680,26721,25451,25970), .\n ' There are some who call me Tim? Dave Dykstra [EMAIL PROTECTED] Sent by: [EMAIL PROTECTED] 02/06/2002 10:16 AM To: David Birnbaum [EMAIL PROTECTED] cc: [EMAIL PROTECTED] (bcc: Tim Conway/LMT/SC/PHILIPS) Subject:Re: SIGUSR1 or SIGINT error Classification: On Tue, Feb 05, 2002 at 11:28:54AM -0500, David Birnbaum wrote: I suspected that might be the case...now...how to determine the real problem? Does rsync log it somewhere? lsof shows that STDERR/STDOUT are going to /dev/null, so I hope it's not writing it there. Nothing informative in syslog, just the message about the SIG: Feb 5 09:49:41 hite rsyncd[9279]: [ID 702911 daemon.warning] rsync error: received SIGUSR1 or SIGINT (code 20) at rsync.c(229) Any clues? I'm sorry, but I don't have any more suggestions. - Dave Dykstra
Re: rsh syntax - Help
rsync -a /opt/config/. DevlServ:/opt/config (see rsync.1) Tim Conway [EMAIL PROTECTED] 303.682.4917 Philips Semiconductor - Longmont TC 1880 Industrial Circle, Suite D Longmont, CO 80501 Available via SameTime Connect within Philips, n9hmg on AIM perl -e 'print pack(, 19061,29556,8289,28271,29800,25970,8304,25970,27680,26721,25451,25970), .\n ' There are some who call me Tim? Leslie Turkson [EMAIL PROTECTED] Sent by: [EMAIL PROTECTED] 02/06/2002 11:21 AM To: [EMAIL PROTECTED] cc: (bcc: Tim Conway/LMT/SC/PHILIPS) Subject:rsh syntax - Help Classification: Please can someone provide me with the exact syntax for using the rsh protocol. I am trying to copy files in a directory named /opt/config/ on server DevlServ to the same directory on server ApplServ1. I have tried the following without much success: rsync --rsh= cp /opt/config/on DevlServ:/opt/config rsync rsh cp /opt/config/on DevlServ:/opt/config Both of the above results in error messages. I can rsh to server DevlServ from the UNIX command line without being requested to enter a password, so I am pretty confident that rsh is setup correctly and I have full permissions to both directories. I think I just need to get the syntax right. HELP The error message I get is as follows: config cp: cannot access DevlServ cp: cannot access --server cp: .: is a directory rsync: connection unexpectedly closed (0 bytes read so far) rsync error: error in rsync protocol data stream (code 12) at io.c(140) I would appreciate any suggestions. Thanks. Leslie Join the world's largest e-mail service with MSN Hotmail. Click Here
Re: rsync error: partial transfer (code 23) at main.c(576)
Well, Mike. I'd guess it's a permission problem. Either on the destination itself or teh module defined in rsyncd.conf. Let's concentrate on that second one. It's documented that by default, if you do not specify a uid for a module (in rsyncd.conf), it will use -2 (usually corresponds to nobody. same for gid. I suspect that's probably all you need to hear. (see rsyncd.conf.5) Tim Conway [EMAIL PROTECTED] 303.682.4917 Philips Semiconductor - Longmont TC 1880 Industrial Circle, Suite D Longmont, CO 80501 Available via SameTime Connect within Philips, n9hmg on AIM perl -e 'print pack(, 19061,29556,8289,28271,29800,25970,8304,25970,27680,26721,25451,25970), .\n ' There are some who call me Tim? MICHAEL R. LEFEBVRE [EMAIL PROTECTED] Sent by: [EMAIL PROTECTED] 02/06/2002 01:14 PM To: [EMAIL PROTECTED] cc: (bcc: Tim Conway/LMT/SC/PHILIPS) Subject:rsync error: partial transfer (code 23) at main.c(576) Classification: I want to rsync my home directory from interd1 to interp1. interd1 is the server, so I enter: rsync -vraz --delete ./ interp1::test rsync: building file list ... rsync: 6 files to consider. delete_one: unlinek .rhosts: Permission denied ./ .profile rsync: failed to open .sh_history, continuing : Permission denied .sh_history a.a rsync.tst test mkstemp ..profile.KIa00D failed mkstemp ..sh_history.LIa00D failed mkstemp .a.a.MIa00D failed mkstemp .rsync.test.NIa00d failed mkstemp .test.OIa00D failed wrote 1959 bytes read 126 bytes 1930.00 bytes/sec total size is 6173 speedup is 2.96 rsync error: patrial transfer (code 23) at main.c(576) Do you know why this is only partialy transfering? Regards, Mike Lefebvre
Re: SIGUSR1 or SIGINT error
Currently 2.5.1pre3. I haven't tested that problem lately, though. I'll get the newest up and try a full sync. It's worth a try. I'll feel really stupid, though, if i've put all this work into newsync (perl driving find|diff|tar|lzop) and it's fixed in rsync. I think our case will always create problems, though, with the broken nfs unlink in the nfs3 interface on the NAS, and the broken nfs2 client on the solaris machines (mtime bug). I won't let this influence my test, though ;-). Tim Conway [EMAIL PROTECTED] 303.682.4917 Philips Semiconductor - Longmont TC 1880 Industrial Circle, Suite D Longmont, CO 80501 Available via SameTime Connect within Philips, n9hmg on AIM perl -e 'print pack(, 19061,29556,8289,28271,29800,25970,8304,25970,27680,26721,25451,25970), .\n ' There are some who call me Tim? Dave Dykstra [EMAIL PROTECTED] Sent by: [EMAIL PROTECTED] 02/06/2002 03:41 PM To: Eric Whiting [EMAIL PROTECTED] cc: Tim Conway/LMT/SC/PHILIPS@AMEC David Birnbaum [EMAIL PROTECTED] [EMAIL PROTECTED] Subject:Re: SIGUSR1 or SIGINT error Classification: Looks like a fix for that went into 2.5.0. See revision 1.87 at http://cvs.samba.org/cgi-bin/cvsweb/rsync/io.c Tim David, what version are you running? 2.5.2 has some serious problems, Eric. Try the latest development snapshot at rsync://rsync.samba.org/ftp/unpacked/rsync/ or ftp://rsync.samba.org/pub/unpacked/rsync/ - Dave Dykstra On Wed, Feb 06, 2002 at 11:33:43AM -0700, Eric Whiting wrote: Make that 2 of us who need to specify a large timeout. I have found that I have to set the timeout to a large value (1) to get the rsyncs to run successfully. Leaving it at the default seemed to cause timeout/hang problems. Of course I still running a 2.4.6dev version. I had troubles with 2.5.[01]. (solaris/linux mix of of rsync clients/servers) I need to try 2.5.2 as soon as I get a chance. Looks like some good fixes are happening in 2.5.2. eric On Wed, 2002-02-06 at 10:39, [EMAIL PROTECTED] wrote: When i was getting these, I traced the process and its children (solaris: truss -f). I found that one of the spawned threads was experiencing an io timeout while the filelist was building. I had set no timeout, but it did it at 60 seconds every time. I found that this corresponded to a SELECT_TIMEOUT parameter, which was set to 60 if IO_TIMEOUT was 0. BY setting my timeout to 86400 (1 day), i stopped those. Of course, then, it choked farther along, but that's another story. Try setting a timeout, even if you don't want one. Make it the longest the process should ever take. Tim Conway [EMAIL PROTECTED] 303.682.4917 Philips Semiconductor - Longmont TC 1880 Industrial Circle, Suite D Longmont, CO 80501 Available via SameTime Connect within Philips, n9hmg on AIM perl -e 'print pack(, 19061,29556,8289,28271,29800,25970,8304,25970,27680,26721,25451,25970), .\n ' There are some who call me Tim? Dave Dykstra [EMAIL PROTECTED] Sent by: [EMAIL PROTECTED] 02/06/2002 10:16 AM To: David Birnbaum [EMAIL PROTECTED] cc: [EMAIL PROTECTED] (bcc: Tim Conway/LMT/SC/PHILIPS) Subject:Re: SIGUSR1 or SIGINT error Classification: On Tue, Feb 05, 2002 at 11:28:54AM -0500, David Birnbaum wrote: I suspected that might be the case...now...how to determine the real problem? Does rsync log it somewhere? lsof shows that STDERR/STDOUT are going to /dev/null, so I hope it's not writing it there. Nothing informative in syslog, just the message about the SIG: Feb 5 09:49:41 hite rsyncd[9279]: [ID 702911 daemon.warning] rsync error: received SIGUSR1 or SIGINT (code 20) at rsync.c(229) Any clues? I'm sorry, but I don't have any more suggestions. - Dave Dykstra
Re: SIGUSR1 or SIGINT error
Well, it ran to completion this way, in about 7.5h, but i'm not certain i believe it. While i left --delete --force off (I have been horribly burned testing those on really big chunks before), I would expect that the destination would then end up with at least as much as the source. big and big1 are subdirectories of the same volume, and its only contents aside from very small directory containing between 10 and 20 Kb of scripts, so i don't see how the destination could end up 6-1/2Gb short. When my current operations complete, I'll try one with all the options turned on, and run the filesystem map generator from my project to see what differences it left. I have an idea of a mod to make the hard links check more efficient, but I don't understand C well enough. What i was thinking of was to keep the st_nlink part of the stat, and if it'snot a directory and nlink 1, save the path and inode in a seperate list. and leave them out of the main flist. That way, there's no processing of the items for which there's no possibility of a need to track hard links, then fix only one copy of each linked file, delete all the others, and link them back to it. I'm guessing that's a complete redo of the protocol, though. ++ Tools@lonnetsvr /users/Toolscat doit #!/bin/sh /cadappl/encap/packages/rsync-cvs/bin/rsync --rsync-path=/cadappl/encap/packages/rsync-cvs/bin/rsync -WHav --stats --progress alta:/wan/lon-tools1/lon-tools1/big* /wan/lon-tools2/lon-tools2 doit.log 21 /dev/null Tools@lonnetsvr /users/Toolsgrep ' ' doit.log receiving file list ... done big/tools/Tools/.microsoft/Favorites/Channels/Arcadia Bay Demo Channel/ big/tools/Tools/.microsoft/Favorites/Channels/The Microsoft Channel/ big1/cadappl1/hpux/iclibs/CMOS18/EXTERNALS/PcCMOS18sfliolib_nlm_ex/2.1/tools/adf/vital/sfliolib_nlm - ../../vha/sfliolib_nlm big1/cadappl1/hpux/iclibs/CMOS18/EXTERNALS/PcCMOS18shliolib_nlm_ex/2.1/tools/adf/vital/shliolib_nlm - ../../vha/shliolib_nlm big1/cadappl1/hpux/iclibs/CMOS18/PcCMOS18flviolib_spm/2.1.1/lib/flviolib_spm.src - ../tools/vital/timing/flviolib_spm.src big1/cadappl1/hpux/iclibs/CMOS18/PcCMOS18flviolib_spm/2.1.1/tools/adf/vital/flviolib_spm - ../../vha/flviolib_spm big1/cadappl1/hpux/latest - /cadappl/perl/5.6.1 Number of files: 2727469 Number of files transferred: 0 Total file size: 114067347318 bytes Total transferred file size: 0 bytes Literal data: 0 bytes Matched data: 0 bytes File list size: 68790028 Total bytes written: 16 Total bytes read: 68790044 wrote 16 bytes read 68790044 bytes 2531.79 bytes/sec total size is 114067347318 speedup is 1658.20 Tools@lonnetsvr /users/Toolsdf -k /wan/lon-tools*/big/tools Filesystemkbytesused avail capacity Mounted on lon-tools1:big 150147795 121588653 2855914281% /wan/lon-tools1/big lon-tools2:big 150147795 115027617 3512017877% /wan/lon-tools2/big Tools@lonnetsvr /users/Tools ++ Tim Conway [EMAIL PROTECTED] 303.682.4917 Philips Semiconductor - Longmont TC 1880 Industrial Circle, Suite D Longmont, CO 80501 Available via SameTime Connect within Philips, n9hmg on AIM perl -e 'print pack(, 19061,29556,8289,28271,29800,25970,8304,25970,27680,26721,25451,25970), .\n ' There are some who call me Tim? Dave Dykstra [EMAIL PROTECTED] Sent by: [EMAIL PROTECTED] 02/07/2002 10:28 AM To: David Birnbaum [EMAIL PROTECTED] cc: Tim Conway/LMT/SC/PHILIPS@AMEC Eric Whiting [EMAIL PROTECTED] [EMAIL PROTECTED] Subject:Re: SIGUSR1 or SIGINT error Classification: The fix that went into 2.5.0 was for timeouts that were happening even when --timeout=0 (the default). Can any of you say for sure that it makes a difference with a new version when you go from --timeout=0 to a very large timeout? I want to see if Tim's experience with timeouts defaulting to 60 seconds is still happening, or if that was only something earlier. Of course, it's also entirely possible that the SIGUSR1 or SIGINT error message is being caused by a different problem. - Dave Dykstra On Thu, Feb 07, 2002 at 10:22:23AM -0500, David Birnbaum wrote: I'm running 2.5.2. However, we had the same type of problem with 2.4.6, which is what we were running before. If I had to guess, I would say that we're seeing this error a little more often in 2.5.2. David. - On Thu, 7 Feb 2002 [EMAIL PROTECTED] wrote: Currently 2.5.1pre3. I haven't tested that problem lately, though. I'll get the newest up and try a full sync. It's worth a try. I'll feel really stupid, though, if i've put all this work into newsync (perl driving find|diff|tar|lzop) and it's fixed in rsync. I think our case will always create problems, though, with the broken nfs unlink in the nfs3 interface on the NAS, and the broken nfs2 client on the solaris machines
Re: Rsync - TAR
If you're planning to rsync it over, tar it up, and delete the directory tree, you should just tar|gzip it on the work system and catch that in a file on the other end. Tim Conway [EMAIL PROTECTED] 303.682.4917 Philips Semiconductor - Longmont TC 1880 Industrial Circle, Suite D Longmont, CO 80501 Available via SameTime Connect within Philips, n9hmg on AIM perl -e 'print pack(, 19061,29556,8289,28271,29800,25970,8304,25970,27680,26721,25451,25970), .\n ' There are some who call me Tim? Anthony Rabaa [EMAIL PROTECTED] Sent by: [EMAIL PROTECTED] 02/08/2002 11:09 AM To: [EMAIL PROTECTED] cc: (bcc: Tim Conway/LMT/SC/PHILIPS) Subject:Rsync - TAR Classification: Hello, I have rsync doing a nightly backup of my work computer to my home. It's a rather large backup that I would like to have save in a large tar.gz file. I'm hoping this would save a lot of hard drive space which could be used more productively. Is something like this feasible? If so, how could it be accomplished. Thanks in advance! -- Anthony Rabaa Software Developer Network Administrator AMCon Research Inc., Ottawa, Ontario, Canada http://www.amconresearch.com
