Re: configure --with-rsh=CMD and default blocking-IO support
On Thu, 7 Feb 2002, Martin Pool wrote: > A general-purpose RSYNC_OPTS variable would be more tasteful. I think > popt makes supporting this fairly straightforward. That's a nice idea. One area we'll want to be careful of is how the two options interact. For instance, we want to support old scripts that might set RSYNC_RSH and then run a bunch of rsync commands. It would be nice to make this work without conflicting with a user's also-existing RSYNC_OPTS var. A potential solution is to ignore RSYNC_OPTS if RSYNC_RSH is set (which also serves to wean people away from RSYNC_RSH if they want to be able to set other default options). Another potential problem area is how to override already-set options. If someone wants to put -a into their RSYNC_OPTS variable, how can they then turn it off? I suppose we could just say that the user gets what she deserves in such a case. So, perhaps I'm trying to solve a problem that isn't really all that important. Just having the ability to set the default remote shell and its IO mode might be good enough for most people, and we let the rest use shell scripts or aliases, like you said. I could trim down my last patch to avoid the extra RSYNC_RSH parsing if you'd like to just apply the other part of it. Or, feel free to tweak it yourself -- it should be pretty easy. ..wayne..
RE: [path] & module options with SSH
Many thanks Dave! I very much appreciate you taking the time to answer ALL my questions! It was very instructive and helpful to me! I have to say, two days ago, I never eared of rsync and today, I honestly can't say how useful it is to me! I guess someone doesn't know how much they miss something until they find it! I was doing (or more correctly, trying to...) a lots of this stuff with plenty of others script and it wasn't near as efficient and clean as this solution is for me! Not even close!!! By reading archive emails I realize you must be very busy with this project and many others, so it make me even more appreciate you taking the time to help me out with my questions and quest on understand this! This is a GREAT tools! Daniel
Re: rsync dir in _both_ directions?
On 6 Feb 2002, Jack McKinney <[EMAIL PROTECTED]> wrote: > > #!/bin/sh > > eval `ssh-agent` > > ssh-add > > rsync foo bar > > rsync qux zump > > .. > > kill $SSH_AGENT_PID > > As I indicated, I do not want to use ssh-agent. For religious reasons? -- Martin
Re: error in rsync protocol data stream (code 12) at token.c(288)
The newer zlib-1.1.3 patch still has the problem. According to Dave Dykstra: > I was able to reproduce this between a Solaris 2.7 and 2.8 machine with and > without ssh, although I don't have any time to debug it today. Can > somebody try applying Jos Backus latest zlib 1.1.3 upgrade patch to see > if that helps? > > - Dave > > On Fri, Feb 01, 2002 at 02:01:03PM -0800, Stuart Anderson wrote: > > According to Dave Dykstra: > > > On Fri, Feb 01, 2002 at 09:09:18AM -0800, Stuart Anderson wrote: > > > > According to sba: > > > > > > > > No I did not apply the patch, but I verified the same problem with > > > > > rsync-2.4.8, is that sufficient? > > > > > > No, because rsync-2.4.8 include the same security patch that went into > > > 2.5.2 last weekend. If you could try 2.4.6 or 2.5.1 that would tell us if > > > it was some other problem introduced last weekend or not. > > > > The problem was introduced in version 2.4.4 and exists in all later > > versions up to and including 2.5.2. Whereas version 2.4.3 and 2.4.0 > > are both fine (the only earlier versions checked). > > > > The problem only exists for compression and large files. > > > > I encountered and tested the various rsync versions with the file, > > redhat/7.1/en/iso/i386/seawolf-i386-powertools.iso > > gotten from your nearest RedHat mirror, but it is more general > > than just this file. > > > > However, the problem is not reproducible with any of the 1018 "small" > > files from redhat/7.1/en/os/i386/RedHat/RPMS which range in size from > > 1810 Byte to 21541821 Byte. > > > > > > To reproduce: > > > > 1) rsync --archive a large file (ISO size?) to a remote machine > > 2) /bin/touch it on the destination machine > > 3) rsync --archive --compress again > > 4) Check for error message: "deflate on token returned 0 (16384 bytes left)" > > > > I don't know if it matters but I am running rsync between two Solaris 8 > > machines, and using ssh as the transport: > > > > > ssh -V > > OpenSSH_2.9p2, SSH protocols 1.5/2.0, OpenSSL 0x0090601f > > > > -- > > Stuart Anderson [EMAIL PROTECTED] http://www.srl.caltech.edu/personnel/sba -- Stuart Anderson [EMAIL PROTECTED] http://www.srl.caltech.edu/personnel/sba
Re: configure --with-rsh=CMD and default blocking-IO support
On 6 Feb 2002, Wayne Davison <[EMAIL PROTECTED]> wrote: > Cool. I like that one as well. Here's an implementation. This patch > adds the configure option --with(out)-blocking-io and defines a new > variable that gets put into config.h: DEFAULT_BLOCKING_IO. OK, that's good. Thanks. > The code now allows the remote shell value to contain a single prefixed > IO-blocking option. If the string starts with "--" and it has a space > in it, the string must start with "--blocking-io ", "--no-blocking-io ", > or "-- " (the last item allows someone to use a program name that > matches one of our options -- just for completeness). I really think we will regret this later. I don't see any compelling reason to overload this variable with more than one meaning, especially because it's only a workaround for an OS/SSH bug. A general-purpose RSYNC_OPTS variable would be more tasteful. I think popt makes supporting this fairly straightforward. Alternatively people can use shell functions or scripts. > and also to not talk like rsh is always the default remote shell. OK, that's good -- Martin
Re: SIGUSR1 or SIGINT error
Dave, I tried the snapshot... I get an error (after a ./configure;make). gcc -I. -I. -g -O2 -DHAVE_CONFIG_H -Wall -W -c main.c -o main.o main.c: In function `do_cmd': main.c:184: `RSYNC_RSH' undeclared (first use in this function) I added the #define to config.h and it compiled ok with a lot of warnings.. (SUSE 7.3) I"ll try it out. eric On Wed, 2002-02-06 at 15:41, Dave Dykstra wrote: > Looks like a fix for that went into 2.5.0. See revision 1.87 at > http://cvs.samba.org/cgi-bin/cvsweb/rsync/io.c > > Tim & David, what version are you running? > > 2.5.2 has some serious problems, Eric. Try the latest development > snapshot at > rsync://rsync.samba.org/ftp/unpacked/rsync/ > or > ftp://rsync.samba.org/pub/unpacked/rsync/ > > - Dave Dykstra > > > On Wed, Feb 06, 2002 at 11:33:43AM -0700, Eric Whiting wrote: > > Make that 2 of us who need to specify a large timeout. > > > > I have found that I have to set the timeout to a large value (1) to > > get the rsyncs to run successfully. Leaving it at the default seemed to > > cause timeout/hang problems. Of course I still running a 2.4.6dev > > version. I had troubles with 2.5.[01]. (solaris/linux mix of of rsync > > clients/servers) > > > > I need to try 2.5.2 as soon as I get a chance. Looks like some good > > fixes are happening in 2.5.2. > > > > eric > > > > > > > > On Wed, 2002-02-06 at 10:39, [EMAIL PROTECTED] wrote: > > > When i was getting these, I traced the process and its children (solaris: > > > truss -f). I found that one of the spawned threads was experiencing an io > > > timeout while the filelist was building. I had set no timeout, but it did > > > it at 60 seconds every time. I found that this corresponded to a > > > SELECT_TIMEOUT parameter, which was set to 60 if IO_TIMEOUT was 0. BY > > > setting my timeout to 86400 (1 day), i stopped those. Of course, then, it > > > choked farther along, but that's another story. > > > Try setting a timeout, even if you don't want one. Make it the longest > > > the process should ever take. > > > > > > Tim Conway > > > [EMAIL PROTECTED] > > > 303.682.4917 > > > Philips Semiconductor - Longmont TC > > > 1880 Industrial Circle, Suite D > > > Longmont, CO 80501 > > > Available via SameTime Connect within Philips, n9hmg on AIM > > > perl -e 'print pack(, > > > 19061,29556,8289,28271,29800,25970,8304,25970,27680,26721,25451,25970), > > > ".\n" ' > > > "There are some who call me Tim?" > > > > > > > > > > > > > > > Dave Dykstra <[EMAIL PROTECTED]> > > > Sent by: [EMAIL PROTECTED] > > > 02/06/2002 10:16 AM > > > > > > > > > To: David Birnbaum <[EMAIL PROTECTED]> > > > cc: [EMAIL PROTECTED] > > > (bcc: Tim Conway/LMT/SC/PHILIPS) > > > Subject:Re: SIGUSR1 or SIGINT error > > > Classification: > > > > > > > > > > > > On Tue, Feb 05, 2002 at 11:28:54AM -0500, David Birnbaum wrote: > > > > I suspected that might be the case...now...how to determine the "real" > > > > problem? Does rsync log it somewhere? lsof shows that STDERR/STDOUT > > > are > > > > going to /dev/null, so I hope it's not writing it there. Nothing > > > > informative in syslog, just the message about the SIG: > > > > > > > > Feb 5 09:49:41 hite rsyncd[9279]: [ID 702911 daemon.warning] rsync > > > error: received SIGUSR1 or SIGINT (code 20) at rsync.c(229) > > > > > > > > Any clues? > > > > > > > > > I'm sorry, but I don't have any more suggestions. > > > > > > - Dave Dykstra > > > > > > > > > > > > > > > > > >
Re: error in rsync protocol data stream (code 12) at token.c(288)
I was able to reproduce this between a Solaris 2.7 and 2.8 machine with and without ssh, although I don't have any time to debug it today. Can somebody try applying Jos Backus latest zlib 1.1.3 upgrade patch to see if that helps? - Dave On Fri, Feb 01, 2002 at 02:01:03PM -0800, Stuart Anderson wrote: > According to Dave Dykstra: > > On Fri, Feb 01, 2002 at 09:09:18AM -0800, Stuart Anderson wrote: > > > According to sba: > > > > > > No I did not apply the patch, but I verified the same problem with > > > > rsync-2.4.8, is that sufficient? > > > > No, because rsync-2.4.8 include the same security patch that went into > > 2.5.2 last weekend. If you could try 2.4.6 or 2.5.1 that would tell us if > > it was some other problem introduced last weekend or not. > > The problem was introduced in version 2.4.4 and exists in all later > versions up to and including 2.5.2. Whereas version 2.4.3 and 2.4.0 > are both fine (the only earlier versions checked). > > The problem only exists for compression and large files. > > I encountered and tested the various rsync versions with the file, > redhat/7.1/en/iso/i386/seawolf-i386-powertools.iso > gotten from your nearest RedHat mirror, but it is more general > than just this file. > > However, the problem is not reproducible with any of the 1018 "small" > files from redhat/7.1/en/os/i386/RedHat/RPMS which range in size from > 1810 Byte to 21541821 Byte. > > > To reproduce: > > 1) rsync --archive a large file (ISO size?) to a remote machine > 2) /bin/touch it on the destination machine > 3) rsync --archive --compress again > 4) Check for error message: "deflate on token returned 0 (16384 bytes left)" > > I don't know if it matters but I am running rsync between two Solaris 8 > machines, and using ssh as the transport: > > > ssh -V > OpenSSH_2.9p2, SSH protocols 1.5/2.0, OpenSSL 0x0090601f > > -- > Stuart Anderson [EMAIL PROTECTED] http://www.srl.caltech.edu/personnel/sba
Re: configure --with-rsh=CMD and default blocking-IO support
On Wed, 6 Feb 2002, Dave Dykstra wrote:
> Of the proposed alternatives, I like this latter the best, changing
> --non-blocking-io to --no-blocking-io.
Cool. I like that one as well. Here's an implementation. This patch
adds the configure option --with(out)-blocking-io and defines a new
variable that gets put into config.h: DEFAULT_BLOCKING_IO.
The default for configure is just as before: remsh or rsh gets used
with blocking IO on by default. If the user specifies --with-rsh=CMD
then the default is --without-blocking-io unless the user also specifies
the --with-blocking-io configure option.
The code in main.c now uses the DEFAULT_BLOCKING_IO value, but only when
we use the default RSYNC_RSH (internal) value. If the user specifies an
RSYNC_RSH environment variable (or a remote shell via the command-line),
the default is to use non-blocking IO. (This is a slight change in
behavior if the user had set RSYNC_RSH=rsh in their environment -- is
this acceptable?)
The code now allows the remote shell value to contain a single prefixed
IO-blocking option. If the string starts with "--" and it has a space
in it, the string must start with "--blocking-io ", "--no-blocking-io ",
or "-- " (the last item allows someone to use a program name that
matches one of our options -- just for completeness).
I also updated the main man page to mention the new RSYNC_RSH syntax,
and also to not talk like rsh is always the default remote shell. In
the --blocking-io section, it used to say that ssh prefers blocking IO.
I've never used anything but non-blocking IO with ssh, so is this
statement backwards? I tweaked the statement to say that only some
versions of ssh prefer blocking IO.
Don't forget to run autoconf and autoheader after applying this patch.
..wayne..
---8<--8<--8<--8<---cut here--->8-->8-->8-->8---
Index: rsync/configure.in
--- rsync/configure.in 6 Feb 2002 04:37:09 - 1.131
+++ rsync/configure.in 6 Feb 2002 22:45:04 -
@@ -102,6 +102,23 @@
fi
AC_DEFINE_UNQUOTED(RSYNC_RSH, "$RSYNC_RSH", [default -e command])
+
+AC_ARG_WITH(blocking-io,
+ AC_HELP_STRING([--with-blocking-io], [set blocking IO for your remote shell]))
+
+case "$with_blocking_io" in
+'')
+if test x"$with_rsh" != x; then
+ IO=0
+else
+ IO=1
+fi
+;;
+no) IO=0 ;;
+*) IO=1 ;;
+esac
+
+AC_DEFINE_UNQUOTED(DEFAULT_BLOCKING_IO, $IO, [default to blocking IO])
# arrgh. libc in the current debian stable screws up the largefile
# stuff, getting byte range locking wrong
Index: rsync/main.c
--- rsync/main.c5 Feb 2002 23:05:32 - 1.139
+++ rsync/main.c6 Feb 2002 22:45:08 -
@@ -178,10 +178,25 @@
extern int read_batch;
if (!read_batch && !local_server) { /* dw -- added read_batch */
+ int def_io = DEFAULT_BLOCKING_IO;
if (!cmd)
cmd = getenv(RSYNC_RSH_ENV);
if (!cmd)
cmd = RSYNC_RSH;
+ else
+ def_io = 0;
+ if (*cmd == '-' && cmd[1] == '-' && (tok = strchr(cmd, ' '))) {
+ if (strncmp(cmd+2, "blocking-io ", 12) == 0)
+ def_io = 1;
+ else if (strncmp(cmd+2, "no-blocking-io ", 15) == 0)
+ def_io = 0;
+ else if (cmd[2] != ' ') {
+ rprintf(FERROR,"Invalid remote-shell-IO option: %s\n",
+ cmd);
+ exit_cleanup(RERR_SYNTAX);
+ }
+ cmd = tok + 1;
+ }
cmd = strdup(cmd);
if (!cmd)
goto oom;
@@ -207,8 +222,8 @@
args[argc++] = rsync_path;
- if ((blocking_io == -1) && (strcmp(cmd, RSYNC_RSH) == 0))
- blocking_io = 1;
+ if (blocking_io < 0)
+ blocking_io = def_io;
server_options(args,&argc);
Index: rsync/options.c
--- rsync/options.c 5 Feb 2002 23:05:32 - 1.78
+++ rsync/options.c 6 Feb 2002 22:45:09 -
@@ -206,7 +206,7 @@
rprintf(F," --no-whole-file turn off --whole-file\n");
rprintf(F," -x, --one-file-system don't cross filesystem boundaries\n");
rprintf(F," -B, --block-size=SIZE checksum blocking size (default
%d)\n",BLOCK_SIZE);
- rprintf(F," -e, --rsh=COMMAND specify rsh replacement\n");
+ rprintf(F," -e, --rsh=COMMAND specify the remote shell\n");
rprintf(F," --rsync-path=PATH specify path to rsync on the remote
machine\n");
rprintf(F," -C, --cvs-exclude auto ignore files in the same way CVS
does\n");
rprintf(F," --existing only update files that already exist\n");
Index: rsync/rsync.yo
--- rsync/rsync.yo 5 Feb 2002 23:05:33 -
Re: SIGUSR1 or SIGINT error
Looks like a fix for that went into 2.5.0. See revision 1.87 at http://cvs.samba.org/cgi-bin/cvsweb/rsync/io.c Tim & David, what version are you running? 2.5.2 has some serious problems, Eric. Try the latest development snapshot at rsync://rsync.samba.org/ftp/unpacked/rsync/ or ftp://rsync.samba.org/pub/unpacked/rsync/ - Dave Dykstra On Wed, Feb 06, 2002 at 11:33:43AM -0700, Eric Whiting wrote: > Make that 2 of us who need to specify a large timeout. > > I have found that I have to set the timeout to a large value (1) to > get the rsyncs to run successfully. Leaving it at the default seemed to > cause timeout/hang problems. Of course I still running a 2.4.6dev > version. I had troubles with 2.5.[01]. (solaris/linux mix of of rsync > clients/servers) > > I need to try 2.5.2 as soon as I get a chance. Looks like some good > fixes are happening in 2.5.2. > > eric > > > > On Wed, 2002-02-06 at 10:39, [EMAIL PROTECTED] wrote: > > When i was getting these, I traced the process and its children (solaris: > > truss -f). I found that one of the spawned threads was experiencing an io > > timeout while the filelist was building. I had set no timeout, but it did > > it at 60 seconds every time. I found that this corresponded to a > > SELECT_TIMEOUT parameter, which was set to 60 if IO_TIMEOUT was 0. BY > > setting my timeout to 86400 (1 day), i stopped those. Of course, then, it > > choked farther along, but that's another story. > > Try setting a timeout, even if you don't want one. Make it the longest > > the process should ever take. > > > > Tim Conway > > [EMAIL PROTECTED] > > 303.682.4917 > > Philips Semiconductor - Longmont TC > > 1880 Industrial Circle, Suite D > > Longmont, CO 80501 > > Available via SameTime Connect within Philips, n9hmg on AIM > > perl -e 'print pack(, > > 19061,29556,8289,28271,29800,25970,8304,25970,27680,26721,25451,25970), > > ".\n" ' > > "There are some who call me Tim?" > > > > > > > > > > Dave Dykstra <[EMAIL PROTECTED]> > > Sent by: [EMAIL PROTECTED] > > 02/06/2002 10:16 AM > > > > > > To: David Birnbaum <[EMAIL PROTECTED]> > > cc: [EMAIL PROTECTED] > > (bcc: Tim Conway/LMT/SC/PHILIPS) > > Subject:Re: SIGUSR1 or SIGINT error > > Classification: > > > > > > > > On Tue, Feb 05, 2002 at 11:28:54AM -0500, David Birnbaum wrote: > > > I suspected that might be the case...now...how to determine the "real" > > > problem? Does rsync log it somewhere? lsof shows that STDERR/STDOUT > > are > > > going to /dev/null, so I hope it's not writing it there. Nothing > > > informative in syslog, just the message about the SIG: > > > > > > Feb 5 09:49:41 hite rsyncd[9279]: [ID 702911 daemon.warning] rsync > > error: received SIGUSR1 or SIGINT (code 20) at rsync.c(229) > > > > > > Any clues? > > > > > > I'm sorry, but I don't have any more suggestions. > > > > - Dave Dykstra > > > > > > > > > > >
Re: rsync dir in _both_ directions?
Big Brother tells me that David Bolen wrote: > > Since it's an easy experiment - why "suspect" - did you try this? It > should take virtually no time for the second (sans the initial > checksum computation and transmission, which to be fair for large > files and small block sizes can be quite significant). I hadn't checked it because I haven't had the time to set up the experiment. I was planning to do this at home tonight. I must say, I am quite impressed by the algorithm rsync is implementing. I am going to have to look at the source on this when I get some time. -- "As you gain experience, you'll realise that Jack McKinney all logical questions are considered [EMAIL PROTECTED] insubordination." -Dogbert http://www.lorentz.com 1024D/FBED2DAA 4096g/3F93879F 2002 Chicago Cubs magic number: 163 msg03008/pgp0.pgp Description: PGP signature
Re: [path] & module options with SSH
On Wed, Feb 06, 2002 at 02:57:29PM -0500, Daniel Ouellet wrote: > Dave Dykstra wrote: > > JD Paul's patch which I previously referred to will give you what > > you want. > > So, this patch would apply only when the :: is in use correct? That's what I > understand. Yes. > > ":" syntax uses rsh (or ssh if you use -e ssh) to run another copy of the > > rsync program on the remote side. "::" syntax skips that completely, > > ignores -e, and instead connects to a daemon separately started to listen > > on port 873 on the remote host. In the future, when JD Paul's patch is > > accepted, the expectation will be that if you use "::" and "-e ssh" > > together it will still use ssh to connect but it will run rsync -daemon > > interactively so it can honor your rsyncd.conf. > > > > Does that make it clear? > > > Now I re-read the documents a few more times and I did some more test. I > just got the light I guess! (:> I was running a daemon on my server or > source and it wasn't needed with the : syntax I used! Duh!! (:< > > Here again correct me if I am wrong, but the : as explain in the DOC ( but I > didn't get it the first time, and the second and...) is use when your SSH > remote shell will start a rsync process on the remote server and YOU DO NOT > NEED a --daemon running there, but at the moment, without JD Paul's patch, > the rsyncd.conf file is ignore. Right. > The :: will use the --daemon on the remote server and should accept the > rsyncd.conf correctly with SSH if JD Paul's patch is apply. I think I am > clear on that as well, right? Yes. With JD Paul's patch, the remote side will not be listening on port 873 but it will start up a new instance of rsync --daemon under ssh and honor rsyncd.conf. > At the moment if SSH is in use, regardless if you use : or ::, the > rsync.conf is not use without JD Paul's patch apply, so there is no way to > limit access to part of the file system under SSH other then changing the > right on the user on the remote server? Right. If you use :: now, you are not using SSH. > Many thanks for your help and answer! It put some light for me on this. > > Last part if I may abuse of your generosity so far. In the 2.5.2, there is > the use of malloc.h that wasn't there n the 2.5.1 and will make my compiler > complain and provide me plenty of warning. 2.5.2 added the gcc -Wall option but hasn't yet tried to fix many of the warnings. I expect those to come along some time in the future. > Any patch to use the more resent > stdlib.h instead or am I crazy to ask? rsync.h is already including stdlib.h if it exists, so that must not be the problem. - Dave
URL style syntax in destination?
On Wed, Feb 06, 2002 at 01:09:22PM -0500, Alberto Accomazzi wrote: > Subject: Re: [path] & module options with SSH > > The discussion about syntax for remote file specification and the exchange > between Martin and Wayne about configure options for rsh make me wonder if > we should push some alternative syntax for specifying the transport protocol > to be used by rsync. > > I, for one, always stick to the rsync://host/module syntax when pulling from > an rsync server, and have often wished that the same syntax were available > when doing a push. I find the URL-style syntax easy to remember and understand, > while the "::" seems much less intuitive (it actually looks perlish to me > because of the way modules are specified in perl). > Among other things, I notice that SUN uses the url syntax in its man pages > describing NFS (they use nfs://host[:port]/pathname). > > So what came to mind is to have rsync recognize and use both for push and > pull remote specifications of the form: > > rsync://host/module/file > ssh://[username@]host/dir/file > rsh://[username@]host/dir/file > > I'm not crazy about the last two, but thought of them while reading messages > about ssh/rsh issues. Hmm... one problem that this wouldn't solve is the > use of ssh-over-rsyncd that somebody has proposed, though. Also I'm not > sure how I would handle the passing of additional options to the external > transport program (what we do now with -e 'shell [OPTIONS]'). Ok, so > maybe this is not so hot, but rsync:// is cool, IMHO. It seems like there would be some pitfall, but I can't think what it might be right now. Web browsers typically only do reading so that's why I think it was only thought of for the source side. I looked back through my saved email, and it was Andrew Tridgell who added the syntax while I was the primary rsync maintainer. I didn't find any discussion of whether or not it should also be on the destination side. I suggest that you try implementing it and see how well it fits in and how clean the implementation is. I too am not crazy about the rsh and ssh URLs. - Dave Dykstra
Re: List of rsync output / error messages
Joseph, check out the header file errcode.h in the rsync distribution. That file and the structure found in log.c map the system exit codes to the error messages you refer to, so the best way to programmatically catch errors is simply to check the exit status returned by rsync. -- Alberto In message <[EMAIL PROTECTED]>, Joseph Annino writes: > Is there a good place to get information about the list of all possible > output and error messages rsync generates? Or should I just muck around the > source code (which I haven't looked at yet) and find them? > > I am doing something where I would like to parse rsync's output using Perl > into a set of data structures. I already have something that works under > normal conditions. Eventually I'd like to use that data as part of building > a Perl/Tk interface. Of course to parse the output successfully, I need to > know all the possibilities as surprises can throw things out of whack. > > And this is just an idea, but ways to make rsync's output more easily > parseable, and more verbose in terms to reporting information that would be > useful for say making a progress bar would be nice to discuss. > > Thanks. > > > -- > Joseph Annino Consulting - Perl, PHP, MySQL, Oracle, etc. > [EMAIL PROTECTED] - http://www.jannino.com > Alberto Accomazzi mailto:[EMAIL PROTECTED] NASA Astrophysics Data System http://adsabs.harvard.edu Harvard-Smithsonian Center for Astrophysicshttp://cfawww.harvard.edu 60 Garden Street, MS 83, Cambridge, MA 02138 USA
Re: rsync error: partial transfer (code 23) at main.c(576)
Probably Mike needs to use "read only = false" in his rsyncd.conf. The default is "read only = true". - Dave Dykstra On Wed, Feb 06, 2002 at 02:16:55PM -0700, [EMAIL PROTECTED] wrote: > Well, Mike. I'd guess it's a permission problem. Either on the > destination itself or teh module defined in rsyncd.conf. Let's > concentrate on that second one. It's documented that by default, if you > do not specify a uid for a module (in rsyncd.conf), it will use -2 > (usually corresponds to "nobody". same for gid. I suspect that's > probably all you need to hear. > (see rsyncd.conf.5) > > Tim Conway > [EMAIL PROTECTED] > 303.682.4917 > Philips Semiconductor - Longmont TC > 1880 Industrial Circle, Suite D > Longmont, CO 80501 > Available via SameTime Connect within Philips, n9hmg on AIM > perl -e 'print pack(, > 19061,29556,8289,28271,29800,25970,8304,25970,27680,26721,25451,25970), > ".\n" ' > "There are some who call me Tim?" > > > > > "MICHAEL R. LEFEBVRE" <[EMAIL PROTECTED]> > Sent by: [EMAIL PROTECTED] > 02/06/2002 01:14 PM > > > To: <[EMAIL PROTECTED]> > cc: (bcc: Tim Conway/LMT/SC/PHILIPS) > Subject:rsync error: partial transfer (code 23) at main.c(576) > Classification: > > > > I want to rsync my home directory from interd1 to interp1. interd1 is > the server, so I enter: > > rsync -vraz --delete ./ interp1::test > > rsync: building file list ... > rsync: 6 files to consider. > delete_one: unlinek .rhosts: Permission denied > ./ > .profile > rsync: failed to open ".sh_history", continuing : Permission denied > .sh_history > a.a > rsync.tst > test > mkstemp ..profile.KIa00D failed > mkstemp ..sh_history.LIa00D failed > mkstemp .a.a.MIa00D failed > mkstemp .rsync.test.NIa00d failed > mkstemp .test.OIa00D failed > wrote 1959 bytes read 126 bytes 1930.00 bytes/sec > total size is 6173 speedup is 2.96 > rsync error: patrial transfer (code 23) at main.c(576) > > Do you know why this is only partialy transfering? > > Regards, > > Mike Lefebvre > > > > >
RE: rsync dir in _both_ directions?
Jack McKinney [[EMAIL PROTECTED]] writes: > If I add 512 bytes at the begining of the file, then I would expect > it. If I only add 14 bytes, then I don't think rsync will detect this, > as it would require it to compute checksums start at EVERY byte instead > of 512 byte checksums at offsets 0, 512, 1024, 1536, et al. Yep, and that's precisely what rsync does. It actually uses two types of checksums. One is a fast rolling checksum that can be efficiently computed with a block starting at _every_ byte in the file. The nature of the checksum is that you can compute its new value starting at byte X+1, based on its old value from a block starting at X by only performing a single computation based on the new byte at the end of the block starting at X+1. But the penalty you pay for the speed is that it's a "weaker" checksum - you can have inaccurately identified matches (e.g., overlaps in the checksum). So there's a second, much stronger checksum, but much slower, that is used to validate a match once the first checksum thinks it found a match. When you transmit a file, the sender computes both checksums for each block in the file it has and sends them over. The receiver then walks its current file, taking block size chunks _at every byte_ and computing the weak/fast checksum. If the weak matches, it then does the stronger checksum, and if that matches, it knows it need not request that block of data from the sender. This will match common blocks located anywhere within the file at any offset (including re-using a source block multiple times to reproduce the target). You might want to read the tech paper on rsync and its protocol, since it goes into this in much more detail. If all rsync did was match on finite block boundaries, it would be _way_ less useful than it really is. >It is an easy experiment. (...) > (...) I suspect that your xfer time will be comparable to > the first one, not to the second. Since it's an easy experiment - why "suspect" - did you try this? It should take virtually no time for the second (sans the initial checksum computation and transmission, which to be fair for large files and small block sizes can be quite significant). -- David /---\ \ David Bolen\ E-mail: [EMAIL PROTECTED] / | FitLinxx, Inc.\ Phone: (203) 708-5192| / 860 Canal Street, Stamford, CT 06902 \ Fax: (203) 316-5150 \ \---/
Re: Latest version of the batch mode cleanup patch
On Wed, Feb 06, 2002 at 11:55:29AM -0800, Jos Backus wrote: > Here's the latest version of the batch mode cleanup patch. Because I've been testing with Jos on that patch I went ahead and submitted it to CVS. - Dave
Re: rsync error: partial transfer (code 23) at main.c(576)
Well, Mike. I'd guess it's a permission problem. Either on the destination itself or teh module defined in rsyncd.conf. Let's concentrate on that second one. It's documented that by default, if you do not specify a uid for a module (in rsyncd.conf), it will use -2 (usually corresponds to "nobody". same for gid. I suspect that's probably all you need to hear. (see rsyncd.conf.5) Tim Conway [EMAIL PROTECTED] 303.682.4917 Philips Semiconductor - Longmont TC 1880 Industrial Circle, Suite D Longmont, CO 80501 Available via SameTime Connect within Philips, n9hmg on AIM perl -e 'print pack(, 19061,29556,8289,28271,29800,25970,8304,25970,27680,26721,25451,25970), ".\n" ' "There are some who call me Tim?" "MICHAEL R. LEFEBVRE" <[EMAIL PROTECTED]> Sent by: [EMAIL PROTECTED] 02/06/2002 01:14 PM To: <[EMAIL PROTECTED]> cc: (bcc: Tim Conway/LMT/SC/PHILIPS) Subject:rsync error: partial transfer (code 23) at main.c(576) Classification: I want to rsync my home directory from interd1 to interp1. interd1 is the server, so I enter: rsync -vraz --delete ./ interp1::test rsync: building file list ... rsync: 6 files to consider. delete_one: unlinek .rhosts: Permission denied ./ .profile rsync: failed to open ".sh_history", continuing : Permission denied .sh_history a.a rsync.tst test mkstemp ..profile.KIa00D failed mkstemp ..sh_history.LIa00D failed mkstemp .a.a.MIa00D failed mkstemp .rsync.test.NIa00d failed mkstemp .test.OIa00D failed wrote 1959 bytes read 126 bytes 1930.00 bytes/sec total size is 6173 speedup is 2.96 rsync error: patrial transfer (code 23) at main.c(576) Do you know why this is only partialy transfering? Regards, Mike Lefebvre
Re: rsh syntax - Help
rsync -a /opt/config/. DevlServ:/opt/config (see rsync.1) Tim Conway [EMAIL PROTECTED] 303.682.4917 Philips Semiconductor - Longmont TC 1880 Industrial Circle, Suite D Longmont, CO 80501 Available via SameTime Connect within Philips, n9hmg on AIM perl -e 'print pack(, 19061,29556,8289,28271,29800,25970,8304,25970,27680,26721,25451,25970), ".\n" ' "There are some who call me Tim?" "Leslie Turkson" <[EMAIL PROTECTED]> Sent by: [EMAIL PROTECTED] 02/06/2002 11:21 AM To: [EMAIL PROTECTED] cc: (bcc: Tim Conway/LMT/SC/PHILIPS) Subject:rsh syntax - Help Classification: Please can someone provide me with the exact syntax for using the rsh protocol. I am trying to copy files in a directory named /opt/config/ on server DevlServ to the same directory on server ApplServ1. I have tried the following without much success: rsync --rsh= cp /opt/config/on DevlServ:/opt/config rsync rsh cp /opt/config/on DevlServ:/opt/config Both of the above results in error messages. I can rsh to server DevlServ from the UNIX command line without being requested to enter a password, so I am pretty confident that rsh is setup correctly and I have full permissions to both directories. I think I just need to get the syntax right. HELP The error message I get is as follows: config cp: cannot access DevlServ cp: cannot access --server cp: .: is a directory rsync: connection unexpectedly closed (0 bytes read so far) rsync error: error in rsync protocol data stream (code 12) at io.c(140) I would appreciate any suggestions. Thanks. Leslie Join the world's largest e-mail service with MSN Hotmail. Click Here
Re: rsync dir in _both_ directions?
On Wed, Feb 06, 2002 at 02:14:23PM -0600, Jack McKinney wrote: ... > If I add 512 bytes at the begining of the file, then I would expect > it. If I only add 14 bytes, then I don't think rsync will detect this, > as it would require it to compute checksums start at EVERY byte instead > of 512 byte checksums at offsets 0, 512, 1024, 1536, et al. > It is an easy experiment. Create a large text file (e.g., uuencode > a decently large binary) and rsync it to another machine and time it. > Then, edit it, and change ONE BYTE and rsync again. It should go much > faster because only one block has changed. Then, edit it a second time > and add a new line at the top that says Kilroy was here. This will add > 17 bytes to the file (13 letters, two spaces, one period, one newline). > Then rsync it again. I suspect that your xfer time will be comparable to > the first one, not to the second. I did try it. Did you? It works. Look at the "speedup" printed by -v and the total bytes transferred. - Dave
List of rsync output / error messages
Is there a good place to get information about the list of all possible output and error messages rsync generates? Or should I just muck around the source code (which I haven't looked at yet) and find them? I am doing something where I would like to parse rsync's output using Perl into a set of data structures. I already have something that works under normal conditions. Eventually I'd like to use that data as part of building a Perl/Tk interface. Of course to parse the output successfully, I need to know all the possibilities as surprises can throw things out of whack. And this is just an idea, but ways to make rsync's output more easily parseable, and more verbose in terms to reporting information that would be useful for say making a progress bar would be nice to discuss. Thanks. -- Joseph Annino Consulting - Perl, PHP, MySQL, Oracle, etc. [EMAIL PROTECTED] - http://www.jannino.com
Solaris 2.7 error in file IO (code 11) at main.c(883)
I'm updating to version 2.5.2 on a Solaris 2.7 (and 2.6) system. The rsync process seems to run but finishes with the following: wrote 91830 bytes read 4589600 bytes 105200.67 bytes/sec total size is 127959731 speedup is 27.33 rsync error: error in file IO (code 11) at main.c(883) I've tried the binary downloaded version and recompiled myself from the source. Is the error message an indication that the process didn't correctly sync the systems or that some closing cleanup was incorrect? Thanks Stan Silverman
Re: rsync dir in _both_ directions?
Big Brother tells me that Martin Pool wrote: > > With your commands, all messages will end up on both machines, which > is fine to start with. But if you delete a message it will presumably > keep on being copied back from the other machine. I don't know a good > way for rsync to distinguish between the case of a message deleted on > machine A, and a message newly added on machine B. I don't delete messages, so this is not an issue. -- "Charlie, here comes the deuce; and when Jack McKinney you speak of me, speak well." [EMAIL PROTECTED] -Crash Davis, Bull Durham http://www.lorentz.com 1024D/FBED2DAA 4096g/3F93879F 2002 Chicago Cubs magic number: 163 msg02996/pgp0.pgp Description: PGP signature
rsync error: partial transfer (code 23) at main.c(576)
I want to rsync my home directory from interd1 to interp1. interd1 is the server, so I enter: rsync -vraz --delete ./ interp1::test rsync: building file list ... rsync: 6 files to consider. delete_one: unlinek .rhosts: Permission denied ./ .profile rsync: failed to open ".sh_history", continuing : Permission denied .sh_history a.a rsync.tst test mkstemp ..profile.KIa00D failed mkstemp ..sh_history.LIa00D failed mkstemp .a.a.MIa00D failed mkstemp .rsync.test.NIa00d failed mkstemp .test.OIa00D failed wrote 1959 bytes read 126 bytes 1930.00 bytes/sec total size is 6173 speedup is 2.96 rsync error: patrial transfer (code 23) at main.c(576) Do you know why this is only partialy transfering? Regards, Mike Lefebvre
Re: rsync dir in _both_ directions?
Big Brother tells me that Martin Pool wrote: > On 5 Feb 2002, Jack McKinney <[EMAIL PROTECTED]> wrote: > > > > Yes. blocks are 512 bytes long. If I add a 14 byte header on a message > > near the beginning, then there are no identical blocks. > > In this case there are identical blocks starting just after the > inserted header. You might not expect rsync to handle this case, but > it does! If I add 512 bytes at the begining of the file, then I would expect it. If I only add 14 bytes, then I don't think rsync will detect this, as it would require it to compute checksums start at EVERY byte instead of 512 byte checksums at offsets 0, 512, 1024, 1536, et al. It is an easy experiment. Create a large text file (e.g., uuencode a decently large binary) and rsync it to another machine and time it. Then, edit it, and change ONE BYTE and rsync again. It should go much faster because only one block has changed. Then, edit it a second time and add a new line at the top that says Kilroy was here. This will add 17 bytes to the file (13 letters, two spaces, one period, one newline). Then rsync it again. I suspect that your xfer time will be comparable to the first one, not to the second. > > It can take 5-30 minutes to finish the first, and I want it to then > > start the second one without having to watch for the first one. I prefer > > not to use ssh's feature to remember keys or to use .rhosts/.shosts. > > Make a little shell script: > > eval `ssh-agent` > ssh-add > rsync foo bar > rsync qux zump > .. > kill $SSH_AGENT_PID As I indicated, I do not want to use ssh-agent. -- Sherry: To me, humor is like medicine!Jack McKinney Niles: Guess we're in the placebo group. [EMAIL PROTECTED] 1024D/FBED2DAA 4096g/3F93879F http://www.lorentz.com 2002 Chicago Cubs magic number: 163 msg02994/pgp0.pgp Description: PGP signature
RE: [path] & module options with SSH
> > On Wed, Feb 06, 2002 at 12:04:49PM -0500, Daniel Ouellet wrote: > > So, I guess this is the answer to my question as to why the > config wasn't > > limiting the users. > > > > Great, thanks for that answer, but this still leave me with how > can I limit > > access to specific portion of the file system as I put in my > email to make > > sure the users only access what I would like them to access via > rsync other > > then having to change the right of each users on the server side of the > > connection? > > JD Paul's patch which I previously referred to will give you what > you want. So, this patch would apply only when the :: is in use correct? That's what I understand. > > Now as to -daemon and ssh not working together. You may be > right, I don't > > know. I discover rsyn just two days ago. SO, I don't know much > about it yet, > > just plenty of reading, trial and error you may say! > > > > Am I understanding you correctly when you say ssh and -daemon are not > > working together when you use the :: syntax or are you saying > that they just > > don't period regardless of : or ::? > > ":" syntax uses rsh (or ssh if you use -e ssh) to run another copy of the > rsync program on the remote side. "::" syntax skips that completely, > ignores -e, and instead connects to a daemon separately started to listen > on port 873 on the remote host. In the future, when JD Paul's patch is > accepted, the expectation will be that if you use "::" and "-e ssh" > together it will still use ssh to connect but it will run rsync -daemon > interactively so it can honor your rsyncd.conf. > > Does that make it clear? Now I re-read the documents a few more times and I did some more test. I just got the light I guess! (:> I was running a daemon on my server or source and it wasn't needed with the : syntax I used! Duh!! (:< Here again correct me if I am wrong, but the : as explain in the DOC ( but I didn't get it the first time, and the second and...) is use when your SSH remote shell will start a rsync process on the remote server and YOU DO NOT NEED a --daemon running there, but at the moment, without JD Paul's patch, the rsyncd.conf file is ignore. The :: will use the --daemon on the remote server and should accept the rsyncd.conf correctly with SSH if JD Paul's patch is apply. I think I am clear on that as well, right? At the moment if SSH is in use, regardless if you use : or ::, the rsync.conf is not use without JD Paul's patch apply, so there is no way to limit access to part of the file system under SSH other then changing the right on the user on the remote server? Many thanks for your help and answer! It put some light for me on this. Last part if I may abuse of your generosity so far. In the 2.5.2, there is the use of malloc.h that wasn't there n the 2.5.1 and will make my compiler complain and provide me plenty of warning. Any patch to use the more resent stdlib.h instead or am I crazy to ask? Thanks again, you were very helpful! Daniel
Latest version of the batch mode cleanup patch
Here's the latest version of the batch mode cleanup patch.
Index: batch.c
===
RCS file: /cvsroot/rsync/batch.c,v
retrieving revision 1.12
diff -u -r1.12 batch.c
--- batch.c 24 Jan 2002 08:09:46 - 1.12
+++ batch.c 6 Feb 2002 19:47:57 -
@@ -8,55 +8,38 @@
#include "rsync.h"
#include
-char rsync_flist_file[27] = "rsync_flist.";
-char rsync_csums_file[27] = "rsync_csums.";
-char rsync_delta_file[27] = "rsync_delta.";
-char rsync_argvs_file[27] = "rsync_argvs.";
-
-char batch_file_ext[15];
-
-int fdb;
-int fdb_delta;
-int fdb_open;
-int fdb_close;
+extern char *batch_prefix;
struct file_list *batch_flist;
-void create_batch_file_ext()
-{
- struct tm *timeptr;
- time_t elapsed_seconds;
-
- /* Save run date and time to use for batch file extensions */
- time(&elapsed_seconds);
- timeptr = localtime(&elapsed_seconds);
-
- sprintf(batch_file_ext, "%4d%02d%02d%02d%02d%02d",
- timeptr->tm_year + 1900, timeptr->tm_mon + 1,
- timeptr->tm_mday, timeptr->tm_hour, timeptr->tm_min,
- timeptr->tm_sec);
- rprintf(FINFO,"batch file extension: %s\n", batch_file_ext);
-}
-
-void set_batch_file_ext(char *ext)
-{
- strcpy(batch_file_ext, ext);
-}
+static char rsync_flist_file[] = ".rsync_flist";
+static char rsync_csums_file[] = ".rsync_csums";
+static char rsync_delta_file[] = ".rsync_delta";
+static char rsync_argvs_file[] = ".rsync_argvs";
+
+static int fdb;
+static int fdb_delta;
+static int fdb_open;
+static int fdb_close;
void write_batch_flist_file(char *buff, int bytes_to_write)
{
+ char filename[MAXPATHLEN];
if (fdb_open) {
/* Set up file extension */
- strcat(rsync_flist_file, batch_file_ext);
+ strlcpy(filename, batch_prefix, sizeof(filename));
+ strlcat(filename, rsync_flist_file, sizeof(filename));
- /* Open batch flist file for writing; create it if it doesn't exist */
- fdb =
- do_open(rsync_flist_file, O_WRONLY | O_CREAT | O_TRUNC,
+ /*
+* Open batch flist file for writing;
+* create it if it doesn't exist
+*/
+ fdb = do_open(filename, O_WRONLY | O_CREAT | O_TRUNC,
S_IREAD | S_IWRITE);
if (fdb == -1) {
rprintf(FERROR, "Batch file %s open error: %s\n",
- rsync_flist_file, strerror(errno));
+ filename, strerror(errno));
close(fdb);
exit_cleanup(1);
}
@@ -67,12 +50,11 @@
if (write(fdb, buff, bytes_to_write) == -1) {
rprintf(FERROR, "Batch file %s write error: %s\n",
- rsync_flist_file, strerror(errno));
+ filename, strerror(errno));
close(fdb);
exit_cleanup(1);
}
-
if (fdb_close) {
close(fdb);
}
@@ -110,7 +92,6 @@
}
write_char_bufs(fptr[i]->sum);
}
-
}
void write_char_bufs(char *buf)
@@ -118,11 +99,8 @@
/* Write the size of the string which will follow */
char b[4];
- if (buf != NULL)
- SIVAL(b, 0, strlen(buf));
- else {
- SIVAL(b, 0, 0);
- }
+
+ SIVAL(b, 0, buf != NULL ? strlen(buf) : 0);
write_batch_flist_file(b, sizeof(int));
@@ -137,36 +115,52 @@
{
int fdb;
int i;
- char buff[256];
-
- strcat(rsync_argvs_file, batch_file_ext);
-
-
- /* Open batch argvs file for writing; create it if it doesn't exist */
- fdb = do_open(rsync_argvs_file, O_WRONLY | O_CREAT | O_TRUNC,
+ char buff[256]; /* XXX */
+ char buff2[MAXPATHLEN + 6];
+ char filename[MAXPATHLEN];
+
+ /* Set up file extension */
+ strlcpy(filename, batch_prefix, sizeof(filename));
+ strlcat(filename, rsync_argvs_file, sizeof(filename));
+
+ /*
+* Open batch argvs file for writing;
+* create it if it doesn't exist
+*/
+ fdb = do_open(filename, O_WRONLY | O_CREAT | O_TRUNC,
S_IREAD | S_IWRITE | S_IEXEC);
if (fdb == -1) {
rprintf(FERROR, "Batch file %s open error: %s\n",
- rsync_argvs_file, strerror(errno));
+ filename, strerror(errno));
close(fdb);
exit_cleanup(1);
}
buff[0] = '\0';
+
/* Write argvs info to batch file */
for (i = 0; i < argc; ++i) {
- if (i == argc - 2)
+ if (i == argc - 2) /* Skip source directory on cmdline */
continue;
/*
* FIXME:
* I think directl
Re: SIGUSR1 or SIGINT error
Make that 2 of us who need to specify a large timeout. I have found that I have to set the timeout to a large value (1) to get the rsyncs to run successfully. Leaving it at the default seemed to cause timeout/hang problems. Of course I still running a 2.4.6dev version. I had troubles with 2.5.[01]. (solaris/linux mix of of rsync clients/servers) I need to try 2.5.2 as soon as I get a chance. Looks like some good fixes are happening in 2.5.2. eric On Wed, 2002-02-06 at 10:39, [EMAIL PROTECTED] wrote: > When i was getting these, I traced the process and its children (solaris: > truss -f). I found that one of the spawned threads was experiencing an io > timeout while the filelist was building. I had set no timeout, but it did > it at 60 seconds every time. I found that this corresponded to a > SELECT_TIMEOUT parameter, which was set to 60 if IO_TIMEOUT was 0. BY > setting my timeout to 86400 (1 day), i stopped those. Of course, then, it > choked farther along, but that's another story. > Try setting a timeout, even if you don't want one. Make it the longest > the process should ever take. > > Tim Conway > [EMAIL PROTECTED] > 303.682.4917 > Philips Semiconductor - Longmont TC > 1880 Industrial Circle, Suite D > Longmont, CO 80501 > Available via SameTime Connect within Philips, n9hmg on AIM > perl -e 'print pack(, > 19061,29556,8289,28271,29800,25970,8304,25970,27680,26721,25451,25970), > ".\n" ' > "There are some who call me Tim?" > > > > > Dave Dykstra <[EMAIL PROTECTED]> > Sent by: [EMAIL PROTECTED] > 02/06/2002 10:16 AM > > > To: David Birnbaum <[EMAIL PROTECTED]> > cc: [EMAIL PROTECTED] > (bcc: Tim Conway/LMT/SC/PHILIPS) > Subject:Re: SIGUSR1 or SIGINT error > Classification: > > > > On Tue, Feb 05, 2002 at 11:28:54AM -0500, David Birnbaum wrote: > > I suspected that might be the case...now...how to determine the "real" > > problem? Does rsync log it somewhere? lsof shows that STDERR/STDOUT > are > > going to /dev/null, so I hope it's not writing it there. Nothing > > informative in syslog, just the message about the SIG: > > > > Feb 5 09:49:41 hite rsyncd[9279]: [ID 702911 daemon.warning] rsync > error: received SIGUSR1 or SIGINT (code 20) at rsync.c(229) > > > > Any clues? > > > I'm sorry, but I don't have any more suggestions. > > - Dave Dykstra > > > > >
rsh syntax - Help
Please can someone provide me with the exact syntax for using the rsh protocol. I am trying to copy files in a directory named /opt/config/ on server DevlServ to the same directory on server ApplServ1. I have tried the following without much success: rsync --rsh= cp /opt/config/on DevlServ:/opt/config rsync rsh cp /opt/config/on DevlServ:/opt/config Both of the above results in error messages. I can rsh to server DevlServ from the UNIX command line without being requested to enter a password, so I am pretty confident that rsh is setup correctly and I have full permissions to both directories. I think I just need to get the syntax right. HELP The error message I get is as follows: config cp: cannot access DevlServcp: cannot access --servercp: .: is a directoryrsync: connection unexpectedly closed (0 bytes read so far)rsync error: error in rsync protocol data stream (code 12) at io.c(140) I would appreciate any suggestions. Thanks. Leslie Join the worlds largest e-mail service with MSN Hotmail. Click Here
Re: [path] & module options with SSH
The discussion about syntax for remote file specification and the exchange between Martin and Wayne about configure options for rsh make me wonder if we should push some alternative syntax for specifying the transport protocol to be used by rsync. I, for one, always stick to the rsync://host/module syntax when pulling from an rsync server, and have often wished that the same syntax were available when doing a push. I find the URL-style syntax easy to remember and understand, while the "::" seems much less intuitive (it actually looks perlish to me because of the way modules are specified in perl). Among other things, I notice that SUN uses the url syntax in its man pages describing NFS (they use nfs://host[:port]/pathname). So what came to mind is to have rsync recognize and use both for push and pull remote specifications of the form: rsync://host/module/file ssh://[username@]host/dir/file rsh://[username@]host/dir/file I'm not crazy about the last two, but thought of them while reading messages about ssh/rsh issues. Hmm... one problem that this wouldn't solve is the use of ssh-over-rsyncd that somebody has proposed, though. Also I'm not sure how I would handle the passing of additional options to the external transport program (what we do now with -e 'shell [OPTIONS]'). Ok, so maybe this is not so hot, but rsync:// is cool, IMHO. -- Alberto In message <[EMAIL PROTECTED]>, Dave Dykstra writes: > > Am I understanding you correctly when you say ssh and -daemon are not > > working together when you use the :: syntax or are you saying that they just > > don't period regardless of : or ::? > > ":" syntax uses rsh (or ssh if you use -e ssh) to run another copy of the > rsync program on the remote side. "::" syntax skips that completely, > ignores -e, and instead connects to a daemon separately started to listen > on port 873 on the remote host. In the future, when JD Paul's patch is > accepted, the expectation will be that if you use "::" and "-e ssh" > together it will still use ssh to connect but it will run rsync -daemon > interactively so it can honor your rsyncd.conf. > > Does that make it clear? > > > > Because, I do not have RSH, only SSH on my server and it does work for me. I > > do have to use the SSH Verion 2 as I wasn't able to do it with the version 1 > > and I use DSA not RSA. > > That doesn't matter; "::" syntax bypasses both RSH and SSH. Alberto Accomazzi mailto:[EMAIL PROTECTED] NASA Astrophysics Data System http://adsabs.harvard.edu Harvard-Smithsonian Center for Astrophysicshttp://cfawww.harvard.edu 60 Garden Street, MS 83, Cambridge, MA 02138 USA
Re: SIGUSR1 or SIGINT error
When i was getting these, I traced the process and its children (solaris: truss -f). I found that one of the spawned threads was experiencing an io timeout while the filelist was building. I had set no timeout, but it did it at 60 seconds every time. I found that this corresponded to a SELECT_TIMEOUT parameter, which was set to 60 if IO_TIMEOUT was 0. BY setting my timeout to 86400 (1 day), i stopped those. Of course, then, it choked farther along, but that's another story. Try setting a timeout, even if you don't want one. Make it the longest the process should ever take. Tim Conway [EMAIL PROTECTED] 303.682.4917 Philips Semiconductor - Longmont TC 1880 Industrial Circle, Suite D Longmont, CO 80501 Available via SameTime Connect within Philips, n9hmg on AIM perl -e 'print pack(, 19061,29556,8289,28271,29800,25970,8304,25970,27680,26721,25451,25970), ".\n" ' "There are some who call me Tim?" Dave Dykstra <[EMAIL PROTECTED]> Sent by: [EMAIL PROTECTED] 02/06/2002 10:16 AM To: David Birnbaum <[EMAIL PROTECTED]> cc: [EMAIL PROTECTED] (bcc: Tim Conway/LMT/SC/PHILIPS) Subject:Re: SIGUSR1 or SIGINT error Classification: On Tue, Feb 05, 2002 at 11:28:54AM -0500, David Birnbaum wrote: > I suspected that might be the case...now...how to determine the "real" > problem? Does rsync log it somewhere? lsof shows that STDERR/STDOUT are > going to /dev/null, so I hope it's not writing it there. Nothing > informative in syslog, just the message about the SIG: > > Feb 5 09:49:41 hite rsyncd[9279]: [ID 702911 daemon.warning] rsync error: received SIGUSR1 or SIGINT (code 20) at rsync.c(229) > > Any clues? I'm sorry, but I don't have any more suggestions. - Dave Dykstra
Re: SIGUSR1 or SIGINT error
On Tue, Feb 05, 2002 at 11:28:54AM -0500, David Birnbaum wrote: > I suspected that might be the case...now...how to determine the "real" > problem? Does rsync log it somewhere? lsof shows that STDERR/STDOUT are > going to /dev/null, so I hope it's not writing it there. Nothing > informative in syslog, just the message about the SIG: > > Feb 5 09:49:41 hite rsyncd[9279]: [ID 702911 daemon.warning] rsync error: >received SIGUSR1 or SIGINT (code 20) at rsync.c(229) > > Any clues? I'm sorry, but I don't have any more suggestions. - Dave Dykstra
Re: [path] & module options with SSH
On Wed, Feb 06, 2002 at 12:04:49PM -0500, Daniel Ouellet wrote: > So, I guess this is the answer to my question as to why the config wasn't > limiting the users. > > Great, thanks for that answer, but this still leave me with how can I limit > access to specific portion of the file system as I put in my email to make > sure the users only access what I would like them to access via rsync other > then having to change the right of each users on the server side of the > connection? JD Paul's patch which I previously referred to will give you what you want. > Now as to -daemon and ssh not working together. You may be right, I don't > know. I discover rsyn just two days ago. SO, I don't know much about it yet, > just plenty of reading, trial and error you may say! > > Am I understanding you correctly when you say ssh and -daemon are not > working together when you use the :: syntax or are you saying that they just > don't period regardless of : or ::? ":" syntax uses rsh (or ssh if you use -e ssh) to run another copy of the rsync program on the remote side. "::" syntax skips that completely, ignores -e, and instead connects to a daemon separately started to listen on port 873 on the remote host. In the future, when JD Paul's patch is accepted, the expectation will be that if you use "::" and "-e ssh" together it will still use ssh to connect but it will run rsync -daemon interactively so it can honor your rsyncd.conf. Does that make it clear? > Because, I do not have RSH, only SSH on my server and it does work for me. I > do have to use the SSH Verion 2 as I wasn't able to do it with the version 1 > and I use DSA not RSA. That doesn't matter; "::" syntax bypasses both RSH and SSH. > When I setup my box to use the daemon and have my cron on the client, I > generate my key with: > > ssh-keygen -t dsa > > Is that help. > > I am not trying to make a debate at all, I would like to understand it > right. > > You are 100% right as to the ::. That I wasn't able to and that's why in my > email I did put the : and ask if there was a problem with SSH and rsync. > > I can tell you that it does work with : but not with ::, that I knock my > head on it a lot two night ago! So, do you provide me the answer here as to > why. Thanks! > > Finally, when you say that the -daemon doesn't see the rsync.conf, you mean > the module portions right? As I just test it and if I change: > > log file = /var/log/rsyncd.log > pid file = /var/run/rsyncd.pid > lock file = /var/run/rsync.lock > read only=yes > uid=nobody > gid=nobody > > to > > log file = /var/log/rsyncd.logss > pid file = /var/run/rsyncd.pid > lock file = /var/run/rsync.lock > read only=yes > uid=nobody > gid=nobody > > for the test, then the log file use rsyncd.logss and I see inside: > > webfarm1# more /var/log/rsyncd.logss > 2002/02/06 11:57:30 [12838] rsyncd version 2.5.2 starting, listening on port > 873 > > So, I am not sure that I follow up 100% yet. The --daemon is seeing the rsyncd.conf, but the client never talks to that process if you're using the ":" syntax. - Dave Dykstra
RE: [path] & module options with SSH
So, I guess this is the answer to my question as to why the config wasn't limiting the users. Great, thanks for that answer, but this still leave me with how can I limit access to specific portion of the file system as I put in my email to make sure the users only access what I would like them to access via rsync other then having to change the right of each users on the server side of the connection? Now as to -daemon and ssh not working together. You may be right, I don't know. I discover rsyn just two days ago. SO, I don't know much about it yet, just plenty of reading, trial and error you may say! Am I understanding you correctly when you say ssh and -daemon are not working together when you use the :: syntax or are you saying that they just don't period regardless of : or ::? Because, I do not have RSH, only SSH on my server and it does work for me. I do have to use the SSH Verion 2 as I wasn't able to do it with the version 1 and I use DSA not RSA. When I setup my box to use the daemon and have my cron on the client, I generate my key with: ssh-keygen -t dsa Is that help. I am not trying to make a debate at all, I would like to understand it right. You are 100% right as to the ::. That I wasn't able to and that's why in my email I did put the : and ask if there was a problem with SSH and rsync. I can tell you that it does work with : but not with ::, that I knock my head on it a lot two night ago! So, do you provide me the answer here as to why. Thanks! Finally, when you say that the -daemon doesn't see the rsync.conf, you mean the module portions right? As I just test it and if I change: log file = /var/log/rsyncd.log pid file = /var/run/rsyncd.pid lock file = /var/run/rsync.lock read only=yes uid=nobody gid=nobody to log file = /var/log/rsyncd.logss pid file = /var/run/rsyncd.pid lock file = /var/run/rsync.lock read only=yes uid=nobody gid=nobody for the test, then the log file use rsyncd.logss and I see inside: webfarm1# more /var/log/rsyncd.logss 2002/02/06 11:57:30 [12838] rsyncd version 2.5.2 starting, listening on port 873 So, I am not sure that I follow up 100% yet. I would appreciate if you would take a bit of your time to put light on this for me if you have time of course! Many thanks for your help! Daniel -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]]On Behalf Of Dave Dykstra Sent: Wednesday, February 06, 2002 9:56 AM To: Daniel Ouellet Cc: [EMAIL PROTECTED] Subject: Re: [path] & module options with SSH rsync --daemon and ssh do not currently work together at all. If you use the syntax "hostname::module", rsync ignores the '-e ssh'. It sounds like you are not using the double-colon syntax so rsyncd.conf is ignored. Someone has posted a patch that enable the two to work together but it hasn't yet been integrated. - Dave Dykstra On Tue, Feb 05, 2002 at 08:45:52PM -0500, Daniel Ouellet wrote: > Hi, > > I am running rsync 2.5.2 and have a server running rsync --daemon over ssh. > > Now, I read plenty of information on rsync, all the man & all the info on > the rsync.samba.org + many other sites that for the most part all say the > same with a few exceptions. Did search on google and even look at the marc > lists. What I am looking for, unless I do not understand it and please > correct me if that's the case, it look like that the module section would be > use to limit the access to specific users to a limit part of the files > system. Example, if I have: > > [simple_path_name] >path = /rsync/files/here >comment = My Very Own Rsync Server >uid = nobody >gid = nobody >read only = no >list = yes >auth users = username >secrets file = /etc/rsyncd.secrets > > Then I would expect the users ( username) to have access only to my files > inside the /rsync/files/here, but with ssh anyway, you have access to all of > it and only the user rights on the server will limit you. I don't know under > RSH if that's the same or not as I only have servers with SSH, so I didn't > test that part out. > > Also, the hosts allow option if I put it in the global section to limit > access to my box, is not recognize either. > > I can have: hosts allow 192.168.2.2 and obviously this is not a routable IP > and it is not my IP either, but I will still have access to the rsync > server. > > So, in short, is there a place that would list the options that are not > active under SSH? > > What I want to do is to limit access to some IP's, or block of IP under SSH > and also limit access to a portion of the files system as above. I wouldn't > mind if the auth users is not used as SSH is find for that, but I sure would > love if I could have limit on the files system and as a bonus if the auth > users would recognize the user that sign up via SSH and provide access to > the path only. Obviously the secrets file wouldn't be use in the SSH case, > but would be use only to limit access per users. > > So, is that make sense or am I way off i
Re: configure --with-rsh=CMD and default blocking-IO support
On Tue, Feb 05, 2002 at 09:38:04PM -0800, Wayne Davison wrote: > On Wed, 6 Feb 2002, Martin Pool wrote: > > OK, I agree --with-rsh should go in, but I think putting magic > > characters into it is needlessly confusing. I would feel much better > > about a separate configure option to set the default O_NONBLOCK mode. > > The complicating factor then becomes: how does the RSYNC_RSH environment > variable interact with this default O_NONBLOCK mode, and how can the > default blocking be changed via the environment? I came up with the > magic character idea in order to try to keep things simple (using only > one environment variable instead of trying to keep two different ones in > sync). I admit that it's quirky, though. > > So the obvious alternative is something like this: > > export RSYNC_RSH=ssh > export RSYNC_BLOCKING_IO=1 > > Perhaps a better idiom might be allow RSYNC_RSH to begin with a > command-line option? If the string begins with "--blocking-io " we > strip it off and twiddle that command-line flag? If we want to make > this orthogonal we could also add support for the --non-blocking-io > command-line option and allow this string to appear at the start of > the RSYNC_RSH value. What do you think of something like this? > > export RSYNC_RSH='--blocking-io /usr/bin/ssh -l username' > export RSYNC_RSH='--non-blocking-io rsh' Of the proposed alternatives, I like this latter the best, changing --non-blocking-io to --no-blocking-io. I added --no-blocking-io and --no-whole-file command line options yesterday. - Dave Dykstra
Re[2]: [Fwd: Re: meaning of "IO Error: skipping the delete...."]]
Nitin Agarwal <[EMAIL PROTECTED]> wrote: > Dear Mr. Rusty, > Thanks for the reply. The problem was sorted out by changing the uid option in > rsyncd.conf file to root. > We are facing two more problem now > 1) while transferring the files, sometimes the transfer breaks in between and gives > us the error message: "readerror: connection reset by peer" I know that there's a patch someone submitted a while back that may (or may not) fix that one. I don't remember where it is, so I'm sending this to the list also in hopes that this will jog someone's memory ;-) > 2) sometimes it shows error message: "Erroring writing 4092 bytes - exiting". It may be related to (1) above (that's what I'd guess first, anyway). Again, hopefully someone on the list (probably the patch author) will chime in with the info. > Thanks & Regards > > We are having a large file system (approx. 100 GB). Oh - now that's an interesting fact. Do you perhaps have LOTS of little files, or a few really big files? I recall that there's a possible issue of one side deciding that the other side had gone away because that other side was busy calculating and did not reply in time. There's a timeout value somewhere that you can make longer that may fix the problem also - I'd try that first. Unfortunately I'm not sure of the name of the config option - check the documents and also look in the config file itself... > Regards > Nitin Agarwal Hope this helps! rc
Re: 2.5.2 will not compile
Several people have come up with fixes for that, and I'm not sure which one is the best. Here's mine: --- rsync.h.O Mon Jan 28 08:57:17 2002 +++ rsync.h Mon Jan 28 08:57:31 2002 @@ -303,8 +303,8 @@ * cope with platforms on which this is an unsigned int or even a * struct. Later. */ -#define INO64_T unsigned int64 -#define DEV64_T unsigned int64 +#define INO64_T int64 +#define DEV64_T int64 #ifndef MIN #define MIN(a,b) ((a)<(b)?(a):(b)) - Dave Dykstra On Tue, Feb 05, 2002 at 03:36:47PM -0500, Richard Ureña wrote: > Trying to compile rsync 2.5.2 after "./configure --prefix=/usr" > I get the following make errors: > >gcc -I. -I. -g -O2 -DHAVE_CONFIG_H -Wall -W -c rsync.c -o rsync.o >In file included from rsync.c:23: >rsync.h:339: warning: no semicolon at end of struct or union >rsync.h:339: parse error before `inode' >rsync.h:341: parse error before `dev' >rsync.h:341: warning: type defaults to `int' in declaration of `dev' >rsync.h:341: warning: data definition has no type or storage class >rsync.h:344: parse error before `rdev' >rsync.h:344: warning: type defaults to `int' in declaration of `rdev' >rsync.h:344: warning: data definition has no type or storage class >rsync.h:347: `basename' redeclared as different kind of symbol >/usr/include/string.h:317: previous declaration of `basename' >rsync.h:350: `link' redeclared as different kind of symbol >/usr/include/unistd.h:678: previous declaration of `link' >rsync.h:352: parse error before `}' >rsync.h: In function `flist_up': >rsync.h:420: dereferencing pointer to incomplete type >rsync.c: In function `set_perms': >rsync.c:165: dereferencing pointer to incomplete type >rsync.c:168: dereferencing pointer to incomplete type >rsync.c:178: dereferencing pointer to incomplete type >rsync.c:179: dereferencing pointer to incomplete type >rsync.c:179: dereferencing pointer to incomplete type >rsync.c:184: dereferencing pointer to incomplete type >rsync.c:188: dereferencing pointer to incomplete type >rsync.c:189: dereferencing pointer to incomplete type >rsync.c:206: dereferencing pointer to incomplete type >rsync.c:208: dereferencing pointer to incomplete type >rsync.c:152: warning: `change_uid' might be used uninitialized in this function >rsync.c:152: warning: `change_gid' might be used uninitialized in this function >rsync.c: In function `finish_transfer': >rsync.c:245: dereferencing pointer to incomplete type >make: *** [rsync.o] Error 1 > > Some info about the build environment: > > uname -sr >Linux 2.2.20 > > gcc --version >egcs-2.91.66 > > ls -d /lib/libc* >/lib/libc-2.1.2.so >/lib/libc.so.6 >/lib/libcrypt.so.1 > > make --version >GNU Make version 3.77, by Richard Stallman and Roland McGrath. > > Output of "configure" appended below. > > I'd like to be able to upgrade to the new version due to the > security concerns.Any help will be greatly appreciated. > > Regards, > > Richard > > (output of "./configure --prefix=/usr 2>&1") > >configure: Configuring rsync 2.5.2 >checking build system type... i686-pc-linux-gnu >checking host system type... i686-pc-linux-gnu >checking target system type... i686-pc-linux-gnu >checking for gcc... gcc >checking for C compiler default output... a.out >checking whether the C compiler works... yes >checking whether we are cross compiling... no >checking for executable suffix... >checking for object suffix... o >checking whether we are using the GNU C compiler... yes >checking whether gcc accepts -g... yes >checking how to run the C preprocessor... gcc -E >checking for a BSD compatible install... /usr/bin/ginstall -c >checking for gcc option to accept ANSI C... none needed >checking whether to include debugging symbols... yes >checking for remsh... 0 >checking for broken largefile support... yes >checking ipv6 stack type... linux-glibc >checking for library containing getaddrinfo... none required >checking whether byte ordering is bigendian... no >checking for dirent.h that defines DIR... yes >checking for opendir in -ldir... no >checking whether time.h and sys/time.h may both be included... yes >checking for sys/wait.h that is POSIX.1 compatible... yes >checking for sys/fcntl.h... yes >checking for sys/select.h... yes >checking for fcntl.h... yes >checking for sys/time.h... yes >checking for sys/unistd.h... yes >checking for unistd.h... yes >checking for utime.h... yes >checking for grp.h... yes >checking for compat.h... no >checking for sys/param.h... yes >checking for ctype.h... yes >checking for sys/wait.h... (cached) yes >checking for sys/ioctl.h... yes >checking for sys/filio.h... no >checking for string.h... yes >checking for stdlib.h... yes >
Re: rsync usage... and I did RTFM
Sorry, but that advice is wrong. Daniel's using the single colon so his rsyncd.conf is being ignored. Ross needs JD Paul's patch which hasn't bee integrated yet: http://lists.samba.org/pipermail/rsync/2001-December/005488.html - Dave On Tue, Feb 05, 2002 at 11:58:55PM -0500, Daniel Ouellet wrote: > Sure you can. Unless you are looking for a log that I don't know about. > > All my server only have ssh and not only the log is written to a file on the > /var/log/rsyncd.log, but I did set it up to email me a copy of the transfer > activity. > > My crontab have: > > 15 * * * * /usr/local/bin/rsync -e > ssh -auvz --exclude "/*/logs/" --delete youruser@yourserver:/sites/ /sites > 2>&1 | tee /var/log/rsync.out | mail -s "`/bin/hostname` rsync update" root > > I run the server in daemon mode. > > The server side is: > > log file = /var/log/rsyncd.log > pid file = /var/run/rsyncd.pid > lock file = /var/run/rsync.lock > read only=yes > uid=nobody > gid=nobody > > And I setup my DSA key on the server to be trusted so that the cron don't > have to enter the password at every time. > > Finally there is two great articles on how to setup the SSH side of it with > the passphare instead of empty one. > > http://www-106.ibm.com/developerworks/library/l-keyc.html > http://www-106.ibm.com/developerworks/library/l-keyc2/ > > Hope this help. > > Daniel > > -Original Message- > From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]]On > Behalf Of Ross Becker > Sent: Tuesday, February 05, 2002 11:48 AM > To: '[EMAIL PROTECTED]' > Subject: rsync usage... and I did RTFM > > > Hey folks, >I've been through the rsync documentation, and to the best of my ability, > I dont see how to do exactly what I wish to do. I'd go through the mailing > list archives, but there doesnt seem to be a search function. > > Essentially, I need to rsync securely between some sites, but I want logs > written. The problem is, it appears that using rsyncd is the only way to > get logs, but if you use rsyncd, there's no way to use ssh for secure > transfers. Is there any way to both get logs written and use ssh at the same > time? > > > Thanks > --Ross Becker >
Re: [path] & module options with SSH
rsync --daemon and ssh do not currently work together at all. If you use the syntax "hostname::module", rsync ignores the '-e ssh'. It sounds like you are not using the double-colon syntax so rsyncd.conf is ignored. Someone has posted a patch that enable the two to work together but it hasn't yet been integrated. - Dave Dykstra On Tue, Feb 05, 2002 at 08:45:52PM -0500, Daniel Ouellet wrote: > Hi, > > I am running rsync 2.5.2 and have a server running rsync --daemon over ssh. > > Now, I read plenty of information on rsync, all the man & all the info on > the rsync.samba.org + many other sites that for the most part all say the > same with a few exceptions. Did search on google and even look at the marc > lists. What I am looking for, unless I do not understand it and please > correct me if that's the case, it look like that the module section would be > use to limit the access to specific users to a limit part of the files > system. Example, if I have: > > [simple_path_name] >path = /rsync/files/here >comment = My Very Own Rsync Server >uid = nobody >gid = nobody >read only = no >list = yes >auth users = username >secrets file = /etc/rsyncd.secrets > > Then I would expect the users ( username) to have access only to my files > inside the /rsync/files/here, but with ssh anyway, you have access to all of > it and only the user rights on the server will limit you. I don't know under > RSH if that's the same or not as I only have servers with SSH, so I didn't > test that part out. > > Also, the hosts allow option if I put it in the global section to limit > access to my box, is not recognize either. > > I can have: hosts allow 192.168.2.2 and obviously this is not a routable IP > and it is not my IP either, but I will still have access to the rsync > server. > > So, in short, is there a place that would list the options that are not > active under SSH? > > What I want to do is to limit access to some IP's, or block of IP under SSH > and also limit access to a portion of the files system as above. I wouldn't > mind if the auth users is not used as SSH is find for that, but I sure would > love if I could have limit on the files system and as a bonus if the auth > users would recognize the user that sign up via SSH and provide access to > the path only. Obviously the secrets file wouldn't be use in the SSH case, > but would be use only to limit access per users. > > So, is that make sense or am I way off in the understanding of the system > and the intention behind the module portion. > > As a last question, this is not a big deal, but I was curious as if anyone > would know of an option that would only send out the actual name&path of the > files transfer without the final small stats and not show the portion where > you get the initial directory list. The reason why I am asking is that if > so, I could plug the output of it to MySQL and log the files transfer for > audit reason. > > Many thanks for your help and time! > > Daniel > > >
Re: rsync error: unexplained error (code -1) at main.c(842)
I've not seen that one before, but I tried to trace it in the source code. A -1 from main.c line 842 means that start_client() returned a -1 which means that start_socket_client returned -1 and there are several things that can cause that. Mostly they seem to be if the server doesn't do its part of the protocol properly. The client error message should definitely be more descriptive. Is there anything more descriptive in /var/log/rsyncd.log on the server? The real problem is probably on the server. - Dave Dykstra On Tue, Feb 05, 2002 at 09:52:52AM -0500, MICHAEL R. LEFEBVRE wrote: > I'm running rsync 2.5.1 on solaris 7 in server mode and getting the > above message when entering the following command: > > rsync -vraz --delete hostname::test ./ > > this is my rsyncd.conf > > use chroot = no > max connections = 25 > syslog facility = local 3 > read only = false > > log file = /var/log/rsyncd.log > pid file = /var/log/rsyncd.pid > lock file = /var/log/rsyncd.lock > > [test] > path = /export/home/mrl315 > comment = rsync test > hosts allow = hostname > secrets file = /etc/rsyncd.secrets > > The rsyncd.conf and rsyncd.secrets are owned by root:other and have > read and write permission only for root. > > I have setup an rsync server idential to this one on an ultra 10 (this > one is a Sun Enterprise 250) and it works fine. > > Regards, > > Mike Lefebvre
Re: rsync with openssh
On Tue, Feb 05, 2002 at 09:17:16AM -0500, Jim Ogilvie wrote: > We've used rsync with an old ssh for a couple of years with no problems. > Now trying to switch to openssh 3.0.2p1 so we can use protocol 2. Rsync > makes > the connection via ssh and does it's work. The problem is after rsync runs > and exits > on the server side the sshd session keeps running and isn't closing the > connection. > So the rsync client hangs waiting for the connection to end. > Is anyone else having this problem? Know of a solution? OpenSSH has a thoroughly-discussed difference between ssh 1.2.* of hanging in some situations where server-side subprocesses haven't exitted and have left some file descriptors open. I've never heard anybody saying that this has happened to them with rsync, however. Tell us more about your situation: what versions of rsync, what operating system, what filesystem types, what rsync command line, what your data is like. When it hangs, see if there are any subprocesses still running on the server. - Dave Dykstra
Re: configure --with-rsh=CMD and default blocking-IO support
On Wed, 6 Feb 2002, Martin Pool wrote: > OK, I agree --with-rsh should go in, but I think putting magic > characters into it is needlessly confusing. I would feel much better > about a separate configure option to set the default O_NONBLOCK mode. The complicating factor then becomes: how does the RSYNC_RSH environment variable interact with this default O_NONBLOCK mode, and how can the default blocking be changed via the environment? I came up with the magic character idea in order to try to keep things simple (using only one environment variable instead of trying to keep two different ones in sync). I admit that it's quirky, though. So the obvious alternative is something like this: export RSYNC_RSH=ssh export RSYNC_BLOCKING_IO=1 Perhaps a better idiom might be allow RSYNC_RSH to begin with a command-line option? If the string begins with "--blocking-io " we strip it off and twiddle that command-line flag? If we want to make this orthogonal we could also add support for the --non-blocking-io command-line option and allow this string to appear at the start of the RSYNC_RSH value. What do you think of something like this? export RSYNC_RSH='--blocking-io /usr/bin/ssh -l username' export RSYNC_RSH='--non-blocking-io rsh' Or can you think of a better way to go? ..wayne..
Re: Error from rsync-2.5.2
On Tue, Feb 05, 2002 at 12:31:02PM -0800, Jennifer Lu wrote: > I have compiled the new version 2.5.2 rsync in our servers, > then ran rsync last night, there were some files not copied > to destination server, both source and destination servers are > running 2.5.2, I got the following error messages, can you > please let me know what would be caused the errors? I copied > back 2.3.2 version on both servers, and rsync went well. > > Thanks for the help, here is the errors: > > ch_what > Warning: unexpected read size of 0 in map_ptr > Warning: unexpected read size of 0 in map_ptr > collect_tests > Warning: unexpected read size of 0 in map_ptr > Warning: unexpected read size of 0 in map_ptr Dave Dykstra created a patch for this. See: http://lists.samba.org/pipermail/rsync/2002-January/005966.html -- albert chin ([EMAIL PROTECTED])
Error in RSYNC
Dear Sir, We encountered one more error while running 'rsync' on our data. The error is : "Erroring writing 4092 bytes - exiting" Kindly let us know the reason and solution for the same. Thanks & Regards Nitin Agarwal
One More error in RSYNC
Dear Mr. Martin, We encountered one more error while running 'rsync' on our data. The error is : "Erroring writing 4092 bytes - exiting" Kindly let us know the reason and solution for the same. Thanks & Regards Nitin Agarwal
Re: rsync dir in _both_ directions?
On 4 Feb 2002, Jack McKinney <[EMAIL PROTECTED]> wrote: > Well, I read my email on my laptop, and it is sometimes necessary to > login to a central server to read my email (firewall issues). What I do is > to peridocally sync with the central server (which we'll call "server") by: > > rsync -e ssh -rptlv Mail server:. > rsync -e ssh -rptlv server:Mail . I think the problem is not so much that it takes two commands, but rather that there is domain-specific merge knowledge that rsync does not have. With your commands, all messages will end up on both machines, which is fine to start with. But if you delete a message it will presumably keep on being copied back from the other machine. I don't know a good way for rsync to distinguish between the case of a message deleted on machine A, and a message newly added on machine B. If you use --update, and only flag files rather than deleting them then I guess you would be OK. -- Martin
Re: rsync dir in _both_ directions?
On 5 Feb 2002, Jack McKinney <[EMAIL PROTECTED]> wrote: > Big Brother tells me that Dave Dykstra wrote: > > On Mon, Feb 04, 2002 at 11:58:04AM -0600, Jack McKinney wrote: > > > If I change the status on the first message in the box (reply to it, delete > > > it), then the ENTIRE mailbox is re-copied. Not so in maildir format. > > > > Are you sure about that? Rsync is able to locate identical blocks that > > are in a different position. > > Yes. blocks are 512 bytes long. If I add a 14 byte header on a message > near the beginning, then there are no identical blocks. In this case there are identical blocks starting just after the inserted header. You might not expect rsync to handle this case, but it does! > It can take 5-30 minutes to finish the first, and I want it to then > start the second one without having to watch for the first one. I prefer > not to use ssh's feature to remember keys or to use .rhosts/.shosts. Make a little shell script: eval `ssh-agent` ssh-add rsync foo bar rsync qux zump .. kill $SSH_AGENT_PID As before, you type your unlock passphrase exactly once; on conclusion of the script it is gone from memory. -- Martin
RE: rsync usage... and I did RTFM
Sure you can. Unless you are looking for a log that I don't know about. All my server only have ssh and not only the log is written to a file on the /var/log/rsyncd.log, but I did set it up to email me a copy of the transfer activity. My crontab have: 15 * * * * /usr/local/bin/rsync -e ssh -auvz --exclude "/*/logs/" --delete youruser@yourserver:/sites/ /sites 2>&1 | tee /var/log/rsync.out | mail -s "`/bin/hostname` rsync update" root I run the server in daemon mode. The server side is: log file = /var/log/rsyncd.log pid file = /var/run/rsyncd.pid lock file = /var/run/rsync.lock read only=yes uid=nobody gid=nobody And I setup my DSA key on the server to be trusted so that the cron don't have to enter the password at every time. Finally there is two great articles on how to setup the SSH side of it with the passphare instead of empty one. http://www-106.ibm.com/developerworks/library/l-keyc.html http://www-106.ibm.com/developerworks/library/l-keyc2/ Hope this help. Daniel -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]]On Behalf Of Ross Becker Sent: Tuesday, February 05, 2002 11:48 AM To: '[EMAIL PROTECTED]' Subject: rsync usage... and I did RTFM Hey folks, I've been through the rsync documentation, and to the best of my ability, I dont see how to do exactly what I wish to do. I'd go through the mailing list archives, but there doesnt seem to be a search function. Essentially, I need to rsync securely between some sites, but I want logs written. The problem is, it appears that using rsyncd is the only way to get logs, but if you use rsyncd, there's no way to use ssh for secure transfers. Is there any way to both get logs written and use ssh at the same time? Thanks --Ross Becker
