Re: rsync 2.5.5 fails to kernel.org
On Wed, May 01, 2002 at 03:58:26PM -0500, Dave Dykstra wrote: I tried the same command using rsync 2.5.5 on a Solaris 7 machine and it got around 860MB before I cancelled it. Is this one still a problem for you? I was copying to an empty directory and using a different operating system, but at least it rules out a widespread general problem. I haven't seen anybody else reporting a similar problem. Perhaps it was a temporary problem on rsync.kernel.org. So sorry for not following this up. The problem was complicated by tsocks 1.7 and the socks5 firewall (Aventail) used here. A few weeks ago we switched from socks5 to Cisco PIX. After I disabled tsocks everything Just Worked(tm). A best guess, it's an rsync + socks5 issue maybe. To many variables to tell and no way to replicate it. -- Regards, Scott Russell ([EMAIL PROTECTED]) Linux Technology Center, System Admin, RHCE. T/L 441-9289 / External 919-543-9289 http://bzimage.raleigh.ibm.com/webcam -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.tuxedo.org/~esr/faqs/smart-questions.html
rsync 2.5.5 fails to kernel.org
Greets. I thought I saw someone asking for problem reports against 2.5.5. Red Hat 7.1 + errata with rsync 2.5.5 from tgz sources. I use the rsync command: rsync -rltv --partial --delete --delete-excluded \ rsync://rsync.kernel.org/pub/linux/kernel/ \ /var/ftp/pub/mirrors/kernel.org/ I get the following error: rsync: connection unexpectedly closed (1493971 bytes read so far) rsync error: error in rsync protocol data stream (code 12) at io.c(151) rsync: error writing 4092 unbuffered bytes - exiting: Broken pipe rsync error: error in rsync protocol data stream (code 12) at io.c(464) Here's the results from another run: rsync: connection unexpectedly closed (1493991 bytes read so far) rsync error: error in rsync protocol data stream (code 12) at io.c(151) rsync: connection unexpectedly closed (1493971 bytes read so far) rsync error: error in rsync protocol data stream (code 12) at io.c(151) And yet another... rsync: connection unexpectedly closed (1502394 bytes read so far) rsync error: error in rsync protocol data stream (code 12) at io.c(151) rsync: connection unexpectedly closed (1502374 bytes read so far) rsync error: error in rsync protocol data stream (code 12) at io.c(151) Anyway, it's pretty consistent. I'm not sure which version of rsync kernel.org is running but I know the recommend a 2.4.x release on their web site. -- Regards, Scott Russell ([EMAIL PROTECTED]) Linux Technology Center, System Admin, RHCE. T/L 441-9289 / External 919-543-9289 http://bzimage.raleigh.ibm.com/webcam -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.tuxedo.org/~esr/faqs/smart-questions.html
Re: rsync processes never die
Martin - Thanks. I'll grab it and give it a try. -- Scott On Mon, Mar 11, 2002 at 01:54:41PM +1100, Martin Pool wrote: On 7 Mar 2002, Scott Russell [EMAIL PROTECTED] wrote: I'm using rsync 2.5.2 but have seen this with 2.5.1. My system is Red Hat 7.2 with the current errata updates. Using rsyncd with xinetd it seems as if old processes never go away. For example right now I have 3 open rsync connections according to netstat -nap but 30 rsync --daemon processes going! I think this might be fixed in 2.5.3pre: * Fix for rsync server processes hanging around after the client unexpectedly disconnects. (Colin Walters) (Debian bug #128632) Please let us know if that does not fix the problem. -- Martin -- Regards, Scott Russell ([EMAIL PROTECTED]) Linux Technology Center, System Admin, RHCE. T/L 441-9289 / External 919-543-9289 http://bzimage.raleigh.ibm.com/webcam -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.tuxedo.org/~esr/faqs/smart-questions.html
Re: rsync processes never die
On Mon, Mar 11, 2002 at 01:54:41PM +1100, Martin Pool wrote: On 7 Mar 2002, Scott Russell [EMAIL PROTECTED] wrote: I'm using rsync 2.5.2 but have seen this with 2.5.1. My system is Red Hat 7.2 with the current errata updates. Using rsyncd with xinetd it seems as if old processes never go away. For example right now I have 3 open rsync connections according to netstat -nap but 30 rsync --daemon processes going! I think this might be fixed in 2.5.3pre: * Fix for rsync server processes hanging around after the client unexpectedly disconnects. (Colin Walters) (Debian bug #128632) Please let us know if that does not fix the problem. Bah. [root@ftp3 rsync-2.5.3pre1]# autoconf FATAL ERROR: Autoconf version 2.52 or higher is required for this script [root@ftp3 rsync-2.5.3pre1]# rpm -q autoconf autoconf-2.13-10 This is on Red Hat 7.1. or 7.2. Looks like I'll have to wait a while on this update. Thanks anyway. -- Regards, Scott Russell ([EMAIL PROTECTED]) Linux Technology Center, System Admin, RHCE. T/L 441-9289 / External 919-543-9289 http://bzimage.raleigh.ibm.com/webcam -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.tuxedo.org/~esr/faqs/smart-questions.html
Re: Compile 2.5.2 on RedHat 6.2
For what it's worth I saw the same error. 2.5.1 worked fine however. I never found a solution. -- Scott In mailinglists.external.rsync, you wrote: I am running RedHat 6.2 on many of my servers and am receiving the following errors when compiling Rsync 2.5.2: [root@wrapguy rsync-2.5.2]# make gcc -I. -I. -g -O2 -DHAVE_CONFIG_H -Wall -W -c rsync.c -o rsync.o In file included from rsync.c:23: rsync.h:339: warning: no semicolon at end of struct or union rsync.h:339: parse error before `inode' rsync.h:341: parse error before `dev' rsync.h:341: warning: type defaults to `int' in declaration of `dev' rsync.h:341: warning: data definition has no type or storage class rsync.h:344: parse error before `rdev' rsync.h:344: warning: type defaults to `int' in declaration of `rdev' rsync.h:344: warning: data definition has no type or storage class rsync.h:347: `basename' redeclared as different kind of symbol /usr/include/string.h:317: previous declaration of `basename' rsync.h:350: `link' redeclared as different kind of symbol /usr/include/unistd.h:678: previous declaration of `link' rsync.h:352: parse error before `}' rsync.h: In function `flist_up': rsync.h:420: dereferencing pointer to incomplete type rsync.c: In function `set_perms': rsync.c:165: dereferencing pointer to incomplete type rsync.c:168: dereferencing pointer to incomplete type rsync.c:178: dereferencing pointer to incomplete type rsync.c:179: dereferencing pointer to incomplete type rsync.c:179: dereferencing pointer to incomplete type rsync.c:184: dereferencing pointer to incomplete type rsync.c:188: dereferencing pointer to incomplete type rsync.c:189: dereferencing pointer to incomplete type rsync.c:206: dereferencing pointer to incomplete type rsync.c:208: dereferencing pointer to incomplete type rsync.c:152: warning: `change_uid' might be used uninitialized in this function rsync.c:152: warning: `change_gid' might be used uninitialized in this function rsync.c: In function `finish_transfer': rsync.c:245: dereferencing pointer to incomplete type make: *** [rsync.o] Error 1 -- Regards, Scott Russell ([EMAIL PROTECTED]) Linux Technology Center, System Admin, RHCE. T/L 441-9289 / External 919-543-9289 http://bzimage.raleigh.ibm.com/webcam -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.tuxedo.org/~esr/faqs/smart-questions.html
rsync processes never die
Greets. I'm using rsync 2.5.2 but have seen this with 2.5.1. My system is Red Hat 7.2 with the current errata updates. Using rsyncd with xinetd it seems as if old processes never go away. For example right now I have 3 open rsync connections according to netstat -nap but 30 rsync --daemon processes going! All of them show: [root@root]$ ps ax | grep rsync 24950 ?S 0:05 rsync --daemon 25930 ?S 0:31 rsync --daemon ... Any clues as to what's going on? Any way I can get more info about this issue? Nothing is logged into my /var/log/rsyncd.log that is helpful. Oh yeah, the only options in my /etc/rsyncd.conf are timeout 600 and the logging directive. Thanks. -- Regards, Scott Russell ([EMAIL PROTECTED]) Linux Technology Center, System Admin, RHCE. T/L 441-9289 / External 919-543-9289 http://bzimage.raleigh.ibm.com/webcam -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.tuxedo.org/~esr/faqs/smart-questions.html
Re: Excludes in /etc/rsyncd.conf
On Wed, Oct 31, 2001 at 01:38:15PM -0800, Jason Helfman wrote: Thought that you are able to add a simple - + with paths or file names at the bottom of a file and run an rsync and those are then recognized inclusions or exclusions from the synchronization You can do this but keep in mind that the file is read top to bottom. The first matching + / - rule is the one operated upon and the others below it are ignored. -- Regards, Scott Russell ([EMAIL PROTECTED]) Linux Technology Center, System Admin, RHCE. T/L 441-9289 / External 919-543-9289 http://bzimage.raleigh.ibm.com/webcam
rsyncd log file option and logrotate
Greets. I'm running rsyncd out of xinetd and will be logging to /var/log/rsyncd.log. Is there anything special I need to do with rsync if I use logrotate to manage the rsyncd.log files? Do I need to 'restart' the rsyncd process if one is running and the log gets rotated? -- Regards, Scott Russell ([EMAIL PROTECTED]) Linux Technology Center, System Admin, RHCE. T/L 441-9289 / External 919-543-9289 http://bzimage.raleigh.ibm.com/webcam
Re: rsyncd log file option and logrotate
David - No, no problem, yet. Just trying to avoid problems. :) I think the question really boils down to what happens if the logfile rsyncd is writing to gets moved out from under it? Maybe I should just run rsyncd in standalone mode and use /sbin/kill -HUP to restart it after moving the logfile. -- Scott On Wed, Jun 13, 2001 at 03:32:49PM -0500, Dave Dykstra wrote: On Wed, Jun 13, 2001 at 04:03:11PM -0400, Scott Russell wrote: Greets. I'm running rsyncd out of xinetd and will be logging to /var/log/rsyncd.log. Is there anything special I need to do with rsync if I use logrotate to manage the rsyncd.log files? Do I need to 'restart' the rsyncd process if one is running and the log gets rotated? I haven't used logrotate, but modified rsync some time ago to have it close and re-open the log file on every connection, to allow the log file to be moved away when running as a stand-alone background daemon. When running from xinetd you're re-starting rsync completely on every connection so it would have worked even without my modification. Each individual connection is pretty short lived so you shouldn't have to worry about any running rsync daemon processes. Are you having a problem? - Dave Dykstra -- Regards, Scott Russell ([EMAIL PROTECTED]) Linux Technology Center, System Admin, RHCE. T/L 441-9289 / External 919-543-9289 http://bzimage.raleigh.ibm.com/webcam
Re: rsyncd log file option and logrotate
On Wed, Jun 13, 2001 at 03:54:31PM -0500, Dave Dykstra wrote:
On Wed, Jun 13, 2001 at 04:38:52PM -0400, Scott Russell wrote:
David -
No, no problem, yet. Just trying to avoid problems. :) I think the question
really boils down to what happens if the logfile rsyncd is writing to gets
moved out from under it?
If it is during an open connection, it will keep the old file open from
wherever it is moved to and write out there. That's assuming it's the
same filesystem;
This is good news. Based on this a logrotate script like should work:
/var/log/rsyncd.log {
rotate 6
monthly
compress
missingok
}
--
Regards,
Scott Russell ([EMAIL PROTECTED])
Linux Technology Center, System Admin, RHCE.
T/L 441-9289 / External 919-543-9289
http://bzimage.raleigh.ibm.com/webcam
Feature Req: multiple include / exclude from statments in rsyncd.conf
All - I would like to see a patch so that multiple include from and exclude from statments can be used withing the rsyncd.conf file. Here's an example of what I'm doing that illustrates the need for this feature. [60GB] path = /home/ftp/pub comment = Basic mirror. Maximum of 60GB in size. include from = /etc/rsync.d/60gb.conf [90GB] path = /home/ftp/pub comment = Basic mirror. Maximum of 90GB in size. include from = /etc/rsync.d/90gb.conf include from = /etc/rsync.d/60gb.conf The idea is that the 90GB module contains all the include / exclude statements from the 60gb.conf file and the 90gb.conf file. In this manor since rsync applys include / exclude statements on a first match basis I could define something like this: /etc/rsync.d/90gb.conf: # Include all PPC code + /linux/PPC/ /etc/rsync.d/60gb.conf: # include only the i386 code - /linux/PPC/ - /linux/S390/ - /linux/IA64/ + /linux/i386/ In the above rsyncd.conf examples this would mean people who mirror the 60GB module do not get the PPC directory while those who mirror the 90GB module get the PPC directory and the i386 directory defined in the 60gb.conf file. I think this would be fairly powerful feature when dealing with complex mirror include / excludes and suspect it is a fairly easy change to the code. Unfortunately my abilities are with interpreted languages and C / C++ is not my strong point. If someone writes a patch against 2.4.6 for this I'll be happy to do testing and give feedback. Thanks. -- Regards, Scott Russell ([EMAIL PROTECTED]) Linux Technology Center, System Admin, RHCE. T/L 441-9289 / External 919-543-9289 http://bzimage.raleigh.ibm.com/webcam
Re: I also am getting hang/timeout using rsync 2.4.6 --daemon
All - I understand the focus of the discussion below is Linux - Solaris time outs but is anyone else seeing the same problem under Linux - Linux when running 2.4.6 rsync --deamon? Currently I'm seeing it from the client end. Both of the servers I'm pulling from were updated to 2.4.6 and then I started seeing problems. Since I don't have visibility to the server side there isn't to much else I can say for sure. On my end (client side) the system is Red Hat 6.0 + errata and a 2.2.18 kernel using rsync 2.4.6. In my logs I see "unexpected EOF in read_timeout" usually right after the file list is received. Running rysnc with -v -v -v doesn't show anything special about the problem. -- Scott On Tue, Feb 27, 2001 at 01:29:45PM -0600, Dave Dykstra wrote: Re: http://lists.samba.org/pipermail/rsync/2001-February/003628.html Some more email was exhanged the last couple days on the subject of a TCP hang between Solaris and Linux: Date: Mon, 26 Feb 2001 12:06:08 -0600 From: Dave Dykstra [EMAIL PROTECTED] To: "David S. Miller" [EMAIL PROTECTED] Cc: [EMAIL PROTECTED], [EMAIL PROTECTED] Subject: Re: Linux 2.2.16 through 2.2.18preX TCP hang bug triggered by rsync David, Regarding the patch you sent us (below) to try to help solve the problem of Solaris sending acks with a sequence number that was out-of-window: We have now completed extensive testing with it and analysis of tcpdumps and have determined that the patch is working as expected (accepting the acks) but it isn't enough to work around the state that Solaris gets itself into; the connection still hangs. It looks like Alexey was right. Linux is able to make further progress getting data sent to Solaris but it isn't enough to recover the whole session; the Linux receive window stays at 0 so I presume the rsync application isn't reading the data because it's waiting for the Solaris side to complete something. Oddly, every 30 or 60 seconds after this situation occurs, Linux sends another 2-4 1460 byte packets and they're acknowledged by Solaris. It seems unlikely that the rsync application would be sending exact multiples of 1460, but I didn't do a trace during the hang to see if it was generating extra data for some reason. I have attached the tcpdump in case you're interested. Recall that 'static' is Linux and 'dynamic' is Solaris. We have added our interpretation on some of the lines. We also have had an initial response from Sun where they recommended upgrading with a certain patch but that too hasn't solved the problem (the attached tcpdump is with the Solaris patch in place). Thanks for your help and I'll let you know if we do ever get a satisfactory answer from Sun. - Dave Dykstra -- Regards, Scott Russell ([EMAIL PROTECTED]) Linux Technology Center, System Admin, RHCE. T/L 441-9289 / External 919-543-9289 http://bzimage.raleigh.ibm.com/webcam
Re: I also am getting hang/timeout using rsync 2.4.6 --daemon
On Tue, Feb 27, 2001 at 02:57:40PM -0600, Dave Dykstra wrote: Those symptoms are quite different. I suggest checking the server side logs first. The EOF occurs on the client side anytime the server side goes away prematurely. You probably aren't yet stressing TCP because I don't think much bidirectional traffic is exchanged so early, unless your first file is very large. - Dave Dykstra I would love to. I still cannot get the site owners at the server end to tell me what rsyncd reports in their logs when I connect then drop. It's a quest. All though rsync itself may not be stressing TCP my system (clinet side) is under heavy load (avg 75% capacity on tr0) from ftp clients. Regardless, I see the timeouts on my end from rsync even if I disable ftpd and only run rsync requests to the upstream rsync servers. I'll continue to look into it. Thanks for the feedback. -- Regards, Scott Russell ([EMAIL PROTECTED]) Linux Technology Center, System Admin, RHCE. T/L 441-9289 / External 919-543-9289 http://bzimage.raleigh.ibm.com/webcam
