Re: rsync error: error in rsync protocol data stream (Broken pipe)
Just for the record, the patch to token.c for the -z bug that was discovered by Yasuoka Masahiko and patched by him and Wayne Davison has fixed the problem that I reported here: http://www.mail-archive.com/[EMAIL PROTECTED]/msg07289.html Thanks guys, this bug has been biting me for the past 6 months... -Andy -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: rsync error: error in rsync protocol data stream (Broken pipe)
On Tue, Jun 17, 2003 at 06:35:50AM -0700, jw schultz wrote: You could try turning on transfer logging i suppose. If you haven't already done so you might want to use the log file option in case chroot is getting in the way. Beyond this i have no suggestions; i dont use rsyncd. I may be having a similar problem. I'm using rsync version 2.5.6cvs-20030205 every night on Solaris 8/x86 to do a backup from a client system to a backup server using rsyncd. Almost every night I see the following errors logged on the rsyncd server: Jun 18 00:48:18 ead62 rsyncd[9632]: [ID 702911 daemon.warning] inflate returned -3 (0 bytes) Jun 18 00:48:18 ead62 rsyncd[9632]: [ID 702911 daemon.warning] rsync error: error in rsync protocol data stream (code 12) at ../token.c(416) Jun 18 00:48:18 ead62 rsyncd[9632]: [ID 702911 daemon.warning] rsync: connection unexpectedly closed (197041 bytes read so far) Jun 18 00:48:18 ead62 rsyncd[9632]: [ID 702911 daemon.warning] rsync error: error in rsync protocol data stream (code 12) at ../io.c(187) And on the client side, I see the following error: rsync: writefd_unbuffered failed to write 42 bytes: phase unknown: Broken pipe rsync error: error in rsync protocol data stream (code 12) at ../io.c(622) This has been happening to me for months almost every night. I find that I can check the return code from rsync to see whether the transfer succeeded. If it failed, I simply try again, and it almost always finishes the backup successfully on the second invocation. But if not I keep retrying up to 10 times. I think one evening it took 9 attempts, but most of the time it works in 2 tries. It's a hack, but it gets the job done, and I was having no luck debugging the problem. -Andy -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: rsync error: error in rsync protocol data stream (Broken pipe)
On Wed, Jun 18, 2003 at 10:38:54AM -0400, Andrew J. Schorr wrote: On Tue, Jun 17, 2003 at 06:35:50AM -0700, jw schultz wrote: You could try turning on transfer logging i suppose. If you haven't already done so you might want to use the log file option in case chroot is getting in the way. Beyond this i have no suggestions; i dont use rsyncd. I may be having a similar problem. I'm using rsync version 2.5.6cvs-20030205 every night on Solaris 8/x86 to do a backup from a client system to a backup server using rsyncd. Almost every night I see the following errors logged on the rsyncd server: Jun 18 00:48:18 ead62 rsyncd[9632]: [ID 702911 daemon.warning] inflate returned -3 (0 bytes) Jun 18 00:48:18 ead62 rsyncd[9632]: [ID 702911 daemon.warning] rsync error: error in rsync protocol data stream (code 12) at ../token.c(416) Jun 18 00:48:18 ead62 rsyncd[9632]: [ID 702911 daemon.warning] rsync: connection unexpectedly closed (197041 bytes read so far) Jun 18 00:48:18 ead62 rsyncd[9632]: [ID 702911 daemon.warning] rsync error: error in rsync protocol data stream (code 12) at ../io.c(187) And on the client side, I see the following error: rsync: writefd_unbuffered failed to write 42 bytes: phase unknown: Broken pipe rsync error: error in rsync protocol data stream (code 12) at ../io.c(622) This has been happening to me for months almost every night. I find that I can check the return code from rsync to see whether the transfer succeeded. If it failed, I simply try again, and it almost always finishes the backup successfully on the second invocation. But if not I keep retrying up to 10 times. I think one evening it took 9 attempts, but most of the time it works in 2 tries. It's a hack, but it gets the job done, and I was having no luck debugging the problem. inflate returned -3 is a compression error where it has bad data. It could be there is a bug in zlib but it seems more likely there is something wrong on your systems, perhaps flaky hardware. I haven't looked into that part of the code much, it is possible this could be a symptom of a file being changed during the transfer. If this is happening that frequently i'd try not using the -z option. -- J.W. SchultzPegasystems Technologies email address: [EMAIL PROTECTED] Remember Cernan and Schmitt -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: rsync error: error in rsync protocol data stream (Broken pipe)
On Mon, 16 Jun 2003 at 16:24:25 -0700, jw schultz wrote: After looking at io.c, it seems like the problem (rather, the symptom) for this error message is a failed write() call (which explains the Broken Pipe message to stderr). I thought maybe --blocking-io would avoid this issue, but it still happened when using --blocking-io. It would seem that the server process (forked off from the daemon) is dying. Look in the rsyncd log file (may be syslog) for clues. Yes, this does appear to be what is happening. I actually had tried looking at the daemon logs, but there didn't appear to be anything remotely interesting in it. This is the only line I see in the log: 2003/06/16 16:23:48 [17924] rsync to blah/blah/Blah/ from [EMAIL PROTECTED] (10.x.x.x) Is there any way to increase the verbosity for this? Adding -v's to the daemon process didn't appear to affect it. Oh well, I guess it's time to start debugging. - Nate Case [EMAIL PROTECTED] (not subscribed) -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: rsync error: error in rsync protocol data stream (Broken pipe)
On Tue, Jun 17, 2003 at 09:10:38AM -0500, Nathaniel Case wrote: On Mon, 16 Jun 2003 at 16:24:25 -0700, jw schultz wrote: After looking at io.c, it seems like the problem (rather, the symptom) for this error message is a failed write() call (which explains the Broken Pipe message to stderr). I thought maybe --blocking-io would avoid this issue, but it still happened when using --blocking-io. It would seem that the server process (forked off from the daemon) is dying. Look in the rsyncd log file (may be syslog) for clues. Yes, this does appear to be what is happening. I actually had tried looking at the daemon logs, but there didn't appear to be anything remotely interesting in it. This is the only line I see in the log: 2003/06/16 16:23:48 [17924] rsync to blah/blah/Blah/ from [EMAIL PROTECTED] (10.x.x.x) Is there any way to increase the verbosity for this? Adding -v's to the daemon process didn't appear to affect it. Oh well, I guess it's time to start debugging. You could try turning on transfer logging i suppose. If you haven't already done so you might want to use the log file option in case chroot is getting in the way. Beyond this i have no suggestions; i dont use rsyncd. -- J.W. SchultzPegasystems Technologies email address: [EMAIL PROTECTED] Remember Cernan and Schmitt -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
rsync error: error in rsync protocol data stream (Broken pipe)
I'm experiencing some unexpected behavior with rsync v2.5.6. After looking around it seems like several people have experienced similar problems in the past, but those appear to be old bugs that are solved now. Also doesn't appear to be any direct bugfixes made for this issue since 2.5.6. The strange thing is that things seem to work when I use ssh for transport. However now I need to switch to using the rsync daemon directly and this problem has arose. I'm pushing local data to a remote rsync backup server: ---[snip]--- # rsync -a -v --modify-window=2 /backup/Source/Dir/ [EMAIL PROTECTED]::backup/dest Password: building file list ... rsync: writefd_unbuffered failed to write 4092 bytes: phase send_file_entry: Broken pipe rsync error: error in rsync protocol data stream (code 12) at io.c(515) ---[snip]--- With more -v's, I end up with the same message at the end, but it's preceeded by a bunch of lines like this (filenames altered): make_file(3,file1) make_file(3,file2) make_file(3,file3) make_file(3,file4) It stops on the same file each time. I tried deleting this file to see if it's just choking on a particular file (or type of file), but in that case it just chokes on the next file (different type) in the directory. The files in this directory are no bigger than 2MB. If I remove the subdirectory entirely, it still just dies on the next file it encounters. The directories that I'm syncing do have a large number of files, but as I mentioned earlier, it does seem to work when using -e ssh. I do not seem to be running out of memory. I also tried the same parameters but breaking it up into smaller subdirectories, and it still failed after X amount of files with the same error (I did not determine whether or not X is always the same with using other directories). Note that this is always occuring in the building file list stage, and not the actual file transfer. After looking at io.c, it seems like the problem (rather, the symptom) for this error message is a failed write() call (which explains the Broken Pipe message to stderr). I thought maybe --blocking-io would avoid this issue, but it still happened when using --blocking-io. Does anyone have any ideas about what could be causing this? Thanks, - Nate Case [EMAIL PROTECTED] -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: rsync error: error in rsync protocol data stream (Broken pipe)
On Mon, Jun 16, 2003 at 11:53:04AM -0500, Nate Case wrote: I'm experiencing some unexpected behavior with rsync v2.5.6. After looking around it seems like several people have experienced similar problems in the past, but those appear to be old bugs that are solved now. Also doesn't appear to be any direct bugfixes made for this issue since 2.5.6. The strange thing is that things seem to work when I use ssh for transport. However now I need to switch to using the rsync daemon directly and this problem has arose. I'm pushing local data to a remote rsync backup server: ---[snip]--- # rsync -a -v --modify-window=2 /backup/Source/Dir/ [EMAIL PROTECTED]::backup/dest Password: building file list ... rsync: writefd_unbuffered failed to write 4092 bytes: phase send_file_entry: Broken pipe rsync error: error in rsync protocol data stream (code 12) at io.c(515) ---[snip]--- With more -v's, I end up with the same message at the end, but it's preceeded by a bunch of lines like this (filenames altered): make_file(3,file1) make_file(3,file2) make_file(3,file3) make_file(3,file4) It stops on the same file each time. I tried deleting this file to see if it's just choking on a particular file (or type of file), but in that case it just chokes on the next file (different type) in the directory. The files in this directory are no bigger than 2MB. If I remove the subdirectory entirely, it still just dies on the next file it encounters. The directories that I'm syncing do have a large number of files, but as I mentioned earlier, it does seem to work when using -e ssh. I do not seem to be running out of memory. I also tried the same parameters but breaking it up into smaller subdirectories, and it still failed after X amount of files with the same error (I did not determine whether or not X is always the same with using other directories). Note that this is always occuring in the building file list stage, and not the actual file transfer. After looking at io.c, it seems like the problem (rather, the symptom) for this error message is a failed write() call (which explains the Broken Pipe message to stderr). I thought maybe --blocking-io would avoid this issue, but it still happened when using --blocking-io. It would seem that the server process (forked off from the daemon) is dying. Look in the rsyncd log file (may be syslog) for clues. -- J.W. SchultzPegasystems Technologies email address: [EMAIL PROTECTED] Remember Cernan and Schmitt -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
