Matt, I'm happy to take a look at the logs. The "io timeout after 72035 seconds" suggests something has stalled, and we'd like to find out what is going on just before that.
Yes, the first step will be to increase the XferLogLevel to see what is happening before things stall. Maybe try 5? Also, you could add "-vv" to the RsyncArgs to increase the client's rsync verbosity. It probably makes sense to reduce the ClientTimeout so you don't have to wait for ~20 hours each run. It would be good to know if the stall is repeatable (ie, same place in the transfer each time). However, since the XferLOG file writes are buffered, it's quite likely we'll miss the most recent debug output when the processes get killed after the io timeout. So it might be better to run BackupPC_dump from the command-line with the -v option, which writes all the log output to stderr in addition to the XferLOG file. First disable automatic backups on that host, to make sure none start when you are running a backup manually. Then run BackupPC_dump from the command-line as the backuppc user. Either do it from a terminal with a large scroll-back capability (eg, 2k+ lines that you can cut-and-paste), or use "script" or something similar that captures all the output. Please send to me directly. Thanks, Craig On Thu, Oct 4, 2018 at 7:31 AM Bedynek, Matthew J. <bedyne...@ornl.gov> wrote: > All, > > > > Am using BackupPC 4.2.1 on a Redhat 7.5 host to backup a rather large > repository of data. I believe things worked OK with tar but after we > changed the file system we switched back to Rsync. I think we have plenty > of others which roughly equal in size but backup fine. Not sure if there > is something specific about this path that causes issue. > > > > I have worked with Craig in the past to troubleshoot and provide data. I > suspect he or someone will ask I turn up logging and repeat. I can provide > the logs on request -- just not here since would be very large. > > > > It seems our environment is good for testing / finding issues since some > of them are large in size and number of files. > > > > Thanks and take care! > > > > (..list of files..) > > [receiver] io timeout after 72035 seconds -- exiting > > Done: 0 errors, 0 filesExist, 0 sizeExist, 0 sizeExistComp, 0 filesTotal, 0 > sizeTotal, 0 filesNew, 0 sizeNew, 0 sizeNewComp, 233977 inode > > rsync error: timeout in data send/receive (code 30) at io.c(140) > [receiver=3.0.9.12] > > rsync_bpc: connection unexpectedly closed (39 bytes received so far) > [generator] > > DoneGen: 0 errors, 14204 filesExist, 809628 sizeExist, 809628 sizeExistComp, > 131190 filesTotal, 2915859542718 sizeTotal, 0 filesNew, 0 sizeNew, 0 > sizeNewComp, 305814 inode > > rsync error: error in rsync protocol data stream (code 12) at io.c(629) > [generator=3.0.9.12] > > rsync_bpc exited with fatal status 12 (3072) (rsync error: error in rsync > protocol data stream (code 12) at io.c(629) [generator=3.0.9.12]) > > Xfer PIDs are now > > Got fatal error during xfer (rsync error: error in rsync protocol data stream > (code 12) at io.c(629) [generator=3.0.9.12]) > > Backup aborted (rsync error: error in rsync protocol data stream (code 12) at > io.c(629) [generator=3.0.9.12]) > > BackupFailCleanup: nFilesTotal = 131190, type = full, BackupCase = 1, inPlace > = 1, lastBkupNum = > > Keeping non-empty backup #0 (/backups/BackupPC/pc/cncs5to9-8-repo/0) > > Running BackupPC_refCountUpdate -h cncs5to9-8-repo -f on cncs5to9-8-repo > > Xfer PIDs are now 12182 > > BackupPC_refCountUpdate: cncs5to9-8-repo #0 inodeLast set to 305812 (was 1) > > BackupPC_refCountUpdate: host cncs5to9-8-repo got 0 errors (took 141 secs) > > Xfer PIDs are now > > Finished BackupPC_refCountUpdate (running time: 142 sec) > > > > > > Thanks, > > > > Matt > > > _______________________________________________ > BackupPC-users mailing list > BackupPC-users@lists.sourceforge.net > List: https://lists.sourceforge.net/lists/listinfo/backuppc-users > Wiki: http://backuppc.wiki.sourceforge.net > Project: http://backuppc.sourceforge.net/ >
_______________________________________________ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List: https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki: http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
