RE: [rdiff-backup-users] Crash in 1.3.3
Hi Josh, Yes, it could be flaky hardware, in particular it could be the destination drive, which is a very well known brand of USB external drive that seem in my experience to fail quite often. But I can do a successful backup of the same data to the same destination drive using rsync, with no problem. Any thoughts on this? The rest of the hardware I trust because it is a server we use all the time for other critical tasks. Cheers, Craig -Original Message- From: rdiff-backup-users-bounces+craig=plasmatronics.com...@nongnu.org [mailto:rdiff-backup-users-bounces+craig=plasmatronics.com...@nongnu.org ] On Behalf Of Josh Nisly Sent: Friday, 1 May 2009 1:58 AM To: Craig Findlay Cc: rdiff-backup Subject: Re: [rdiff-backup-users] Crash in 1.3.3 CRC check failures typically indicate faulty or dying hardware. JoshN Craig Findlay wrote: Backing up from Unix (OpenBSD) to Unix, both 1.3.3: Cheers, Craig 2009-04-28 09:32:05 Backing up bellfs_backup::/data/home to /mnt/backup/serverbackup/bellfs/home with args: SpecialFileError zopeuser/var/zopectlsock Socket error: AF_UNIX path too long Exception 'CRC check failed' raised of class 'type 'exceptions.IOError'': File /usr/local/lib/python2.5/site-packages/rdiff_backup/Main.py, line 306, in error_check_Main try: Main(arglist) File /usr/local/lib/python2.5/site-packages/rdiff_backup/Main.py, line 326, in Main take_action(rps) File /usr/local/lib/python2.5/site-packages/rdiff_backup/Main.py, line 282, in take_action elif action == backup: Backup(rps[0], rps[1]) File /usr/local/lib/python2.5/site-packages/rdiff_backup/Main.py, line 345, in Backup backup.Mirror_and_increment(rpin, rpout, incdir) File /usr/local/lib/python2.5/site-packages/rdiff_backup/backup.py, line 51, in Mirror_and_increment DestS.patch_and_increment(dest_rpath, source_diffiter, inc_rpath) File /usr/local/lib/python2.5/site-packages/rdiff_backup/backup.py, line 253, in patch_and_increment cls.CCPP.close() File /usr/local/lib/python2.5/site-packages/rdiff_backup/backup.py, line 499, in close metadata.ManagerObj.ConvertMetaToDiff() File /usr/local/lib/python2.5/site-packages/rdiff_backup/metadata.py, line 647, in ConvertMetaToDiff for diff_rorp in self.get_diffiter(new_iter, old_iter): File /usr/local/lib/python2.5/site-packages/rdiff_backup/metadata.py, line 609, in get_diffiter for new_rorp, old_rorp in rorpiter.Collate2Iters(new_iter, old_iter): File /usr/local/lib/python2.5/site-packages/rdiff_backup/rorpiter.py, line 100, in Collate2Iters try: relem2 = riter2.next() File /usr/local/lib/python2.5/site-packages/rdiff_backup/metadata.py, line 274, in iterate for record in self.iterate_records(): File /usr/local/lib/python2.5/site-packages/rdiff_backup/metadata.py, line 283, in iterate_records next_pos = self.get_next_pos() File /usr/local/lib/python2.5/site-packages/rdiff_backup/metadata.py, line 266, in get_next_pos newbuf = self.fileobj.read(self.blocksize) File /usr/local/lib/python2.5/gzip.py, line 227, in read self._read(readsize) File /usr/local/lib/python2.5/gzip.py, line 292, in _read self._read_eof() File /usr/local/lib/python2.5/gzip.py, line 311, in _read_eof raise IOError, CRC check failed Traceback (most recent call last): File /usr/local/bin/rdiff-backup, line 30, in module rdiff_backup.Main.error_check_Main(sys.argv[1:]) File /usr/local/lib/python2.5/site-packages/rdiff_backup/Main.py, line 306, in error_check_Main try: Main(arglist) File /usr/local/lib/python2.5/site-packages/rdiff_backup/Main.py, line 326, in Main take_action(rps) File /usr/local/lib/python2.5/site-packages/rdiff_backup/Main.py, line 282, in take_action elif action == backup: Backup(rps[0], rps[1]) File /usr/local/lib/python2.5/site-packages/rdiff_backup/Main.py, line 345, in Backup backup.Mirror_and_increment(rpin, rpout, incdir) File /usr/local/lib/python2.5/site-packages/rdiff_backup/backup.py, line 51, in Mirror_and_increment DestS.patch_and_increment(dest_rpath, source_diffiter, inc_rpath) File /usr/local/lib/python2.5/site-packages/rdiff_backup/backup.py, line 253, in patch_and_increment cls.CCPP.close() File /usr/local/lib/python2.5/site-packages/rdiff_backup/backup.py, line 499, in close metadata.ManagerObj.ConvertMetaToDiff() File /usr/local/lib/python2.5/site-packages/rdiff_backup/metadata.py, line 647, in ConvertMetaToDiff for diff_rorp in self.get_diffiter(new_iter, old_iter): File /usr/local/lib/python2.5/site-packages/rdiff_backup/metadata.py, line 609, in get_diffiter for new_rorp, old_rorp in rorpiter.Collate2Iters(new_iter, old_iter): File /usr/local/lib/python2.5/site-packages/rdiff_backup/rorpiter.py, line 100, in Collate2Iters try: relem2 = riter2.next() File /usr/local/lib/python2.5/site-packages
[rdiff-backup-users] Crash in 1.3.3
Backing up from Unix (OpenBSD) to Unix, both 1.3.3: Cheers, Craig 2009-04-28 09:32:05 Backing up bellfs_backup::/data/home to /mnt/backup/serverbackup/bellfs/home with args: SpecialFileError zopeuser/var/zopectlsock Socket error: AF_UNIX path too long Exception 'CRC check failed' raised of class 'type 'exceptions.IOError'': File /usr/local/lib/python2.5/site-packages/rdiff_backup/Main.py, line 306, in error_check_Main try: Main(arglist) File /usr/local/lib/python2.5/site-packages/rdiff_backup/Main.py, line 326, in Main take_action(rps) File /usr/local/lib/python2.5/site-packages/rdiff_backup/Main.py, line 282, in take_action elif action == backup: Backup(rps[0], rps[1]) File /usr/local/lib/python2.5/site-packages/rdiff_backup/Main.py, line 345, in Backup backup.Mirror_and_increment(rpin, rpout, incdir) File /usr/local/lib/python2.5/site-packages/rdiff_backup/backup.py, line 51, in Mirror_and_increment DestS.patch_and_increment(dest_rpath, source_diffiter, inc_rpath) File /usr/local/lib/python2.5/site-packages/rdiff_backup/backup.py, line 253, in patch_and_increment cls.CCPP.close() File /usr/local/lib/python2.5/site-packages/rdiff_backup/backup.py, line 499, in close metadata.ManagerObj.ConvertMetaToDiff() File /usr/local/lib/python2.5/site-packages/rdiff_backup/metadata.py, line 647, in ConvertMetaToDiff for diff_rorp in self.get_diffiter(new_iter, old_iter): File /usr/local/lib/python2.5/site-packages/rdiff_backup/metadata.py, line 609, in get_diffiter for new_rorp, old_rorp in rorpiter.Collate2Iters(new_iter, old_iter): File /usr/local/lib/python2.5/site-packages/rdiff_backup/rorpiter.py, line 100, in Collate2Iters try: relem2 = riter2.next() File /usr/local/lib/python2.5/site-packages/rdiff_backup/metadata.py, line 274, in iterate for record in self.iterate_records(): File /usr/local/lib/python2.5/site-packages/rdiff_backup/metadata.py, line 283, in iterate_records next_pos = self.get_next_pos() File /usr/local/lib/python2.5/site-packages/rdiff_backup/metadata.py, line 266, in get_next_pos newbuf = self.fileobj.read(self.blocksize) File /usr/local/lib/python2.5/gzip.py, line 227, in read self._read(readsize) File /usr/local/lib/python2.5/gzip.py, line 292, in _read self._read_eof() File /usr/local/lib/python2.5/gzip.py, line 311, in _read_eof raise IOError, CRC check failed Traceback (most recent call last): File /usr/local/bin/rdiff-backup, line 30, in module rdiff_backup.Main.error_check_Main(sys.argv[1:]) File /usr/local/lib/python2.5/site-packages/rdiff_backup/Main.py, line 306, in error_check_Main try: Main(arglist) File /usr/local/lib/python2.5/site-packages/rdiff_backup/Main.py, line 326, in Main take_action(rps) File /usr/local/lib/python2.5/site-packages/rdiff_backup/Main.py, line 282, in take_action elif action == backup: Backup(rps[0], rps[1]) File /usr/local/lib/python2.5/site-packages/rdiff_backup/Main.py, line 345, in Backup backup.Mirror_and_increment(rpin, rpout, incdir) File /usr/local/lib/python2.5/site-packages/rdiff_backup/backup.py, line 51, in Mirror_and_increment DestS.patch_and_increment(dest_rpath, source_diffiter, inc_rpath) File /usr/local/lib/python2.5/site-packages/rdiff_backup/backup.py, line 253, in patch_and_increment cls.CCPP.close() File /usr/local/lib/python2.5/site-packages/rdiff_backup/backup.py, line 499, in close metadata.ManagerObj.ConvertMetaToDiff() File /usr/local/lib/python2.5/site-packages/rdiff_backup/metadata.py, line 647, in ConvertMetaToDiff for diff_rorp in self.get_diffiter(new_iter, old_iter): File /usr/local/lib/python2.5/site-packages/rdiff_backup/metadata.py, line 609, in get_diffiter for new_rorp, old_rorp in rorpiter.Collate2Iters(new_iter, old_iter): File /usr/local/lib/python2.5/site-packages/rdiff_backup/rorpiter.py, line 100, in Collate2Iters try: relem2 = riter2.next() File /usr/local/lib/python2.5/site-packages/rdiff_backup/metadata.py, line 274, in iterate for record in self.iterate_records(): File /usr/local/lib/python2.5/site-packages/rdiff_backup/metadata.py, line 283, in iterate_records next_pos = self.get_next_pos() File /usr/local/lib/python2.5/site-packages/rdiff_backup/metadata.py, line 266, in get_next_pos newbuf = self.fileobj.read(self.blocksize) File /usr/local/lib/python2.5/gzip.py, line 227, in read self._read(readsize) File /usr/local/lib/python2.5/gzip.py, line 292, in _read self._read_eof() File /usr/local/lib/python2.5/gzip.py, line 311, in _read_eof raise IOError, CRC check failed IOError: CRC check failed Fatal Error: Lost connection to the remote system 2009-04-28 15:17:36 Exit code 1 ___ rdiff-backup-users mailing list at rdiff-backup-users@nongnu.org http://lists.nongnu.org/mailman/listinfo/rdiff-backup-users Wiki URL:
[rdiff-backup-users] Extremely slow performance problem
I have a problem with version 1.3.1 backing up from a Windows server to a Linux box. Following is the output from a test backup script. Both ends are at 1.3.1, the Windows server is running the native Windows version, not a cygwin version. Note the Elapsed time of 1 hour 54 minutes. This is an extremely long time it seems to me to do the job, since the source is only 2292 files and 2.2 GB in size. This test represents only one directory tree out of 40 or 50 on the Windows server. At that rate it will take days to a single backup run on the server :( Any thoughts why it is so slow? What testing can I do to narrow down the cause of the problem? Cheers, Craig 2009-02-19 01:00:01 Starting backup 2009-02-19 01:00:01 backup disk status before backup: /dev/sda1 459G 148G 288G 34% /mnt/backup 2009-02-19 01:00:02 Backing up plsbs_backup::f:/shared/public/Travis to /mnt/backup/serverbackup/plsbs/shared/public/Travis with args: --[ Session statistics ]-- StartTime 1234965607.00 (Thu Feb 19 01:00:07 2009) EndTime 1234972469.94 (Thu Feb 19 02:54:29 2009) ElapsedTime 6862.94 (1 hour 54 minutes 22.94 seconds) SourceFiles 2292 SourceFileSize 2362487027 (2.20 GB) MirrorFiles 2284 MirrorFileSize 2357293068 (2.20 GB) NewFiles 29 NewFileSize 6938672 (6.62 MB) DeletedFiles 21 DeletedFileSize 1785944 (1.70 MB) ChangedFiles 2259 ChangedSourceSize 2355548355 (2.19 GB) ChangedMirrorSize 2355507124 (2.19 GB) IncrementFiles 2309 IncrementFileSize 992506 (969 KB) TotalDestinationSizeChange 6186465 (5.90 MB) Errors 0 -- 2009-02-19 02:54:30 Exit code 0 2009-02-19 02:54:30 Finished backing up plsbs_backup::f:/shared/public/Travis to /mnt/backup/serverbackup/plsbs/shared/public/Travis 2009-02-19 02:54:30 backup disk status after backup: /dev/sda1 459G 148G 288G 34% /mnt/backup 2009-02-19 02:54:30 Finished backup ___ rdiff-backup-users mailing list at rdiff-backup-users@nongnu.org http://lists.nongnu.org/mailman/listinfo/rdiff-backup-users Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki
[rdiff-backup-users] Exception thrown in 1.3.1
The following crash happens consistently in 1.3.1. The process is backing up from a Windows Small Business Server to a Linux backup server. I have checked the source directory and there does not seem to be anything unusual with the permissions. Could it be something to do with the dot in the directory name? (This is a new rdiff-backup set being created from an existing data mirror, so the rdiff-backup-data directory is empty at this point, just mentioning that in case it may be relevant.) Cheers, Craig Found interrupted initial backup. Removing... Exception '[Errno 1] Operation not permitted: '/mnt/backup/serverbackup/plsbs/shared/public/Australian.Standards/Emerg ency_lighting'' raised of class 'type 'exceptions.OSError'': File /usr/lib/python2.5/site-packages/rdiff_backup/Main.py, line 306, in error_check_Main try: Main(arglist) File /usr/lib/python2.5/site-packages/rdiff_backup/Main.py, line 326, in Main take_action(rps) File /usr/lib/python2.5/site-packages/rdiff_backup/Main.py, line 282, in take_action elif action == backup: Backup(rps[0], rps[1]) File /usr/lib/python2.5/site-packages/rdiff_backup/Main.py, line 348, in Backup backup.Mirror(rpin, rpout) File /usr/lib/python2.5/site-packages/rdiff_backup/backup.py, line 38, in Mirror DestS.patch(dest_rpath, source_diffiter) File /usr/lib/python2.5/site-packages/rdiff_backup/backup.py, line 240, in patch ITR(diff.index, diff) File /usr/lib/python2.5/site-packages/rdiff_backup/rorpiter.py, line 277, in __call__ if self.finish_branches(index) is None: File /usr/lib/python2.5/site-packages/rdiff_backup/rorpiter.py, line 229, in finish_branches to_be_finished.end_process() File /usr/lib/python2.5/site-packages/rdiff_backup/backup.py, line 668, in end_process rpath.copy_attribs(self.dir_update, self.base_rp) File /usr/lib/python2.5/site-packages/rdiff_backup/rpath.py, line 189, in copy_attribs rpout.chmod(rpin.getperms()) File /usr/lib/python2.5/site-packages/rdiff_backup/rpath.py, line 927, in chmod self.conn.os.chmod(self.path, permissions Globals.permission_mask) Traceback (most recent call last): File /usr/bin/rdiff-backup, line 30, in module rdiff_backup.Main.error_check_Main(sys.argv[1:]) File /usr/lib/python2.5/site-packages/rdiff_backup/Main.py, line 306, in error_check_Main try: Main(arglist) File /usr/lib/python2.5/site-packages/rdiff_backup/Main.py, line 326, in Main take_action(rps) File /usr/lib/python2.5/site-packages/rdiff_backup/Main.py, line 282, in take_action elif action == backup: Backup(rps[0], rps[1]) File /usr/lib/python2.5/site-packages/rdiff_backup/Main.py, line 348, in Backup backup.Mirror(rpin, rpout) File /usr/lib/python2.5/site-packages/rdiff_backup/backup.py, line 38, in Mirror DestS.patch(dest_rpath, source_diffiter) File /usr/lib/python2.5/site-packages/rdiff_backup/backup.py, line 240, in patch ITR(diff.index, diff) File /usr/lib/python2.5/site-packages/rdiff_backup/rorpiter.py, line 277, in __call__ if self.finish_branches(index) is None: File /usr/lib/python2.5/site-packages/rdiff_backup/rorpiter.py, line 229, in finish_branches to_be_finished.end_process() File /usr/lib/python2.5/site-packages/rdiff_backup/backup.py, line 668, in end_process rpath.copy_attribs(self.dir_update, self.base_rp) File /usr/lib/python2.5/site-packages/rdiff_backup/rpath.py, line 189, in copy_attribs rpout.chmod(rpin.getperms()) File /usr/lib/python2.5/site-packages/rdiff_backup/rpath.py, line 927, in chmod self.conn.os.chmod(self.path, permissions Globals.permission_mask) OSError: [Errno 1] Operation not permitted: '/mnt/backup/serverbackup/plsbs/shared/public/Australian.Standards/Emerg ency_lighting' ___ rdiff-backup-users mailing list at rdiff-backup-users@nongnu.org http://lists.nongnu.org/mailman/listinfo/rdiff-backup-users Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki
[rdiff-backup-users] Possible to convert an existing data set to rdiff-backup?
Hi all, Is there a way to convert an existing data set, such as might be produced by rsync, to an rdiff-backup data set. The rational behind this is that I already have a large amount of data in an rsync mirror, but want to convert it to rdiff-backup without having to take the significant time and bandwidth to rdiff-backup the data from the source from scratch. Cheers, Craig ___ rdiff-backup-users mailing list at rdiff-backup-users@nongnu.org http://lists.nongnu.org/mailman/listinfo/rdiff-backup-users Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki
[rdiff-backup-users] Rdiff-backup 1.30
Hi all, I just upgraded a couple of machines to 1.3.0 Below is a traceback when running on a standard script of mine: (Used to work with 1.1.15) It's backing up from a Windows server to a unix box. The script runs on the unix box. Any thoughts? (The first line is output from my script, the rest is from rdiff-backup) Cheers, Craig 2009-01-07 11:54:36 Backing up plsbs_backup::d:/users to /mnt/backup/serverbackup/plsbs/users with args: Traceback (most recent call last): File /usr/bin/rdiff-backup, line 30, in module rdiff_backup.Main.error_check_Main(sys.argv[1:]) File /usr/lib/python2.5/site-packages/rdiff_backup/Main.py, line 309, in error_check_Main errmsg = robust.is_routine_fatal(exc) File /usr/lib/python2.5/site-packages/rdiff_backup/robust.py, line 77, in is_routine_fatal elif isinstance(exc, EnvironmentError) and e.errno == errno.ENOTCONN: NameError: global name 'e' is not defined ___ rdiff-backup-users mailing list at rdiff-backup-users@nongnu.org http://lists.nongnu.org/mailman/listinfo/rdiff-backup-users Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki
[rdiff-backup-users] Rdiff-backup 1.1.16 crash
Hi all, I know 1.1.17 has been released, however a colleague and I have been chasing a crash in 1.1.16 that appears to be due to filenames that contain unicode type characters. A typical filename that causes this is: '/Volumes/LACIE/b/Project/2005/2005389AL/Docs Out/Da003 2005389 facade glazing apartments - BLL proposed opt\x85.pdf' The \x85 appears to be the problem. We have a PDF generator that seems to like putting these types of characters in the names of the files it generates. Typical crash output follows below: Sorry for the html formatting of this email. I am currently using Microsoft OWA so it can't be helped. Cheers, Craig Exception '[Errno 22] Invalid argument: '/Volumes/LACIE/b/Project/2005/2005389AL/Docs Out/Da003 2005389 facade glazing apartments - BLL proposed opt\x 85.pdf'' raised of class 'type 'exceptions.OSError'': File /sw/lib/python2.5/site-packages/rdiff_backup/Main.py, line 299, in error_check_Main try: Main(arglist) File /sw/lib/python2.5/site-packages/rdiff_backup/Main.py, line 319, in Main take_action(rps) File /sw/lib/python2.5/site-packages/rdiff_backup/Main.py, line 275, in take_action elif action == backup: Backup(rps[0], rps[1]) File /sw/lib/python2.5/site-packages/rdiff_backup/Main.py, line 341, in Backup backup.Mirror(rpin, rpout) File /sw/lib/python2.5/site-packages/rdiff_backup/backup.py, line 38, in Mirror DestS.patch(dest_rpath, source_diffiter) File /sw/lib/python2.5/site-packages/rdiff_backup/backup.py, line 218, in patch ITR(diff.index, diff) File /sw/lib/python2.5/site-packages/rdiff_backup/rorpiter.py, line 281, in __call__ last_branch.fast_process(*args) File /sw/lib/python2.5/site-packages/rdiff_backup/backup.py, line 512, in fast_process self.CCPP.get_rorps(index), self.basis_root_rp) File /sw/lib/python2.5/site-packages/rdiff_backup/longname.py, line 200, in get_mirror_inc_rps mirror_rp, alt_mirror, alt_inc = mir_triple_new(new_rorp) File /sw/lib/python2.5/site-packages/rdiff_backup/longname.py, line 171, in mir_triple_new mirror_rp = check_new_index(mirror_root, new_rorp.index) File /sw/lib/python2.5/site-packages/rdiff_backup/longname.py, line 125, in check_new_index rp = wrap_call(base.new_index, index) File /sw/lib/python2.5/site-packages/rdiff_backup/longname.py, line 111, in wrap_call try: result = func(*args) File /sw/lib/python2.5/site-packages/rdiff_backup/rpath.py, line 1017, in new_index return self.__class__(self.conn, self.base, index) File /sw/lib/python2.5/site-packages/rdiff_backup/rpath.py, line 747, in __init__ else: self.setdata() File /sw/lib/python2.5/site-packages/rdiff_backup/rpath.py, line 771, in setdata self.data = self.conn.C.make_file_dict(self.path) Traceback (most recent call last): File /usr/local/bin/rdiff-backup, line 23, in module rdiff_backup.Main.error_check_Main(sys.argv[1:]) File /sw/lib/python2.5/site-packages/rdiff_backup/Main.py, line 299, in error_check_Main try: Main(arglist) File /sw/lib/python2.5/site-packages/rdiff_backup/Main.py, line 319, in Main take_action(rps) File /sw/lib/python2.5/site-packages/rdiff_backup/Main.py, line 275, in take_action elif action == backup: Backup(rps[0], rps[1]) File /sw/lib/python2.5/site-packages/rdiff_backup/Main.py, line 341, in Backup backup.Mirror(rpin, rpout) File /sw/lib/python2.5/site-packages/rdiff_backup/backup.py, line 38, in Mirror DestS.patch(dest_rpath, source_diffiter) File /sw/lib/python2.5/site-packages/rdiff_backup/backup.py, line 218, in patch ITR(diff.index, diff) File /sw/lib/python2.5/site-packages/rdiff_backup/rorpiter.py, line 281, in __call__ last_branch.fast_process(*args) File /sw/lib/python2.5/site-packages/rdiff_backup/backup.py, line 512, in fast_process self.CCPP.get_rorps(index), self.basis_root_rp) File /sw/lib/python2.5/site-packages/rdiff_backup/longname.py, line 200, in get_mirror_inc_rps mirror_rp, alt_mirror, alt_inc = mir_triple_new(new_rorp) File /sw/lib/python2.5/site-packages/rdiff_backup/longname.py, line 171, in mir_triple_new mirror_rp = check_new_index(mirror_root, new_rorp.index) File /sw/lib/python2.5/site-packages/rdiff_backup/longname.py, line 125, in check_new_index rp = wrap_call(base.new_index, index) File /sw/lib/python2.5/site-packages/rdiff_backup/longname.py, line 111, in wrap_call try: result = func(*args) File /sw/lib/python2.5/site-packages/rdiff_backup/rpath.py, line 1017, in new_index return self.__class__(self.conn, self.base, index) File /sw/lib/python2.5/site-packages/rdiff_backup/rpath.py, line 747, in __init__ else: self.setdata() File /sw/lib/python2.5/site-packages/rdiff_backup/rpath.py, line 771, in setdata self.data = self.conn.C.make_file_dict(self.path) OSError: [Errno 22] Invalid argument: '/Volumes/LACIE/b/Project/2005/2005389AL/Docs Out/Da003
