Re: [BackupPC-users] Restore issues
Hi Matthias, Sorry it took me so long to get back to you - I've had a lot of tight deadlines. I'm not backing up a windows client... It's a Linux server offering a samba share. To be fair, thinking about it, since I'm using rsync, it's just a Linux server, and the fact that I access it with Windows clients is irrelevant. No VSS - because Linux... Good news on the backup front... Still, very strange about the different numbers of files reported on Windows when looking at the share, but I guess that's one of those strange glitchy things with MS products... I think pretty much the only thing I backup is the /home, /etc, /usr and /var... Thanks for your help on this... Cheers, Jx On Thu, Jul 30, 2009 at 8:00 PM, Matthias Meyer matthias.me...@gmx.liwrote: Vetch wrote: Hi Matthias, All my xferlogs say they have 0 errors (apart from one, but that was after the problem occurred anyway)... I've had a look at them, but they are... quite long... Without knowing what to search for, I'm not sure what I can do with them... If they report no errors, I guess I can assume all files are backing up properly? Xfer Error Summary Backup# Type View #Xfer errs #bad files #bad share #tar errs 0 full XferLOG, Errors 0 0 0 0 28 full XferLOG, Errors 0 0 0 0 56 full XferLOG, Errors 0 0 0 0 84 full XferLOG, Errors 0 0 0 0 112 full XferLOG, Errors 0 0 0 0 126 full XferLOG, Errors 0 0 0 0 133 full XferLOG, Errors 0 0 0 0 140 full XferLOG, Errors 0 0 0 0 147 full XferLOG, Errors 0 0 0 0 150 incr XferLOG, Errors 0 0 0 0 151 incr XferLOG, Errors 0 0 0 0 152 incr XferLOG, Errors 0 0 0 0 153 full XferLOG, Errors 1 0 0 0 154 incr XferLOG, Errors 0 0 0 0 155 incr XferLOG, Errors 0 0 0 0 156 incr XferLOG, Errors 0 0 0 0 Thanks, Jx That is unbelievable. You backup a windows client, right? There should be a lot of files which can not be backuped because they are in use. Do you use volume shadow copies in windows? But nevertheless, you have backups of all files specified in your configuration. If you check your backup include/exclude configuration you should find which files are not backuped. br Matthias -- Don't Panic -- Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/ -- Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Restore issues
Hi Jeffrey, Sounds doable, but as I say, I'm backing a Linux server, not a Win client, so I don't think I need to worry too much about the busy files... Thanks for the suggestion though... Cheers, Jx On Thu, Jul 30, 2009 at 8:19 PM, Jeffrey J. Kosowsky backu...@kosowsky.orgwrote: Matthias Meyer wrote at about 21:00:54 +0200 on Thursday, July 30, 2009: Vetch wrote: Hi Matthias, All my xferlogs say they have 0 errors (apart from one, but that was after the problem occurred anyway)... I've had a look at them, but they are... quite long... Without knowing what to search for, I'm not sure what I can do with them... If they report no errors, I guess I can assume all files are backing up properly? Xfer Error Summary Backup# Type View #Xfer errs #bad files #bad share #tar errs 0 full XferLOG, Errors 0 0 0 0 28 full XferLOG, Errors 0 0 0 0 56 full XferLOG, Errors 0 0 0 0 84 full XferLOG, Errors 0 0 0 0 112 full XferLOG, Errors 0 0 0 0 126 full XferLOG, Errors 0 0 0 0 133 full XferLOG, Errors 0 0 0 0 140 full XferLOG, Errors 0 0 0 0 147 full XferLOG, Errors 0 0 0 0 150 incr XferLOG, Errors 0 0 0 0 151 incr XferLOG, Errors 0 0 0 0 152 incr XferLOG, Errors 0 0 0 0 153 full XferLOG, Errors 1 0 0 0 154 incr XferLOG, Errors 0 0 0 0 155 incr XferLOG, Errors 0 0 0 0 156 incr XferLOG, Errors 0 0 0 0 Thanks, Jx That is unbelievable. You backup a windows client, right? There should be a lot of files which can not be backuped because they are in use. Do you use volume shadow copies in windows? Alternatively, you could just exclude the files that tend to be busy. Before I wrote my volume shadow copy script, I had a short list of excludes that eliminated all busy files. But nevertheless, you have backups of all files specified in your configuration. If you check your backup include/exclude configuration you should find which files are not backuped. br Matthias -- Don't Panic -- Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/ -- Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/ -- Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Restore issues
Vetch wrote: Hi Matthias, All my xferlogs say they have 0 errors (apart from one, but that was after the problem occurred anyway)... I've had a look at them, but they are... quite long... Without knowing what to search for, I'm not sure what I can do with them... If they report no errors, I guess I can assume all files are backing up properly? Xfer Error Summary Backup# Type View #Xfer errs #bad files #bad share #tar errs 0 full XferLOG, Errors 0 0 0 0 28 full XferLOG, Errors 0 0 0 0 56 full XferLOG, Errors 0 0 0 0 84 full XferLOG, Errors 0 0 0 0 112 full XferLOG, Errors 0 0 0 0 126 full XferLOG, Errors 0 0 0 0 133 full XferLOG, Errors 0 0 0 0 140 full XferLOG, Errors 0 0 0 0 147 full XferLOG, Errors 0 0 0 0 150 incr XferLOG, Errors 0 0 0 0 151 incr XferLOG, Errors 0 0 0 0 152 incr XferLOG, Errors 0 0 0 0 153 full XferLOG, Errors 1 0 0 0 154 incr XferLOG, Errors 0 0 0 0 155 incr XferLOG, Errors 0 0 0 0 156 incr XferLOG, Errors 0 0 0 0 Thanks, Jx That is unbelievable. You backup a windows client, right? There should be a lot of files which can not be backuped because they are in use. Do you use volume shadow copies in windows? But nevertheless, you have backups of all files specified in your configuration. If you check your backup include/exclude configuration you should find which files are not backuped. br Matthias -- Don't Panic -- Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Restore issues
Matthias Meyer wrote at about 21:00:54 +0200 on Thursday, July 30, 2009: Vetch wrote: Hi Matthias, All my xferlogs say they have 0 errors (apart from one, but that was after the problem occurred anyway)... I've had a look at them, but they are... quite long... Without knowing what to search for, I'm not sure what I can do with them... If they report no errors, I guess I can assume all files are backing up properly? Xfer Error Summary Backup# Type View #Xfer errs #bad files #bad share #tar errs 0 full XferLOG, Errors 0 0 0 0 28 full XferLOG, Errors 0 0 0 0 56 full XferLOG, Errors 0 0 0 0 84 full XferLOG, Errors 0 0 0 0 112 full XferLOG, Errors 0 0 0 0 126 full XferLOG, Errors 0 0 0 0 133 full XferLOG, Errors 0 0 0 0 140 full XferLOG, Errors 0 0 0 0 147 full XferLOG, Errors 0 0 0 0 150 incr XferLOG, Errors 0 0 0 0 151 incr XferLOG, Errors 0 0 0 0 152 incr XferLOG, Errors 0 0 0 0 153 full XferLOG, Errors 1 0 0 0 154 incr XferLOG, Errors 0 0 0 0 155 incr XferLOG, Errors 0 0 0 0 156 incr XferLOG, Errors 0 0 0 0 Thanks, Jx That is unbelievable. You backup a windows client, right? There should be a lot of files which can not be backuped because they are in use. Do you use volume shadow copies in windows? Alternatively, you could just exclude the files that tend to be busy. Before I wrote my volume shadow copy script, I had a short list of excludes that eliminated all busy files. But nevertheless, you have backups of all files specified in your configuration. If you check your backup include/exclude configuration you should find which files are not backuped. br Matthias -- Don't Panic -- Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/ -- Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Restore issues
You should check the backup log file XferLOG. Each file which can not backup will be logged theire, including the reason. br Matthias -- Don't Panic -- ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Restore issues
Hi Matthias, All my xferlogs say they have 0 errors (apart from one, but that was after the problem occurred anyway)... I've had a look at them, but they are... quite long... Without knowing what to search for, I'm not sure what I can do with them... If they report no errors, I guess I can assume all files are backing up properly? Xfer Error Summary Backup# Type View #Xfer errs #bad files #bad share #tar errs 0 full XferLOG, Errors 0 0 0 0 28 full XferLOG, Errors 0 0 0 0 56 full XferLOG, Errors 0 0 0 0 84 full XferLOG, Errors 0 0 0 0 112 full XferLOG, Errors 0 0 0 0 126 full XferLOG, Errors 0 0 0 0 133 full XferLOG, Errors 0 0 0 0 140 full XferLOG, Errors 0 0 0 0 147 full XferLOG, Errors 0 0 0 0 150 incr XferLOG, Errors 0 0 0 0 151 incr XferLOG, Errors 0 0 0 0 152 incr XferLOG, Errors 0 0 0 0 153 full XferLOG, Errors 1 0 0 0 154 incr XferLOG, Errors 0 0 0 0 155 incr XferLOG, Errors 0 0 0 0 156 incr XferLOG, Errors 0 0 0 0 Thanks, Jx On Fri, Jul 24, 2009 at 9:33 AM, Matthias Meyer matthias.me...@gmx.liwrote: You should check the backup log file XferLOG. Each file which can not backup will be logged theire, including the reason. br Matthias -- Don't Panic -- ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/ -- ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Restore issues
Vetch wrote: Ok - I tried direct restores back into the original directories over the network - and it came up with successful restores for all the home directories... Does this mean that the data has been fully successfully restored? Probably. But as I said, BackupPC didn't check file consistency. I think I have about 1000 files missing (out of about 35000)... You think? Do you really miss one of this 1000 files? Now, this wouldn't be the end of the world, but I'd be interested to know if when it reports success, it has definitely brought back the entire dataset... Yes. But possible not all the files YOU expected in the backup dataset. ... and if so... do you have any suggestions as to why I may have different numbers of files? How do you measure the file counts? I've just tried this - I booted to a live CD and e2fsck-ed the device... On first scan, it reported clean... I'm now running a e2fsck -f to force it to check, but assuming that it reports the device as clean, then can I assume that the backups are not corrupted? Yes! In which case, I have to wonder about the missing files... Am I just worrying unneccessarily? Probably. We didn't know yet if really files are missing or if your measurement is wrong. br Matthias -- Don't Panic -- ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Restore issues
Hi Matthias, Thanks for your help on this... See replies below... On Tue, Jul 21, 2009 at 6:40 PM, Matthias Meyer matthias.me...@gmx.liwrote: Vetch wrote: Ok - I tried direct restores back into the original directories over the network - and it came up with successful restores for all the home directories... Does this mean that the data has been fully successfully restored? Probably. But as I said, BackupPC didn't check file consistency. Ok... So it could be that the files are corrupt in the database... Though given the positive e2fsck results, that seems unlikely... I think I have about 1000 files missing (out of about 35000)... You think? Do you really miss one of this 1000 files? No - I haven't noticed any missing files, so I'm not too worried (realistically, I tend to keep numerous versions of any important document I'm working on anyway, so...) Now, this wouldn't be the end of the world, but I'd be interested to know if when it reports success, it has definitely brought back the entire dataset... Yes. But possible not all the files YOU expected in the backup dataset. Ok... so backuppc managed to restore all the data it has available... ... but perhaps I didn't count properly, or alternatively, didn't backup properly in the first place... ... and if so... do you have any suggestions as to why I may have different numbers of files? How do you measure the file counts? I took the somewhat unscientific approach of using an offline backup and using windows to count the files based on the properties... Essentially, I had one of the folders stored as an offline backup (the one with 35000-ish files in) which I synchronise most days... I used the windows properties to count files in the folder (36026) and then I took an archive (zip) copy of my offline directory which also 36026 files in it, based on the archive file count... I then connected to the server, synchronised and used the windows properties on it again... It showed (I can't remember now exactly), but I believe it was around 35200... I then restored the archive and now the folder properties report 36093 (I attribute the extra files to being ones which are not offline synchronisable using the Windows XP offline files (e.g. .pst files, etc)... Now, potentially, I guess that means that there could have just been more files on my offline copy (I expected about 14 since the offline synch claimed that 14 files had changed and needed to be synchronised)... Equally, possibly there were files on my offline copy that couldn't be copied to the server through synch and I didn't know... ... though I would have expected to have noticed previously... Equally, it's possible that the backuppc user didn't have rights on the server to backup all files on the server, but I was ssh-ing in and sudo-ing the command, so I believe that should give it root access for the rsync command... I don't know - it just seems like there should have been more files... Like I say, I'm not particualarly bothered, as I think it's highly unlikely any of the files I genuinely need have been completely lost, but still... I'd be interested in knowing what caused the discrepancy - if it's my counting, my setup, my configuration or the system behaving strangely... I've just tried this - I booted to a live CD and e2fsck-ed the device... On first scan, it reported clean... I'm now running a e2fsck -f to force it to check, but assuming that it reports the device as clean, then can I assume that the backups are not corrupted? Yes! Excellent... Well, that's good news... In which case, I have to wonder about the missing files... Am I just worrying unneccessarily? Probably. We didn't know yet if really files are missing or if your measurement is wrong. Heh - it's probably my measurement, isn't it? ;) Oh well - let's hope so ;) Once again many thanks, Jx br Matthias -- Don't Panic -- ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/ -- ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Restore issues
Vetch wrote at about 00:14:06 +0100 on Wednesday, July 22, 2009: Hi Matthias, Thanks for your help on this... See replies below... On Tue, Jul 21, 2009 at 6:40 PM, Matthias Meyer matthias.me...@gmx.liwrote: Vetch wrote: Ok - I tried direct restores back into the original directories over the network - and it came up with successful restores for all the home directories... Does this mean that the data has been fully successfully restored? Probably. But as I said, BackupPC didn't check file consistency. Ok... So it could be that the files are corrupt in the database... Though given the positive e2fsck results, that seems unlikely... I think I have about 1000 files missing (out of about 35000)... You think? Do you really miss one of this 1000 files? No - I haven't noticed any missing files, so I'm not too worried (realistically, I tend to keep numerous versions of any important document I'm working on anyway, so...) Now, this wouldn't be the end of the world, but I'd be interested to know if when it reports success, it has definitely brought back the entire dataset... Yes. But possible not all the files YOU expected in the backup dataset. Ok... so backuppc managed to restore all the data it has available... ... but perhaps I didn't count properly, or alternatively, didn't backup properly in the first place... ... and if so... do you have any suggestions as to why I may have different numbers of files? How do you measure the file counts? I took the somewhat unscientific approach of using an offline backup and using windows to count the files based on the properties... Essentially, I had one of the folders stored as an offline backup (the one with 35000-ish files in) which I synchronise most days... I used the windows properties to count files in the folder (36026) and then I took an archive (zip) copy of my offline directory which also 36026 files in it, based on the archive file count... I then connected to the server, synchronised and used the windows properties on it again... It showed (I can't remember now exactly), but I believe it was around 35200... I then restored the archive and now the folder properties report 36093 (I attribute the extra files to being ones which are not offline synchronisable using the Windows XP offline files (e.g. .pst files, etc)... Now, potentially, I guess that means that there could have just been more files on my offline copy (I expected about 14 since the offline synch claimed that 14 files had changed and needed to be synchronised)... Equally, possibly there were files on my offline copy that couldn't be copied to the server through synch and I didn't know... ... though I would have expected to have noticed previously... Equally, it's possible that the backuppc user didn't have rights on the server to backup all files on the server, but I was ssh-ing in and sudo-ing the command, so I believe that should give it root access for the rsync command... I don't know - it just seems like there should have been more files... Like I say, I'm not particualarly bothered, as I think it's highly unlikely any of the files I genuinely need have been completely lost, but still... I'd be interested in knowing what caused the discrepancy - if it's my counting, my setup, my configuration or the system behaving strangely... Well, with Windows there can be all kinds of reasons for mismatches including: 1. Busy files that can't be backed up (not just .pst files but also other open files and various registry and system files) 2. Permissions/acl issues - even with ssh as admin/root there may still be files you can't access. I'm not a Windows guru but I know it's not as simple as linux where root can read everything 3. Junctions can end up being double counted 4. Other weird Windows detritus - I have at times had weird ntfs files that hang around and are non-deletable (until I boot into Linux) Maybe I'm biased, but I always find Windows to be way more cumbersome, obscure, and unpredictable than the simple metaphors of *nix filesystems. I've just tried this - I booted to a live CD and e2fsck-ed the device... On first scan, it reported clean... I'm now running a e2fsck -f to force it to check, but assuming that it reports the device as clean, then can I assume that the backups are not corrupted? Yes! Excellent... Well, that's good news... In which case, I have to wonder about the missing files... Am I just worrying unneccessarily? Probably. We didn't know yet if really files are missing or if your measurement is wrong. Heh - it's probably my measurement, isn't it? ;) Oh well - let's hope so ;) Once again many thanks, Jx br Matthias -- Don't Panic
Re: [BackupPC-users] Restore issues
Hi, sorry for replying so late, busy week. Matthias Meyer wrote on 2009-07-20 21:59:49 +0200 [Re: [BackupPC-users] Restore issues]: Vetch wrote: I tried restoring a backup to my system - it kept on failing with aborted by signal=PIPE. I have a feeling it may have been related to corrupt files within the hardlinks, since when I tried restoring using tar, the tar files were not readable, How did you create the tar file(s), what was the error reported? Was it completely unreadable, or did it abort on a specific file? Always the same file? Did you try with different source backups? Even though you might be interested in the lastest versions of your files, choosing an older backup to restore might give you some clues (or older versions of missing files). and the zip files didn't have as many files in as I would have expected... How did you create the zip file(s)? I guess if there are problems with the data, there's probably nothing that can be done? Probably not much :-( Well, the part of the data that is unaffected should be, err, unaffected. There is probably a lot you can do, assuming a partial restore is better than nothing. You might be able to restore older versions of files that are missing in the latest backup. One of the virtues of BackupPC is that you have file level access to all files. It's not like a read error on a tape or a compressed archive, beyond which all data may be lost. And you can quickly locate different versions of the same file in different backups. If the problem is with the tar stream (e.g. a huge file that is incorrectly encoded), you can always directly access the files with BackupPC_zcat, though that won't scale to many files. But it does mean you may be able to access more data than at first apparent. How does backuppc check the consistency of the data once it's been added to the pool? I know that it only backs up the data once, but how does it make sure that the data is in a valid state? That is not completely true. Depending on the XferMethod, each *full* backup will either re-transfer the complete data set and reuse only perfectly matching pool files (i.e. on-disk corruption would *not* cause a corrupted file to be reused), or at least check all data against the stored backup and re-create any changed files (also causing corrupted files not to be reused). With rsync(d) and --checksum-seed in RsyncArgs this checking is limited to RsyncCsumCacheVerifyProb - 1% of the files by default - so a corrupted file may in fact be reused. This is why this behaviour is turned *off* by default (i.e. if you didn't enable it, it's off; if you did, you know it, and it's your fault ;-). I didn't believe that backuppc will check data consistency. That is the job of a filesystem. e2fsck will do that for ext2/3 file systems. That is incorrect. e2fsck will check *file system metadata* for *detectable* corruption. I don't believe e2fsck will attempt to read all used data blocks - that could take hours on large file systems, and would make the badblocks(8) program somewhat pointless - and, in any case, it couldn't check data consistency beyond detecting read errors reported by the disk hardware. So, no, e2fsck will do nothing more than confirm that your file system metadata has no detectable errors. You should run it regulary. Yes, probably. Though I have not yet heard any reports of if and in what time e2fsck successfully completes for BackupPC pool file systems. It may or may not have problems with the large number of files with more than one link. I don't know. I assume you are running BackupPC on Linux and use ext3 as filesystem. So I would advise: 1) make a image backup (e.g. partimage) of your /var/lib/backuppc or wherever your __TOPDIR__ resides. This part is the most important of all. e2fsck can effectively lose data you could have accessed before (or, at least, make it next to impossible to find). If your file system is already damaged beyond recognition, you don't have much options, but it doesn't seem to be. I'd recommend mounting it read-only instead of checking it. As soon as you have a copy of as much data as you can (and need to) retrieve, you can run e2fsck and see if the situation improves. For your BackupPC pool, I wouldn't recommend trusting a repaired file system (unless only trivial things were repaired that didn't make much of a difference in the first place). File system metadata doesn't contain much redundancy to allow for reconstructing lost information. Unattached inodes can be found, but will you have any idea, where in which backup(s) lost+found/#123456 really belongs? Incorrect link counts can be corrected, but that does not restore the missing links, it simply acknowledges the fact that they have been lost. Contradictory information, that would crash the file system driver, can be resolved, so that it will not, but that does not mean the result is correct in the sense of as it was before corruption. If your
[BackupPC-users] Restore issues
Hi all, I have a question... I tried restoring a backup to my system - it kept on failing with aborted by signal=PIPE. I have a feeling it may have been related to corrupt files within the hardlinks, since when I tried restoring using tar, the tar files were not readable, and the zip files didn't have as many files in as I would have expected... I didn't seem to be able to resolve that, so I wiped the system (keeping the home directory in tact), and rebuilt the server. Now, unfortunately, the software I use to run the file server decided to delete my home directories when I restored the configuration... Go figure... So now, I'm really worried because I think that I will have lost about 100GB of data that I believed was backed up... Anyway, I'm trying to restore now, but the tar files are still coming up as unreadable... I guess if there are problems with the data, there's probably nothing that can be done? How does backuppc check the consistency of the data once it's been added to the pool? I know that it only backs up the data once, but how does it make sure that the data is in a valid state? Is it checksums? If so, then I guess it can't be a corruption error... ... but I don't get why I would be finding directories of 3000 files showing only 1500 in a full backup... Can anyone help? Any support would be greatly appreciated... I've got all the coursework from a Masters and a PhD on there! Jx -- Enter the BlackBerry Developer Challenge This is your chance to win up to $100,000 in prizes! For a limited time, vendors submitting new applications to BlackBerry App World(TM) will have the opportunity to enter the BlackBerry Developer Challenge. See full prize details at: http://p.sf.net/sfu/Challenge___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Restore issues
Vetch wrote: Hi all, I have a question... I tried restoring a backup to my system - it kept on failing with aborted by signal=PIPE. I have a feeling it may have been related to corrupt files within the hardlinks, since when I tried restoring using tar, the tar files were not readable, and the zip files didn't have as many files in as I would have expected... I didn't seem to be able to resolve that, so I wiped the system (keeping the home directory in tact), and rebuilt the server. Which system? Your BackupPC server or your backup client? Now, unfortunately, the software I use to run the file server decided to delete my home directories when I restored the configuration... Go figure... So now, I'm really worried because I think that I will have lost about 100GB of data that I believed was backed up... Anyway, I'm trying to restore now, but the tar files are still coming up as unreadable... I guess if there are problems with the data, there's probably nothing that can be done? Probably not much :-( How does backuppc check the consistency of the data once it's been added to the pool? I know that it only backs up the data once, but how does it make sure that the data is in a valid state? Is it checksums? If so, then I guess it can't be a corruption error... ... but I don't get why I would be finding directories of 3000 files showing only 1500 in a full backup... I didn't believe that backuppc will check data consistency. That is the job of a filesystem. e2fsck will do that for ext2/3 file systems. You should run it regulary. Can anyone help? Any support would be greatly appreciated... I've got all the coursework from a Masters and a PhD on there! Jx I assume you are running BackupPC on Linux and use ext3 as filesystem. So I would advise: 1) make a image backup (e.g. partimage) of your /var/lib/backuppc or wherever your __TOPDIR__ resides. 2) try to open some files (select files which are importend for you) from the BackupPC GUI (use your backup client, open http:\\your server/backuppc, login, left hand menu, Browse Backups, click on a file and say open) 3) run e2fsck on the device where /var/lib/backuppc located (unmount the device first) and expect to lost data if e2fsck find file system errors. You should run e2fsck as soon as possible. Elsewere filesystem errors can grows and destroy more data then necessary. I have had a similiar problem 7 month ago. A lot of file system errors and I lost most of my backups. Fortunately I didn't lost data on my backup clients at the same time. br Matthias -- Don't Panic -- Enter the BlackBerry Developer Challenge This is your chance to win up to $100,000 in prizes! For a limited time, vendors submitting new applications to BlackBerry App World(TM) will have the opportunity to enter the BlackBerry Developer Challenge. See full prize details at: http://p.sf.net/sfu/Challenge ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Restore Issues with larger files
Lee writes: Sorr I missed information. I am restoring to RHEL4 box. I can transfer 500MB files no problem, but once I try 2GB or more it fails with a PIPE error. I have been able to transfer a 2GB file maybe twice out of the 20 times I've tried. Any ideas? Not directly related to the original post but i seem to have lots of problems relating to larger files with rsync, even when I am backing up large files, like Exchange database, it causes issues, but it doesn't have this issue with every deployment of backuppc, I'm not sure what to do. I've tried changing my transfer method from rsync to smb and this is what I get in the log when backing up a windows machine. What version of BackupPC and smbclient are you using? Earlier versions of both have problems with large files. Craig - This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/ ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Restore Issues with larger files
Craig I switched back to rsync, as I found the issue, it wasn't backuppc's client timeout I needed to change, but I totally forgot about the host's rsync timeout, after increasing that, everything is working like a champ!!! Thanks for reply! Lee Connell Ammonoosuc Computer Services, Inc. Network Engineer 15 Main St. Suite 10 Littleton, NH 03561 603-444-3937 If you require immediate response please send your inquiry to [EMAIL PROTECTED] -Original Message- From: Craig Barratt [mailto:[EMAIL PROTECTED] Sent: Tuesday, May 20, 2008 4:40 PM To: Lee A. Connell Cc: backuppc-users@lists.sourceforge.net Subject: Re: [BackupPC-users] Restore Issues with larger files Lee writes: Sorr I missed information. I am restoring to RHEL4 box. I can transfer 500MB files no problem, but once I try 2GB or more it fails with a PIPE error. I have been able to transfer a 2GB file maybe twice out of the 20 times I've tried. Any ideas? Not directly related to the original post but i seem to have lots of problems relating to larger files with rsync, even when I am backing up large files, like Exchange database, it causes issues, but it doesn't have this issue with every deployment of backuppc, I'm not sure what to do. I've tried changing my transfer method from rsync to smb and this is what I get in the log when backing up a windows machine. What version of BackupPC and smbclient are you using? Earlier versions of both have problems with large files. Craig No virus found in this incoming message. Checked by AVG. Version: 7.5.524 / Virus Database: 269.23.21/1456 - Release Date: 5/20/2008 6:45 AM - This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/ ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Restore Issues with larger files
I get the error below with large files, this one is around 2GB, files around 500MB transfer and restore with no problem. What can be causing this? I am running rsync 2.6.8 on both client and server. You are restoring to Linux, Windows, MacOSX? 2G is, for example the size limit for FAT32, no way you can restore a bigger file to a, say WinXP with FAT32. Eduardo. - This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/ ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Restore Issues with larger files
You are restoring to Linux, Windows, MacOSX? 2G is, for example the size limit for FAT32, no way you can restore a bigger file to a, say WinXP with FAT32. Eduardo. I think you mean that 2 GB is the limit for FAT16. Windows 95a was the last FAT16 operating system released by Microsoft. The limit for FAT32 is 8 TB. This was released with Windows 95b in 1996. How many of us in 1996 figured that we would have terabyte drives in 2008? It is reported that FAT64 will support up to 16 exabytes. Chris Baker -- [EMAIL PROTECTED] systems administrator Intera Inc. -- 512-425-2006 - This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/ ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Restore Issues with larger files
Chris Baker wrote: You are restoring to Linux, Windows, MacOSX? 2G is, for example the size limit for FAT32, no way you can restore a bigger file to a, say WinXP with FAT32. Eduardo. I think you mean that 2 GB is the limit for FAT16. Windows 95a was the last FAT16 operating system released by Microsoft. No, he means that 2GB is the _file_ size limit for FAT32 for any single file. The limit for FAT32 is 8 TB. That's the volume size limit with the maximum cluster size. This was released with Windows 95b in 1996. How many of us in 1996 figured that we would have terabyte drives in 2008? Apparently no one at all, since scandisk will only check volumes of 128 gb or less. -- Les Mikesell [EMAIL PROTECTED] - This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/ ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Restore Issues with larger files
Sorr I missed information. I am restoring to RHEL4 box. I can transfer 500MB
files no problem, but once I try 2GB or more it fails with a PIPE error. I
have been able to transfer a 2GB file maybe twice out of the 20 times I've
tried. Any ideas?
Not directly related to the original post but i seem to have lots of problems
relating to larger files with rsync, even when I am backing up large files,
like Exchange database, it causes issues, but it doesn't have this issue with
every deployment of backuppc, I'm not sure what to do.
I've tried changing my transfer method from rsync to smb and this is what I get
in the log when backing up a windows machine.
}Üh �ÜUðßàöÄzÜ` �ÜUøé{ÿÿÿÝØ3À[^ÉÂ�Ì‹ÿU‹ìQ‹E
S3Û;Ãu
[EMAIL PROTECTED]/attrib (ed8d4fc28920fb5327c53f64019fa61b)
tarExtract: Unexpected end of tar archive (tot = 1048576, num = 287232, posn = )
tarExtract: Removing partial file
�‹ø;û|I8 Ôt‹E‹@@8X/ujôÿÌ�Pè”øÿÿ„ÀtVjëVS‹Eü‹jPÿ‘€���‹ø‹Eü‹PÿQ;û}‹Çë‹E
‹
‰‹6‹VÿP3À_^[ÉÂ�Ì‹ÿU‹ìVÿu
ÿuÿÔ�‹ð…ö}G�þó€u ÿuh
tarExtract: BackupPC_tarExtact aborting (Unexpected end of tar archive)
tarExtract: Done: 188 errors, 48 filesExist, 802 sizeExist, 181034
sizeExistComp, 66 filesTotal, 21520584660 sizeTotal
Got fatal error during xfer (Unexpected end of tar archive)
Backup aborted (Unexpected end of tar archive)
Lee Connell
Ammonoosuc Computer Services, Inc.
Network Engineer
15 Main St. Suite 10
Littleton, NH 03561
603-444-3937
If you require immediate response please send your inquiry to
[EMAIL PROTECTED]
-Original Message-
From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Chris Baker
Sent: Monday, May 19, 2008 3:16 PM
To: backuppc-users@lists.sourceforge.net
Subject: Re: [BackupPC-users] Restore Issues with larger files
You are restoring to Linux, Windows, MacOSX? 2G is, for example the size
limit for FAT32, no way you can restore a bigger file to a, say WinXP with
FAT32.
Eduardo.
I think you mean that 2 GB is the limit for FAT16. Windows 95a was the last
FAT16 operating system released by Microsoft.
The limit for FAT32 is 8 TB. This was released with Windows 95b in 1996. How
many of us in 1996 figured that we would have terabyte drives in 2008?
It is reported that FAT64 will support up to 16 exabytes.
Chris Baker -- [EMAIL PROTECTED]
systems administrator
Intera Inc. -- 512-425-2006
-
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/
No virus found in this incoming message.
Checked by AVG.
Version: 7.5.524 / Virus Database: 269.23.21/1454 - Release Date: 5/19/2008
7:44 AM
-
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Restore Issues with larger files
You are restoring to Linux, Windows, MacOSX? 2G is, for example the size limit for FAT32, no way you can restore a bigger file to a, say WinXP with FAT32. I think you mean that 2 GB is the limit for FAT16. Windows 95a was the last FAT16 operating system released by Microsoft. No, I did mean FAT32. The size limit (for *files*) is 4G (not 2G, sorry). The limit for FAT32 is 8 TB. This was released with Windows 95b in 1996. How many of us in 1996 figured that we would have terabyte drives in 2008? That is the limit for the *drive* size. But whoever posted the original message was trying to recover a file from backuppc, not a disk. 4G for a restore can be reached pretty easily, it just happened to me yesterday on a WindowsXP with FAT32 and the error message you get is in the lines of disk full. Eduardo. - This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/ ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
[BackupPC-users] Restore Issues with larger files
I get the error below with large files, this one is around 2GB, files around 500MB transfer and restore with no problem. What can be causing this? I am running rsync 2.6.8 on both client and server. Server Log Sorted file list has 1 entries PostSortFile 0: BACKUP_DB_WBBI_S_3201_P_2_T_654815103 Got #0 (BACKUP_DB_WBBI_S_3201_P_2_T_654815103), blkCnt=0, blkSize=0, rem=0 /var/lib/backuppc/pc/prod/62/fWBBIBACKUP/fBACKUP_DB_WBBI_S_3201_P_2_T_65 4815103 cache = , invalid = , phase = 0 BACKUP_DB_WBBI_S_3201_P_2_T_654815103: opened for read exiting after signal PIPE Restore failed: aborted by signal=PIPE Client Log 2008/05/16 16:06:02 [12664] connect from prometheus (192.168.0.200) 2008/05/16 16:06:02 [12664] rsync to oracle/ from [EMAIL PROTECTED] (192.168.0.200) 2008/05/16 20:11:02 [12664] rsync error: timeout in data send/receive (code 30) at io.c(171) [receiver=2.6.8] 2008/05/16 20:11:02 [12664] rsync: connection unexpectedly closed (258 bytes received so far) [generator] 2008/05/16 20:11:02 [12664] rsync error: error in rsync protocol data stream (code 12) at io.c(463) [generator=2.6.8] Lee Connell Ammonoosuc Computer Services, Inc. Network Engineer 15 Main St. Suite 10 Littleton, NH 03561 603-444-3937 If you require immediate response please send your inquiry to [EMAIL PROTECTED] mailto:[EMAIL PROTECTED] - This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Restore Issues with larger files
Has anyone used the Western Digital My Book studio drives with Linux? Do they work with Linux? Western Digital says that they don't support them. Chris Baker -- [EMAIL PROTECTED] systems administrator Intera Inc. -- 512-425-2006 _ From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Lee A. Connell Sent: Friday, May 16, 2008 3:34 PM To: backuppc-users@lists.sourceforge.net Subject: [BackupPC-users] Restore Issues with larger files I get the error below with large files, this one is around 2GB, files around 500MB transfer and restore with no problem. What can be causing this? I am running rsync 2.6.8 on both client and server. Server Log Sorted file list has 1 entries PostSortFile 0: BACKUP_DB_WBBI_S_3201_P_2_T_654815103 Got #0 (BACKUP_DB_WBBI_S_3201_P_2_T_654815103), blkCnt=0, blkSize=0, rem=0 /var/lib/backuppc/pc/prod/62/fWBBIBACKUP/fBACKUP_DB_WBBI_S_3201_P_2_T_654815 103 cache = , invalid = , phase = 0 BACKUP_DB_WBBI_S_3201_P_2_T_654815103: opened for read exiting after signal PIPE Restore failed: aborted by signal=PIPE Client Log 2008/05/16 16:06:02 [12664] connect from prometheus (192.168.0.200) 2008/05/16 16:06:02 [12664] rsync to oracle/ from [EMAIL PROTECTED] (192.168.0.200) 2008/05/16 20:11:02 [12664] rsync error: timeout in data send/receive (code 30) at io.c(171) [receiver=2.6.8] 2008/05/16 20:11:02 [12664] rsync: connection unexpectedly closed (258 bytes received so far) [generator] 2008/05/16 20:11:02 [12664] rsync error: error in rsync protocol data stream (code 12) at io.c(463) [generator=2.6.8] Lee Connell Ammonoosuc Computer Services, Inc. Network Engineer 15 Main St. Suite 10 Littleton, NH 03561 603-444-3937 If you require immediate response please send your inquiry to mailto:[EMAIL PROTECTED] [EMAIL PROTECTED] - This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
