On 08/27/2012 07:19 PM, martin f krafft wrote: > Dear list, > > I am very happy with BackupPC, except that it hasn't been able to > back up my workstation for more than 3 months. It started a full > backup on 14 May 2012, shortly after I had stored a set of large > files to disk. Since my workstation is behind an ADSL connection, > the backup eventually timed out and was stored as a partial backup > (#507). > > Every day since then, the full backup started again, and every day > since then, it has been timing out. > > This is especially annoying since the exact same files have already > been backed up to the server once from another host, so one would > like to assume that BackupPC does not need to transfer them all over > again. > > I understand that the rsync protocol does not allow this and > a BackupPC client would be needed to shortcut uploading of files > that already exist on the server. > > However, the status quo seems broken to me. If BackupPC times out on > a backup and stores a partial backup, it should be able to resume > the next day. But this is not what seems to happen. The log seems to > suggest that each backup run uses the previous full backup (#506, > not the partial backup #507) as baseline: > > full backup started for directory / (baseline backup #506) > > I only just turned on verbose logging to find out what's actually > going on and whether there is any progress being made, but to me it > seems like BackupPC is failing to build upon the work done for the > last partial backup and keeps trying again and again. > > Has anyone seen this behaviour and do you have any suggestions how > to mitigate this problem? > Yes, and I've found the best way to backup large files over slow links is to split them into smaller files using split. However, for a random user who dumps a DVD image or HDD image to their computer, this is never going to work. This only works if you can control the system and how the user behaves.
Solutions: 1) Add a max size option to rsync and inform users that files larger than X will not be backed up (probably not a good solution if that large file just happened to be very important) 2) Split large files prior to backup, again not so useful if you can't control the user, or this isn't a standard backup file. ie, random users doing random things 3) Do not use backuppc to get a copy of the files across the slow link. Use standard rsync with options like partial to make a copy of the files local to the backup server, then use backuppc to backup this local store. The second option is what I frequently use when a remote server has a large file (ie, MS SQL dump, or disk image, etc), so I just split the large file prior to backuppc doing the backup and exclude the original file. I am seriously considering looking more into option 3, however, the downside of this is that backuppc will no longer be able to inform me of a lack of recent backups, since it will always succeed to backup the local copy. I would need a fairly robust system which will ensure regular copies of the remote system are successfully being made to the local system. PS, the real problem with what you are seeing is possibly that backuppc isn't backing up one of the large files before being interrupted/timing out. Since it doesn't continue the backup from the middle of the file, you never complete that file, and therefore will never complete the backup. It would be ideal if backuppc was able to keep a partial file in a partial backup, so that it could continue the backup on the next run. Even more intelligent would be the ability to continue the backup using both the partial file AND the previous complete copy (if one existed) so that when a large file (disk image) was regularly backed up, and interrupted half way through (ADSL dropped sync) then it would continue instead of doing a full copy of the second half of the file by using the second half of the original file. Regards, Adam ------------------------------------------------------------------------------ Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ _______________________________________________ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List: https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki: http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
