Hi, Clemens von Musil wrote on 20.01.2007 at 00:31:12 [[BackupPC-users] Avoiding long backup times]: > [...] > One configured backup host has a very slow network connection - a full > backup lasts about two days. Because of the exclusive run of > backuppc_nightly in 2.xx, I learned, this full backup stalls all other > backups. I want to avoid tihs situation and got stuck in following: > > What exactly happens during full backup?
that depends on the transfer method. For a slow connection, you want rsync (or rsyncd), not tar or smb. For a very slow connection, you *definitely* want rsync/rsyncd (which I'll just call "rsync" for simplicity). > I read, that backuppc stores every identical file only one time. Basically true, but BackupPC needs to determine that the file is identical to something and to what. Sparing you the transfer (if possible with reasonable cost) is rsyncs job. Doing so comes at a cost in terms of CPU usage, so you've got the option of using tar if bandwidth is cheaper than CPU power. In your case, it obviously is not. > [...] What happens with an unchanged file in a full backup? tar and smb will transfer it, rsync will not (presuming you mean "unchanged" as in "same name, same content"). For a changed file, rsync will try to speed up the transfer. If you append a few bytes to a large file, tar/smb will transfer the whole file (even on an incremental), while rsync will (basically) transfer some checksums and the few bytes only (on full and incremental backups). > If the file will not be transferred again - what is the difference > between full and incremenal? This only applies to rsync (as tar/smb will transfer it). rsync will always transfer missing files as well as update files that have apparently changed. The difference between full and incremental backups lies firstly in the amount of trouble rsync will go to to determine whether a file has changed or not. For an incremental backup, I believe rsync will only look at size and modification time, whereas for a full backup, checksums of the files are calculated, thus consuming much more CPU-time and disk-I/O-bandwidth at least on the client (I understand the server caches checksums if you tell it to), with the benefit of detecting files that have been changed and the modification time reset. The second difference is that a full backup gives a new reference point for following backups while an incremental backup does not necessarily. In version 2 (of BackupPC) all incrementals were relative to the last full (so after one year of only incrementals you'd be transfering everything changed in that year on each incremental backup), whereas version 3 supports multi-level incrementals, with each incremental transfering everything changed since the last incremental of lower level (resp. full backup at level 0). > And if yes - does it make sense to keep one full for ever and dealing > only with following incrementals? No. With a very slow network connection, you want to avoid transfering changes more often than necessary. Configuring multi-level incrementals in BackupPC 3 seems to be simple enough that you *could* say "do incrementals of increasing level each day for the next, say, 10 years", but that will make browsing, restoring and even doing the backups increasingly expensive (and you'll need to keep all the incrementals), even if you *can* neglect modified files your backups are missing. A full backup each day *might* be your best choice, because it avoids duplicating transfers. With any incrementals in between, you'll have at least the fulls re-transfering everything since the last full (less or equal to the sum of the intervening incrementals (plus the changes since the last incremental, obviously), due to files modified more than once or modified and deleted). You'll have to find out the best trade-off in your specific situation between retransfering file content and calculating checksums. You probably need to find out first, how much data you need to transfer each day. Then you can estimate, the traffic of how many days you could allow yourself to transfer at once. That should be about the maximum interval between full backups ... Regards, Holger ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys - and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV _______________________________________________ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/backuppc-users http://backuppc.sourceforge.net/
