Hi I have a problem with the rsync incremental backups of a 50 GB binary file. My set up consists of: - Backuppc 3.2.1 running on Centos 5.7 with rsync 3.0.6. The server has 2 GB of RAM with a single Xeon processor - Client being backed up is Ubuntu Lucid server - Backup is done over a wireless link between 2 buildings which can provide for a 11 MB/s when tested with a plain rsync of a file between the 2 servers. - The client receives a nightly copy of a ntbackup of a exchange 2003 database to the /opt/samba/ntbackups folder. The file is currently around 51 GB in size
The problem I am having is that the incremental backups take around double the time the full backups. These are the backuppc logs for full backup following by a incremental on the next night: Previous nights full: 2012-01-06 14:47:42 full backup started for directory / (baseline backup #377) 2012-01-06 14:47:44 full backup started for directory /etc (baseline backup #377) 2012-01-06 14:47:57 full backup started for directory /var (baseline backup #377) 2012-01-06 14:50:49 full backup started for directory /opt (baseline backup #377) 2012-01-06 18:38:38 full backup started for directory /root (baseline backup #377) 2012-01-06 18:38:42 full backup started for directory /home (baseline backup #377) 2012-01-06 19:02:59 full backup started for directory /usr (baseline backup #377) 2012-01-06 19:07:07 full backup started for directory /bin (baseline backup #377) 2012-01-06 19:07:10 full backup started for directory /sbin (baseline backup #377) 2012-01-06 19:07:12 full backup started for directory /lib (baseline backup #377) 2012-01-06 19:08:32 full backup 378 complete, 202077 files, 641375636764 bytes, 1 xferErrs (0 bad files, 0 bad shares, 1 other) Last nights incremental 2012-01-07 15:00:10 incr backup started back to 2012-01-06 14:47:42 (backup #378) for directory / 2012-01-07 15:00:14 incr backup started back to 2012-01-06 14:47:42 (backup #378) for directory /etc 2012-01-07 15:00:21 incr backup started back to 2012-01-06 14:47:42 (backup #378) for directory /var 2012-01-07 15:03:39 incr backup started back to 2012-01-06 14:47:42 (backup #378) for directory /opt 2012-01-07 23:56:40 incr backup started back to 2012-01-06 14:47:42 (backup #378) for directory /root 2012-01-07 23:56:45 incr backup started back to 2012-01-06 14:47:42 (backup #378) for directory /home 2012-01-07 23:57:33 incr backup started back to 2012-01-06 14:47:42 (backup #378) for directory /usr 2012-01-07 23:59:42 incr backup started back to 2012-01-06 14:47:42 (backup #378) for directory /bin 2012-01-07 23:59:43 incr backup started back to 2012-01-06 14:47:42 (backup #378) for directory /sbin 2012-01-07 23:59:45 incr backup started back to 2012-01-06 14:47:42 (backup #378) for directory /lib 2012-01-08 00:00:26 incr backup 379 complete, 126 files, 55831250555 bytes, 0 xferErrs (0 bad files, 0 bad shares, 0 other) You can see that the backup of the /opt share takes nearly the total time of the incremental taking about 8 and half hours to complete while the backup of the /opt rsync share in the full backup takes about 3 and half hours. The full backup is slightly longer than what it takes if I just do a rsync over ssh copy of the file from the client server to the backup server. I have found that rsync seems to always transfer the whole file when copying this file from the client server to the backup server: # rsync -avzh --progress -e ssh administrator@isabella:ExchangeDailyBackup.bkf ExchangeDailyBackup.bkf Password: receiving incremental file list ExchangeDailyBackup.bkf 54.44G 100% 10.66MB/s 1:21:10 (xfer#1, to-check=0/1)4 sent 3.31M bytes received 3.27G bytes 486.33K bytes/sec total size is 54.44G speedup is 16.65. My questions for the list are: 1. Is it reasonable for rsync to transfer the whole file when copying a large ntbackup file? 2. Why does an incremental backup of this file take so much longer than a full backup of it or a plain rsync of this file? Thank you -- John Habermann IT Officer Cook Shire Council 10 Furneaux St Cooktown 4895 ph 40820577 ------------------------------------------------------------------------------ Ridiculously easy VDI. With Citrix VDI-in-a-Box, you don't need a complex infrastructure or vast IT resources to deliver seamless, secure access to virtual desktops. With this all-in-one solution, easily deploy virtual desktops for less than the cost of PCs and save 60% on VDI infrastructure costs. Try it free! http://p.sf.net/sfu/Citrix-VDIinabox _______________________________________________ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List: https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki: http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/