[BackupPC-users] rsync or tar?
Hello to all, I am new in this List. We have a small office with 5 Mac OSX (10.4) and 3 PCs (XP). We decided to back up with Backuppc. The last few hours I searche the mailinglist and read a lot. But I am still not sure if: * it is better to use rsync or tar because of the resource forks? * I have to install additional software like xtar on the Macs? * How would a sample.pl configurationfile look if I just want to back up the folder /customers? This I can find out by myself. -- Thank you very much. Markus - This SF.net email is sponsored by DB2 Express Download DB2 Express C - the FREE version of DB2 express and take control of your XML. No limits. Just data. Click to get it now. http://sourceforge.net/powerbar/db2/ ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/backuppc-users http://backuppc.sourceforge.net/
[BackupPC-users] rsync vs Tar - big files
Some stats using rsync vs using tar on a file system with big files Server is a FreeBSD 6.2 box, 2.93Ghz Celeron with 768MB Ram., RAID10 on a 3ware 9500Scontroller. client is a Mac pro dual/dual xeon 2.66 6GB ram source drive 94GB of media files average file size 10MB on a 250GB SATA-300 drive. network switched gig-e tar (baseline) 10.7 MB/sec rsync 1st - 5.63 MB/sec (writing data on server) rsync 2nd - 10.27 MB/sec (reading data for rsync checksum compare but checksum cache not yet written) rsync 3rd - 35.20 MB/sec (rsync with server side checksums in cache) On all runs CPU was not the limiting factor. rsync is a big win with large files on machines with enough memory and CPU. - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT business topics through brief surveys-and earn cash http://www.techsay.com/default.php?page=join.phpp=sourceforgeCID=DEVDEV___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/backuppc-users http://backuppc.sourceforge.net/
Re: [BackupPC-users] RSync v. Tar
Hi, Les Mikesell wrote on 27.03.2007 at 01:03:32 [Re: [BackupPC-users] RSync v. Tar]: Jesse Proudman wrote: I've got one customer who's server has taken 3600 minutes to backup. 77 Gigs of Data. 1,972,859 small files. Would tar be better or make this faster? It's directly connected via 100 Mbit to ^^^ the backup box. ^^ If the files don't change frequently, tar incremental runs will be much faster because they are based only on the target timestamps while rsync will load the entire directory at both ends and compare them. if you ask me, regardless of how your data changes, tar is the way to go, not rsync, especially for *full* backups. With a direct 100 MBit connection, there's not much point in spending (lots of) CPU time for saving bandwidth - not with 2 million files. rsync is good for low bandwidth connections, where the link severely limits the transfer and speeding it up makes a real difference. In your case, your link speed is in the same order of magnitude as your disk I/O performance (considering our favorite topic, the seek times on the pool file system, the network link may in fact not even be the limiting factor - it clearly isn't, as 77 GB would take slightly more than 2 hours to transfer over a 100 MBit link, and rsync is not making it go faster than that ;-). rsync has additional benefits concerning finding and backing up new (or moved) files with old timestamps and deleted files on *incremental* backups, but keeping the list of 2 million files in memory will probably be a problem, as it possibly was with your full (?) backup (how much memory do the BackupPC server and the backed up host have?). (Les: if the files *do* change frequently, there's even less speedup to get from using rsync. Only frequent metadata changes without file content changes would give rsync an advantage - assuming it is faster to figure out that the file is identical than to simply send it over the network.) Regards, Holger P.S.: rsync checksum caching *might* make a difference starting from the third backup, but I read that it's less improvement than one might expect. - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT business topics through brief surveys-and earn cash http://www.techsay.com/default.php?page=join.phpp=sourceforgeCID=DEVDEV ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/backuppc-users http://backuppc.sourceforge.net/
Re: [BackupPC-users] RSync v. Tar
I have a bit of hard data to offer on this subject, as I recently switched a backup from tar+ssh (over cygwin) to rsyncd. The backuppc server is on the same physical LAN, and connect to each other via a 192.168 address. All the cabling and switches support 100 MB full duplex communications, and the servers have gigabit NICs. The backuppc server is dumping the data to the /var partition which is on the 2 80Gb satas in the case, running in a RAID 1 software array, under mdadm on Debian stable. These are the current stats, using rsyncd - Backup# Type #Files Size/MB MB/sec #Files Size/MB #Files Size/MB 30http://mail.stephanco.com/cgi-bin/BackupPC_Admin?action=browsehost=sarahnum=30 full 168480 41758.5 5.66 168333 41639.8 310 118.9 35http://mail.stephanco.com/cgi-bin/BackupPC_Admin?action=browsehost=sarahnum=35 incr 730 191.7 0.19 564 103.6 223 88.2 36http://mail.stephanco.com/cgi-bin/BackupPC_Admin?action=browsehost=sarahnum=36 incr 785 198.4 0.18 725 123.0 106 75.4 37http://mail.stephanco.com/cgi-bin/BackupPC_Admin?action=browsehost=sarahnum=37 full 169010 41836.6 5.73 168876 41750.2 276 86.5 38http://mail.stephanco.com/cgi-bin/BackupPC_Admin?action=browsehost=sarahnum=38 incr 0 0.0 0.00 0 0.0 29 0.0 39http://mail.stephanco.com/cgi-bin/BackupPC_Admin?action=browsehost=sarahnum=39 incr 0 0.0 0.00 0 0.0 0 0.0 40http://mail.stephanco.com/cgi-bin/BackupPC_Admin?action=browsehost=sarahnum=40 incr 155 89.4 0.04 42 5.5 169 83.9 41http://mail.stephanco.com/cgi-bin/BackupPC_Admin?action=browsehost=sarahnum=41 incr 321 124.2 0.05 142 19.8 234 104.4 Backup# Type Filled Level Start Date Duration/mins Age/days Server Backup Path 30http://mail.stephanco.com/cgi-bin/BackupPC_Admin?action=browsehost=sarahnum=30 full yes 0 3/16 18:00 122.9 11.2 /var/lib/backuppc/pc/sarah/30 35http://mail.stephanco.com/cgi-bin/BackupPC_Admin?action=browsehost=sarahnum=35 incr no 1 3/21 18:00 16.9 6.2 /var/lib/backuppc/pc/sarah/35 36http://mail.stephanco.com/cgi-bin/BackupPC_Admin?action=browsehost=sarahnum=36 incr no 1 3/22 18:00 17.9 5.2 /var/lib/backuppc/pc/sarah/36 37http://mail.stephanco.com/cgi-bin/BackupPC_Admin?action=browsehost=sarahnum=37 full yes 0 3/23 18:00 121.7 4.2 /var/lib/backuppc/pc/sarah/37 38http://mail.stephanco.com/cgi-bin/BackupPC_Admin?action=browsehost=sarahnum=38 incr no 1 3/24 18:00 16.5 3.2 /var/lib/backuppc/pc/sarah/38 39http://mail.stephanco.com/cgi-bin/BackupPC_Admin?action=browsehost=sarahnum=39 incr no 1 3/25 18:00 15.4 2.2 /var/lib/backuppc/pc/sarah/39 40http://mail.stephanco.com/cgi-bin/BackupPC_Admin?action=browsehost=sarahnum=40 incr no 1 3/26 18:00 33.1 1.2 /var/lib/backuppc/pc/sarah/40 41http://mail.stephanco.com/cgi-bin/BackupPC_Admin?action=browsehost=sarahnum=41 incr no 1 3/27 18:00 38.6 0.2 When it was doing tar, the full backups took far longer, in the neighborhood of 600 minutes. The incremental backups took around an hour most days. So I clearly made out better with rsyncd. Just as additional info, the server being backed up is a file server for a small company. It is backing up the directory where they store .jpg images of the products they sell. They organize it by date, so obviously everything in the current day's directory is new, but previous directories aren't modified most of the time. I hope that helps. Peace, Jim On 3/27/07, Holger Parplies [EMAIL PROTECTED] wrote: Hi, Les Mikesell wrote on 27.03.2007 at 01:03:32 [Re: [BackupPC-users] RSync v. Tar]: Jesse Proudman wrote: I've got one customer who's server has taken 3600 minutes to backup. 77 Gigs of Data. 1,972,859 small files. Would tar be better or make this faster? It's directly connected via 100 Mbit to ^^^ the backup box. ^^ If the files don't change frequently, tar incremental runs will be much faster because they are based only on the target timestamps while rsync will load the entire directory at both ends and compare them. if you ask me, regardless of how your data changes, tar is the way to go, not rsync, especially for *full* backups. With a direct 100 MBit connection, there's not much point in spending (lots of) CPU time for saving bandwidth - not with 2 million files. rsync is good for low bandwidth connections, where the link severely limits the transfer and speeding it up makes a real difference. In your case, your link speed is in the same order of magnitude as your disk I/O performance (considering our favorite topic, the seek times on the pool file system, the network link may in fact not even be the limiting factor - it clearly isn't, as 77 GB would take slightly more than 2 hours to transfer over a 100 MBit link, and rsync is not making it go faster than that ;-). rsync has additional benefits concerning finding and backing up new (or moved) files with old timestamps and deleted files on *incremental* backups, but keeping the list
[BackupPC-users] RSync v. Tar
I've got one customer who's server has taken 3600 minutes to backup. 77 Gigs of Data. 1,972,859 small files. Would tar be better or make this faster? It's directly connected via 100 Mbit to the backup box. -- Jesse Proudman, Blue Box Group, LLC - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT business topics through brief surveys-and earn cash http://www.techsay.com/default.php?page=join.phpp=sourceforgeCID=DEVDEV ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/backuppc-users http://backuppc.sourceforge.net/
[BackupPC-users] rsync vs tar
I've been using tar to back up my linux clients as I had difficulty getting rsync to work properly (mostly was deciphering now the rsync.conf file should be configured on remote clients) but I'm wondering if there are any disk space advantages to rsync backups in the data pool versus tar? My clients for the most part backup at night so I'm not too worried about how long backups take, but I also back up my primary file server with backuppc with tar and I'm pushing my RAID limit in disk space. So, to re-iterate: are there data pool space advantages to rsync over tar? Phillip M. Bryant Systems Administrator ITT Industries - Advanced Engineering and Sciences Albuquerque, New Mexico 87110 ph. 505-889-7016 fx. 505-889-7040 cell 505-385-8668 RHCT MCSE NT4, 2000 MCP+I This e-mail and any files transmitted with it are proprietary and intended solely for the use of the individual or entity to whom they are addressed. If you have received this e-mail in error please notify the sender. Please note that any views or opinions presented in this e-mail are solely those of the author and do not necessarily represent those of ITT Industries, Inc. The recipient should check this e-mail and any attachments for the presence of viruses. ITT Industries accepts no liability for any damage caused by any virus transmitted by this e-mail. --- This SF.net email is sponsored by: Splunk Inc. Do you grep through log files for problems? Stop! Download the new AJAX search engine that makes searching your log files as easy as surfing the web. DOWNLOAD SPLUNK! http://sel.as-us.falkag.net/sel?cmd=lnkkid3432bid#0486dat1642 ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/backuppc-users http://backuppc.sourceforge.net/
