brien dieterle wrote: > Jason Hughes wrote: >> Evren Yurtesen wrote: >> >>> Jason Hughes wrote: >>> >>>> That drive should be more than adequate. Mine is a 5400rpm 2mb >>>> buffer clunker. Works fine. >>>> Are you running anything else on the backup server, besides >>>> BackupPC? What OS? What filesystem? How many files total? >>>> >>> FreeBSD, UFS2+softupdates, noatime. >>> >>> There are 4 hosts that have been backed up, for a total of: >>> >>> * 16 full backups of total size 72.16GB (prior to pooling and >>> compression), >>> * 24 incr backups of total size 13.45GB (prior to pooling and >>> compression). >>> >>> # Pool is 17.08GB comprising 760528 files and 4369 directories (as of >>> 3/27 05:54), >>> # Pool hashing gives 38 repeated files with longest chain 6, >>> # Nightly cleanup removed 10725 files of size 0.40GB (around 3/27 05:54), >>> # Pool file system was recently at 10% (3/27 07:16), today's max is >>> 10% (3/27 01:00) and yesterday's max was 10%. >>> >>> Host User #Full Full Age (days) Full Size >>> (GB) Speed (MB/s) #Incr Incr Age (days) Last >>> Backup (days) State Last attempt >>> host1 4 5.4 3.88 0.22 6 0.4 >>> 0.4 idle idle >>> host2 4 5.4 2.10 0.06 6 0.4 >>> 0.4 idle idle >>> host3 4 5.4 7.57 0.14 6 0.4 >>> 0.4 idle idle >>> host4 4 5.4 5.56 0.10 6 0.4 >>> 0.4 idle idle >>> >>> >> Hmm. This is a tiny backup setup, even smaller than mine. However, it >> appears that the average size of your file is only 22KB, which is quite >> small. For comparison sake, this is from my own server: >> Pool is 172.91GB comprising 217311 files and 4369 directories (as of >> 3/26 01:08), >> >> The fact that you have tons of little files will probably give >> significantly higher overhead when doing file-oriented work, simply >> because the inodes must be fetched for each file before seeking to the >> file itself. If we assume no files are shared between hosts (very >> conservative), and you have an 8ms access time, you will have 190132 >> files per host and two seeks per file, neglecting actual i/o time, gives >> you 50 minutes. Just to seek them all. If you have a high degree of >> sharing, it can be up to 4x worse. Realize, the same number of seeks >> must be made on the server as well as the client. >> >> Are you sure you need to be backing up everything that you're putting >> across the network? Maybe excluding some useless directories, maybe >> temp files or logs that haven't been cleaned up? Perhaps you can >> archive big chunks of it with a cron job? >> >> I'd start looking for ways to cut down the number of files, because the >> overhead of per-file accesses are probably eating you alive. I'm also >> no expert on UFS2 or FreeBSD, so it may be worthwhile to research its >> behavior with hard links and small files. >> >> JH >> >> > For what it's worth, I have a server that backs up 8.6 million files > averaging 10k in size from one host. It takes a full 10 hours for a > full backup via tar over NFS ( 2.40MB/s for 87GB). CPU usage is low, > around 10-20%, however IOwait is a pretty steady 25%. > > Server info: > HP DL380 G4 > debian sarge > dual processor 3.2ghz xeon > 2GB Ram > 5x10k rpm scsi disks, raid5 > 128MB battery backed cache (50/50 r/w) > ext3 filesystems > > brien
You are distributing the reads and writes on 5 disks here. Dont you think that 2.40mb/s is a little bit slow compared to the horsepower you have? If you scale it down to my system, I think 1/10 performance is normal... Thanks, Evren ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys-and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV _______________________________________________ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/backuppc-users http://backuppc.sourceforge.net/
