David Rees wrote:
> On 3/27/07, Les Mikesell <[EMAIL PROTECTED]> wrote:
>> Evren Yurtesen wrote:
>>
>>>> What is wall clock time for a run and is it
>>>> reasonable for having to read through both the client and server copies?
>>> I am using rsync but the problem is that it still has to go through a
>>> lot of hard links to figure out if files should be backed up or not.
>
> Evren, I didn't see that you mentioned a wall clock time for your
> backups? I want to know how many files are in a single backup, how
> much data is in that backup and how long it takes to perform that
> backup.
I sent the status of the backups earlier today to mailing list?
# Pool is 17.08GB comprising 760528 files and 4369 directories (as of
3/27 05:54),
# Pool hashing gives 38 repeated files with longest chain 6,
# Nightly cleanup removed 10725 files of size 0.40GB (around 3/27 05:54),
# Pool file system was recently at 10% (3/27 22:40), today's max is 10%
(3/27 01:00) and yesterday's max was 10%.
* 16 full backups of total size 72.16GB (prior to pooling and
compression),
* 24 incr backups of total size 13.60GB (prior to pooling and
compression).
This is from 1 machine.
Totals Existing Files New Files
Backup# Type #Files Size/MB MB/sec #Files Size/MB
#Files Size/MB
112 full 149290 1957.8 0.10 149251 1937.9 4601 20.7
224 full 151701 2022.8 0.09 151655 2004.7 100 18.1
238 full 152170 2099.5 0.06 152131 2081.6 115 17.9
244 incr 214 48.9 0.00 165 22.9 78 26.0
245 full 152228 2095.2 0.06 152177 2076.9 108 18.3
246 incr 118 17.3 0.00 76 0.2 69 17.1
247 incr 159 21.4 0.00 111 3.1 75 18.4
248 incr 181 22.1 0.00 132 2.5 79 19.7
249 incr 186 24.0 0.00 146 7.6 54 16.4
250 incr 206 25.5 0.00 159 6.7 70 18.8
>> From the perspective of the backup directories, it doesn't matter
>> whether or not there are additional links to a file. It just looks at
>> the directory entry to find the file it wants. It may matter that the
>> inodes and file contents end up splattered all over the place because
>> they were written at different times, though.
>
> Yep, Lee is right here. Unless BSD handles hard-links in some crazy manner.
I dont know if the problem is hard links. This is not a FreeBSD or Linux
problem. It exists on both. Just that people using ultra fast 5 disk
raid 5 setups are seeing 2mbytes/sec transfer rate means that backuppc
is very very inefficient.
For example this guy is using Linux (problem is OS independent)
http://forum.psoft.net/showpost.php?p=107808&postcount=16
>>> If you check namesys.com benchmarks, you will see that they only tested
>>> reiserfs against ext2/ext3/xfs/jfs and conveniently forgot to test
>>> against ufs2.
>>>
>>> You can see in the end of the page that slight performance increase in
>>> reiserfs is also bringing twice the cpu usage! (plus extX is faster in
>>> certain operations even)
>>> http://www.namesys.com/benchmarks.html
>
> When your overall speed is limited by the speed of your disks and your
> CPU spends all it's time twiddling it's thumbs waiting for disk, who
> cares if CPU doubles and still spends 90% of it's time waiting as long
> as it gets the job done faster?
Whatever, this is not the problem here.The fact is that, according to
reiserfs developers reiserfs is more or less the same speed with ext2. I
dont think the problem is related to any filesystem as it occurs on both
Linux and FreeBSD
> To summarize Evran's setup:
>
> FreeBSD, 250MB ram, CPU unknown, 1 7200rpm Seagate ATA disk
> Filesystem: ufs2, sync, noatime
> Pool is 17.08GB comprising 760528 files (avg file size ~22KB)
> BackupPC reports backup speeds between 0.06 -> 0.22 MB/s
> Total backup time per host: Unknown
> CPU is 99% idle during backups
> Disk shows ~75 IO/s during load and low transfer rate
> Says even small backup w/small number of files is slow.
>
> Can you try mounting the backup partition async so we can see if it
> really is read performance or write performance that is killing backup
> performance?
>
> I would also highly recommend that you limit backups to 1 concurrent
> backup at a time.
>
> I must wonder if ufs2 is really bad at storing inodes on disk...
On Linux with raid setup with async io etc. people are getting slightly
better results. I think ufs2 is just fine. I wonder if there is
something in my explanations...The problem is backuppc. People are
getting ~2mbytes/sec(was it 2 or 5?) speed with raid5 and 5 drives,
using Linux. It is a miracle that backup even finishes in 24 hours using
a standart ide drive.
> BTW, how does BackupPC calculate speed? I think it calculates backup
> speed by reporting files transferred over time, so if you don't have
> many files that change, won't BackupPC report a very low backup speed.
This is like the 'Contact' movie. The sphere took 30 seconds to download
but there were 18 hours of recording. If what you said was true and
backuppc would be backing up very small amount of files and skipping
most, then backups would probably take less time than 2-4 hours each.
Thanks,
Evren
-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/backuppc-users
http://backuppc.sourceforge.net/
