Re: [BackupPC-users] Filesystem benchmarks

Brien Dieterle Wed, 28 Mar 2007 09:01:03 -0800

Here are some benchmarks I ran last week: I think it's important to balance the -s<size> with the -n numbers so that you are dealing with the same amount of data, otherwise caching can bite you and you can have misleading results. Therefore, I used 10k file-size, and adjusted the number of files to give approximately the same amount of megabytes that was specified with -s. IE; 200MB of block IO and 200MB of small-file io. I only did raid5 vs single disk comparisons on 200, 800, and 3200mb data points, for time reasons :-) I think these results show two things going on. 1) the steady decline of ext3 performance as # files increase. 2) that raid5 and multiple disks really doesn't significantly increase read performance for small files. 3) once you're outside the caching zones things go badly: get more ram :-) Then again, my hardware might just be shoddy :-) Machine info: DL380-G4 dual cpu 3.20GHz Debian Sarge 2.6.8-3-686-smp Smart Array 6i controller with 128MB BBC, 50/50r/w Machine Ram limited to 256MB with kernel param mem=256M Ext3 filesystems, defaults logical drives created with defaults: Raid-5 array: Smart Array 6i in Slot 0    logicaldrive 1    Size: 1 TB    Fault Tolerance: RAID 5    Heads: 255    Sectors per Track: 32    Cylinders: 65535    Stripe Size: 64 KB    Array Accelerator: Enabled Single Disk: Smart Array 6i in Slot 0    logicaldrive 2    Size: 279 GB    Fault Tolerance: RAID 0    Heads: 255    Sectors per Track: 32    Cylinders: 65535    Stripe Size: 128 KB    Status: Ok    Array Accelerator: Enabled 200MB:Single DISK
bonnie -u root -s200 -n20:10000:10000:10 -r0 -f Version 1.03       ------Sequential Output------ --Sequential Input- --Random-                     -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP mirror         200M           144896 45 15404   4           159673   9 8163   6                     ------Sequential Create------ --------Random Create--------                     -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- files:max:min        /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP 20:10000:10000/10 4523 41 1976   6 +++++ +++ 6038 55   408   1 25162 99 mirror,200M,,,144896,45,15404,4,,,159673,9,8163.2,6,20:10000:10000/10,4523,41,1976,6,+++++,+++,6038,55,408,1,25162,99 5 Disk Raid5 Version 1.03       ------Sequential Output------ --Sequential Input- --Random-                     -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP mirror         200M           222201 72 35381 10           110627 12 10372 11                     ------Sequential Create------ --------Random Create--------                     -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- files:max:min        /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP 20:10000:10000/10 6753 63 2855   9 +++++ +++ 10416 96   421   1 25088 99 mirror,200M,,,222201,72,35381,10,,,110627,12,10372.3,11,20:10000:10000/10,6753,63,2855,9,+++++,+++,10416,96,421,1,25088,99 300MBSingle DISK
bonnie -u root -s300 -n30:10000:10000:10 -r0 -f Version 1.03       ------Sequential Output------ --Sequential Input- --Random-                     -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP mirror         300M           69272 23 19904   6           45771   7 1439   2                     ------Sequential Create------ --------Random Create--------                     -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- files:max:min        /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP 30:10000:10000/10 3972 43 2067   6 45890 93 2960 32   196   1 21817 99 mirror,300M,,,69272,23,19904,6,,,45771,7,1439.0,2,30:10000:10000/10,3972,43,2067,6,45890,93,2960,32,196,1,21817,99 400MBSingle DISK
bonnie -u root -s400 -n40:10000:10000:10 -r0 -f Version 1.03       ------Sequential Output------ --Sequential Input- --Random-                     -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP mirror         400M           57034 19 20791   6           52027   8 669.3   1                     ------Sequential Create------ --------Random Create--------                     -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- files:max:min        /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP 40:10000:10000/10 2844 37 2142   6 41534 82 2965 39   171   1 18650 98 mirror,400M,,,57034,19,20791,6,,,52027,8,669.3,1,40:10000:10000/10,2844,37,2142,6,41534,82,2965,39,171,1,18650,98 800MBSingle DISK:
bonnie -u root -s800 -n80:10000:10000:10 -r0 -f Version 1.03       ------Sequential Output------ --Sequential Input- --Random-                     -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP mirror         800M           52857 19 23641   7           58240   8 419.0   0                     ------Sequential Create------ --------Random Create--------                     -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- files:max:min        /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP 80:10000:10000/10 3310 70 2858   9 35289 66 2309 49   170   1 8127 68 mirror,800M,,,52857,19,23641,7,,,58240,8,419.0,0,80:10000:10000/10,3310,70,2858,9,35289,66,2309,49,170,1,8127,685 Disk Raid5:
Version 1.03       ------Sequential Output------ --Sequential Input- --Random-                     -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP mirror         800M           107481 39 39768 12           112645 17 545.5 1                     ------Sequential Create------ --------Random Create--------                     -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- files:max:min        /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP 80:10000:10000/10 3874 83 2607   8 34618 67 4518 94   190   2 7702 64 mirror,800M,,,107481,39,39768,12,,,112645,17,545.5,1,80:10000:10000/10,3874,83,2607,8,34618,67,4518,94,190,2,7702,64 1600MBSingle DISK
bonnie -u root -s1600 -n160:10000:10000:10 -r0 -f Version 1.03       ------Sequential Output------ --Sequential Input- --Random-                     -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP mirror        1600M           51865 19 24401   7           63295   9 354.0   0                     ------Sequential Create------ --------Random Create--------                     -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- files:max:min        /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP 160:10000:10000/10 1735 64 2393   8 30133 58 1817 66   157   2 3466 51 mirror,1600M,,,51865,19,24401,7,,,63295,9,354.0,0,160:10000:10000/10,1735,64,2393,8,30133,58,1817,66,157,2,3466,51 3200MBSingle DISK
bonnie -u root -s3200 -n320:10000:10000:10 -r0 -f Version 1.03       ------Sequential Output------ --Sequential Input- --Random-                     -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP mirror        3200M           49334 19 23487   7           72178 11 282.3   0                     ------Sequential Create------ --------Random Create--------                     -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- files:max:min        /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP 320:10000:10000/10 1362 95 2874   9 30390 56 1397 97   160   5   909 24 mirror,3200M,,,49334,19,23487,7,,,72178,11,282.3,0,320:10000:10000/10,1362,95,2874,9,30390,56,1397,97,160,5,909,24

5 Disk Raid5:
Version 1.03       ------Sequential Output------ --Sequential Input- --Random-                     -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP mirror        3200M           74584 29 38126 12           128050 20 295.2   0                    ------Sequential Create------ --------Random Create--------                     -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- files:max:min        /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP 320:10000:10000/10 1374 96 2742   8 30962 58 1397 97   183   5 2240 60 mirror,3200M,,,74584,29,38126,12,,,128050,20,295.2,0,320:10000:10000/10,1374,96,2742,8,30962,58,1397,97,183,5,2240,60

Brien

John Pettitt wrote:

Following the extended discussion of system benchmarks here are some actual numbers from a FreeBSD box - if anybody has the time to run similar numbers on linux boxes I will happily collate the data.

John

2.93 GHz Celeron D, 768 MB ram FreeBSD 6.2 bonnie++ -f 0 -d . -s 3072 -n 10:100000:10:10 Key: IDE = 80 GB IDE, soft upodates, atime on IDE-R1-atime =300gb raid 1 (mirror) IDE, atime on, soft updates
IDE-R1 = 300gb raid 1 (mirror) IDE, no atime, soft updates IDE-R1-sync = 300gb raid 1 (mirror) IDE, no atime, sync IDE-R1-async = 300gb raid 1 (mirror) IDE, no atime, async SATA-R10 = 1.5TB raid 10 sata on 3ware 9500S-12. no atime, soft updates v4 = raiserfs v4 from namesys on a 2.4ghz xeon ext3 = ext3 from namesys on 2.4ghz xeon Version 1.93c ------Sequential Output------ --Sequential Input- --Random- Concurrency 1 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP IDE 3G 40181 22 12106 6 36944 12 99.4 7 IDE-R1 3G 34511 20 14857 8 54482 19 121.7 9 IDE-R1-atime 3G 34426 21 14832 8 54402 18 122.1 9 IDE-R1-sync 3G 4904 8 4248 4 53750 18 103.7 8 IDE-R1-async 3G 34405 20 14877 8 53579 18 122.6 9 SATA-R10 3G 85375 53 25188 14 49751 17 454.1 33 v4 3G 37579 19 15657 11 41531 11 105.8 0 ext3 3G 35221 22 10987 4 41105 6 90.9 0 Version 1.93c ------Sequential Create------ --------Random Create-------- -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- files:max:min /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP IDE 236 8 5071 73 9954 35 229 7 4354 68 +++++ +++ IDE-R1 460 19 5524 76 13606 86 301 10 4576 65 +++++ +++ IDE-R1-atime 377 15 4368 63 11580 70 395 13 5061 78 11642 90 IDE-R1-sync 107 12 6027 86 +++++ +++ 112 12 5466 83 19644 82 IDE-R1-async 370 15 4609 66 18905 49 376 12 5583 84 11427 90 SATA-R10 973 41 7365 98 13281 84 1079 38 5877 79 12875 85 v4 570 39 746 17 1435 23 513 40 104 2 951 15 ext3 221 8 364 4 853 4 204 7 99 1 306 2 Notes: The 3ware card is somewhat bus limited because it's in a 32bit PCI slot in a 64 bit slot I'd expect better sequential read performance. This also drives up the CPU number due to bus contention. The read numbers for ext3 and raiserfs look suspect Stripe size for the 3ware raid 10 is 256k all file sytems were live and had other files on them - virgin file systems may perform very differently.. Conclusion: ufs2 is pretty similar to ext2 if not a little better but not as fast as raiser4. sync is a big drag but async makes almost no difference over soft updates. atime/noatime doesn't make a whole lot of difference on this test. ~
-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/backuppc-users
http://backuppc.sourceforge.net/
  

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV

_______________________________________________
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/backuppc-users
http://backuppc.sourceforge.net/

Re: [BackupPC-users] Filesystem benchmarks

Reply via email to