In message <[EMAIL PROTECTED]>, Mike Tancsa writes:
>Using an amrc controller in RAID5, it doesnt make a difference really on
>the dd stuff - read or write-- perhaps 2-5MB difference on the faster
>side. Raw reads and writes whether I use da0 or da0s1d, the speed is
>pretty well the same as before... HOWEVER, one measurement is really odd on
>bonnie-- the random seeks .... and postmark shows a *big* difference as well.
This is very likely the alignment thing.
OK, some further tests, trying to control for this. I am not sure what values to actually fiddle with via bsdlable as this is the entire disk so I will just vary mounting da0 vs da0s1d. I also cvsup'd to current as of yesterday (FreeBSD nfs.sentex.ca 6.0-CURRENT FreeBSD 6.0-CURRENT #0: Mon May 2 15:03:53 EDT 2005). I will also try with Scott's changes to the driver which went in after.
Also, the newfs params does seem to make a very big difference and vfs.read_max as well.
[nfs]# fdisk da0 ******* Working on device /dev/da0 ******* parameters extracted from in-core disklabel are: cylinders=91182 heads=255 sectors/track=63 (16065 blks/cyl)
Figures below won't work with BIOS for partitions not in cyl 1 parameters to be used for BIOS calculations are: cylinders=91182 heads=255 sectors/track=63 (16065 blks/cyl)
Media sector size is 512
Warning: BIOS sector numbering starts with sector 1
Information from DOS bootblock is:
The data for partition 1 is:
sysid 165 (0xa5),(FreeBSD/NetBSD/386BSD)
start 63, size 1464838767 (715253 Meg), flag 80 (active)
beg: cyl 0/ head 1/ sector 1;
end: cyl 1023/ head 254/ sector 63
The data for partition 2 is:
<UNUSED>
The data for partition 3 is:
<UNUSED>
The data for partition 4 is:
<UNUSED>
[nfs]# bsdlabel da0s1
# /dev/da0s1:
8 partitions:
# size offset fstype [fsize bsize bps/cpg]
c: 1464838767 0 unused 0 0 # "raw" part, don't edit
d: 1464838767 0 4.2BSD 2048 16384 28552
[nfs]# newfs -b 32768 -f 4096 /dev/da0
[nfs]# bsdlabel da0 bsdlabel: /dev/da0: no valid label found [nfs]# even though I can mount it just fine
[nfs]# mount /dev/da0 /mnt [nfs]# df Filesystem 1K-blocks Used Avail Capacity Mounted on /dev/ad0s1a 1012974 356028 575910 38% / devfs 1 1 0 100% /dev /dev/ad0s1e 1012974 580870 351068 62% /tmp /dev/ad0s1d 15231278 5848016 8164760 42% /usr /dev/ad0s1f 19564270 192176 17806954 1% /var /dev/da0 720910352 8 663237516 0% /mnt [nfs]#
-------Sequential Output-------- ---Sequential Input-- --Random--
-Per Char- --Block--- -Rewrite-- -Per Char- --Block--- --Seeks---
Machine MB K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU /sec %CPU
da0s1 4000 61175 53.3 61832 16.3 31829 8.7 63008 60.4 79551 11.2 224.2 0.7
da0s1 v 4000 61278 54.3 61938 16.4 31828 8.8 73566 71.7 81778 12.6 216.5 0.7
da0 4000 62625 54.5 62154 15.7 31907 8.3 63297 60.8 80139 11.0 226.2 0.7
da0 v 4000 62315 54.3 61759 15.4 32094 8.3 74209 72.3 84470 12.6 223.5 0.7
da0 N 4000 75239 62.7 75771 15.5 33596 8.4 81909 78.6 98453 13.3 214.8 0.8
da0 vN 4000 72786 61.7 74939 15.3 33672 8.4 82944 79.8 108106 14.9 214.7 0.8
v = sysctl -w vfs.read_max=16
N = newfs -b 32768 -f 4096 /dev/da0
[nfs]# postmark
PostMark v1.5 : 3/27/01
pm>set location /mnt
pm>set size 300 100000
pm>set transactions 400000
pm>run
Creating files...Done
Performing transactions..........Done
Deleting files...Done
Time:
1418 seconds total
1418 seconds of transactions (282 per second)Files:
200107 created (141 per second)
Creation alone: 500 files (500 per second)
Mixed with transactions: 199607 files (140 per second)
199905 read (140 per second)
199384 appended (140 per second)
200107 deleted (141 per second)
Deletion alone: 889 files (889 per second)
Mixed with transactions: 199218 files (140 per second)Data:
12715.55 megabytes read (8.97 megabytes per second)
12728.92 megabytes written (8.98 megabytes per second)
pm>quit
[nfs]#create a junk file [nfs]# time dd if=/dev/urandom of=/mnt/big-4g-rand bs=1024k count=4000 4000+0 records in 4000+0 records out 4194304000 bytes transferred in 124.694915 secs (33636528 bytes/sec) 0.022u 116.020s 2:04.70 93.0% 21+2910k 124+32000io 0pf+0w
wait 2 min for disks to totally flush
[nfs]# time dd if=/dev/zero of=/mnt/big-4g-zero bs=1024k count=4000 4000+0 records in 4000+0 records out 4194304000 bytes transferred in 66.463634 secs (63106751 bytes/sec) 0.000u 11.265s 1:06.50 16.9% 21+2926k 126+32000io 0pf+0w
sysctl -w vfs.read_max=16
(clear out cache by reading via md5 4G of other files) and then wait a bit
[nfs]# time dd if=/mnt/big-4g-rand of=/dev/null bs=1024k count=4000 4000+0 records in 4000+0 records out 4194304000 bytes transferred in 59.178470 secs (70875506 bytes/sec) 0.000u 7.677s 0:59.18 12.9% 20+2784k 32128+0io 0pf+0w
[nfs]# postmark PostMark v1.5 : 3/27/01 pm>set locatime /mnt Eh? pm>set location /mnt pm>set size 300 100000 pm>set transactions 400000 pm>run Creating files...Done Performing transactions..........Done Deleting files...Done Time: 1369 seconds total 1369 seconds of transactions (292 per second)
Files:
200107 created (146 per second)
Creation alone: 500 files (500 per second)
Mixed with transactions: 199607 files (145 per second)
199905 read (146 per second)
199384 appended (145 per second)
200107 deleted (146 per second)
Deletion alone: 889 files (889 per second)
Mixed with transactions: 199218 files (145 per second)Data:
12715.55 megabytes read (9.29 megabytes per second)
12728.92 megabytes written (9.30 megabytes per second)
pm>
With vfs.read_max=8
[nfs]# postmark PostMark v1.5 : 3/27/01 pm>set location /mnt pm>set size 300 100000 pm>set transactions 400000 pm>run Creating files...Done Performing transactions..........Done Deleting files...Done Time: 1013 seconds total 1013 seconds of transactions (394 per second)
Files:
200107 created (197 per second)
Creation alone: 500 files (500 per second)
Mixed with transactions: 199607 files (197 per second)
199905 read (197 per second)
199384 appended (196 per second)
200107 deleted (197 per second)
Deletion alone: 889 files (889 per second)
Mixed with transactions: 199218 files (196 per second)Data:
12715.55 megabytes read (12.55 megabytes per second)
12728.92 megabytes written (12.57 megabytes per second)
pm>
pm>quit
[nfs]#--------------------------------------------------------------------- newfs -b 32768 -f 4096 /dev/da0
[nfs]# postmark
PostMark v1.5 : 3/27/01
pm>set location /mnt
pm>set transactions 400000
pm>set size 300 100000
pm>run
Creating files...Done
Performing transactions..........Done
Deleting files...Done
Time:
542 seconds total
541 seconds of transactions (739 per second)Files:
200107 created (369 per second)
Creation alone: 500 files (500 per second)
Mixed with transactions: 199607 files (368 per second)
199905 read (369 per second)
199384 appended (368 per second)
200107 deleted (369 per second)
Deletion alone: 889 files (889 per second)
Mixed with transactions: 199218 files (368 per second)Data:
12715.55 megabytes read (23.46 megabytes per second)
12728.92 megabytes written (23.49 megabytes per second)
pm>
[nfs]# sysctl -w vfs.read_max=16
vfs.read_max: 8 -> 16pm>set location /mnt
pm>set transactions 400000
pm>set size 300 100000
pm>run
Creating files...Done
Performing transactions..........Done
Deleting files...Done
Time:
542 seconds total
541 seconds of transactions (739 per second)Files:
200107 created (369 per second)
Creation alone: 500 files (500 per second)
Mixed with transactions: 199607 files (368 per second)
199905 read (369 per second)
199384 appended (368 per second)
200107 deleted (369 per second)
Deletion alone: 889 files (889 per second)
Mixed with transactions: 199218 files (368 per second)Data:
12715.55 megabytes read (23.46 megabytes per second)
12728.92 megabytes written (23.49 megabytes per second)
pm>
[nfs]# time dd if=/mnt/big-2g-rand count=2000 bs=1024k of=/dev/null 2000+0 records in 2000+0 records out 2097152000 bytes transferred in 19.298799 secs (108667489 bytes/sec) 0.000u 3.603s 0:19.32 18.6% 21+2965k 16019+0io 0pf+0w [nfs]# time dd if=/mnt/big-2g-rand2 count=2000 bs=1024k of=/dev/null 2000+0 records in 2000+0 records out 2097152000 bytes transferred in 20.188331 secs (103879414 bytes/sec) 0.015u 3.515s 0:20.24 17.3% 20+2865k 16020+0io 3pf+0w [nfs]# sysctl -w vfs.read_max=8
vfs.read_max: 16 -> 8
[nfs]# time dd if=/mnt/big-2g-rand count=2000 bs=1024k of=/dev/null
2000+0 records in
2000+0 records out
2097152000 bytes transferred in 21.421867 secs (97897722 bytes/sec)
0.007u 3.482s 0:21.43 16.2% 21+2873k 16018+0io 0pf+0w
[nfs]# time dd if=/mnt/big-2g-rand2 count=2000 bs=1024k of=/dev/null
2000+0 records in
2000+0 records out
2097152000 bytes transferred in 23.306603 secs (89981024 bytes/sec)
0.000u 3.590s 0:23.37 15.3% 19+2687k 16020+0io 3pf+0w
[nfs]#
_______________________________________________ [email protected] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-performance To unsubscribe, send any mail to "[EMAIL PROTECTED]"
