Re: ciss(4) write very slow w/o bbwc
Hi! I tested the performance w and w/o the patch. There is no difference. ciss0 at pci3 dev 3 function 0 Compaq Smart Array 64xx rev 0x01: apic 10 int 3 ciss0: 2 LDs, HW rev 1, FW 2.84/2.84, 64bit fifo scsibus0 at ciss0: 2 targets sd0 at scsibus0 targ 0 lun 0: HP, LOGICAL VOLUME, 2.84 SCSI2 0/direct fixed sd0: 69459MB, 512 bytes/sector, 142253280 sectors sd1 at scsibus0 targ 1 lun 0: HP, LOGICAL VOLUME, 2.84 SCSI2 0/direct fixed sd1: 140006MB, 512 bytes/sector, 286734240 sectors kern.bufcachepercent=20 w/o patch raw device # dd if=/dev/zero of=/dev/rsd1d bs=1m count=1000 1000+0 records in 1000+0 records out 1048576000 bytes transferred in 75.550 secs (13879230 bytes/sec) file # dd if=/dev/zero of=test bs=1m count=1000 1000+0 records in 1000+0 records out 1048576000 bytes transferred in 16.986 secs (61728607 bytes/sec) w patch raw device # dd if=/dev/zero of=/dev/rsd1d bs=1m count=1000 1000+0 records in 1000+0 records out 1048576000 bytes transferred in 75.609 secs (13868396 bytes/sec) file # dd if=/dev/zero of=test bs=1m count=1000 1000+0 records in 1000+0 records out 1048576000 bytes transferred in 16.165 secs (64863961 bytes/sec) In fact, the file test performance is acceptable for me. The raw performace (eg. newfs) is not so important. Thx csszep 2012/5/29 Andreas Bartelt o...@bartula.de: Hello, On 05/29/12 17:28, Kenneth R Westerback wrote: On Tue, May 29, 2012 at 03:48:02PM +0200, csszep wrote: Hi! So i tested the ciss performance with Openbsd 5.1 and Netbsd 5.1.2 and the numbers are the same. :( approx 13Mbyte/s write with dd if=/dev/zero of=/dev/rsd1c bs=1m count=500 But why Linux is four times faster (approx 40Mbyte/s)? Dunno. But the diff below should apply the NetBSD 'fix' for the INQUIRY command. Ken Dunno. But the diff below should apply the NetBSD 'fix' for the INQUIRY command. I also can confirm relatively slow ciss(4) performance on OpenBSD. Enabling the (not battery backed) cache via BIOS doesn't help significantly. I just did some tests on a HP Proliant DL360G7 with RAID1 via ciss(4) with 2x300GB 6G SAS 1 rpm HDDs (cache disabled on this box): # disklabel sd0 # /dev/rsd0c: type: SCSI disk: SCSI disk label: LOGICAL VOLUME duid: 410f0efc5a9d86dd flags: bytes/sector: 512 sectors/track: 63 tracks/cylinder: 255 sectors/cylinder: 16065 cylinders: 36468 total sectors: 585871964 boundstart: 64 boundend: 585858420 drivedata: 0 16 partitions: #size offset fstype [fsize bsize cpg] a: 1028096 64 4.2BSD 2048 16384 1 # / c:5858719640 unused d: 1028160 1028160 4.2BSD 2048 16384 1 # /var e:146801952 2056320 4.2BSD 2048 16384 1 # /usr f: 20964832148858272 4.2BSD 2048 16384 1 # /home g:416035264169823104 4.2BSD 4096 32768 1 # /log # mount /dev/sd0a on / type ffs (local, noatime, softdep) /dev/sd0f on /home type ffs (local, noatime, nodev, nosuid, softdep) /dev/sd0g on /log type ffs (local, noatime, nodev, nosuid, softdep) /dev/sd0e on /usr type ffs (local, noatime, nodev, softdep) /dev/sd0d on /var type ffs (local, noatime, nodev, nosuid, softdep) # dmesg|grep ciss ciss0 at pci1 dev 0 function 0 Hewlett-Packard Smart Array rev 0x01: apic 0 int 4 ciss0: 2 LDs, HW rev 2, FW 3.66/3.66, 64bit fifo rro scsibus0 at ciss0: 2 targets before applying your patch: [/usr] # dd if=/dev/zero of=testfile bs=1m count=1000 1000+0 records in 1000+0 records out 1048576000 bytes transferred in 16.428 secs (63825353 bytes/sec) [/usr] # dd if=/dev/zero of=testfile bs=1m count=1 1+0 records in 1+0 records out 1048576 bytes transferred in 153.910 secs (68128911 bytes/sec) [/log] # dd if=/dev/zero of=testfile bs=1m count=1000 1000+0 records in 1000+0 records out 1048576000 bytes transferred in 8.122 secs (129087680 bytes/sec) [/log] # dd if=/dev/zero of=testfile bs=1m count=1 1+0 records in 1+0 records out 1048576 bytes transferred in 87.701 secs (119561580 bytes/sec) after applying your patch: [/usr] # dd if=/dev/zero of=testfile bs=1m count=1000 1000+0 records in 1000+0 records out 1048576000 bytes transferred in 14.113 secs (74296489 bytes/sec) [/usr] # dd if=/dev/zero of=testfile bs=1m count=1 1+0 records in 1+0 records out 1048576 bytes transferred in 154.600 secs (67824996 bytes/sec) [/log] # dd if=/dev/zero of=testfile bs=1m count=1000 1000+0 records in 1000+0 records out 1048576000 bytes transferred in 6.836 secs (153379539 bytes/sec) [/log] # dd if=/dev/zero of=testfile bs=1m count=1 1+0 records in 1+0 records out 1048576 bytes transferred in 82.955 secs (126402027 bytes/sec) The larger fsize/bsize of partition sd0g almost seems to double the writing throughput in comparison to partition sd0e. I didn't expect this much of a difference. Regarding
ciss(4) write very slow w/o bbwc
Hi Misc! We have some older HP Dl360, Dl380 G4 machines with Smart Array 6i controllores w/o battery backed cache. The disk performance in this case is really poor, for examle the disklabel operation on a 72GB disk lasted for about 5 mins. I found a commit in a NetBSD ciss driver (which is a port of OpenBSD driver), that solve the problem i think: http://cvsweb.netbsd.org/bsdweb.cgi/src/sys/dev/ic/ciss.c?rev=1.23content-type=text/x-cvsweb-markuponly_with_tag=MAIN The problem is, that the NetBSD scsi midlayer is very different from the OpenBSD, so is it possible to integrate this patch? My C and OpenBSD internals knowledge is not enough for this task. PS: Yes i know, my english is terrible. thx csszep
Re: ciss(4) write very slow w/o bbwc
I don't think that commit will fix the problem. HP shouldn't sell machines without the battery, but they do. From memory the firmware on the raid controller has no way of turning on caching without the battery being present. On Tue, May 29, 2012 at 11:30:34AM +0200, csszep wrote: Hi Misc! We have some older HP Dl360, Dl380 G4 machines with Smart Array 6i controllores w/o battery backed cache. The disk performance in this case is really poor, for examle the disklabel operation on a 72GB disk lasted for about 5 mins. I found a commit in a NetBSD ciss driver (which is a port of OpenBSD driver), that solve the problem i think: http://cvsweb.netbsd.org/bsdweb.cgi/src/sys/dev/ic/ciss.c?rev=1.23content-type=text/x-cvsweb-markuponly_with_tag=MAIN The problem is, that the NetBSD scsi midlayer is very different from the OpenBSD, so is it possible to integrate this patch? My C and OpenBSD internals knowledge is not enough for this task. PS: Yes i know, my english is terrible. thx csszep
Re: ciss(4) write very slow w/o bbwc
On Tue, May 29, 2012 at 08:31:06PM +1000, Jonathan Gray wrote: From memory the firmware on the raid controller has no way of turning on caching without the battery being present. I run some ciss, like the HP P212 and P410 and I can override the cache setting, that is I can turn it on even without a battery installed. It gives me a warning though :-P cheers, rodolfo
Re: ciss(4) write very slow w/o bbwc
Ok, but i installed Linux (Debian 6) and there is no performance degradation. I will install NetBSD too, and i will do a test. The commit does not turn on the cache, it enable tagged queing if i understand it well. thx csszep 2012/5/29 Jonathan Gray j...@jsg.id.au: I don't think that commit will fix the problem. HP shouldn't sell machines without the battery, but they do. From memory the firmware on the raid controller has no way of turning on caching without the battery being present. On Tue, May 29, 2012 at 11:30:34AM +0200, csszep wrote: Hi Misc! We have some older HP Dl360, Dl380 G4 machines with Smart Array 6i controllores w/o battery backed cache. The disk performance in this case is really poor, for examle the disklabel operation on a 72GB disk lasted for about 5 mins. I found a commit in a NetBSD ciss driver (which is a port of OpenBSD driver), that solve the problem i think: http://cvsweb.netbsd.org/bsdweb.cgi/src/sys/dev/ic/ciss.c?rev=1.23content-type=text/x-cvsweb-markuponly_with_tag=MAIN The problem is, that the NetBSD scsi midlayer is very different from the OpenBSD, so is it possible to integrate this patch? My C and OpenBSD internals knowledge is not enough for this task. PS: Yes i know, my english is terrible. thx csszep
Re: ciss(4) write very slow w/o bbwc
On Tue, May 29, 2012 at 12:25:51PM +0100, Rodolfo Gouveia wrote: On Tue, May 29, 2012 at 08:31:06PM +1000, Jonathan Gray wrote: From memory the firmware on the raid controller has no way of turning on caching without the battery being present. I run some ciss, like the HP P212 and P410 and I can override the cache setting, that is I can turn it on even without a battery installed. It gives me a warning though :-P Perhaps it is the older ones I am thinking of then, probably something like the 64xx.
Re: ciss(4) write very slow w/o bbwc
Hi! So i tested the ciss performance with Openbsd 5.1 and Netbsd 5.1.2 and the numbers are the same. :( approx 13Mbyte/s write with dd if=/dev/zero of=/dev/rsd1c bs=1m count=500 But why Linux is four times faster (approx 40Mbyte/s)? thx csszep 2012/5/29 csszep css...@gmail.com: Ok, but i installed Linux (Debian 6) and there is no performance degradation. I will install NetBSD too, and i will do a test. The commit does not turn on the cache, it enable tagged queing if i understand it well. thx csszep 2012/5/29 Jonathan Gray j...@jsg.id.au: I don't think that commit will fix the problem. HP shouldn't sell machines without the battery, but they do. From memory the firmware on the raid controller has no way of turning on caching without the battery being present. On Tue, May 29, 2012 at 11:30:34AM +0200, csszep wrote: Hi Misc! We have some older HP Dl360, Dl380 G4 machines with Smart Array 6i controllores w/o battery backed cache. The disk performance in this case is really poor, for examle the disklabel operation on a 72GB disk lasted for about 5 mins. I found a commit in a NetBSD ciss driver (which is a port of OpenBSD driver), that solve the problem i think: http://cvsweb.netbsd.org/bsdweb.cgi/src/sys/dev/ic/ciss.c?rev=1.23content-type=text/x-cvsweb-markuponly_with_tag=MAIN The problem is, that the NetBSD scsi midlayer is very different from the OpenBSD, so is it possible to integrate this patch? My C and OpenBSD internals knowledge is not enough for this task. PS: Yes i know, my english is terrible. thx csszep
Re: ciss(4) write very slow w/o bbwc
On Tue, May 29, 2012 at 03:48:02PM +0200, csszep wrote:
Hi!
So i tested the ciss performance with Openbsd 5.1 and Netbsd 5.1.2 and
the numbers are the same. :(
approx 13Mbyte/s write with dd if=/dev/zero of=/dev/rsd1c bs=1m count=500
But why Linux is four times faster (approx 40Mbyte/s)?
Dunno. But the diff below should apply the NetBSD 'fix' for the INQUIRY
command.
Ken
Dunno. But the diff below should apply the NetBSD 'fix' for the INQUIRY
command.
Ken
Index: ciss.c
===
RCS file: /cvs/src/sys/dev/ic/ciss.c,v
retrieving revision 1.67
diff -u -p -r1.67 ciss.c
--- ciss.c 10 Nov 2011 12:05:11 - 1.67
+++ ciss.c 29 May 2012 15:18:30 -
@@ -642,6 +642,14 @@ ciss_done(struct ciss_ccb *ccb)
if (xs) {
xs-resid = 0;
+ if (xs-cmd-opcode == INQUIRY) {
+ struct scsi_inquiry_data *inq;
+ inq = (struct scsi_inquiry_data *)xs-data;
+ if ((inq-version SID_ANSII) == 0
+(inq-flags SID_CmdQue) != 0) {
+ inq-version |= 2;
+}
+ }
scsi_done(xs);
}
Re: ciss(4) write very slow w/o bbwc
Hello, On 05/29/12 17:28, Kenneth R Westerback wrote: On Tue, May 29, 2012 at 03:48:02PM +0200, csszep wrote: Hi! So i tested the ciss performance with Openbsd 5.1 and Netbsd 5.1.2 and the numbers are the same. :( approx 13Mbyte/s write with dd if=/dev/zero of=/dev/rsd1c bs=1m count=500 But why Linux is four times faster (approx 40Mbyte/s)? Dunno. But the diff below should apply the NetBSD 'fix' for the INQUIRY command. Ken Dunno. But the diff below should apply the NetBSD 'fix' for the INQUIRY command. I also can confirm relatively slow ciss(4) performance on OpenBSD. Enabling the (not battery backed) cache via BIOS doesn't help significantly. I just did some tests on a HP Proliant DL360G7 with RAID1 via ciss(4) with 2x300GB 6G SAS 1 rpm HDDs (cache disabled on this box): # disklabel sd0 # /dev/rsd0c: type: SCSI disk: SCSI disk label: LOGICAL VOLUME duid: 410f0efc5a9d86dd flags: bytes/sector: 512 sectors/track: 63 tracks/cylinder: 255 sectors/cylinder: 16065 cylinders: 36468 total sectors: 585871964 boundstart: 64 boundend: 585858420 drivedata: 0 16 partitions: #size offset fstype [fsize bsize cpg] a: 1028096 64 4.2BSD 2048 163841 # / c:5858719640 unused d: 1028160 1028160 4.2BSD 2048 163841 # /var e:146801952 2056320 4.2BSD 2048 163841 # /usr f: 20964832148858272 4.2BSD 2048 163841 # /home g:416035264169823104 4.2BSD 4096 327681 # /log # mount /dev/sd0a on / type ffs (local, noatime, softdep) /dev/sd0f on /home type ffs (local, noatime, nodev, nosuid, softdep) /dev/sd0g on /log type ffs (local, noatime, nodev, nosuid, softdep) /dev/sd0e on /usr type ffs (local, noatime, nodev, softdep) /dev/sd0d on /var type ffs (local, noatime, nodev, nosuid, softdep) # dmesg|grep ciss ciss0 at pci1 dev 0 function 0 Hewlett-Packard Smart Array rev 0x01: apic 0 int 4 ciss0: 2 LDs, HW rev 2, FW 3.66/3.66, 64bit fifo rro scsibus0 at ciss0: 2 targets before applying your patch: [/usr] # dd if=/dev/zero of=testfile bs=1m count=1000 1000+0 records in 1000+0 records out 1048576000 bytes transferred in 16.428 secs (63825353 bytes/sec) [/usr] # dd if=/dev/zero of=testfile bs=1m count=1 1+0 records in 1+0 records out 1048576 bytes transferred in 153.910 secs (68128911 bytes/sec) [/log] # dd if=/dev/zero of=testfile bs=1m count=1000 1000+0 records in 1000+0 records out 1048576000 bytes transferred in 8.122 secs (129087680 bytes/sec) [/log] # dd if=/dev/zero of=testfile bs=1m count=1 1+0 records in 1+0 records out 1048576 bytes transferred in 87.701 secs (119561580 bytes/sec) after applying your patch: [/usr] # dd if=/dev/zero of=testfile bs=1m count=1000 1000+0 records in 1000+0 records out 1048576000 bytes transferred in 14.113 secs (74296489 bytes/sec) [/usr] # dd if=/dev/zero of=testfile bs=1m count=1 1+0 records in 1+0 records out 1048576 bytes transferred in 154.600 secs (67824996 bytes/sec) [/log] # dd if=/dev/zero of=testfile bs=1m count=1000 1000+0 records in 1000+0 records out 1048576000 bytes transferred in 6.836 secs (153379539 bytes/sec) [/log] # dd if=/dev/zero of=testfile bs=1m count=1 1+0 records in 1+0 records out 1048576 bytes transferred in 82.955 secs (126402027 bytes/sec) The larger fsize/bsize of partition sd0g almost seems to double the writing throughput in comparison to partition sd0e. I didn't expect this much of a difference. Regarding performance, copying many small files (~190 MB) is much worse (time is identical before and after the patch): [/log/test] # date ; cp -Rp /usr/src/sys . ; date Tue May 29 20:42:32 CEST 2012 Tue May 29 20:43:15 CEST 2012 Best Regards Andreas
