Re: ciss(4) write very slow w/o bbwc
Hi! I tested the performance w and w/o the patch. There is no difference. ciss0 at pci3 dev 3 function 0 "Compaq Smart Array 64xx" rev 0x01: apic 10 int 3 ciss0: 2 LDs, HW rev 1, FW 2.84/2.84, 64bit fifo scsibus0 at ciss0: 2 targets sd0 at scsibus0 targ 0 lun 0: SCSI2 0/direct fixed sd0: 69459MB, 512 bytes/sector, 142253280 sectors sd1 at scsibus0 targ 1 lun 0: SCSI2 0/direct fixed sd1: 140006MB, 512 bytes/sector, 286734240 sectors kern.bufcachepercent=20 w/o patch raw device # dd if=/dev/zero of=/dev/rsd1d bs=1m count=1000 1000+0 records in 1000+0 records out 1048576000 bytes transferred in 75.550 secs (13879230 bytes/sec) file # dd if=/dev/zero of=test bs=1m count=1000 1000+0 records in 1000+0 records out 1048576000 bytes transferred in 16.986 secs (61728607 bytes/sec) w patch raw device # dd if=/dev/zero of=/dev/rsd1d bs=1m count=1000 1000+0 records in 1000+0 records out 1048576000 bytes transferred in 75.609 secs (13868396 bytes/sec) file # dd if=/dev/zero of=test bs=1m count=1000 1000+0 records in 1000+0 records out 1048576000 bytes transferred in 16.165 secs (64863961 bytes/sec) In fact, the file test performance is acceptable for me. The raw performace (eg. newfs) is not so important. Thx csszep 2012/5/29 Andreas Bartelt : > Hello, > > > On 05/29/12 17:28, Kenneth R Westerback wrote: >> >> On Tue, May 29, 2012 at 03:48:02PM +0200, csszep wrote: >>> >>> Hi! >>> >>> So i tested the ciss performance with Openbsd 5.1 and Netbsd 5.1.2 and >>> the numbers are the same. :( >>> >>> approx 13Mbyte/s write with dd if=/dev/zero of=/dev/rsd1c bs=1m count=500 >>> >>> But why Linux is four times faster (approx 40Mbyte/s)? >> >> >> Dunno. But the diff below should apply the NetBSD 'fix' for the INQUIRY >> command. >> >> Ken >> >> >> Dunno. But the diff below should apply the NetBSD 'fix' for the INQUIRY >> command. >> > > I also can confirm relatively slow ciss(4) performance on OpenBSD. Enabling > the (not battery backed) cache via BIOS doesn't help significantly. > > I just did some tests on a HP Proliant DL360G7 with RAID1 via ciss(4) with > 2x300GB 6G SAS 1 rpm HDDs (cache disabled on this box): > > # disklabel sd0 > # /dev/rsd0c: > type: SCSI > disk: SCSI disk > label: LOGICAL VOLUME > duid: 410f0efc5a9d86dd > flags: > bytes/sector: 512 > sectors/track: 63 > tracks/cylinder: 255 > sectors/cylinder: 16065 > cylinders: 36468 > total sectors: 585871964 > boundstart: 64 > boundend: 585858420 > drivedata: 0 > > 16 partitions: > #size offset fstype [fsize bsize cpg] > a: 1028096 64 4.2BSD 2048 16384 1 # / > c:5858719640 unused > d: 1028160 1028160 4.2BSD 2048 16384 1 # /var > e:146801952 2056320 4.2BSD 2048 16384 1 # /usr > f: 20964832148858272 4.2BSD 2048 16384 1 # /home > g:416035264169823104 4.2BSD 4096 32768 1 # /log > > # mount > /dev/sd0a on / type ffs (local, noatime, softdep) > /dev/sd0f on /home type ffs (local, noatime, nodev, nosuid, softdep) > /dev/sd0g on /log type ffs (local, noatime, nodev, nosuid, softdep) > /dev/sd0e on /usr type ffs (local, noatime, nodev, softdep) > /dev/sd0d on /var type ffs (local, noatime, nodev, nosuid, softdep) > > > # dmesg|grep ciss > ciss0 at pci1 dev 0 function 0 "Hewlett-Packard Smart Array" rev 0x01: apic > 0 int 4 > ciss0: 2 LDs, HW rev 2, FW 3.66/3.66, 64bit fifo rro > scsibus0 at ciss0: 2 targets > > before applying your patch: > > [/usr] > # dd if=/dev/zero of=testfile bs=1m count=1000 > 1000+0 records in > 1000+0 records out > 1048576000 bytes transferred in 16.428 secs (63825353 bytes/sec) > > [/usr] > # dd if=/dev/zero of=testfile bs=1m count=1 > 1+0 records in > 1+0 records out > 1048576 bytes transferred in 153.910 secs (68128911 bytes/sec) > > [/log] > # dd if=/dev/zero of=testfile bs=1m count=1000 > 1000+0 records in > 1000+0 records out > 1048576000 bytes transferred in 8.122 secs (129087680 bytes/sec) > > [/log] > # dd if=/dev/zero of=testfile bs=1m count=1 > 1+0 records in > 1+0 records out > 1048576 bytes transferred in 87.701 secs (119561580 bytes/sec) > > after applying your patch: > > [/usr] > # dd if=/dev/zero of=testfile bs=1m count=1000 > 1000+0 records in > 1000+0 records out > 1048576000 bytes transferred in 14.113 secs (74296489 bytes/sec) > > [/usr] > # dd if=/dev/zero of=testfile bs=1m count=1 > 1+0 records in > 1+0 records out > 1048576 bytes transferred in 154.600 secs (67824996 bytes/sec) > > [/log] > # dd if=/dev/zero of=testfile bs=1m count=1000 > 1000+0 records in > 1000+0 records out > 1048576000 bytes transferred in 6.836 secs (153379539 bytes/sec) > > [/log] > # dd if=/dev/zero of=testfile bs=1m count=1 > 1+0 records in > 1+0 records out > 1048576 bytes transferred in 82.955 secs (126402027 bytes/sec) > > The larger fsize/bsize of partition sd0g almost seems to double the writing > t
Re: ciss(4) write very slow w/o bbwc
Hello, On 05/29/12 17:28, Kenneth R Westerback wrote: On Tue, May 29, 2012 at 03:48:02PM +0200, csszep wrote: Hi! So i tested the ciss performance with Openbsd 5.1 and Netbsd 5.1.2 and the numbers are the same. :( approx 13Mbyte/s write with dd if=/dev/zero of=/dev/rsd1c bs=1m count=500 But why Linux is four times faster (approx 40Mbyte/s)? Dunno. But the diff below should apply the NetBSD 'fix' for the INQUIRY command. Ken Dunno. But the diff below should apply the NetBSD 'fix' for the INQUIRY command. I also can confirm relatively slow ciss(4) performance on OpenBSD. Enabling the (not battery backed) cache via BIOS doesn't help significantly. I just did some tests on a HP Proliant DL360G7 with RAID1 via ciss(4) with 2x300GB 6G SAS 1 rpm HDDs (cache disabled on this box): # disklabel sd0 # /dev/rsd0c: type: SCSI disk: SCSI disk label: LOGICAL VOLUME duid: 410f0efc5a9d86dd flags: bytes/sector: 512 sectors/track: 63 tracks/cylinder: 255 sectors/cylinder: 16065 cylinders: 36468 total sectors: 585871964 boundstart: 64 boundend: 585858420 drivedata: 0 16 partitions: #size offset fstype [fsize bsize cpg] a: 1028096 64 4.2BSD 2048 163841 # / c:5858719640 unused d: 1028160 1028160 4.2BSD 2048 163841 # /var e:146801952 2056320 4.2BSD 2048 163841 # /usr f: 20964832148858272 4.2BSD 2048 163841 # /home g:416035264169823104 4.2BSD 4096 327681 # /log # mount /dev/sd0a on / type ffs (local, noatime, softdep) /dev/sd0f on /home type ffs (local, noatime, nodev, nosuid, softdep) /dev/sd0g on /log type ffs (local, noatime, nodev, nosuid, softdep) /dev/sd0e on /usr type ffs (local, noatime, nodev, softdep) /dev/sd0d on /var type ffs (local, noatime, nodev, nosuid, softdep) # dmesg|grep ciss ciss0 at pci1 dev 0 function 0 "Hewlett-Packard Smart Array" rev 0x01: apic 0 int 4 ciss0: 2 LDs, HW rev 2, FW 3.66/3.66, 64bit fifo rro scsibus0 at ciss0: 2 targets before applying your patch: [/usr] # dd if=/dev/zero of=testfile bs=1m count=1000 1000+0 records in 1000+0 records out 1048576000 bytes transferred in 16.428 secs (63825353 bytes/sec) [/usr] # dd if=/dev/zero of=testfile bs=1m count=1 1+0 records in 1+0 records out 1048576 bytes transferred in 153.910 secs (68128911 bytes/sec) [/log] # dd if=/dev/zero of=testfile bs=1m count=1000 1000+0 records in 1000+0 records out 1048576000 bytes transferred in 8.122 secs (129087680 bytes/sec) [/log] # dd if=/dev/zero of=testfile bs=1m count=1 1+0 records in 1+0 records out 1048576 bytes transferred in 87.701 secs (119561580 bytes/sec) after applying your patch: [/usr] # dd if=/dev/zero of=testfile bs=1m count=1000 1000+0 records in 1000+0 records out 1048576000 bytes transferred in 14.113 secs (74296489 bytes/sec) [/usr] # dd if=/dev/zero of=testfile bs=1m count=1 1+0 records in 1+0 records out 1048576 bytes transferred in 154.600 secs (67824996 bytes/sec) [/log] # dd if=/dev/zero of=testfile bs=1m count=1000 1000+0 records in 1000+0 records out 1048576000 bytes transferred in 6.836 secs (153379539 bytes/sec) [/log] # dd if=/dev/zero of=testfile bs=1m count=1 1+0 records in 1+0 records out 1048576 bytes transferred in 82.955 secs (126402027 bytes/sec) The larger fsize/bsize of partition sd0g almost seems to double the writing throughput in comparison to partition sd0e. I didn't expect this much of a difference. Regarding performance, copying many small files (~190 MB) is much worse (time is identical before and after the patch): [/log/test] # date ; cp -Rp /usr/src/sys . ; date Tue May 29 20:42:32 CEST 2012 Tue May 29 20:43:15 CEST 2012 Best Regards Andreas
Re: ciss(4) write very slow w/o bbwc
On Tue, May 29, 2012 at 03:48:02PM +0200, csszep wrote:
> Hi!
>
> So i tested the ciss performance with Openbsd 5.1 and Netbsd 5.1.2 and
> the numbers are the same. :(
>
> approx 13Mbyte/s write with dd if=/dev/zero of=/dev/rsd1c bs=1m count=500
>
> But why Linux is four times faster (approx 40Mbyte/s)?
Dunno. But the diff below should apply the NetBSD 'fix' for the INQUIRY
command.
Ken
Dunno. But the diff below should apply the NetBSD 'fix' for the INQUIRY
command.
Ken
Index: ciss.c
===
RCS file: /cvs/src/sys/dev/ic/ciss.c,v
retrieving revision 1.67
diff -u -p -r1.67 ciss.c
--- ciss.c 10 Nov 2011 12:05:11 - 1.67
+++ ciss.c 29 May 2012 15:18:30 -
@@ -642,6 +642,14 @@ ciss_done(struct ciss_ccb *ccb)
if (xs) {
xs->resid = 0;
+ if (xs->cmd->opcode == INQUIRY) {
+ struct scsi_inquiry_data *inq;
+ inq = (struct scsi_inquiry_data *)xs->data;
+ if ((inq->version & SID_ANSII) == 0 &&
+(inq->flags & SID_CmdQue) != 0) {
+ inq->version |= 2;
+}
+ }
scsi_done(xs);
}
Re: ciss(4) write very slow w/o bbwc
Hi! So i tested the ciss performance with Openbsd 5.1 and Netbsd 5.1.2 and the numbers are the same. :( approx 13Mbyte/s write with dd if=/dev/zero of=/dev/rsd1c bs=1m count=500 But why Linux is four times faster (approx 40Mbyte/s)? thx csszep 2012/5/29 csszep : > Ok, but i installed Linux (Debian 6) and there is no performance degradation. > > I will install NetBSD too, and i will do a test. > > The commit does not turn on the cache, it enable tagged queing if i > understand it well. > > thx > csszep > > > 2012/5/29 Jonathan Gray : >> I don't think that commit will fix the problem. >> HP shouldn't sell machines without the battery, but they do. >> From memory the firmware on the raid controller has no way >> of turning on caching without the battery being present. >> >> On Tue, May 29, 2012 at 11:30:34AM +0200, csszep wrote: >>> Hi Misc! >>> >>> We have some older HP Dl360, Dl380 G4 machines with Smart Array 6i >>> controllores w/o battery backed cache. >>> >>> The disk performance in this case is really poor, for examle the >>> disklabel operation on a 72GB disk lasted for about 5 mins. >>> >>> I found a commit in a NetBSD ciss driver (which is a port of OpenBSD >>> driver), that solve the problem i think: >>> >>> http://cvsweb.netbsd.org/bsdweb.cgi/src/sys/dev/ic/ciss.c?rev=1.23&content-type=text/x-cvsweb-markup&only_with_tag=MAIN >>> >>> The problem is, that the NetBSD scsi midlayer is very different from >>> the OpenBSD, so is it possible to integrate this patch? My C and >>> OpenBSD internals knowledge is not enough for this task. >>> >>> PS: Yes i know, my english is terrible. >>> >>> thx >>> csszep
Re: ciss(4) write very slow w/o bbwc
On Tue, May 29, 2012 at 12:25:51PM +0100, Rodolfo Gouveia wrote: > On Tue, May 29, 2012 at 08:31:06PM +1000, Jonathan Gray wrote: > > From memory the firmware on the raid controller has no way > > of turning on caching without the battery being present. > > I run some ciss, like the HP P212 and P410 and I can override > the cache setting, that is I can turn it on even without a > battery installed. It gives me a warning though :-P Perhaps it is the older ones I am thinking of then, probably something like the 64xx.
Re: ciss(4) write very slow w/o bbwc
Ok, but i installed Linux (Debian 6) and there is no performance degradation. I will install NetBSD too, and i will do a test. The commit does not turn on the cache, it enable tagged queing if i understand it well. thx csszep 2012/5/29 Jonathan Gray : > I don't think that commit will fix the problem. > HP shouldn't sell machines without the battery, but they do. > From memory the firmware on the raid controller has no way > of turning on caching without the battery being present. > > On Tue, May 29, 2012 at 11:30:34AM +0200, csszep wrote: >> Hi Misc! >> >> We have some older HP Dl360, Dl380 G4 machines with Smart Array 6i >> controllores w/o battery backed cache. >> >> The disk performance in this case is really poor, for examle the >> disklabel operation on a 72GB disk lasted for about 5 mins. >> >> I found a commit in a NetBSD ciss driver (which is a port of OpenBSD >> driver), that solve the problem i think: >> >> http://cvsweb.netbsd.org/bsdweb.cgi/src/sys/dev/ic/ciss.c?rev=1.23&content-type=text/x-cvsweb-markup&only_with_tag=MAIN >> >> The problem is, that the NetBSD scsi midlayer is very different from >> the OpenBSD, so is it possible to integrate this patch? My C and >> OpenBSD internals knowledge is not enough for this task. >> >> PS: Yes i know, my english is terrible. >> >> thx >> csszep
Re: ciss(4) write very slow w/o bbwc
On Tue, May 29, 2012 at 08:31:06PM +1000, Jonathan Gray wrote: > From memory the firmware on the raid controller has no way > of turning on caching without the battery being present. I run some ciss, like the HP P212 and P410 and I can override the cache setting, that is I can turn it on even without a battery installed. It gives me a warning though :-P cheers, rodolfo
Re: ciss(4) write very slow w/o bbwc
I don't think that commit will fix the problem. HP shouldn't sell machines without the battery, but they do. >From memory the firmware on the raid controller has no way of turning on caching without the battery being present. On Tue, May 29, 2012 at 11:30:34AM +0200, csszep wrote: > Hi Misc! > > We have some older HP Dl360, Dl380 G4 machines with Smart Array 6i > controllores w/o battery backed cache. > > The disk performance in this case is really poor, for examle the > disklabel operation on a 72GB disk lasted for about 5 mins. > > I found a commit in a NetBSD ciss driver (which is a port of OpenBSD > driver), that solve the problem i think: > > http://cvsweb.netbsd.org/bsdweb.cgi/src/sys/dev/ic/ciss.c?rev=1.23&content-type=text/x-cvsweb-markup&only_with_tag=MAIN > > The problem is, that the NetBSD scsi midlayer is very different from > the OpenBSD, so is it possible to integrate this patch? My C and > OpenBSD internals knowledge is not enough for this task. > > PS: Yes i know, my english is terrible. > > thx > csszep
ciss(4) write very slow w/o bbwc
Hi Misc! We have some older HP Dl360, Dl380 G4 machines with Smart Array 6i controllores w/o battery backed cache. The disk performance in this case is really poor, for examle the disklabel operation on a 72GB disk lasted for about 5 mins. I found a commit in a NetBSD ciss driver (which is a port of OpenBSD driver), that solve the problem i think: http://cvsweb.netbsd.org/bsdweb.cgi/src/sys/dev/ic/ciss.c?rev=1.23&content-type=text/x-cvsweb-markup&only_with_tag=MAIN The problem is, that the NetBSD scsi midlayer is very different from the OpenBSD, so is it possible to integrate this patch? My C and OpenBSD internals knowledge is not enough for this task. PS: Yes i know, my english is terrible. thx csszep
