3-4MB/s vinum performance with two Promise PDC20268 UDMA100 controllers
Hi I had four 160G IDE drives in raid5 on a single controller and it worked just fine, never benchmarked it since it was faster than the network anyway. But when adding a second controller card and two more drives the new array has a terrible write performance. I've tried various stripe and block sizes in desperation but that didnt help. Then I've tried assinging the same irq to both controller card in case it was interrupts causing the slow down, so I set both pci slots to use irq3 in the bios (freebsd wants irq 3 for a non existent sio1 port - so I figure that'll be 'free'?) but despite my setting in the bios the cards still show up in dmesg with irq 21 and 22? So I looked thourgh the handbook and tried setting the irq in /boot/device.hints both as hint.atapci.x.irq=3 and hint.ata.x.irq=3 but this didnt work either. The problem is the same in freebsd 5.1 and 5.2 (output below is form 5.2): home# dmesg | grep atapci atapci0: Promise PDC20268 UDMA100 controller port 0xb000-0xb00f,0xb400-0xb403,0xb800-0xb807,0xd000-0xd003,0xd400-0xd407 mem 0xf900-0xf9003fff irq 21 at device 9.0 on pci1 atapci0: [MPSAFE] ata2: at 0xd400 on atapci0 ata3: at 0xb800 on atapci0 atapci1: Promise PDC20268 UDMA100 controller port 0x9400-0x940f,0x9800-0x9803,0xa000-0xa007,0xa400-0xa403,0xa800-0xa807 mem 0xf880-0xf8803fff irq 22 at device 10.0 on pci1 atapci1: [MPSAFE] ata4: at 0xa800 on atapci1 ata5: at 0xa000 on atapci1 atapci2: Intel ICH2 UDMA100 controller port 0x8800-0x880f at device 31.1 on pci0 ata0: at 0x1f0 irq 14 on atapci2 ata1: at 0x170 irq 15 on atapci2 Any thoughts anyone? Bjorn ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: 3-4MB/s vinum performance with two Promise PDC20268 UDMA100 controllers
Bjorn Eikeland wrote: Hi I had four 160G IDE drives in raid5 on a single controller and it worked just fine, never benchmarked it since it was faster than the network anyway. But when adding a second controller card and two more drives the new array has a terrible write performance. I've tried various stripe and block sizes in desperation but that didnt help. Then I've tried assinging the same irq to both controller card in case it was interrupts causing the slow down, so I set both pci slots to use irq3 in the bios (freebsd wants irq 3 for a non existent sio1 port - so I figure that'll be 'free'?) but despite my setting in the bios the cards still show up in dmesg with irq 21 and 22? So I looked thourgh the handbook and tried setting the irq in /boot/device.hints both as hint.atapci.x.irq=3 and hint.ata.x.irq=3 but this didnt work either. The problem is the same in freebsd 5.1 and 5.2 (output below is form 5.2): home# dmesg | grep atapci atapci0: Promise PDC20268 UDMA100 controller port 0xb000-0xb00f,0xb400-0xb403,0xb800-0xb807,0xd000-0xd003,0xd400-0xd407 mem 0xf900-0xf9003fff irq 21 at device 9.0 on pci1 atapci0: [MPSAFE] ata2: at 0xd400 on atapci0 ata3: at 0xb800 on atapci0 atapci1: Promise PDC20268 UDMA100 controller port 0x9400-0x940f,0x9800-0x9803,0xa000-0xa007,0xa400-0xa403,0xa800-0xa807 mem 0xf880-0xf8803fff irq 22 at device 10.0 on pci1 atapci1: [MPSAFE] ata4: at 0xa800 on atapci1 ata5: at 0xa000 on atapci1 atapci2: Intel ICH2 UDMA100 controller port 0x8800-0x880f at device 31.1 on pci0 ata0: at 0x1f0 irq 14 on atapci2 ata1: at 0x170 irq 15 on atapci2 Any thoughts anyone? Bjorn ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED] Don't assign the same irq to 2 devices, thats a conflict! You may not have used win95, but you don't won't to do that. I know raid 5 isslower than raid 1, but I don't remember any numbers. Also the more complex you make the system the slower it will go, hince raid 5 slower than raid 1. Also pci is a shared bus meaning 1 device talks at a time, so maybe, just maybe if you chipset has a pci bridge because you have like 8 slots, or the make was real kind, you could try card 1 in say slot 2, and card 2 in slot 6? You maybe able to get simultainsuos writes and reads that way. The best option is pcix, or a controler that support 8 drives on its own. Also try to adjust pci latincy or waiting, do a google on it, I hear 95-128 clicks is good. I just noticed the buillt in ide is on pci0 while the rest are on pci1, theres a bridge you can use. If you can get it setup, have 2 drives connected to the onboard ide and the rest to your card. Jason ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: 3-4MB/s vinum performance with two Promise PDC20268 UDMA100 controllers
On Thursday, 15 January 2004 at 10:44:20 +0100, Bjorn Eikeland wrote: Hi I had four 160G IDE drives in raid5 on a single controller and it worked just fine, never benchmarked it since it was faster than the network anyway. But when adding a second controller card and two more drives the new array has a terrible write performance. ... Any thoughts anyone? How about some information about your configuration and how you measured the performance? Greg -- When replying to this message, please copy the original recipients. If you don't, I may ignore the reply or reply to the original recipients. For more information, see http://www.lemis.com/questions.html See complete headers for address and phone numbers. pgp0.pgp Description: PGP signature
Re: 3-4MB/s vinum performance with two Promise PDC20268 UDMA100 controllers
På Thu, 15 Jan 2004 09:43:01 -0200, skrev jason [EMAIL PROTECTED]: Bjorn Eikeland wrote: Hi I had four 160G IDE drives in raid5 on a single controller and it worked just fine, never benchmarked it since it was faster than the network anyway. But when adding a second controller card and two more drives the new array has a terrible write performance. I've tried various stripe and block sizes in desperation but that didnt help. Then I've tried assinging the same irq to both controller card in case it was interrupts causing the slow down, so I set both pci slots to use irq3 in the bios (freebsd wants irq 3 for a non existent sio1 port - so I figure that'll be 'free'?) but despite my setting in the bios the cards still show up in dmesg with irq 21 and 22? So I looked thourgh the handbook and tried setting the irq in /boot/device.hints both as hint.atapci.x.irq=3 and hint.ata.x.irq=3 but this didnt work either. The problem is the same in freebsd 5.1 and 5.2 (output below is form 5.2): home# dmesg | grep atapci atapci0: Promise PDC20268 UDMA100 controller port 0xb000-0xb00f,0xb400-0xb403,0xb800-0xb807,0xd000-0xd003,0xd400-0xd407 mem 0xf900-0xf9003fff irq 21 at device 9.0 on pci1 atapci0: [MPSAFE] ata2: at 0xd400 on atapci0 ata3: at 0xb800 on atapci0 atapci1: Promise PDC20268 UDMA100 controller port 0x9400-0x940f,0x9800-0x9803,0xa000-0xa007,0xa400-0xa403,0xa800-0xa807 mem 0xf880-0xf8803fff irq 22 at device 10.0 on pci1 atapci1: [MPSAFE] ata4: at 0xa800 on atapci1 ata5: at 0xa000 on atapci1 atapci2: Intel ICH2 UDMA100 controller port 0x8800-0x880f at device 31.1 on pci0 ata0: at 0x1f0 irq 14 on atapci2 ata1: at 0x170 irq 15 on atapci2 Any thoughts anyone? Bjorn ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED] Don't assign the same irq to 2 devices, thats a conflict! You may not have used win95, but you don't won't to do that. I know raid 5 isslower than raid 1, but I don't remember any numbers. Also the more complex you make the system the slower it will go, hince raid 5 slower than raid 1. Also pci is a shared bus meaning 1 device talks at a time, so maybe, just maybe if you chipset has a pci bridge because you have like 8 slots, or the make was real kind, you could try card 1 in say slot 2, and card 2 in slot 6? You maybe able to get simultainsuos writes and reads that way. The best option is pcix, or a controler that support 8 drives on its own. Also try to adjust pci latincy or waiting, do a google on it, I hear 95-128 clicks is good. I just noticed the buillt in ide is on pci0 while the rest are on pci1, theres a bridge you can use. If you can get it setup, have 2 drives connected to the onboard ide and the rest to your card. Jason Thank you for your thoughts Jason! (Your penny is in the mail ;) About the irq thing I think I read (while reading up on bridges) that interrupts were level triggered (as apposed to edge triggered) and thus two NICs sharing a interrupt and asserting it at the same time would only cause one context switch - so I figured it worth a try. As for the pci bus etc, all three pci slots are pci1 - and the secondary onboard ide channel gives me read and write errors (the same type as for a bad udma100 cable - but im sure its the controller as the cable and drives work fine elsewhere). I've found some info on pci latency but will try it tomorrow as the box is headless - just made some refrence measurements tonight. But I acutally remembered that the previos setup wasn't raid5 - the four first drives were striped since it was a temporary arrangement while waiting for the 2nd controller - however I do have a linux machine at home with 4 of the same drives (all on the onboard UDMA100 controller) running raid5 and it does perform better (cant do any mesurements now, but it does accept data at about 60Mbps over a smb share and i think the client maxed out at that). I've done some measurements on the drives with different setups - the results were quite long so I've posted it on a web page instad http://www.eikeland.info/bjorn/archive/040117vinumperf1.txt (Test was dd count=1 bs=65536 if=/dev/zero of=/dev/vinum/test) Briefly summarized: any raid5 (with 3, 4 or 6 drives) writes ~4M/s and reads 27M, 29M and 32M/s respectly. A single drive reads and writes ~40M/s. Raid0 (4 and 6 drives) writes ~50M and reads 50M and 62M/s respectively. Test done to from /dev/zero with 625M or 512M test file. Is this really what write performance I can expect from a raid5 array usch as this? I knew it wouldnt be writing way fast, but I was quite sure it would accept what the 100Mbit network had to offer? (Should I maybe move this over to freebsd-performance?) -Bjorn ___ [EMAIL PROTECTED] mailing list
