Poor SCSI disk preformance
Hi Everyone, I'm having difficulty getting my SCSI hard disks to preform well. I don't know what tools are available to help me diagnose this issue, or if there are specific tweaks that I need to make. Attached is the output from dmesg. The reason why I say that performance is slow is that when I run the following: # dd if=/dev/zero of=/dev/da0 count=200 bs=128k 200+0 records in 200+0 records out 26214400 bytes transferred in 5.100589 secs (5139485 bytes/sec) You can see that it doesn't transfer very fast. If I do the whole drive, I still get the same throughput. It strikes me as odd that an older ATA disk preforms better than the newer SCSI disk. I am under the impression that a standard run-of-the mill ATA drive will do anywhere from 10-15 MB/s sequential transfer, which is what I get when writing to an actual filesystem (with no soft-updates): # mount /dev/ad0s1a on / (ufs, local) /dev/ad0s1f on /tmp (ufs, local, soft-updates) /dev/ad0s1g on /usr (ufs, local, soft-updates) /dev/ad0s1e on /var (ufs, local, soft-updates) procfs on /proc (procfs, local) # dd if=/dev/zero of=/temp.234233 count=200 bs=128k 200+0 records in 200+0 records out 26214400 bytes transferred in 2.437368 secs (10755208 bytes/sec) And with soft-updates: # dd if=/dev/zero of=/usr/temp.234233 count=200 bs=128k 200+0 records in 200+0 records out 26214400 bytes transferred in 2.759680 secs (9499072 bytes/sec) I also had (meaning it is not currently attached) a different SCSI drive attached on the bus, with the same results. Has anyone any tips for this from a FreeBSD point of view? TIA, Derek Copyright (c) 1992-2003 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD 4.8-RELEASE #0: Thu Apr 3 10:53:38 GMT 2003 [EMAIL PROTECTED]:/usr/obj/usr/src/sys/GENERIC Timecounter i8254 frequency 1193182 Hz CPU: Pentium II/Pentium II Xeon/Celeron (298.54-MHz 686-class CPU) Origin = GenuineIntel Id = 0x660 Stepping = 0 Features=0x183f9ffFPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,MMX,FXSR real memory = 134217728 (131072K bytes) avail memory = 125378560 (122440K bytes) Preloaded elf kernel kernel at 0xc051d000. Pentium Pro MTRR support enabled md0: Malloc disk Using $PIR table, 9 entries at 0xc00ede10 npx0: math processor on motherboard npx0: INT 16 interface pcib0: Intel 82443BX (440 BX) host to PCI bridge on motherboard pci0: PCI bus on pcib0 agp0: Intel 82443BX (440 BX) host to PCI bridge mem 0x4400-0x47ff at device 0.0 on pci0 pcib1: Intel 82443BX (440 BX) PCI-PCI (AGP) bridge at device 1.0 on pci0 pci1: PCI bus on pcib1 pci1: ATI Mach64-GD graphics accelerator at 0.0 irq 11 fxp0: Intel Pro 10/100B/100+ Ethernet port 0x2400-0x241f mem 0x4010-0x401f,0x4200-0x42000fff irq 11 at device 15.0 on pci0 fxp0: Ethernet address 00:08:c7:89:c4:28 inphy0: i82555 10/100 media interface on miibus0 inphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto ahc0: Adaptec 2940 Ultra SCSI adapter port 0x2000-0x20ff mem 0x4020-0x40200fff irq 11 at device 16.0 on pci0 aic7880: Ultra Wide Channel A, SCSI Id=7, 16/253 SCBs isab0: Intel 82371AB PCI to ISA bridge at device 20.0 on pci0 isa0: ISA bus on isab0 atapci0: Intel PIIX4 ATA33 controller port 0x2440-0x244f at device 20.1 on pci0 ata0: at 0x1f0 irq 14 on atapci0 ata1: at 0x170 irq 15 on atapci0 uhci0: Intel 82371AB/EB (PIIX4) USB controller port 0x2420-0x243f irq 11 at device 20.2 on pci0 usb0: Intel 82371AB/EB (PIIX4) USB controller on uhci0 usb0: USB revision 1.0 uhub0: Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1 uhub0: 2 ports with 2 removable, self powered chip0: Intel 82371AB Power management controller port 0xfc00-0xfc0f at device 20.3 on pci0 orm0: Option ROMs at iomem 0xc-0xc7fff,0xc8000-0xc97ff,0xc9800-0xd07ff,0xe-0xe7fff on isa0 fdc0: NEC 72065B or clone at port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 on isa0 fdc0: FIFO enabled, 8 bytes threshold fd0: 1440-KB 3.5 drive on fdc0 drive 0 atkbdc0: Keyboard controller (i8042) at port 0x60,0x64 on isa0 atkbd0: AT Keyboard flags 0x1 irq 1 on atkbdc0 kbd0 at atkbd0 psm0: PS/2 Mouse irq 12 on atkbdc0 psm0: model Generic PS/2 mouse, device ID 0 vga0: Generic ISA VGA at port 0x3c0-0x3df iomem 0xa-0xb on isa0 sc0: System console at flags 0x100 on isa0 sc0: VGA 16 virtual consoles, flags=0x300 sio0 at port 0x3f8-0x3ff irq 4 flags 0x10 on isa0 sio0: type 16550A sio1 at port 0x2f8-0x2ff irq 3 on isa0 sio1: type 16550A ppc0: Parallel port at port 0x378-0x37f irq 7 on isa0 ppc0: SMC-like chipset (ECP/EPP/PS2/NIBBLE) in COMPATIBLE mode ppc0: FIFO with 16/16/8 bytes threshold plip0: PLIP network interface on ppbus0 lpt0: Printer on ppbus0 lpt0: Interrupt-driven port ppi0: Parallel I/O on ppbus0 ad0: 3093MB FUJITSU MPB3032ATU [6286/16/63] at ata0-master UDMA33 Waiting 15 seconds for SCSI devices to settle Mounting root from ufs:/dev/ad0s1a da0 at ahc0
Re: Poor SCSI disk preformance
I also had (meaning it is not currently attached) a different SCSI drive attached on the bus, with the same results. Has anyone any tips for this from a FreeBSD point of view? I wouldn't say that dd is the greatest benchmarking tool. You may want to try benchmarks/rawio. Also, try monitoring diffferent types of transfers to and from another physical disk with iostat. I'm not sure what your speed expectations are, but you're running a 7200 RPM Ultra Wide disk on an Ultra Wide host adapter; not exactly the fastest SCSI technology. ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Poor SCSI disk preformance
In the last episode (Jan 06), Mike Maltese said: I also had (meaning it is not currently attached) a different SCSI drive attached on the bus, with the same results. Has anyone any tips for this from a FreeBSD point of view? I wouldn't say that dd is the greatest benchmarking tool. You may want to try benchmarks/rawio. Also, try monitoring diffferent types of transfers to and from another physical disk with iostat. I'm not sure what your speed expectations are, but you're running a 7200 RPM Ultra Wide disk on an Ultra Wide host adapter; not exactly the fastest SCSI technology. It should go faster than 5MB/sec, though. Seagate's specs say that drive should do 14MB/sec max. UW's top speed is 40MB/sec, so there shouldn't be any bottlenecks. -- Dan Nelson [EMAIL PROTECTED] ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Poor SCSI disk preformance
It should go faster than 5MB/sec, though. Seagate's specs say that drive should do 14MB/sec max. UW's top speed is 40MB/sec, so there shouldn't be any bottlenecks. 14MB/s is the maximum internal transfer rate. Also, we're talking about write performance here, which will likely be quite a bit slower than read performance. As I said before, a more realistic number should be had with rawio or by measuring actual real-world transfers. Unless all you do is write zeros with dd all day long, I don't think that that is the best measure of performance. ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Poor SCSI disk preformance
I wouldn't say that dd is the greatest benchmarking tool. You may want to try benchmarks/rawio. I'll check that out just for kicks, but I _actually want_ to write zeros to the drive first, not just as a benchmark. The reasoning for this is that I'm trying to create a dedicated box to format HDDs in parallel. I wish to first zero the drives to make data recovery without an electron microscope difficult. Then, to test for bad sectors I do a checksum of the number of zero bytes written to disk, and then I read back from the disk and compare checksums. Not exactly an extensive test, and perhaps there is a verify option or trick in dd that I'm not aware of. I think that this would catch any blatantly bad drives... If not, there are 2 full disk operations that should be going faster. Actually, just for kicks: # dd if=/dev/da0 of=/dev/null bs=128k [1] 1839 # iostat -K -w 1 da0 tty da0 cpu tin tout KB/t tps MB/s us ni sy in id 1 42 64.00 607 37.92 1 0 1 0 98 0 43 64.00 222 13.87 0 0 2 0 98 0 43 64.00 223 13.92 0 0 0 2 98 0 42 64.00 224 13.98 0 0 2 0 98 0 43 64.00 222 13.86 0 0 3 0 97 0 43 64.00 223 13.92 0 0 1 2 98 0 43 64.00 223 13.92 0 0 2 1 97 0 42 64.00 223 13.92 0 0 3 0 97 0 43 64.00 223 13.92 0 0 1 0 99 Seems to give me the performance that I expect... Also, try monitoring diffferent types of transfers to and from another physical disk with iostat. Actually, interestingly enough, when I copy a file, or do a newfs_msdos I only get 0.06-0.89MB/s transfers, which is what first tipped me off to the problems... Obviously less acceptable... ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Poor SCSI disk preformance
In the last episode (Jan 06), Derek Marcotte said: Actually, just for kicks: # dd if=/dev/da0 of=/dev/null bs=128k [1] 1839 # iostat -K -w 1 da0 tty da0 cpu tin tout KB/t tps MB/s us ni sy in id 1 42 64.00 607 37.92 1 0 1 0 98 0 43 64.00 222 13.87 0 0 2 0 98 0 43 64.00 223 13.92 0 0 0 2 98 0 42 64.00 224 13.98 0 0 2 0 98 0 43 64.00 222 13.86 0 0 3 0 97 0 43 64.00 223 13.92 0 0 1 2 98 0 43 64.00 223 13.92 0 0 2 1 97 0 42 64.00 223 13.92 0 0 3 0 97 0 43 64.00 223 13.92 0 0 1 0 99 Seems to give me the performance that I expect... Aha. Check the WCE bit to see if your write cache is enabled on the disk: # camcontrol mode da0 -m 8 | grep WCE If it's not set, that could be contributing to the speed difference between reads and writes. Set it by running cmcontrol mode da0 -m 8 -e -P 2, and set WCE: 1. -- Dan Nelson [EMAIL PROTECTED] ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Poor SCSI disk preformance [more on camcontrol please!]
Aha. Check the WCE bit to see if your write cache is enabled on the disk Bingo: # dd if=/dev/zero of=/dev/da0 bs=64k [1] 2253 # iostat -K -w 1 da0 tty da0 cpu tin tout KB/t tps MB/s us ni sy in id 2 38 0.00 0 0.00 1 0 1 0 98 0 43 64.00 223 13.91 0 0 8 1 91 0 43 64.00 223 13.92 0 0 5 0 95 0 43 64.00 223 13.92 0 0 8 1 91 0 43 64.00 223 13.92 0 0 6 0 94 0 42 64.00 223 13.92 0 0 5 1 94 0 43 64.00 223 13.92 1 0 6 1 92 Set it by running cmcontrol mode da0 -m 8 -e -P 2, and set WCE: 1 I needed to modify your command slightly to: camcontrol mode da0 -m 8 -e -P 0 I guess I don't have a page 2 for some reason... This will probably cause this bit to be reset on reboot as well, because it is the current page? Is it prudent to attempt to set the WCE:1 on all drives that get attached? I will be formatting a large number of greatly varying drives, including ATA converted to SCSI type drives, and really old, and really new drive types. I've had a look at man camcontrol earlier, but I don't know enough about the inner workings of SCSI for this to mean much to me. It seems to be pretty obscure (like how would I know to enable features/specs to edit a modepage?), but extremely powerful. Where can I read more about this, is there a good camcontrol FAQ/tutorial out there that explains what these details actually mean/do? Thanks for the help! Derek ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Poor SCSI disk preformance [more on camcontrol please!]
In the last episode (Jan 06), Derek Marcotte said: Aha. Check the WCE bit to see if your write cache is enabled on the disk Bingo: # dd if=/dev/zero of=/dev/da0 bs=64k # iostat -K -w 1 da0 tty da0 cpu tin tout KB/t tps MB/s us ni sy in id 0 43 64.00 223 13.92 1 0 6 1 92 I guess I don't have a page 2 for some reason... This will probably cause this bit to be reset on reboot as well, because it is the current page? Possibly. Power the drive off and see if the change sticks. :) Is it prudent to attempt to set the WCE:1 on all drives that get attached? I will be formatting a large number of greatly varying drives, including ATA converted to SCSI type drives, and really old, and really new drive types. I've never seen WCE hurt sequential write access, so it's probably safe to turn on. If you're paranoid about possibly getting damaged filesystems during power outages, you might want to turn it back off, although every year or so there's a thread that pops up debating its merits. I've had a look at man camcontrol earlier, but I don't know enough about the inner workings of SCSI for this to mean much to me. It seems to be pretty obscure (like how would I know to enable features/specs to edit a modepage?), but extremely powerful. Where can I read more about this, is there a good camcontrol FAQ/tutorial out there that explains what these details actually mean/do? Everything under camcontrol modepage and cmd is pretty much straight from the SCSI spec. You can buy copies of it from ANSI (I think you can download draft copies from www.t10.org somewhere), and sometimes disk vendors will ship copies with vendor-specific info. About 15 years ago, I bought a Maxtor disk that didn't include the little sheet saying which jumpers were which SCSI id. I called up and asked them to send me a copy, and they sent me the whole reference manual for the drive, detailing every SCSI command and modepage. -- Dan Nelson [EMAIL PROTECTED] ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
