Hi,
I'm (still) trying to get my new-to-me server on-line.
Basic specs (dmesg below) are dual XEONs at 2.4GHz, 4GB of ram, LSI MegaRAID
320-1LP controller card with 64MB of ram on board and the BBU installed and
working, 6 15K SCSI U320 72GB drives configured as 3 RAID1 volumes, all
partitions mounted with softdep.
Disk performance is really slow, especially if I have cache writeback enabled.
For example, when /etc/daily does the dd to altroot , I get this with
writeback:
Backing up root=/dev/rsd0a to /dev/rsd2e:
64255+1 records in
64255+1 records out
526377472 bytes transferred in 125.916 secs (4180379 bytes/sec)
When writethru is enabled, I get this:
Backing up root=/dev/rsd0a to /dev/rsd2e:
64255+1 records in
64255+1 records out
526377472 bytes transferred in 76.841 secs (6850130 bytes/sec)
Better, but holy cow, the server its supposed to be an upgrade for does better:
Backing up root=/dev/rsd0a to /dev/rsd1k:
64010+1 records in
64010+1 records out
524370432 bytes transferred in 28.017 secs (18715708 bytes/sec)
Those are 10K drives on the stock Adaptec U160 AIC-7899.
I've replaced the cable that connects the LSI MegaRAID controller to the
enclosure but that didn't change anything.
I've left most of the rest of the settings on the raid card on the default.
This server (if it ever gets on-line ;-) is supposed to be a postgresql server
so writecache should help alot. Right now pgbench tops out at just under 4000
TPS with the basic SELECT tests, and that's not very impressive either.
Anything I can test or tweak or is what I thought to be a pretty nice bit of
kit really not so much.
Here's the dmesg:
OpenBSD 4.6-current (GENERIC.MP) #381: Tue Jan 5 13:43:29 MST 2010
[email protected]:/usr/src/sys/arch/i386/compile/GENERIC.MP
cpu0: Intel(R) Xeon(TM) CPU 2.40GHz ("GenuineIntel" 686-class) 2.41 GHz
cpu0: FPU,V86,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CF
LUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,CNXT-ID,xTPR
real mem = 4160253952 (3967MB)
avail mem = 4053819392 (3866MB)
mainbus0 at root
bios0 at mainbus0: AT/286+ BIOS, date 09/18/03, BIOS32 rev. 0 @ 0xf0010, SMBIOS
rev. 2.3 @ 0xf84b0 (39 entries)
bios0: vendor American Megatrends Inc. version "080009" date 09/18/2003
bios0: Supermicro X5DPA
acpi0 at bios0: rev 2
acpi0: tables DSDT FACP APIC OEMB
acpi0: wakeup devices PS2K(S1) PS2M(S1) SMBS(S1) AUDI(S1) MODM(S1) USB0(S1) USB1
(S1) USB2(S1) P0P1(S1) GLN1(S1) GLN2(S1) BT64(S1) PWRB(S1) SLPB(S1)
acpitimer0 at acpi0: 3579545 Hz, 24 bits
acpimadt0 at acpi0 addr 0xfee00000: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: apic clock running at 133MHz
cpu1 at mainbus0: apid 6 (application processor)
cpu1: Intel(R) Xeon(TM) CPU 2.40GHz ("GenuineIntel" 686-class) 2.41 GHz
cpu1: FPU,V86,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CF
LUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,CNXT-ID,xTPR
cpu2 at mainbus0: apid 1 (application processor)
cpu2: Intel(R) Xeon(TM) CPU 2.40GHz ("GenuineIntel" 686-class) 2.41 GHz
cpu2: FPU,V86,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CF
LUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,CNXT-ID,xTPR
cpu3 at mainbus0: apid 7 (application processor)
cpu3: Intel(R) Xeon(TM) CPU 2.40GHz ("GenuineIntel" 686-class) 2.41 GHz
cpu3: FPU,V86,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CF
LUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,CNXT-ID,xTPR
ioapic0 at mainbus0: apid 8 pa 0xfec00000, version 20, 24 pins
ioapic1 at mainbus0: apid 9 pa 0xfec80000, version 20, 24 pins
ioapic2 at mainbus0: apid 10 pa 0xfec80400, version 20, 24 pins
acpiprt0 at acpi0: bus 0 (PCI0)
acpiprt1 at acpi0: bus 1 (P0P1)
acpiprt2 at acpi0: bus 3 (P2P3)
acpiprt3 at acpi0: bus 4 (BT64)
acpicpu0 at acpi0
acpicpu1 at acpi0
acpicpu2 at acpi0
acpicpu3 at acpi0
acpibtn0 at acpi0: PWRB
acpibtn1 at acpi0: SLPB
bios0: ROM list: 0xc0000/0x8000 0xc8000/0x1000 0xc9000/0x1000 0xca000/0x2600
pci0 at mainbus0 bus 0: configuration mode 1 (bios)
pchb0 at pci0 dev 0 function 0 "Intel E7501 Host" rev 0x01
ppb0 at pci0 dev 2 function 0 "Intel E7500 PCI" rev 0x01
pci1 at ppb0 bus 2
"Intel 82870P2 IOxAPIC" rev 0x04 at pci1 dev 28 function 0 not configured
ppb1 at pci1 dev 29 function 0 "Intel 82870P2 PCIX-PCIX" rev 0x04
pci2 at ppb1 bus 4
ami0 at pci2 dev 2 function 0 "Symbios Logic MegaRAID" rev 0x01: apic 10 int 4 (
irq 10)
ami0: LSI 520, 64b/lhc, FW 1L47, BIOS vG121, 64MB RAM
ami0: 1 channels, 0 FC loops, 3 logical drives
scsibus0 at ami0: 40 targets
sd0 at scsibus0 targ 0 lun 0: <AMI, Host drive #00, > SCSI2 0/direct fixed
sd0: 70135MB, 512 bytes/sec, 143636480 sec total
sd1 at scsibus0 targ 1 lun 0: <AMI, Host drive #01, > SCSI2 0/direct fixed
sd1: 70135MB, 512 bytes/sec, 143636480 sec total
sd2 at scsibus0 targ 2 lun 0: <AMI, Host drive #02, > SCSI2 0/direct fixed
sd2: 70135MB, 512 bytes/sec, 143636480 sec total
scsibus1 at ami0: 16 targets
ahd0 at pci2 dev 3 function 0 "Adaptec AIC-7902B U320" rev 0x10: apic 10 int 2 (
irq 10)
ahd0: aic7902, U320 Wide Channel A, PCI 33 or 66MHz, 512 SCBs
scsibus2 at ahd0: 16 targets, initiator 7
ahd1 at pci2 dev 3 function 1 "Adaptec AIC-7902B U320" rev 0x10: apic 10 int 3 (
irq 10)
ahd1: aic7902, U320 Wide Channel B, PCI 33 or 66MHz, 512 SCBs
scsibus3 at ahd1: 16 targets, initiator 7
"Intel 82870P2 IOxAPIC" rev 0x04 at pci1 dev 30 function 0 not configured
ppb2 at pci1 dev 31 function 0 "Intel 82870P2 PCIX-PCIX" rev 0x04
pci3 at ppb2 bus 3
em0 at pci3 dev 1 function 0 "Intel PRO/1000MT (82541GI)" rev 0x00: apic 9 int 4
(irq 10), address 00:30:48:76:07:dc
em1 at pci3 dev 2 function 0 "Intel PRO/1000MT (82541GI)" rev 0x00: apic 9 int 5
(irq 10), address 00:30:48:76:07:dd
uhci0 at pci0 dev 29 function 0 "Intel 82801CA/CAM USB" rev 0x02: apic 8 int 16
(irq 11)
uhci1 at pci0 dev 29 function 1 "Intel 82801CA/CAM USB" rev 0x02: apic 8 int 19
(irq 5)
uhci2 at pci0 dev 29 function 2 "Intel 82801CA/CAM USB" rev 0x02: apic 8 int 18
(irq 9)
ppb3 at pci0 dev 30 function 0 "Intel 82801BA Hub-to-PCI" rev 0x42
pci4 at vga1 at pci4 dev 4 function 0 "ATI Rage XL" rev 0x27
wsdisplay0 at vga1 mux 1: console (80x25, vt100 emulation)
wsdisplay0: screen 1-5 added (80x25, vt100 emulation)
ichpcib0 at pci0 dev 31 function 0 "Intel 82801CA LPC" rev 0x02
pciide0 at pci0 dev 31 function 1 "Intel 82801CA IDE" rev 0x02: DMA, channel 0
configured to compatibility, channel 1 configured to compatibility
atapiscsi0 at pciide0 channel 0 drive 1
scsibus4 at atapiscsi0: 2 targets
cd0 at scsibus4 targ 0 lun 0: <MATSHITA, DVD-ROM SR-8178, PZ16> ATAPI 5/cdrom
removable
cd0(pciide0:0:1): using PIO mode 4, Ultra-DMA mode 2
pciide0: channel 1 disabled (no drives)
ichiic0 at pci0 dev 31 function 3 "Intel 82801CA/CAM SMBus" rev 0x02: apic 8 int
17 (irq 0)
iic0 at ichiic0
admtemp0 at iic0 addr 0x18: Xeon
admtemp1 at iic0 addr 0x1a: Xeon
lm1 at iic0 addr 0x2d: W83627HF
spdmem0 at iic0 addr 0x51: 1GB DDR SDRAM registered ECC PC2300CL2.5
spdmem1 at iic0 addr 0x52: 1GB DDR SDRAM registered ECC PC2300CL2.5
spdmem2 at iic0 addr 0x55: 1GB DDR SDRAM registered ECC PC2300CL2.5
spdmem3 at iic0 addr 0x56: 1GB DDR SDRAM registered ECC PC2300CL2.5
usb0 at uhci0: USB revision 1.0
uhub0 at usb0 "Intel UHCI root hub" rev 1.00/1.00 addr 1
usb1 at uhci1: USB revision 1.0
uhub1 at usb1 "Intel UHCI root hub" rev 1.00/1.00 addr 1
usb2 at uhci2: USB revision 1.0
uhub2 at usb2 "Intel UHCI root hub" rev 1.00/1.00 addr 1
isa0 at ichpcib0
isadma0 at isa0
com0 at isa0 port 0x3f8/8 irq 4: ns16550a, 16 byte fifo
com0: console
com1 at isa0 port 0x2f8/8 irq 3: ns16550a, 16 byte fifo
pckbc0 at isa0 port 0x60/5
pmsi0 at pckbc0 (aux slot)
pckbc0: using irq 12 for aux slot
wsmouse0 at pmsi0 mux 0
pcppi0 at isa0 port 0x61
midi0 at pcppi0: <PC speaker>
spkr0 at pcppi0
lpt0 at isa0 port 0x378/4 irq 7
wbsio0 at isa0 port 0x2e/2: W83627HF rev 0x17
lm2 at wbsio0 port 0x290/8: W83627HF
lm1 detached
ppb3 bus 1
npx0 at isa0 port 0xf0/16: reported by CPUID; using exception 16
fdc0 at isa0 port 0x3f0/6 irq 6 drq 2
fd0 at fdc0 drive 0: 1.44MB 80 cyl, 2 head, 18 sec
mtrr: Pentium Pro MTRR support
vscsi0 at root
scsibus5 at vscsi0: 256 targets
softraid0 at root
root on sd0a swap on sd0b dump on sd0b
uvm_mapent_alloc: out of static map entries
Huh...as I pasted this in I found the last line. Googling shows a thread from
2006, then a short thread from Sept 2009. I am having a hard time getting
more work mem to postgresql (pgbench croaks if it requests more than 64MB,
even though I have postgresql in its own login class as suggested by the
package readme) but I thought that's another issue.
Last bit of sysctl.conf where I've tweaked some stuff:
# For PostgreSQL Port
kern.seminfo.semmni=4096
kern.seminfo.semmns=9082
kern.shminfo.shmall=128000
kern.shminfo.shmmax=2048000000
Thanks, as always,
Jeff Ross
