As I continue to work on my previous issue with my Sun V120 and network hangs, I decided to install 4.6 release onto an HP DL360 G4 box with the latest BIOS and firmware updates as a possible replacement for the Sun. After many hours of load testing and changing configurations, I found that I always get input errors on the network interfaces when running the multiprocessor kernel. But I get no errors at all with the uniprocessor kernel.
I can reproduce this problem with the internal bge (BCM5704C) and also with a PCI-X Intel Pro/1000 MT (82546GB) card. All I need to do is bring up the system using the MP kernel and push traffic through it. I'm using a simple wget on an internal machine to repeatedly pull a large file from a webserver on the external LAN. Within an hour I easily have over 1000 input errors. With the uniprocessor kernel, I sustained 90Mbps through the firewall for 8 hours straight with 0 errors. I'm running separate 100Mbps switches for internal and external LANs. I don't see any ifq.drops in either case. I'm thinking this is not a hardware issue since it works fine in one case but not in the other, without changing any hardware or cables. I understand that the interrupt handling is different in the MP kernel, so could that be where this issue is originating? It would be great to have both CPUs available as I plan to run some other things (aside from pf) on this box but I can settle for one CPU if that is the only solution. I tried disabling hyperthreading but that did not affect the issue. Here's the relevant netstat -i output for my 1-hour load test with em interfaces and the MP kernel: em0 1500 <Link> 00:04:23:a6:b4:a6 24029262 710 12738132 0 0 em1 1500 <Link> 00:04:23:a6:b4:a7 12753283 1009 24038738 0 0 After switching to the SP kernel: em0 1500 <Link> 00:04:23:a6:b4:a6 16393437 0 14391074 0 0 em1 1500 <Link> 00:04:23:a6:b4:a7 14431184 0 16445995 0 0 Searching the lists, I only found one reference to something like this but it was on 4.0 and I didn't see a resolution. Has anyone else seen this behavior? http://www.mail-archive.com/misc@openbsd.org/msg31490.html As a next step, I'm planning to install the latest snapshot to see if the issue still exists. In the meantime, here is the dmesg from the system. The kernel is #0 because I installed patches 002_xmm.patch and 003_getsockopt.patch. OpenBSD 4.6 (GENERIC.MP) #0: Mon Nov 2 11:43:12 EST 2009 lea...@fw1.bitbytes.com:/usr/src/sys/arch/i386/compile/GENERIC.MP cpu0: Intel(R) Xeon(TM) CPU 3.40GHz ("GenuineIntel" 686-class) 3.41 GHz cpu0: FPU,V86,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,SSE3,MWAIT,DS-CPL,EST,CNXT-ID,CX16,xTPR real mem = 3757613056 (3583MB) avail mem = 3648847872 (3479MB) mainbus0 at root bios0 at mainbus0: AT/286+ BIOS, date 12/31/99, BIOS32 rev. 0 @ 0xf0000, SMBIOS rev. 2.3 @ 0xec000 (56 entries) bios0: vendor HP version "P52" date 07/16/2007 bios0: HP ProLiant DL360 G4 acpi0 at bios0: rev 2 acpi0: tables DSDT FACP SPCR MCFG APIC SSDT acpi0: wakeup devices acpitimer0 at acpi0: 3579545 Hz, 24 bits acpimadt0 at acpi0 addr 0xfee00000: PC-AT compat cpu0 at mainbus0: apid 0 (boot processor) cpu0: apic clock running at 200MHz cpu1 at mainbus0: apid 6 (application processor) cpu1: Intel(R) Xeon(TM) CPU 3.40GHz ("GenuineIntel" 686-class) 3.41 GHz cpu1: FPU,V86,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,SSE3,MWAIT,DS-CPL,EST,CNXT-ID,CX16,xTPR ioapic0 at mainbus0: apid 8 pa 0xfec00000, version 20, 24 pins ioapic1 at mainbus0: apid 9 pa 0xfec10000, version 20, 24 pins ioapic1: misconfigured as apic 0, remapped to apid 9 ioapic2 at mainbus0: apid 10 pa 0xfec82000, version 20, 24 pins ioapic3 at mainbus0: apid 11 pa 0xfec82400, version 20, 24 pins acpiprt0 at acpi0: bus 1 (IP2P) acpiprt1 at acpi0: bus 2 (ICHR) acpiprt2 at acpi0: bus 7 (PCXA) acpiprt3 at acpi0: bus 10 (PCXB) acpiprt4 at acpi0: bus 6 (PTB0) acpiprt5 at acpi0: bus 13 (PTA0) acpiprt6 at acpi0: bus 3 (PTC0) acpiprt7 at acpi0: bus 0 (PCI0) acpicpu0 at acpi0: FVS, 3400, 2800 MHz acpicpu1 at acpi0: FVS, 3400, 2800 MHz acpitz0 at acpi0: critical temperature 31 degC bios0: ROM list: 0xc0000/0x8000 0xc8000/0x4000! 0xcc000/0x1600 0xee000/0x2000! pci0 at mainbus0 bus 0: configuration mode 1 (bios) pchb0 at pci0 dev 0 function 0 "Intel E7520 Host" rev 0x0c ppb0 at pci0 dev 2 function 0 "Intel E7520 PCIE" rev 0x0c pci1 at ppb0 bus 13 ppb1 at pci0 dev 4 function 0 "Intel E7520 PCIE" rev 0x0c pci2 at ppb1 bus 6 ppb2 at pci2 dev 0 function 0 "Intel PCIE-PCIE" rev 0x09 pci3 at ppb2 bus 7 em0 at pci3 dev 1 function 0 "Intel PRO/1000MT (82546GB)" rev 0x03: apic 10 int 0 (irq 5), address 00:04:23:a6:b4:a6 em1 at pci3 dev 1 function 1 "Intel PRO/1000MT (82546GB)" rev 0x03: apic 10 int 1 (irq 5), address 00:04:23:a6:b4:a7 ppb3 at pci2 dev 0 function 2 "Intel PCIE-PCIE" rev 0x09 pci4 at ppb3 bus 10 ppb4 at pci0 dev 6 function 0 "Intel E7520 PCIE" rev 0x0c pci5 at ppb4 bus 3 ppb5 at pci0 dev 28 function 0 "Intel 6300ESB PCIX" rev 0x02 pci6 at ppb5 bus 2 ciss0 at pci6 dev 1 function 0 "Compaq Smart Array 64xx" rev 0x01: apic 9 int 0 (irq 5) ciss0: 1 LD, HW rev 1, FW 2.84/2.84, 64bit fifo scsibus0 at ciss0: 1 targets sd0 at scsibus0 targ 0 lun 0: <HP, LOGICAL VOLUME, 2.84> SCSI2 0/direct fixed sd0: 69459MB, 512 bytes/sec, 142253280 sec total bge0 at pci6 dev 2 function 0 "Broadcom BCM5704C" rev 0x10, BCM5704 B0 (0x2100): apic 9 int 1 (irq 5), address 00:14:38:4c:b5:de brgphy0 at bge0 phy 1: BCM5704 10/100/1000baseT PHY, rev. 0 bge1 at pci6 dev 2 function 1 "Broadcom BCM5704C" rev 0x10, BCM5704 B0 (0x2100): apic 9 int 2 (irq 5), address 00:14:38:4c:b5:dd brgphy1 at bge1 phy 1: BCM5704 10/100/1000baseT PHY, rev. 0 uhci0 at pci0 dev 29 function 0 "Intel 6300ESB USB" rev 0x02: apic 8 int 16 (irq 5) uhci1 at pci0 dev 29 function 1 "Intel 6300ESB USB" rev 0x02: apic 8 int 19 (irq 5) "Intel 6300ESB WDT" rev 0x02 at pci0 dev 29 function 4 not configured "Intel 6300ESB APIC" rev 0x02 at pci0 dev 29 function 5 not configured ehci0 at pci0 dev 29 function 7 "Intel 6300ESB USB" rev 0x02: apic 8 int 23 (irq 5) usb0 at ehci0: USB revision 2.0 uhub0 at usb0 "Intel EHCI root hub" rev 2.00/1.00 addr 1 ppb6 at pci0 dev 30 function 0 "Intel 82801BA Hub-to-PCI" rev 0x0a pci7 at ppb6 bus 1 vga1 at pci7 dev 3 function 0 "ATI Rage XL" rev 0x27 wsdisplay0 at vga1 mux 1: console (80x25, vt100 emulation) wsdisplay0: screen 1-5 added (80x25, vt100 emulation) "Compaq iLO" rev 0x01 at pci7 dev 4 function 0 not configured "Compaq iLO" rev 0x01 at pci7 dev 4 function 2 not configured ichpcib0 at pci0 dev 31 function 0 "Intel 6300ESB LPC" rev 0x02 pciide0 at pci0 dev 31 function 1 "Intel 6300ESB IDE" rev 0x02: DMA, channel 0 configured to compatibility, channel 1 configured to compatibility atapiscsi0 at pciide0 channel 0 drive 0 scsibus1 at atapiscsi0: 2 targets cd0 at scsibus1 targ 0 lun 0: <COMPAQ, CD-ROM SN-124, N104> ATAPI 5/cdrom removable cd0(pciide0:0:0): using PIO mode 4 pciide0: channel 1 disabled (no drives) usb1 at uhci0: USB revision 1.0 uhub1 at usb1 "Intel UHCI root hub" rev 1.00/1.00 addr 1 usb2 at uhci1: USB revision 1.0 uhub2 at usb2 "Intel UHCI root hub" rev 1.00/1.00 addr 1 isa0 at ichpcib0 isadma0 at isa0 com0 at isa0 port 0x3f8/8 irq 4: ns16550a, 16 byte fifo com0: console com1 at isa0 port 0x2f8/8 irq 3: ns16550a, 16 byte fifo com1: probed fifo depth: 0 bytes pckbc0 at isa0 port 0x60/5 pckbd0 at pckbc0 (kbd slot) pckbc0: using irq 1 for kbd slot wskbd0 at pckbd0: console keyboard, using wsdisplay0 pms0 at pckbc0 (aux slot) pckbc0: using irq 12 for aux slot wsmouse0 at pms0 mux 0 pcppi0 at isa0 port 0x61 midi0 at pcppi0: <PC speaker> spkr0 at pcppi0 npx0 at isa0 port 0xf0/16: reported by CPUID; using exception 16 fdc0 at isa0 port 0x3f0/6 irq 6 drq 2 fd0 at fdc0 drive 0: 1.44MB 80 cyl, 2 head, 18 sec mtrr: Pentium Pro MTRR support uhidev0 at uhub2 port 1 configuration 1 interface 0 "Darfon USB Combo Keyboard" rev 1.10/3.02 addr 2 uhidev0: iclass 3/1 ukbd0 at uhidev0: 8 modifier keys, 6 key codes wskbd1 at ukbd0 mux 1 wskbd1: connecting to wsdisplay0 uhidev1 at uhub2 port 1 configuration 1 interface 1 "Darfon USB Combo Keyboard" rev 1.10/3.02 addr 2 uhidev1: iclass 3/0, 2 report ids uhid0 at uhidev1 reportid 1: input=1, output=0, feature=0 uhid1 at uhidev1 reportid 2: input=4, output=0, feature=0 softraid0 at root root on sd0a swap on sd0b dump on sd0b Thanks, Bryan
