high interrupt level

2009-01-02 Thread Jeroen Hofstee

Hello,

A Dell Poweregde 1850 constantly spents about one third of its time in 
interrupt 16. This box has run for a year without this issue.
It has an almost generic kernel, only quotas are compiled in. Rebooting 
(without ipmi) does not help nor updating; the issue was already present
before updating to 6.4 p1, but persists. The software hardly changes, 
except that ipmitool is recently installed to monitor temperatures.


I tried to replicate the issue by loading / unloading the ipmi kernel 
module on a similiar box, but running 7.0-RELEASE-p7 .
This does not lead to any problem however. Any pointers would be 
appreciated. It might not at all be related to ipmi, but I suspect it

to be cause since it is the only change and I loaded a kernel module for it.

Jeroen Hofstee



last pid: 12675;  load averages:  1.27,  0.75,  
0.65  
up 6+05:56:47  23:50:57

85 processes:  1 running, 84 sleeping
CPU:  0.3% user,  0.0% nice,  0.3% system, 33.8% interrupt, 65.5% idle
Mem: 174M Active, 1514M Inact, 209M Wired, 88M Cache, 112M Buf, 14M Free
Swap: 2048M Total, 28K Used, 2048M Free

FreeBSD vh3.virtualhost.nl 6.4-RELEASE-p1 FreeBSD 6.4-RELEASE-p1 #2: Tue 
Dec 23 23:50:15 CET 2008


vmstat -i
interrupt  total   rate
irq14: ata0   47  0
irq16: uhci0   313487360578
irq18: uhci2  16  0
irq23: ehci0   3  0
irq34: mpt0   16  0
irq37: amr0  8894886 16
irq64: em0  54502901100
cpu0: timer904348488   1669
cpu1: timer1  0
cpu2: timer   1058243496   1953
cpu3: timer1  0
Total 2339477215   4317

vh3# kldstat
Id Refs AddressSize Name
17 0xc040 7dbd28   kernel
22 0xc0bdc000 5c838acpi.ko
31 0xc6d1 d000 ipfw.ko
41 0xc6f0b000 2000 accf_http.ko
51 0xcc472000 a000 ipmi.ko
61 0xcc47c000 2000 smbus.ko


Copyright (c) 1992-2008 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 6.4-RELEASE-p1 #2: Tue Dec 23 23:50:15 CET 2008
  r...@vh3.virtualhost.nl:/usr/obj/usr/src/sys/VIRTUALHOST
Timecounter "i8254" frequency 1193182 Hz quality 0
CPU: Intel(R) Xeon(TM) CPU 3.00GHz (2992.70-MHz 686-class CPU)
Origin = "GenuineIntel"  Id = 0xf43  Stepping = 3
Features=0xbfebfbff
Features2=0x641d
AMD Features=0x2010
Logical CPUs per core: 2
real memory  = 2147221504 (2047 MB)
avail memory = 2091900928 (1994 MB)
ACPI APIC Table: 
FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs
cpu0 (BSP): APIC ID:  0
cpu1 (AP): APIC ID:  1
cpu2 (AP): APIC ID:  6
cpu3 (AP): APIC ID:  7
ioapic0: Changing APIC ID to 8
ioapic1: Changing APIC ID to 9
ioapic2: Changing APIC ID to 10
ioapic0  irqs 0-23 on motherboard
ioapic1  irqs 32-55 on motherboard
ioapic2  irqs 64-87 on motherboard
kbd1 at kbdmux0
ath_hal: 0.9.20.3 (AR5210, AR5211, AR5212, RF5111, RF5112, RF2413, RF5413)
hptrr: HPT RocketRAID controller driver v1.1 (Dec 23 2008 23:49:48)
acpi0:  on motherboard
acpi0: Power Button (fixed)
Timecounter "ACPI-fast" frequency 3579545 Hz quality 1000
acpi_timer0: <24-bit timer at 3.579545MHz> port 0x808-0x80b on acpi0
acpi_hpet0:  iomem 0xfed0-0xfed003ff on acpi0
Timecounter "HPET" frequency 14318180 Hz quality 900
cpu0:  on acpi0
cpu1:  on acpi0
cpu2:  on acpi0
cpu3:  on acpi0
pcib0:  port 0xcf8-0xcff on acpi0
pci0:  on pcib0
pcib1:  at device 2.0 on pci0
pci1:  on pcib1
pcib2:  at device 0.0 on pci1
pci2:  on pcib2
mpt0:  port 0xec00-0xecff mem 
0xdfdf-0xdfdf,0xdfde-0xdfde irq 34 at device 5.0 on pci2
mpt0: [GIANT-LOCKED]
mpt0: MPI Version=1.2.12.0
pcib3:  at device 0.2 on pci1
pci3:  on pcib3
amr0:  mem 0xd8ff-0xd8ff irq 37 at device 11.0 
on pci3
amr0: delete logical drives supported by controller
amr0:  Firmware 350O, BIOS 1.09, 64MB RAM
pcib4:  at device 4.0 on pci0
pci4:  on pcib4
pcib5:  at device 5.0 on pci0
pci5:  on pcib5
pcib6:  at device 0.0 on pci5
pci6:  on pcib6
em0:  port 0xdcc0-0xdcff 
mem 0xdf9e-0xdf9f irq 64 at device 7.0 on pci6
em0: Ethernet address: 00:11:43:e6:e9:e8
pcib7:  at device 0.2 on pci5
pci7:  on pcib7
em1:  port 0xccc0-0xccff 
mem 0xdf7e-0xdf7f irq 65 at device 8.0 on pci7
em1: Ethernet address: 00:11:43:e6:e9:e9
pcib8:  at device 6.0 on pci0
pci8:  on pcib8
uhci0:  port 0xace0-0xacff irq 16 at 
device 29.0 on pci0
uhci0: [GIANT-LOCKED]
usb0:  on uhci0
usb0: USB revision 1.0
uhub0: Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1
uhub0: 2 ports with 2 removable, self powered
uhci1:  port 0xacc

high interrupt level

2009-01-02 Thread Jeroen Hofstee

Hello,

A Dell Poweregde 1850 constantly spents about one third of its time in 
interrupt 16. This box has run for a year without this issue.
It has an almost generic kernel, only quotas are compiled in. Rebooting 
(without ipmi) does not help nor updating; the issue was already present
before updating to 6.4 p1, but persists. The software hardly changes, 
except that ipmitool is recently installed to monitor temperatures.


I tried to replicate the issue by loading / unloading the ipmi kernel 
module on a similiar box, but running 7.0-RELEASE-p7 .
This does not lead to any problem however. Any pointers would be 
appreciated. It might not at all be related to ipmi, but I suspect it

to be cause since it is the only change and I loaded a kernel module for it.

Jeroen Hofstee



last pid: 12675;  load averages:  1.27,  0.75,  
0.65  
up 6+05:56:47  23:50:57

85 processes:  1 running, 84 sleeping
CPU:  0.3% user,  0.0% nice,  0.3% system, 33.8% interrupt, 65.5% idle
Mem: 174M Active, 1514M Inact, 209M Wired, 88M Cache, 112M Buf, 14M Free
Swap: 2048M Total, 28K Used, 2048M Free

FreeBSD vh3.virtualhost.nl 6.4-RELEASE-p1 FreeBSD 6.4-RELEASE-p1 #2: Tue 
Dec 23 23:50:15 CET 2008


vmstat -i
interrupt  total   rate
irq14: ata0   47  0
irq16: uhci0   313487360578
irq18: uhci2  16  0
irq23: ehci0   3  0
irq34: mpt0   16  0
irq37: amr0  8894886 16
irq64: em0  54502901100
cpu0: timer904348488   1669
cpu1: timer1  0
cpu2: timer   1058243496   1953
cpu3: timer1  0
Total 2339477215   4317

vh3# kldstat
Id Refs AddressSize Name
17 0xc040 7dbd28   kernel
22 0xc0bdc000 5c838acpi.ko
31 0xc6d1 d000 ipfw.ko
41 0xc6f0b000 2000 accf_http.ko
51 0xcc472000 a000 ipmi.ko
61 0xcc47c000 2000 smbus.ko


Copyright (c) 1992-2008 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 6.4-RELEASE-p1 #2: Tue Dec 23 23:50:15 CET 2008
   r...@vh3.virtualhost.nl:/usr/obj/usr/src/sys/VIRTUALHOST
Timecounter "i8254" frequency 1193182 Hz quality 0
CPU: Intel(R) Xeon(TM) CPU 3.00GHz (2992.70-MHz 686-class CPU)
 Origin = "GenuineIntel"  Id = 0xf43  Stepping = 3
 
Features=0xbfebfbff
 Features2=0x641d
 AMD Features=0x2010
 Logical CPUs per core: 2
real memory  = 2147221504 (2047 MB)
avail memory = 2091900928 (1994 MB)
ACPI APIC Table: 
FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs
cpu0 (BSP): APIC ID:  0
cpu1 (AP): APIC ID:  1
cpu2 (AP): APIC ID:  6
cpu3 (AP): APIC ID:  7
ioapic0: Changing APIC ID to 8
ioapic1: Changing APIC ID to 9
ioapic2: Changing APIC ID to 10
ioapic0  irqs 0-23 on motherboard
ioapic1  irqs 32-55 on motherboard
ioapic2  irqs 64-87 on motherboard
kbd1 at kbdmux0
ath_hal: 0.9.20.3 (AR5210, AR5211, AR5212, RF5111, RF5112, RF2413, RF5413)
hptrr: HPT RocketRAID controller driver v1.1 (Dec 23 2008 23:49:48)
acpi0:  on motherboard
acpi0: Power Button (fixed)
Timecounter "ACPI-fast" frequency 3579545 Hz quality 1000
acpi_timer0: <24-bit timer at 3.579545MHz> port 0x808-0x80b on acpi0
acpi_hpet0:  iomem 0xfed0-0xfed003ff on acpi0
Timecounter "HPET" frequency 14318180 Hz quality 900
cpu0:  on acpi0
cpu1:  on acpi0
cpu2:  on acpi0
cpu3:  on acpi0
pcib0:  port 0xcf8-0xcff on acpi0
pci0:  on pcib0
pcib1:  at device 2.0 on pci0
pci1:  on pcib1
pcib2:  at device 0.0 on pci1
pci2:  on pcib2
mpt0:  port 0xec00-0xecff mem 
0xdfdf-0xdfdf,0xdfde-0xdfde irq 34 at device 5.0 on pci2
mpt0: [GIANT-LOCKED]
mpt0: MPI Version=1.2.12.0
pcib3:  at device 0.2 on pci1
pci3:  on pcib3
amr0:  mem 0xd8ff-0xd8ff irq 37 at device 11.0 
on pci3
amr0: delete logical drives supported by controller
amr0:  Firmware 350O, BIOS 1.09, 64MB RAM
pcib4:  at device 4.0 on pci0
pci4:  on pcib4
pcib5:  at device 5.0 on pci0
pci5:  on pcib5
pcib6:  at device 0.0 on pci5
pci6:  on pcib6
em0:  port 0xdcc0-0xdcff 
mem 0xdf9e-0xdf9f irq 64 at device 7.0 on pci6
em0: Ethernet address: 00:11:43:e6:e9:e8
pcib7:  at device 0.2 on pci5
pci7:  on pcib7
em1:  port 0xccc0-0xccff 
mem 0xdf7e-0xdf7f irq 65 at device 8.0 on pci7
em1: Ethernet address: 00:11:43:e6:e9:e9
pcib8:  at device 6.0 on pci0
pci8:  on pcib8
uhci0:  port 0xace0-0xacff irq 16 at 
device 29.0 on pci0
uhci0: [GIANT-LOCKED]
usb0:  on uhci0
usb0: USB revision 1.0
uhub0: Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1
uhub0: 2 ports with 2 removable, self powered
uhci1:  por