Hello misc,
After having installed 4.0-current on two identically configured SUN
V210 (see dmesg below), I found that their performance was unusually
bad, notably with disk I/O.
top reveals a permanent interrupt load of between 30 to over 50% !?
# top
load averages: 0.09, 0.17, 0.08
19:52:16
13 processes: 12 idle, 1 on processor
CPU states: 1.9% user, 0.0% nice, 1.6% system, 39.0% interrupt, 57.5% idle
Memory: Real: 11M/122M act/tot Free: 880M Swap: 0K/487M used/tot
PID USERNAME PRI NICE SIZE RES STATE WAIT TIME CPU COMMAND
1643 root 2 0 832K 2360K idle select 0:02 0.00% sshd
30242 root 2 0 3544K 3504K sleep select 0:01 0.00% sshd
24667 root 2 0 480K 1072K idle select 0:00 0.00% inetd
2435 root 2 0 1592K 2136K sleep select 0:00 0.00% sendmail
20073 root 18 0 824K 616K sleep pause 0:00 0.00% ksh
24745 root 2 0 536K 1056K idle poll 0:00 0.00% ntpd
29783 root 2 0 664K 1184K sleep select 0:00 0.00% cron
25532 root 3 0 352K 1104K idle ttyin 0:00 0.00% getty
21605 _syslogd 2 0 544K 1000K idle poll 0:00 0.00% syslogd
1 root 10 0 536K 424K idle wait 0:00 0.00% init
24060 root 2 0 520K 976K idle netio 0:00 0.00% syslogd
3809 _ntp 2 0 408K 1056K idle poll 0:00 0.00% ntpd
24849 root 31 0 504K 1648K onproc - 0:00 0.00% top
vmstat hints that pciide0 generates interrupts with a very high rate,
although there is hardly anything running on the boxes:
# vmstat -i
interrupt total rate
bge0 514 3
com0 82 0
pciide0 134564203 879504
siop0 2158 14
siop1 1 0
clock 15327 100
Total 134582285 879622
# uptime
7:50PM up 3 mins, 1 user, load averages: 0.27, 0.21, 0.09
#
# iostat -w 1
tty cd0 sd0 cpu
tin tout KB/t t/s MB/s KB/t t/s MB/s us ni sy in id
0 29 0.00 0 0.00 15.09 1 0.01 0 0 0 36 64
0 171 0.00 0 0.00 0.00 0 0.00 0 0 0 38 62
0 57 0.00 0 0.00 0.00 0 0.00 0 0 0 33 67
0 57 0.00 0 0.00 0.00 0 0.00 0 0 0 38 62
0 57 0.00 0 0.00 0.00 0 0.00 0 0 0 28 72
0 57 0.00 0 0.00 0.00 0 0.00 0 0 0 34 66
0 57 0.00 0 0.00 0.00 0 0.00 0 0 0 27 73
0 57 0.00 0 0.00 0.00 0 0.00 0 0 0 40 60
0 57 0.00 0 0.00 0.00 0 0.00 0 0 0 31 69
0 57 0.00 0 0.00 0.00 0 0.00 0 0 0 36 64
0 57 0.00 0 0.00 0.00 0 0.00 0 0 0 29 71
0 57 0.00 0 0.00 0.00 0 0.00 0 0 0 31 69
0 57 0.00 0 0.00 0.00 0 0.00 0 0 0 34 66
0 57 0.00 0 0.00 0.00 0 0.00 0 0 0 37 63
0 57 0.00 0 0.00 0.00 0 0.00 0 0 0 32 68
Maybe I shoud mention that I was lazy and installed everything under /
into a single partition because it's only a lab setup.
Also, in both boxes, I installed a SK-9S91 PCI 1 Gbit/s fiber NIC, in
addition to the four on-board bge NICs.
Rebooting multiple times of both boxes did not cure the high interrupt
rate, and enabling softupdates on /dev/sd0a did not help either.
The third-lat line in the output of dmesg below is somewhat intrigueing:
"No counter-timer -- using %tick at 1336MHz as system clock. root on sd0a"
But according to other dmesg from the archives, this seems to be
common among sparc64 installs.
In the archives, I found postings regarding a similar problem on an
Ultra 5. Apparently, the only recommendation there was to enable
softupdates. I remember having observed similar performance problems
on my own Ultra 5 some months ago under OpenBSD 3.9, which I was
unable to resolve.
Shall I try a re-install with more partitions, respectively to add
more partitions for the usual mount points?
Anything to do from with OpenBoot in order to avoid interrupt
conflicts between pciide0 and the SK-9S91 PCI NIC, for example?
Is there anything else that I should try in order to silence that
interrupt source?
I am happy to rebuild the kernel after patching and to re-test.
Thanks for your attention and any suggestions,
Rolf
# disklabel sd0
# /dev/rsd0c:
type: SCSI
disk: SCSI disk
label: SUN72G cyl 14087
flags:
bytes/sector: 512
sectors/track: 424
tracks/cylinder: 24
sectors/cylinder: 10176
cylinders: 14087
total sectors: 143349312
rpm: 10025
interleave: 1
trackskew: 0
cylinderskew: 0
headswitch: 0 # microseconds
track-to-track seek: 0 # microseconds
drivedata: 0
16 partitions:
# size offset fstype [fsize bsize cpg]
a: 70000704 0 4.2BSD 2048 16384 16 # Cyl 0 - 6878
b: 997248 70000704 swap # Cyl 6879 - 6976
c: 143349312 0 unused 0 0 # Cyl 0 - 14086
#
# dmesg
console is /[EMAIL PROTECTED],600000/[EMAIL PROTECTED]/[EMAIL PROTECTED],3f8
Copyright (c) 1982, 1986, 1989, 1991, 1993
The Regents of the University of California. All rights reserved.
Copyright (c) 1995-2007 OpenBSD. All rights reserved. http://www.OpenBSD.org
OpenBSD 4.0-current (GENERIC) #1049: Fri Jan 19 18:36:23 MST 2007
[EMAIL PROTECTED]:/usr/src/sys/arch/sparc64/compile/GENERIC
total memory = 1073741824
avail memory = 969416704
using 6553 buffers containing 53682176 bytes of memory
bootpath: /[EMAIL PROTECTED],600000/[EMAIL PROTECTED],0/[EMAIL PROTECTED],0
mainbus0 (root): Sun Fire V210
cpu0 at mainbus0: SUNW,UltraSPARC-IIIi (rev 3.4) @ 1336 MHz, version 0 FPU
cpu0: physical 32K instruction (32 b/l), 64K data (32 b/l), 1024K
external (64 b/l)
memory-controller at mainbus0 not configured
schizo0 at mainbus0: "Tomatillo", version 4, ign 7c0, bus B 0 to 0
schizo0: dvma map c0000000-dfffffff, iotdb 4d16000-4d96000
pci0 at schizo0
bge0 at pci0 dev 2 function 0 "Broadcom BCM5704C" rev 0x00, BCM5704 B0
(0x2100): ivec 0x7c8, address 00:14:4f:64:0c:52
brgphy0 at bge0 phy 1: BCM5704 10/100/1000baseT PHY, rev. 0
bge1 at pci0 dev 2 function 1 "Broadcom BCM5704C" rev 0x00, BCM5704 B0
(0x2100): ivec 0x7c9, address 00:14:4f:64:0c:53
brgphy1 at bge1 phy 1: BCM5704 10/100/1000baseT PHY, rev. 0
schizo1 at mainbus0: "Tomatillo", version 4, ign 780, bus A 0 to 0
schizo1: dvma map c0000000-dfffffff, iotdb 547e000-54fe000
pci1 at schizo1
ebus0 at pci1 dev 7 function 0 "Acer Labs M1533 ISA" rev 0x00
"flashprom" at ebus0 addr 0-fffff, 290-290 not configured
rtc0 at ebus0 addr 70-71: m5819p
pcfiic0 at ebus0 addr 320-321 ipl 46
iic0 at pcfiic0
"SUNW,i2c-imax" at iic0 addr 0xb not configured
"SUNW,i2c-imax" at iic0 addr 0xc not configured
"at24c64" at iic0 addr 0x51 not configured
"at24c64" at iic0 addr 0x54 not configured
"at24c64" at iic0 addr 0x58 not configured
"at34c02" at iic0 addr 0x5b not configured
"at34c02" at iic0 addr 0x5c not configured
"at34c02" at iic0 addr 0x5d not configured
"at34c02" at iic0 addr 0x5e not configured
"ds1307" at iic0 addr 0x68 not configured
"at24c64" at iic0 addr 0x28 not configured
"pca9555" at iic0 addr 0x22 not configured
"pca9555" at iic0 addr 0x23 not configured
"pca9555" at iic0 addr 0x34 not configured
"pca9556" at iic0 addr 0x38 not configured
power0 at ebus0 addr 800-82f ipl 32: can't map register space
com0 at ebus0 addr 3f8-3ff ipl 44: ns16550a, 16 byte fifo
com0: console
com1 at ebus0 addr 2e8-2ef ipl 44: ns16550a, 16 byte fifo
"rmc-comm" at ebus0 addr 3e8-3ef ipl 44 not configured
"Acer Labs M7101 Power" rev 0x00 at pci1 dev 6 function 0 not configured
ohci0 at pci1 dev 10 function 0 "Acer Labs M5237 USB" rev 0x03: ivec
0x7a7, version 1.0, legacy support
usb0 at ohci0: USB revision 1.0
uhub0 at usb0
uhub0: Acer Labs OHCI root hub, rev 1.00/1.00, addr 1
uhub0: 2 ports with 2 removable, self powered
pciide0 at pci1 dev 13 function 0 "Acer Labs M5229 UDMA IDE" rev 0xc4:
DMA, channel 0 configured to native-PCI, channel 1 configured to
native-PCI
pciide0: using ivec 0x798 for native-PCI interrupt
atapiscsi0 at pciide0 channel 0 drive 0
scsibus0 at atapiscsi0: 2 targets
cd0 at scsibus0 targ 0 lun 0: <TEAC, DV-28E-R, 1.8A> SCSI0 5/cdrom removable
cd0(pciide0:0:0): using PIO mode 4, Ultra-DMA mode 2
pciide0: channel 1 disabled (no drives)
schizo2 at mainbus0: "Tomatillo", version 4, ign 700, bus A 0 to 0
schizo2: dvma map c0000000-dfffffff, iotdb 5514000-5594000
pci2 at schizo2
siop0 at pci2 dev 2 function 0 "Symbios Logic 53c1010-66" rev 0x01:
ivec 0x729, using 8K of on-board RAM
scsibus1 at siop0: 16 targets
sd0 at scsibus1 targ 0 lun 0: <SEAGATE, ST373207LSUN72G, 045A> SCSI3
0/direct fixed
sd0: 70007MB, 14089 cyl, 24 head, 424 sec, 512 bytes/sec, 143374738 sec total
siop1 at pci2 dev 2 function 1 "Symbios Logic 53c1010-66" rev 0x01:
ivec 0x728, using 8K of on-board RAM
scsibus2 at siop1: 16 targets
schizo3 at mainbus0: "Tomatillo", version 4, ign 740, bus B 0 to 0
schizo3: dvma map c0000000-dfffffff, iotdb 55c2000-5642000
pci3 at schizo3
bge2 at pci3 dev 2 function 0 "Broadcom BCM5704C" rev 0x00, BCM5704 B0
(0x2100): ivec 0x75c, address 00:14:4f:64:0c:54
brgphy2 at bge2 phy 1: BCM5704 10/100/1000baseT PHY, rev. 0
bge3 at pci3 dev 2 function 1 "Broadcom BCM5704C" rev 0x00, BCM5704 B0
(0x2100): ivec 0x75d, address 00:14:4f:64:0c:55
brgphy3 at bge3 phy 1: BCM5704 10/100/1000baseT PHY, rev. 0
mskc0 at pci3 dev 1 function 0 "Schneider & Koch SK-9Sxx" rev 0x11,
Yukon-2 XL rev. A1 (0x1): ivec 0x744
msk0 at mskc0 port A, address 00:00:5a:72:fc:58
eephy0 at msk0 phy 0: Marvell 88E1112 Gigabit PHY, rev. 1
pcons at mainbus0 not configured
No counter-timer -- using %tick at 1336MHz as system clock.
root on sd0a
siop0: target 0 now using tagged DT 16 bit 80.0 MHz 62 REQ/ACK offset xfers
rootdev=0x700 rrootdev=0x1100 rawdev=0x1102
#
