Snapshot hangs at lm1 detached (continued)

2011-06-09 Thread Jeff Ross

Today's June 10 snapshot hangs in the same place.

I also installed the June 6 snap below on a standby remote server with a 
SuperMicro motherboard and the lm sensors and it also hangs in the same 
place.  It's locked up tight right now at the hang so I can't get a 
current dmesg from it.


However, it was fine with a snapshot installed on May 25, so that 
narrows the window down considerably!  (This is the server referenced in 
the thread on misc with the subject Bad frame pointer crash again).


I'm going to try building kernels for the days between May 25 and June 8 
to see if I can narrow the problem down a little more.


Jeff


I upgraded one of my CARPed firewalls this morning and on reboot with 
the new kernel it hangs just before it detaches lm1.  bsd.rd will boot, 
but neither bsd.sp or bsd.mp will.


I disabled lm* at boot.  Here's the dmesg from that, followed by the 
dmesg from the last kernel (May 2).  I marked the last line printed on 
the screen.


Hooray for CARP!  I was able to do all of this messing around and the 
rest of the users didn't know a thing.


Jeff

OpenBSD 4.9-current (GENERIC.MP) #65: Mon Jun  6 14:30:48 MDT 2011
dera...@i386.openbsd.org:/usr/src/sys/arch/i386/compile/GENERIC.MP
cpu0: Intel(R) Xeon(TM) CPU 3.20GHz (GenuineIntel 686-class) 3.21 GHz
cpu0: 
FPU,V86,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,SSE3,MWAIT,DS-CPL,CNXT-ID,CX16,xTPR

real mem  = 3757490176 (3583MB)
avail mem = 3685838848 (3515MB)
User Kernel Config
UKC disable lm
306 lm0 disabled
307 lm* disabled
308 lm* disabled
UKC quit
Continuing...
mainbus0 at root
bios0 at mainbus0: AT/286+ BIOS, date 03/29/05, BIOS32 rev. 0 @ 0xf0010, 
SMBIOS rev. 2.3 @ 0xfa380 (61 entries)

bios0: vendor American Megatrends Inc. version 080010 date 03/29/2005
bios0: SiMech R200
acpi0 at bios0: rev 2
acpi0: sleep states S0 S1 S4 S5
acpi0: tables DSDT FACP APIC OEMB
acpi0: wakeup devices PXHA(S4) PXHB(S4) EPA0(S4) EPA1(S4) EPB0(S4) 
EPB1(S4) EPC0(S4) P0P1(S4) MC97(S4) USB1(S1) USB2(S1) EUSB(S4) PS2K(S4) 
PS2M(S4) P0PC(S4) SLPB(S4)

acpitimer0 at acpi0: 3579545 Hz, 24 bits
acpimadt0 at acpi0 addr 0xfee0: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: apic clock running at 200MHz
cpu1 at mainbus0: apid 6 (application processor)
cpu1: Intel(R) Xeon(TM) CPU 3.20GHz (GenuineIntel 686-class) 3.21 GHz
cpu1: 
FPU,V86,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,SSE3,MWAIT,DS-CPL,CNXT-ID,CX16,xTPR

cpu2 at mainbus0: apid 1 (application processor)
cpu2: Intel(R) Xeon(TM) CPU 3.20GHz (GenuineIntel 686-class) 3.21 GHz
cpu2: 
FPU,V86,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,SSE3,MWAIT,DS-CPL,CNXT-ID,CX16,xTPR

cpu3 at mainbus0: apid 7 (application processor)
cpu3: Intel(R) Xeon(TM) CPU 3.20GHz (GenuineIntel 686-class) 3.21 GHz
cpu3: 
FPU,V86,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,SSE3,MWAIT,DS-CPL,CNXT-ID,CX16,xTPR

ioapic0 at mainbus0: apid 8 pa 0xfec0, version 20, 24 pins
ioapic1 at mainbus0: apid 11 pa 0xfec1, version 20, 24 pins
ioapic1: misconfigured as apic 9, remapped to apid 11
ioapic2 at mainbus0: apid 9 pa 0xfec8, version 20, 24 pins
ioapic2: misconfigured as apic 10, remapped to apid 9
ioapic3 at mainbus0: apid 10 pa 0xfec80400, version 20, 24 pins
ioapic3: misconfigured as apic 11, remapped to apid 10
acpiprt0 at acpi0: bus 0 (PCI0)
acpiprt1 at acpi0: bus 1 (EPA0)
acpiprt2 at acpi0: bus 3 (PXHA)
acpiprt3 at acpi0: bus 2 (PXHB)
acpiprt4 at acpi0: bus 4 (EPA1)
acpiprt5 at acpi0: bus 6 (P0P1)
acpiprt6 at acpi0: bus 5 (P0PC)
acpicpu0 at acpi0
acpicpu1 at acpi0
acpicpu2 at acpi0
acpicpu3 at acpi0
acpibtn0 at acpi0: PWRB
acpibtn1 at acpi0: SLPB
bios0: ROM list: 0xc/0x8000 0xc8000/0x800 0xc8800/0x1000 0xc9800/0x1000
ipmi at mainbus0 not configured
pci0 at mainbus0 bus 0: configuration mode 1 (bios)
pchb0 at pci0 dev 0 function 0 Intel E7320 Host rev 0x0c
ppb0 at pci0 dev 2 function 0 Intel E7520 PCIE rev 0x0c
pci1 at ppb0 bus 1
ppb1 at pci1 dev 0 function 0 Intel 6700PXH PCIE-PCIX rev 0x09
pci2 at ppb1 bus 3
mpi0 at pci2 dev 5 function 0 Symbios Logic 53c1030 rev 0xc1: msi
scsibus0 at mpi0: 16 targets, initiator 7
ppb2 at pci1 dev 0 function 2 Intel 6700PXH PCIE-PCIX rev 0x09
pci3 at ppb2 bus 2
em0 at pci3 dev 1 function 0 Intel PRO/1000MT (82546GB) rev 0x03: apic 
10 int 0, address 00:1b:21:15:6a:82
em1 at pci3 dev 1 function 1 Intel PRO/1000MT (82546GB) rev 0x03: apic 
10 int 1, address 00:1b:21:15:6a:83

ppb3 at pci0 dev 3 function 0 Intel E7520 PCIE rev 0x0c
pci4 at ppb3 bus 4
ppb4 at pci0 dev 28 function 0 Intel 6300ESB PCIX rev 0x02
pci5 at ppb4 bus 5
em2 at pci5 dev 1 function 0 Intel PRO/1000MT (82541GI) rev 0x05: apic 
11 int 2, address 00:30:48:87:2e:b0
em3 at pci5 dev 2 function 0 Intel PRO/1000MT (82541GI) rev 0x05: 

Snapshot hangs at lm1 detached

2011-06-08 Thread Jeff Ross
I upgraded one of my CARPed firewalls this morning and on reboot with 
the new kernel it hangs just before it detaches lm1.  bsd.rd will boot, 
but neither bsd.sp or bsd.mp will.


I disabled lm* at boot.  Here's the dmesg from that, followed by the 
dmesg from the last kernel (May 2).  I marked the last line printed on 
the screen.


Hooray for CARP!  I was able to do all of this messing around and the 
rest of the users didn't know a thing.


Jeff

OpenBSD 4.9-current (GENERIC.MP) #65: Mon Jun  6 14:30:48 MDT 2011
dera...@i386.openbsd.org:/usr/src/sys/arch/i386/compile/GENERIC.MP
cpu0: Intel(R) Xeon(TM) CPU 3.20GHz (GenuineIntel 686-class) 3.21 GHz
cpu0: 
FPU,V86,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,SSE3,MWAIT,DS-CPL,CNXT-ID,CX16,xTPR

real mem  = 3757490176 (3583MB)
avail mem = 3685838848 (3515MB)
User Kernel Config
UKC disable lm
306 lm0 disabled
307 lm* disabled
308 lm* disabled
UKC quit
Continuing...
mainbus0 at root
bios0 at mainbus0: AT/286+ BIOS, date 03/29/05, BIOS32 rev. 0 @ 0xf0010, 
SMBIOS rev. 2.3 @ 0xfa380 (61 entries)

bios0: vendor American Megatrends Inc. version 080010 date 03/29/2005
bios0: SiMech R200
acpi0 at bios0: rev 2
acpi0: sleep states S0 S1 S4 S5
acpi0: tables DSDT FACP APIC OEMB
acpi0: wakeup devices PXHA(S4) PXHB(S4) EPA0(S4) EPA1(S4) EPB0(S4) 
EPB1(S4) EPC0(S4) P0P1(S4) MC97(S4) USB1(S1) USB2(S1) EUSB(S4) PS2K(S4) 
PS2M(S4) P0PC(S4) SLPB(S4)

acpitimer0 at acpi0: 3579545 Hz, 24 bits
acpimadt0 at acpi0 addr 0xfee0: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: apic clock running at 200MHz
cpu1 at mainbus0: apid 6 (application processor)
cpu1: Intel(R) Xeon(TM) CPU 3.20GHz (GenuineIntel 686-class) 3.21 GHz
cpu1: 
FPU,V86,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,SSE3,MWAIT,DS-CPL,CNXT-ID,CX16,xTPR

cpu2 at mainbus0: apid 1 (application processor)
cpu2: Intel(R) Xeon(TM) CPU 3.20GHz (GenuineIntel 686-class) 3.21 GHz
cpu2: 
FPU,V86,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,SSE3,MWAIT,DS-CPL,CNXT-ID,CX16,xTPR

cpu3 at mainbus0: apid 7 (application processor)
cpu3: Intel(R) Xeon(TM) CPU 3.20GHz (GenuineIntel 686-class) 3.21 GHz
cpu3: 
FPU,V86,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,SSE3,MWAIT,DS-CPL,CNXT-ID,CX16,xTPR

ioapic0 at mainbus0: apid 8 pa 0xfec0, version 20, 24 pins
ioapic1 at mainbus0: apid 11 pa 0xfec1, version 20, 24 pins
ioapic1: misconfigured as apic 9, remapped to apid 11
ioapic2 at mainbus0: apid 9 pa 0xfec8, version 20, 24 pins
ioapic2: misconfigured as apic 10, remapped to apid 9
ioapic3 at mainbus0: apid 10 pa 0xfec80400, version 20, 24 pins
ioapic3: misconfigured as apic 11, remapped to apid 10
acpiprt0 at acpi0: bus 0 (PCI0)
acpiprt1 at acpi0: bus 1 (EPA0)
acpiprt2 at acpi0: bus 3 (PXHA)
acpiprt3 at acpi0: bus 2 (PXHB)
acpiprt4 at acpi0: bus 4 (EPA1)
acpiprt5 at acpi0: bus 6 (P0P1)
acpiprt6 at acpi0: bus 5 (P0PC)
acpicpu0 at acpi0
acpicpu1 at acpi0
acpicpu2 at acpi0
acpicpu3 at acpi0
acpibtn0 at acpi0: PWRB
acpibtn1 at acpi0: SLPB
bios0: ROM list: 0xc/0x8000 0xc8000/0x800 0xc8800/0x1000 0xc9800/0x1000
ipmi at mainbus0 not configured
pci0 at mainbus0 bus 0: configuration mode 1 (bios)
pchb0 at pci0 dev 0 function 0 Intel E7320 Host rev 0x0c
ppb0 at pci0 dev 2 function 0 Intel E7520 PCIE rev 0x0c
pci1 at ppb0 bus 1
ppb1 at pci1 dev 0 function 0 Intel 6700PXH PCIE-PCIX rev 0x09
pci2 at ppb1 bus 3
mpi0 at pci2 dev 5 function 0 Symbios Logic 53c1030 rev 0xc1: msi
scsibus0 at mpi0: 16 targets, initiator 7
ppb2 at pci1 dev 0 function 2 Intel 6700PXH PCIE-PCIX rev 0x09
pci3 at ppb2 bus 2
em0 at pci3 dev 1 function 0 Intel PRO/1000MT (82546GB) rev 0x03: apic 
10 int 0, address 00:1b:21:15:6a:82
em1 at pci3 dev 1 function 1 Intel PRO/1000MT (82546GB) rev 0x03: apic 
10 int 1, address 00:1b:21:15:6a:83

ppb3 at pci0 dev 3 function 0 Intel E7520 PCIE rev 0x0c
pci4 at ppb3 bus 4
ppb4 at pci0 dev 28 function 0 Intel 6300ESB PCIX rev 0x02
pci5 at ppb4 bus 5
em2 at pci5 dev 1 function 0 Intel PRO/1000MT (82541GI) rev 0x05: apic 
11 int 2, address 00:30:48:87:2e:b0
em3 at pci5 dev 2 function 0 Intel PRO/1000MT (82541GI) rev 0x05: apic 
11 int 3, address 00:30:48:87:2e:b1

uhci0 at pci0 dev 29 function 0 Intel 6300ESB USB rev 0x02: apic 8 int 16
uhci1 at pci0 dev 29 function 1 Intel 6300ESB USB rev 0x02: apic 8 int 19
Intel 6300ESB WDT rev 0x02 at pci0 dev 29 function 4 not configured
Intel 6300ESB APIC rev 0x02 at pci0 dev 29 function 5 not configured
ehci0 at pci0 dev 29 function 7 Intel 6300ESB USB rev 0x02: apic 8 int 23
usb0 at ehci0: USB revision 2.0
uhub0 at usb0 Intel EHCI root hub rev 2.00/1.00 addr 1
ppb5 at pci0 dev 30 function 0 Intel 82801BA Hub-to-PCI rev 0x0a
pci6 at ppb5 bus 6
vga1 at pci6 dev 2 function 0 ATI Rage XL rev 0x27
wsdisplay0 at