Hi,
One of my firewall/pimd multicast router got today in ddb after 2
months, without any obvious reason.
There was no panic. However I got this trace which might be usefull to you.
This was one of the first snapshots of 5.0. I updated after that to the
latest snapshot.
regards
Giannis
ddb{0}> trace
pf_find_state(d41f2100,e1b45ad8,2,dff4b800,1c) at pf_find_state+0x54
pf_test_state_udp(e1b45b60,e1b45b50,2,2,d41f2100) at pf_test_state_udp+0x99
pf_test(2,2,d4239000,e1b45ca4,0) at pf_test+0x78e
ip_output(dff4b800,0,0,1,e1b45d20) at ip_output+0x776
mrt_ioctl(e00f3400,0,3b9aca00,2,d0a2dcc0) at mrt_ioctl+0x1b6
mrt_ioctl(40,e1b45e3c,2c,0,40) at mrt_ioctl+0xa4a
ip_mrouter_set(d9cd5dd8,68,e1b45ef0,68,e1b45ef0) at ip_mrouter_set+0x7c9
sosetopt(d9cd5dd8,0,68,dff4bc00,246) at sosetopt+0x54
sys_setsockopt(d9e8f818,e1b45f64,e1b45f84,0,d3ffe540) at sys_setsockopt+0xc4
syscall() at syscall+0x12d
PID PPID PGRP UID S FLAGS WAIT COMMAND
23773 10764 10764 507 3 0x80 kqread pickup
*16400 1 16400 0 7 0x801000 pimd
15109 1 15109 0 3 0x80 ttyin getty
4115 1 4115 0 3 0x80 ttyin getty
23977 1 23977 0 3 0x80 ttyin getty
26477 1 26477 0 3 0x80 ttyin getty
25508 1 25508 0 3 0x80 ttyin getty
6819 1 6819 0 3 0x80 ttyin getty
10151 1 10151 0 3 0x80 select cron
2742 1 3724 0 3 0x80 select snmpd
17907 10764 10764 507 3 0x80 kqread qmgr
10764 1 10764 0 3 0x80 kqread master
14727 1 14727 0 3 0x80 kqread ifstated
25072 1 25072 0 3 0x80 select sshd
27575 5262 30317 83 3 0x80 poll ntpd
5262 30317 30317 83 3 0x80 poll ntpd
30317 1 30317 0 3 0x80 poll ntpd
1418 17063 17063 74 3 0x80 bpf pflogd
17063 1 17063 0 3 0x80 netio pflogd
20596 1021 1021 73 7 0x80 syslogd
1021 1 1021 0 3 0x80 netio syslogd
17 0 0 0 3 0x100200 aiodoned aiodoned
16 0 0 0 3 0x100200 syncer update
15 0 0 0 3 0x100200 cleaner cleaner
14 0 0 0 3 0x100200 reaper reaper
13 0 0 0 3 0x100200 pgdaemon pagedaemon
12 0 0 0 3 0x100200 bored crypto
11 0 0 0 3 0x100200 pftm pfpurge
10 0 0 0 3 0x100200 usbtsk usbtask
9 0 0 0 3 0x100200 usbatsk usbatsk
8 0 0 0 3 0x100200 acpi0 acpi0
7 0 0 0 7 0x40100200 idle3
6 0 0 0 7 0x40100200 idle2
5 0 0 0 3 0x40100200 idle1
4 0 0 0 3 0x100200 bored syswq
3 0 0 0 3 0x40100200 idle0
2 0 0 0 3 0x100200 kmalloc kmthread
1 0 1 0 3 0x80 wait init
0 -1 0 0 3 0x200 scheduler swapper
OpenBSD 5.0-current (GENERIC.MP) #126: Sun Nov 20 16:48:02 MST 2011
[email protected]:/usr/src/sys/arch/i386/compile/GENERIC.MP
cpu0: Intel(R) Xeon(R) CPU E5640 @ 2.67GHz ("GenuineIntel" 686-class)
2.67 GHz
cpu0:
FPU,V86,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,NXE,LONG,SSE3,PCLMUL,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,DCA,SSE4.1,SSE4.2,POPCNT,AES,LAHF
real mem = 3210981376 (3062MB)
avail mem = 3148365824 (3002MB)
mainbus0 at root
bios0 at mainbus0: AT/286+ BIOS, date 12/13/10, BIOS32 rev. 0 @ 0xfdb60,
SMBIOS rev. 2.5 @ 0xbf6c8000 (94 entries)
bios0: vendor FUJITSU // Phoenix Technologies Ltd. version "6.00 Rev.
1.09.2619.N1" date 12/13/2010
bios0: FUJITSU PRIMERGY RX300 S6
acpi0 at bios0: rev 2
acpi0: sleep states S0 S1 S4 S5
acpi0: tables DSDT FACP TCPA SLIT SSDT SSDT SSDT SSDT SSDT SSDT SSDT
SSDT EINJ HEST BERT SSDT ERST SPCR DMAR MCFG HPET APIC BOOT
acpi0: wakeup devices PE0_(S4) PE1_(S4) PE3_(S4) PE5_(S4) PE7_(S4)
PE8_(S4) PE9_(S4) PE10(S4) PE1A(S4) PE1E(S4) USB1(S1) USB2(S1) USB3(S1)
USB4(S1) USB5(S1) USB6(S1) USB7(S1) USB8(S1) COM1(S1) COM2(S1)
acpitimer0 at acpi0: 3579545 Hz, 24 bits
acpimcfg0 at acpi0 addr 0xe0000000, bus 0-10
acpihpet0 at acpi0: 14318179 Hz
acpimadt0 at acpi0 addr 0xfee00000: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: apic clock running at 133MHz
cpu1 at mainbus0: apid 2 (application processor)
cpu1: Intel(R) Xeon(R) CPU E5640 @ 2.67GHz ("GenuineIntel" 686-class)
2.67 GHz
cpu1:
FPU,V86,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,NXE,LONG,SSE3,PCLMUL,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,DCA,SSE4.1,SSE4.2,POPCNT,AES,LAHF
cpu2 at mainbus0: apid 18 (application processor)
cpu2: Intel(R) Xeon(R) CPU E5640 @ 2.67GHz ("GenuineIntel" 686-class)
2.67 GHz
cpu2:
FPU,V86,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,NXE,LONG,SSE3,PCLMUL,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,DCA,SSE4.1,SSE4.2,POPCNT,AES,LAHF
cpu3 at mainbus0: apid 20 (application processor)
cpu3: Intel(R) Xeon(R) CPU E5640 @ 2.67GHz ("GenuineIntel" 686-class)
2.67 GHz
cpu3:
FPU,V86,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,NXE,LONG,SSE3,PCLMUL,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,DCA,SSE4.1,SSE4.2,POPCNT,AES,LAHF
ioapic0 at mainbus0: apid 1 pa 0xfec00000, version 20, 24 pins
ioapic1 at mainbus0: apid 3 pa 0xfec80000, version 20, 24 pins
acpiprt0 at acpi0: bus 0 (PCI0)
acpiprt1 at acpi0: bus 1 (PE1_)
acpiprt2 at acpi0: bus 2 (PE3_)
acpiprt3 at acpi0: bus 3 (PE5_)
acpiprt4 at acpi0: bus 4 (PE7_)
acpiprt5 at acpi0: bus 5 (PE8_)
acpiprt6 at acpi0: bus 6 (PE9_)
acpiprt7 at acpi0: bus 7 (PE10)
acpiprt8 at acpi0: bus 8 (PE1A)
acpiprt9 at acpi0: bus 9 (PE1E)
acpicpu0 at acpi0: C3, C2, C1, PSS
acpicpu1 at acpi0: C3, C2, C1, PSS
acpicpu2 at acpi0: C3, C2, C1, PSS
acpicpu3 at acpi0: C3, C2, C1, PSS
acpibtn0 at acpi0: PWRB
acpiac0 at acpi0: AC unit online
bios0: ROM list: 0xc0000/0x8000 0xc8000/0x2a00! 0xcb000/0x5c00
0xd0c00/0x200! 0xd0e00/0x1000 0xd1e00/0x1000
ipmi at mainbus0 not configured
cpu0: Enhanced SpeedStep 2667 MHz: speeds: 2661, 2660, 2527, 2394, 2261,
2128, 1995, 1862, 1729, 1596 MHz
pci0 at mainbus0 bus 0: configuration mode 1 (bios)
pchb0 at pci0 dev 0 function 0 "Intel 5520 Host" rev 0x13
ppb0 at pci0 dev 1 function 0 "Intel X58 PCIE" rev 0x13: apic 3 int 4
pci1 at ppb0 bus 1
mfi0 at pci1 dev 0 function 0 "Symbios Logic MegaRAID SAS2108 GEN2" rev
0x04: apic 3 int 4, 0x11761734
mfi0: logical drives 1, version 12.4.0-0031, 512MB RAM
scsibus0 at mfi0: 1 targets
sd0 at scsibus0 targ 0 lun 0: <LSI, RAID 5/6 SAS 6G, 2.40> SCSI3
0/direct fixed naa.6003005700d24c20151664951f3f94ab
sd0: 139488MB, 512 bytes/sector, 285671424 sectors
ppb1 at pci0 dev 3 function 0 "Intel X58 PCIE" rev 0x13: apic 3 int 0
pci2 at ppb1 bus 2
em0 at pci2 dev 0 function 0 "Intel PRO/1000 PT (82571EB)" rev 0x06:
msi, address 00:1b:21:8c:41:96
em1 at pci2 dev 0 function 1 "Intel PRO/1000 PT (82571EB)" rev 0x06:
msi, address 00:1b:21:8c:41:97
ppb2 at pci0 dev 5 function 0 "Intel X58 PCIE" rev 0x13: apic 3 int 2
pci3 at ppb2 bus 3
ppb3 at pci0 dev 7 function 0 "Intel X58 PCIE" rev 0x13: apic 3 int 6
pci4 at ppb3 bus 4
ppb4 at pci0 dev 8 function 0 "Intel X58 PCIE" rev 0x13: apic 3 int 7
pci5 at ppb4 bus 5
ppb5 at pci0 dev 9 function 0 "Intel X58 PCIE" rev 0x13: apic 3 int 8
pci6 at ppb5 bus 6
ppb6 at pci0 dev 10 function 0 "Intel X58 PCIE" rev 0x13: apic 3 int 9
pci7 at ppb6 bus 7
"Intel X58 QuickPath" rev 0x13 at pci0 dev 16 function 0 not configured
"Intel X58 QuickPath" rev 0x13 at pci0 dev 16 function 1 not configured
"Intel X58 QuickPath" rev 0x13 at pci0 dev 17 function 0 not configured
"Intel X58 QuickPath" rev 0x13 at pci0 dev 17 function 1 not configured
"Intel X58 Misc" rev 0x13 at pci0 dev 20 function 0 not configured
"Intel X58 GPIO" rev 0x13 at pci0 dev 20 function 1 not configured
"Intel X58 RAS" rev 0x13 at pci0 dev 20 function 2 not configured
"Intel X58 Throttle" rev 0x13 at pci0 dev 20 function 3 not configured
vendor "Intel", unknown product 0x342f (class system subclass interrupt,
rev 0x13) at pci0 dev 21 function 0 not configured
uhci0 at pci0 dev 26 function 0 "Intel 82801JI USB" rev 0x00: apic 1 int 19
uhci1 at pci0 dev 26 function 1 "Intel 82801JI USB" rev 0x00: apic 1 int 18
uhci2 at pci0 dev 26 function 2 "Intel 82801JI USB" rev 0x00: apic 1 int 18
ehci0 at pci0 dev 26 function 7 "Intel 82801JI USB" rev 0x00: apic 1 int 18
usb0 at ehci0: USB revision 2.0
uhub0 at usb0 "Intel EHCI root hub" rev 2.00/1.00 addr 1
ppb7 at pci0 dev 28 function 0 "Intel 82801JI PCIE" rev 0x00: apic 1 int 16
pci8 at ppb7 bus 8
em2 at pci8 dev 0 function 0 "Intel PRO/1000 PT (82575EB)" rev 0x02:
msi, address 00:19:99:98:e4:ea
em3 at pci8 dev 0 function 1 "Intel PRO/1000 PT (82575EB)" rev 0x02:
msi, address 00:19:99:98:e4:eb
ppb8 at pci0 dev 28 function 4 "Intel 82801JI PCIE" rev 0x00: apic 1 int 16
pci9 at ppb8 bus 9
vga1 at pci9 dev 0 function 0 "Matrox MGA G200e (ServerEngines)" rev 0x02
wsdisplay0 at vga1 mux 1: console (80x25, vt100 emulation)
wsdisplay0: screen 1-5 added (80x25, vt100 emulation)
uhci3 at pci0 dev 29 function 0 "Intel 82801JI USB" rev 0x00: apic 1 int 23
uhci4 at pci0 dev 29 function 1 "Intel 82801JI USB" rev 0x00: apic 1 int 22
uhci5 at pci0 dev 29 function 2 "Intel 82801JI USB" rev 0x00: apic 1 int 21
ehci1 at pci0 dev 29 function 7 "Intel 82801JI USB" rev 0x00: apic 1 int 23
usb1 at ehci1: USB revision 2.0
uhub1 at usb1 "Intel EHCI root hub" rev 2.00/1.00 addr 1
ppb9 at pci0 dev 30 function 0 "Intel 82801BA Hub-to-PCI" rev 0x90
pci10 at ppb9 bus 10
pcib0 at pci0 dev 31 function 0 "Intel 82801JIB LPC" rev 0x00
ahci0 at pci0 dev 31 function 2 "Intel 82801JI AHCI" rev 0x00: msi, AHCI 1.2
scsibus1 at ahci0: 32 targets
cd0 at scsibus1 targ 0 lun 0: <HL-DT-ST, DVDRAM_GT30N, FT03> ATAPI
5/cdrom removable
ichiic0 at pci0 dev 31 function 3 "Intel 82801JI SMBus" rev 0x00: apic 1
int 17
iic0 at ichiic0
iic0: addr 0x2e 00=40 words 00=4040 01=0000 02=0000 03=0000 04=0000
05=0000 06=0000 07=0000
usb2 at uhci0: USB revision 1.0
uhub2 at usb2 "Intel UHCI root hub" rev 1.00/1.00 addr 1
usb3 at uhci1: USB revision 1.0
uhub3 at usb3 "Intel UHCI root hub" rev 1.00/1.00 addr 1
usb4 at uhci2: USB revision 1.0
uhub4 at usb4 "Intel UHCI root hub" rev 1.00/1.00 addr 1
usb5 at uhci3: USB revision 1.0
uhub5 at usb5 "Intel UHCI root hub" rev 1.00/1.00 addr 1
usb6 at uhci4: USB revision 1.0
uhub6 at usb6 "Intel UHCI root hub" rev 1.00/1.00 addr 1
usb7 at uhci5: USB revision 1.0
uhub7 at usb7 "Intel UHCI root hub" rev 1.00/1.00 addr 1
isa0 at pcib0
isadma0 at isa0
com0 at isa0 port 0x3f8/8 irq 4: ns16550a, 16 byte fifo
com0: console
com1 at isa0 port 0x2f8/8 irq 3: ns16550a, 16 byte fifo
pckbc0 at isa0 port 0x60/5
kbc: cmd word write error
pcppi0 at isa0 port 0x61
spkr0 at pcppi0
npx0 at isa0 port 0xf0/16: reported by CPUID; using exception 16
mtrr: Pentium Pro MTRR support
uhidev0 at uhub7 port 1 configuration 1 interface 0 "SILITEK USB
Keyboard and Mouse" rev 1.00/1.20 addr 2
uhidev0: iclass 3/1
ukbd0 at uhidev0: 8 modifier keys, 6 key codes
wskbd0 at ukbd0 mux 1
wskbd0: connecting to wsdisplay0
uhidev1 at uhub7 port 1 configuration 1 interface 1 "SILITEK USB
Keyboard and Mouse" rev 1.00/1.20 addr 2
uhidev1: iclass 3/0, 2 report ids
ums0 at uhidev1 reportid 1: 3 buttons, Z dir
wsmouse0 at ums0 mux 0
uhid0 at uhidev1 reportid 2: input=1, output=0, feature=0
vscsi0 at root
scsibus2 at vscsi0: 256 targets
softraid0 at root
scsibus3 at softraid0: 256 targets
root on sd0a swap on sd0b dump on sd0b
WARNING: / was not properly unmounted
carp: carp0 demoted group carp by 1 to 129 (carpdev)
carp: carp1 demoted group carp by 1 to 130 (carpdev)
carp: carp2 demoted group carp by 1 to 131 (carpdev)
carp: carp3 demoted group carp by 1 to 132 (carpdev)
carp: pfsync0 demoted group carp by 1 to 133 (pfsync bulk start)
carp: pfsync0 demoted group pfsync by 1 to 1 (pfsync bulk start)
carp: carp1 demoted group carp by -1 to 132 (carpdev)
carp: carp0 demoted group carp by -1 to 131 (carpdev)
carp: carp3 demoted group carp by -1 to 130 (carpdev)
carp: carp2 demoted group carp by -1 to 129 (carpdev)
carp: carp0 demoted group carp by 1 to 130 (carpdev)
carp: carp3 demoted group carp by 1 to 131 (carpdev)
carp: carp2 demoted group carp by 1 to 132 (carpdev)
carp1: state transition: BACKUP -> MASTER
arp_rtrequest: bad gateway value
carp: carp0 demoted group carp by -1 to 3 (carpdev)
carp: carp3 demoted group carp by -1 to 2 (carpdev)
carp: carp2 demoted group carp by -1 to 1 (carpdev)
carp: carp0 demoted group carp by 1 to 2 (carpdev)
carp: carp3 demoted group carp by 1 to 3 (carpdev)
carp: carp2 demoted group carp by 1 to 4 (carpdev)
carp: carp0 demoted group carp by -1 to 3 (carpdev)
carp: carp3 demoted group carp by -1 to 2 (carpdev)
carp: carp2 demoted group carp by -1 to 1 (carpdev)
carp0: state transition: BACKUP -> MASTER
arp_rtrequest: bad gateway value
carp3: state transition: BACKUP -> MASTER
arp_rtrequest: bad gateway value
carp2: state transition: BACKUP -> MASTER
arp_rtrequest: bad gateway value
carp1: state transition: MASTER -> BACKUP
carp0: state transition: MASTER -> BACKUP
carp3: state transition: MASTER -> BACKUP
carp2: state transition: MASTER -> BACKUP
carp: pfsync0 demoted group carp by -1 to 0 (pfsync bulk done)
carp: pfsync0 demoted group pfsync by -1 to 0 (pfsync bulk done)
carp0: state transition: BACKUP -> MASTER
arp_rtrequest: bad gateway value
carp3: state transition: BACKUP -> MASTER
arp_rtrequest: bad gateway value
carp2: state transition: BACKUP -> MASTER
arp_rtrequest: bad gateway value
carp1: state transition: BACKUP -> MASTER
arp_rtrequest: bad gateway value
