I have had both firewalls in a carp/pfsync pair which are running the same
snapshot crash;
uvm_fault(0xd07f81e0, 0x0, 0, 1) -> e
kernel: page fault trap, code=0
Stopped at pf_test_rule+0x8a0: movl 0x58(%eax),%ecx
ddb> pf_test_rule(d51efd64,d51efd5c,1,d0ba8500,db53ad00) at pf_test_rule+0x8a0
pf_test(1,d0c54800,d51efe64,0) at pf_test+0x8c1
ipv4_input(db53ad00,d0b9a180,68400000,7c10c) at ipv4_input+0x124
ipintr(58,10,d51e0010,d0480010,7c10c) at ipintr+0x64
Bad frame pointer: 0xd51efe7c
ddb> PID PPID PGRP UID S FLAGS WAIT COMMAND
29392 24940 24940 0 2 0x40100 sendmail
897 22729 897 0 2 0x4002 top
22729 16691 22729 0 3 0x4082 pause ksh
16691 9570 16691 0 2 0x4180 sshd
17835 3928 17835 0 3 0x4082 ttyin ksh
3928 9570 3928 0 2 0x4180 sshd
6950 1 6950 0 3 0x4082 ttyin getty
27487 1 27487 0 3 0x4082 ttyin getty
1375 1 1375 0 3 0x4082 ttyin getty
11547 1 11547 0 3 0x4082 ttyin getty
3455 1 3455 0 3 0x4082 ttyin getty
4587 1 4587 0 2 0 cron
9570 1 9570 0 3 0x80 select sshd
24940 1 24940 0 2 0x40100 sendmail
138 1 138 0 3 0x180 select inetd
30566 14663 14663 83 2 0x100 ntpd
14663 1 14663 0 2 0 ntpd
10627 31688 31688 70 3 0x100 uvn_getpage named
31688 1 31688 0 3 0x180 netio named
5097 8724 8724 74 2 0x100 pflogd
8724 1 8724 0 3 0x80 netio pflogd
16751 7245 7245 73 2 0x100 syslogd
7245 1 7245 0 3 0x88 netio syslogd
12 0 0 0 3 0x100200 bored crypto
11 0 0 0 3 0x100200 aiodoned aiodoned
10 0 0 0 2 0x100200 update
9 0 0 0 3 0x100200 cleaner cleaner
8 0 0 0 3 0x100200 reaper reaper
* 7 0 0 0 7 0x100200 pagedaemon
6 0 0 0 2 0x100600 pfpurge
5 0 0 0 3 0x100200 acpi_idle acpi0
4 0 0 0 3 0x100200 bored syswq
3 0 0 0 3 0x100200 idle0
2 0 0 0 3 0x100200 km_alloc1w kmthread
1 0 1 0 3 0x4080 wait init
0 -1 0 0 3 0x80200 scheduler swapper
Here is the last pf related info I have off of one of the machines
before it crashed (less than 30 minutes later):
Sun Jun 8 01:30:03 NZST 2008
Status: Enabled for 5 days 05:53:11 Debug: Misc
State Table Total Rate
current entries 513
searches 112428343 248.1/s
inserts 919608 2.0/s
removals 919095 2.0/s
Counters
match 973538 2.1/s
bad-offset 0 0.0/s
fragment 0 0.0/s
short 0 0.0/s
normalize 0 0.0/s
memory 0 0.0/s
bad-timestamp 0 0.0/s
congestion 0 0.0/s
ip-option 0 0.0/s
proto-cksum 0 0.0/s
state-mismatch 108 0.0/s
state-insert 0 0.0/s
state-limit 0 0.0/s
src-limit 0 0.0/s
synproxy 0 0.0/s
Name Size Requests Fail Releases Pgreq Pgrel Npage Hiwat Minpg
Maxpg Idle
pfiaddrpl 100 4 0 0 1 0 1 1 0
8 0
pfrulepl 848 30 0 10 8 0 8 8 0
8 2
pfstatepl 192 5122566 0 5122068 49 0 49 49 0
477 24
pfstatekeypl 64 5585028 0 1378048 66778 0 66778 66778 0
8 0
pfstateitempl 12 1378790 0 1378048 5 0 5 5 0
8 2
pfpooladdrpl 68 4 0 0 1 0 1 1 0
8 0
pfrktable 1240 13 0 6 4 0 4 4 0
334 0
pfrkentry 156 10 0 4 1 0 1 1 0
7693 0
pfrkentry2 156 4 0 0 1 0 1 1 0
8 0
pfosfpen 108 1392 0 696 30 11 19 19 0
8 0
pfosfp 28 814 0 407 3 0 3 3 0
8 0
carp:
5854725 packets received (IPv4)
3607274 packets received (IPv6)
9461993 discarded for unknown vhid
897315 packets sent (IPv4)
2 transitions to master
pfsync:
9080346 packets received (IPv4)
8406004 failed state lookup/inserts
10660708 packets sent (IPv4)
OpenBSD 4.3-current (GENERIC) #884: Sat May 31 11:49:16 MDT 2008
[EMAIL PROTECTED]:/usr/src/sys/arch/i386/compile/GENERIC
cpu0: Intel Celeron ("GenuineIntel" 686-class, 128KB L2 cache) 496 MHz
cpu0:
FPU,V86,DE,PSE,TSC,MSR,PAE,MCE,CX8,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,MMX,FXSR
real mem = 334065664 (318MB)
avail mem = 314511360 (299MB)
mainbus0 at root
bios0 at mainbus0: AT/286+ BIOS, date 01/10/99, BIOS32 rev. 0 @ 0xeba00,
SMBIOS rev. 2.3 @ 0xfa248 (43 entries)
bios0: vendor Compaq version "686J1 v1.10" date 08/12/1999
bios0: Compaq Deskpro SFF Series
acpi0 at bios0: rev 0
acpi0: tables DSDT FACP SSDT SSDT SSDT
acpi0: wakeup devices COM1(S4) COM2(S4) USB0(S1) PCI0(S4) PBTN(S4)
acpitimer0 at acpi0: 3579545 Hz, 24 bits
acpiprt0 at acpi0: bus 0 (PCI0)
acpicpu0 at acpi0: C2
acpibtn0 at acpi0: PBTN
bios0: ROM list: 0xc0000/0x8000 0xc8000/0x1000 0xc9000/0x1000
0xe0000/0x10000!
cpu0 at mainbus0
pci0 at mainbus0 bus 0: configuration mode 1 (no bios)
pchb0 at pci0 dev 0 function 0 "Intel 82810-DC100 Host" rev 0x02
vga1 at pci0 dev 1 function 0 "Intel 82810-DC100 Video" rev 0x02
wsdisplay0 at vga1 mux 1: console (80x25, vt100 emulation)
wsdisplay0: screen 1-5 added (80x25, vt100 emulation)
agp0 at vga1: aperture at 0x44000000, size 0x4000000
ppb0 at pci0 dev 30 function 0 "Intel 82801AA Hub-to-PCI" rev 0x01
pci1 at ppb0 bus 1
fxp0 at pci1 dev 2 function 0 "Intel 8255x" rev 0x08, i82559: irq 10,
address 00:50:8b:79:7d:f4
inphy0 at fxp0 phy 1: i82555 10/100 PHY, rev. 4
fxp1 at pci1 dev 9 function 0 "Intel 8255x" rev 0x08, i82559: irq 9,
address 00:04:ac:23:9b:f8
inphy1 at fxp1 phy 1: i82555 10/100 PHY, rev. 4
ichpcib0 at pci0 dev 31 function 0 "Intel 82801AA LPC" rev 0x01
pciide0 at pci0 dev 31 function 1 "Intel 82801AA IDE" rev 0x01: DMA,
channel 0 wired to compatibility, channel 1 wired to compatibility
wd0 at pciide0 channel 0 drive 0: <ST36424A>
wd0: 16-sector PIO, LBA, 6149MB, 12594960 sectors
wd0(pciide0:0:0): using PIO mode 4, Ultra-DMA mode 2
pciide0: channel 1 disabled (no drives)
uhci0 at pci0 dev 31 function 2 "Intel 82801AA USB" rev 0x01: can't map
i/o space
auich0 at pci0 dev 31 function 5 "Intel 82801AA AC97" rev 0x01: can't
map codec i/o space
isa0 at ichpcib0
isadma0 at isa0
com0 at isa0 port 0x3f8/8 irq 4: ns16550a, 16 byte fifo
pckbc0 at isa0 port 0x60/5
pckbd0 at pckbc0 (kbd slot)
pckbc0: using irq 1 for kbd slot
wskbd0 at pckbd0: console keyboard, using wsdisplay0
pcppi0 at isa0 port 0x61
midi0 at pcppi0: <PC speaker>
spkr0 at pcppi0
lpt0 at isa0 port 0x378/4 irq 7
npx0 at isa0 port 0xf0/16: reported by CPUID; using exception 16
fdc0 at isa0 port 0x3f0/6 irq 6 drq 2
fd0 at fdc0 drive 0: 1.44MB 80 cyl, 2 head, 18 sec
biomask f96d netmask ff6d ttymask ffff
mtrr: Pentium Pro MTRR support
softraid0 at root
root on wd0a swap on wd0b dump on wd0b
