Two physical machines with a bunch of vlans and carp interfaces with pfsync. I
have a script that pushes pf.conf to the machines and runs pfctl -f
/etc/pf.conf on them. One of the invocations killed both of them and left a
crash dump.
I’m mostly wondering if this is a known issue or not. If not, I’m curious
about what methods I’d best use to debug the issue? I don’t have a bsd.gdb
available - I was testing syspatch a while ago so I never had a chance to
build it myself.
Thanks for any help!
dmesg -N /var/crash/bsd.0 -M /var/crash/bsd.0.core says:
OpenBSD 6.0 (GENERIC.MP) #2: Mon Nov 28 23:02:49 CET 2016
[email protected]:/usr/src/sys/arch/amd64/compile/GENERIC.MP
RTC BIOS diagnostic error 80<clock_battery>
real mem = 8396206080 (8007MB)
avail mem = 8137265152 (7760MB)
mpath0 at root
scsibus0 at mpath0: 256 targets
mainbus0 at root
bios0 at mainbus0: SMBIOS rev. 2.7 @ 0xe5320 (51 entries)
bios0: vendor Dell Inc. version "1.4.0" date 10/23/2014
bios0: Dell Inc. PowerEdge R220
acpi0 at bios0: rev 2
acpi0: sleep states S0 S4 S5
acpi0: tables DSDT FACP UEFI UEFI ASF! ASPT BOOT DBGP HPET APIC MCFG SLIC WDAT
SSDT SPMI SSDT SSDT SSDT DMAR SSDT FPDT HEST ERST BERT EINJ
acpi0: wakeup devices P0P1(S4) UAR1(S3) ECIR(S4) GLAN(S4) EHC1(S4) EHC2(S4)
XHC_(S4) HDEF(S4) PXSX(S4) RP01(S4) PXSX(S4) RP02(S4) PXSX(S4) RP03(S4)
PXSX(S4) RP04(S4) [...]
acpitimer0 at acpi0: 3579545 Hz, 24 bits
acpihpet0 at acpi0: 14318179 Hz
acpimadt0 at acpi0 addr 0xfee00000: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: Intel(R) Xeon(R) CPU E3-1271 v3 @ 3.60GHz, 3592.44 MHz
cpu0:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUS
H,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX
,SMX,EST,TM2,SSSE3,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT
,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,LONG,LAHF,ABM,PERF,ITSC,FSGSB
ASE,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,SENSOR,ARAT
cpu0: 256KB 64b/line 8-way L2 cache
cpu0: smt 0, core 0, package 0
mtrr: Pentium Pro MTRR support, 10 var ranges, 88 fixed ranges
cpu0: apic clock running at 99MHz
cpu0: mwait min=64, max=64, C-substates=0.2.1.2.4, IBE
cpu1 at mainbus0: apid 1 (application processor)
cpu1: Intel(R) Xeon(R) CPU E3-1271 v3 @ 3.60GHz, 3591.68 MHz
cpu1:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUS
H,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX
,SMX,EST,TM2,SSSE3,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT
,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,LONG,LAHF,ABM,PERF,ITSC,FSGSB
ASE,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,SENSOR,ARAT
cpu1: 256KB 64b/line 8-way L2 cache
cpu1: smt 1, core 0, package 0
cpu2 at mainbus0: apid 2 (application processor)
cpu2: Intel(R) Xeon(R) CPU E3-1271 v3 @ 3.60GHz, 3591.68 MHz
cpu2:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUS
H,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX
,SMX,EST,TM2,SSSE3,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT
,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,LONG,LAHF,ABM,PERF,ITSC,FSGSB
ASE,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,SENSOR,ARAT
cpu2: 256KB 64b/line 8-way L2 cache
cpu2: smt 0, core 1, package 0
cpu3 at mainbus0: apid 3 (application processor)
cpu3: Intel(R) Xeon(R) CPU E3-1271 v3 @ 3.60GHz, 3591.68 MHz
cpu3:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUS
H,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX
,SMX,EST,TM2,SSSE3,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT
,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,LONG,LAHF,ABM,PERF,ITSC,FSGSB
ASE,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,SENSOR,ARAT
cpu3: 256KB 64b/line 8-way L2 cache
cpu3: smt 1, core 1, package 0
cpu4 at mainbus0: apid 4 (application processor)
cpu4: Intel(R) Xeon(R) CPU E3-1271 v3 @ 3.60GHz, 3591.68 MHz
cpu4:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUS
H,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX
,SMX,EST,TM2,SSSE3,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT
,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,LONG,LAHF,ABM,PERF,ITSC,FSGSB
ASE,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,SENSOR,ARAT
cpu4: 256KB 64b/line 8-way L2 cache
cpu4: smt 0, core 2, package 0
cpu5 at mainbus0: apid 5 (application processor)
cpu5: Intel(R) Xeon(R) CPU E3-1271 v3 @ 3.60GHz, 3591.68 MHz
cpu5:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUS
H,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX
,SMX,EST,TM2,SSSE3,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT
,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,LONG,LAHF,ABM,PERF,ITSC,FSGSB
ASE,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,SENSOR,ARAT
cpu5: 256KB 64b/line 8-way L2 cache
cpu5: smt 1, core 2, package 0
cpu6 at mainbus0: apid 6 (application processor)
cpu6: Intel(R) Xeon(R) CPU E3-1271 v3 @ 3.60GHz, 3591.68 MHz
cpu6:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUS
H,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX
,SMX,EST,TM2,SSSE3,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT
,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,LONG,LAHF,ABM,PERF,ITSC,FSGSB
ASE,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,SENSOR,ARAT
cpu6: 256KB 64b/line 8-way L2 cache
cpu6: smt 0, core 3, package 0
cpu7 at mainbus0: apid 7 (application processor)
cpu7: Intel(R) Xeon(R) CPU E3-1271 v3 @ 3.60GHz, 3591.68 MHz
cpu7:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUS
H,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX
,SMX,EST,TM2,SSSE3,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT
,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,LONG,LAHF,ABM,PERF,ITSC,FSGSB
ASE,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,SENSOR,ARAT
cpu7: 256KB 64b/line 8-way L2 cache
cpu7: smt 1, core 3, package 0
ioapic0 at mainbus0: apid 2 pa 0xfec00000, version 20, 24 pins
acpimcfg0 at acpi0 addr 0xf0000000, bus 0-127
acpiprt0 at acpi0: bus 0 (PCI0)
acpiprt1 at acpi0: bus -1 (P0P1)
acpiprt2 at acpi0: bus 2 (RP01)
acpiprt3 at acpi0: bus -1 (RP02)
acpiprt4 at acpi0: bus 3 (RP03)
acpiprt5 at acpi0: bus -1 (RP04)
acpiprt6 at acpi0: bus -1 (RP05)
acpiprt7 at acpi0: bus -1 (RP06)
acpiprt8 at acpi0: bus -1 (RP07)
acpiprt9 at acpi0: bus -1 (RP08)
acpiprt10 at acpi0: bus 1 (PEG0)
acpiprt11 at acpi0: bus -1 (PEG1)
acpiprt12 at acpi0: bus -1 (PEG2)
acpiec0 at acpi0: not present
acpicpu0 at acpi0: C2(200@148 mwait.1@0x33), C1(1000@1 mwait.1), PSS
acpicpu1 at acpi0: C2(200@148 mwait.1@0x33), C1(1000@1 mwait.1), PSS
acpicpu2 at acpi0: C2(200@148 mwait.1@0x33), C1(1000@1 mwait.1), PSS
acpicpu3 at acpi0: C2(200@148 mwait.1@0x33), C1(1000@1 mwait.1), PSS
acpicpu4 at acpi0: C2(200@148 mwait.1@0x33), C1(1000@1 mwait.1), PSS
acpicpu5 at acpi0: C2(200@148 mwait.1@0x33), C1(1000@1 mwait.1), PSS
acpicpu6 at acpi0: C2(200@148 mwait.1@0x33), C1(1000@1 mwait.1), PSS
acpicpu7 at acpi0: C2(200@148 mwait.1@0x33), C1(1000@1 mwait.1), PSS
acpipwrres0 at acpi0: FN00, resource for FAN0
acpipwrres1 at acpi0: FN01, resource for FAN1
acpipwrres2 at acpi0: FN02, resource for FAN2
acpipwrres3 at acpi0: FN03, resource for FAN3
acpipwrres4 at acpi0: FN04, resource for FAN4
acpitz0 at acpi0: critical temperature is 105 degC
acpitz1 at acpi0: critical temperature is 105 degC
acpibat0 at acpi0: BAT0 model "CRB Battery 0" serial Battery 0 type Fake oem
"-Virtual Battery 0-"
acpibat1 at acpi0: BAT1 not present
acpibat2 at acpi0: BAT2 not present
"INT33D6" at acpi0 not configured
"INT3F0D" at acpi0 not configured
"IPI0001" at acpi0 not configured
"PNP0401" at acpi0 not configured
"PNP0A05" at acpi0 not configured
"PNP0501" at acpi0 not configured
"PNP0501" at acpi0 not configured
"PNP0C33" at acpi0 not configured
"INT340E" at acpi0 not configured
"PNP0C0B" at acpi0 not configured
"PNP0C0B" at acpi0 not configured
"PNP0C0B" at acpi0 not configured
"PNP0C0B" at acpi0 not configured
"PNP0C0B" at acpi0 not configured
acpidock0 at acpi0: DOCK not docked (0)
acpivideo0 at acpi0: GFX0
acpivout0 at acpivideo0: DD1F
ipmi at mainbus0 not configured
cpu0: Enhanced SpeedStep 3592 MHz: speeds: 3601, 3600, 3400, 3200, 3000, 2800,
2600, 2400, 2200, 2000, 1800, 1600, 1400, 1200, 1000, 800 MHz
pci0 at mainbus0 bus 0
pchb0 at pci0 dev 0 function 0 "Intel Xeon E3-1200 v3 Host" rev 0x06
ppb0 at pci0 dev 1 function 0 "Intel Core 4G PCIE" rev 0x06: msi
pci1 at ppb0 bus 1
ix0 at pci1 dev 0 function 0 "Intel 82599" rev 0x01: msi, address
a0:36:9f:80:76:c8
ix1 at pci1 dev 0 function 1 "Intel 82599" rev 0x01: msi, address
a0:36:9f:80:76:ca
xhci0 at pci0 dev 20 function 0 "Intel 8 Series xHCI" rev 0x05: msi
usb0 at xhci0: USB revision 3.0
uhub0 at usb0 "Intel xHCI root hub" rev 3.00/1.00 addr 1
ehci0 at pci0 dev 26 function 0 "Intel 8 Series USB" rev 0x05: apic 2 int 16
usb1 at ehci0: USB revision 2.0
uhub1 at usb1 "Intel EHCI root hub" rev 2.00/1.00 addr 1
ppb1 at pci0 dev 28 function 0 "Intel 8 Series PCIE" rev 0xd5: msi
pci2 at ppb1 bus 2
bge0 at pci2 dev 0 function 0 "Broadcom BCM5720" rev 0x00, BCM5720 A0
(0x5720000), APE firmware NCSI 1.3.7.0: msi, address 20:47:47:86:65:98
brgphy0 at bge0 phy 1: BCM5720C 10/100/1000baseT PHY, rev. 0
bge1 at pci2 dev 0 function 1 "Broadcom BCM5720" rev 0x00, BCM5720 A0
(0x5720000), APE firmware NCSI 1.3.7.0: msi, address 20:47:47:86:65:9a
brgphy1 at bge1 phy 2: BCM5720C 10/100/1000baseT PHY, rev. 0
ppb2 at pci0 dev 28 function 2 "Intel 8 Series PCIE" rev 0xd5: msi
pci3 at ppb2 bus 3
ppb3 at pci3 dev 0 function 0 "Renesas SH7757 PCIE Switch" rev 0x00
pci4 at ppb3 bus 4
ppb4 at pci4 dev 0 function 0 "Renesas SH7757 PCIE Switch" rev 0x00
pci5 at ppb4 bus 5
ppb5 at pci5 dev 0 function 0 "Renesas SH7757 PCIE-PCI" rev 0x00
pci6 at ppb5 bus 6
vga1 at pci6 dev 0 function 0 "Matrox MGA G200eR" rev 0x00
wsdisplay0 at vga1 mux 1: console (80x25, vt100 emulation)
wsdisplay0: screen 1-5 added (80x25, vt100 emulation)
ppb6 at pci4 dev 1 function 0 "Renesas SH7757 PCIE Switch" rev 0x00
pci7 at ppb6 bus 7
vendor "Renesas", unknown product 0x0011 (class memory subclass RAM, rev 0x00)
at pci7 dev 0 function 0 not configured
ehci1 at pci0 dev 29 function 0 "Intel 8 Series USB" rev 0x05: apic 2 int 23
usb2 at ehci1: USB revision 2.0
uhub2 at usb2 "Intel EHCI root hub" rev 2.00/1.00 addr 1
pcib0 at pci0 dev 31 function 0 "Intel C222 LPC" rev 0x05
ahci0 at pci0 dev 31 function 2 "Intel 8 Series AHCI" rev 0x05: msi, AHCI 1.3
ahci0: port 0: 6.0Gb/s
ahci0: port 4: 1.5Gb/s
scsibus1 at ahci0: 32 targets
sd0 at scsibus1 targ 0 lun 0: <ATA, TOSHIBA DT01ACA0, MS1O> SCSI3 0/direct
fixed naa.5000039fe0f0972c
sd0: 476940MB, 512 bytes/sector, 976773168 sectors
cd0 at scsibus1 targ 4 lun 0: <HL-DT-ST, DVD-ROM DTA0N, D3B0> ATAPI 5/cdrom
removable
ichiic0 at pci0 dev 31 function 3 "Intel 8 Series SMBus" rev 0x05: apic 2 int
18
iic0 at ichiic0
sdtemp0 at iic0 addr 0x19: mcp98243
sdtemp1 at iic0 addr 0x1b: mcp98243
spdmem0 at iic0 addr 0x51: 4GB DDR3 SDRAM ECC PC3-12800 with thermal sensor
spdmem1 at iic0 addr 0x53: 4GB DDR3 SDRAM ECC PC3-12800 with thermal sensor
isa0 at pcib0
isadma0 at isa0
com0 at isa0 port 0x3f8/8 irq 4: ns16550a, 16 byte fifo
com1 at isa0 port 0x2f8/8 irq 3: ns16550a, 16 byte fifo
pckbc0 at isa0 port 0x60/5 irq 1 irq 12
pcppi0 at isa0 port 0x61
spkr0 at pcppi0
uhub3 at uhub0 port 9 "no manufacturer Gadget USB HUB" rev 2.00/0.00 addr 2
uhidev0 at uhub3 port 1 configuration 1 interface 0 "Avocent Keyboard/Mouse
Function" rev 2.00/0.00 addr 3
uhidev0: iclass 3/1
ukbd0 at uhidev0: 8 variable keys, 6 key codes
wskbd0 at ukbd0: console keyboard, using wsdisplay0
uhidev1 at uhub3 port 1 configuration 1 interface 1 "Avocent Keyboard/Mouse
Function" rev 2.00/0.00 addr 3
uhidev1: iclass 3/1
ums0 at uhidev1: 3 buttons, Z dir
wsmouse0 at ums0 mux 0
uhidev2 at uhub3 port 1 configuration 1 interface 2 "Avocent Keyboard/Mouse
Function" rev 2.00/0.00 addr 3
uhidev2: iclass 3/1
ums1 at uhidev2: 3 buttons, Z dir
wsmouse1 at ums1 mux 0
uhub4 at uhub1 port 1 "Intel Rate Matching Hub" rev 2.00/0.05 addr 2
uhub5 at uhub2 port 1 "Intel Rate Matching Hub" rev 2.00/0.05 addr 2
vscsi0 at root
scsibus2 at vscsi0: 256 targets
softraid0 at root
scsibus3 at softraid0: 256 targets
root on sd0a (e7ce0d3cdff1479c.a) swap on sd0b dump on sd0b
ukbd0: was console keyboard
wskbd0 detached
ukbd0 detached
uhidev0 detached
wsmouse0 detached
ums0 detached
uhidev1 detached
wsmouse1 detached
ums1 detached
uhidev2 detached
uhidev0 at uhub3 port 1 configuration 1 interface 0 "Avocent Keyboard/Mouse
Function" rev 2.00/0.00 addr 3
uhidev0: iclass 3/1
ukbd0 at uhidev0: 8 variable keys, 6 key codes
wskbd0 at ukbd0: console keyboard, using wsdisplay0
uhidev1 at uhub3 port 1 configuration 1 interface 1 "Avocent Keyboard/Mouse
Function" rev 2.00/0.00 addr 3
uhidev1: iclass 3/1
ums0 at uhidev1: 3 buttons, Z dir
wsmouse0 at ums0 mux 0
uhidev2 at uhub3 port 1 configuration 1 interface 2 "Avocent Keyboard/Mouse
Function" rev 2.00/0.00 addr 3
uhidev2: iclass 3/1
ums1 at uhidev2: 3 buttons, Z dir
wsmouse1 at ums1 mux 0
carp: pfsync0 demoted group carp by 32 to 160 (pfsync init)
carp: pfsync0 demoted group pfsync by 32 to 32 (pfsync init)
carp: pfsync0 demoted group carp by 1 to 161 (pfsync bulk start)
carp: pfsync0 demoted group pfsync by 1 to 33 (pfsync bulk start)
carp124: state transition: BACKUP -> MASTER
carp131: state transition: BACKUP -> MASTER
carp133: state transition: BACKUP -> MASTER
carp134: state transition: BACKUP -> MASTER
carp139: state transition: BACKUP -> MASTER
carp163: state transition: BACKUP -> MASTER
carp17: state transition: BACKUP -> MASTER
carp82: state transition: BACKUP -> MASTER
carp83: state transition: BACKUP -> MASTER
carp93: state transition: BACKUP -> MASTER
carp124: state transition: MASTER -> BACKUP
carp131: state transition: MASTER -> BACKUP
carp133: state transition: MASTER -> BACKUP
carp163: state transition: MASTER -> BACKUP
carp17: state transition: MASTER -> BACKUP
carp83: state transition: MASTER -> BACKUP
carp134: state transition: MASTER -> BACKUP
carp139: state transition: MASTER -> BACKUP
carp82: state transition: MASTER -> BACKUP
carp93: state transition: MASTER -> BACKUP
carp: pfsync0 demoted group carp by -1 to 160 (pfsync bulk fail)
carp: pfsync0 demoted group pfsync by -1 to 32 (pfsync bulk fail)
carp: pfsync0 demoted group carp by -32 to 128 (pfsync init)
carp: pfsync0 demoted group pfsync by -32 to 0 (pfsync init)
pfsync: failed to receive bulk update
carp124: state transition: BACKUP -> MASTER
carp131: state transition: BACKUP -> MASTER
carp133: state transition: BACKUP -> MASTER
carp163: state transition: BACKUP -> MASTER
carp17: state transition: BACKUP -> MASTER
carp83: state transition: BACKUP -> MASTER
carp134: state transition: BACKUP -> MASTER
carp139: state transition: BACKUP -> MASTER
carp82: state transition: BACKUP -> MASTER
carp93: state transition: BACKUP -> MASTER
carp: pfsync0 demoted group carp by 1 to 1 (pfsync link state down)
carp: pfsync0 demoted group pfsync by 1 to 1 (pfsync link state down)
carp: pfsync0 demoted group carp by -1 to 0 (pfsync bulk cancelled)
carp: pfsync0 demoted group pfsync by -1 to 0 (pfsync bulk cancelled)
carp: pfsync0 demoted group carp by -1 to 0 (pfsync bulk cancelled)
carp: pfsync0 demoted group pfsync by -1 to 0 (pfsync bulk cancelled)
carp: pfsync0 demoted group carp by -1 to 0 (pfsync bulk cancelled)
carp: pfsync0 demoted group pfsync by -1 to 0 (pfsync bulk cancelled)
carp: pfsync0 demoted group carp by -1 to 0 (pfsync link state up)
carp: pfsync0 demoted group pfsync by -1 to 0 (pfsync link state up)
pfsync: failed to receive bulk update
carp: pfsync0 demoted group carp by 1 to 1 (pfsync link state down)
carp: pfsync0 demoted group pfsync by 1 to 1 (pfsync link state down)
carp: pfsync0 demoted group carp by -1 to 0 (pfsync bulk cancelled)
carp: pfsync0 demoted group pfsync by -1 to 0 (pfsync bulk cancelled)
carp: pfsync0 demoted group carp by -1 to 0 (pfsync bulk cancelled)
carp: pfsync0 demoted group pfsync by -1 to 0 (pfsync bulk cancelled)
carp: pfsync0 demoted group carp by -1 to 0 (pfsync link state up)
carp: pfsync0 demoted group pfsync by -1 to 0 (pfsync link state up)
pfsync: failed to receive bulk update
carp: carp163 demoted group carp by 1 to 1 (carpdev)
carp: carp163 demoted group vpn by 1 to 1 (carpdev)
carp124: state transition: MASTER -> BACKUP
carp: carp163 demoted group carp by -1 to 0 (detach)
carp: carp163 demoted group vpn by -1 to 0 (detach)
carp124: state transition: BACKUP -> MASTER
panic: kernel diagnostic assertion "sc->sc_carpdev != NULL" failed: file
"../../../../netinet/ip_carp.c", line 2312
Starting stack trace...
panic() at panic+0x10b
__assert() at __assert+0x25
carp_output() at carp_output+0x106
ip_output() at ip_output+0x69d
ip_forward() at ip_forward+0x1ab
ipv4_input() at ipv4_input+0x35b
ipintr() at ipintr+0x1e
if_netisr() at if_netisr+0x105
taskq_thread() at taskq_thread+0x6c
end trace frame: 0x0, count: 248
End of stack trace.
syncing disks... 13 12 2 done
panic: sleep: softnet failed insomnia
Starting stack trace...
panic() at panic+0x10b
sleep_setup() at sleep_setup+0xff
sched_barrier() at sched_barrier+0x9a
ixgbe_stop() at ixgbe_stop+0x9a
ixgbe_ioctl() at ixgbe_ioctl+0x16c
if_downall() at if_downall+0x8a
boot() at boot+0xe4
reboot() at reboot+0x26
panic() at panic+0x106
__assert() at __assert+0x25
carp_output() at carp_output+0x106
ip_output() at ip_output+0x69d
ip_forward() at ip_forward+0x1ab
ipv4_input() at ipv4_input+0x35b
ipintr() at ipintr+0x1e
if_netisr() at if_netisr+0x105
taskq_thread() at taskq_thread+0x6c
end trace frame: 0x0, count: 240
End of stack trace.
panic: sleep: softnet failed insomnia
Starting stack trace...
panic() at panic+0x10b
sleep_setup() at sleep_setup+0xff
sched_barrier() at sched_barrier+0x9a
ixgbe_stop() at ixgbe_stop+0x9a
ixgbe_ioctl() at ixgbe_ioctl+0x16c
if_downall() at if_downall+0x8a
boot() at boot+0xe4
reboot() at reboot+0x26
panic() at panic+0x106
sleep_setup() at sleep_setup+0xff
sched_barrier() at sched_barrier+0x9a
ixgbe_stop() at ixgbe_stop+0x9a
ixgbe_ioctl() at ixgbe_ioctl+0x16c
if_downall() at if_downall+0x8a
boot() at boot+0xe4
reboot() at reboot+0x26
panic() at panic+0x106
__assert() at __assert+0x25
carp_output() at carp_output+0x106
ip_output() at ip_output+0x69d
ip_forward() at ip_forward+0x1ab
ipv4_input() at ipv4_input+0x35b
ipintr() at ipintr+0x1e
if_netisr() at if_netisr+0x105
taskq_thread() at taskq_thread+0x6c
end trace frame: 0x0, count: 232
End of stack trace.
panic: sleep: softnet failed insomnia
Starting stack trace...
panic() at panic+0x10b
sleep_setup() at sleep_setup+0xff
sched_barrier() at sched_barrier+0x9a
bge_stop() at bge_stop+0x6e
bge_ioctl() at bge_ioctl+0x206
if_downall() at if_downall+0x8a
boot() at boot+0xe4
reboot() at reboot+0x26
panic() at panic+0x106
sleep_setup() at sleep_setup+0xff
sched_barrier() at sched_barrier+0x9a
ixgbe_stop() at ixgbe_stop+0x9a
ixgbe_ioctl() at ixgbe_ioctl+0x16c
if_downall() at if_downall+0x8a
boot() at boot+0xe4
reboot() at reboot+0x26
panic() at panic+0x106
sleep_setup() at sleep_setup+0xff
sched_barrier() at sched_barrier+0x9a
ixgbe_stop() at ixgbe_stop+0x9a
ixgbe_ioctl() at ixgbe_ioctl+0x16c
if_downall() at if_downall+0x8a
boot() at boot+0xe4
reboot() at reboot+0x26
panic() at panic+0x106
__assert() at __assert+0x25
carp_output() at carp_output+0x106
ip_output() at ip_output+0x69d
ip_forward() at ip_forward+0x1ab
ipv4_input() at ipv4_input+0x35b
ipintr() at ipintr+0x1e
if_netisr() at if_netisr+0x105
taskq_thread() at taskq_thread+0x6c
end trace frame: 0x0, count: 224
End of stack trace.
panic: sleep: softnet failed insomnia
Starting stack trace...
panic() at panic+0x10b
sleep_setup() at sleep_setup+0xff
sched_barrier() at sched_barrier+0x9a
bge_stop() at bge_stop+0x6e
bge_ioctl() at bge_ioctl+0x206
if_downall() at if_downall+0x8a
boot() at boot+0xe4
reboot() at reboot+0x26
panic() at panic+0x106
sleep_setup() at sleep_setup+0xff
sched_barrier() at sched_barrier+0x9a
bge_stop() at bge_stop+0x6e
bge_ioctl() at bge_ioctl+0x206
if_downall() at if_downall+0x8a
boot() at boot+0xe4
reboot() at reboot+0x26
panic() at panic+0x106
sleep_setup() at sleep_setup+0xff
sched_barrier() at sched_barrier+0x9a
ixgbe_stop() at ixgbe_stop+0x9a
ixgbe_ioctl() at ixgbe_ioctl+0x16c
if_downall() at if_downall+0x8a
boot() at boot+0xe4
reboot() at reboot+0x26
panic() at panic+0x106
sleep_setup() at sleep_setup+0xff
sched_barrier() at sched_barrier+0x9a
ixgbe_stop() at ixgbe_stop+0x9a
ixgbe_ioctl() at ixgbe_ioctl+0x16c
if_downall() at if_downall+0x8a
boot() at boot+0xe4
reboot() at reboot+0x26
panic() at panic+0x106
__assert() at __assert+0x25
carp_output() at carp_output+0x106
ip_output() at ip_output+0x69d
ip_forward() at ip_forward+0x1ab
ipv4_input() at ipv4_input+0x35b
ipintr() at ipintr+0x1e
if_netisr() at if_netisr+0x105
taskq_thread() at taskq_thread+0x6c
end trace frame: 0x0, count: 216
End of stack trace.
carp: carp124 demoted group carp by 1 to 1 (carpdev)
carp: carp131 demoted group carp by 1 to 2 (carpdev)
carp: carp131 demoted group mgmt by 1 to 1 (carpdev)
carp: carp133 demoted group carp by 1 to 3 (carpdev)
carp: carp134 demoted group carp by 1 to 4 (carpdev)
carp: carp139 demoted group carp by 1 to 5 (carpdev)
carp: carp17 demoted group carp by 1 to 6 (carpdev)
carp: carp17 demoted group internet by 1 to 1 (carpdev)
carp: carp82 demoted group carp by 1 to 7 (carpdev)
carp: carp82 demoted group intra-router by 1 to 1 (carpdev)
carp: carp83 demoted group carp by 1 to 8 (carpdev)
carp: carp83 demoted group intra-router by 1 to 2 (carpdev)
carp: carp83 demoted group van by 1 to 1 (carpdev)
carp: carp93 demoted group carp by 1 to 9 (carpdev)
carp: carp93 demoted group intra-router by 1 to 3 (carpdev)
carp: carp93 demoted group tor by 1 to 1 (carpdev)
dumping to dev 4,1 offset 492575
dump 8022 8021 8020 8019 8018 8017 8016 8015 8014 8013 8012 8011 8010 8009
8008 8007 8006 8005 8004 8003 8002 8001 8000 7999 7998
[...]
--
Brent Graveland
[email protected]
