Hi
I'm seeing random udp package drops on OpenBSD version 7.3,
that are not present on 7.2. I tested this with dns queries,
but it might not be limited to dns protocol.
I have reproduced this problem in 2 different systems - production router
running on dell FC430 hardware and a clean OpenBSD 7.3 install on
testing environment running on Virtualbox VM-s. To reproduce the problem,
i run a simple dns query with dig in a 500-interation loop.
During a single run, i usually get timeout at least once. I don't believe
i overload the router or run into any other limits, as the forwarding test
is
the only load on the environment and amount of packets is is insignificant -
single dns query every 0.2 seconds.
Environment is following:
dns client (10.10.11.3) > vlan5(10.10.11.2)-Router-em2(nat-to 10.10.10.2)
> dns server (10.10.10.1)
Problem occurs randomly somewhere around nat. I can see package
entering the router, also pass in and nat-to rule matches from pflog.
I'm not seeing the package exiting router. Every time problem occurs,
pf state-insert counter is incremented. I'm not able to reproduce the
problem
without nat-to or when i use tcp instead of udp.
pf.conf is following:
set skip on lo
block log all
pass in log on vlan5 inet tag ACCEPTED
pass in log inet proto tcp to (self) port 22
pass out log on em2 tagged ACCEPTED nat-to (em2)
This is tcpdump capture of a problem from em2 (that is also parent
interface for vlan5) - note the retransmission of the first package
after dig timeout (first package is not exiting router on em2):
03:36:02.807194 08:00:27:a3:eb:09 08:00:27:52:ed:50 8100 81: 802.1Q vid 5
pri 3 10.10.11.3.17189 > 10.10.10.1.53: 24319+ [1au] A? rt.lan.(35)
03:36:07.810308 08:00:27:a3:eb:09 08:00:27:52:ed:50 8100 81: 802.1Q vid 5
pri 3 10.10.11.3.17189 > 10.10.10.1.53: 24319+ [1au] A? rt.lan.(35)
03:36:07.810528 08:00:27:52:ed:50 08:00:27:6c:b0:65 0800 77:
10.10.10.2.57351 > 10.10.10.1.53: 24319+ [1au] A? rt.lan.(35)
03:36:07.810911 08:00:27:6c:b0:65 08:00:27:52:ed:50 0800 93: 10.10.10.1.53
> 10.10.10.2.57351: 24319* 1/0/1 A 192.168.1.195(51)
03:36:07.810944 08:00:27:52:ed:50 08:00:27:a3:eb:09 8100 97: 802.1Q vid 5
pri 3 10.10.10.1.53 > 10.10.11.3.17189: 24319* 1/0/1 A 192.168.1.195(51)
The same event from the pflog interface - pass out nat-to rule matches
on the original and retransmitted package, pass in does not:
03:36:02.807245 rule 1/(match) pass in on vlan5: 10.10.11.3.17189 >
10.10.10.1.53: 24319+ [1au] A? rt.lan.(35)
03:36:02.807277 rule 3/(match) pass out on em2: 10.10.11.3.17189 >
10.10.10.1.53: 24319+ [1au] A? rt.lan.(35)
03:36:07.810487 rule 3/(match) pass out on em2: 10.10.11.3.17189 >
10.10.10.1.53: 24319+ [1au] A? rt.lan.(35)
When looking through the pf debug-level logs, 2 events stand up - first one
has no problems, second one fails.Note that source port 53081 for nat exist
in
2 different events (this particular test run was 66 dns requests long
on an otherwise idle system):
May 8 03:35:53 lab73 /bsd: pf: key search, in on vlan5: UDP wire: (0)
10.10.11.3:25217 10.10.10.1:53
May 8 03:35:53 lab73 /bsd: pf: key setup: UDP wire: (0) 10.10.11.3:25217
10.10.10.1:53 stack: (0) -
May 8 03:35:53 lab73 /bsd: pf: key search, out on em2: UDP wire: (0)
10.10.10.1:53 10.10.11.3:25217
May 8 03:35:53 lab73 /bsd: pf: pf_map_addr: selected address 10.10.10.2
May 8 03:35:53 lab73 /bsd: pf: key setup: UDP wire: (0) 10.10.10.1:53
10.10.10.2:53081 stack: (0) 10.10.10.1:53 10.10.11.3:25217
May 8 03:35:53 lab73 /bsd: pf: key search, in on em2: UDP wire: (0)
10.10.10.1:53 10.10.10.2:53081
May 8 03:35:53 lab73 /bsd: pf: key search, out on vlan5: UDP wire: (0)
10.10.11.3:25217 10.10.10.1:53
.....
May 8 03:36:02 lab73 /bsd: pf: key search, in on vlan5: UDP wire: (0)
10.10.11.3:17189 10.10.10.1:53
May 8 03:36:02 lab73 /bsd: pf: key setup: UDP wire: (0) 10.10.11.3:17189
10.10.10.1:53 stack: (0) -
May 8 03:36:02 lab73 /bsd: pf: key search, out on em2: UDP wire: (0)
10.10.10.1:53 10.10.11.3:17189
May 8 03:36:02 lab73 /bsd: pf: pf_map_addr: selected address 10.10.10.2
May 8 03:36:02 lab73 /bsd: pf: key setup: UDP wire: (0) 10.10.10.1:53
10.10.10.2:53081 stack: (0) 10.10.10.1:53 10.10.11.3:17189
May 8 03:36:02 lab73 /bsd: pf: wire key attach failed on all: UDP out
wire: (0) 10.10.10.1:53 10.10.10.2:53081 1:0 @3, existing: UDP out wire:
(0) 10.10.10.1:53 10.10.10.2:53081 stack: (0) 10.10.10.1:53 10.10.11.3:25217
2:1 @3
May 8 03:36:07 lab73 /bsd: pf: key search, in on vlan5: UDP wire: (0)
10.10.11.3:17189 10.10.10.1:53
May 8 03:36:07 lab73 /bsd: pf: key search, out on em2: UDP wire: (0)
10.10.10.1:53 10.10.11.3:17189
May 8 03:36:07 lab73 /bsd: pf: pf_map_addr: selected address 10.10.10.2
May 8 03:36:07 lab73 /bsd: pf: key setup: UDP wire: (0) 10.10.10.1:53
10.10.10.2:57351 stack: (0) 10.10.10.1:53 10.10.11.3:17189
May 8 03:36:07 lab73 /bsd: pf: key search, in on em2: UDP wire: (0)
10.10.10.1:53 10.10.10.2:57351
May 8 03:36:07 lab73 /bsd: pf: key search, out on vlan5: UDP wire: (0)
10.10.11.3:17189 10.10.10.1:53
It seems to me (i might be wrong), that system selects source ports
for nat in a way, that makes collisions likely.
PF information:
lab73# pfctl -si
Status: Enabled for 0 days 12:52:28 Debug: err
State Table Total Rate
current entries 2
half-open tcp 0
searches 116673 2.5/s
inserts 3578 0.1/s
removals 3576 0.1/s
Counters
match 15802 0.3/s
bad-offset 0 0.0/s
fragment 0 0.0/s
short 0 0.0/s
normalize 0 0.0/s
memory 0 0.0/s
bad-timestamp 0 0.0/s
congestion 0 0.0/s
ip-option 0 0.0/s
proto-cksum 0 0.0/s
state-mismatch 0 0.0/s
state-insert 9 0.0/s
state-limit 0 0.0/s
src-limit 0 0.0/s
synproxy 0 0.0/s
translate 0 0.0/s
no-route 0 0.0/s
Dmesg from test system:
OpenBSD 7.3 (GENERIC) #1072: Sat Mar 25 10:26:08 MDT 2023
dera...@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC
real mem = 1056899072 (1007MB)
avail mem = 1005633536 (959MB)
random: good seed from bootblocks
mpath0 at root
scsibus0 at mpath0: 256 targets
mainbus0 at root
bios0 at mainbus0: SMBIOS rev. 2.5 @ 0xe1000 (10 entries)
bios0: vendor innotek GmbH version "VirtualBox" date 12/01/2006
bios0: innotek GmbH VirtualBox
acpi0 at bios0: ACPI 4.0
acpi0: sleep states S0 S5
acpi0: tables DSDT FACP APIC SSDT
acpi0: wakeup devices
acpitimer0 at acpi0: 3579545 Hz, 32 bits
acpimadt0 at acpi0 addr 0xfee00000: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: 12th Gen Intel(R) Core(TM) i5-1235U, 2504.67 MHz, 06-9a-04
cpu0:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,HTT,SSE3,PCLMUL,MWAIT,SSSE3,CX16,PCID,SSE4.1,SSE4.2,MOVBE,POPCNT,AES,XSAVE,AVX,RDRAND,NXE,RDTSCP,LONG,LAHF,ABM,3DNOWP,ITSC,FSGSBASE,B
MI1,AVX2,BMI2,INVPCID,RDSEED,CLFLUSHOPT,MD_CLEAR,L1DF,MELTDOWN
cpu0: 48KB 64b/line 12-way D-cache, 32KB 64b/line 8-way I-cache, 1MB
64b/line 10-way L2 cache, 12MB 64b/line 12-way L3 cache
cpu0: smt 0, core 0, package 0
mtrr: CPU supports MTRRs but not enabled by BIOS
cpu0: apic clock running at 1003MHz
cpu0: mwait min=64, max=64
ioapic0 at mainbus0: apid 1 pa 0xfec00000, version 20, 24 pins, remapped
acpiprt0 at acpi0: bus 0 (PCI0)
acpipci0 at acpi0 PCI0: 0x00000000 0x00000011 0x00000001
acpibat0 at acpi0: BAT0 model "1" serial 0 type VBOX oem "innotek"
acpiac0 at acpi0: AC unit online
acpicpu0 at acpi0: C1(@1 halt!)
acpivideo0 at acpi0: GFX0
cpu0: using VERW MDS workaround (except on vmm entry)
pci0 at mainbus0 bus 0
pchb0 at pci0 dev 0 function 0 "Intel 82441FX" rev 0x02
pcib0 at pci0 dev 1 function 0 "Intel 82371SB ISA" rev 0x00
pciide0 at pci0 dev 1 function 1 "Intel 82371AB IDE" rev 0x01: DMA, channel
0 configured to compatibility, channel 1 configured to compatibility
wd0 at pciide0 channel 0 drive 0: <VBOX HARDDISK>
wd0: 128-sector PIO, LBA, 16384MB, 33554432 sectors
wd0(pciide0:0:0): using PIO mode 4, Ultra-DMA mode 2
atapiscsi0 at pciide0 channel 1 drive 0
scsibus1 at atapiscsi0: 2 targets
cd0 at scsibus1 targ 0 lun 0: <VBOX, CD-ROM, 1.0> removable
cd0(pciide0:1:0): using PIO mode 4, Ultra-DMA mode 2
vga1 at pci0 dev 2 function 0 "VMware SVGA II" rev 0x00
wsdisplay0 at vga1 mux 1: console (80x25, vt100 emulation)
wsdisplay0: screen 1-5 added (80x25, vt100 emulation)
em0 at pci0 dev 3 function 0 "Intel 82540EM" rev 0x02: apic 1 int 19,
address 08:00:27:29:f9:df
"InnoTek Guest Service" rev 0x00 at pci0 dev 4 function 0 not configured
auich0 at pci0 dev 5 function 0 "Intel 82801AA AC97" rev 0x01: apic 1 int
21, ICH
ac97: codec id 0x83847600 (SigmaTel STAC9700)
audio0 at auich0
ohci0 at pci0 dev 6 function 0 "Apple Intrepid USB" rev 0x00: apic 1 int
22, version 1.0
piixpm0 at pci0 dev 7 function 0 "Intel 82371AB Power" rev 0x08: apic 1 int
23
iic0 at piixpm0
em1 at pci0 dev 8 function 0 "Intel 82540EM" rev 0x02: apic 1 int 16,
address 08:00:27:50:fc:5b
em2 at pci0 dev 9 function 0 "Intel 82540EM" rev 0x02: apic 1 int 17,
address 08:00:27:52:ed:50
ehci0 at pci0 dev 11 function 0 "Intel 82801FB USB" rev 0x00: apic 1 int 19
usb0 at ehci0: USB revision 2.0
uhub0 at usb0 configuration 1 interface 0 "Intel EHCI root hub" rev
2.00/1.00 addr 1
isa0 at pcib0
isadma0 at isa0
pckbc0 at isa0 port 0x60/5 irq 1 irq 12
pckbd0 at pckbc0 (kbd slot)
wskbd0 at pckbd0: console keyboard, using wsdisplay0
pms0 at pckbc0 (aux slot)
wsmouse0 at pms0 mux 0
pcppi0 at isa0 port 0x61
spkr0 at pcppi0
usb1 at ohci0: USB revision 1.0
uhub1 at usb1 configuration 1 interface 0 "Apple OHCI root hub" rev
1.00/1.00 addr 1
vscsi0 at root
scsibus2 at vscsi0: 256 targets
softraid0 at root
scsibus3 at softraid0: 256 targets
root on wd0a (0e31354297303b72.a) swap on wd0b dump on wd0b
Dmesg from production system:
OpenBSD 7.3 (GENERIC.MP) #1125: Sat Mar 25 10:36:29 MDT 2023
dera...@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP
real mem = 68604694528 (65426MB)
avail mem = 66506113024 (63425MB)
random: good seed from bootblocks
mpath0 at root
scsibus0 at mpath0: 256 targets
mainbus0 at root
bios0 at mainbus0: SMBIOS rev. 2.8 @ 0x7af09000 (57 entries)
bios0: vendor Dell Inc. version "2.7.1" date 01/23/2018
bios0: Dell Inc. PowerEdge FC430
acpi0 at bios0: ACPI 4.0
acpi0: sleep states S0 S5
acpi0: tables DSDT FACP MCEJ SSDT TPM2 WD__ SLIC HPET APIC MCFG MSCT SLIT
SRAT SSDT SSDT SSDT PRAD SPCR DMAR HEST BERT ERST EINJ
acpi0: wakeup devices PCI0(S4) BR1A(S4) BR1B(S4) BR2A(S4) BR2C(S4) BR3A(S4)
BR3C(S4) XHC_(S0) RP02(S4) RP03(S4) RP05(S4) RP08(S4) QRP0(S4) QR1A(S4)
QR2A(S4) QR2C(S4) [...]
acpitimer0 at acpi0: 3579545 Hz, 24 bits
acpihpet0 at acpi0: 14318179 Hz
acpimadt0 at acpi0 addr 0xfee00000: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: Intel(R) Xeon(R) CPU E5-2623 v4 @ 2.60GHz, 2800.02 MHz, 06-4f-01
cpu0:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,MO
VBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,3DNOWP,PERF,ITSC,FSGSBASE,TSC_ADJUST,BMI1,HLE,AVX2,SMEP,BMI2,ERMS,INVPCID,RTM,PQM,RDSEED,ADX,SMAP,PT,MD_CLEAR,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,
MELTDOWN
cpu0: 32KB 64b/line 8-way D-cache, 32KB 64b/line 8-way I-cache, 256KB
64b/line 8-way L2 cache, 10MB 64b/line 20-way L3 cache
cpu0: smt 0, core 0, package 0
mtrr: Pentium Pro MTRR support, 10 var ranges, 88 fixed ranges
cpu0: apic clock running at 100MHz
cpu0: mwait min=64, max=64, C-substates=0.2.1.2, IBE
cpu1 at mainbus0: apid 2 (application processor)
cpu1: Intel(R) Xeon(R) CPU E5-2623 v4 @ 2.60GHz, 2800.05 MHz, 06-4f-01
cpu1:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,MO
VBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,3DNOWP,PERF,ITSC,FSGSBASE,TSC_ADJUST,BMI1,HLE,AVX2,SMEP,BMI2,ERMS,INVPCID,RTM,PQM,RDSEED,ADX,SMAP,PT,MD_CLEAR,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,
MELTDOWN
cpu1: 32KB 64b/line 8-way D-cache, 32KB 64b/line 8-way I-cache, 256KB
64b/line 8-way L2 cache, 10MB 64b/line 20-way L3 cache
cpu1: smt 0, core 1, package 0
cpu2 at mainbus0: apid 4 (application processor)
cpu2: Intel(R) Xeon(R) CPU E5-2623 v4 @ 2.60GHz, 2800.09 MHz, 06-4f-01
cpu2:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,MO
VBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,3DNOWP,PERF,ITSC,FSGSBASE,TSC_ADJUST,BMI1,HLE,AVX2,SMEP,BMI2,ERMS,INVPCID,RTM,PQM,RDSEED,ADX,SMAP,PT,MD_CLEAR,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,
MELTDOWN
cpu2: 32KB 64b/line 8-way D-cache, 32KB 64b/line 8-way I-cache, 256KB
64b/line 8-way L2 cache, 10MB 64b/line 20-way L3 cache
cpu2: smt 0, core 2, package 0
cpu3 at mainbus0: apid 6 (application processor)
cpu3: Intel(R) Xeon(R) CPU E5-2623 v4 @ 2.60GHz, 2800.16 MHz, 06-4f-01
cpu3:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,MO
VBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,3DNOWP,PERF,ITSC,FSGSBASE,TSC_ADJUST,BMI1,HLE,AVX2,SMEP,BMI2,ERMS,INVPCID,RTM,PQM,RDSEED,ADX,SMAP,PT,MD_CLEAR,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,
MELTDOWN
cpu3: 32KB 64b/line 8-way D-cache, 32KB 64b/line 8-way I-cache, 256KB
64b/line 8-way L2 cache, 10MB 64b/line 20-way L3 cache
cpu3: smt 0, core 3, package 0
cpu4 at mainbus0: apid 1 (application processor)
cpu4: Intel(R) Xeon(R) CPU E5-2623 v4 @ 2.60GHz, 2800.12 MHz, 06-4f-01
cpu4:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,MO
VBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,3DNOWP,PERF,ITSC,FSGSBASE,TSC_ADJUST,BMI1,HLE,AVX2,SMEP,BMI2,ERMS,INVPCID,RTM,PQM,RDSEED,ADX,SMAP,PT,MD_CLEAR,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,
MELTDOWN
cpu4: 32KB 64b/line 8-way D-cache, 32KB 64b/line 8-way I-cache, 256KB
64b/line 8-way L2 cache, 10MB 64b/line 20-way L3 cache
cpu4: smt 1, core 0, package 0
cpu5 at mainbus0: apid 3 (application processor)
cpu5: Intel(R) Xeon(R) CPU E5-2623 v4 @ 2.60GHz, 2800.12 MHz, 06-4f-01
cpu5:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,MO
VBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,3DNOWP,PERF,ITSC,FSGSBASE,TSC_ADJUST,BMI1,HLE,AVX2,SMEP,BMI2,ERMS,INVPCID,RTM,PQM,RDSEED,ADX,SMAP,PT,MD_CLEAR,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,
MELTDOWN
cpu5: 32KB 64b/line 8-way D-cache, 32KB 64b/line 8-way I-cache, 256KB
64b/line 8-way L2 cache, 10MB 64b/line 20-way L3 cache
cpu5: smt 1, core 1, package 0
cpu6 at mainbus0: apid 5 (application processor)
cpu6: Intel(R) Xeon(R) CPU E5-2623 v4 @ 2.60GHz, 2800.15 MHz, 06-4f-01
cpu6:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,MO
VBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,3DNOWP,PERF,ITSC,FSGSBASE,TSC_ADJUST,BMI1,HLE,AVX2,SMEP,BMI2,ERMS,INVPCID,RTM,PQM,RDSEED,ADX,SMAP,PT,MD_CLEAR,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,
MELTDOWN
cpu6: 32KB 64b/line 8-way D-cache, 32KB 64b/line 8-way I-cache, 256KB
64b/line 8-way L2 cache, 10MB 64b/line 20-way L3 cache
cpu6: smt 1, core 2, package 0
cpu7 at mainbus0: apid 7 (application processor)
cpu7: Intel(R) Xeon(R) CPU E5-2623 v4 @ 2.60GHz, 2800.26 MHz, 06-4f-01
cpu7:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,MO
VBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,3DNOWP,PERF,ITSC,FSGSBASE,TSC_ADJUST,BMI1,HLE,AVX2,SMEP,BMI2,ERMS,INVPCID,RTM,PQM,RDSEED,ADX,SMAP,PT,MD_CLEAR,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,
MELTDOWN
cpu7: 32KB 64b/line 8-way D-cache, 32KB 64b/line 8-way I-cache, 256KB
64b/line 8-way L2 cache, 10MB 64b/line 20-way L3 cache
cpu7: smt 1, core 3, package 0
ioapic0 at mainbus0: apid 8 pa 0xfec00000, version 20, 24 pins
ioapic1 at mainbus0: apid 9 pa 0xfec01000, version 20, 24 pins
acpimadt0: unknown apic structure type 7f
acpimadt0: unknown apic structure type 7f
acpimadt0: unknown apic structure type 7f
acpimcfg0 at acpi0
acpimcfg0: addr 0x80000000, bus 0-255
acpiprt0 at acpi0: bus 255 (UNC0)
acpiprt1 at acpi0: bus 0 (PCI0)
acpiprt2 at acpi0: bus 1 (BR1A)
acpiprt3 at acpi0: bus -1 (BR1B)
acpiprt4 at acpi0: bus -1 (BR2A)
acpiprt5 at acpi0: bus -1 (BR2C)
acpiprt6 at acpi0: bus 2 (BR3A)
acpiprt7 at acpi0: bus -1 (BR3C)
acpiprt8 at acpi0: bus -1 (RP02)
acpiprt9 at acpi0: bus -1 (RP03)
acpiprt10 at acpi0: bus -1 (RP05)
acpiprt11 at acpi0: bus 12 (RP08)
acpipci0 at acpi0 UNC0: 0x00000010 0x00000011 0x00000000
"ACPI0004" at acpi0 not configured
"PNP0C33" at acpi0 not configured
"PNP0C14" at acpi0 not configured
acpipci1 at acpi0 PCI0: 0x00000000 0x00000011 0x00000001
"PNP0003" at acpi0 not configured
"PNP0C14" at acpi0 not configured
acpicmos0 at acpi0
com0 at acpi0 COMA addr 0x3f8/0x8 irq 4: ns16550a, 16 byte fifo
com0: console
com1 at acpi0 COMB addr 0x2f8/0x8 irq 3: ns16550a, 16 byte fifo
acpicpu0 at acpi0: C1(1000@1 mwait.1)
acpicpu1 at acpi0: C1(1000@1 mwait.1)
acpicpu2 at acpi0: C1(1000@1 mwait.1)
acpicpu3 at acpi0: C1(1000@1 mwait.1)
acpicpu4 at acpi0: C1(1000@1 mwait.1)
acpicpu5 at acpi0: C1(1000@1 mwait.1)
acpicpu6 at acpi0: C1(1000@1 mwait.1)
acpicpu7 at acpi0: C1(1000@1 mwait.1)
ipmi at mainbus0 not configured
cpu0: using VERW MDS workaround (except on vmm entry)
pci0 at mainbus0 bus 255
"Intel Xeon-D QPI Link" rev 0x01 at pci0 dev 11 function 0 not configured
"Intel Xeon-D QPI Link" rev 0x01 at pci0 dev 11 function 1 not configured
"Intel Xeon-D QPI Link" rev 0x01 at pci0 dev 11 function 2 not configured
"Intel Xeon-D QPI Debug" rev 0x01 at pci0 dev 11 function 3 not configured
"Intel Xeon-D Cache" rev 0x01 at pci0 dev 12 function 0 not configured
"Intel Xeon-D Cache" rev 0x01 at pci0 dev 12 function 1 not configured
"Intel Xeon-D Cache" rev 0x01 at pci0 dev 12 function 2 not configured
"Intel Xeon-D Cache" rev 0x01 at pci0 dev 12 function 3 not configured
"Intel Xeon-D Cache" rev 0x01 at pci0 dev 15 function 0 not configured
"Intel E5 v4 Cache" rev 0x01 at pci0 dev 15 function 1 not configured
"Intel Xeon-D Cache" rev 0x01 at pci0 dev 15 function 4 not configured
"Intel Xeon-D Cache" rev 0x01 at pci0 dev 15 function 5 not configured
"Intel Xeon-D Cache" rev 0x01 at pci0 dev 15 function 6 not configured
"Intel Xeon-D PCIE" rev 0x01 at pci0 dev 16 function 0 not configured
"Intel E5 v4 R2PCIe Agent" rev 0x01 at pci0 dev 16 function 1 not configured
"Intel Xeon-D Ubox" rev 0x01 at pci0 dev 16 function 5 not configured
"Intel Xeon-D Ubox" rev 0x01 at pci0 dev 16 function 6 not configured
"Intel Xeon-D Ubox" rev 0x01 at pci0 dev 16 function 7 not configured
"Intel Xeon-D Home Agent" rev 0x01 at pci0 dev 18 function 0 not configured
"Intel Xeon-D Home Agent" rev 0x01 at pci0 dev 18 function 1 not configured
vendor "Intel", unknown product 0x6f70 (class system subclass
miscellaneous, rev 0x01) at pci0 dev 18 function 2 not configured
"Intel Xeon-D Memory" rev 0x01 at pci0 dev 19 function 0 not configured
"Intel Xeon-D Memory" rev 0x01 at pci0 dev 19 function 1 not configured
"Intel Xeon-D Memory" rev 0x01 at pci0 dev 19 function 2 not configured
"Intel Xeon-D Memory" rev 0x01 at pci0 dev 19 function 3 not configured
"Intel Xeon-D Memory" rev 0x01 at pci0 dev 19 function 4 not configured
"Intel Xeon-D Memory" rev 0x01 at pci0 dev 19 function 5 not configured
"Intel Xeon-D Memory" rev 0x01 at pci0 dev 19 function 6 not configured
"Intel Xeon-D Memory" rev 0x01 at pci0 dev 19 function 7 not configured
"Intel Xeon-D Memory" rev 0x01 at pci0 dev 20 function 0 not configured
"Intel Xeon-D Memory" rev 0x01 at pci0 dev 20 function 1 not configured
"Intel Xeon-D Memory" rev 0x01 at pci0 dev 20 function 2 not configured
"Intel Xeon-D Memory" rev 0x01 at pci0 dev 20 function 3 not configured
"Intel Xeon-D Memory" rev 0x01 at pci0 dev 20 function 4 not configured
"Intel Xeon-D Memory" rev 0x01 at pci0 dev 20 function 5 not configured
"Intel Xeon-D Memory" rev 0x01 at pci0 dev 20 function 6 not configured
"Intel Xeon-D Memory" rev 0x01 at pci0 dev 20 function 7 not configured
"Intel Xeon-D Memory" rev 0x01 at pci0 dev 21 function 0 not configured
"Intel Xeon-D Memory" rev 0x01 at pci0 dev 21 function 1 not configured
"Intel Xeon-D Memory" rev 0x01 at pci0 dev 21 function 2 not configured
"Intel Xeon-D Memory" rev 0x01 at pci0 dev 21 function 3 not configured
"Intel E5 v4 RAS" rev 0x01 at pci0 dev 22 function 0 not configured
"Intel E5 v4 DDRIO" rev 0x01 at pci0 dev 22 function 6 not configured
"Intel E5 v4 DDRIO" rev 0x01 at pci0 dev 22 function 7 not configured
"Intel E5 v4 Thermal" rev 0x01 at pci0 dev 23 function 0 not configured
"Intel E5 v4 DDRIO" rev 0x01 at pci0 dev 23 function 4 not configured
"Intel E5 v4 DDRIO" rev 0x01 at pci0 dev 23 function 5 not configured
"Intel E5 v4 DDRIO" rev 0x01 at pci0 dev 23 function 6 not configured
"Intel E5 v4 DDRIO" rev 0x01 at pci0 dev 23 function 7 not configured
"Intel Xeon-D PCU" rev 0x01 at pci0 dev 30 function 0 not configured
"Intel Xeon-D PCU" rev 0x01 at pci0 dev 30 function 1 not configured
"Intel Xeon-D PCU" rev 0x01 at pci0 dev 30 function 2 not configured
"Intel Xeon-D PCU" rev 0x01 at pci0 dev 30 function 3 not configured
"Intel Xeon-D PCU" rev 0x01 at pci0 dev 30 function 4 not configured
"Intel Xeon-D PCU" rev 0x01 at pci0 dev 31 function 0 not configured
"Intel Xeon-D PCU" rev 0x01 at pci0 dev 31 function 2 not configured
pci1 at mainbus0 bus 0
pchb0 at pci1 dev 0 function 0 "Intel Xeon-D Host" rev 0x01
ppb0 at pci1 dev 1 function 0 "Intel Xeon-D PCIE" rev 0x01
pci2 at ppb0 bus 1
1:0:0: rom address conflict 0xfff80000/0x80000
1:0:1: rom address conflict 0xfff80000/0x80000
"Broadcom BCM57810" rev 0x10 at pci2 dev 0 function 0 not configured
"Broadcom BCM57810" rev 0x10 at pci2 dev 0 function 1 not configured
ppb1 at pci1 dev 3 function 0 "Intel Xeon-D PCIE" rev 0x01
pci3 at ppb1 bus 2
ppb2 at pci3 dev 0 function 0 "PLX PEX 8617" rev 0xca
pci4 at ppb2 bus 3
ppb3 at pci4 dev 1 function 0 "PLX PEX 8617" rev 0xca
pci5 at ppb3 bus 4
ppb4 at pci5 dev 0 function 0 "PLX PEX 8780" rev 0xab
pci6 at ppb4 bus 5
ppb5 at pci6 dev 12 function 0 "PLX PEX 8780" rev 0xab: msi
pci7 at ppb5 bus 6
ix0 at pci7 dev 0 function 0 "Intel 82599" rev 0x01, msix, 4 queues,
address b4:96:91:1c:95:b8
ix1 at pci7 dev 0 function 1 "Intel 82599" rev 0x01, msix, 4 queues,
address b4:96:91:1c:95:ba
ppb6 at pci6 dev 16 function 0 "PLX PEX 8780" rev 0xab: msi
pci8 at ppb6 bus 7
ppb7 at pci8 dev 0 function 0 "IDT 89HPES12N3A" rev 0x0e
pci9 at ppb7 bus 8
ppb8 at pci9 dev 2 function 0 "IDT 89HPES12N3A" rev 0x0e
pci10 at ppb8 bus 9
em0 at pci10 dev 0 function 0 "Intel 82571EB" rev 0x06: apic 9 int 16,
address 00:1c:c4:47:07:c9
em1 at pci10 dev 0 function 1 "Intel 82571EB" rev 0x06: apic 9 int 22,
address 00:1c:c4:47:07:c8
ppb9 at pci9 dev 4 function 0 "IDT 89HPES12N3A" rev 0x0e
pci11 at ppb9 bus 10
em2 at pci11 dev 0 function 0 "Intel 82571EB" rev 0x06: apic 9 int 21,
address 00:1c:c4:47:07:cb
em3 at pci11 dev 0 function 1 "Intel 82571EB" rev 0x06: apic 9 int 20,
address 00:1c:c4:47:07:ca
"Intel Xeon-D Address Map" rev 0x01 at pci1 dev 5 function 0 not configured
"Intel Xeon-D Hot Plug" rev 0x01 at pci1 dev 5 function 1 not configured
"Intel Xeon-D RAS" rev 0x01 at pci1 dev 5 function 2 not configured
"Intel Xeon-D I/O APIC" rev 0x01 at pci1 dev 5 function 4 not configured
"Intel C610 MS SPSR" rev 0x05 at pci1 dev 17 function 0 not configured
ahci0 at pci1 dev 17 function 4 "Intel C610 AHCI" rev 0x05: msi, AHCI 1.3
ahci0: port 0: 6.0Gb/s
ahci0: port 1: 6.0Gb/s
scsibus1 at ahci0: 32 targets
sd0 at scsibus1 targ 0 lun 0: <ATA, THNSF8400CAME, > naa.500080d910d12447
sd0: 381554MB, 512 bytes/sector, 781422768 sectors, thin
sd1 at scsibus1 targ 1 lun 0: <ATA, INTEL SSDSC1BG40, G201>
naa.55cd2e404c045fbb
sd1: 381554MB, 512 bytes/sector, 781422768 sectors, thin
xhci0 at pci1 dev 20 function 0 "Intel C610 xHCI" rev 0x05: msi, xHCI 1.0
usb0 at xhci0: USB revision 3.0
uhub0 at usb0 configuration 1 interface 0 "Intel xHCI root hub" rev
3.00/1.00 addr 1
"Intel C610 MEI" rev 0x05 at pci1 dev 22 function 0 not configured
"Intel C610 MEI" rev 0x05 at pci1 dev 22 function 1 not configured
ehci0 at pci1 dev 26 function 0 "Intel C610 USB" rev 0x05: apic 8 int 18
usb1 at ehci0: USB revision 2.0
uhub1 at usb1 configuration 1 interface 0 "Intel EHCI root hub" rev
2.00/1.00 addr 1
ppb10 at pci1 dev 28 function 0 "Intel C610 PCIE" rev 0xd5
pci12 at ppb10 bus 11
ppb11 at pci1 dev 28 function 7 "Intel C610 PCIE" rev 0xd5: msi
pci13 at ppb11 bus 12
ppb12 at pci13 dev 0 function 0 "Renesas SH7758 PCIE Switch" rev 0x00
pci14 at ppb12 bus 13
ppb13 at pci14 dev 0 function 0 "Renesas SH7758 PCIE Switch" rev 0x00
pci15 at ppb13 bus 14
ppb14 at pci15 dev 0 function 0 "Renesas SH7758 PCIE-PCI" rev 0x00
pci16 at ppb14 bus 15
vga1 at pci16 dev 0 function 0 "Matrox MGA G200eR" rev 0x01
wsdisplay at vga1 not configured
ehci1 at pci1 dev 29 function 0 "Intel C610 USB" rev 0x05: apic 8 int 18
usb2 at ehci1: USB revision 2.0
uhub2 at usb2 configuration 1 interface 0 "Intel EHCI root hub" rev
2.00/1.00 addr 1
pcib0 at pci1 dev 31 function 0 "Intel C610 LPC" rev 0x05
ahci1 at pci1 dev 31 function 2 "Intel C610 AHCI" rev 0x05: msi, AHCI 1.3
scsibus2 at ahci1: 32 targets
isa0 at pcib0
isadma0 at isa0
pcppi0 at isa0 port 0x61
spkr0 at pcppi0
vmm0 at mainbus0: VMX/EPT
uhub3 at uhub0 port 12 configuration 1 interface 0 "no manufacturer Gadget
USB HUB" rev 2.00/0.00 addr 2
uhidev0 at uhub3 port 1 configuration 1 interface 0 "Avocent Keyboard/Mouse
Function" rev 2.00/0.00 addr 3
uhidev0: iclass 3/1
ukbd0 at uhidev0: 8 variable keys, 6 key codes
wskbd0 at ukbd0 mux 1
uhidev1 at uhub3 port 1 configuration 1 interface 1 "Avocent Keyboard/Mouse
Function" rev 2.00/0.00 addr 3
uhidev1: iclass 3/1
ums0 at uhidev1: 3 buttons, Z dir
wsmouse0 at ums0 mux 0
uhidev2 at uhub3 port 1 configuration 1 interface 2 "Avocent Keyboard/Mouse
Function" rev 2.00/0.00 addr 3
uhidev2: iclass 3/1
ums1 at uhidev2: 3 buttons, Z dir
wsmouse1 at ums1 mux 0
uhub4 at uhub1 port 1 configuration 1 interface 0 "Intel Rate Matching Hub"
rev 2.00/0.05 addr 2
uhub5 at uhub2 port 1 configuration 1 interface 0 "Intel Rate Matching Hub"
rev 2.00/0.05 addr 2
vscsi0 at root
scsibus3 at vscsi0: 256 targets
softraid0 at root
scsibus4 at softraid0: 256 targets
sd2 at scsibus4 targ 1 lun 0: <OPENBSD, SR RAID 1, 006>
sd2: 381553MB, 512 bytes/sector, 781422176 sectors
root on sd2a (a5d5ac3c3b882b85.a) swap on sd2b dump on sd2b
Kind Regards
Joosep
