Dear List Members,
I have a weird problem, when I try to ssh to an OpenBSD server. (I use
OpenBSD 7.3 with GENERIC.MP #1125 kernel.)
I perform benchmarking tests to measure the performance of OpenBSD PF.
I use the below test setup:
2001:2::[0000-ffff]:2/64 198.19.0.0/15 - 198.19.255.254/15
\ +--------------------------------------+ /
IPv6 \ |Initiator Responder| /
+-------------| Tester |<------------+
| addresses | [state table]| public IPv4 |
| +--------------------------------------+ |
| |
| +--------------------------------------+ |
| 2001:2::1/64| DUT: | public IPv4 |
+------------>| Stateful NAT64 gateway |-------------+
IPv6 address | [connection tracking table] | \
+--------------------------------------+ \
198.18.0.1/15
(As for the actual tests, I use only sub-ranges from the potential IP
address ranges shown above.)
The Tester is executed on a Linux server. During my tests, a bash shell
script (running on the Linux server) executes various commands on the
DUT (Device Under Test), which is the OpenBSD server. To that end, I use
ssh with key based authentication. Usually everything goes well, but
after a while, things "go wrong", and I cannot ssh from the Linux server
to the OpenBSD server any more. I get the following error message:
root@tester:~/siitperf# ssh 172.16.17.102
ssh_exchange_identification: read: Connection reset by peer
root@tester:~/siitperf#
Then I even cannot ssh from the OpenBSD server to itself:
dut# ssh localhost
getsockname failed: Connection reset by peer
banner exchange: Connection to 127.0.0.1 port -1: Broken pipe
dut# ssh 172.16.17.102
getsockname failed: Connection reset by peer
banner exchange: Connection to UNKNOWN port -1: Broken pipe
dut#
To be able to perform the tests, I set various things by my scripts, and
perhaps one of them could be the culprit, but I cannot find it. I
execute the scripts in the /root/DUT-settings directory of the OpenBSD
server from the bash shell script running on the tester using ssh. The
relevant scripts are:
dut# pwd
/root/DUT-settings
dut# cat set-nat64-varip # this one sets static NDP and ARP entries
/root/DUT-settings/set-ndm-left 0 3999
/root/DUT-settings/set-arpm-right 2 1001
dut# cat set-ndm-left
for i in $(seq $1 $2)
do
h=$(printf "%x" $i)
ndp -s 2001:2::$h:2 24:6e:96:3c:3f:40 permanent
done
dut# cat set-ndm-right
for i in $(seq $1 $2)
do
h=$(printf "%x" $i)
ndp -s 2001:2:0:8000::$h:2 24:6e:96:3c:3f:42 permanent
done
dut# cat set-pf
pfctl -f /etc/pf-set-nat64
dut# cat /etc/pf-set-nat64
# $OpenBSD: pf.conf,v 1.55 2017/12/03 20:40:04 sthen Exp $
#
# See pf.conf(5) and /etc/examples/pf.conf
set skip on lo
block return # block stateless traffic
pass # establish keep-state
# By default, do not permit remote connections to X11
block return in on ! lo0 proto tcp to port 6000:6010
# Port build user does not need network
block return out log proto {tcp udp} user _pbuild
set skip on em1 # to protect ssh
set limit states 1000000000 # 1000M
set timeout interval 3600 # 1 hour
pass in on ix0 inet6 from any to 64:ff9b::/96 af-to inet from 198.19.0.1
dut#
When everything is set, then the test follows. I have two kinds of tests.
1) Maximum connection establishment rate test. It sends 4M test frames
with all different source IP address and destination IP address
combinations to establish 4M connections. The test uses a binary search
to find the highest rate at which all connections are established. (In
fact it is not checked. What is checked, is that all test frames arrive
back the the Tester.)
2) Throughput test. First, the 4M connections are loaded into the
connection tracking table of PF. Then comes the throughput test with
bidirectional traffic. One elementary test last for 60s. A binary search
is used to find the highest rate at which all frames are forwarded.
In the case of both tests, I reboot the DUT after each elementary step
of the binary search. Its aim is to completely clear the connection
tracking table of PF. And, IMHO, it should put the OpenBSD server into a
well defined, clear state. After which, it should behave the in the same
way, every time.
And now come the weird things. The maximum connection establishment rate
test was successful. The binary search was executed 10 times without any
problem. As for the throughput test, the binary search was done ones
fully. (It means 9 steps.)
Here is the first result:
No, Size, Dir, n, m, Duration, Initial Rate, N, M, R, T, D, Error, Date,
Iterations needed, rate
1, 84, b, 2, 2, 60, 200000, 4000000, 4000000, 80000, 500, 51000, 1000,
2023-09-03 18:23:27, 9, 361718
root@tester:~/siitperf#
And when the binary search was executed the second time, it stopped
working after the second iteration. This is the relevant part from the
nohup.out file:
Preliminary frames received: 4000000
Info: Preliminary phase finished.
Info: Testing initiated at 2023-09-03 18:31:35
Info: Forward sender's sending took 59.9999967862 seconds.
Forward frames sent: 18000000
Info: Reverse sender's sending took 59.9999967887 seconds.
Reverse frames sent: 18000000
Forward frames received: 17726784
Reverse frames received: 17782805
Info: Test finished.
Rebooting the DUT and then waiting for 240 seconds...
ssh_exchange_identification: read: Connection reset by peer^M
Done.
The script waited for 240s and then continued the work, but from this
point it could never ssh to the DUT again, and thus all its further
results are rubbish...
Some more information: I could execute the throughput test without any
problem, when I used only a single IP address pair and 4M different port
number combinations. This makes me think that perhaps the usage of the
high number of IP addresses (4000 static NDP and 1000 static ARP entries
are set) could cause the problem? But I reboot the system after every
single step. Why it does not have a clear state then?
Could there be some random event?
Did I make a mistake in pf.conf? -- I am not familiar with PF, so it has
a chance, too!
Could you please advise me?
Thank you very much in advance!
Best regards,
Gábor
p.s.: Although I do not suspect any hardware problem, I have attached
the dmesg of the DUT.
OpenBSD 7.3 (GENERIC.MP) #1125: Sat Mar 25 10:36:29 MDT 2023
dera...@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP
real mem = 137324171264 (130962MB)
avail mem = 133142933504 (126974MB)
random: good seed from bootblocks
mpath0 at root
scsibus0 at mpath0: 256 targets
mainbus0 at root
bios0 at mainbus0: SMBIOS rev. 2.8 @ 0x7af09000 (81 entries)
bios0: vendor Dell Inc. version "2.16.0" date 07/20/2022
bios0: Dell Inc. PowerEdge R730
acpi0 at bios0: ACPI 4.0
acpi0: sleep states S0 S5
acpi0: tables DSDT FACP MCEJ WD__ SLIC HPET APIC MCFG MSCT SLIT SRAT SSDT SSDT
SSDT PRAD DMAR HEST BERT ERST EINJ
acpi0: wakeup devices PCI0(S4) BR1A(S4) BR1B(S4) BR2A(S4) BR2B(S4) BR2C(S4)
BR2D(S4) BR3A(S4) BR3B(S4) BR3C(S4) BR3D(S4) XHC_(S0) RP02(S4) RP03(S4)
RP05(S4) RP08(S4) [...]
acpitimer0 at acpi0: 3579545 Hz, 24 bits
acpihpet0 at acpi0: 14318179 Hz
acpimadt0 at acpi0 addr 0xfee00000: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: Intel(R) Xeon(R) CPU E5-2667 v3 @ 3.20GHz, 3400.03 MHz, 06-3f-02
cpu0:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,PERF,ITSC,FSGSBASE,TSC_ADJUST,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,PQM,MD_CLEAR,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
cpu0: 32KB 64b/line 8-way D-cache, 32KB 64b/line 8-way I-cache, 256KB 64b/line
8-way L2 cache, 20MB 64b/line 20-way L3 cache
cpu0: smt 0, core 0, package 0
mtrr: Pentium Pro MTRR support, 10 var ranges, 88 fixed ranges
cpu0: apic clock running at 99MHz
cpu0: mwait min=64, max=64, C-substates=0.2.1.2, IBE
cpu1 at mainbus0: apid 16 (application processor)
cpu1: Intel(R) Xeon(R) CPU E5-2667 v3 @ 3.20GHz, 3400.06 MHz, 06-3f-02
cpu1:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,PERF,ITSC,FSGSBASE,TSC_ADJUST,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,PQM,MD_CLEAR,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
cpu1: 32KB 64b/line 8-way D-cache, 32KB 64b/line 8-way I-cache, 256KB 64b/line
8-way L2 cache, 20MB 64b/line 20-way L3 cache
cpu1: smt 0, core 0, package 1
cpu2 at mainbus0: apid 2 (application processor)
cpu2: Intel(R) Xeon(R) CPU E5-2667 v3 @ 3.20GHz, 3400.06 MHz, 06-3f-02
cpu2:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,PERF,ITSC,FSGSBASE,TSC_ADJUST,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,PQM,MD_CLEAR,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
cpu2: 32KB 64b/line 8-way D-cache, 32KB 64b/line 8-way I-cache, 256KB 64b/line
8-way L2 cache, 20MB 64b/line 20-way L3 cache
cpu2: smt 0, core 1, package 0
cpu3 at mainbus0: apid 18 (application processor)
cpu3: Intel(R) Xeon(R) CPU E5-2667 v3 @ 3.20GHz, 3400.06 MHz, 06-3f-02
cpu3:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,PERF,ITSC,FSGSBASE,TSC_ADJUST,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,PQM,MD_CLEAR,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
cpu3: 32KB 64b/line 8-way D-cache, 32KB 64b/line 8-way I-cache, 256KB 64b/line
8-way L2 cache, 20MB 64b/line 20-way L3 cache
cpu3: smt 0, core 1, package 1
cpu4 at mainbus0: apid 4 (application processor)
cpu4: Intel(R) Xeon(R) CPU E5-2667 v3 @ 3.20GHz, 3400.16 MHz, 06-3f-02
cpu4:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,PERF,ITSC,FSGSBASE,TSC_ADJUST,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,PQM,MD_CLEAR,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
cpu4: 32KB 64b/line 8-way D-cache, 32KB 64b/line 8-way I-cache, 256KB 64b/line
8-way L2 cache, 20MB 64b/line 20-way L3 cache
cpu4: smt 0, core 2, package 0
cpu5 at mainbus0: apid 20 (application processor)
cpu5: Intel(R) Xeon(R) CPU E5-2667 v3 @ 3.20GHz, 3400.15 MHz, 06-3f-02
cpu5:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,PERF,ITSC,FSGSBASE,TSC_ADJUST,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,PQM,MD_CLEAR,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
cpu5: 32KB 64b/line 8-way D-cache, 32KB 64b/line 8-way I-cache, 256KB 64b/line
8-way L2 cache, 20MB 64b/line 20-way L3 cache
cpu5: smt 0, core 2, package 1
cpu6 at mainbus0: apid 6 (application processor)
cpu6: Intel(R) Xeon(R) CPU E5-2667 v3 @ 3.20GHz, 3400.17 MHz, 06-3f-02
cpu6:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,PERF,ITSC,FSGSBASE,TSC_ADJUST,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,PQM,MD_CLEAR,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
cpu6: 32KB 64b/line 8-way D-cache, 32KB 64b/line 8-way I-cache, 256KB 64b/line
8-way L2 cache, 20MB 64b/line 20-way L3 cache
cpu6: smt 0, core 3, package 0
cpu7 at mainbus0: apid 22 (application processor)
cpu7: Intel(R) Xeon(R) CPU E5-2667 v3 @ 3.20GHz, 3400.14 MHz, 06-3f-02
cpu7:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,PERF,ITSC,FSGSBASE,TSC_ADJUST,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,PQM,MD_CLEAR,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
cpu7: 32KB 64b/line 8-way D-cache, 32KB 64b/line 8-way I-cache, 256KB 64b/line
8-way L2 cache, 20MB 64b/line 20-way L3 cache
cpu7: smt 0, core 3, package 1
cpu8 at mainbus0: apid 8 (application processor)
cpu8: Intel(R) Xeon(R) CPU E5-2667 v3 @ 3.20GHz, 3400.32 MHz, 06-3f-02
cpu8:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,PERF,ITSC,FSGSBASE,TSC_ADJUST,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,PQM,MD_CLEAR,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
cpu8: 32KB 64b/line 8-way D-cache, 32KB 64b/line 8-way I-cache, 256KB 64b/line
8-way L2 cache, 20MB 64b/line 20-way L3 cache
cpu8: smt 0, core 4, package 0
cpu9 at mainbus0: apid 24 (application processor)
cpu9: Intel(R) Xeon(R) CPU E5-2667 v3 @ 3.20GHz, 3400.25 MHz, 06-3f-02
cpu9:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,PERF,ITSC,FSGSBASE,TSC_ADJUST,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,PQM,MD_CLEAR,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
cpu9: 32KB 64b/line 8-way D-cache, 32KB 64b/line 8-way I-cache, 256KB 64b/line
8-way L2 cache, 20MB 64b/line 20-way L3 cache
cpu9: smt 0, core 4, package 1
cpu10 at mainbus0: apid 10 (application processor)
cpu10: Intel(R) Xeon(R) CPU E5-2667 v3 @ 3.20GHz, 3400.25 MHz, 06-3f-02
cpu10:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,PERF,ITSC,FSGSBASE,TSC_ADJUST,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,PQM,MD_CLEAR,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
cpu10: 32KB 64b/line 8-way D-cache, 32KB 64b/line 8-way I-cache, 256KB 64b/line
8-way L2 cache, 20MB 64b/line 20-way L3 cache
cpu10: smt 0, core 5, package 0
cpu11 at mainbus0: apid 26 (application processor)
cpu11: Intel(R) Xeon(R) CPU E5-2667 v3 @ 3.20GHz, 3400.25 MHz, 06-3f-02
cpu11:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,PERF,ITSC,FSGSBASE,TSC_ADJUST,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,PQM,MD_CLEAR,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
cpu11: 32KB 64b/line 8-way D-cache, 32KB 64b/line 8-way I-cache, 256KB 64b/line
8-way L2 cache, 20MB 64b/line 20-way L3 cache
cpu11: smt 0, core 5, package 1
cpu12 at mainbus0: apid 12 (application processor)
cpu12: Intel(R) Xeon(R) CPU E5-2667 v3 @ 3.20GHz, 3400.39 MHz, 06-3f-02
cpu12:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,PERF,ITSC,FSGSBASE,TSC_ADJUST,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,PQM,MD_CLEAR,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
cpu12: 32KB 64b/line 8-way D-cache, 32KB 64b/line 8-way I-cache, 256KB 64b/line
8-way L2 cache, 20MB 64b/line 20-way L3 cache
cpu12: smt 0, core 6, package 0
cpu13 at mainbus0: apid 28 (application processor)
cpu13: Intel(R) Xeon(R) CPU E5-2667 v3 @ 3.20GHz, 3400.31 MHz, 06-3f-02
cpu13:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,PERF,ITSC,FSGSBASE,TSC_ADJUST,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,PQM,MD_CLEAR,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
cpu13: 32KB 64b/line 8-way D-cache, 32KB 64b/line 8-way I-cache, 256KB 64b/line
8-way L2 cache, 20MB 64b/line 20-way L3 cache
cpu13: smt 0, core 6, package 1
cpu14 at mainbus0: apid 14 (application processor)
cpu14: Intel(R) Xeon(R) CPU E5-2667 v3 @ 3.20GHz, 3400.29 MHz, 06-3f-02
cpu14:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,PERF,ITSC,FSGSBASE,TSC_ADJUST,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,PQM,MD_CLEAR,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
cpu14: 32KB 64b/line 8-way D-cache, 32KB 64b/line 8-way I-cache, 256KB 64b/line
8-way L2 cache, 20MB 64b/line 20-way L3 cache
cpu14: smt 0, core 7, package 0
cpu15 at mainbus0: apid 30 (application processor)
cpu15: Intel(R) Xeon(R) CPU E5-2667 v3 @ 3.20GHz, 3400.28 MHz, 06-3f-02
cpu15:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,PERF,ITSC,FSGSBASE,TSC_ADJUST,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,PQM,MD_CLEAR,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
cpu15: 32KB 64b/line 8-way D-cache, 32KB 64b/line 8-way I-cache, 256KB 64b/line
8-way L2 cache, 20MB 64b/line 20-way L3 cache
cpu15: smt 0, core 7, package 1
ioapic0 at mainbus0: apid 8 pa 0xfec00000, version 20, 24 pins
ioapic1 at mainbus0: apid 9 pa 0xfec01000, version 20, 24 pins
ioapic2 at mainbus0: apid 10 pa 0xfec40000, version 20, 24 pins
acpimadt0: unknown apic structure type 7f
acpimadt0: unknown apic structure type 7f
acpimcfg0 at acpi0
acpimcfg0: addr 0x80000000, bus 0-255
acpiprt0 at acpi0: bus 255 (UNC1)
acpiprt1 at acpi0: bus 127 (UNC0)
acpiprt2 at acpi0: bus 0 (PCI0)
acpiprt3 at acpi0: bus 2 (BR1A)
acpiprt4 at acpi0: bus -1 (BR1B)
acpiprt5 at acpi0: bus 3 (BR2A)
acpiprt6 at acpi0: bus -1 (BR2B)
acpiprt7 at acpi0: bus 4 (BR2C)
acpiprt8 at acpi0: bus -1 (BR2D)
acpiprt9 at acpi0: bus 1 (BR3A)
acpiprt10 at acpi0: bus -1 (BR3B)
acpiprt11 at acpi0: bus 5 (BR3C)
acpiprt12 at acpi0: bus -1 (BR3D)
acpiprt13 at acpi0: bus -1 (RP02)
acpiprt14 at acpi0: bus -1 (RP03)
acpiprt15 at acpi0: bus 7 (RP05)
acpiprt16 at acpi0: bus 8 (RP08)
acpiprt17 at acpi0: bus 128 (PCI1)
acpiprt18 at acpi0: bus -1 (QRP0)
acpiprt19 at acpi0: bus 129 (QR1A)
acpiprt20 at acpi0: bus -1 (QR1B)
acpiprt21 at acpi0: bus 130 (QR2A)
acpiprt22 at acpi0: bus -1 (QR2B)
acpiprt23 at acpi0: bus -1 (QR2C)
acpiprt24 at acpi0: bus -1 (QR2D)
acpiprt25 at acpi0: bus 131 (QR3A)
acpiprt26 at acpi0: bus -1 (QR3B)
acpiprt27 at acpi0: bus 132 (QR3C)
acpiprt28 at acpi0: bus -1 (QR3D)
acpipci0 at acpi0 UNC1: 0x00000000 0x00000011 0x00000001
acpipci1 at acpi0 UNC0: 0x00000010 0x00000011 0x00000000
"ACPI0004" at acpi0 not configured
"ACPI0004" at acpi0 not configured
"PNP0C33" at acpi0 not configured
"PNP0C14" at acpi0 not configured
acpipci2 at acpi0 PCI0: 0x00000000 0x00000011 0x00000001
"PNP0003" at acpi0 not configured
"PNP0C14" at acpi0 not configured
acpicmos0 at acpi0
com1 at acpi0 COMA addr 0x2f8/0x8 irq 3: ns16550a, 16 byte fifo
com0 at acpi0 COMB addr 0x3f8/0x8 irq 4: ns16550a, 16 byte fifo
acpipci3 at acpi0 PCI1: 0x00000000 0x00000011 0x00000001
acpicpu0 at acpi0: C1(1000@1 mwait.1)
acpicpu1 at acpi0: C1(1000@1 mwait.1)
acpicpu2 at acpi0: C1(1000@1 mwait.1)
acpicpu3 at acpi0: C1(1000@1 mwait.1)
acpicpu4 at acpi0: C1(1000@1 mwait.1)
acpicpu5 at acpi0: C1(1000@1 mwait.1)
acpicpu6 at acpi0: C1(1000@1 mwait.1)
acpicpu7 at acpi0: C1(1000@1 mwait.1)
acpicpu8 at acpi0: C1(1000@1 mwait.1)
acpicpu9 at acpi0: C1(1000@1 mwait.1)
acpicpu10 at acpi0: C1(1000@1 mwait.1)
acpicpu11 at acpi0: C1(1000@1 mwait.1)
acpicpu12 at acpi0: C1(1000@1 mwait.1)
acpicpu13 at acpi0: C1(1000@1 mwait.1)
acpicpu14 at acpi0: C1(1000@1 mwait.1)
acpicpu15 at acpi0: C1(1000@1 mwait.1)
ipmi at mainbus0 not configured
cpu0: using VERW MDS workaround (except on vmm entry)
pci0 at mainbus0 bus 255
255:30:3: mem address conflict 0x20/0x20
"Intel E5 v3 QPI" rev 0x02 at pci0 dev 8 function 0 not configured
"Intel E5 v3 QPI" rev 0x02 at pci0 dev 8 function 2 not configured
"Intel E5 v3 QPI" rev 0x02 at pci0 dev 8 function 3 not configured
"Intel E5 v3 QPI" rev 0x02 at pci0 dev 9 function 0 not configured
"Intel E5 v3 QPI" rev 0x02 at pci0 dev 9 function 2 not configured
"Intel E5 v3 QPI" rev 0x02 at pci0 dev 9 function 3 not configured
"Intel E5 v3 QPI Monitor" rev 0x02 at pci0 dev 11 function 0 not configured
"Intel E5 v3 QPI Monitor" rev 0x02 at pci0 dev 11 function 1 not configured
"Intel E5 v3 QPI Monitor" rev 0x02 at pci0 dev 11 function 2 not configured
"Intel E5 v3 Unicast" rev 0x02 at pci0 dev 12 function 0 not configured
"Intel E5 v3 Unicast" rev 0x02 at pci0 dev 12 function 1 not configured
"Intel E5 v3 Unicast" rev 0x02 at pci0 dev 12 function 2 not configured
"Intel E5 v3 Unicast" rev 0x02 at pci0 dev 12 function 3 not configured
"Intel E5 v3 Unicast" rev 0x02 at pci0 dev 12 function 4 not configured
"Intel E5 v3 Unicast" rev 0x02 at pci0 dev 12 function 5 not configured
"Intel E5 v3 Unicast" rev 0x02 at pci0 dev 12 function 6 not configured
"Intel E5 v3 Unicast" rev 0x02 at pci0 dev 12 function 7 not configured
"Intel E5 v3 Ring Agent" rev 0x02 at pci0 dev 15 function 0 not configured
"Intel E5 v3 Ring Agent" rev 0x02 at pci0 dev 15 function 1 not configured
"Intel E5 v3 SAD" rev 0x02 at pci0 dev 15 function 4 not configured
"Intel E5 v3 SAD" rev 0x02 at pci0 dev 15 function 5 not configured
"Intel E5 v3 SAD" rev 0x02 at pci0 dev 15 function 6 not configured
"Intel E5 v3 PCIE Ring" rev 0x02 at pci0 dev 16 function 0 not configured
"Intel E5 v3 PCIE Monitor" rev 0x02 at pci0 dev 16 function 1 not configured
"Intel E5 v3 Scratch" rev 0x02 at pci0 dev 16 function 5 not configured
"Intel E5 v3 Scratch" rev 0x02 at pci0 dev 16 function 6 not configured
"Intel E5 v3 Scratch" rev 0x02 at pci0 dev 16 function 7 not configured
"Intel E5 v3 Home Agent" rev 0x02 at pci0 dev 18 function 0 not configured
"Intel E5 v3 Home Agent" rev 0x02 at pci0 dev 18 function 1 not configured
"Intel E5 v3 Home Agent Debug" rev 0x02 at pci0 dev 18 function 2 not configured
"Intel E5 v3 Memory" rev 0x02 at pci0 dev 19 function 0 not configured
"Intel E5 v3 RAS" rev 0x02 at pci0 dev 19 function 1 not configured
"Intel E5 v3 TAD" rev 0x02 at pci0 dev 19 function 2 not configured
"Intel E5 v3 TAD" rev 0x02 at pci0 dev 19 function 3 not configured
"Intel E5 v3 TAD" rev 0x02 at pci0 dev 19 function 4 not configured
"Intel E5 v3 TAD" rev 0x02 at pci0 dev 19 function 5 not configured
"Intel E5 v3 DDR Broadcast" rev 0x02 at pci0 dev 19 function 6 not configured
"Intel E5 v3 DDR Broadcast" rev 0x02 at pci0 dev 19 function 7 not configured
"Intel E5 v3 Thermal" rev 0x02 at pci0 dev 20 function 0 not configured
"Intel E5 v3 Thermal" rev 0x02 at pci0 dev 20 function 1 not configured
"Intel E5 v3 Error" rev 0x02 at pci0 dev 20 function 2 not configured
"Intel E5 v3 Error" rev 0x02 at pci0 dev 20 function 3 not configured
"Intel E5 v3 DDRIO" rev 0x02 at pci0 dev 20 function 4 not configured
"Intel E5 v3 DDRIO" rev 0x02 at pci0 dev 20 function 5 not configured
"Intel E5 v3 DDRIO" rev 0x02 at pci0 dev 20 function 6 not configured
"Intel E5 v3 DDRIO" rev 0x02 at pci0 dev 20 function 7 not configured
"Intel E5 v3 Thermal" rev 0x02 at pci0 dev 21 function 0 not configured
"Intel E5 v3 Thermal" rev 0x02 at pci0 dev 21 function 1 not configured
"Intel E5 v3 Error" rev 0x02 at pci0 dev 21 function 2 not configured
"Intel E5 v3 Error" rev 0x02 at pci0 dev 21 function 3 not configured
"Intel E5 v3 TA" rev 0x02 at pci0 dev 22 function 0 not configured
"Intel E5 v3 DDR Broadcast" rev 0x02 at pci0 dev 22 function 6 not configured
"Intel E5 v3 DDR Broadcast" rev 0x02 at pci0 dev 22 function 7 not configured
"Intel E5 v3 Thermal" rev 0x02 at pci0 dev 23 function 0 not configured
"Intel E5 v3 DDRIO" rev 0x02 at pci0 dev 23 function 4 not configured
"Intel E5 v3 DDRIO" rev 0x02 at pci0 dev 23 function 5 not configured
"Intel E5 v3 DDRIO" rev 0x02 at pci0 dev 23 function 6 not configured
"Intel E5 v3 DDRIO" rev 0x02 at pci0 dev 23 function 7 not configured
"Intel E5 v3 PCU" rev 0x02 at pci0 dev 30 function 0 not configured
"Intel E5 v3 PCU" rev 0x02 at pci0 dev 30 function 1 not configured
"Intel E5 v3 PCU" rev 0x02 at pci0 dev 30 function 2 not configured
"Intel E5 v3 PCU" rev 0x02 at pci0 dev 30 function 3 not configured
"Intel E5 v3 PCU" rev 0x02 at pci0 dev 30 function 4 not configured
"Intel E5 v3 VCU" rev 0x02 at pci0 dev 31 function 0 not configured
"Intel E5 v3 VCU" rev 0x02 at pci0 dev 31 function 2 not configured
pci1 at mainbus0 bus 127
127:30:3: mem address conflict 0x20/0x20
"Intel E5 v3 QPI" rev 0x02 at pci1 dev 8 function 0 not configured
"Intel E5 v3 QPI" rev 0x02 at pci1 dev 8 function 2 not configured
"Intel E5 v3 QPI" rev 0x02 at pci1 dev 8 function 3 not configured
"Intel E5 v3 QPI" rev 0x02 at pci1 dev 9 function 0 not configured
"Intel E5 v3 QPI" rev 0x02 at pci1 dev 9 function 2 not configured
"Intel E5 v3 QPI" rev 0x02 at pci1 dev 9 function 3 not configured
"Intel E5 v3 QPI Monitor" rev 0x02 at pci1 dev 11 function 0 not configured
"Intel E5 v3 QPI Monitor" rev 0x02 at pci1 dev 11 function 1 not configured
"Intel E5 v3 QPI Monitor" rev 0x02 at pci1 dev 11 function 2 not configured
"Intel E5 v3 Unicast" rev 0x02 at pci1 dev 12 function 0 not configured
"Intel E5 v3 Unicast" rev 0x02 at pci1 dev 12 function 1 not configured
"Intel E5 v3 Unicast" rev 0x02 at pci1 dev 12 function 2 not configured
"Intel E5 v3 Unicast" rev 0x02 at pci1 dev 12 function 3 not configured
"Intel E5 v3 Unicast" rev 0x02 at pci1 dev 12 function 4 not configured
"Intel E5 v3 Unicast" rev 0x02 at pci1 dev 12 function 5 not configured
"Intel E5 v3 Unicast" rev 0x02 at pci1 dev 12 function 6 not configured
"Intel E5 v3 Unicast" rev 0x02 at pci1 dev 12 function 7 not configured
"Intel E5 v3 Ring Agent" rev 0x02 at pci1 dev 15 function 0 not configured
"Intel E5 v3 Ring Agent" rev 0x02 at pci1 dev 15 function 1 not configured
"Intel E5 v3 SAD" rev 0x02 at pci1 dev 15 function 4 not configured
"Intel E5 v3 SAD" rev 0x02 at pci1 dev 15 function 5 not configured
"Intel E5 v3 SAD" rev 0x02 at pci1 dev 15 function 6 not configured
"Intel E5 v3 PCIE Ring" rev 0x02 at pci1 dev 16 function 0 not configured
"Intel E5 v3 PCIE Monitor" rev 0x02 at pci1 dev 16 function 1 not configured
"Intel E5 v3 Scratch" rev 0x02 at pci1 dev 16 function 5 not configured
"Intel E5 v3 Scratch" rev 0x02 at pci1 dev 16 function 6 not configured
"Intel E5 v3 Scratch" rev 0x02 at pci1 dev 16 function 7 not configured
"Intel E5 v3 Home Agent" rev 0x02 at pci1 dev 18 function 0 not configured
"Intel E5 v3 Home Agent" rev 0x02 at pci1 dev 18 function 1 not configured
"Intel E5 v3 Home Agent Debug" rev 0x02 at pci1 dev 18 function 2 not configured
"Intel E5 v3 Memory" rev 0x02 at pci1 dev 19 function 0 not configured
"Intel E5 v3 RAS" rev 0x02 at pci1 dev 19 function 1 not configured
"Intel E5 v3 TAD" rev 0x02 at pci1 dev 19 function 2 not configured
"Intel E5 v3 TAD" rev 0x02 at pci1 dev 19 function 3 not configured
"Intel E5 v3 TAD" rev 0x02 at pci1 dev 19 function 4 not configured
"Intel E5 v3 TAD" rev 0x02 at pci1 dev 19 function 5 not configured
"Intel E5 v3 DDR Broadcast" rev 0x02 at pci1 dev 19 function 6 not configured
"Intel E5 v3 DDR Broadcast" rev 0x02 at pci1 dev 19 function 7 not configured
"Intel E5 v3 Thermal" rev 0x02 at pci1 dev 20 function 0 not configured
"Intel E5 v3 Thermal" rev 0x02 at pci1 dev 20 function 1 not configured
"Intel E5 v3 Error" rev 0x02 at pci1 dev 20 function 2 not configured
"Intel E5 v3 Error" rev 0x02 at pci1 dev 20 function 3 not configured
"Intel E5 v3 DDRIO" rev 0x02 at pci1 dev 20 function 4 not configured
"Intel E5 v3 DDRIO" rev 0x02 at pci1 dev 20 function 5 not configured
"Intel E5 v3 DDRIO" rev 0x02 at pci1 dev 20 function 6 not configured
"Intel E5 v3 DDRIO" rev 0x02 at pci1 dev 20 function 7 not configured
"Intel E5 v3 Thermal" rev 0x02 at pci1 dev 21 function 0 not configured
"Intel E5 v3 Thermal" rev 0x02 at pci1 dev 21 function 1 not configured
"Intel E5 v3 Error" rev 0x02 at pci1 dev 21 function 2 not configured
"Intel E5 v3 Error" rev 0x02 at pci1 dev 21 function 3 not configured
"Intel E5 v3 TA" rev 0x02 at pci1 dev 22 function 0 not configured
"Intel E5 v3 DDR Broadcast" rev 0x02 at pci1 dev 22 function 6 not configured
"Intel E5 v3 DDR Broadcast" rev 0x02 at pci1 dev 22 function 7 not configured
"Intel E5 v3 Thermal" rev 0x02 at pci1 dev 23 function 0 not configured
"Intel E5 v3 DDRIO" rev 0x02 at pci1 dev 23 function 4 not configured
"Intel E5 v3 DDRIO" rev 0x02 at pci1 dev 23 function 5 not configured
"Intel E5 v3 DDRIO" rev 0x02 at pci1 dev 23 function 6 not configured
"Intel E5 v3 DDRIO" rev 0x02 at pci1 dev 23 function 7 not configured
"Intel E5 v3 PCU" rev 0x02 at pci1 dev 30 function 0 not configured
"Intel E5 v3 PCU" rev 0x02 at pci1 dev 30 function 1 not configured
"Intel E5 v3 PCU" rev 0x02 at pci1 dev 30 function 2 not configured
"Intel E5 v3 PCU" rev 0x02 at pci1 dev 30 function 3 not configured
"Intel E5 v3 PCU" rev 0x02 at pci1 dev 30 function 4 not configured
"Intel E5 v3 VCU" rev 0x02 at pci1 dev 31 function 0 not configured
"Intel E5 v3 VCU" rev 0x02 at pci1 dev 31 function 2 not configured
pci2 at mainbus0 bus 0
pchb0 at pci2 dev 0 function 0 "Intel E5 v3 Host" rev 0x02
ppb0 at pci2 dev 1 function 0 "Intel E5 v3 PCIE" rev 0x02
pci3 at ppb0 bus 2
mfii0 at pci3 dev 0 function 0 "Symbios Logic MegaRAID SAS3108" rev 0x02: msi
mfii0: "PERC H730 Mini", firmware 25.5.9.0001, 1024MB cache
scsibus1 at mfii0: 64 targets
sd0 at scsibus1 targ 1 lun 0: <DELL, PERC H730 Mini, 4.30>
naa.644a84201ef70e002bb4233c330e6c54
sd0: 953344MB, 512 bytes/sector, 1952448512 sectors
scsibus2 at mfii0: 256 targets
ppb1 at pci2 dev 2 function 0 "Intel E5 v3 PCIE" rev 0x02: msi
pci4 at ppb1 bus 3
ppb2 at pci2 dev 2 function 2 "Intel E5 v3 PCIE" rev 0x02: msi
pci5 at ppb2 bus 4
ppb3 at pci2 dev 3 function 0 "Intel E5 v3 PCIE" rev 0x02
pci6 at ppb3 bus 1
1:0:0: rom address conflict 0xfff80000/0x80000
1:0:1: rom address conflict 0xfff80000/0x80000
ix0 at pci6 dev 0 function 0 "Intel X540T" rev 0x01, msix, 16 queues, address
b8:ca:3a:5e:25:a8
ix1 at pci6 dev 0 function 1 "Intel X540T" rev 0x01, msix, 16 queues, address
b8:ca:3a:5e:25:aa
ppb4 at pci2 dev 3 function 2 "Intel E5 v3 PCIE" rev 0x02: msi
pci7 at ppb4 bus 5
ix2 at pci7 dev 0 function 0 "Intel 82599" rev 0x01, msix, 16 queues, address
a0:36:9f:43:dd:ec
ix3 at pci7 dev 0 function 1 "Intel 82599" rev 0x01, msix, 16 queues, address
a0:36:9f:43:dd:ee
"Intel E5 v3 Address Map" rev 0x02 at pci2 dev 5 function 0 not configured
"Intel E5 v3 Hot Plug" rev 0x02 at pci2 dev 5 function 1 not configured
"Intel E5 v3 Error Reporting" rev 0x02 at pci2 dev 5 function 2 not configured
"Intel E5 v3 I/O APIC" rev 0x02 at pci2 dev 5 function 4 not configured
"Intel C610 MS SPSR" rev 0x05 at pci2 dev 17 function 0 not configured
ahci0 at pci2 dev 17 function 4 "Intel C610 AHCI" rev 0x05: msi, AHCI 1.3
scsibus3 at ahci0: 32 targets
"Intel C610 MEI" rev 0x05 at pci2 dev 22 function 0 not configured
"Intel C610 MEI" rev 0x05 at pci2 dev 22 function 1 not configured
ehci0 at pci2 dev 26 function 0 "Intel C610 USB" rev 0x05: apic 8 int 18
usb0 at ehci0: USB revision 2.0
uhub0 at usb0 configuration 1 interface 0 "Intel EHCI root hub" rev 2.00/1.00
addr 1
ppb5 at pci2 dev 28 function 0 "Intel C610 PCIE" rev 0xd5
pci8 at ppb5 bus 6
ppb6 at pci2 dev 28 function 4 "Intel C610 PCIE" rev 0xd5: msi
pci9 at ppb6 bus 7
em0 at pci9 dev 0 function 0 "Intel I350" rev 0x01: msi, address
b8:ca:3a:5e:25:ac
em1 at pci9 dev 0 function 1 "Intel I350" rev 0x01: msi, address
b8:ca:3a:5e:25:ad
ppb7 at pci2 dev 28 function 7 "Intel C610 PCIE" rev 0xd5: msi
pci10 at ppb7 bus 8
ppb8 at pci10 dev 0 function 0 "Renesas SH7758 PCIE Switch" rev 0x00
pci11 at ppb8 bus 9
ppb9 at pci11 dev 0 function 0 "Renesas SH7758 PCIE Switch" rev 0x00
pci12 at ppb9 bus 10
ppb10 at pci12 dev 0 function 0 "Renesas SH7758 PCIE-PCI" rev 0x00
pci13 at ppb10 bus 11
vga1 at pci13 dev 0 function 0 "Matrox MGA G200eR" rev 0x01
wsdisplay0 at vga1 mux 1: console (80x25, vt100 emulation)
wsdisplay0: screen 1-5 added (80x25, vt100 emulation)
ehci1 at pci2 dev 29 function 0 "Intel C610 USB" rev 0x05: apic 8 int 18
usb1 at ehci1: USB revision 2.0
uhub1 at usb1 configuration 1 interface 0 "Intel EHCI root hub" rev 2.00/1.00
addr 1
pcib0 at pci2 dev 31 function 0 "Intel C610 LPC" rev 0x05
ahci1 at pci2 dev 31 function 2 "Intel C610 AHCI" rev 0x05: msi, AHCI 1.3
ahci1: port 5: 1.5Gb/s
scsibus4 at ahci1: 32 targets
cd0 at scsibus4 targ 5 lun 0: <HL-DT-ST, DVD+-RW GTA0N, A3B0> removable
isa0 at pcib0
isadma0 at isa0
pckbc0 at isa0 port 0x60/5 irq 1 irq 12
pcppi0 at isa0 port 0x61
spkr0 at pcppi0
pci14 at mainbus0 bus 128
ppb11 at pci14 dev 1 function 0 "Intel E5 v3 PCIE" rev 0x02: msi
pci15 at ppb11 bus 129
129:0:0: rom address conflict 0xfffc0000/0x40000
129:0:1: rom address conflict 0xfffc0000/0x40000
bnxt0 at pci15 dev 0 function 0 "Broadcom BCM57404" rev 0x01: fw ver 212.0.162,
msix, 8 queues, address 00:0a:f7:ae:a9:e6
bnxt1 at pci15 dev 0 function 1 "Broadcom BCM57404" rev 0x01: fw ver 212.0.162,
msix, 8 queues, address 00:0a:f7:ae:a9:e7
ppb12 at pci14 dev 2 function 0 "Intel E5 v3 PCIE" rev 0x02: msi
pci16 at ppb12 bus 130
ppb13 at pci14 dev 3 function 0 "Intel E5 v3 PCIE" rev 0x02: msi
pci17 at ppb13 bus 131
ppb14 at pci14 dev 3 function 2 "Intel E5 v3 PCIE" rev 0x02: msi
pci18 at ppb14 bus 132
"Intel E5 v3 Address Map" rev 0x02 at pci14 dev 5 function 0 not configured
"Intel E5 v3 Hot Plug" rev 0x02 at pci14 dev 5 function 1 not configured
"Intel E5 v3 Error Reporting" rev 0x02 at pci14 dev 5 function 2 not configured
"Intel E5 v3 I/O APIC" rev 0x02 at pci14 dev 5 function 4 not configured
vmm0 at mainbus0: VMX/EPT
uhub2 at uhub0 port 1 configuration 1 interface 0 "Intel Rate Matching Hub" rev
2.00/0.05 addr 2
umass0 at uhub2 port 1 configuration 1 interface 0 "BUFFALO USB Flash Disk" rev
2.10/1.10 addr 3
umass0: using SCSI over Bulk-Only
scsibus5 at umass0: 2 targets, initiator 0
sd1 at scsibus5 targ 1 lun 0: <BUFFALO, USB Flash Disk, 1.00> removable
serial.0411023a2B2DACB41813
sd1: 14806MB, 512 bytes/sector, 30322688 sectors
umass1 at uhub2 port 4 configuration 1 interface 0 "USB Flash Disk" rev
2.00/2.00 addr 4
umass1: using SCSI over Bulk-Only
scsibus6 at umass1: 2 targets, initiator 0
sd2 at scsibus6 targ 1 lun 0: <, USB BAR, 2.00> removable
sd2: 999MB, 512 bytes/sector, 2047744 sectors
uhub3 at uhub2 port 6 configuration 1 interface 0 "no manufacturer Gadget USB
HUB" rev 2.00/0.00 addr 5
uhidev0 at uhub3 port 1 configuration 1 interface 0 "Avocent Keyboard/Mouse
Function" rev 2.00/0.00 addr 6
uhidev0: iclass 3/1
ukbd0 at uhidev0: 8 variable keys, 6 key codes
wskbd0 at ukbd0: console keyboard, using wsdisplay0
uhidev1 at uhub3 port 1 configuration 1 interface 1 "Avocent Keyboard/Mouse
Function" rev 2.00/0.00 addr 6
uhidev1: iclass 3/1
ums0 at uhidev1: 3 buttons, Z dir
wsmouse0 at ums0 mux 0
uhidev2 at uhub3 port 1 configuration 1 interface 2 "Avocent Keyboard/Mouse
Function" rev 2.00/0.00 addr 6
uhidev2: iclass 3/1
ums1 at uhidev2: 3 buttons, Z dir
wsmouse1 at ums1 mux 0
uhub4 at uhub1 port 1 configuration 1 interface 0 "Intel Rate Matching Hub" rev
2.00/0.05 addr 2
vscsi0 at root
scsibus7 at vscsi0: 256 targets
softraid0 at root
scsibus8 at softraid0: 256 targets
ukbd0: was console keyboard
wskbd0 detached
ukbd0 detached
uhidev0 detached
wsmouse0 detached
ums0 detached
uhidev1 detached
wsmouse1 detached
ums1 detached
uhidev2 detached
root on sd0a (30e0f7089003b889.a) swap on sd0b dump on sd0b
WARNING: / was not properly unmounted
uhidev0 at uhub3 port 1 configuration 1 interface 0 "Avocent Keyboard/Mouse
Function" rev 2.00/0.00 addr 6
uhidev0: iclass 3/1
ukbd0 at uhidev0: 8 variable keys, 6 key codes
wskbd0 at ukbd0: console keyboard, using wsdisplay0
uhidev1 at uhub3 port 1 configuration 1 interface 1 "Avocent Keyboard/Mouse
Function" rev 2.00/0.00 addr 6
uhidev1: iclass 3/1
ums0 at uhidev1: 3 buttons, Z dir
wsmouse0 at ums0 mux 0
uhidev2 at uhub3 port 1 configuration 1 interface 2 "Avocent Keyboard/Mouse
Function" rev 2.00/0.00 addr 6
uhidev2: iclass 3/1
ums1 at uhidev2: 3 buttons, Z dir
wsmouse1 at ums1 mux 0
