On Mon, Apr 05, 2021 at 09:29:10PM +0200, Marcus MERIGHI wrote:
> Hello!
>
> scottchel...@gmail.com (Scott Cheloha), 2021.03.25 (Thu) 23:28 (CET):
> > Feel free to share your raw data.
>
> I think I followed the instructions; is it expected to get a lot of
>
> pstat: cannot read 0x9704: invalid address (9704)
No, not expected. Don't know why that would happen. Your data looks
plausible though, so I'm just going to assume it isn't bogus.
> and that the script takes about six minutes to run?
It will be slow, yes. We're extracting a single 8-byte value at a
time. For your CPU that's 15000 pstat(8) runs, so 6 minutes doesn't
sound outlandish.
> Anyway, data in tar.gz attached. dmesg as well.
It looks like every AP is basically synchronized but the BP is way
behind. Unsure about the source of the large stddev, though.
First package:
$ ministat -q $(jot -w cpu%d-skew 7 1)
x cpu1-skew
+ cpu2-skew
* cpu3-skew
% cpu4-skew
# cpu5-skew
@ cpu6-skew
O cpu7-skew
N Min Max Median Avg Stddev
x 1000 -281984 -19984 -149392 -148027.28 74570.233
+ 1000 -282000 -20176 -145824 -150002.46 75860.807
No difference proven at 95.0% confidence
* 1000 -281920 -19904 -149840 -150290.77 76264.271
No difference proven at 95.0% confidence
% 1000 -281504 -20160 -155712 -153770.51 76155.187
No difference proven at 95.0% confidence
# 1000 -281456 -19984 -154496 -150543.97 73409.211
No difference proven at 95.0% confidence
@ 1000 -281792 -20032 -153264 -152657.66 75292.624
No difference proven at 95.0% confidence
O 1000 -281984 -20112 -154640 -150771.23 76194.772
No difference proven at 95.0% confidence
Second package (except cpu15):
$ ministat -q $(jot -w cpu%d-skew 7 8)
x cpu8-skew
+ cpu9-skew
* cpu10-skew
% cpu11-skew
# cpu12-skew
@ cpu13-skew
O cpu14-skew
N Min Max Median Avg Stddev
x 1000 -281968 -20128 -147616 -149659.41 75748.332
+ 1000 -280736 -20016 -157520 -152365.76 75622.919
No difference proven at 95.0% confidence
* 1000 -281952 -20000 -157040 -153879.23 73912.486
No difference proven at 95.0% confidence
% 1000 -281696 -19968 -154384 -150910.83 74784.57
No difference proven at 95.0% confidence
# 1000 -281872 -19920 -149744 -150493.36 75257.127
No difference proven at 95.0% confidence
@ 1000 -281728 -20064 -152896 -152678.35 75612.029
No difference proven at 95.0% confidence
O 1000 -281808 -20656 -156800 -155226.9 76674.412
No difference proven at 95.0% confidence
To be clear, userland TSC is disabled on this machine, yes? And this
is a physical server, not a VM?
> OpenBSD 6.8 (GENERIC.MP) #5: Mon Feb 22 04:36:10 MST 2021
>
> r...@syspatch-68-amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP
> real mem = 34224320512 (32638MB)
> avail mem = 33172041728 (31635MB)
> random: good seed from bootblocks
> mpath0 at root
> scsibus0 at mpath0: 256 targets
> mainbus0 at root
> bios0 at mainbus0: SMBIOS rev. 3.0 @ 0xec9b0 (74 entries)
> bios0: vendor American Megatrends Inc. version "3.1" date 06/07/2018
> bios0: www.1he-server.com GN#15069
> acpi0 at bios0: ACPI 5.0
> acpi0: sleep states S0 S4 S5
> acpi0: tables DSDT FACP APIC FPDT FIDT SPMI MCFG UEFI HPET MSCT NFIT SLIT
> SRAT WDDT SSDT NITR SSDT SSDT PRAD DMAR HEST BERT ERST EINJ
> acpi0: wakeup devices IP2P(S4) EHC1(S4) EHC2(S4) RP01(S4) RP02(S4) RP03(S4)
> RP04(S4) RP05(S4) RP06(S4) RP07(S4) RP08(S4) BR1B(S4) BR3A(S4) BR3B(S4)
> BR3C(S4) BR3D(S4) [...]
> acpitimer0 at acpi0: 3579545 Hz, 24 bits
> acpimadt0 at acpi0 addr 0xfee00000: PC-AT compat
> cpu0 at mainbus0: apid 0 (boot processor)
> cpu0: Intel(R) Xeon(R) CPU E5-2620 v4 @ 2.10GHz, 2100.31 MHz, 06-4f-01
> cpu0:
> FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,3DNOWP,PERF,ITSC,FSGSBASE,TSC_ADJUST,BMI1,HLE,AVX2,SMEP,BMI2,ERMS,INVPCID,RTM,PQM,RDSEED,ADX,SMAP,PT,MD_CLEAR,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
> cpu0: 256KB 64b/line 8-way L2 cache
> cpu0: smt 0, core 0, package 0
> mtrr: Pentium Pro MTRR support, 10 var ranges, 88 fixed ranges
> cpu0: apic clock running at 99MHz
> cpu0: mwait min=64, max=64, C-substates=0.2.1.2, IBE
> cpu1 at mainbus0: apid 2 (application processor)
> cpu1: Intel(R) Xeon(R) CPU E5-2620 v4 @ 2.10GHz, 2100.01 MHz, 06-4f-01
> cpu1:
> FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,3DNOWP,PERF,ITSC,FSGSBASE,TSC_ADJUST,BMI1,HLE,AVX2,SMEP,BMI2,ERMS,INVPCID,RTM,PQM,RDSEED,ADX,SMAP,PT,MD_CLEAR,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
> cpu1: 256KB 64b/line 8-way L2 cache
> cpu1: smt 0, core 1, package 0
> cpu2 at mainbus0: apid 4 (application processor)
> cpu2: Intel(R) Xeon(R) CPU E5-2620 v4 @ 2.10GHz, 2100.01 MHz, 06-4f-01
> cpu2:
> FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,3DNOWP,PERF,ITSC,FSGSBASE,TSC_ADJUST,BMI1,HLE,AVX2,SMEP,BMI2,ERMS,INVPCID,RTM,PQM,RDSEED,ADX,SMAP,PT,MD_CLEAR,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
> cpu2: 256KB 64b/line 8-way L2 cache
> cpu2: smt 0, core 2, package 0
> cpu3 at mainbus0: apid 6 (application processor)
> cpu3: Intel(R) Xeon(R) CPU E5-2620 v4 @ 2.10GHz, 2100.01 MHz, 06-4f-01
> cpu3:
> FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,3DNOWP,PERF,ITSC,FSGSBASE,TSC_ADJUST,BMI1,HLE,AVX2,SMEP,BMI2,ERMS,INVPCID,RTM,PQM,RDSEED,ADX,SMAP,PT,MD_CLEAR,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
> cpu3: 256KB 64b/line 8-way L2 cache
> cpu3: smt 0, core 3, package 0
> cpu4 at mainbus0: apid 8 (application processor)
> cpu4: Intel(R) Xeon(R) CPU E5-2620 v4 @ 2.10GHz, 2100.01 MHz, 06-4f-01
> cpu4:
> FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,3DNOWP,PERF,ITSC,FSGSBASE,TSC_ADJUST,BMI1,HLE,AVX2,SMEP,BMI2,ERMS,INVPCID,RTM,PQM,RDSEED,ADX,SMAP,PT,MD_CLEAR,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
> cpu4: 256KB 64b/line 8-way L2 cache
> cpu4: smt 0, core 4, package 0
> cpu5 at mainbus0: apid 10 (application processor)
> cpu5: Intel(R) Xeon(R) CPU E5-2620 v4 @ 2.10GHz, 2100.01 MHz, 06-4f-01
> cpu5:
> FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,3DNOWP,PERF,ITSC,FSGSBASE,TSC_ADJUST,BMI1,HLE,AVX2,SMEP,BMI2,ERMS,INVPCID,RTM,PQM,RDSEED,ADX,SMAP,PT,MD_CLEAR,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
> cpu5: 256KB 64b/line 8-way L2 cache
> cpu5: smt 0, core 5, package 0
> cpu6 at mainbus0: apid 12 (application processor)
> cpu6: Intel(R) Xeon(R) CPU E5-2620 v4 @ 2.10GHz, 2100.01 MHz, 06-4f-01
> cpu6:
> FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,3DNOWP,PERF,ITSC,FSGSBASE,TSC_ADJUST,BMI1,HLE,AVX2,SMEP,BMI2,ERMS,INVPCID,RTM,PQM,RDSEED,ADX,SMAP,PT,MD_CLEAR,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
> cpu6: 256KB 64b/line 8-way L2 cache
> cpu6: smt 0, core 6, package 0
> cpu7 at mainbus0: apid 14 (application processor)
> cpu7: Intel(R) Xeon(R) CPU E5-2620 v4 @ 2.10GHz, 2100.01 MHz, 06-4f-01
> cpu7:
> FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,3DNOWP,PERF,ITSC,FSGSBASE,TSC_ADJUST,BMI1,HLE,AVX2,SMEP,BMI2,ERMS,INVPCID,RTM,PQM,RDSEED,ADX,SMAP,PT,MD_CLEAR,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
> cpu7: 256KB 64b/line 8-way L2 cache
> cpu7: smt 0, core 7, package 0
> cpu8 at mainbus0: apid 16 (application processor)
> cpu8: Intel(R) Xeon(R) CPU E5-2620 v4 @ 2.10GHz, 2099.69 MHz, 06-4f-01
> cpu8:
> FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,3DNOWP,PERF,ITSC,FSGSBASE,TSC_ADJUST,BMI1,HLE,AVX2,SMEP,BMI2,ERMS,INVPCID,RTM,PQM,RDSEED,ADX,SMAP,PT,MD_CLEAR,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
> cpu8: 256KB 64b/line 8-way L2 cache
> cpu8: smt 0, core 0, package 1
> cpu9 at mainbus0: apid 18 (application processor)
> cpu9: Intel(R) Xeon(R) CPU E5-2620 v4 @ 2.10GHz, 2100.01 MHz, 06-4f-01
> cpu9:
> FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,3DNOWP,PERF,ITSC,FSGSBASE,TSC_ADJUST,BMI1,HLE,AVX2,SMEP,BMI2,ERMS,INVPCID,RTM,PQM,RDSEED,ADX,SMAP,PT,MD_CLEAR,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
> cpu9: 256KB 64b/line 8-way L2 cache
> cpu9: smt 0, core 1, package 1
> cpu10 at mainbus0: apid 20 (application processor)
> cpu10: Intel(R) Xeon(R) CPU E5-2620 v4 @ 2.10GHz, 2100.01 MHz, 06-4f-01
> cpu10:
> FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,3DNOWP,PERF,ITSC,FSGSBASE,TSC_ADJUST,BMI1,HLE,AVX2,SMEP,BMI2,ERMS,INVPCID,RTM,PQM,RDSEED,ADX,SMAP,PT,MD_CLEAR,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
> cpu10: 256KB 64b/line 8-way L2 cache
> cpu10: smt 0, core 2, package 1
> cpu11 at mainbus0: apid 22 (application processor)
> cpu11: Intel(R) Xeon(R) CPU E5-2620 v4 @ 2.10GHz, 2100.01 MHz, 06-4f-01
> cpu11:
> FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,3DNOWP,PERF,ITSC,FSGSBASE,TSC_ADJUST,BMI1,HLE,AVX2,SMEP,BMI2,ERMS,INVPCID,RTM,PQM,RDSEED,ADX,SMAP,PT,MD_CLEAR,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
> cpu11: 256KB 64b/line 8-way L2 cache
> cpu11: smt 0, core 3, package 1
> cpu12 at mainbus0: apid 24 (application processor)
> cpu12: Intel(R) Xeon(R) CPU E5-2620 v4 @ 2.10GHz, 2100.01 MHz, 06-4f-01
> cpu12:
> FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,3DNOWP,PERF,ITSC,FSGSBASE,TSC_ADJUST,BMI1,HLE,AVX2,SMEP,BMI2,ERMS,INVPCID,RTM,PQM,RDSEED,ADX,SMAP,PT,MD_CLEAR,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
> cpu12: 256KB 64b/line 8-way L2 cache
> cpu12: smt 0, core 4, package 1
> cpu13 at mainbus0: apid 26 (application processor)
> cpu13: Intel(R) Xeon(R) CPU E5-2620 v4 @ 2.10GHz, 2100.01 MHz, 06-4f-01
> cpu13:
> FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,3DNOWP,PERF,ITSC,FSGSBASE,TSC_ADJUST,BMI1,HLE,AVX2,SMEP,BMI2,ERMS,INVPCID,RTM,PQM,RDSEED,ADX,SMAP,PT,MD_CLEAR,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
> cpu13: 256KB 64b/line 8-way L2 cache
> cpu13: smt 0, core 5, package 1
> cpu14 at mainbus0: apid 28 (application processor)
> cpu14: Intel(R) Xeon(R) CPU E5-2620 v4 @ 2.10GHz, 2100.01 MHz, 06-4f-01
> cpu14:
> FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,3DNOWP,PERF,ITSC,FSGSBASE,TSC_ADJUST,BMI1,HLE,AVX2,SMEP,BMI2,ERMS,INVPCID,RTM,PQM,RDSEED,ADX,SMAP,PT,MD_CLEAR,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
> cpu14: 256KB 64b/line 8-way L2 cache
> cpu14: smt 0, core 6, package 1
> cpu15 at mainbus0: apid 30 (application processor)
> cpu15: Intel(R) Xeon(R) CPU E5-2620 v4 @ 2.10GHz, 2100.01 MHz, 06-4f-01
> cpu15:
> FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,3DNOWP,PERF,ITSC,FSGSBASE,TSC_ADJUST,BMI1,HLE,AVX2,SMEP,BMI2,ERMS,INVPCID,RTM,PQM,RDSEED,ADX,SMAP,PT,MD_CLEAR,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
> cpu15: 256KB 64b/line 8-way L2 cache
> cpu15: smt 0, core 7, package 1
> ioapic0 at mainbus0: apid 1 pa 0xfec00000, version 20, 24 pins
> ioapic1 at mainbus0: apid 2 pa 0xfec01000, version 20, 24 pins
> ioapic2 at mainbus0: apid 3 pa 0xfec40000, version 20, 24 pins
> acpimcfg0 at acpi0
> acpimcfg0: addr 0x80000000, bus 0-255
> acpihpet0 at acpi0: 14318179 Hz
> acpiprt0 at acpi0: bus 255 (UNC1)
> acpiprt1 at acpi0: bus 127 (UNC0)
> acpiprt2 at acpi0: bus 0 (PCI0)
> acpiprt3 at acpi0: bus 5 (RP05)
> acpiprt4 at acpi0: bus 6 (BR6F)
> acpiprt5 at acpi0: bus 3 (BR3A)
> acpiprt6 at acpi0: bus 1 (BR1A)
> acpiprt7 at acpi0: bus 2 (BR2A)
> acpiprt8 at acpi0: bus -1 (BR2B)
> acpiprt9 at acpi0: bus -1 (BR2C)
> acpiprt10 at acpi0: bus -1 (BR2D)
> acpiprt11 at acpi0: bus 128 (PCI1)
> acpiprt12 at acpi0: bus -1 (QR2A)
> acpiprt13 at acpi0: bus -1 (QR2C)
> acpiprt14 at acpi0: bus -1 (QR3A)
> acpiprt15 at acpi0: bus -1 (QR3C)
> acpipci0 at acpi0 UNC1: 0x00000010 0x00000011 0x00000000
> acpipci1 at acpi0 UNC0: 0x00000010 0x00000011 0x00000000
> "ACPI0004" at acpi0 not configured
> "ACPI0007" at acpi0 not configured
> "ACPI0007" at acpi0 not configured
> "ACPI0007" at acpi0 not configured
> "ACPI0007" at acpi0 not configured
> "ACPI0007" at acpi0 not configured
> "ACPI0007" at acpi0 not configured
> "ACPI0007" at acpi0 not configured
> "ACPI0007" at acpi0 not configured
> "ACPI0004" at acpi0 not configured
> "ACPI0007" at acpi0 not configured
> "ACPI0007" at acpi0 not configured
> "ACPI0007" at acpi0 not configured
> "ACPI0007" at acpi0 not configured
> "ACPI0007" at acpi0 not configured
> "ACPI0007" at acpi0 not configured
> "ACPI0007" at acpi0 not configured
> "ACPI0007" at acpi0 not configured
> acpipci2 at acpi0 PCI0: 0x00000010 0x00000011 0x00000000
> "PNP0003" at acpi0 not configured
> acpicmos0 at acpi0
> "IPI0001" at acpi0 not configured
> acpipci3 at acpi0 PCI1: 0x00000010 0x00000011 0x00000000
> acpibtn0 at acpi0: PWRB
> "PNP0C33" at acpi0 not configured
> "PNP0C14" at acpi0 not configured
> "ACPI000D" at acpi0 not configured
> acpicpu0 at acpi0: C2(350@41 mwait.3@0x20), C1(1016@1 mwait.1), PSS
> acpicpu1 at acpi0: C2(350@41 mwait.3@0x20), C1(1016@1 mwait.1), PSS
> acpicpu2 at acpi0: C2(350@41 mwait.3@0x20), C1(1016@1 mwait.1), PSS
> acpicpu3 at acpi0: C2(350@41 mwait.3@0x20), C1(1016@1 mwait.1), PSS
> acpicpu4 at acpi0: C2(350@41 mwait.3@0x20), C1(1016@1 mwait.1), PSS
> acpicpu5 at acpi0: C2(350@41 mwait.3@0x20), C1(1016@1 mwait.1), PSS
> acpicpu6 at acpi0: C2(350@41 mwait.3@0x20), C1(1016@1 mwait.1), PSS
> acpicpu7 at acpi0: C2(350@41 mwait.3@0x20), C1(1016@1 mwait.1), PSS
> acpicpu8 at acpi0: C2(350@41 mwait.3@0x20), C1(1016@1 mwait.1), PSS
> acpicpu9 at acpi0: C2(350@41 mwait.3@0x20), C1(1016@1 mwait.1), PSS
> acpicpu10 at acpi0: C2(350@41 mwait.3@0x20), C1(1016@1 mwait.1), PSS
> acpicpu11 at acpi0: C2(350@41 mwait.3@0x20), C1(1016@1 mwait.1), PSS
> acpicpu12 at acpi0: C2(350@41 mwait.3@0x20), C1(1016@1 mwait.1), PSS
> acpicpu13 at acpi0: C2(350@41 mwait.3@0x20), C1(1016@1 mwait.1), PSS
> acpicpu14 at acpi0: C2(350@41 mwait.3@0x20), C1(1016@1 mwait.1), PSS
> acpicpu15 at acpi0: C2(350@41 mwait.3@0x20), C1(1016@1 mwait.1), PSS
> ipmi at mainbus0 not configured
> cpu0: using VERW MDS workaround (except on vmm entry)
> cpu0: Enhanced SpeedStep 2100 MHz: speeds: 2101, 2100, 2000, 1900, 1800,
> 1700, 1600, 1500, 1400, 1300, 1200 MHz
> pci0 at mainbus0 bus 255
> 255:30:3: io address conflict 0x14/0x4
> vendor "Intel", unknown product 0x6f80 (class system subclass miscellaneous,
> rev 0x01) at pci0 dev 8 function 0 not configured
> vendor "Intel", unknown product 0x6f32 (class DASP subclass Time and
> Frequency, rev 0x01) at pci0 dev 8 function 2 not configured
> vendor "Intel", unknown product 0x6f83 (class system subclass miscellaneous,
> rev 0x01) at pci0 dev 8 function 3 not configured
> vendor "Intel", unknown product 0x6f90 (class system subclass miscellaneous,
> rev 0x01) at pci0 dev 9 function 0 not configured
> vendor "Intel", unknown product 0x6f33 (class DASP subclass Time and
> Frequency, rev 0x01) at pci0 dev 9 function 2 not configured
> vendor "Intel", unknown product 0x6f93 (class system subclass miscellaneous,
> rev 0x01) at pci0 dev 9 function 3 not configured
> "Intel Xeon-D QPI Link" rev 0x01 at pci0 dev 11 function 0 not configured
> "Intel Xeon-D QPI Link" rev 0x01 at pci0 dev 11 function 1 not configured
> "Intel Xeon-D QPI Link" rev 0x01 at pci0 dev 11 function 2 not configured
> "Intel Xeon-D QPI Debug" rev 0x01 at pci0 dev 11 function 3 not configured
> "Intel Xeon-D Cache" rev 0x01 at pci0 dev 12 function 0 not configured
> "Intel Xeon-D Cache" rev 0x01 at pci0 dev 12 function 1 not configured
> "Intel Xeon-D Cache" rev 0x01 at pci0 dev 12 function 2 not configured
> "Intel Xeon-D Cache" rev 0x01 at pci0 dev 12 function 3 not configured
> "Intel E5 v4 Cache" rev 0x01 at pci0 dev 12 function 4 not configured
> "Intel E5 v4 Cache" rev 0x01 at pci0 dev 12 function 5 not configured
> vendor "Intel", unknown product 0x6fe6 (class system subclass miscellaneous,
> rev 0x01) at pci0 dev 12 function 6 not configured
> vendor "Intel", unknown product 0x6fe7 (class system subclass miscellaneous,
> rev 0x01) at pci0 dev 12 function 7 not configured
> "Intel Xeon-D Cache" rev 0x01 at pci0 dev 15 function 0 not configured
> "Intel E5 v4 Cache" rev 0x01 at pci0 dev 15 function 1 not configured
> "Intel Xeon-D Cache" rev 0x01 at pci0 dev 15 function 4 not configured
> "Intel Xeon-D Cache" rev 0x01 at pci0 dev 15 function 5 not configured
> "Intel Xeon-D Cache" rev 0x01 at pci0 dev 15 function 6 not configured
> "Intel Xeon-D PCIE" rev 0x01 at pci0 dev 16 function 0 not configured
> "Intel E5 v4 R2PCIe Agent" rev 0x01 at pci0 dev 16 function 1 not configured
> "Intel Xeon-D Ubox" rev 0x01 at pci0 dev 16 function 5 not configured
> "Intel Xeon-D Ubox" rev 0x01 at pci0 dev 16 function 6 not configured
> "Intel Xeon-D Ubox" rev 0x01 at pci0 dev 16 function 7 not configured
> "Intel Xeon-D Home Agent" rev 0x01 at pci0 dev 18 function 0 not configured
> "Intel Xeon-D Home Agent" rev 0x01 at pci0 dev 18 function 1 not configured
> "Intel Xeon-D Memory" rev 0x01 at pci0 dev 19 function 0 not configured
> "Intel Xeon-D Memory" rev 0x01 at pci0 dev 19 function 1 not configured
> "Intel Xeon-D Memory" rev 0x01 at pci0 dev 19 function 2 not configured
> "Intel Xeon-D Memory" rev 0x01 at pci0 dev 19 function 3 not configured
> "Intel Xeon-D Memory" rev 0x01 at pci0 dev 19 function 4 not configured
> "Intel Xeon-D Memory" rev 0x01 at pci0 dev 19 function 5 not configured
> "Intel Xeon-D Memory" rev 0x01 at pci0 dev 19 function 6 not configured
> "Intel Xeon-D Memory" rev 0x01 at pci0 dev 19 function 7 not configured
> "Intel Xeon-D Memory" rev 0x01 at pci0 dev 20 function 0 not configured
> "Intel Xeon-D Memory" rev 0x01 at pci0 dev 20 function 1 not configured
> "Intel Xeon-D Memory" rev 0x01 at pci0 dev 20 function 2 not configured
> "Intel Xeon-D Memory" rev 0x01 at pci0 dev 20 function 3 not configured
> "Intel Xeon-D Memory" rev 0x01 at pci0 dev 20 function 4 not configured
> "Intel Xeon-D Memory" rev 0x01 at pci0 dev 20 function 5 not configured
> "Intel Xeon-D Memory" rev 0x01 at pci0 dev 20 function 6 not configured
> "Intel Xeon-D Memory" rev 0x01 at pci0 dev 20 function 7 not configured
> "Intel Xeon-D Memory" rev 0x01 at pci0 dev 21 function 0 not configured
> "Intel Xeon-D Memory" rev 0x01 at pci0 dev 21 function 1 not configured
> "Intel Xeon-D Memory" rev 0x01 at pci0 dev 21 function 2 not configured
> "Intel Xeon-D Memory" rev 0x01 at pci0 dev 21 function 3 not configured
> "Intel E5 v4 RAS" rev 0x01 at pci0 dev 22 function 0 not configured
> "Intel E5 v4 DDRIO" rev 0x01 at pci0 dev 22 function 6 not configured
> "Intel E5 v4 DDRIO" rev 0x01 at pci0 dev 22 function 7 not configured
> "Intel E5 v4 Thermal" rev 0x01 at pci0 dev 23 function 0 not configured
> "Intel E5 v4 DDRIO" rev 0x01 at pci0 dev 23 function 4 not configured
> "Intel E5 v4 DDRIO" rev 0x01 at pci0 dev 23 function 5 not configured
> "Intel E5 v4 DDRIO" rev 0x01 at pci0 dev 23 function 6 not configured
> "Intel E5 v4 DDRIO" rev 0x01 at pci0 dev 23 function 7 not configured
> "Intel Xeon-D PCU" rev 0x01 at pci0 dev 30 function 0 not configured
> "Intel Xeon-D PCU" rev 0x01 at pci0 dev 30 function 1 not configured
> "Intel Xeon-D PCU" rev 0x01 at pci0 dev 30 function 2 not configured
> "Intel Xeon-D PCU" rev 0x01 at pci0 dev 30 function 3 not configured
> "Intel Xeon-D PCU" rev 0x01 at pci0 dev 30 function 4 not configured
> "Intel Xeon-D PCU" rev 0x01 at pci0 dev 31 function 0 not configured
> "Intel Xeon-D PCU" rev 0x01 at pci0 dev 31 function 2 not configured
> pci1 at mainbus0 bus 127
> 127:30:3: io address conflict 0x14/0x4
> vendor "Intel", unknown product 0x6f80 (class system subclass miscellaneous,
> rev 0x01) at pci1 dev 8 function 0 not configured
> vendor "Intel", unknown product 0x6f32 (class DASP subclass Time and
> Frequency, rev 0x01) at pci1 dev 8 function 2 not configured
> vendor "Intel", unknown product 0x6f83 (class system subclass miscellaneous,
> rev 0x01) at pci1 dev 8 function 3 not configured
> vendor "Intel", unknown product 0x6f90 (class system subclass miscellaneous,
> rev 0x01) at pci1 dev 9 function 0 not configured
> vendor "Intel", unknown product 0x6f33 (class DASP subclass Time and
> Frequency, rev 0x01) at pci1 dev 9 function 2 not configured
> vendor "Intel", unknown product 0x6f93 (class system subclass miscellaneous,
> rev 0x01) at pci1 dev 9 function 3 not configured
> "Intel Xeon-D QPI Link" rev 0x01 at pci1 dev 11 function 0 not configured
> "Intel Xeon-D QPI Link" rev 0x01 at pci1 dev 11 function 1 not configured
> "Intel Xeon-D QPI Link" rev 0x01 at pci1 dev 11 function 2 not configured
> "Intel Xeon-D QPI Debug" rev 0x01 at pci1 dev 11 function 3 not configured
> "Intel Xeon-D Cache" rev 0x01 at pci1 dev 12 function 0 not configured
> "Intel Xeon-D Cache" rev 0x01 at pci1 dev 12 function 1 not configured
> "Intel Xeon-D Cache" rev 0x01 at pci1 dev 12 function 2 not configured
> "Intel Xeon-D Cache" rev 0x01 at pci1 dev 12 function 3 not configured
> "Intel E5 v4 Cache" rev 0x01 at pci1 dev 12 function 4 not configured
> "Intel E5 v4 Cache" rev 0x01 at pci1 dev 12 function 5 not configured
> vendor "Intel", unknown product 0x6fe6 (class system subclass miscellaneous,
> rev 0x01) at pci1 dev 12 function 6 not configured
> vendor "Intel", unknown product 0x6fe7 (class system subclass miscellaneous,
> rev 0x01) at pci1 dev 12 function 7 not configured
> "Intel Xeon-D Cache" rev 0x01 at pci1 dev 15 function 0 not configured
> "Intel E5 v4 Cache" rev 0x01 at pci1 dev 15 function 1 not configured
> "Intel Xeon-D Cache" rev 0x01 at pci1 dev 15 function 4 not configured
> "Intel Xeon-D Cache" rev 0x01 at pci1 dev 15 function 5 not configured
> "Intel Xeon-D Cache" rev 0x01 at pci1 dev 15 function 6 not configured
> "Intel Xeon-D PCIE" rev 0x01 at pci1 dev 16 function 0 not configured
> "Intel E5 v4 R2PCIe Agent" rev 0x01 at pci1 dev 16 function 1 not configured
> "Intel Xeon-D Ubox" rev 0x01 at pci1 dev 16 function 5 not configured
> "Intel Xeon-D Ubox" rev 0x01 at pci1 dev 16 function 6 not configured
> "Intel Xeon-D Ubox" rev 0x01 at pci1 dev 16 function 7 not configured
> "Intel Xeon-D Home Agent" rev 0x01 at pci1 dev 18 function 0 not configured
> "Intel Xeon-D Home Agent" rev 0x01 at pci1 dev 18 function 1 not configured
> "Intel Xeon-D Memory" rev 0x01 at pci1 dev 19 function 0 not configured
> "Intel Xeon-D Memory" rev 0x01 at pci1 dev 19 function 1 not configured
> "Intel Xeon-D Memory" rev 0x01 at pci1 dev 19 function 2 not configured
> "Intel Xeon-D Memory" rev 0x01 at pci1 dev 19 function 3 not configured
> "Intel Xeon-D Memory" rev 0x01 at pci1 dev 19 function 4 not configured
> "Intel Xeon-D Memory" rev 0x01 at pci1 dev 19 function 5 not configured
> "Intel Xeon-D Memory" rev 0x01 at pci1 dev 19 function 6 not configured
> "Intel Xeon-D Memory" rev 0x01 at pci1 dev 19 function 7 not configured
> "Intel Xeon-D Memory" rev 0x01 at pci1 dev 20 function 0 not configured
> "Intel Xeon-D Memory" rev 0x01 at pci1 dev 20 function 1 not configured
> "Intel Xeon-D Memory" rev 0x01 at pci1 dev 20 function 2 not configured
> "Intel Xeon-D Memory" rev 0x01 at pci1 dev 20 function 3 not configured
> "Intel Xeon-D Memory" rev 0x01 at pci1 dev 20 function 4 not configured
> "Intel Xeon-D Memory" rev 0x01 at pci1 dev 20 function 5 not configured
> "Intel Xeon-D Memory" rev 0x01 at pci1 dev 20 function 6 not configured
> "Intel Xeon-D Memory" rev 0x01 at pci1 dev 20 function 7 not configured
> "Intel Xeon-D Memory" rev 0x01 at pci1 dev 21 function 0 not configured
> "Intel Xeon-D Memory" rev 0x01 at pci1 dev 21 function 1 not configured
> "Intel Xeon-D Memory" rev 0x01 at pci1 dev 21 function 2 not configured
> "Intel Xeon-D Memory" rev 0x01 at pci1 dev 21 function 3 not configured
> "Intel E5 v4 RAS" rev 0x01 at pci1 dev 22 function 0 not configured
> "Intel E5 v4 DDRIO" rev 0x01 at pci1 dev 22 function 6 not configured
> "Intel E5 v4 DDRIO" rev 0x01 at pci1 dev 22 function 7 not configured
> "Intel E5 v4 Thermal" rev 0x01 at pci1 dev 23 function 0 not configured
> "Intel E5 v4 DDRIO" rev 0x01 at pci1 dev 23 function 4 not configured
> "Intel E5 v4 DDRIO" rev 0x01 at pci1 dev 23 function 5 not configured
> "Intel E5 v4 DDRIO" rev 0x01 at pci1 dev 23 function 6 not configured
> "Intel E5 v4 DDRIO" rev 0x01 at pci1 dev 23 function 7 not configured
> "Intel Xeon-D PCU" rev 0x01 at pci1 dev 30 function 0 not configured
> "Intel Xeon-D PCU" rev 0x01 at pci1 dev 30 function 1 not configured
> "Intel Xeon-D PCU" rev 0x01 at pci1 dev 30 function 2 not configured
> "Intel Xeon-D PCU" rev 0x01 at pci1 dev 30 function 3 not configured
> "Intel Xeon-D PCU" rev 0x01 at pci1 dev 30 function 4 not configured
> "Intel Xeon-D PCU" rev 0x01 at pci1 dev 31 function 0 not configured
> "Intel Xeon-D PCU" rev 0x01 at pci1 dev 31 function 2 not configured
> pci2 at mainbus0 bus 0
> pchb0 at pci2 dev 0 function 0 "Intel Xeon-D Host" rev 0x01
> ppb0 at pci2 dev 1 function 0 "Intel Xeon-D PCIE" rev 0x01: msi
> pci3 at ppb0 bus 1
> em0 at pci3 dev 0 function 0 "Intel I350" rev 0x01: msi, address
> ac:1f:6b:48:c6:08
> em1 at pci3 dev 0 function 1 "Intel I350" rev 0x01: msi, address
> ac:1f:6b:48:c6:09
> ppb1 at pci2 dev 2 function 0 "Intel Xeon-D PCIE" rev 0x01: msi
> pci4 at ppb1 bus 2
> nvme0 at pci4 dev 0 function 0 "Intel NVMe" rev 0x02: msix, NVMe 1.0
> nvme0: INTEL SSDPEDMX012T7, firmware MDV10290, serial CVPF738500M61P2JGN
> scsibus1 at nvme0: 2 targets, initiator 0
> sd0 at scsibus1 targ 1 lun 0: <NVMe, INTEL SSDPEDMX01, MDV1>
> sd0: 1144641MB, 512 bytes/sector, 2344225968 sectors
> ppb2 at pci2 dev 3 function 0 "Intel Xeon-D PCIE" rev 0x01: msi
> pci5 at ppb2 bus 3
> "Intel E5 v4 DMA" rev 0x01 at pci2 dev 4 function 0 not configured
> "Intel E5 v4 DMA" rev 0x01 at pci2 dev 4 function 1 not configured
> "Intel E5 v4 DMA" rev 0x01 at pci2 dev 4 function 2 not configured
> "Intel E5 v4 DMA" rev 0x01 at pci2 dev 4 function 3 not configured
> "Intel E5 v4 DMA" rev 0x01 at pci2 dev 4 function 4 not configured
> "Intel E5 v4 DMA" rev 0x01 at pci2 dev 4 function 5 not configured
> "Intel E5 v4 DMA" rev 0x01 at pci2 dev 4 function 6 not configured
> "Intel E5 v4 DMA" rev 0x01 at pci2 dev 4 function 7 not configured
> "Intel Xeon-D Address Map" rev 0x01 at pci2 dev 5 function 0 not configured
> "Intel Xeon-D Hot Plug" rev 0x01 at pci2 dev 5 function 1 not configured
> "Intel Xeon-D RAS" rev 0x01 at pci2 dev 5 function 2 not configured
> "Intel Xeon-D I/O APIC" rev 0x01 at pci2 dev 5 function 4 not configured
> "Intel C610 MS SPSR" rev 0x05 at pci2 dev 17 function 0 not configured
> ahci0 at pci2 dev 17 function 4 "Intel C610 AHCI" rev 0x05: msi, AHCI 1.3
> scsibus2 at ahci0: 32 targets
> xhci0 at pci2 dev 20 function 0 "Intel C610 xHCI" rev 0x05: msi, xHCI 1.0
> usb0 at xhci0: USB revision 3.0
> uhub0 at usb0 configuration 1 interface 0 "Intel xHCI root hub" rev 3.00/1.00
> addr 1
> "Intel C610 MEI" rev 0x05 at pci2 dev 22 function 0 not configured
> "Intel C610 MEI" rev 0x05 at pci2 dev 22 function 1 not configured
> ehci0 at pci2 dev 26 function 0 "Intel C610 USB" rev 0x05: apic 1 int 18
> usb1 at ehci0: USB revision 2.0
> uhub1 at usb1 configuration 1 interface 0 "Intel EHCI root hub" rev 2.00/1.00
> addr 1
> ppb3 at pci2 dev 28 function 0 "Intel C610 PCIE" rev 0xd5
> pci6 at ppb3 bus 4
> ppb4 at pci2 dev 28 function 4 "Intel C610 PCIE" rev 0xd5: msi
> pci7 at ppb4 bus 5
> ppb5 at pci7 dev 0 function 0 "ASPEED Technology AST1150 PCI" rev 0x03
> pci8 at ppb5 bus 6
> vga1 at pci8 dev 0 function 0 "ASPEED Technology AST2000" rev 0x30
> wsdisplay0 at vga1 mux 1: console (80x25, vt100 emulation)
> wsdisplay0: screen 1-5 added (80x25, vt100 emulation)
> ehci1 at pci2 dev 29 function 0 "Intel C610 USB" rev 0x05: apic 1 int 18
> usb2 at ehci1: USB revision 2.0
> uhub2 at usb2 configuration 1 interface 0 "Intel EHCI root hub" rev 2.00/1.00
> addr 1
> pcib0 at pci2 dev 31 function 0 "Intel C610 LPC" rev 0x05
> ahci1 at pci2 dev 31 function 2 "Intel C610 AHCI" rev 0x05: msi, AHCI 1.3
> ahci1: port 0: 6.0Gb/s
> ahci1: port 1: 6.0Gb/s
> scsibus3 at ahci1: 32 targets
> sd1 at scsibus3 targ 0 lun 0: <ATA, INTEL SSDSC2KB24, XCV1>
> naa.55cd2e41503ffbab
> sd1: 228936MB, 512 bytes/sector, 468862128 sectors, thin
> sd2 at scsibus3 targ 1 lun 0: <ATA, INTEL SSDSC2KB24, XCV1>
> naa.55cd2e41508029db
> sd2: 228936MB, 512 bytes/sector, 468862128 sectors, thin
> ichiic0 at pci2 dev 31 function 3 "Intel C610 SMBus" rev 0x05: apic 1 int 18
> iic0 at ichiic0
> isa0 at pcib0
> isadma0 at isa0
> pckbc0 at isa0 port 0x60/5 irq 1 irq 12
> pcppi0 at isa0 port 0x61
> spkr0 at pcppi0
> pci9 at mainbus0 bus 128
> "Intel E5 v4 DMA" rev 0x01 at pci9 dev 4 function 0 not configured
> "Intel E5 v4 DMA" rev 0x01 at pci9 dev 4 function 1 not configured
> "Intel E5 v4 DMA" rev 0x01 at pci9 dev 4 function 2 not configured
> "Intel E5 v4 DMA" rev 0x01 at pci9 dev 4 function 3 not configured
> "Intel E5 v4 DMA" rev 0x01 at pci9 dev 4 function 4 not configured
> "Intel E5 v4 DMA" rev 0x01 at pci9 dev 4 function 5 not configured
> "Intel E5 v4 DMA" rev 0x01 at pci9 dev 4 function 6 not configured
> "Intel E5 v4 DMA" rev 0x01 at pci9 dev 4 function 7 not configured
> "Intel Xeon-D Address Map" rev 0x01 at pci9 dev 5 function 0 not configured
> "Intel Xeon-D Hot Plug" rev 0x01 at pci9 dev 5 function 1 not configured
> "Intel Xeon-D RAS" rev 0x01 at pci9 dev 5 function 2 not configured
> "Intel Xeon-D I/O APIC" rev 0x01 at pci9 dev 5 function 4 not configured
> vmm0 at mainbus0: VMX/EPT
> uhidev0 at uhub0 port 1 configuration 1 interface 0 "American Power
> Conversion Smart-UPS 1500 RM FW:667.18.I USB FW:7.3" rev 1.10/0.06 addr 2
> uhidev0: iclass 3/0, 54 report ids
> upd0 at uhidev0
> uhidev1 at uhub0 port 3 configuration 1 interface 0 "Tangtop USB CAT5" rev
> 1.10/0.01 addr 3
> uhidev1: iclass 3/1
> ukbd0 at uhidev1: 8 variable keys, 6 key codes
> wskbd0 at ukbd0: console keyboard, using wsdisplay0
> uhidev2 at uhub0 port 3 configuration 1 interface 1 "Tangtop USB CAT5" rev
> 1.10/0.01 addr 3
> uhidev2: iclass 3/1
> ums0 at uhidev2: 3 buttons, Z dir
> wsmouse0 at ums0 mux 0
> uhub3 at uhub0 port 13 configuration 1 interface 0 "ATEN International
> product 0x7000" rev 2.00/0.00 addr 4
> uhidev3 at uhub3 port 1 configuration 1 interface 0 "ATEN International
> product 0x2419" rev 1.10/1.00 addr 5
> uhidev3: iclass 3/1
> ukbd1 at uhidev3: 8 variable keys, 6 key codes
> wskbd1 at ukbd1 mux 1
> wskbd1: connecting to wsdisplay0
> uhidev4 at uhub3 port 1 configuration 1 interface 1 "ATEN International
> product 0x2419" rev 1.10/1.00 addr 5
> uhidev4: iclass 3/1
> ums1 at uhidev4: 3 buttons, Z dir
> wsmouse1 at ums1 mux 0
> uhub4 at uhub1 port 1 configuration 1 interface 0 "Intel Rate Matching Hub"
> rev 2.00/0.05 addr 2
> uhub5 at uhub2 port 1 configuration 1 interface 0 "Intel Rate Matching Hub"
> rev 2.00/0.05 addr 2
> vscsi0 at root
> scsibus4 at vscsi0: 256 targets
> softraid0 at root
> scsibus5 at softraid0: 256 targets
> sd3 at scsibus5 targ 1 lun 0: <OPENBSD, SR RAID 1, 006>
> sd3: 228933MB, 512 bytes/sector, 468856433 sectors
> root on sd3a (ee2d305bf6524cc7.a) swap on sd3b dump on sd3b
> carp: carp2 demoted group carp by 1 to 129 (carpdev)
> carp: carp2 demoted group lan by 1 to 1 (carpdev)
> carp: carp2 demoted group int by 1 to 1 (carpdev)
> carp2: state transition: INIT -> BACKUP
> carp: carp2 demoted group carp by -1 to 128 (carpdev)
> carp: carp2 demoted group lan by -1 to 0 (carpdev)
> carp: carp2 demoted group int by -1 to 0 (carpdev)
