As a final update to this (for now): I was unable to work out why the correct timestamps from pvclock_get_timecount() were not being used to correct the system clock. I suspect I don't have a full enough understanding of how the return value from this function used by the kernel timekeeping processes.
I did notice a thread on bugs@ (https://marc.info/?l=openbsd-bugs&m=157403970412386&w=2) which included a patch with a correction to the shift parameter sent from vmm to the pvclock driver. Although I couldn't test this particular patch (I don't control the hypervisor) I was able to set the shift parameter in pvclock.c to 12 to achieve the same net result. I can confirm that change resolved the precision issue described in the linked thread, but it also seems to have resulted in much improved clock stability (4 steps in 24hr, 1.0s, 1.0s, 0.5s, 0.5s). At this point ntpd is able to keep the clock synced so in the absence of further lines of investigation I'm inclined to leave as-is and continue to monitor. I will update the list if I discover anything new. Ian On Fri, 15 Nov 2019 at 09:17, Ian Gregory <[email protected]> wrote: > > I continued to investigate this and added some debugging output to the > pvclock driver to attempt to work out what was going on. > > In my most recent test I rebooted the client VM at 08:10 yesterday. > Over the following 24h, there were 16 "clock step" events which caused > the time to lag real time by a total of 21.3 seconds. In all but 3 of > the steps, the change in the offset was 1.0 seconds almost exactly. > During the test the VM was loaded with it's usual workload (running > net/zabbix) and ntpd was disabled. > > I added a printf to the end of pvclock_get_timecount which outputs the > state of the variables within the function for each 1000000000 steps > of system_time. > > Here is an example of the output. The prefixed date is from syslog and > is the incorrect system time. The actual time of the log entry (as > reported from a reliable NTP source) was 09:05:23 > > Nov 15 09:05:02 starbug /bsd: pvclock: > tsc_timestamp=3627858654285868 rdtsc=3627858654563637 delta_1=277769 > shift=-20 delta=0 mul_frac=342781 system_time=1573808723797914701 > ctr=1573808723797914701 > > The ctr value is the return value of pvclock_get_timecount - the value > 1573808723797914701 translates to Fri Nov 15 09:05:23.797 > > I'm no expert in kernel timekeeping internals (far from it), but it > seems that the pvclock driver is returning correct timestamps from > pvclock_get_timecount and thus I conclude both the pvclock device in > vmm and the pvclock driver in the kernel are working as designed. > > Can anyone advise if I've missed something? Happy to provide further > data if needed. > > Thanks > Ian > > > > On Fri, 8 Nov 2019 at 13:53, Ian Gregory <[email protected]> wrote: > > > > Hi > > > > Since the 6.6 release I've been experimenting with using pvclock as > > the selected timecounter on a virtual machine running under vmm. Both > > the host and guest are running 6.6-stable (the environment is provided > > by openbsd.amsterdam). > > > > With 6.5 and the tsc source, the clock would drift linearly by about 2 > > seconds per minute. This was too large a drift for ntpd to compensate > > for and so I used a cron job to force-correct the clock at regular > > intervals. > > > > With 6.6 I have changed the timecounter source to pvclock. In > > frequency terms this has proven to be much more stable, with minimal > > drift. However, at irregular intervals the clock will step out of time > > by a small whole number of seconds. Over 24 hours following a reboot > > the clock now differs from real time (verified against multiple ntp > > sources) by just over 23 seconds, it having stepped 9 times during > > that time window > > > > I ran the following command every 60s following a reboot of the guest > > to log the output > > echo -n `date` && rdate -pv time.cloudflare.com | tail -1 | awk > > '{ print " " $6 }' > > Note that the data points are not consistently 60s apart - I'm using > > 'sleep' to delay the loop. > > > > Raw data and chart of the offset over the 24 hours is available in > > this Google sheet: http://bit.ly/34NTaUh > > > > Is this likely to point to a bug in the pvclock implementation or an > > environment/configuration issue? > > > > Thanks > > Ian > > > > > > dmesg (guest) > > ============= > > > > OpenBSD 6.6 (GENERIC) #0: Sat Oct 26 06:47:50 MDT 2019 > > > > [email protected]:/usr/src/sys/arch/amd64/compile/GENERIC > > real mem = 2130698240 (2031MB) > > avail mem = 2053558272 (1958MB) > > mpath0 at root > > scsibus0 at mpath0: 256 targets > > mainbus0 at root > > bios0 at mainbus0: SMBIOS rev. 2.4 @ 0xf3f40 (10 entries) > > bios0: vendor SeaBIOS version "1.11.0p2-OpenBSD-vmm" date 01/01/2011 > > bios0: OpenBSD VMM > > acpi at bios0 not configured > > cpu0 at mainbus0: (uniprocessor) > > cpu0: Intel(R) Xeon(R) CPU X5675 @ 3.07GHz, 3062.26 MHz, 06-2c-02 > > cpu0: > > FPU,VME,DE,PSE,TSC,MSR,PAE,CX8,SEP,PGE,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,SSE3,PCLMUL,SSSE3,CX16,SSE4.1,SSE4.2,POPCNT,AES,HV,NXE,PAGE1GB,LONG,LAHF,ITSC,MELTDOWN > > cpu0: 256KB 64b/line 8-way L2 cache > > cpu0: smt 0, core 0, package 0 > > cpu0: using IvyBridge MDS workaround > > pvbus0 at mainbus0: OpenBSD > > pvclock0 at pvbus0 > > pci0 at mainbus0 bus 0 > > pchb0 at pci0 dev 0 function 0 "OpenBSD VMM Host" rev 0x00 > > virtio0 at pci0 dev 1 function 0 "Qumranet Virtio RNG" rev 0x00 > > viornd0 at virtio0 > > virtio0: irq 3 > > virtio1 at pci0 dev 2 function 0 "Qumranet Virtio Network" rev 0x00 > > vio0 at virtio1: address fe:e1:bb:d4:c4:03 > > virtio1: irq 5 > > virtio2 at pci0 dev 3 function 0 "Qumranet Virtio Storage" rev 0x00 > > vioblk0 at virtio2 > > scsibus1 at vioblk0: 2 targets > > sd0 at scsibus1 targ 0 lun 0: <VirtIO, Block Device, > > > sd0: 51200MB, 512 bytes/sector, 104857600 sectors > > virtio2: irq 6 > > virtio3 at pci0 dev 4 function 0 "OpenBSD VMM Control" rev 0x00 > > vmmci0 at virtio3 > > virtio3: irq 7 > > isa0 at mainbus0 > > isadma0 at isa0 > > com0 at isa0 port 0x3f8/8 irq 4: ns8250, no fifo > > com0: console > > vscsi0 at root > > scsibus2 at vscsi0: 256 targets > > softraid0 at root > > scsibus3 at softraid0: 256 targets > > root on sd0a (886e07e83005c94c.a) swap on sd0b dump on sd0b > > > > dmesg (host) > > ============ > > > > OpenBSD 6.6 (GENERIC.MP) #0: Sat Oct 26 08:08:07 MDT 2019 > > > > [email protected]:/usr/src/sys/arch/amd64/compile/GENERIC.MP > > real mem = 34329825280 (32739MB) > > avail mem = 33276682240 (31735MB) > > mpath0 at root > > scsibus0 at mpath0: 256 targets > > mainbus0 at root > > bios0 at mainbus0: SMBIOS rev. 2.6 @ 0xcf49c000 (84 entries) > > bios0: vendor Dell Inc. version "6.4.0" date 07/23/2013 > > bios0: Dell Inc. PowerEdge R610 > > acpi0 at bios0: ACPI 3.0 > > acpi0: sleep states S0 S4 S5 > > acpi0: tables DSDT FACP APIC SPCR HPET DMAR MCFG WD__ SLIC ERST HEST > > BERT EINJ SRAT TCPA SSDT > > acpi0: wakeup devices PCI0(S5) > > acpitimer0 at acpi0: 3579545 Hz, 24 bits > > acpimadt0 at acpi0 addr 0xfee00000: PC-AT compat > > cpu0 at mainbus0: apid 32 (boot processor) > > cpu0: Intel(R) Xeon(R) CPU X5675 @ 3.07GHz, 3325.38 MHz, 06-2c-02 > > cpu0: > > FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,POPCNT,AES,NXE,PAGE1GB,RDTSCP,LONG,LAHF,PERF,ITSC,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,MELTDOWN > > cpu0: 256KB 64b/line 8-way L2 cache > > cpu0: smt 0, core 0, package 1 > > mtrr: Pentium Pro MTRR support, 10 var ranges, 88 fixed ranges > > cpu0: apic clock running at 133MHz > > cpu0: mwait min=64, max=64, C-substates=0.2.1.1, IBE > > cpu1 at mainbus0: apid 0 (application processor) > > cpu1: Intel(R) Xeon(R) CPU X5675 @ 3.07GHz, 1596.02 MHz, 06-2c-02 > > cpu1: > > FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,POPCNT,AES,NXE,PAGE1GB,RDTSCP,LONG,LAHF,PERF,ITSC,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,MELTDOWN > > cpu1: 256KB 64b/line 8-way L2 cache > > cpu1: smt 0, core 0, package 0 > > cpu2 at mainbus0: apid 34 (application processor) > > cpu2: Intel(R) Xeon(R) CPU X5675 @ 3.07GHz, 3325.02 MHz, 06-2c-02 > > cpu2: > > FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,POPCNT,AES,NXE,PAGE1GB,RDTSCP,LONG,LAHF,PERF,ITSC,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,MELTDOWN > > cpu2: 256KB 64b/line 8-way L2 cache > > cpu2: smt 0, core 1, package 1 > > cpu3 at mainbus0: apid 2 (application processor) > > cpu3: Intel(R) Xeon(R) CPU X5675 @ 3.07GHz, 3325.02 MHz, 06-2c-02 > > cpu3: > > FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,POPCNT,AES,NXE,PAGE1GB,RDTSCP,LONG,LAHF,PERF,ITSC,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,MELTDOWN > > cpu3: 256KB 64b/line 8-way L2 cache > > cpu3: smt 0, core 1, package 0 > > cpu4 at mainbus0: apid 36 (application processor) > > cpu4: Intel(R) Xeon(R) CPU X5675 @ 3.07GHz, 3325.02 MHz, 06-2c-02 > > cpu4: > > FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,POPCNT,AES,NXE,PAGE1GB,RDTSCP,LONG,LAHF,PERF,ITSC,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,MELTDOWN > > cpu4: 256KB 64b/line 8-way L2 cache > > cpu4: smt 0, core 2, package 1 > > cpu5 at mainbus0: apid 4 (application processor) > > cpu5: Intel(R) Xeon(R) CPU X5675 @ 3.07GHz, 3325.02 MHz, 06-2c-02 > > cpu5: > > FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,POPCNT,AES,NXE,PAGE1GB,RDTSCP,LONG,LAHF,PERF,ITSC,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,MELTDOWN > > cpu5: 256KB 64b/line 8-way L2 cache > > cpu5: smt 0, core 2, package 0 > > cpu6 at mainbus0: apid 48 (application processor) > > cpu6: Intel(R) Xeon(R) CPU X5675 @ 3.07GHz, 3325.02 MHz, 06-2c-02 > > cpu6: > > FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,POPCNT,AES,NXE,PAGE1GB,RDTSCP,LONG,LAHF,PERF,ITSC,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,MELTDOWN > > cpu6: 256KB 64b/line 8-way L2 cache > > cpu6: smt 0, core 8, package 1 > > cpu7 at mainbus0: apid 16 (application processor) > > cpu7: Intel(R) Xeon(R) CPU X5675 @ 3.07GHz, 3325.02 MHz, 06-2c-02 > > cpu7: > > FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,POPCNT,AES,NXE,PAGE1GB,RDTSCP,LONG,LAHF,PERF,ITSC,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,MELTDOWN > > cpu7: 256KB 64b/line 8-way L2 cache > > cpu7: smt 0, core 8, package 0 > > cpu8 at mainbus0: apid 50 (application processor) > > cpu8: Intel(R) Xeon(R) CPU X5675 @ 3.07GHz, 3325.02 MHz, 06-2c-02 > > cpu8: > > FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,POPCNT,AES,NXE,PAGE1GB,RDTSCP,LONG,LAHF,PERF,ITSC,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,MELTDOWN > > cpu8: 256KB 64b/line 8-way L2 cache > > cpu8: smt 0, core 9, package 1 > > cpu9 at mainbus0: apid 18 (application processor) > > cpu9: Intel(R) Xeon(R) CPU X5675 @ 3.07GHz, 3325.02 MHz, 06-2c-02 > > cpu9: > > FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,POPCNT,AES,NXE,PAGE1GB,RDTSCP,LONG,LAHF,PERF,ITSC,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,MELTDOWN > > cpu9: 256KB 64b/line 8-way L2 cache > > cpu9: smt 0, core 9, package 0 > > cpu10 at mainbus0: apid 52 (application processor) > > cpu10: Intel(R) Xeon(R) CPU X5675 @ 3.07GHz, 3325.02 MHz, 06-2c-02 > > cpu10: > > FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,POPCNT,AES,NXE,PAGE1GB,RDTSCP,LONG,LAHF,PERF,ITSC,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,MELTDOWN > > cpu10: 256KB 64b/line 8-way L2 cache > > cpu10: smt 0, core 10, package 1 > > cpu11 at mainbus0: apid 20 (application processor) > > cpu11: Intel(R) Xeon(R) CPU X5675 @ 3.07GHz, 3325.02 MHz, 06-2c-02 > > cpu11: > > FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,POPCNT,AES,NXE,PAGE1GB,RDTSCP,LONG,LAHF,PERF,ITSC,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,MELTDOWN > > cpu11: 256KB 64b/line 8-way L2 cache > > cpu11: smt 0, core 10, package 0 > > cpu12 at mainbus0: apid 33 (application processor) > > cpu12: Intel(R) Xeon(R) CPU X5675 @ 3.07GHz, 3325.01 MHz, 06-2c-02 > > cpu12: > > FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,POPCNT,AES,NXE,PAGE1GB,RDTSCP,LONG,LAHF,PERF,ITSC,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,MELTDOWN > > cpu12: 256KB 64b/line 8-way L2 cache > > cpu12: smt 1, core 0, package 1 > > cpu13 at mainbus0: apid 1 (application processor) > > cpu13: Intel(R) Xeon(R) CPU X5675 @ 3.07GHz, 3325.01 MHz, 06-2c-02 > > cpu13: > > FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,POPCNT,AES,NXE,PAGE1GB,RDTSCP,LONG,LAHF,PERF,ITSC,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,MELTDOWN > > cpu13: 256KB 64b/line 8-way L2 cache > > cpu13: smt 1, core 0, package 0 > > cpu14 at mainbus0: apid 35 (application processor) > > cpu14: Intel(R) Xeon(R) CPU X5675 @ 3.07GHz, 3325.01 MHz, 06-2c-02 > > cpu14: > > FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,POPCNT,AES,NXE,PAGE1GB,RDTSCP,LONG,LAHF,PERF,ITSC,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,MELTDOWN > > cpu14: 256KB 64b/line 8-way L2 cache > > cpu14: smt 1, core 1, package 1 > > cpu15 at mainbus0: apid 3 (application processor) > > cpu15: Intel(R) Xeon(R) CPU X5675 @ 3.07GHz, 3325.01 MHz, 06-2c-02 > > cpu15: > > FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,POPCNT,AES,NXE,PAGE1GB,RDTSCP,LONG,LAHF,PERF,ITSC,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,MELTDOWN > > cpu15: 256KB 64b/line 8-way L2 cache > > cpu15: smt 1, core 1, package 0 > > cpu16 at mainbus0: apid 37 (application processor) > > cpu16: Intel(R) Xeon(R) CPU X5675 @ 3.07GHz, 3325.01 MHz, 06-2c-02 > > cpu16: > > FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,POPCNT,AES,NXE,PAGE1GB,RDTSCP,LONG,LAHF,PERF,ITSC,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,MELTDOWN > > cpu16: 256KB 64b/line 8-way L2 cache > > cpu16: smt 1, core 2, package 1 > > cpu17 at mainbus0: apid 5 (application processor) > > cpu17: Intel(R) Xeon(R) CPU X5675 @ 3.07GHz, 3325.01 MHz, 06-2c-02 > > cpu17: > > FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,POPCNT,AES,NXE,PAGE1GB,RDTSCP,LONG,LAHF,PERF,ITSC,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,MELTDOWN > > cpu17: 256KB 64b/line 8-way L2 cache > > cpu17: smt 1, core 2, package 0 > > cpu18 at mainbus0: apid 49 (application processor) > > cpu18: Intel(R) Xeon(R) CPU X5675 @ 3.07GHz, 3299.21 MHz, 06-2c-02 > > cpu18: > > FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,POPCNT,AES,NXE,PAGE1GB,RDTSCP,LONG,LAHF,PERF,ITSC,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,MELTDOWN > > cpu18: 256KB 64b/line 8-way L2 cache > > cpu18: smt 1, core 8, package 1 > > cpu19 at mainbus0: apid 17 (application processor) > > cpu19: Intel(R) Xeon(R) CPU X5675 @ 3.07GHz, 3310.51 MHz, 06-2c-02 > > cpu19: > > FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,POPCNT,AES,NXE,PAGE1GB,RDTSCP,LONG,LAHF,PERF,ITSC,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,MELTDOWN > > cpu19: 256KB 64b/line 8-way L2 cache > > cpu19: smt 1, core 8, package 0 > > cpu20 at mainbus0: apid 51 (application processor) > > cpu20: Intel(R) Xeon(R) CPU X5675 @ 3.07GHz, 3259.43 MHz, 06-2c-02 > > cpu20: > > FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,POPCNT,AES,NXE,PAGE1GB,RDTSCP,LONG,LAHF,PERF,ITSC,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,MELTDOWN > > cpu20: 256KB 64b/line 8-way L2 cache > > cpu20: smt 1, core 9, package 1 > > cpu21 at mainbus0: apid 19 (application processor) > > cpu21: Intel(R) Xeon(R) CPU X5675 @ 3.07GHz, 3270.82 MHz, 06-2c-02 > > cpu21: > > FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,POPCNT,AES,NXE,PAGE1GB,RDTSCP,LONG,LAHF,PERF,ITSC,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,MELTDOWN > > cpu21: 256KB 64b/line 8-way L2 cache > > cpu21: smt 1, core 9, package 0 > > cpu22 at mainbus0: apid 53 (application processor) > > cpu22: Intel(R) Xeon(R) CPU X5675 @ 3.07GHz, 3225.92 MHz, 06-2c-02 > > cpu22: > > FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,POPCNT,AES,NXE,PAGE1GB,RDTSCP,LONG,LAHF,PERF,ITSC,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,MELTDOWN > > cpu22: 256KB 64b/line 8-way L2 cache > > cpu22: smt 1, core 10, package 1 > > cpu23 at mainbus0: apid 21 (application processor) > > cpu23: Intel(R) Xeon(R) CPU X5675 @ 3.07GHz, 3242.14 MHz, 06-2c-02 > > cpu23: > > FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,POPCNT,AES,NXE,PAGE1GB,RDTSCP,LONG,LAHF,PERF,ITSC,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,MELTDOWN > > cpu23: 256KB 64b/line 8-way L2 cache > > cpu23: smt 1, core 10, package 0 > > ioapic0 at mainbus0: apid 0 pa 0xfec00000, version 20, 24 pins > > ioapic1 at mainbus0: apid 1 pa 0xfec80000, version 20, 24 pins, remapped > > acpihpet0 at acpi0: 14318179 Hz > > acpimcfg0 at acpi0 > > acpimcfg0: addr 0xe0000000, bus 0-255 > > acpiprt0 at acpi0: bus 0 (PCI0) > > acpiprt1 at acpi0: bus 1 (PEX1) > > acpiprt2 at acpi0: bus 2 (PEX3) > > acpiprt3 at acpi0: bus -1 (PEX4) > > acpiprt4 at acpi0: bus -1 (PEX5) > > acpiprt5 at acpi0: bus -1 (PEX6) > > acpiprt6 at acpi0: bus 4 (PEX7) > > acpiprt7 at acpi0: bus 8 (PEX9) > > acpiprt8 at acpi0: bus -1 (PEXA) > > acpiprt9 at acpi0: bus 3 (SBEX) > > acpiprt10 at acpi0: bus 9 (COMP) > > acpicpu0 at acpi0: C3(350@96 mwait.1@0x20), C1(1000@1 mwait.1) > > acpicpu1 at acpi0: C3(350@96 mwait.1@0x20), C1(1000@1 mwait.1) > > acpicpu2 at acpi0: C3(350@96 mwait.1@0x20), C1(1000@1 mwait.1) > > acpicpu3 at acpi0: C3(350@96 mwait.1@0x20), C1(1000@1 mwait.1) > > acpicpu4 at acpi0: C3(350@96 mwait.1@0x20), C1(1000@1 mwait.1) > > acpicpu5 at acpi0: C3(350@96 mwait.1@0x20), C1(1000@1 mwait.1) > > acpicpu6 at acpi0: C3(350@96 mwait.1@0x20), C1(1000@1 mwait.1) > > acpicpu7 at acpi0: C3(350@96 mwait.1@0x20), C1(1000@1 mwait.1) > > acpicpu8 at acpi0: C3(350@96 mwait.1@0x20), C1(1000@1 mwait.1) > > acpicpu9 at acpi0: C3(350@96 mwait.1@0x20), C1(1000@1 mwait.1) > > acpicpu10 at acpi0: C3(350@96 mwait.1@0x20), C1(1000@1 mwait.1) > > acpicpu11 at acpi0: C3(350@96 mwait.1@0x20), C1(1000@1 mwait.1) > > acpicpu12 at acpi0: C3(350@96 mwait.1@0x20), C1(1000@1 mwait.1) > > acpicpu13 at acpi0: C3(350@96 mwait.1@0x20), C1(1000@1 mwait.1) > > acpicpu14 at acpi0: C3(350@96 mwait.1@0x20), C1(1000@1 mwait.1) > > acpicpu15 at acpi0: C3(350@96 mwait.1@0x20), C1(1000@1 mwait.1) > > acpicpu16 at acpi0: C3(350@96 mwait.1@0x20), C1(1000@1 mwait.1) > > acpicpu17 at acpi0: C3(350@96 mwait.1@0x20), C1(1000@1 mwait.1) > > acpicpu18 at acpi0: C3(350@96 mwait.1@0x20), C1(1000@1 mwait.1) > > acpicpu19 at acpi0: C3(350@96 mwait.1@0x20), C1(1000@1 mwait.1) > > acpicpu20 at acpi0: C3(350@96 mwait.1@0x20), C1(1000@1 mwait.1) > > acpicpu21 at acpi0: C3(350@96 mwait.1@0x20), C1(1000@1 mwait.1) > > acpicpu22 at acpi0: C3(350@96 mwait.1@0x20), C1(1000@1 mwait.1) > > acpicpu23 at acpi0: C3(350@96 mwait.1@0x20), C1(1000@1 mwait.1) > > acpipci0 at acpi0 PCI0: 0x00000010 0x00000011 0x00000000 > > acpicmos0 at acpi0 > > "PNP0C14" at acpi0 not configured > > ipmi at mainbus0 not configured > > cpu0: using IvyBridge MDS workaround > > pci0 at mainbus0 bus 0 > > pchb0 at pci0 dev 0 function 0 "Intel 5500 Host" rev 0x13 > > ppb0 at pci0 dev 1 function 0 "Intel X58 PCIE" rev 0x13 > > pci1 at ppb0 bus 1 > > bnx0 at pci1 dev 0 function 0 "Broadcom BCM5709" rev 0x20: apic 1 int 4 > > bnx1 at pci1 dev 0 function 1 "Broadcom BCM5709" rev 0x20: apic 1 int 16 > > ppb1 at pci0 dev 3 function 0 "Intel X58 PCIE" rev 0x13 > > pci2 at ppb1 bus 2 > > bnx2 at pci2 dev 0 function 0 "Broadcom BCM5709" rev 0x20: apic 1 int 0 > > bnx3 at pci2 dev 0 function 1 "Broadcom BCM5709" rev 0x20: apic 1 int 10 > > ppb2 at pci0 dev 7 function 0 "Intel X58 PCIE" rev 0x13: msi > > pci3 at ppb2 bus 4 > > ppb3 at pci3 dev 0 function 0 "IDT 89HPES12N3A" rev 0x0e > > pci4 at ppb3 bus 5 > > ppb4 at pci4 dev 2 function 0 "IDT 89HPES12N3A" rev 0x0e > > pci5 at ppb4 bus 6 > > em0 at pci5 dev 0 function 0 "Intel 82571EB" rev 0x06: apic 1 int 14, > > address 00:26:55:eb:7d:6d > > em1 at pci5 dev 0 function 1 "Intel 82571EB" rev 0x06: apic 1 int 15, > > address 00:26:55:eb:7d:6c > > ppb5 at pci4 dev 4 function 0 "IDT 89HPES12N3A" rev 0x0e > > pci6 at ppb5 bus 7 > > em2 at pci6 dev 0 function 0 "Intel 82571EB" rev 0x06: apic 1 int 13, > > address 00:26:55:eb:7d:6f > > em3 at pci6 dev 0 function 1 "Intel 82571EB" rev 0x06: apic 1 int 6, > > address 00:26:55:eb:7d:6e > > ppb6 at pci0 dev 9 function 0 "Intel X58 PCIE" rev 0x13: msi > > pci7 at ppb6 bus 8 > > "Intel X58 Misc" rev 0x13 at pci0 dev 20 function 0 not configured > > "Intel X58 GPIO" rev 0x13 at pci0 dev 20 function 1 not configured > > "Intel X58 RAS" rev 0x13 at pci0 dev 20 function 2 not configured > > uhci0 at pci0 dev 26 function 0 "Intel 82801I USB" rev 0x02: apic 0 int 17 > > uhci1 at pci0 dev 26 function 1 "Intel 82801I USB" rev 0x02: apic 0 int 18 > > ehci0 at pci0 dev 26 function 7 "Intel 82801I USB" rev 0x02: apic 0 int 19 > > usb0 at ehci0: USB revision 2.0 > > uhub0 at usb0 configuration 1 interface 0 "Intel EHCI root hub" rev > > 2.00/1.00 addr 1 > > ppb7 at pci0 dev 28 function 0 "Intel 82801I PCIE" rev 0x02 > > pci8 at ppb7 bus 3 > > mfi0 at pci8 dev 0 function 0 "Symbios Logic SAS1078" rev 0x04: apic 0 int > > 16 > > mfi0: "PERC 6/i Integrated", firmware 6.3.3.0002, 256MB cache > > scsibus1 at mfi0: 64 targets > > sd0 at scsibus1 targ 0 lun 0: <DELL, PERC 6/i, 1.22> > > naa.60024e807b228b00238ea8a404afdf3d > > sd0: 139392MB, 512 bytes/sector, 285474816 sectors > > sd1 at scsibus1 targ 1 lun 0: <DELL, PERC 6/i, 1.22> > > naa.60024e807b228b00238fe1a0062cc8ce > > sd1: 2573568MB, 512 bytes/sector, 5270667264 sectors > > uhci2 at pci0 dev 29 function 0 "Intel 82801I USB" rev 0x02: apic 0 int 21 > > uhci3 at pci0 dev 29 function 1 "Intel 82801I USB" rev 0x02: apic 0 int 20 > > ehci1 at pci0 dev 29 function 7 "Intel 82801I USB" rev 0x02: apic 0 int 21 > > usb1 at ehci1: USB revision 2.0 > > uhub1 at usb1 configuration 1 interface 0 "Intel EHCI root hub" rev > > 2.00/1.00 addr 1 > > ppb8 at pci0 dev 30 function 0 "Intel 82801BA Hub-to-PCI" rev 0x92 > > pci9 at ppb8 bus 9 > > vga1 at pci9 dev 3 function 0 "Matrox MGA G200eW" rev 0x0a > > wsdisplay0 at vga1 mux 1: console (80x25, vt100 emulation) > > wsdisplay0: screen 1-5 added (80x25, vt100 emulation) > > pcib0 at pci0 dev 31 function 0 "Intel 82801IB LPC" rev 0x02 > > pciide0 at pci0 dev 31 function 2 "Intel 82801I SATA" rev 0x02: DMA, > > channel 0 configured to native-PCI, channel 1 configured to native-PCI > > pciide0: using apic 0 int 23 for native-PCI interrupt > > usb2 at uhci0: USB revision 1.0 > > uhub2 at usb2 configuration 1 interface 0 "Intel UHCI root hub" rev > > 1.00/1.00 addr 1 > > usb3 at uhci1: USB revision 1.0 > > uhub3 at usb3 configuration 1 interface 0 "Intel UHCI root hub" rev > > 1.00/1.00 addr 1 > > usb4 at uhci2: USB revision 1.0 > > uhub4 at usb4 configuration 1 interface 0 "Intel UHCI root hub" rev > > 1.00/1.00 addr 1 > > usb5 at uhci3: USB revision 1.0 > > uhub5 at usb5 configuration 1 interface 0 "Intel UHCI root hub" rev > > 1.00/1.00 addr 1 > > isa0 at pcib0 > > isadma0 at isa0 > > com0 at isa0 port 0x3f8/8 irq 4: ns16550a, 16 byte fifo > > com0: console > > com1 at isa0 port 0x2f8/8 irq 3: ns16550a, 16 byte fifo > > pckbc0 at isa0 port 0x60/5 irq 1 irq 12 > > pckbd0 at pckbc0 (kbd slot) > > wskbd0 at pckbd0: console keyboard, using wsdisplay0 > > pcppi0 at isa0 port 0x61 > > spkr0 at pcppi0 > > vmm0 at mainbus0: VMX/EPT > > uhub6 at uhub0 port 3 configuration 1 interface 0 "Standard > > Microsystems product 0x2514" rev 2.00/0.00 addr 2 > > uhidev0 at uhub4 port 2 configuration 1 interface 0 "Avocent USB > > Composite Device-0" rev 1.10/0.00 addr 2 > > uhidev0: iclass 3/1 > > ukbd0 at uhidev0: 8 variable keys, 6 key codes > > wskbd1 at ukbd0 mux 1 > > wskbd1: connecting to wsdisplay0 > > uhidev1 at uhub4 port 2 configuration 1 interface 1 "Avocent USB > > Composite Device-0" rev 1.10/0.00 addr 2 > > uhidev1: iclass 3/1 > > ums0 at uhidev1: 3 buttons, Z dir > > wsmouse0 at ums0 mux 0 > > vscsi0 at root > > scsibus2 at vscsi0: 256 targets > > softraid0 at root > > scsibus3 at softraid0: 256 targets > > root on sd0a (6d624779831add1c.a) swap on sd0b dump on sd0b > > bnx0: address 00:22:19:65:bb:0a > > brgphy0 at bnx0 phy 1: BCM5709 10/100/1000baseT PHY, rev. 8 > > bnx1: address 00:22:19:65:bb:0c > > brgphy1 at bnx1 phy 1: BCM5709 10/100/1000baseT PHY, rev. 8 > > bnx2: address 00:22:19:65:bb:0e > > brgphy2 at bnx2 phy 1: BCM5709 10/100/1000baseT PHY, rev. 8 > > bnx3: address 00:22:19:65:bb:10 > > brgphy3 at bnx3 phy 1: BCM5709 10/100/1000baseT PHY, rev. 8
