On Wed, Jan 09, 2019 at 11:18:16PM -0600, Brian Conway wrote:
> After looking through the mailing list archives, I'm seeking advice on
> running an OpenBSD i386 guest successfully without losing too much
> time. My host is an 8th-gen Intel NUC (dmesg *2 follows), and both the
> host and any OpenBSD amd64 guest keep time beautifully using the tsc
> time source. I've needed to make no changes to sysctl or HZ as
> compiled in the kernel.
>
> When firing up an i386 guest to do some release(8) builds, I have no
> issues with running those tasks, but the system loses time on the
> order of 4-5 for every 20, even when sitting idle. It appears that
> there is no tsc source found on i386, and taking a peak at
> src/sys/arch/i386/i386 indicates that perhaps this isn't available?
>
> What's the preferred method to get such a guest in line? I've included
> various sysctl, ntpd, and dmesg output below. All systems are running
> 6.4-stable. Thanks!
>
> Brian Conway
>
> amd64 host kern.timecounter (amd64 guest is similar):
> # sysctl kern.timecounter
> kern.timecounter.tick=1
> kern.timecounter.timestepwarnings=0
> kern.timecounter.hardware=tsc
> kern.timecounter.choice=i8254(0) acpihpet0(1000) tsc(2000)
> acpitimer0(1000) dummy(-1000000)
>
> i386 guest:
> kern.timecounter.tick=1
> kern.timecounter.timestepwarnings=0
> kern.timecounter.hardware=i8254
> kern.timecounter.choice=i8254(0) dummy(-1000000)
>
> i386 guest after 20 minutes of uptime (jitter and delay reflect
> redirection to a local ntp server):
> -----
> $ ntpctl -s all
> 4/4 peers valid, 1/1 sensors valid, constraint offset 3s (4 errors),
> clock unsynced, clock offset is 26743.868ms
>
> peer
> wt tl st next poll offset delay jitter
> 107.181.191.189 from pool us.pool.ntp.org
> 1 10 3 1945s 3135s 6537.870ms 1.062ms 0.316ms
> 142.147.92.5 from pool us.pool.ntp.org
> 1 10 3 1898s 3086s 5915.575ms 0.854ms 0.082ms
> 216.229.4.69 from pool us.pool.ntp.org
> 1 10 3 1928s 3115s 5919.290ms 0.887ms 0.110ms
> 216.6.2.70 from pool us.pool.ntp.org
> 1 10 3 1825s 3017s 3889.790ms 0.852ms 0.115ms
>
> sensor
> wt gd st next poll offset correction
> vmmci0
> 1 1 0 10s 15s 1277789.475ms 0.000ms
> -----
>
> i386 guest ntpd logs:
> -----
> Jan 10 01:41:46 b64-i386 ntpd[5306]: ntp engine ready
> Jan 10 01:41:47 b64-i386 ntpd[98096]: set local clock to Thu Jan 10
> 01:41:47 UTC 2019 (offset 0.647554s)
> Jan 10 01:41:47 b64-i386 savecore: no core dump
> Jan 10 01:41:49 b64-i386 ntpd[5306]: constraint reply from
> 172.217.2.228: offset 2.592767
> Jan 10 01:42:09 b64-i386 ntpd[5306]: peer 216.229.4.69 now valid
> Jan 10 01:42:11 b64-i386 ntpd[5306]: peer 216.6.2.70 now valid
> Jan 10 01:42:12 b64-i386 ntpd[5306]: peer 142.147.92.5 now valid
> Jan 10 01:42:14 b64-i386 ntpd[5306]: peer 107.181.191.189 now valid
> Jan 10 01:43:05 b64-i386 ntpd[81307]: adjusting local clock by 33.008869s
> Jan 10 01:44:06 b64-i386 ntpd[5306]: reply from 216.6.2.70: constraint
> check failed
> Jan 10 01:44:08 b64-i386 ntpd[5306]: reply from 107.181.191.189:
> constraint check failed
> Jan 10 01:44:10 b64-i386 ntpd[5306]: reply from 142.147.92.5:
> constraint check failed
> Jan 10 01:44:11 b64-i386 ntpd[5306]: reply from 216.229.4.69:
> constraint check failed
> ...
> -----
>
> amd64 host dmesg:
> -----
> OpenBSD 6.4-stable (GENERIC.MP) #3: Mon Jan 7 01:46:29 UTC 2019
> bcon...@nc.int.rcesoftware.com:/usr/src/sys/arch/amd64/compile/GENERIC.MP
> real mem = 8452431872 (8060MB)
> avail mem = 8187002880 (7807MB)
> mpath0 at root
> scsibus0 at mpath0: 256 targets
> mainbus0 at root
> bios0 at mainbus0: SMBIOS rev. 2.8 @ 0x7a9a4000 (76 entries)
> bios0: vendor Intel Corp. version "BECFL357.86A.0056.2018.1128.1717"
> date 11/28/2018
> bios0: Intel(R) Client Systems NUC8i5BEH
> acpi0 at bios0: rev 2
> acpi0: sleep states S0 S3 S4 S5
> acpi0: tables DSDT FACP APIC FPDT FIDT MCFG SSDT SSDT SSDT HPET SSDT
> SSDT UEFI LPIT SSDT SSDT DBGP DBG2 SSDT DMAR BGRT WSMT
> acpi0: wakeup devices SIO1(S3) PXSX(S4) RP01(S4) PXSX(S4) RP02(S4)
> PXSX(S4) RP03(S4) PXSX(S4) RP04(S4) PXSX(S4) RP05(S4) PXSX(S4)
> RP06(S4) PXSX(S4) RP07(S4) PXSX(S4) [...]
> acpitimer0 at acpi0: 3579545 Hz, 24 bits
> acpimadt0 at acpi0 addr 0xfee00000: PC-AT compat
> cpu0 at mainbus0: apid 0 (boot processor)
> cpu0: Intel(R) Core(TM) i5-8259U CPU @ 2.30GHz, 2195.91 MHz, 06-8e-0a
> cpu0:
> FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,3DNOWP,PERF,ITSC,FSGSBASE,SGX,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,MPX,RDSEED,ADX,SMAP,CLFLUSHOPT,PT,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,XSAVEC,XGETBV1,XSAVES,MELTDOWN
> cpu0: 256KB 64b/line 8-way L2 cache
> cpu0: smt 0, core 0, package 0
> mtrr: Pentium Pro MTRR support, 10 var ranges, 88 fixed ranges
> cpu0: apic clock running at 24MHz
> cpu0: mwait min=64, max=64, C-substates=0.2.1.2.4.1.1.1, IBE
> cpu1 at mainbus0: apid 2 (application processor)
> cpu1: Intel(R) Core(TM) i5-8259U CPU @ 2.30GHz, 2194.93 MHz, 06-8e-0a
> cpu1:
> FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,3DNOWP,PERF,ITSC,FSGSBASE,SGX,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,MPX,RDSEED,ADX,SMAP,CLFLUSHOPT,PT,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,XSAVEC,XGETBV1,XSAVES,MELTDOWN
> cpu1: 256KB 64b/line 8-way L2 cache
> cpu1: smt 0, core 1, package 0
> cpu2 at mainbus0: apid 4 (application processor)
> cpu2: Intel(R) Core(TM) i5-8259U CPU @ 2.30GHz, 2194.93 MHz, 06-8e-0a
> cpu2:
> FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,3DNOWP,PERF,ITSC,FSGSBASE,SGX,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,MPX,RDSEED,ADX,SMAP,CLFLUSHOPT,PT,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,XSAVEC,XGETBV1,XSAVES,MELTDOWN
> cpu2: 256KB 64b/line 8-way L2 cache
> cpu2: smt 0, core 2, package 0
> cpu3 at mainbus0: apid 6 (application processor)
> cpu3: Intel(R) Core(TM) i5-8259U CPU @ 2.30GHz, 2194.93 MHz, 06-8e-0a
> cpu3:
> FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,3DNOWP,PERF,ITSC,FSGSBASE,SGX,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,MPX,RDSEED,ADX,SMAP,CLFLUSHOPT,PT,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,XSAVEC,XGETBV1,XSAVES,MELTDOWN
> cpu3: 256KB 64b/line 8-way L2 cache
> cpu3: smt 0, core 3, package 0
> ioapic0 at mainbus0: apid 2 pa 0xfec00000, version 20, 120 pins
> acpimcfg0 at acpi0
> acpimcfg0: addr 0xe0000000, bus 0-255
> acpihpet0 at acpi0: 23999999 Hz
> acpiprt0 at acpi0: bus 0 (PCI0)
> acpiprt1 at acpi0: bus -1 (PEG0)
> acpiprt2 at acpi0: bus -1 (PEG1)
> acpiprt3 at acpi0: bus -1 (PEG2)
> acpiprt4 at acpi0: bus -1 (RP01)
> acpiprt5 at acpi0: bus -1 (RP02)
> acpiprt6 at acpi0: bus -1 (RP03)
> acpiprt7 at acpi0: bus -1 (RP04)
> acpiprt8 at acpi0: bus -1 (RP05)
> acpiprt9 at acpi0: bus -1 (RP06)
> acpiprt10 at acpi0: bus -1 (RP07)
> acpiprt11 at acpi0: bus -1 (RP08)
> acpiprt12 at acpi0: bus -1 (RP09)
> acpiprt13 at acpi0: bus -1 (RP10)
> acpiprt14 at acpi0: bus -1 (RP11)
> acpiprt15 at acpi0: bus -1 (RP12)
> acpiprt16 at acpi0: bus -1 (RP13)
> acpiprt17 at acpi0: bus -1 (RP14)
> acpiprt18 at acpi0: bus -1 (RP15)
> acpiprt19 at acpi0: bus -1 (RP16)
> acpiprt20 at acpi0: bus -1 (RP17)
> acpiprt21 at acpi0: bus -1 (RP18)
> acpiprt22 at acpi0: bus -1 (RP19)
> acpiprt23 at acpi0: bus -1 (RP20)
> acpiprt24 at acpi0: bus -1 (RP21)
> acpiprt25 at acpi0: bus -1 (RP22)
> acpiprt26 at acpi0: bus -1 (RP23)
> acpiprt27 at acpi0: bus -1 (RP24)
> acpiec0 at acpi0
> acpitz0 at acpi0acpitz0: TZ0_: failed to read _HOT
> acpitz0: TZ0_: failed to read _TMP
> acpitz0: TZ0_: failed to read _TMP
> acpipwrres0 at acpi0: USBC, resource for XDCI
> acpipwrres1 at acpi0: V0PR
> acpipwrres2 at acpi0: V1PR
> acpipwrres3 at acpi0: V2PR
> acpipwrres4 at acpi0: WRST
> acpicpu0 at acpi0: C3(200@1034 mwait.1@0x60), C2(200@151
> mwait.1@0x33), C1(1000@1 mwait.1), PSS
> acpicpu1 at acpi0: C3(200@1034 mwait.1@0x60), C2(200@151
> mwait.1@0x33), C1(1000@1 mwait.1), PSS
> acpicpu2 at acpi0: C3(200@1034 mwait.1@0x60), C2(200@151
> mwait.1@0x33), C1(1000@1 mwait.1), PSS
> acpicpu3 at acpi0: C3(200@1034 mwait.1@0x60), C2(200@151
> mwait.1@0x33), C1(1000@1 mwait.1), PSS
> acpipwrres5 at acpi0: FN00, resource for FAN0
> acpipwrres6 at acpi0: FN01, resource for FAN1
> acpipwrres7 at acpi0: FN02, resource for FAN2
> acpipwrres8 at acpi0: FN03, resource for FAN3
> acpipwrres9 at acpi0: FN04, resource for FAN4
> acpitz1 at acpi0: critical temperature is 119 degC
> acpipwrres10 at acpi0: PIN_
> acpicmos0 at acpi0
> "INT34BB" at acpi0 not configured
> "PNP0C14" at acpi0 not configured
> acpibtn0 at acpi0: SLPB
> "PNP0C14" at acpi0 not configured
> "PNP0C14" at acpi0 not configured
> "INT33A1" at acpi0 not configured
> acpibtn1 at acpi0: PWRB
> "PNP0C0B" at acpi0 not configured
> "PNP0C0B" at acpi0 not configured
> "PNP0C0B" at acpi0 not configured
> "PNP0C0B" at acpi0 not configured
> "PNP0C0B" at acpi0 not configured
> "PNP0C14" at acpi0 not configured
> acpivideo0 at acpi0: GFX0
> acpivout0 at acpivideo0: DD1F
> cpu0: Enhanced SpeedStep 2195 MHz: speeds: 2301, 2300, 2200, 2000,
> 1900, 1800, 1600, 1500, 1300, 1100, 1000, 900, 800, 700, 500, 400 MHz
> pci0 at mainbus0 bus 0
> 0:31:5: mem address conflict 0xfe010000/0x1000
> pchb0 at pci0 dev 0 function 0 vendor "Intel", unknown product 0x3ed0 rev 0x08
> vendor "Intel", unknown product 0x3ea5 (class display subclass VGA,
> rev 0x01) at pci0 dev 2 function 0 not configured
> "Intel Core GMM" rev 0x00 at pci0 dev 8 function 0 not configured
> vendor "Intel", unknown product 0x9df9 (class DASP subclass
> miscellaneous, rev 0x30) at pci0 dev 18 function 0 not configured
> xhci0 at pci0 dev 20 function 0 vendor "Intel", unknown product 0x9ded
> rev 0x30: msi, xHCI 1.16
> usb0 at xhci0: USB revision 3.0
> uhub0 at usb0 configuration 1 interface 0 "Intel xHCI root hub" rev
> 3.00/1.00 addr 1
> vendor "Intel", unknown product 0x9def (class memory subclass RAM, rev
> 0x30) at pci0 dev 20 function 2 not configured
> vendor "Intel", unknown product 0x9df0 (class network subclass
> miscellaneous, rev 0x30) at pci0 dev 20 function 3 not configured
> vendor "Intel", unknown product 0x9de0 (class communications subclass
> miscellaneous, rev 0x30) at pci0 dev 22 function 0 not configured
> ahci0 at pci0 dev 23 function 0 vendor "Intel", unknown product 0x9dd3
> rev 0x30: msi, AHCI 1.3.1
> ahci0: port 2: 6.0Gb/s
> scsibus1 at ahci0: 32 targets
> sd0 at scsibus1 targ 2 lun 0: <ATA, Crucial_CT240M50, MU05> SCSI3
> 0/direct fixed naa.500a07510c06ed9d
> sd0: 228936MB, 512 bytes/sector, 468862128 sectors, thin
> pcib0 at pci0 dev 31 function 0 vendor "Intel", unknown product 0x9d84 rev
> 0x30
> vendor "Intel", unknown product 0x9da3 (class serial bus subclass
> SMBus, rev 0x30) at pci0 dev 31 function 4 not configured
> vendor "Intel", unknown product 0x9da4 (class serial bus unknown
> subclass 0x80, rev 0x30) at pci0 dev 31 function 5 not configured
> em0 at pci0 dev 31 function 6 "Intel I219-V" rev 0x30: msi, address
> 94:c6:91:a8:54:23
> isa0 at pcib0
> isadma0 at isa0
> com0 at isa0 port 0x3f8/8 irq 4: ns16550a, 16 byte fifo
> com0: probed fifo depth: 0 bytes
> pcppi0 at isa0 port 0x61
> spkr0 at pcppi0
> vmm0 at mainbus0: VMX/EPT
> efifb0 at mainbus0: 1680x1050, 32bpp
> wsdisplay0 at efifb0 mux 1: console (std, vt100 emulation)
> wsdisplay0: screen 1-5 added (std, vt100 emulation)
> vscsi0 at root
> scsibus2 at vscsi0: 256 targets
> softraid0 at root
> scsibus3 at softraid0: 256 targets
> root on sd0a (207110c0facab1e3.a) swap on sd0b dump on sd0b
> -----
>
> i386 guest dmesg:
> -----
> OpenBSD 6.4-stable (GENERIC) #1: Tue Jan 8 07:37:53 UTC 2019
>
> bcon...@b64-i386.int.rcesoftware.com:/usr/src/sys/arch/i386/compile/GENERIC
> real mem = 1073225728 (1023MB)
> avail mem = 1038622720 (990MB)
> mpath0 at root
> scsibus0 at mpath0: 256 targets
> mainbus0 at root
> bios0 at mainbus0: date 06/23/99, BIOS32 rev. 0 @ 0xfdc65, SMBIOS rev.
> 2.4 @ 0xf1950 (10 entries)
> bios0: vendor SeaBIOS version "1.11.0p0-OpenBSD-vmm" date 01/01/2011
> bios0: OpenBSD VMM
> acpi at bios0 function 0x0 not configured
> pcibios at bios0 function 0x1a not configured
> bios0: ROM list: 0xef000/0x1000!
> cpu0 at mainbus0: (uniprocessor)
> cpu0: Intel(R) Core(TM) i5-8259U CPU @ 2.30GHz ("GenuineIntel"
> 686-class) 2.31 GHz, 06-8e-0a
> cpu0:
> FPU,V86,DE,PSE,TSC,MSR,PAE,MCE,CX8,SEP,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,SSE3,PCLMUL,SSSE3,FMA3,CX16,SSE4.1,SSE4.2,MOVBE,POPCNT,AES,XSAVE,AVX,F16C,RDRAND,HV,NXE,PAGE1GB,LONG,LAHF,ABM,3DNOWP,ITSC,FSGSBASE,BMI1,AVX2,SMEP,BMI2,ERMS,RDSEED,ADX,SMAP,CLFLUSHOPT,MELTDOWN
> pvbus0 at mainbus0: OpenBSD
> pci0 at mainbus0 bus 0: configuration mode 1 (no bios)
> pchb0 at pci0 dev 0 function 0 "OpenBSD VMM Host" rev 0x00
> virtio0 at pci0 dev 1 function 0 "Qumranet Virtio RNG" rev 0x00
> viornd0 at virtio0
> virtio0: irq 3
> virtio1 at pci0 dev 2 function 0 "Qumranet Virtio Storage" rev 0x00
> vioblk0 at virtio1
> scsibus1 at vioblk0: 2 targets
> sd0 at scsibus1 targ 0 lun 0: <VirtIO, Block Device, > SCSI3 0/direct fixed
> sd0: 15360MB, 512 bytes/sector, 31457280 sectors
> virtio1: irq 5
> virtio2 at pci0 dev 3 function 0 "Qumranet Virtio Network" rev 0x00
> vio0 at virtio2: address fe:e1:bb:d1:15:e5
> virtio2: irq 6
> virtio3 at pci0 dev 4 function 0 "OpenBSD VMM Control" rev 0x00
> vmmci0 at virtio3
> virtio3: irq 7
> isa0 at mainbus0
> isadma0 at isa0
> com0 at isa0 port 0x3f8/8 irq 4: ns16450, no fifo
> com0: console
> pcdisplay0 at isa0 port 0x3d0/16 iomem 0xb8000/32768
> wsdisplay0 at pcdisplay0 mux 1: console (80x25, vt100 emulation)
> npx0 at isa0 port 0xf0/16: reported by CPUID; using exception 16
> vscsi0 at root
> scsibus2 at vscsi0: 256 targets
> softraid0 at root
> scsibus3 at softraid0: 256 targets
> root on sd0a (f5af63032b1e4a54.a) swap on sd0b dump on sd0b
> -----
>
i386 is really bad since it doesn't have TSC.
Eventually I got tired of trying to make that work and just put this in
root's crontab on the VM:
1,15,30,45 * * * * /usr/sbin/rdate pool.ntp.org >
/dev/null
plus run ntpd to keep things in sync as best as it can be during the 15 minute
intervals there. It's not particularly secure, but my only i386 VMs are just for
testing builds and kernels.
I doubt i386 will get better until someone implements an HPET in vmd.
-ml
