Hi,
Is using TSC also valid for linux guests? I see that there are no
desynchronization problems with OpenBSD guests using TSC. On the other
hand, I haven't found a method to obtain a similar behavior under
Linux (at least Ubuntu 18.04 and Alpine Linux 3.7).
alpine:~$ dmesg | grep -i tsc
[ 0.000000] Command line: BOOT_IMAGE=vmlinuz-virthardened
root=UUID=72e7a928-6a6e-4ae5-96a6-2637af5153b2
modules=sd-mod,usb-storage,ext4 rootfstype=ext4 acpi=off noapic
console=ttyS0,
115200 clocksource=tsc initrd=initramfs-virthardened
[ 0.00000000] Kernel command line: BOOT_IMAGE=vmlinuz-virthardened
root=UUID=72e7a928-6a6e-4ae5-96a6-2637af5153b2
modules=sd-mod,usb-storage,ext4 rootfstype=ext4 acpi=off noapic
console
=ttyS0,115200 clocksource=tsc initrd=initramfs-virthardened
[ 0.00000000] tsc: Fast TSC calibration failed
[ 0.00000000] tsc: Unable to calibrate against PIT
[ 0.00000000] tsc: No reference (HPET/PMTIMER) available
[ 0.00000000] tsc: Marking TSC unstable due to could not calculate TSC khz
alpine:~$ dmesg | grep clocksource
[ 0.000000] Command line: BOOT_IMAGE=vmlinuz-virthardened
root=UUID=72e7a928-6a6e-4ae5-96a6-2637af5153b2
modules=sd-mod,usb-storage,ext4 rootfstype=ext4 acpi=off noapic
console=ttyS0,115200 clocksource=tsc initrd=initramfs-virthardened
[ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff
max_cycles: 0xffffffff, max_idle_ns: 19112604462750000 ns
[ 0.000000] Kernel command line: BOOT_IMAGE=vmlinuz-virthardened
root=UUID=72e7a928-6a6e-4ae5-96a6-2637af5153b2
modules=sd-mod,usb-storage,ext4 rootfstype=ext4 acpi=off noapic
console
=ttyS0,115200 clocksource=tsc initrd=initramfs-virthardened
[ 1.230000] clocksource: jiffies: mask: 0xffffffff max_cycles:
0xffffffff, max_idle_ns: 19112604462750000 ns
[ 1.310000] clocksource: Switched to clocksource refined-jiffies
As a result, the guest on linux drifts terribly.
VMM(8) host dmesg :
OpenBSD 6.4-beta (GENERIC.MP) #287: Tue Sep 4 20:38:56 MDT 2018
[email protected]:/usr/src/sys/arch/amd64/compile/GENERIC.MP
real mem = 8410218496 (8020MB)
avail mem = 8146083840 (7768MB)
mpath0 at root
scsibus0 at mpath0: 256 targets
mainbus0 at root
bios0 at mainbus0: SMBIOS rev. 2.7 @ 0xdabf4000 (48 entries)
bios0: vendor LENOVO version "H4ET99WW (2.59 )" date 04/11/2018
bios0: LENOVO 3358AXG
acpi0 at bios0: rev 2
acpi0: sleep states S0 S3 S4 S5
acpi0: tables DSDT FACP SLIC ASF! HPET APIC MCFG FPDT SSDT SSDT UEFI
UEFI MSDM UEFI DBG2
acpi0: wakeup devices P0P1(S4) EHC1(S3) EHC2(S3) XHC_(S3) HDEF(S4)
RP04(S4) PXSX(S4) RP06(S4) PXSX(S4) BLAN(S4) PEG0(S4) PEGP(S4)
PEG1(S4) PEG2(S4) PEG3(S4) LID_(S4)
acpitimer0 at acpi0: 3579545 Hz, 24 bits
acpihpet0 at acpi0: 14318179 Hz
acpimadt0 at acpi0 addr 0xfee00000: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: Intel(R) Core(TM) i3-3227U CPU @ 1.90GHz, 1796.22 MHz, 06-3a-09
cpu0:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLINE,XSAVE,AVX,F16C,NXE,RDTSCP,LONG,LAHF,PERF,ITSC,FSGSBASE,SMEP,ERMS,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
cpu0: 256KB 64b/line 8-way L2 cache
cpu0: smt 0, core 0, package 0
mtrr: Pentium Pro MTRR support, 10 var ranges, 88 fixed ranges
cpu0: apic clock running at 99MHz
cpu0: mwait min=64, max=64, C-substates=0.2.1.1.2, IBE
cpu1 at mainbus0: apid 2 (application processor)
cpu1: Intel(R) Core(TM) i3-3227U CPU @ 1.90GHz, 1795.93 MHz, 06-3a-09
cpu1:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLINE,XSAVE,AVX,F16C,NXE,RDTSCP,LONG,LAHF,PERF,ITSC,FSGSBASE,SMEP,ERMS,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
cpu1: 256KB 64b/line 8-way L2 cache
cpu1: smt 0, core 1, package 0
ioapic0 at mainbus0: apid 2 pa 0xfec00000, version 20, 24 pins
acpimcfg0 at acpi0
acpimcfg0: addr 0xf8000000, bus 0-63
acpiprt0 at acpi0: bus 0 (PCI0)
acpiprt1 at acpi0: bus -1 (P0P1)
acpiprt2 at acpi0: bus 2 (RP01)
acpiprt3 at acpi0: bus 3 (RP02)
acpiprt4 at acpi0: bus 4 (RP03)
acpiprt5 at acpi0: bus -1 (RP04)
acpiprt6 at acpi0: bus -1 (RP05)
acpiprt7 at acpi0: bus 9 (RP06)
acpiprt8 at acpi0: bus -1 (RP07)
acpiprt9 at acpi0: bus -1 (RP08)
acpiprt10 at acpi0: bus -1 (PEG0)
acpiprt11 at acpi0: bus -1 (PEG1)
acpiprt12 at acpi0: bus -1 (PEG2)
acpiprt13 at acpi0: bus -1 (PEG3)
acpiec0 at acpi0
acpicpu0 at acpi0: C2(500@59 mwait.1@0x10), C1(1000@1 mwait.1), PSS
cpicpu1 at acpi0: C2(500@59 mwait.1@0x10), C1(1000@1 mwait.1), PSS
acpitz0 at acpi0: critical temperature is 99 degC
acpicmos0 at acpi0
acpithinkpad0 at acpi0
acpiac0 at acpi0: AC unit online
acpibat0 at acpi0: BAT1 model "45N1174" serial 494 type LION oem "SANYO"
acpibtn0 at acpi0: LID_
"PNP0C14" at acpi0 not configured
"PNP0C14" at acpi0 not configured
"PNP0C14" at acpi0 not configured
acpibtn1 at acpi0: PWRB
acpivideo0 at acpi0: GFX0
acpivout at acpivideo0 not configured
cpu0: Enhanced SpeedStep 1796 MHz: speeds: 1901, 1900, 1800, 1700,
1600, 1500, 1400, 1300, 1200, 1100, 1000, 900, 800, 779 MHz
pci0 at mainbus0 bus 0
pchb0 at pci0 dev 0 function 0 "Intel Core 3G Host" rev 0x09
inteldrm0 at pci0 dev 2 function 0 "Intel HD Graphics 4000" rev 0x09
drm0 at inteldrm0
inteldrm0: msi
inteldrm0: 1366x768, 32bpp
wsdisplay0 at inteldrm0 mux 1: console (std, vt100 emulation)
wsdisplay0: screen 1-5 added (std, vt100 emulation)
xhci0 at pci0 dev 20 function 0 "Intel 7 Series xHCI" rev 0x04: msi, xHCI 1.0
usb0 at xhci0: USB revision 3.0
uhub0 at usb0 configuration 1 interface 0 "Intel xHCI root hub" rev
3.00/1.00 addr 1
"Intel 7 Series MEI" rev 0x04 at pci0 dev 22 function 0 not configured
ehci0 at pci0 dev 26 function 0 "Intel 7 Series USB" rev 0x04: apic 2 int 16
usb1 at ehci0: USB revision 2.0
uhub1 at usb1 configuration 1 interface 0 "Intel EHCI root hub" rev
2.00/1.00 addr 1
azalia0 at pci0 dev 27 function 0 "Intel 7 Series HD Audio" rev 0x04: msi
azalia0: codecs: Realtek ALC269, Intel/0x2806, using Realtek ALC269
audio0 at azalia0
ppb0 at pci0 dev 28 function 0 "Intel 7 Series PCIE" rev 0xc4: msi
pci1 at ppb0 bus 2
ppb1 at pci0 dev 28 function 1 "Intel 7 Series PCIE" rev 0xc4: msi
pci2 at ppb1 bus 3
rtwn0 at pci2 dev 0 function 0 "Realtek 8188CE" rev 0x01: msi
rtwn0: MAC/BB RTL8188CE, RF 6052 1T1R, address 60:d8:19:c6:01:4b
ppb2 at pci0 dev 28 function 2 "Intel 7 Series PCIE" rev 0xc4: msi
pci3 at ppb2 bus 4
rtsx0 at pci3 dev 0 function 0 "Realtek RTS5209 Card Reader" rev 0x01: msi
sdmmc0 at rtsx0: 4-bit, dma
ppb3 at pci0 dev 28 function 5 "Intel 7 Series PCIE" rev 0xc4: msi
pci4 at ppb3 bus 9
re0 at pci4 dev 0 function 0 "Realtek 8168" rev 0x07:
RTL8168E/8111E-VL (0x2c80), msi, address c4:54:44:1f:16:10
rgephy0 at re0 phy 7: RTL8169S/8110S/8211 PHY, rev. 5
ehci1 at pci0 dev 29 function 0 "Intel 7 Series USB" rev 0x04: apic 2 int 23
usb2 at ehci1: USB revision 2.0
uhub2 at usb2 configuration 1 interface 0 "Intel EHCI root hub" rev
2.00/1.00 addr 1
pcib0 at pci0 dev 31 function 0 "Intel HM77 LPC" rev 0x04
ahci0 at pci0 dev 31 function 2 "Intel 7 Series AHCI" rev 0x04: msi, AHCI 1.3
ahci0: port 0: 6.0Gb/s
scsibus1 at ahci0: 32 targets
sd0 at scsibus1 targ 0 lun 0: <ATA, HGST HTS725050A7, GH2Z> SCSI3
0/direct fixed naa.5000cca76ce19389
sd0: 476940MB, 512 bytes/sector, 976773168 sectors
ichiic0 at pci0 dev 31 function 3 "Intel 7 Series SMBus" rev 0x04: apic 2 int 18
iic0 at ichiic0
spdmem0 at iic0 addr 0x50: 4GB DDR3 SDRAM PC3-10600 SO-DIMM
spdmem1 at iic0 addr 0x52: 4GB DDR3 SDRAM PC3-12800 SO-DIMM
isa0 at pcib0
isadma0 at isa0
pckbc0 at isa0 port 0x60/5 irq 1 irq 12
pckbd0 at pckbc0 (kbd slot)
wskbd0 at pckbd0: console keyboard, using wsdisplay0
pms0 at pckbc0 (aux slot)
wsmouse0 at pms0 mux 0
wsmouse1 at pms0 mux 0
pms0: Synaptics clickpad, firmware 8.1, 0x1e2b1 0x940300
pcppi0 at isa0 port 0x61
spkr0 at pcppi0
vmm0 at mainbus0: VMX/EPT
efifb at mainbus0 not configured
error: [drm:pid0:cpt_set_fifo_underrun_reporting] *ERROR* uncleared
pch fifo underrun on pch transcoder A
error: [drm:pid0:intel_pch_fifo_underrun_irq_handler] *ERROR* PCH
transcoder A FIFO underrun
uvideo0 at uhub0 port 4 configuration 1 interface 0 "Vimicro corp.
Integrated Camera" rev 2.00/0.10 addr 2
video0 at uvideo0
uhub3 at uhub1 port 1 configuration 1 interface 0 "Intel Rate Matching
Hub" rev 2.00/0.00 addr 2
uhub4 at uhub2 port 1 configuration 1 interface 0 "Intel Rate Matching
Hub" rev 2.00/0.00 addr 2
vscsi0 at root
scsibus2 at vscsi0: 256 targets
softraid0 at root
scsibus3 at softraid0: 256 targets
sd1 at scsibus3 targ 1 lun 0: <OPENBSD, SR CRYPTO, 006> SCSI2 0/direct fixed
sd1: 476939MB, 512 bytes/sector, 976771553 sectors
root on sd1a (e93c6174e549f1f1.a) swap on sd1b dump on sd1b
On Wed, Sep 5, 2018 at 11:22 PM Mike Larkin <[email protected]> wrote:
>
> On Wed, Sep 05, 2018 at 08:38:53PM +0200, Paul de Weerd wrote:
> > Hi all,
> >
> > Last year in May, i wrote to bugs@ about vmm VMs having slow time[1].
> > Mike Larkin replied that this was a known issue[2], so I mostly
> > ignored it, thinking things would improve over time. And indeed they
> > have: these days, the play VM I have on my home workstation runs time
> > at normal speed.
> >
> > So I set about building another VM on another machine. Here time
> > seems to run slower again, but I don't really understand why. Of
> > course, the hardware of the host machines differs; my workstation uses
> > an i7-4770, the server with the slow VM runs on a Xeon E31260L, full
> > dmesgs at the end of this mail.
> >
> > I noticed that the VM on the Xeon runs at 100% interrupt according to
> > top / systat, but then found out the VM on the i7 is (almost) the
> > same, this one runs at 80-90% interrupt.
> >
> > Then I noticed that the interrupt rate on the i7 guest is half that of
> > the Xeon guest:
> >
>
> Also:
>
> -proxy- ~> uptime
> 2:20PM up 229 days, 15:39, 1 user, load averages: 0.00, 0.00, 0.00
>
>
> that VM has been up for 229 days and the clock has drifted less than 3
> seconds over that time. Use tsc.
>
> -ml
>
