On Fri, Jan 26, 2024 at 04:16:57PM +0800, Xiaoyao Li wrote: > On 1/25/2024 6:05 AM, Marcelo Tosatti wrote: > > On Wed, Jan 24, 2024 at 10:52:46PM +0800, Xiaoyao Li wrote: > > > On 1/23/2024 11:39 PM, Marcelo Tosatti wrote: > > > > On Sat, Jan 20, 2024 at 05:44:07PM +0800, Xiaoyao Li wrote: > > > > > On 1/20/2024 12:14 AM, Marcelo Tosatti wrote: > > > > > > On Fri, Jan 19, 2024 at 02:46:22PM +0800, Xiaoyao Li wrote: > > > > > > > I'm wondering why CPUID_APM_INVTSC is set as unmigratable_flags. > > > > > > > Could > > > > > > > anyone explain it? > > > > > > > > > > > > > > > > > > commit 68bfd0ad4a1dcc4c328d5db85dc746b49c1ec07e > > > > > > Author: Marcelo Tosatti <mtosa...@redhat.com> > > > > > > Date: Wed May 14 16:30:09 2014 -0300 > > > > > > > > > > > > target-i386: block migration and savevm if invariant tsc is > > > > > > exposed > > > > > > Invariant TSC documentation mentions that "invariant TSC > > > > > > will run at a > > > > > > constant rate in all ACPI P-, C-. and T-states". > > > > > > This is not the case if migration to a host with different > > > > > > TSC frequency > > > > > > is allowed, or if savevm is performed. So block > > > > > > migration/savevm. > > > > > > > > > > > > So the rationale here was that without ensuring the destination host > > > > > > has the same TSC clock frequency, we can't migrate. > > > > > > > > > > It seems to me the concept of invtsc was extended to "tsc freq will > > > > > not > > > > > change even after the machine is live migrated". I'm not sure it is > > > > > correct > > > > > to extend the concept of invtsc. > > > > > > > > > > The main reason of introducing invtsc is to tell the tsc hardware > > > > > keeps > > > > > counting (at the same rate) even at deep C state, so long as other > > > > > states. > > > > > > > > > > For example, a guest is created on machine A with X GHz tsc, and > > > > > invtsc > > > > > exposed (machine A can ensure the guest's tsc counts at X GHz at any > > > > > state). > > > > > If the guest is migrated to machine B with Y GHz tsc, and machine B > > > > > can also > > > > > ensure the invtsc of its guest, i.e., the guest's tsc counts at Y GHz > > > > > at any > > > > > state. IMHO, in this case, the invtsc is supported at both src and > > > > > dest, > > > > > which means it is a migratable feature. However, the migration itself > > > > > fails, > > > > > due to mismatched/different configuration of tsc freq, not due to > > > > > invtsc. > > > > > > > > > > > However, this was later extended to allow invtsc migratioon when > > > > > > setting > > > > > > tsc-khz explicitly: > > > > > > > > > > > > commit d99569d9d8562c480e0befab601756b0b7b5d0e0 > > > > > > Author: Eduardo Habkost <ehabk...@redhat.com> > > > > > > Date: Sun Jan 8 15:32:34 2017 -0200 > > > > > > > > > > > > kvm: Allow invtsc migration if tsc-khz is set explicitly > > > > > > We can safely allow a VM to be migrated with invtsc enabled > > > > > > if > > > > > > tsc-khz is set explicitly, because: > > > > > > * QEMU already refuses to start if it can't set the TSC > > > > > > frequency > > > > > > to the configured value. > > > > > > * Management software is already required to keep device > > > > > > configuration (including CPU configuration) the same on > > > > > > migration source and destination. > > > > > > Signed-off-by: Eduardo Habkost <ehabk...@redhat.com> > > > > > > Message-Id: <20170108173234.25721-3-ehabk...@redhat.com> > > > > > > Signed-off-by: Eduardo Habkost <ehabk...@redhat.com> > > > > > > > > > > But in the case that user doesn't set tsc freq explicitly, the live > > > > > migration is likely to fail or have issues even without invtsc > > > > > exposed to > > > > > guest, > > > > > > > > Depends on how the guest is using the TSC, but yes. > > > > > > > > > if the destination host has a different tsc frequency than src host. > > > > > > > > > > So why bother checking invtsc only? > > > > > > > > Well, if invtsc is exposed to the guest, then it might use the TSC for > > > > timekeeping purposes. > > > > > > > > Therefore you don't want to fail (on the invtsc clock characteristics) > > > > otherwise timekeeping in the guest might be problematic. > > > > > > > > But this are all just heuristics. > > > > > > > > Do you have a suggestion for different behaviour? > > > > > > I think we need to block live migration when user doesn't specify a > > > certain > > > tsc frequency explicitly, regardless of invtsc. > > > > Problem is if that guest is using kvmclock for timekeeping, then live > > migration > > is safe (kvmclock logic will update the tsc frequency of the destination > > host upon migration). > > > > It surprises me kvmclock can do it. Cloud you please elaborate it a bit how > kvmclock achieve it during migration or point me to some codes in Linux?
MSR_KVM_SYSTEM_TIME_NEW description at Documentation/virt/kvm/x86/msr.rst and arch/x86/kernel/pvclock.c, __pvclock_clocksource_read function.
