KVM clock synchronization

2010-12-28 Thread Zachary Amsden
A further set of improvements to KVM clock. Now it actually can stay synchronized on SMP systems with a stable TSC, does not get destabilized by host suspend, and is resistant to migration. I did look a bit into the second to last remaining migration issue; that is, we read the TSCs for all VCPUs

[KVM Clock Synchronization 1/4] Make cyc_to_nsec conversions more reliable

2010-12-28 Thread Zachary Amsden
Rather than use the host CPU TSC rate, which can change, compute cycle to nanosecond conversion in the guest TSC rate, which is fixed. This allows the math for write compensation detection to be made more reliable. Signed-off-by: Zachary Amsden --- arch/x86/kvm/x86.c | 58

[KVM Clock Synchronization 2/4] Keep TSC synchronized across host suspend

2010-12-28 Thread Zachary Amsden
During a host suspend, TSC may go backwards, which KVM interprets as an unstable TSC. Technically, KVM should not be marking the TSC unstable, which causes the TSC clocksource to go bad, but should be adjusting the TSC offsets in such a case. Signed-off-by: Zachary Amsden --- arch/x86/include

Re: [Qemu-devel] [RFC][PATCH 0/3] Let RTC follow backward jumps of host clock immediately

2010-12-23 Thread Zachary Amsden
On 12/17/2010 04:58 AM, Jan Kiszka wrote: By default, we base the mc146818 RTC on the host clock (CLOCK_REALTIME). This works fine if only the frequency of the host clock is tuned (e.g. by NTP) or if it is set to a future time. However, if the host is tuned backward, e.g. because NTP obtained the

Re: vread in kvm_clock

2010-12-20 Thread Zachary Amsden
On 12/19/2010 08:16 PM, Avi Kivity wrote: On 12/20/2010 03:15 AM, Zachary Amsden wrote: On 12/19/2010 05:27 AM, Avi Kivity wrote: On 12/17/2010 07:43 PM, Zachary Amsden wrote: On 12/15/2010 10:16 AM, Julien Desfossez wrote: Hi, I'm currently working with the kvm clocksource an

Re: vread in kvm_clock

2010-12-19 Thread Zachary Amsden
On 12/19/2010 05:27 AM, Avi Kivity wrote: On 12/17/2010 07:43 PM, Zachary Amsden wrote: On 12/15/2010 10:16 AM, Julien Desfossez wrote: Hi, I'm currently working with the kvm clocksource and I'm wondering if we could implement the vread function for this clock source when we are

Re: vread in kvm_clock

2010-12-17 Thread Zachary Amsden
On 12/15/2010 10:16 AM, Julien Desfossez wrote: Hi, I'm currently working with the kvm clocksource and I'm wondering if we could implement the vread function for this clock source when we are running on a host with constant_tsc. If I understand correctly the hv_clock structure is per_cpu becau

Re: linux-next: Tree for November 22 (kvm)

2010-11-30 Thread Zachary Amsden
On 11/30/2010 02:17 AM, Glauber Costa wrote: On Mon, 2010-11-29 at 08:08 -1000, Zachary Amsden wrote: On 11/29/2010 07:52 AM, Randy Dunlap wrote: On 11/29/10 09:47, Zachary Amsden wrote: On 11/29/2010 06:35 AM, Avi Kivity wrote: On 11/29/2010 06:33 PM, Randy

Re: linux-next: Tree for November 22 (kvm)

2010-11-29 Thread Zachary Amsden
On 11/29/2010 07:52 AM, Randy Dunlap wrote: On 11/29/10 09:47, Zachary Amsden wrote: On 11/29/2010 06:35 AM, Avi Kivity wrote: On 11/29/2010 06:33 PM, Randy Dunlap wrote: On Mon, 22 Nov 2010 13:26:27 -0800 Randy Dunlap wrote: On Mon, 22 Nov 2010 13:49:11 +1100

Re: linux-next: Tree for November 22 (kvm)

2010-11-29 Thread Zachary Amsden
On 11/29/2010 06:35 AM, Avi Kivity wrote: On 11/29/2010 06:33 PM, Randy Dunlap wrote: On Mon, 22 Nov 2010 13:26:27 -0800 Randy Dunlap wrote: > On Mon, 22 Nov 2010 13:49:11 +1100 Stephen Rothwell wrote: > > > Hi all, > > > > Changes since 20101119: > > > kvm.c:(.init.text+0x11f49): undefined

Re: [PATCH 1/2] KVM: take kvm_lock for hardware_disable() during cpu hotplug

2010-11-17 Thread Zachary Amsden
On 11/17/2010 04:41 PM, Takuya Yoshikawa wrote: (2010/11/18 11:33), Zachary Amsden wrote: On 11/17/2010 04:04 PM, Takuya Yoshikawa wrote: (2010/11/18 10:59), Zachary Amsden wrote: On 11/15/2010 10:35 PM, Takuya Yoshikawa wrote: In kvm_cpu_hotplug(), only CPU_STARTING case is protected by

Re: [PATCH 1/2] KVM: take kvm_lock for hardware_disable() during cpu hotplug

2010-11-17 Thread Zachary Amsden
On 11/17/2010 04:04 PM, Takuya Yoshikawa wrote: (2010/11/18 10:59), Zachary Amsden wrote: On 11/15/2010 10:35 PM, Takuya Yoshikawa wrote: In kvm_cpu_hotplug(), only CPU_STARTING case is protected by kvm_lock. This patch adds missing protection for CPU_DYING case. Signed-off-by: Takuya

Re: [PATCH 1/2] KVM: take kvm_lock for hardware_disable() during cpu hotplug

2010-11-17 Thread Zachary Amsden
On 11/15/2010 10:35 PM, Takuya Yoshikawa wrote: In kvm_cpu_hotplug(), only CPU_STARTING case is protected by kvm_lock. This patch adds missing protection for CPU_DYING case. Signed-off-by: Takuya Yoshikawa --- virt/kvm/kvm_main.c |2 ++ 1 files changed, 2 insertions(+), 0 deletions(-) di

Re: TSC host kernel message with kvm.git as host kernel and qemu-kvm.git as userspace

2010-10-21 Thread Zachary Amsden
On 10/21/2010 09:49 AM, Lucas Meneghel Rodrigues wrote: Hi folks, I was doing some work at one of our host machines, the one that runs upstream (Fedora 14, kvm.git as the kernel and qemu-kvm.git as userspace), and I noticed the kernel message: kvm: unreliable cycle conversion on adjustable rate

Re: [PATCH 25/27] nVMX: Additional TSC-offset handling

2010-10-19 Thread Zachary Amsden
On 10/17/2010 12:16 AM, Nadav Har'El wrote: In the unlikely case that L1 does not capture MSR_IA32_TSC, L0 needs to emulate this MSR write by L2 by modifying vmcs02.tsc_offset. We also need to set vmcs12.tsc_offset, for this change to survive the next nested entry (see prepare_vmcs02()). Signed-

Re: 2.6.35-rc1 regression with pvclock and smp guests

2010-10-11 Thread Zachary Amsden
On 10/08/2010 10:59 PM, Arjan Koers wrote: On 2010-10-09 08:29, Michael Tokarev wrote: ... The result is that no released linux kernel boots in smp in kvm, which is a linux virtual machine. That's irony, isn't it? I wonder how distributions (which are almost all based on 2.6.32 nowadays) wi

Re: 2.6.35-rc1 regression with pvclock and smp guests

2010-10-08 Thread Zachary Amsden
On 10/08/2010 03:10 PM, Arjan Koers wrote: On 2010-10-09 00:06, Marcelo Tosatti wrote: On Thu, Oct 07, 2010 at 04:47:11PM -1000, Zachary Amsden wrote: On 10/07/2010 02:12 PM, Arjan Koers wrote: On 2010-10-03 01:42, Zachary Amsden wrote: ... Umm... do you guys

Re: 2.6.35-rc1 regression with pvclock and smp guests

2010-10-08 Thread Zachary Amsden
On 10/08/2010 03:10 PM, Arjan Koers wrote: On 2010-10-09 00:06, Marcelo Tosatti wrote: On Thu, Oct 07, 2010 at 04:47:11PM -1000, Zachary Amsden wrote: On 10/07/2010 02:12 PM, Arjan Koers wrote: On 2010-10-03 01:42, Zachary Amsden wrote: ... Umm... do you guys

Re: 2.6.35-rc1 regression with pvclock and smp guests

2010-10-07 Thread Zachary Amsden
On 10/07/2010 02:12 PM, Arjan Koers wrote: On 2010-10-03 01:42, Zachary Amsden wrote: ... Umm... do you guys have this commit? This is supposed to address the issue where the guest keeps resetting the TSC. A guest which does that will break kvmclock. It only happens on SMP, and it&#

Re: last_kernel_ns ?

2010-10-04 Thread Zachary Amsden
On 10/04/2010 06:50 AM, Glauber Costa wrote: Zach, vcpu->hv_clock.tsc_timestamp = tsc_timestamp; vcpu->hv_clock.system_time = kernel_ns + v->kvm->arch.kvmclock_offset; vcpu->last_kernel_ns = kernel_ns;<= (1) vcpu->last_guest_tsc = tsc_timestamp; v

Re: 2.6.35-rc1 regression with pvclock and smp guests

2010-10-02 Thread Zachary Amsden
On 10/02/2010 06:10 AM, Arjan Koers wrote: On 2010-10-02 09:35, Michael Tokarev wrote: 02.10.2010 09:35, Zachary Amsden wrote: [] Can you try this patch to see if it helps? I believe it is also safe for Xen, but cc'ing to double check. It makes no visible difference.

Re: TSC in nested SVM and VMX

2010-10-02 Thread Zachary Amsden
On 10/02/2010 01:19 AM, Alexander Graf wrote: On 02.10.2010, at 03:56, Alexander Graf wrote: Am 01.10.2010 um 21:22 schrieb Zachary Amsden: On 10/01/2010 04:46 AM, Alexander Graf wrote: On 01.10.2010, at 13:21, Nadav Har'El wrote: On Thu, Sep 30, 2010, Za

Re: 2.6.35-rc1 regression with pvclock and smp guests

2010-10-02 Thread Zachary Amsden
On 10/01/2010 09:35 PM, Michael Tokarev wrote: 02.10.2010 09:35, Zachary Amsden wrote: [] Can you try this patch to see if it helps? I believe it is also safe for Xen, but cc'ing to double check. It makes no visible difference. For some reason one of my test guests - 2.6.35.6

Re: 2.6.35-rc1 regression with pvclock and smp guests

2010-10-01 Thread Zachary Amsden
On 09/30/2010 01:07 PM, Michael Tokarev wrote: 01.10.2010 03:02, Michael Tokarev wrote: 30.09.2010 23:05, Marcelo Tosatti wrote: [] Arjan, Michael, can you try the following: From 3823c018162dc708b543cbdc680a4c7d63533fee Mon Sep 17 00:00:00 2001 From: Zachary Amsden Date: Sat, 29

Re: TSC in nested SVM and VMX

2010-10-01 Thread Zachary Amsden
On 10/01/2010 04:46 AM, Alexander Graf wrote: On 01.10.2010, at 13:21, Nadav Har'El wrote: On Thu, Sep 30, 2010, Zachary Amsden wrote about "Re: TSC in nested SVM and VMX": 1) When reading an MSR, we are not emulating the L2 guest; we are DIRECTLY reading the

Re: TSC in nested SVM and VMX

2010-09-30 Thread Zachary Amsden
On 09/30/2010 12:50 PM, Nadav Har'El wrote: Hi, I noticed that the TSC handling code has recently changed, and since it wasn't done correctly in the nested VMX patch, I wanted to take the opportunity to fix it. I looked at what nested SVM does about TSC, and most of it I think I understand, but

Re: 2.6.35-rc1 regression with pvclock and smp guests

2010-09-30 Thread Zachary Amsden
On 09/30/2010 01:07 PM, Michael Tokarev wrote: 01.10.2010 03:02, Michael Tokarev wrote: 30.09.2010 23:05, Marcelo Tosatti wrote: [] Arjan, Michael, can you try the following: From 3823c018162dc708b543cbdc680a4c7d63533fee Mon Sep 17 00:00:00 2001 From: Zachary Amsden Date: Sat, 29

Re: 2.6.35-rc1 regression with pvclock and smp guests

2010-09-30 Thread Zachary Amsden
On 09/30/2010 05:12 AM, Michael Tokarev wrote: 30.09.2010 17:54, Zachary Amsden wrote: [] The printk movement is just a bandaid patch, correct? Anything which does printk before kvmclock is registered could trigger the same bug. Well, I'd not say it's just a bandaid patch,

Re: 2.6.35-rc1 regression with pvclock and smp guests

2010-09-30 Thread Zachary Amsden
On 09/29/2010 11:59 PM, Michael Tokarev wrote: 30.09.2010 11:55, Michael Tokarev wrote: 29.09.2010 23:26, Arjan Koers wrote: On 2010-09-29 11:19, Michael Tokarev wrote: 29.09.2010 13:17, Michael Tokarev wrote: [] Avi, this is definitely a -stable material, f

Re: [PATCH] fix kvmclock bug

2010-09-27 Thread Zachary Amsden
On 09/25/2010 11:54 PM, Jan Kiszka wrote: Am 24.09.2010 09:28, Jan Kiszka wrote: Am 19.09.2010 02:15, Zachary Amsden wrote: For CPUs with unstable TSC, we null time offset between not just VCPU switches, but all preemptions of the kvm thread. This makes a bug much more likely where

Re: [KVM timekeeping fixes 4/4] TSC catchup mode

2010-09-22 Thread Zachary Amsden
On 09/21/2010 08:18 AM, Marcelo Tosatti wrote: On Mon, Sep 20, 2010 at 03:11:30PM -1000, Zachary Amsden wrote: On 09/20/2010 05:38 AM, Marcelo Tosatti wrote: On Sat, Sep 18, 2010 at 02:38:15PM -1000, Zachary Amsden wrote: Negate the effects of AN TYM spell while kvm thread

Re: [KVM timekeeping fixes 4/4] TSC catchup mode

2010-09-20 Thread Zachary Amsden
On 09/20/2010 05:38 AM, Marcelo Tosatti wrote: On Sat, Sep 18, 2010 at 02:38:15PM -1000, Zachary Amsden wrote: Negate the effects of AN TYM spell while kvm thread is preempted by tracking conversion factor to the highest TSC rate and catching the TSC up when it has fallen behind the kernel

[KVM timekeeping fixes 4/4] TSC catchup mode

2010-09-18 Thread Zachary Amsden
is is possible, which only does catchup when TSC rate drops, and which specifically targets only CPUs with broken TSC, but since these all are considered unstable_tsc(), this patch covers all necessary cases. Signed-off-by: Zachary Amsden --- arch/x86/include/asm/kvm_host.h |6 +++ arch/x86/kvm/

[KVM timekeeping fixes 3/4] Rename timer function

2010-09-18 Thread Zachary Amsden
This just changes some names to better reflect the usage they will be given. Separated out to keep confusion to a minimum. Signed-off-by: Zachary Amsden --- arch/x86/kvm/x86.c | 12 ++-- include/linux/kvm_host.h |2 +- 2 files changed, 7 insertions(+), 7 deletions(-) diff

[KVM timekeeping fixes 1/4] Fix kvmclock bug

2010-09-18 Thread Zachary Amsden
last_tsc to the newly read tsc value so that any computed nsec advance of kvmclock is nulled. Signed-off-by: Zachary Amsden --- arch/x86/kvm/x86.c |1 + 1 files changed, 1 insertions(+), 0 deletions(-) diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index a51635e..0b021e1 100644 --- a

[KVM timekeeping fixes 2/4] Make math work for other scales

2010-09-18 Thread Zachary Amsden
ake care to avoid an arithmetic overflow when scaling up the tps32 value (this could not happen with the fixed scaled value of NSEC_PER_SEC, but can happen with scaled rates above 2^31. Signed-off-by: Zachary Amsden --- arch/x86/kvm/x86.c | 30 ++ 1 files changed,

[PATCH] fix kvmclock bug

2010-09-18 Thread Zachary Amsden
now, this patch is sufficient to get things working again for me. commit 1abe7e8806fd71ea802c6622ed3ce7821a18f271 Author: Zachary Amsden Date: Sat Sep 18 13:58:37 2010 -1000 Fix kvmclock bug If preempted after kvmclock values are updated, but before hardware virtualization is

Re: [KVM timekeeping 10/35] Fix deep C-state TSC desynchronization

2010-09-18 Thread Zachary Amsden
On 09/17/2010 12:31 PM, Zachary Amsden wrote: On 09/17/2010 12:09 PM, Zachary Amsden wrote: On 09/15/2010 08:27 AM, Jan Kiszka wrote: Am 15.09.2010 14:32, Glauber Costa wrote: On Wed, Sep 15, 2010 at 10:09:33AM +0200, Jan Kiszka wrote: In any case, I'll proceed with the forcing of uns

Re: [KVM timekeeping 10/35] Fix deep C-state TSC desynchronization

2010-09-17 Thread Zachary Amsden
On 09/17/2010 12:09 PM, Zachary Amsden wrote: On 09/15/2010 08:27 AM, Jan Kiszka wrote: Am 15.09.2010 14:32, Glauber Costa wrote: On Wed, Sep 15, 2010 at 10:09:33AM +0200, Jan Kiszka wrote: In any case, I'll proceed with the forcing of unstable TSC and HPET clocksource and see what ha

Re: [KVM timekeeping 10/35] Fix deep C-state TSC desynchronization

2010-09-17 Thread Zachary Amsden
On 09/15/2010 08:27 AM, Jan Kiszka wrote: Am 15.09.2010 14:32, Glauber Costa wrote: On Wed, Sep 15, 2010 at 10:09:33AM +0200, Jan Kiszka wrote: In any case, I'll proceed with the forcing of unstable TSC and HPET clocksource and see what happens. I tried that before, but it

[PATCH] kvm-kmod

2010-09-17 Thread Zachary Amsden
Working on an older Fedora, I hit the need for this.. Add missing MSR definition Signed-off-by: Zachary Amsden diff --git a/x86/external-module-compat.h b/x86/external-module-compat.h index cc51b0f..8cb936d 100644 --- a/x86/external-module-compat.h +++ b/x86/external-module-compat.h @@ -1091,3

Re: [KVM timekeeping 10/35] Fix deep C-state TSC desynchronization

2010-09-14 Thread Zachary Amsden
On 09/13/2010 11:10 PM, Jan Kiszka wrote: Am 20.08.2010 10:07, Zachary Amsden wrote: When CPUs with unstable TSCs enter deep C-state, TSC may stop running. This causes us to require resynchronization. Since we can't tell when this may potentially happen, we assume the worst by forci

Re: [KVM timekeeping 10/35] Fix deep C-state TSC desynchronization

2010-09-14 Thread Zachary Amsden
On 09/14/2010 12:26 PM, Jan Kiszka wrote: Am 14.09.2010 21:32, Zachary Amsden wrote: On 09/14/2010 12:40 AM, Jan Kiszka wrote: Am 14.09.2010 11:27, Avi Kivity wrote: On 09/14/2010 11:10 AM, Jan Kiszka wrote: Am 20.08.2010 10:07, Zachary Amsden wrote

Re: [KVM timekeeping 10/35] Fix deep C-state TSC desynchronization

2010-09-14 Thread Zachary Amsden
On 09/14/2010 12:40 AM, Jan Kiszka wrote: Am 14.09.2010 11:27, Avi Kivity wrote: On 09/14/2010 11:10 AM, Jan Kiszka wrote: Am 20.08.2010 10:07, Zachary Amsden wrote: When CPUs with unstable TSCs enter deep C-state, TSC may stop running. This causes us to require

Re: [KVM timekeeping 26/35] Catchup slower TSC to guest rate

2010-09-07 Thread Zachary Amsden
On 09/06/2010 05:44 PM, Dong, Eddie wrote: > Zachary: > Will you extend the logic to cover the situation when the guest runs at > higher than the guest rate but the PCPU is over committed. In that case, > likely we can use the time spent when the VCPU is scheduled out to catch up > as well

Re: [KVM timekeeping] Revert getnsboottime() kernel API

2010-09-03 Thread Zachary Amsden
wo commits: 2616ac0be4b12bf31cfe8880a5b21c5a2da7d150 b8a578a03d18fab78ea1f79886e32c90f1d071a1 Zach On 08/31/2010 07:57 PM, Zachary Amsden wrote: Turns out this doesn't actually save any math or locking, name is chosen rather poorly, it doesn't match the existing kernel APIs, and requires kvm-k

Re: [RFC 1/8] Implement getnsboottime kernel API

2010-09-01 Thread Zachary Amsden
On 08/30/2010 06:06 AM, Glauber Costa wrote: From: Zachary Amsden Add a kernel call to get the number of nanoseconds since boot. This is generally useful enough to make it a generic call. Signed-off-by: Zachary Amsden --- include/linux/time.h |1 + kernel/time/timekeeping.c | 27

[KVM timekeeping] Revert getnsboottime() kernel API

2010-08-31 Thread Zachary Amsden
Turns out this doesn't actually save any math or locking, name is chosen rather poorly, it doesn't match the existing kernel APIs, and requires kvm-kmod changes. Signed-off-by: Zachary Amsden --- arch/x86/kvm/x86.c| 18 ++ include/linux/time.h |1

Re: [PATCH] KVM: pit: Do not check pending pit timer in vcpu thread

2010-08-30 Thread Zachary Amsden
On 08/26/2010 11:15 PM, Jason Wang wrote: Pit interrupt injection was done by workqueue, so no need to check pending pit timer in vcpu thread which could lead unnecessary unblocking of vcpu. Is this actually correct? There were a bunch of workarounds and fixes put into this code over time

Re: [PATCH kvm-unit-test 6/6] Add a test for kvm-clock

2010-08-27 Thread Zachary Amsden
On 08/26/2010 07:49 PM, Jason Wang wrote: This patch implements two tests for kvmclock. First one check whether the date of time returned by kvmclock matches the value got from host. Second one check whether the cycle of kvmclock grows monotonically in smp guest. Technically, it's not monot

Re: [KVM timekeeping 17/35] Implement getnsboottime kernel API

2010-08-27 Thread Zachary Amsden
On 08/27/2010 08:05 AM, Jan Kiszka wrote: Zachary Amsden wrote: Add a kernel call to get the number of nanoseconds since boot. This is generally useful enough to make it a generic call. Signed-off-by: Zachary Amsden --- include/linux/time.h |1 + kernel/time/timekeeping.c

Re: [KVM timekeeping 13/35] Perform hardware_enable in CPU_STARTING callback

2010-08-27 Thread Zachary Amsden
On 08/27/2010 06:32 AM, Jan Kiszka wrote: Zachary Amsden wrote: The CPU_STARTING callback was added upstream with the intention of being used for KVM, specifically for the hardware enablement that must be done before we can run in hardware virt. It had bugs on the x86_64 architecture at

Re: [RFC 3/7] measure time out of guest

2010-08-26 Thread Zachary Amsden
On 08/25/2010 11:43 AM, Glauber Costa wrote: By measuring time between a vcpu_put and a vcpu_load, we can estimate how much time did the guest stay out of the cpu. This is exported to the guest at every clock update. Signed-off-by: Glauber Costa --- arch/x86/include/asm/kvm_host.h |2 ++

Re: [RFC 2/7] change headers preparing for steal time

2010-08-26 Thread Zachary Amsden
On 08/25/2010 11:43 AM, Glauber Costa wrote: This guest/host common patch prepares infrastructure for the steal time implementation. Some constants are added, and a name change happens in pvclock vcpu structure. Signed-off-by: Glauber Costa --- arch/x86/include/asm/kvm_para.h|1 + arc

Re: [KVM timekeeping 25/35] Add clock catchup mode

2010-08-25 Thread Zachary Amsden
On 08/25/2010 12:01 PM, Marcelo Tosatti wrote: On Wed, Aug 25, 2010 at 10:48:20AM -1000, Zachary Amsden wrote: On 08/25/2010 07:27 AM, Marcelo Tosatti wrote: On Thu, Aug 19, 2010 at 10:07:39PM -1000, Zachary Amsden wrote: Make the clock update handler handle generic clock

Re: [KVM timekeeping 25/35] Add clock catchup mode

2010-08-25 Thread Zachary Amsden
On 08/25/2010 07:27 AM, Marcelo Tosatti wrote: On Thu, Aug 19, 2010 at 10:07:39PM -1000, Zachary Amsden wrote: Make the clock update handler handle generic clock synchronization, not just KVM clock. We add a catchup mode which keeps passthrough TSC in line with absolute guest TSC. Signed

Re: KVM timekeeping and TSC virtualization

2010-08-25 Thread Zachary Amsden
On 08/25/2010 06:55 AM, Marcelo Tosatti wrote: On Tue, Aug 24, 2010 at 01:01:38PM -1000, Zachary Amsden wrote: With this patchset, KVM now has a much stronger guarantee: If you have old guest software running on broken hardware, using SMP virtual machines, you do not get hardware

Re: KVM timekeeping and TSC virtualization

2010-08-24 Thread Zachary Amsden
On 08/24/2010 12:13 PM, Marcelo Tosatti wrote: On Thu, Aug 19, 2010 at 10:07:14PM -1000, Zachary Amsden wrote: This patch set implements full TSC virtualization, with both trapping and passthrough modes, and intelligent mode switching. As a result, TSC will never go backwards, we are stable

Re: KVM timekeeping and TSC virtualization

2010-08-24 Thread Zachary Amsden
On 08/24/2010 03:32 AM, David S. Ahern wrote: On 08/23/10 23:47, Zachary Amsden wrote: I've heard the rumor that TSC is orders of magnitude faster under VMware than under KVM from three people now, and I thought you were part of that camp. Needless to say, they are either laug

Re: KVM timekeeping and TSC virtualization

2010-08-23 Thread Zachary Amsden
On 08/23/2010 05:04 PM, David S. Ahern wrote: On 08/23/10 19:44, Zachary Amsden wrote: I have also looked at time keeping and performance of getimeofday on a certain proprietary hypervisor. KVM lags severely here and workloads dependent on timestamps are dramatically impacted. Evaluations

Re: KVM timekeeping and TSC virtualization

2010-08-23 Thread Zachary Amsden
On 08/21/2010 03:32 PM, David S. Ahern wrote: On 08/20/10 17:24, Zachary Amsden wrote: On 08/20/2010 03:26 AM, David S. Ahern wrote: On 08/20/10 02:07, Zachary Amsden wrote: This patch set implements full TSC virtualization, with both trapping and passthrough modes, and

Re: [KVM timekeeping 33/35] Indicate reliable TSC in kvmclock

2010-08-23 Thread Zachary Amsden
On 08/20/2010 07:45 AM, Glauber Costa wrote: On Thu, Aug 19, 2010 at 10:07:47PM -1000, Zachary Amsden wrote: When no platform bugs have been detected, no TSC warps have been detected, and the hardware guarantees to us TSC does not change rate or stop with P-state or C-state changes, we can

Re: [KVM timekeeping 12/35] Robust TSC compensation

2010-08-23 Thread Zachary Amsden
On 08/20/2010 07:40 AM, Glauber Costa wrote: On Thu, Aug 19, 2010 at 10:07:26PM -1000, Zachary Amsden wrote: Make the match of TSC find TSC writes that are close to each other instead of perfectly identical; this allows the compensator to also work in migration / suspend scenarios. Signed

Re: [KVM timekeeping 11/35] Add helper functions for time computation

2010-08-23 Thread Zachary Amsden
On 08/20/2010 07:34 AM, Glauber Costa wrote: On Thu, Aug 19, 2010 at 10:07:25PM -1000, Zachary Amsden wrote: Add a helper function to compute the kernel time and convert nanoseconds back to CPU specific cycles. Note that these must not be called in preemptible context, as that would mean

Re: [KVM timekeeping 08/35] Warn about unstable TSC

2010-08-23 Thread Zachary Amsden
On 08/20/2010 07:28 AM, Glauber Costa wrote: On Thu, Aug 19, 2010 at 10:07:22PM -1000, Zachary Amsden wrote: If creating an SMP guest with unstable host TSC, issue a warning Signed-off-by: Zachary Amsden Ok, I am not sure I agree 100 % this is needed. I believe we should try to

Re: [KVM timekeeping 05/35] Move TSC reset out of vmcb_init

2010-08-23 Thread Zachary Amsden
On 08/20/2010 07:08 AM, Glauber Costa wrote: On Thu, Aug 19, 2010 at 10:07:19PM -1000, Zachary Amsden wrote: The VMCB is reset whenever we receive a startup IPI, so Linux is setting TSC back to zero happens very late in the boot process and destabilizing the TSC. Instead, just set TSC to

Re: [KVM timekeeping 03/35] Move TSC offset writes to common code

2010-08-23 Thread Zachary Amsden
On 08/20/2010 07:06 AM, Glauber Costa wrote: On Thu, Aug 19, 2010 at 10:07:17PM -1000, Zachary Amsden wrote: Also, ensure that the storing of the offset and the reading of the TSC are never preempted by taking a spinlock. While the lock is overkill now, it is useful later in this patch

Re: [KVM timekeeping 17/35] Implement getnsboottime kernel API

2010-08-20 Thread Zachary Amsden
On 08/20/2010 03:04 PM, john stultz wrote: On Fri, 2010-08-20 at 14:52 -1000, Zachary Amsden wrote: I think gettimefromboot_ns() is a good descriptive name, but slightly too long - it would ruin my indentation. Perhaps getrealtime_ns()? Sigh... So getrealtime_ns would probably be

Re: [KVM timekeeping 17/35] Implement getnsboottime kernel API

2010-08-20 Thread Zachary Amsden
On 08/20/2010 02:02 PM, john stultz wrote: On Fri, 2010-08-20 at 13:37 -1000, Zachary Amsden wrote: On 08/20/2010 08:39 AM, john stultz wrote: On Thu, 2010-08-19 at 22:07 -1000, Zachary Amsden wrote: Add a kernel call to get the number of nanoseconds since boot. This is

Re: [KVM timekeeping 17/35] Implement getnsboottime kernel API

2010-08-20 Thread Zachary Amsden
On 08/20/2010 08:39 AM, john stultz wrote: On Thu, 2010-08-19 at 22:07 -1000, Zachary Amsden wrote: Add a kernel call to get the number of nanoseconds since boot. This is generally useful enough to make it a generic call. Few comments here. Signed-off-by: Zachary Amsden

Re: KVM timekeeping and TSC virtualization

2010-08-20 Thread Zachary Amsden
On 08/20/2010 03:26 AM, David S. Ahern wrote: On 08/20/10 02:07, Zachary Amsden wrote: This patch set implements full TSC virtualization, with both trapping and passthrough modes, and intelligent mode switching. As a result, TSC will never go backwards, we are stable against guest re

[KVM timekeeping 07/35] Make cpu_tsc_khz updates use local CPU

2010-08-20 Thread Zachary Amsden
will issue IPIs to the local CPU to perform this very same task). Signed-off-by: Zachary Amsden --- arch/x86/kvm/x86.c | 157 +-- 1 files changed, 114 insertions(+), 43 deletions(-) diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 62c58f9

[KVM timekeeping 03/35] Move TSC offset writes to common code

2010-08-20 Thread Zachary Amsden
Also, ensure that the storing of the offset and the reading of the TSC are never preempted by taking a spinlock. While the lock is overkill now, it is useful later in this patch series. Signed-off-by: Zachary Amsden --- arch/x86/include/asm/kvm_host.h |3 +++ arch/x86/kvm/svm.c

[KVM timekeeping 04/35] Fix SVM VMCB reset

2010-08-20 Thread Zachary Amsden
On reset, VMCB TSC should be set to zero. Instead, code was setting tsc_offset to zero, which passes through the underlying TSC. Signed-off-by: Zachary Amsden --- arch/x86/kvm/svm.c |2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/arch/x86/kvm/svm.c b/arch/x86/kvm

[KVM timekeeping 10/35] Fix deep C-state TSC desynchronization

2010-08-20 Thread Zachary Amsden
When CPUs with unstable TSCs enter deep C-state, TSC may stop running. This causes us to require resynchronization. Since we can't tell when this may potentially happen, we assume the worst by forcing re-compensation for it at every point the VCPU task is descheduled. Signed-off-by: Za

[KVM timekeeping 08/35] Warn about unstable TSC

2010-08-20 Thread Zachary Amsden
If creating an SMP guest with unstable host TSC, issue a warning Signed-off-by: Zachary Amsden --- arch/x86/kvm/x86.c |4 1 files changed, 4 insertions(+), 0 deletions(-) diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 3420f25..5e3b10e 100644 --- a/arch/x86/kvm/x86.c +++ b

[KVM timekeeping 15/35] Move scale_delta into common header

2010-08-20 Thread Zachary Amsden
The scale_delta function for shift / multiply with 31-bit precision moves to a common header so it can be used by both kernel and kvm module. Signed-off-by: Zachary Amsden --- arch/x86/include/asm/pvclock.h | 38 ++ arch/x86/kernel/pvclock.c |3

[KVM timekeeping 14/35] Add clock sync request to hardware enable

2010-08-20 Thread Zachary Amsden
worry about it is resume, as resuming a frozen CPU, the spinlock won't be taken. Signed-off-by: Zachary Amsden --- arch/x86/kvm/x86.c |8 virt/kvm/kvm_main.c |6 +- 2 files changed, 13 insertions(+), 1 deletions(-) diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x

[KVM timekeeping 17/35] Implement getnsboottime kernel API

2010-08-20 Thread Zachary Amsden
Add a kernel call to get the number of nanoseconds since boot. This is generally useful enough to make it a generic call. Signed-off-by: Zachary Amsden --- include/linux/time.h |1 + kernel/time/timekeeping.c | 27 +++ 2 files changed, 28 insertions(+), 0

[KVM timekeeping 25/35] Add clock catchup mode

2010-08-20 Thread Zachary Amsden
Make the clock update handler handle generic clock synchronization, not just KVM clock. We add a catchup mode which keeps passthrough TSC in line with absolute guest TSC. Signed-off-by: Zachary Amsden --- arch/x86/include/asm/kvm_host.h |1 + arch/x86/kvm/x86.c | 55

[KVM timekeeping 24/35] Timer request function renaming

2010-08-20 Thread Zachary Amsden
Separate step to prepare for next change. Signed-off-by: Zachary Amsden --- arch/x86/kvm/x86.c | 12 ++-- include/linux/kvm_host.h |2 +- 2 files changed, 7 insertions(+), 7 deletions(-) diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index d715ec0..ac0b2d9 100644

[KVM timekeeping 21/35] Track max tsc_khz

2010-08-20 Thread Zachary Amsden
Signed-off-by: Zachary Amsden --- arch/x86/kvm/x86.c | 10 ++ 1 files changed, 10 insertions(+), 0 deletions(-) diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 718bed9..517daf3 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -898,6 +898,7 @@ static void

[KVM timekeeping 20/35] Make math work for other scales

2010-08-20 Thread Zachary Amsden
ake care to avoid an arithmetic overflow when scaling up the tps32 value (this could not happen with the fixed scaled value of NSEC_PER_SEC, but can happen with scaled rates above 2^31. Signed-off-by: Zachary Amsden --- arch/x86/kvm/x86.c | 30 ++ 1 files changed,

[KVM timekeeping 26/35] Catchup slower TSC to guest rate

2010-08-20 Thread Zachary Amsden
Use the catchup code to continue adjusting the TSC when running at lower than the guest rate Signed-off-by: Zachary Amsden --- arch/x86/kvm/x86.c |9 - 1 files changed, 8 insertions(+), 1 deletions(-) diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index a4215d7..086d56a

[KVM timekeeping 06/35] TSC reset compensation

2010-08-20 Thread Zachary Amsden
Attempt to synchronize TSCs which are reset to the same value. In the case of a reliable hardware TSC, we can just re-use the same offset, but on non-reliable hardware, we can get closer by adjusting the offset to match the elapsed time. Signed-off-by: Zachary Amsden --- arch/x86/include/asm

[KVM timekeeping 12/35] Robust TSC compensation

2010-08-20 Thread Zachary Amsden
Make the match of TSC find TSC writes that are close to each other instead of perfectly identical; this allows the compensator to also work in migration / suspend scenarios. Signed-off-by: Zachary Amsden --- arch/x86/kvm/x86.c | 14 ++ 1 files changed, 10 insertions(+), 4

[KVM timekeeping 11/35] Add helper functions for time computation

2010-08-20 Thread Zachary Amsden
KVM_SET_CLOCK / KVM_GET_CLOCK ioctls to use the kernel time helper, these should be bootbased as well. Signed-off-by: Zachary Amsden --- arch/x86/kvm/x86.c | 48 1 files changed, 28 insertions(+), 20 deletions(-) diff --git a/arch/x86/kvm/x86.c b/arch/x86

[KVM timekeeping 01/35] Drop vm_init_tsc

2010-08-20 Thread Zachary Amsden
This is used only by the VMX code, and is not done properly; if the TSC is indeed backwards, it is out of sync, and will need proper handling in the logic at each and every CPU change. For now, drop this test during init as misguided. Signed-off-by: Zachary Amsden --- arch/x86/include/asm

[KVM timekeeping 05/35] Move TSC reset out of vmcb_init

2010-08-20 Thread Zachary Amsden
The VMCB is reset whenever we receive a startup IPI, so Linux is setting TSC back to zero happens very late in the boot process and destabilizing the TSC. Instead, just set TSC to zero once at VCPU creation time. Why the separate patch? So git-bisect is your friend. Signed-off-by: Zachary

[KVM timekeeping 27/35] Add TSC trapping

2010-08-20 Thread Zachary Amsden
rtual machines, and we don't trap the TSC if the system TSC is "stable". Signed-off-by: Zachary Amsden --- arch/x86/include/asm/kvm_host.h |2 + arch/x86/kvm/svm.c | 22 +++ arch/x86/kvm/vmx.c | 21 ++ arch/x86/kvm/x86.c

[KVM timekeeping 29/35] TSC overrun protection

2010-08-20 Thread Zachary Amsden
be loaded after KVM has already started, in which case our estimated CPU frequency may be shown to be wrong. These problems will be dealt with separately for clarity. Signed-off-by: Zachary Amsden --- arch/x86/include/asm/kvm_host.h |1 + arch/x86/kvm/x86.c

[KVM timekeeping 28/35] Unstable TSC write compensation

2010-08-20 Thread Zachary Amsden
between the guest TSC and system time. With this change, guests no longer exhibit pathological behavior during guest initiatied TSC recalibration. Signed-off-by: Zachary Amsden --- arch/x86/kvm/x86.c | 51 +-- 1 files changed, 21 insertions(+), 30

[KVM timekeeping 35/35] Add some debug stuff

2010-08-20 Thread Zachary Amsden
Very useful, debug-only output Signed-off-by: Zachary Amsden --- arch/x86/kvm/x86.c | 33 +++-- 1 files changed, 31 insertions(+), 2 deletions(-) diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 23d5138..c74a087 100644 --- a/arch/x86/kvm/x86.c +++ b/arch

[KVM timekeeping 32/35] Entry conditions for TSC trapping

2010-08-20 Thread Zachary Amsden
it works. Signed-off-by: Zachary Amsden --- arch/x86/include/asm/kvm_host.h |2 + arch/x86/kvm/x86.c | 58 +++--- 2 files changed, 55 insertions(+), 5 deletions(-) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host

[KVM timekeeping 31/35] Exit conditions for TSC trapping

2010-08-20 Thread Zachary Amsden
preemption point when we are trapping, which we now know definitively because the transition points are all in one place. Signed-off-by: Zachary Amsden --- arch/x86/include/asm/kvm_host.h |4 ++ arch/x86/kvm/x86.c | 93 -- 2 files changed, 72

[KVM timekeeping 33/35] Indicate reliable TSC in kvmclock

2010-08-20 Thread Zachary Amsden
When no platform bugs have been detected, no TSC warps have been detected, and the hardware guarantees to us TSC does not change rate or stop with P-state or C-state changes, we can consider it reliable. Signed-off-by: Zachary Amsden --- arch/x86/kvm/x86.c | 10 +- 1 files changed, 9

[KVM timekeeping 34/35] Remove dead code

2010-08-20 Thread Zachary Amsden
This function is basically almost completely totally useless. N.B. The hardware_enable code is not redundant; we could have an old VCPU thread hanging around that was not rescheduled during a CPU hotremove / hotadd event. Signed-off-by: Zachary Amsden --- arch/x86/kvm/x86.c | 15

[KVM timekeeping 23/35] Set initial TSC rate conversion factors

2010-08-20 Thread Zachary Amsden
Signed-off-by: Zachary Amsden --- arch/x86/include/asm/kvm_host.h |3 +++ arch/x86/kvm/x86.c | 12 2 files changed, 15 insertions(+), 0 deletions(-) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index 638cae5..3a54cc1 100644 --- a

[KVM timekeeping 30/35] IOCTL for setting TSC rate

2010-08-20 Thread Zachary Amsden
Add an IOCTL for setting the TSC rate for a VM, intended to be used to migrate non-kvmclock based VMs which rely on TSC rate staying stable across host migration. Signed-off-by: Zachary Amsden --- arch/x86/kvm/x86.c | 36 include/linux/kvm.h |4

[KVM timekeeping 19/35] Add timekeeping documentation

2010-08-20 Thread Zachary Amsden
Basic informational document about x86 timekeeping and how KVM is affected. Signed-off-by: Zachary Amsden --- Documentation/kvm/timekeeping.txt | 613 + 1 files changed, 613 insertions(+), 0 deletions(-) create mode 100644 Documentation/kvm/timekeeping.txt

<    1   2   3   4   5   6   7   8   9   10   >