On Tue, Oct 27, 2015 at 03:16:15PM -0500, Aravind Gopalakrishnan wrote:
> On 10/22/2015 10:44 AM, Boris Ostrovsky wrote:
> >On 10/22/2015 10:17 AM, Jan Beulich wrote:
> >>>>>On 28.09.15 at 09:13, <haozhong.zh...@intel.com> wrote:
> >>>The existing hvm_set_guest_tsc_fixed() and hvm_get_guest_tsc_fixed()
> >>>calculate the guest TSC by adding the TSC offset to the host TSC. When
> >>>the TSC scaling is enabled, the host TSC should be scaled first. This
> >>>patch adds the scaling logic to those two functions.
> >>Just like mentioned for the first twp patches - I'd first of all like to
> >>understand why the lack of scaling this wasn't an issue for SVM so
> >>far. What you reads plausible, but assuming that SVM TSC scaling
> >>code was tested, I'm hesitant to apply changes to it without
> >>understanding the details (or at least without SVM maintainers'
> >>consent).
> >
> >I don't see that this series will create any regressions in SVM . Most of
> >the changes move SVM-specific code into HVM I didn't see any obvious
> >problems there. I do have concern about patch 5 since I am sure I fully
> >understand whether the new algorithm (in __scale_tsc()) is equivalent to
> >current SVM code. I think you also had questions about that.
> >
> >Having said this, the fact that this patch (and patch 9) fix bugs leads me
> >to believe this feature may not have been thoroughly tested.
> >
> >I don't have a pair of appropriate AMD systems to test this series with
> >migration (which is where this can be verified). Aravind, can you find
> >something and see how this works?
> >
>
> Haozhong, Boris-
>
> I am planning to use a Fam10h system (older processor) and Fam15h Model 60h
> (newer processor) for the test case.
>
> Shall try to run the test on a single system as Haozhong mentioned on a
> different reply.
> I ran into a problem with xl right now which I am trying to solve.
>
> So, shall keep you posted on how testing goes.
>
> Btw, I had issues with applying the patches to my local xen.git branch.
> Patches 9 and 10 did not apply cleanly. Here is the log from git apply-
>
> Patch 9:
> Checking patch xen/arch/x86/time.c...
> error: while searching for:
> }
> else
> {
> _u.tsc_timestamp = t->local_tsc_stamp;
> _u.system_time = t->stime_local_stamp;
> _u.tsc_to_system_mul = t->tsc_scale.mul_frac;
> _u.tsc_shift = (s8)t->tsc_scale.shift;
> }
> if ( is_hvm_domain(d) )
> _u.tsc_timestamp += v->arch.hvm_vcpu.cache_tsc_offset;
>
> error: patch failed: xen/arch/x86/time.c:832
>
> I think the complaint is about "_u.tsc_timestamp = t->local_tsc_stamp;".
> I checked current master
> (http://xenbits.xen.org/gitweb/?p=xen.git;a=blob;f=xen/arch/x86/time.c;h=5d7452a2bf8b8fb830c14f8897cfca65cb1ad39e;hb=refs/heads/master)
> and the line there is "tsc_stamp = t->local_tsc_stamp" inside the else block
> and outside it, we have "_u.tsc_timestamp = tsc_stamp"
>
> The rejected hunk for Patch 10:
> +#define VMX_TSC_MULTIPLIER_DEFAULT 0x0001000000000000ULL
> +#define VMX_TSC_MULTIPLIER_MAX 0xffffffffffffffffULL
> +
> #define cpu_has_wbinvd_exiting \
>
> This seems to be because we have the #defines ordered like so on current
> master-
> #define VMX_MISC_CR3_TARGET 0x01ff0000
> #define VMX_MISC_VMWRITE_ALL 0x20000000
>
> #define cpu_has_wbinvd_exiting \
> (vmx_secondary_exec_control & SECONDARY_EXEC_WBINVD_EXITING)
>
> but the *_VMWRITE_ALL define is missing on your diff for Patch 10..
>
> Maybe I am missing something?
>
> Thanks,
> -Aravind.
Hi Aravind,
This patchset has been sent out for quite a while and is based on
commit 9cc1346. Something has changed in master and broken the
structure of patch 9 and patch 10. You can either try the old commit
9cc1346, or my rebased patch 9 and patch 10 in the attachment (on
commit e08f383).
Thanks,
Haozhong
>From 191effb2beb4d309c70d647c4b0347e15fe6a1d1 Mon Sep 17 00:00:00 2001
From: Haozhong Zhang <haozhong.zh...@intel.com>
Date: Mon, 24 Aug 2015 14:13:35 +0800
Subject: [PATCH 09/13] x86/time.c: Scale host TSC in pvclock properly
This patch makes the pvclock return the scaled host TSC and
corresponding scaling parameters to HVM domains if guest TSC is not
emulated and TSC scaling is enabled.
Signed-off-by: Haozhong Zhang <haozhong.zh...@intel.com>
---
xen/arch/x86/time.c | 16 ++++++++++++----
1 file changed, 12 insertions(+), 4 deletions(-)
diff --git a/xen/arch/x86/time.c b/xen/arch/x86/time.c
index 2487b3a..a3e8fe7 100644
--- a/xen/arch/x86/time.c
+++ b/xen/arch/x86/time.c
@@ -821,10 +821,18 @@ static void __update_vcpu_system_time(struct vcpu *v, int
force)
}
else
{
- tsc_stamp = t->local_tsc_stamp;
-
- _u.tsc_to_system_mul = t->tsc_scale.mul_frac;
- _u.tsc_shift = (s8)t->tsc_scale.shift;
+ if ( is_hvm_domain(d) && hvm_funcs.tsc_scaling_supported )
+ {
+ tsc_stamp = hvm_scale_tsc(v, t->local_tsc_stamp);
+ _u.tsc_to_system_mul = d->arch.vtsc_to_ns.mul_frac;
+ _u.tsc_shift = d->arch.vtsc_to_ns.shift;
+ }
+ else
+ {
+ tsc_stamp = t->local_tsc_stamp;
+ _u.tsc_to_system_mul = t->tsc_scale.mul_frac;
+ _u.tsc_shift = (s8)t->tsc_scale.shift;
+ }
}
_u.tsc_timestamp = tsc_stamp;
--
2.4.8
>From 05dfac3a82bbd227da619e14c27ab4f8df438882 Mon Sep 17 00:00:00 2001
From: Haozhong Zhang <haozhong.zh...@intel.com>
Date: Wed, 19 Aug 2015 13:27:11 +0800
Subject: [PATCH 10/13] vmx: Detect and initialize VMX RDTSC(P) scaling
This patch adds the detection and initialization code for VMX TSC
scaling.
Signed-off-by: Haozhong Zhang <haozhong.zh...@intel.com>
---
xen/arch/x86/hvm/vmx/vmcs.c | 11 +++++++++--
xen/arch/x86/hvm/vmx/vmx.c | 9 +++++++++
xen/include/asm-x86/hvm/vmx/vmcs.h | 7 +++++++
3 files changed, 25 insertions(+), 2 deletions(-)
diff --git a/xen/arch/x86/hvm/vmx/vmcs.c b/xen/arch/x86/hvm/vmx/vmcs.c
index 4ea1ad1..716703d 100644
--- a/xen/arch/x86/hvm/vmx/vmcs.c
+++ b/xen/arch/x86/hvm/vmx/vmcs.c
@@ -148,6 +148,7 @@ static void __init vmx_display_features(void)
P(cpu_has_vmx_vmfunc, "VM Functions");
P(cpu_has_vmx_virt_exceptions, "Virtualisation Exceptions");
P(cpu_has_vmx_pml, "Page Modification Logging");
+ P(cpu_has_vmx_tsc_scaling, "RDTSC(P) Scaling");
#undef P
if ( !printed )
@@ -240,7 +241,8 @@ static int vmx_init_vmcs_config(void)
SECONDARY_EXEC_PAUSE_LOOP_EXITING |
SECONDARY_EXEC_ENABLE_INVPCID |
SECONDARY_EXEC_ENABLE_VM_FUNCTIONS |
- SECONDARY_EXEC_ENABLE_VIRT_EXCEPTIONS);
+ SECONDARY_EXEC_ENABLE_VIRT_EXCEPTIONS |
+ SECONDARY_EXEC_TSC_SCALING);
rdmsrl(MSR_IA32_VMX_MISC, _vmx_misc_cap);
if ( _vmx_misc_cap & VMX_MISC_VMWRITE_ALL )
opt |= SECONDARY_EXEC_ENABLE_VMCS_SHADOWING;
@@ -976,7 +978,7 @@ static int construct_vmcs(struct vcpu *v)
__vmwrite(PIN_BASED_VM_EXEC_CONTROL, vmx_pin_based_exec_control);
v->arch.hvm_vmx.exec_control = vmx_cpu_based_exec_control;
- if ( d->arch.vtsc )
+ if ( d->arch.vtsc && !cpu_has_vmx_tsc_scaling )
v->arch.hvm_vmx.exec_control |= CPU_BASED_RDTSC_EXITING;
v->arch.hvm_vmx.secondary_exec_control = vmx_secondary_exec_control;
@@ -1250,6 +1252,9 @@ static int construct_vmcs(struct vcpu *v)
__vmwrite(GUEST_PAT, guest_pat);
}
+ if ( cpu_has_vmx_tsc_scaling )
+ __vmwrite(TSC_MULTIPLIER, VMX_TSC_MULTIPLIER_DEFAULT);
+
vmx_vmcs_exit(v);
/* PVH: paging mode is updated by arch_set_info_guest(). */
@@ -1840,6 +1845,8 @@ void vmcs_dump_vcpu(struct vcpu *v)
printk("IDTVectoring: info=%08x errcode=%08x\n",
vmr32(IDT_VECTORING_INFO), vmr32(IDT_VECTORING_ERROR_CODE));
printk("TSC Offset = 0x%016lx\n", vmr(TSC_OFFSET));
+ if ( v->arch.hvm_vmx.secondary_exec_control & SECONDARY_EXEC_TSC_SCALING )
+ printk("TSC Multiplier = 0x%016lx\n", vmr(TSC_MULTIPLIER));
if ( (v->arch.hvm_vmx.exec_control & CPU_BASED_TPR_SHADOW) ||
(vmx_pin_based_exec_control & PIN_BASED_POSTED_INTERRUPT) )
printk("TPR Threshold = 0x%02x PostedIntrVec = 0x%02x\n",
diff --git a/xen/arch/x86/hvm/vmx/vmx.c b/xen/arch/x86/hvm/vmx/vmx.c
index 624db1c..454440e 100644
--- a/xen/arch/x86/hvm/vmx/vmx.c
+++ b/xen/arch/x86/hvm/vmx/vmx.c
@@ -151,6 +151,8 @@ static int vmx_vcpu_initialise(struct vcpu *v)
if ( v->vcpu_id == 0 )
v->arch.user_regs.eax = 1;
+ v->arch.tsc_scaling_ratio = VMX_TSC_MULTIPLIER_DEFAULT;
+
return 0;
}
@@ -1965,6 +1967,10 @@ static struct hvm_function_table __initdata
vmx_function_table = {
.altp2m_vcpu_emulate_vmfunc = vmx_vcpu_emulate_vmfunc,
/* support for VMX RDTSC(P) scaling */
.tsc_scaling_supported = 0,
+ .default_tsc_scaling_ratio = VMX_TSC_MULTIPLIER_DEFAULT,
+ .max_tsc_scaling_ratio = VMX_TSC_MULTIPLIER_MAX,
+ .tsc_scaling_ratio_frac_bits = 48,
+ .tsc_scaling_ratio_rsvd = 0x0ULL,
};
const struct hvm_function_table * __init start_vmx(void)
@@ -2017,6 +2023,9 @@ const struct hvm_function_table * __init start_vmx(void)
&& cpu_has_vmx_secondary_exec_control )
vmx_function_table.pvh_supported = 1;
+ if ( cpu_has_vmx_tsc_scaling )
+ vmx_function_table.tsc_scaling_supported = 1;
+
setup_vmcs_dump();
return &vmx_function_table;
diff --git a/xen/include/asm-x86/hvm/vmx/vmcs.h
b/xen/include/asm-x86/hvm/vmx/vmcs.h
index 865d9fc..6a7a6b5 100644
--- a/xen/include/asm-x86/hvm/vmx/vmcs.h
+++ b/xen/include/asm-x86/hvm/vmx/vmcs.h
@@ -225,6 +225,7 @@ extern u32 vmx_vmentry_control;
#define SECONDARY_EXEC_ENABLE_VMCS_SHADOWING 0x00004000
#define SECONDARY_EXEC_ENABLE_PML 0x00020000
#define SECONDARY_EXEC_ENABLE_VIRT_EXCEPTIONS 0x00040000
+#define SECONDARY_EXEC_TSC_SCALING 0x02000000
extern u32 vmx_secondary_exec_control;
#define VMX_EPT_EXEC_ONLY_SUPPORTED 0x00000001
@@ -247,6 +248,9 @@ extern u64 vmx_ept_vpid_cap;
#define VMX_MISC_CR3_TARGET 0x01ff0000
#define VMX_MISC_VMWRITE_ALL 0x20000000
+#define VMX_TSC_MULTIPLIER_DEFAULT 0x0001000000000000ULL
+#define VMX_TSC_MULTIPLIER_MAX 0xffffffffffffffffULL
+
#define cpu_has_wbinvd_exiting \
(vmx_secondary_exec_control & SECONDARY_EXEC_WBINVD_EXITING)
#define cpu_has_vmx_virtualize_apic_accesses \
@@ -290,6 +294,8 @@ extern u64 vmx_ept_vpid_cap;
(vmx_secondary_exec_control & SECONDARY_EXEC_ENABLE_VIRT_EXCEPTIONS)
#define cpu_has_vmx_pml \
(vmx_secondary_exec_control & SECONDARY_EXEC_ENABLE_PML)
+#define cpu_has_vmx_tsc_scaling \
+ (vmx_secondary_exec_control & SECONDARY_EXEC_TSC_SCALING)
#define VMCS_RID_TYPE_MASK 0x80000000
@@ -364,6 +370,7 @@ enum vmcs_field {
VMREAD_BITMAP = 0x00002026,
VMWRITE_BITMAP = 0x00002028,
VIRT_EXCEPTION_INFO = 0x0000202a,
+ TSC_MULTIPLIER = 0x00002032,
GUEST_PHYSICAL_ADDRESS = 0x00002400,
VMCS_LINK_POINTER = 0x00002800,
GUEST_IA32_DEBUGCTL = 0x00002802,
--
2.4.8
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel
