Re: [PATCH 2/3] KVM: X86: implement the logic for spinlock optimization
On 07/08/2017 14:28, Longpeng(Mike) wrote: > * kvm_arch_spin_in_kernel() returns whether the vcpu (which exits due to > spinlock) is CPL=0. It only be called by kvm_vcpu_on_spin(), and the > input vcpu is 'me' which get a PAUSE exit now. * > > I split kvm_arch_vcpu_in_kernel(in RFC) into two functions: > kvm_arch_spin_in_kernel and kvm_arch_preempt_in_kernel > > Because of KVM/VMX L1 never set CPU_BASED_PAUSE_EXITING and only set > SECONDARY_EXEC_PAUSE_LOOP_EXITING if supported, so for L1: I understand better now. I think vmx.c should just return true from vmx_spin_in_kernel. However, kvm_arch_vcpu_spin_in_kernel is not necessary. Instead you should make "in_kern" an argument to kvm_vcpu_on_spin (maybe renamed to "yield_to_kernel_mode_vcpu"). Then vmx.c can just call "kvm_vcpu_on_spin(vcpu, true)". > 1. get a PAUSE exit with CPL=0 if PLE is supported > 2. never get a PAUSE exit if don't support PLE > > So, I think it can direct return true(CPL=0) if supports PLE. > > But for nested KVM/VMX(I'm not familiar with nested), it could set > CPU_BASED_PAUSE_EXITING, so I think get_cpl() is also needed. If the nested hypervisor sets CPU_BASED_PAUSE_EXITING, a PAUSE vmexit while running a nested guest would be reflected to the nested hypervisor. So you wouldn't get to handle_pause and thus to kvm_vcpu_on_spin. Thanks, Paolo
Re: [PATCH 2/3] KVM: X86: implement the logic for spinlock optimization
On 08/07/2017 06:45 PM, Paolo Bonzini wrote:
On 07/08/2017 10:44, Longpeng(Mike) wrote:
+
+ /*
+* Intel sdm vol3 ch-25.1.3 says: The “PAUSE-loop exiting”
+* VM-execution control is ignored if CPL > 0. So the vcpu
+* is always exiting with CPL=0 if it uses PLE.
This is not true (how can it be?). What 25.1.3 says is, the VCPU is
always at CPL=0 if you get a PAUSE exit (reason 40) and PAUSE exiting is
0 (it always is for KVM). But here you're looking for a VCPU that
didn't get a PAUSE exit, so the CPL can certainly be 3.
Hi Paolo,
My comment above is something wrong(please forgive my poor English), my
origin meaning is:
The “PAUSE-loop exiting” VM-execution control is ignored if
CPL > 0. So the vcpu's CPL is must 0 if it exits due to PLE.
* kvm_arch_spin_in_kernel() returns whether the vcpu(which exits due to
spinlock) is CPL=0. It only be called by kvm_vcpu_on_spin(), and the
input vcpu is 'me' which get a PAUSE exit now. *
I split kvm_arch_vcpu_in_kernel(in RFC) into two functions:
kvm_arch_spin_in_kernel and kvm_arch_preempt_in_kernel
Because of KVM/VMX L1 never set CPU_BASED_PAUSE_EXITING and only set
SECONDARY_EXEC_PAUSE_LOOP_EXITING if supported, so for L1:
1. get a PAUSE exit with CPL=0 if PLE is supported
2. never get a PAUSE exit if don't support PLE
So, I think it can direct return true(CPL=0) if supports PLE.
But for nested KVM/VMX(I'm not familiar with nested), it could set
CPU_BASED_PAUSE_EXITING, so I think get_cpl() is also needed.
If the above is correct, what about this way( we can save a vmcs_read
opeartion for L1):
kvm_arch_vcpu_spin_in_kernel(vcpu)
{
if (!is_guest_mode(vcpu))
return true;
return vmx_get_cpl(vcpu) == 0;
}
kvm_vcpu_on_spin()
{
/* @me get a PAUSE exit */
me_in_kernel = kvm_arch_vcpu_spin_in_kernel(me);
...
for each vcpu {
...
if (me_in_kernel && !...preempt_in_kernel(vcpu))
continue;
...
}
...
}
---
Regards,
Longpeng(Mike)
However, I understand that vmx_get_cpl can be a bit slow here. You can
actually read SS's access rights directly in this function and get the
DPL from there, that's going to be just a single VMREAD.
The only difference is when vmx->rmode.vm86_active=1. However,
pause-loop exiting is not working properly anyway if
vmx->rmode.vm86_active=1, because CPL=3 according to the processor.
Paolo
+* The following block needs less cycles than vmx_get_cpl().
+*/
+ if (cpu_has_secondary_exec_ctrls())
+ secondary_exec_ctrl = vmcs_read32(SECONDARY_VM_EXEC_CONTROL);
+ if (secondary_exec_ctrl & SECONDARY_EXEC_PAUSE_LOOP_EXITING)
+ return true;
+
Paolo
Re: [PATCH 2/3] KVM: X86: implement the logic for spinlock optimization
On 07/08/2017 10:44, Longpeng(Mike) wrote: > + > + /* > + * Intel sdm vol3 ch-25.1.3 says: The “PAUSE-loop exiting” > + * VM-execution control is ignored if CPL > 0. So the vcpu > + * is always exiting with CPL=0 if it uses PLE. This is not true (how can it be?). What 25.1.3 says is, the VCPU is always at CPL=0 if you get a PAUSE exit (reason 40) and PAUSE exiting is 0 (it always is for KVM). But here you're looking for a VCPU that didn't get a PAUSE exit, so the CPL can certainly be 3. However, I understand that vmx_get_cpl can be a bit slow here. You can actually read SS's access rights directly in this function and get the DPL from there, that's going to be just a single VMREAD. The only difference is when vmx->rmode.vm86_active=1. However, pause-loop exiting is not working properly anyway if vmx->rmode.vm86_active=1, because CPL=3 according to the processor. Paolo > + * The following block needs less cycles than vmx_get_cpl(). > + */ > + if (cpu_has_secondary_exec_ctrls()) > + secondary_exec_ctrl = vmcs_read32(SECONDARY_VM_EXEC_CONTROL); > + if (secondary_exec_ctrl & SECONDARY_EXEC_PAUSE_LOOP_EXITING) > + return true; > + Paolo
