Re: [PATCH 0/9] Intel Processor Trace virtulization enabling
On 23/10/2017 10:01, Kang, Luwei wrote: > >>> So, can we enable it in L1 >>> guest only first? I think it is not worth to disable EPT for L1 to >>> enable intel PT. what is your opinion? >> Yes, we can enable it. But since KVM sets IA32_VMX_MISC[14]=0, your patches >> must forbid enabling processor trace during VMX >> operation. > > L1 hypervisor can't get the capability of " TraceEn can be set in > VMX operation (IA32_VMX_MISC[bit 14] is 0)" and set it to 0. We need > to trap whether L1 hypervisor have enable VMXON, and forbid enable PT > when vmxon. Is that right? Or have something else? Correct. I don't have the SDM at hand, so I'm not sure what happens if you do a VMXON while TraceEn is 1. However, that has to be handled as well. Paolo
Re: [PATCH 0/9] Intel Processor Trace virtulization enabling
On 23/10/2017 10:01, Kang, Luwei wrote: > >>> So, can we enable it in L1 >>> guest only first? I think it is not worth to disable EPT for L1 to >>> enable intel PT. what is your opinion? >> Yes, we can enable it. But since KVM sets IA32_VMX_MISC[14]=0, your patches >> must forbid enabling processor trace during VMX >> operation. > > L1 hypervisor can't get the capability of " TraceEn can be set in > VMX operation (IA32_VMX_MISC[bit 14] is 0)" and set it to 0. We need > to trap whether L1 hypervisor have enable VMXON, and forbid enable PT > when vmxon. Is that right? Or have something else? Correct. I don't have the SDM at hand, so I'm not sure what happens if you do a VMXON while TraceEn is 1. However, that has to be handled as well. Paolo
RE: [PATCH 0/9] Intel Processor Trace virtulization enabling
> > HI Paolo, Thanks for your clarify. Have understood. So, we should set > > "use GPA for processor tracing" in any way( if we can do it) even in > > system mode. There don't have problem in no nested but have problem in > > nested if not set this bit. Still talking with hardware designer but > > please don't expect it can be change in SDM or hardware(fail vmentry > > if they are not respected) soon. > > No change in hardware is needed. > > What I'm asking for is to define a bit in some architectural MSR such that, > _if the bit is 1_, you must have one of: > > - RTIT_CTL = 0 > > - enable EPT = 0 > > - enable EPT = use GPA for processor tracing = 1, RTIT_CTL != 0 > > or vmentry would fail. > > If the bit is 1 and RTIT_CTL = 0 and enable EPT = 1 and use GPA for processor > tracing = 0, the hypervisor must trap RTIT_CTL writes > or behavior is undefined. > > Processors would just set it to 0 and have absolutely no change in behavior. > Get it. Will update with you when hardware designer have any response. > > So, can we enable it in L1 > > guest only first? I think it is not worth to disable EPT for L1 to > > enable intel PT. what is your opinion? > > Yes, we can enable it. But since KVM sets IA32_VMX_MISC[14]=0, your patches > must forbid enabling processor trace during VMX > operation. L1 hypervisor can't get the capability of " TraceEn can be set in VMX operation (IA32_VMX_MISC[bit 14] is 0)" and set it to 0. We need to trap whether L1 hypervisor have enable VMXON, and forbid enable PT when vmxon. Is that right? Or have something else? Thanks, Luwei Kang > > (In fact, another source of complexity is that we'd have to write the VMPTRLD > packet ourselves to the guest's processor trace > buffer). > > Paolo
RE: [PATCH 0/9] Intel Processor Trace virtulization enabling
> > HI Paolo, Thanks for your clarify. Have understood. So, we should set > > "use GPA for processor tracing" in any way( if we can do it) even in > > system mode. There don't have problem in no nested but have problem in > > nested if not set this bit. Still talking with hardware designer but > > please don't expect it can be change in SDM or hardware(fail vmentry > > if they are not respected) soon. > > No change in hardware is needed. > > What I'm asking for is to define a bit in some architectural MSR such that, > _if the bit is 1_, you must have one of: > > - RTIT_CTL = 0 > > - enable EPT = 0 > > - enable EPT = use GPA for processor tracing = 1, RTIT_CTL != 0 > > or vmentry would fail. > > If the bit is 1 and RTIT_CTL = 0 and enable EPT = 1 and use GPA for processor > tracing = 0, the hypervisor must trap RTIT_CTL writes > or behavior is undefined. > > Processors would just set it to 0 and have absolutely no change in behavior. > Get it. Will update with you when hardware designer have any response. > > So, can we enable it in L1 > > guest only first? I think it is not worth to disable EPT for L1 to > > enable intel PT. what is your opinion? > > Yes, we can enable it. But since KVM sets IA32_VMX_MISC[14]=0, your patches > must forbid enabling processor trace during VMX > operation. L1 hypervisor can't get the capability of " TraceEn can be set in VMX operation (IA32_VMX_MISC[bit 14] is 0)" and set it to 0. We need to trap whether L1 hypervisor have enable VMXON, and forbid enable PT when vmxon. Is that right? Or have something else? Thanks, Luwei Kang > > (In fact, another source of complexity is that we'd have to write the VMPTRLD > packet ourselves to the guest's processor trace > buffer). > > Paolo
Re: [PATCH 0/9] Intel Processor Trace virtulization enabling
On 20/10/2017 02:22, Kang, Luwei wrote: > HI Paolo, Thanks for your clarify. Have understood. So, we should set > "use GPA for processor tracing" in any way( if we can do it) even in > system mode. There don't have problem in no nested but have problem > in nested if not set this bit. Still talking with hardware designer > but please don't expect it can be change in SDM or hardware(fail > vmentry if they are not respected) soon. No change in hardware is needed. What I'm asking for is to define a bit in some architectural MSR such that, _if the bit is 1_, you must have one of: - RTIT_CTL = 0 - enable EPT = 0 - enable EPT = use GPA for processor tracing = 1, RTIT_CTL != 0 or vmentry would fail. If the bit is 1 and RTIT_CTL = 0 and enable EPT = 1 and use GPA for processor tracing = 0, the hypervisor must trap RTIT_CTL writes or behavior is undefined. Processors would just set it to 0 and have absolutely no change in behavior. > So, can we enable it in L1 > guest only first? I think it is not worth to disable EPT for L1 to > enable intel PT. what is your opinion? Yes, we can enable it. But since KVM sets IA32_VMX_MISC[14]=0, your patches must forbid enabling processor trace during VMX operation. (In fact, another source of complexity is that we'd have to write the VMPTRLD packet ourselves to the guest's processor trace buffer). Paolo
Re: [PATCH 0/9] Intel Processor Trace virtulization enabling
On 20/10/2017 02:22, Kang, Luwei wrote: > HI Paolo, Thanks for your clarify. Have understood. So, we should set > "use GPA for processor tracing" in any way( if we can do it) even in > system mode. There don't have problem in no nested but have problem > in nested if not set this bit. Still talking with hardware designer > but please don't expect it can be change in SDM or hardware(fail > vmentry if they are not respected) soon. No change in hardware is needed. What I'm asking for is to define a bit in some architectural MSR such that, _if the bit is 1_, you must have one of: - RTIT_CTL = 0 - enable EPT = 0 - enable EPT = use GPA for processor tracing = 1, RTIT_CTL != 0 or vmentry would fail. If the bit is 1 and RTIT_CTL = 0 and enable EPT = 1 and use GPA for processor tracing = 0, the hypervisor must trap RTIT_CTL writes or behavior is undefined. Processors would just set it to 0 and have absolutely no change in behavior. > So, can we enable it in L1 > guest only first? I think it is not worth to disable EPT for L1 to > enable intel PT. what is your opinion? Yes, we can enable it. But since KVM sets IA32_VMX_MISC[14]=0, your patches must forbid enabling processor trace during VMX operation. (In fact, another source of complexity is that we'd have to write the VMPTRLD packet ourselves to the guest's processor trace buffer). Paolo
RE: [PATCH 0/9] Intel Processor Trace virtulization enabling
> On 19/10/2017 07:54, Kang, Luwei wrote: > >>> Get it. I have feedback to hardware architect. I hope it can be applied > >>> but it may need wait a long time. > >> Note that the hardware need not do anything. However it would be > >> nice if the SDM can define a bit _for the hypervisors_ to enforce the > >> above constraint and fail vmentry if they are not respected. > > > > Hi Paolo, > > Thanks for your response. I have a question want to ask for you. As > > you mentioned in previous mail " We would like the nested hypervisor > > to be forced to set the "use GPA for processor tracing"". Is there > > have any problem if we don't set "use GPA for processor tracing" in > > nested hypervisor? > > If the nested hypervisor doesn't set "use GPA for processor tracing", the > processor should use L1 addresses for processor tracing. > This however is not possible without shadowing the ToPA, same as in > non-nested virtualization for <=Skylake processors. > > So we have > > | mode > |--- > nested? | system-wide | host-guest > +---+--- > no | use HPA for tracing | use GPA for tracing > | (no EPT) | (EPT is GPA->HPA) > +---+--- > yes | use GPA for tracing | use nGPA for tracing > | (EPT is nGPA->HPA!!) | (EPT is nGPA->HPA, so ok) > > (for nested, L0 mode of course must be host-guest). If the nested hypervisor > wants to use system-wide tracing, it cannot use "use > GPA for tracing" because the EPT table doesn't have the right mapping of > L1->L0 physical address. > > So if you want to do system-wide L1 tracing you have to disable EPT for L1, > and if you want to do host-guest L1 tracing you have to > enable it. > HI Paolo, Thanks for your clarify. Have understood. So, we should set "use GPA for processor tracing" in any way( if we can do it) even in system mode. There don't have problem in no nested but have problem in nested if not set this bit. Still talking with hardware designer but please don't expect it can be change in SDM or hardware(fail vmentry if they are not respected) soon. So, can we enable it in L1 guest only first? I think it is not worth to disable EPT for L1 to enable intel PT. what is your opinion? Thanks, Luwei Kang
RE: [PATCH 0/9] Intel Processor Trace virtulization enabling
> On 19/10/2017 07:54, Kang, Luwei wrote: > >>> Get it. I have feedback to hardware architect. I hope it can be applied > >>> but it may need wait a long time. > >> Note that the hardware need not do anything. However it would be > >> nice if the SDM can define a bit _for the hypervisors_ to enforce the > >> above constraint and fail vmentry if they are not respected. > > > > Hi Paolo, > > Thanks for your response. I have a question want to ask for you. As > > you mentioned in previous mail " We would like the nested hypervisor > > to be forced to set the "use GPA for processor tracing"". Is there > > have any problem if we don't set "use GPA for processor tracing" in > > nested hypervisor? > > If the nested hypervisor doesn't set "use GPA for processor tracing", the > processor should use L1 addresses for processor tracing. > This however is not possible without shadowing the ToPA, same as in > non-nested virtualization for <=Skylake processors. > > So we have > > | mode > |--- > nested? | system-wide | host-guest > +---+--- > no | use HPA for tracing | use GPA for tracing > | (no EPT) | (EPT is GPA->HPA) > +---+--- > yes | use GPA for tracing | use nGPA for tracing > | (EPT is nGPA->HPA!!) | (EPT is nGPA->HPA, so ok) > > (for nested, L0 mode of course must be host-guest). If the nested hypervisor > wants to use system-wide tracing, it cannot use "use > GPA for tracing" because the EPT table doesn't have the right mapping of > L1->L0 physical address. > > So if you want to do system-wide L1 tracing you have to disable EPT for L1, > and if you want to do host-guest L1 tracing you have to > enable it. > HI Paolo, Thanks for your clarify. Have understood. So, we should set "use GPA for processor tracing" in any way( if we can do it) even in system mode. There don't have problem in no nested but have problem in nested if not set this bit. Still talking with hardware designer but please don't expect it can be change in SDM or hardware(fail vmentry if they are not respected) soon. So, can we enable it in L1 guest only first? I think it is not worth to disable EPT for L1 to enable intel PT. what is your opinion? Thanks, Luwei Kang
Re: [PATCH 0/9] Intel Processor Trace virtulization enabling
On 19/10/2017 07:54, Kang, Luwei wrote: >>> Get it. I have feedback to hardware architect. I hope it can be applied but >>> it may need wait a long time. >> Note that the hardware need not do anything. However it would be nice if >> the SDM can define a bit _for the hypervisors_ to >> enforce the above constraint and fail vmentry if they are not respected. > > Hi Paolo, > Thanks for your response. I have a question want to ask for you. As > you mentioned in previous mail " We would like the nested hypervisor > to be forced to set the "use GPA for processor tracing"". Is there > have any problem if we don't set "use GPA for processor tracing" in > nested hypervisor? If the nested hypervisor doesn't set "use GPA for processor tracing", the processor should use L1 addresses for processor tracing. This however is not possible without shadowing the ToPA, same as in non-nested virtualization for <=Skylake processors. So we have | mode |--- nested? | system-wide | host-guest +---+--- no | use HPA for tracing | use GPA for tracing | (no EPT) | (EPT is GPA->HPA) +---+--- yes | use GPA for tracing | use nGPA for tracing | (EPT is nGPA->HPA!!) | (EPT is nGPA->HPA, so ok) (for nested, L0 mode of course must be host-guest). If the nested hypervisor wants to use system-wide tracing, it cannot use "use GPA for tracing" because the EPT table doesn't have the right mapping of L1->L0 physical address. So if you want to do system-wide L1 tracing you have to disable EPT for L1, and if you want to do host-guest L1 tracing you have to enable it. Paolo > If yes, what should we do? In patch 9, I pass > though PT MSRs ( IA32_RTIT_* ) to guest.
Re: [PATCH 0/9] Intel Processor Trace virtulization enabling
On 19/10/2017 07:54, Kang, Luwei wrote: >>> Get it. I have feedback to hardware architect. I hope it can be applied but >>> it may need wait a long time. >> Note that the hardware need not do anything. However it would be nice if >> the SDM can define a bit _for the hypervisors_ to >> enforce the above constraint and fail vmentry if they are not respected. > > Hi Paolo, > Thanks for your response. I have a question want to ask for you. As > you mentioned in previous mail " We would like the nested hypervisor > to be forced to set the "use GPA for processor tracing"". Is there > have any problem if we don't set "use GPA for processor tracing" in > nested hypervisor? If the nested hypervisor doesn't set "use GPA for processor tracing", the processor should use L1 addresses for processor tracing. This however is not possible without shadowing the ToPA, same as in non-nested virtualization for <=Skylake processors. So we have | mode |--- nested? | system-wide | host-guest +---+--- no | use HPA for tracing | use GPA for tracing | (no EPT) | (EPT is GPA->HPA) +---+--- yes | use GPA for tracing | use nGPA for tracing | (EPT is nGPA->HPA!!) | (EPT is nGPA->HPA, so ok) (for nested, L0 mode of course must be host-guest). If the nested hypervisor wants to use system-wide tracing, it cannot use "use GPA for tracing" because the EPT table doesn't have the right mapping of L1->L0 physical address. So if you want to do system-wide L1 tracing you have to disable EPT for L1, and if you want to do host-guest L1 tracing you have to enable it. Paolo > If yes, what should we do? In patch 9, I pass > though PT MSRs ( IA32_RTIT_* ) to guest.
RE: [PATCH 0/9] Intel Processor Trace virtulization enabling
> Nested virtualization is interesting. We would like the nested > hypervisor to be forced to set the "use GPA for processor tracing" > secondary execution control whenever "enable EPT" is set and > RTIT_CTL is nonzero. There is no way to encode that in > IA32_VMX_PROCBASED_CTLS2, however. It would be nice if Intel could > reserve a bit in IA32_VMX_EPT_VPID_CAP for KVM to express that > constraint. > >>> > >>> Do you mean if nested hypervisor get the capability of "Guest PT use > >>> GPA" and EPT has enable. Highly recommend nested hypervisor set " > >>> Guest PT use GPA " as well. > >> > >> Well, it's required more than recommended. However, it's only required if > >> "enable EPT" is set and RTIT_CTL is nonzero. > >> > >>> If nested hypervisor is also KVM, "use GPA for processor tracing" > >>> will be set for sure. But other hypervisor may not do that. So, we'd > >>> better add a flag in IA32_VMX_EPT_VPID_CAP to express that constraint. > >> > >> Correct. The constraint would be: > >> > >> * RTIT_CTL on entry is zero if EPT is disabled > >> > >> * RTIT_CTL on entry is zero if EPT is enabled and "Guest PT uses GPA" > >> is zero > >> > >> Maybe IA32_VMX_EPT_VPID_CAP is not the best place. I'll let Intel decide > >> that. > > > > Get it. I have feedback to hardware architect. I hope it can be applied but > > it may need wait a long time. > > Note that the hardware need not do anything. However it would be nice if the > SDM can define a bit _for the hypervisors_ to > enforce the above constraint and fail vmentry if they are not respected. > Hi Paolo, Thanks for your response. I have a question want to ask for you. As you mentioned in previous mail " We would like the nested hypervisor to be forced to set the "use GPA for processor tracing"". Is there have any problem if we don't set "use GPA for processor tracing" in nested hypervisor? If yes, what should we do? In patch 9, I pass though PT MSRs ( IA32_RTIT_* ) to guest. Thanks, Luwei Kang
RE: [PATCH 0/9] Intel Processor Trace virtulization enabling
> Nested virtualization is interesting. We would like the nested > hypervisor to be forced to set the "use GPA for processor tracing" > secondary execution control whenever "enable EPT" is set and > RTIT_CTL is nonzero. There is no way to encode that in > IA32_VMX_PROCBASED_CTLS2, however. It would be nice if Intel could > reserve a bit in IA32_VMX_EPT_VPID_CAP for KVM to express that > constraint. > >>> > >>> Do you mean if nested hypervisor get the capability of "Guest PT use > >>> GPA" and EPT has enable. Highly recommend nested hypervisor set " > >>> Guest PT use GPA " as well. > >> > >> Well, it's required more than recommended. However, it's only required if > >> "enable EPT" is set and RTIT_CTL is nonzero. > >> > >>> If nested hypervisor is also KVM, "use GPA for processor tracing" > >>> will be set for sure. But other hypervisor may not do that. So, we'd > >>> better add a flag in IA32_VMX_EPT_VPID_CAP to express that constraint. > >> > >> Correct. The constraint would be: > >> > >> * RTIT_CTL on entry is zero if EPT is disabled > >> > >> * RTIT_CTL on entry is zero if EPT is enabled and "Guest PT uses GPA" > >> is zero > >> > >> Maybe IA32_VMX_EPT_VPID_CAP is not the best place. I'll let Intel decide > >> that. > > > > Get it. I have feedback to hardware architect. I hope it can be applied but > > it may need wait a long time. > > Note that the hardware need not do anything. However it would be nice if the > SDM can define a bit _for the hypervisors_ to > enforce the above constraint and fail vmentry if they are not respected. > Hi Paolo, Thanks for your response. I have a question want to ask for you. As you mentioned in previous mail " We would like the nested hypervisor to be forced to set the "use GPA for processor tracing"". Is there have any problem if we don't set "use GPA for processor tracing" in nested hypervisor? If yes, what should we do? In patch 9, I pass though PT MSRs ( IA32_RTIT_* ) to guest. Thanks, Luwei Kang
Re: [PATCH 0/9] Intel Processor Trace virtulization enabling
Please send patch series with git's sendemail.thread=shallow and sendemail.chainReplyTo=false settings (see the man git-send-email --thread option for an explanation of the different settings). That way email clients treat the entire series as a single email thread. Patch management and continuous integration tools may have trouble with patch series that are not sent in this standard format. Thanks, Stefan
Re: [PATCH 0/9] Intel Processor Trace virtulization enabling
Please send patch series with git's sendemail.thread=shallow and sendemail.chainReplyTo=false settings (see the man git-send-email --thread option for an explanation of the different settings). That way email clients treat the entire series as a single email thread. Patch management and continuous integration tools may have trouble with patch series that are not sent in this standard format. Thanks, Stefan
Re: [PATCH 0/9] Intel Processor Trace virtulization enabling
On 18/10/2017 13:06, Kang, Luwei wrote: Nested virtualization is interesting. We would like the nested hypervisor to be forced to set the "use GPA for processor tracing" secondary execution control whenever "enable EPT" is set and RTIT_CTL is nonzero. There is no way to encode that in IA32_VMX_PROCBASED_CTLS2, however. It would be nice if Intel could reserve a bit in IA32_VMX_EPT_VPID_CAP for KVM to express that constraint. >>> >>> Do you mean if nested hypervisor get the capability of "Guest PT use >>> GPA" and EPT has enable. Highly recommend nested hypervisor set " >>> Guest PT use GPA " as well. >> >> Well, it's required more than recommended. However, it's only required if >> "enable EPT" is set and RTIT_CTL is nonzero. >> >>> If nested hypervisor is also KVM, "use GPA for processor tracing" >>> will be set for sure. But other hypervisor may not do that. So, we'd >>> better add a flag in IA32_VMX_EPT_VPID_CAP to express that constraint. >> >> Correct. The constraint would be: >> >> * RTIT_CTL on entry is zero if EPT is disabled >> >> * RTIT_CTL on entry is zero if EPT is enabled and "Guest PT uses GPA" is zero >> >> Maybe IA32_VMX_EPT_VPID_CAP is not the best place. I'll let Intel decide >> that. > > Get it. I have feedback to hardware architect. I hope it can be applied but > it may need wait a long time. Note that the hardware need not do anything. However it would be nice if the SDM can define a bit _for the hypervisors_ to enforce the above constraint and fail vmentry if they are not respected. Paolo
Re: [PATCH 0/9] Intel Processor Trace virtulization enabling
On 18/10/2017 13:06, Kang, Luwei wrote: Nested virtualization is interesting. We would like the nested hypervisor to be forced to set the "use GPA for processor tracing" secondary execution control whenever "enable EPT" is set and RTIT_CTL is nonzero. There is no way to encode that in IA32_VMX_PROCBASED_CTLS2, however. It would be nice if Intel could reserve a bit in IA32_VMX_EPT_VPID_CAP for KVM to express that constraint. >>> >>> Do you mean if nested hypervisor get the capability of "Guest PT use >>> GPA" and EPT has enable. Highly recommend nested hypervisor set " >>> Guest PT use GPA " as well. >> >> Well, it's required more than recommended. However, it's only required if >> "enable EPT" is set and RTIT_CTL is nonzero. >> >>> If nested hypervisor is also KVM, "use GPA for processor tracing" >>> will be set for sure. But other hypervisor may not do that. So, we'd >>> better add a flag in IA32_VMX_EPT_VPID_CAP to express that constraint. >> >> Correct. The constraint would be: >> >> * RTIT_CTL on entry is zero if EPT is disabled >> >> * RTIT_CTL on entry is zero if EPT is enabled and "Guest PT uses GPA" is zero >> >> Maybe IA32_VMX_EPT_VPID_CAP is not the best place. I'll let Intel decide >> that. > > Get it. I have feedback to hardware architect. I hope it can be applied but > it may need wait a long time. Note that the hardware need not do anything. However it would be nice if the SDM can define a bit _for the hypervisors_ to enforce the above constraint and fail vmentry if they are not respected. Paolo
RE: [PATCH 0/9] Intel Processor Trace virtulization enabling
> >> Nested virtualization is interesting. We would like the nested > >> hypervisor to be forced to set the "use GPA for processor tracing" > >> secondary execution control whenever "enable EPT" is set and RTIT_CTL > >> is nonzero. There is no way to encode that in > >> IA32_VMX_PROCBASED_CTLS2, however. It would be nice if Intel could > >> reserve a bit in IA32_VMX_EPT_VPID_CAP for KVM to express that > >> constraint. > > > > Do you mean if nested hypervisor get the capability of "Guest PT use > > GPA" and EPT has enable. Highly recommend nested hypervisor set " > > Guest PT use GPA " as well. > > Well, it's required more than recommended. However, it's only required if > "enable EPT" is set and RTIT_CTL is nonzero. > > > If nested hypervisor is also KVM, "use GPA for processor tracing" > > will be set for sure. But other hypervisor may not do that. So, we'd > > better add a flag in IA32_VMX_EPT_VPID_CAP to express that constraint. > > Correct. The constraint would be: > > * RTIT_CTL on entry is zero if EPT is disabled > > * RTIT_CTL on entry is zero if EPT is enabled and "Guest PT uses GPA" is zero > > Maybe IA32_VMX_EPT_VPID_CAP is not the best place. I'll let Intel decide > that. > Get it. I have feedback to hardware architect. I hope it can be applied but it may need wait a long time. Thanks, Luwei Kang
RE: [PATCH 0/9] Intel Processor Trace virtulization enabling
> >> Nested virtualization is interesting. We would like the nested > >> hypervisor to be forced to set the "use GPA for processor tracing" > >> secondary execution control whenever "enable EPT" is set and RTIT_CTL > >> is nonzero. There is no way to encode that in > >> IA32_VMX_PROCBASED_CTLS2, however. It would be nice if Intel could > >> reserve a bit in IA32_VMX_EPT_VPID_CAP for KVM to express that > >> constraint. > > > > Do you mean if nested hypervisor get the capability of "Guest PT use > > GPA" and EPT has enable. Highly recommend nested hypervisor set " > > Guest PT use GPA " as well. > > Well, it's required more than recommended. However, it's only required if > "enable EPT" is set and RTIT_CTL is nonzero. > > > If nested hypervisor is also KVM, "use GPA for processor tracing" > > will be set for sure. But other hypervisor may not do that. So, we'd > > better add a flag in IA32_VMX_EPT_VPID_CAP to express that constraint. > > Correct. The constraint would be: > > * RTIT_CTL on entry is zero if EPT is disabled > > * RTIT_CTL on entry is zero if EPT is enabled and "Guest PT uses GPA" is zero > > Maybe IA32_VMX_EPT_VPID_CAP is not the best place. I'll let Intel decide > that. > Get it. I have feedback to hardware architect. I hope it can be applied but it may need wait a long time. Thanks, Luwei Kang
Re: [PATCH 0/9] Intel Processor Trace virtulization enabling
On 18/10/2017 12:18, Kang, Luwei wrote: >> Nested virtualization is interesting. We would like the nested >> hypervisor to be forced to set the "use GPA for processor tracing" >> secondary execution control whenever "enable EPT" is set and >> RTIT_CTL is nonzero. There is no way to encode that in >> IA32_VMX_PROCBASED_CTLS2, however. It would be nice if Intel could >> reserve a bit in IA32_VMX_EPT_VPID_CAP for KVM to express that >> constraint. > > Do you mean if nested hypervisor get the capability of "Guest PT use > GPA" and EPT has enable. Highly recommend nested hypervisor set " > Guest PT use GPA " as well. Well, it's required more than recommended. However, it's only required if "enable EPT" is set and RTIT_CTL is nonzero. > If nested hypervisor is also KVM, "use GPA for processor tracing" > will be set for sure. But other hypervisor may not do that. So, we'd > better add a flag in IA32_VMX_EPT_VPID_CAP to express that > constraint. Correct. The constraint would be: * RTIT_CTL on entry is zero if EPT is disabled * RTIT_CTL on entry is zero if EPT is enabled and "Guest PT uses GPA" is zero Maybe IA32_VMX_EPT_VPID_CAP is not the best place. I'll let Intel decide that. Paolo
Re: [PATCH 0/9] Intel Processor Trace virtulization enabling
On 18/10/2017 12:18, Kang, Luwei wrote: >> Nested virtualization is interesting. We would like the nested >> hypervisor to be forced to set the "use GPA for processor tracing" >> secondary execution control whenever "enable EPT" is set and >> RTIT_CTL is nonzero. There is no way to encode that in >> IA32_VMX_PROCBASED_CTLS2, however. It would be nice if Intel could >> reserve a bit in IA32_VMX_EPT_VPID_CAP for KVM to express that >> constraint. > > Do you mean if nested hypervisor get the capability of "Guest PT use > GPA" and EPT has enable. Highly recommend nested hypervisor set " > Guest PT use GPA " as well. Well, it's required more than recommended. However, it's only required if "enable EPT" is set and RTIT_CTL is nonzero. > If nested hypervisor is also KVM, "use GPA for processor tracing" > will be set for sure. But other hypervisor may not do that. So, we'd > better add a flag in IA32_VMX_EPT_VPID_CAP to express that > constraint. Correct. The constraint would be: * RTIT_CTL on entry is zero if EPT is disabled * RTIT_CTL on entry is zero if EPT is enabled and "Guest PT uses GPA" is zero Maybe IA32_VMX_EPT_VPID_CAP is not the best place. I'll let Intel decide that. Paolo
RE: [PATCH 0/9] Intel Processor Trace virtulization enabling
> On 16/10/2017 14:09, Luwei Kang wrote: > > 2. Enabling use of EPT to redirect PT output. > > — This enables the VMM to elect to virtualize the PT output buffer using > > EPT. In this mode, the CPU will treat PT output > addresses as Guest Physical Addresses (GPAs) and translate them using EPT. > This means that Intel PT output reads (of the ToPA > table) and writes (of trace output) can cause EPT violations, and other > output events. > > Yay yay yay! ;) > > > Intel Processor Trace virtualization can be work in one of 4 possible modes > > by set new option "pt_mode". Default is host guest > mode. > > a. system-wide: trace both host/guest and output to host buffer; b. > > host-only: only trace host and output to host buffer; c. guest-only: > > only trace guest and output to guest buffer; d. host-guest: trace > > host/guest simultaneous and output to their respective buffer. > > I think the guest-only mode isn't particularly useful, and I would remove it. Agree, will remove it. > > Nested virtualization is interesting. We would like the nested hypervisor to > be forced to set the "use GPA for processor tracing" > secondary execution control whenever "enable EPT" is set and RTIT_CTL is > nonzero. There is no way to encode that in > IA32_VMX_PROCBASED_CTLS2, however. It would be nice if Intel could reserve a > bit in IA32_VMX_EPT_VPID_CAP for KVM to > express that constraint. Do you mean if nested hypervisor get the capability of "Guest PT use GPA" and EPT has enable. Highly recommend nested hypervisor set " Guest PT use GPA " as well. If nested hypervisor is also KVM, "use GPA for processor tracing" will be set for sure. But other hypervisor may not do that. So, we'd better add a flag in IA32_VMX_EPT_VPID_CAP to express that constraint. Is that right? Thanks, Luwei Kang
RE: [PATCH 0/9] Intel Processor Trace virtulization enabling
> On 16/10/2017 14:09, Luwei Kang wrote: > > 2. Enabling use of EPT to redirect PT output. > > — This enables the VMM to elect to virtualize the PT output buffer using > > EPT. In this mode, the CPU will treat PT output > addresses as Guest Physical Addresses (GPAs) and translate them using EPT. > This means that Intel PT output reads (of the ToPA > table) and writes (of trace output) can cause EPT violations, and other > output events. > > Yay yay yay! ;) > > > Intel Processor Trace virtualization can be work in one of 4 possible modes > > by set new option "pt_mode". Default is host guest > mode. > > a. system-wide: trace both host/guest and output to host buffer; b. > > host-only: only trace host and output to host buffer; c. guest-only: > > only trace guest and output to guest buffer; d. host-guest: trace > > host/guest simultaneous and output to their respective buffer. > > I think the guest-only mode isn't particularly useful, and I would remove it. Agree, will remove it. > > Nested virtualization is interesting. We would like the nested hypervisor to > be forced to set the "use GPA for processor tracing" > secondary execution control whenever "enable EPT" is set and RTIT_CTL is > nonzero. There is no way to encode that in > IA32_VMX_PROCBASED_CTLS2, however. It would be nice if Intel could reserve a > bit in IA32_VMX_EPT_VPID_CAP for KVM to > express that constraint. Do you mean if nested hypervisor get the capability of "Guest PT use GPA" and EPT has enable. Highly recommend nested hypervisor set " Guest PT use GPA " as well. If nested hypervisor is also KVM, "use GPA for processor tracing" will be set for sure. But other hypervisor may not do that. So, we'd better add a flag in IA32_VMX_EPT_VPID_CAP to express that constraint. Is that right? Thanks, Luwei Kang
Re: [PATCH 0/9] Intel Processor Trace virtulization enabling
On 16/10/2017 14:09, Luwei Kang wrote: > 2. Enabling use of EPT to redirect PT output. > — This enables the VMM to elect to virtualize the PT output buffer using > EPT. In this mode, the CPU will treat PT output addresses as Guest Physical > Addresses (GPAs) and translate them using EPT. This means that Intel PT > output reads (of the ToPA table) and writes (of trace output) can cause EPT > violations, and other output events. Yay yay yay! ;) > Intel Processor Trace virtualization can be work in one of 4 possible modes > by set new option "pt_mode". Default is host guest mode. > a. system-wide: trace both host/guest and output to host buffer; > b. host-only: only trace host and output to host buffer; > c. guest-only: only trace guest and output to guest buffer; > d. host-guest: trace host/guest simultaneous and output to their respective > buffer. I think the guest-only mode isn't particularly useful, and I would remove it. Nested virtualization is interesting. We would like the nested hypervisor to be forced to set the "use GPA for processor tracing" secondary execution control whenever "enable EPT" is set and RTIT_CTL is nonzero. There is no way to encode that in IA32_VMX_PROCBASED_CTLS2, however. It would be nice if Intel could reserve a bit in IA32_VMX_EPT_VPID_CAP for KVM to express that constraint. Paolo
Re: [PATCH 0/9] Intel Processor Trace virtulization enabling
On 16/10/2017 14:09, Luwei Kang wrote: > 2. Enabling use of EPT to redirect PT output. > — This enables the VMM to elect to virtualize the PT output buffer using > EPT. In this mode, the CPU will treat PT output addresses as Guest Physical > Addresses (GPAs) and translate them using EPT. This means that Intel PT > output reads (of the ToPA table) and writes (of trace output) can cause EPT > violations, and other output events. Yay yay yay! ;) > Intel Processor Trace virtualization can be work in one of 4 possible modes > by set new option "pt_mode". Default is host guest mode. > a. system-wide: trace both host/guest and output to host buffer; > b. host-only: only trace host and output to host buffer; > c. guest-only: only trace guest and output to guest buffer; > d. host-guest: trace host/guest simultaneous and output to their respective > buffer. I think the guest-only mode isn't particularly useful, and I would remove it. Nested virtualization is interesting. We would like the nested hypervisor to be forced to set the "use GPA for processor tracing" secondary execution control whenever "enable EPT" is set and RTIT_CTL is nonzero. There is no way to encode that in IA32_VMX_PROCBASED_CTLS2, however. It would be nice if Intel could reserve a bit in IA32_VMX_EPT_VPID_CAP for KVM to express that constraint. Paolo
