Re: [PATCH v3 4/4] x86/cpu-policy: Derive RSBA/RRSBA for guest policies
On 16/06/2023 1:12 pm, Jan Beulich wrote: > On 15.06.2023 20:17, Andrew Cooper wrote: >> On 15/06/2023 1:13 pm, Jan Beulich wrote: >>> On 15.06.2023 12:41, Andrew Cooper wrote: On 15/06/2023 9:30 am, Jan Beulich wrote: > On 14.06.2023 20:12, Andrew Cooper wrote: >> On 13/06/2023 10:59 am, Jan Beulich wrote: >>> On 12.06.2023 18:13, Andrew Cooper wrote: The RSBA bit, "RSB Alternative", means that the RSB may use alternative predictors when empty. From a practical point of view, this mean "Retpoline not safe". Enhanced IBRS (officially IBRS_ALL in Intel's docs, previously IBRS_ATT) is a statement that IBRS is implemented in hardware (as opposed to the form retrofitted to existing CPUs in microcode). The RRSBA bit, "Restricted-RSBA", is a combination of RSBA, and the eIBRS property that predictions are tagged with the mode in which they were learnt. Therefore, it means "when eIBRS is active, the RSB may fall back to alternative predictors but restricted to the current prediction mode". As such, it's stronger statement than RSBA, but still means "Retpoline not safe". CPUs are not expected to enumerate both RSBA and RRSBA. Add feature dependencies for EIBRS and RRSBA. While technically they're not linked, absolutely nothing good can come of letting the guest see RRSBA without EIBRS. Nor a guest seeing EIBRS without IBRSB. Furthermore, we use this dependency to simplify the max derivation logic. The max policies gets RSBA and RRSBA unconditionally set (with the EIBRS dependency maybe hiding RRSBA). We can run any VM, even if it has been told "somewhere you might run, Retpoline isn't safe". The default policies are more complicated. A guest shouldn't see both bits, but it needs to see one if the current host suffers from any form of RSBA, and which bit it needs to see depends on whether eIBRS is visible or not. Therefore, the calculation must be performed after sanitise_featureset(). Signed-off-by: Andrew Cooper --- CC: Jan Beulich CC: Roger Pau Monné CC: Wei Liu v3: * Minor commit message adjustment. * Drop changes to recalculate_cpuid_policy(). Deferred to a later series. >>> With this dropped, with the title not saying "max/default", and with >>> the description also not mentioning "live" policies at all, I don't >>> think this patch is self-consistent (meaning in particular: leaving >>> aside the fact that there's no way right now to requests e.g. both >>> RSBA and RRSBA for a guest; aiui it is possible for Dom0). >>> >>> As you may imagine I'm also curious why you decided to drop this. >> Because when I tried doing levelling in Xapi, I remembered why I did it >> the way I did in v1, and why the v2 way was wrong. >> >> Xen cannot safely edit what the toolstack provides, so must not. > And this is the part I don't understand: Why can't we correct the > (EIBRS,RSBA,RRSBA) tuple to a combination that is "legal"? At least > as long as ... > >> Instead, failing the set_policy() call is an option, and is what we want >> to do longterm, > ... we aren't there. > >> but also happens to be wrong too in this case. An admin >> may know that a VM isn't using retpoline, and may need to migrate it >> anyway for a number of reasons, so any safety checks need to be in the >> toolstack, and need to be overrideable with something like --force. > Possibly leading to an inconsistent policy exposed to a guest? I > guess this may be the only option when we can't really resolve an > ambiguity, but that isn't the case here, is it? Wrong. Xen does not have any knowledge of other hosts the VM might migrate to. So while Xen can spot problem combinations *on this host*, which way to correct the problem combination depends on where the VM might migrate to. >>> I actually view this as two different levels: With a flawed policy, the >>> guest is liable to not work correctly at all. No point thinking about >>> it being able to migrate. With a fixed up policy it may fail to migrate, >>> but it'll at least work otherwise. >> If you get RSBA and/or RRSBA wrong, nothing is going to malfunction in >> the guest, even if you migrate it. >> >> The consequence of getting RSBA and/or RRSBA wrong is the guest *might* >> think retpoline is safe to use, and *might* end up vulnerable to >> speculative attacks on this or other hardware. > Isn't that some sort of "malfunction"? Perhaps, there's a difference between
Re: [PATCH v3 4/4] x86/cpu-policy: Derive RSBA/RRSBA for guest policies
On 15.06.2023 20:17, Andrew Cooper wrote: > On 15/06/2023 1:13 pm, Jan Beulich wrote: >> On 15.06.2023 12:41, Andrew Cooper wrote: >>> On 15/06/2023 9:30 am, Jan Beulich wrote: On 14.06.2023 20:12, Andrew Cooper wrote: > On 13/06/2023 10:59 am, Jan Beulich wrote: >> On 12.06.2023 18:13, Andrew Cooper wrote: >>> The RSBA bit, "RSB Alternative", means that the RSB may use alternative >>> predictors when empty. From a practical point of view, this mean >>> "Retpoline >>> not safe". >>> >>> Enhanced IBRS (officially IBRS_ALL in Intel's docs, previously >>> IBRS_ATT) is a >>> statement that IBRS is implemented in hardware (as opposed to the form >>> retrofitted to existing CPUs in microcode). >>> >>> The RRSBA bit, "Restricted-RSBA", is a combination of RSBA, and the >>> eIBRS >>> property that predictions are tagged with the mode in which they were >>> learnt. >>> Therefore, it means "when eIBRS is active, the RSB may fall back to >>> alternative predictors but restricted to the current prediction mode". >>> As >>> such, it's stronger statement than RSBA, but still means "Retpoline not >>> safe". >>> >>> CPUs are not expected to enumerate both RSBA and RRSBA. >>> >>> Add feature dependencies for EIBRS and RRSBA. While technically >>> they're not >>> linked, absolutely nothing good can come of letting the guest see RRSBA >>> without EIBRS. Nor a guest seeing EIBRS without IBRSB. Furthermore, >>> we use >>> this dependency to simplify the max derivation logic. >>> >>> The max policies gets RSBA and RRSBA unconditionally set (with the EIBRS >>> dependency maybe hiding RRSBA). We can run any VM, even if it has been >>> told >>> "somewhere you might run, Retpoline isn't safe". >>> >>> The default policies are more complicated. A guest shouldn't see both >>> bits, >>> but it needs to see one if the current host suffers from any form of >>> RSBA, and >>> which bit it needs to see depends on whether eIBRS is visible or not. >>> Therefore, the calculation must be performed after >>> sanitise_featureset(). >>> >>> Signed-off-by: Andrew Cooper >>> --- >>> CC: Jan Beulich >>> CC: Roger Pau Monné >>> CC: Wei Liu >>> >>> v3: >>> * Minor commit message adjustment. >>> * Drop changes to recalculate_cpuid_policy(). Deferred to a later >>> series. >> With this dropped, with the title not saying "max/default", and with >> the description also not mentioning "live" policies at all, I don't >> think this patch is self-consistent (meaning in particular: leaving >> aside the fact that there's no way right now to requests e.g. both >> RSBA and RRSBA for a guest; aiui it is possible for Dom0). >> >> As you may imagine I'm also curious why you decided to drop this. > Because when I tried doing levelling in Xapi, I remembered why I did it > the way I did in v1, and why the v2 way was wrong. > > Xen cannot safely edit what the toolstack provides, so must not. And this is the part I don't understand: Why can't we correct the (EIBRS,RSBA,RRSBA) tuple to a combination that is "legal"? At least as long as ... > Instead, failing the set_policy() call is an option, and is what we want > to do longterm, ... we aren't there. > but also happens to be wrong too in this case. An admin > may know that a VM isn't using retpoline, and may need to migrate it > anyway for a number of reasons, so any safety checks need to be in the > toolstack, and need to be overrideable with something like --force. Possibly leading to an inconsistent policy exposed to a guest? I guess this may be the only option when we can't really resolve an ambiguity, but that isn't the case here, is it? >>> Wrong. Xen does not have any knowledge of other hosts the VM might >>> migrate to. >>> >>> So while Xen can spot problem combinations *on this host*, which way to >>> correct the problem combination depends on where the VM might migrate to. >> I actually view this as two different levels: With a flawed policy, the >> guest is liable to not work correctly at all. No point thinking about >> it being able to migrate. With a fixed up policy it may fail to migrate, >> but it'll at least work otherwise. > > If you get RSBA and/or RRSBA wrong, nothing is going to malfunction in > the guest, even if you migrate it. > > The consequence of getting RSBA and/or RRSBA wrong is the guest *might* > think retpoline is safe to use, and *might* end up vulnerable to > speculative attacks on this or other hardware. Isn't that some sort of "malfunction"? > And the admin might know that they overrode the default settings and > forced the use of some other protection mechanism, so the guest is in > fact safe despite having wrong RSBA/RRSBA
Re: [PATCH v3 4/4] x86/cpu-policy: Derive RSBA/RRSBA for guest policies
On 15/06/2023 1:13 pm, Jan Beulich wrote: > On 15.06.2023 12:41, Andrew Cooper wrote: >> On 15/06/2023 9:30 am, Jan Beulich wrote: >>> On 14.06.2023 20:12, Andrew Cooper wrote: On 13/06/2023 10:59 am, Jan Beulich wrote: > On 12.06.2023 18:13, Andrew Cooper wrote: >> The RSBA bit, "RSB Alternative", means that the RSB may use alternative >> predictors when empty. From a practical point of view, this mean >> "Retpoline >> not safe". >> >> Enhanced IBRS (officially IBRS_ALL in Intel's docs, previously IBRS_ATT) >> is a >> statement that IBRS is implemented in hardware (as opposed to the form >> retrofitted to existing CPUs in microcode). >> >> The RRSBA bit, "Restricted-RSBA", is a combination of RSBA, and the eIBRS >> property that predictions are tagged with the mode in which they were >> learnt. >> Therefore, it means "when eIBRS is active, the RSB may fall back to >> alternative predictors but restricted to the current prediction mode". >> As >> such, it's stronger statement than RSBA, but still means "Retpoline not >> safe". >> >> CPUs are not expected to enumerate both RSBA and RRSBA. >> >> Add feature dependencies for EIBRS and RRSBA. While technically they're >> not >> linked, absolutely nothing good can come of letting the guest see RRSBA >> without EIBRS. Nor a guest seeing EIBRS without IBRSB. Furthermore, we >> use >> this dependency to simplify the max derivation logic. >> >> The max policies gets RSBA and RRSBA unconditionally set (with the EIBRS >> dependency maybe hiding RRSBA). We can run any VM, even if it has been >> told >> "somewhere you might run, Retpoline isn't safe". >> >> The default policies are more complicated. A guest shouldn't see both >> bits, >> but it needs to see one if the current host suffers from any form of >> RSBA, and >> which bit it needs to see depends on whether eIBRS is visible or not. >> Therefore, the calculation must be performed after sanitise_featureset(). >> >> Signed-off-by: Andrew Cooper >> --- >> CC: Jan Beulich >> CC: Roger Pau Monné >> CC: Wei Liu >> >> v3: >> * Minor commit message adjustment. >> * Drop changes to recalculate_cpuid_policy(). Deferred to a later >> series. > With this dropped, with the title not saying "max/default", and with > the description also not mentioning "live" policies at all, I don't > think this patch is self-consistent (meaning in particular: leaving > aside the fact that there's no way right now to requests e.g. both > RSBA and RRSBA for a guest; aiui it is possible for Dom0). > > As you may imagine I'm also curious why you decided to drop this. Because when I tried doing levelling in Xapi, I remembered why I did it the way I did in v1, and why the v2 way was wrong. Xen cannot safely edit what the toolstack provides, so must not. >>> And this is the part I don't understand: Why can't we correct the >>> (EIBRS,RSBA,RRSBA) tuple to a combination that is "legal"? At least >>> as long as ... >>> Instead, failing the set_policy() call is an option, and is what we want to do longterm, >>> ... we aren't there. >>> but also happens to be wrong too in this case. An admin may know that a VM isn't using retpoline, and may need to migrate it anyway for a number of reasons, so any safety checks need to be in the toolstack, and need to be overrideable with something like --force. >>> Possibly leading to an inconsistent policy exposed to a guest? I >>> guess this may be the only option when we can't really resolve an >>> ambiguity, but that isn't the case here, is it? >> Wrong. Xen does not have any knowledge of other hosts the VM might >> migrate to. >> >> So while Xen can spot problem combinations *on this host*, which way to >> correct the problem combination depends on where the VM might migrate to. > I actually view this as two different levels: With a flawed policy, the > guest is liable to not work correctly at all. No point thinking about > it being able to migrate. With a fixed up policy it may fail to migrate, > but it'll at least work otherwise. If you get RSBA and/or RRSBA wrong, nothing is going to malfunction in the guest, even if you migrate it. The consequence of getting RSBA and/or RRSBA wrong is the guest *might* think retpoline is safe to use, and *might* end up vulnerable to speculative attacks on this or other hardware. And the admin might know that they overrode the default settings and forced the use of some other protection mechanism, so the guest is in fact safe despite having wrong RSBA/RRSBA settings. I don't know how to put it any more plainly. Xen *does not* have the information necessary to make a safety judgement in this matter. Only the toolstack (as a proxy for the admin) has the necessary
Re: [PATCH v3 4/4] x86/cpu-policy: Derive RSBA/RRSBA for guest policies
On 15.06.2023 12:41, Andrew Cooper wrote: > On 15/06/2023 9:30 am, Jan Beulich wrote: >> On 14.06.2023 20:12, Andrew Cooper wrote: >>> On 13/06/2023 10:59 am, Jan Beulich wrote: On 12.06.2023 18:13, Andrew Cooper wrote: > The RSBA bit, "RSB Alternative", means that the RSB may use alternative > predictors when empty. From a practical point of view, this mean > "Retpoline > not safe". > > Enhanced IBRS (officially IBRS_ALL in Intel's docs, previously IBRS_ATT) > is a > statement that IBRS is implemented in hardware (as opposed to the form > retrofitted to existing CPUs in microcode). > > The RRSBA bit, "Restricted-RSBA", is a combination of RSBA, and the eIBRS > property that predictions are tagged with the mode in which they were > learnt. > Therefore, it means "when eIBRS is active, the RSB may fall back to > alternative predictors but restricted to the current prediction mode". As > such, it's stronger statement than RSBA, but still means "Retpoline not > safe". > > CPUs are not expected to enumerate both RSBA and RRSBA. > > Add feature dependencies for EIBRS and RRSBA. While technically they're > not > linked, absolutely nothing good can come of letting the guest see RRSBA > without EIBRS. Nor a guest seeing EIBRS without IBRSB. Furthermore, we > use > this dependency to simplify the max derivation logic. > > The max policies gets RSBA and RRSBA unconditionally set (with the EIBRS > dependency maybe hiding RRSBA). We can run any VM, even if it has been > told > "somewhere you might run, Retpoline isn't safe". > > The default policies are more complicated. A guest shouldn't see both > bits, > but it needs to see one if the current host suffers from any form of > RSBA, and > which bit it needs to see depends on whether eIBRS is visible or not. > Therefore, the calculation must be performed after sanitise_featureset(). > > Signed-off-by: Andrew Cooper > --- > CC: Jan Beulich > CC: Roger Pau Monné > CC: Wei Liu > > v3: > * Minor commit message adjustment. > * Drop changes to recalculate_cpuid_policy(). Deferred to a later > series. With this dropped, with the title not saying "max/default", and with the description also not mentioning "live" policies at all, I don't think this patch is self-consistent (meaning in particular: leaving aside the fact that there's no way right now to requests e.g. both RSBA and RRSBA for a guest; aiui it is possible for Dom0). As you may imagine I'm also curious why you decided to drop this. >>> Because when I tried doing levelling in Xapi, I remembered why I did it >>> the way I did in v1, and why the v2 way was wrong. >>> >>> Xen cannot safely edit what the toolstack provides, so must not. >> And this is the part I don't understand: Why can't we correct the >> (EIBRS,RSBA,RRSBA) tuple to a combination that is "legal"? At least >> as long as ... >> >>> Instead, failing the set_policy() call is an option, and is what we want >>> to do longterm, >> ... we aren't there. >> >>> but also happens to be wrong too in this case. An admin >>> may know that a VM isn't using retpoline, and may need to migrate it >>> anyway for a number of reasons, so any safety checks need to be in the >>> toolstack, and need to be overrideable with something like --force. >> Possibly leading to an inconsistent policy exposed to a guest? I >> guess this may be the only option when we can't really resolve an >> ambiguity, but that isn't the case here, is it? > > Wrong. Xen does not have any knowledge of other hosts the VM might > migrate to. > > So while Xen can spot problem combinations *on this host*, which way to > correct the problem combination depends on where the VM might migrate to. I actually view this as two different levels: With a flawed policy, the guest is liable to not work correctly at all. No point thinking about it being able to migrate. With a fixed up policy it may fail to migrate, but it'll at least work otherwise. Jan
Re: [PATCH v3 4/4] x86/cpu-policy: Derive RSBA/RRSBA for guest policies
On 15/06/2023 9:30 am, Jan Beulich wrote: > On 14.06.2023 20:12, Andrew Cooper wrote: >> On 13/06/2023 10:59 am, Jan Beulich wrote: >>> On 12.06.2023 18:13, Andrew Cooper wrote: The RSBA bit, "RSB Alternative", means that the RSB may use alternative predictors when empty. From a practical point of view, this mean "Retpoline not safe". Enhanced IBRS (officially IBRS_ALL in Intel's docs, previously IBRS_ATT) is a statement that IBRS is implemented in hardware (as opposed to the form retrofitted to existing CPUs in microcode). The RRSBA bit, "Restricted-RSBA", is a combination of RSBA, and the eIBRS property that predictions are tagged with the mode in which they were learnt. Therefore, it means "when eIBRS is active, the RSB may fall back to alternative predictors but restricted to the current prediction mode". As such, it's stronger statement than RSBA, but still means "Retpoline not safe". CPUs are not expected to enumerate both RSBA and RRSBA. Add feature dependencies for EIBRS and RRSBA. While technically they're not linked, absolutely nothing good can come of letting the guest see RRSBA without EIBRS. Nor a guest seeing EIBRS without IBRSB. Furthermore, we use this dependency to simplify the max derivation logic. The max policies gets RSBA and RRSBA unconditionally set (with the EIBRS dependency maybe hiding RRSBA). We can run any VM, even if it has been told "somewhere you might run, Retpoline isn't safe". The default policies are more complicated. A guest shouldn't see both bits, but it needs to see one if the current host suffers from any form of RSBA, and which bit it needs to see depends on whether eIBRS is visible or not. Therefore, the calculation must be performed after sanitise_featureset(). Signed-off-by: Andrew Cooper --- CC: Jan Beulich CC: Roger Pau Monné CC: Wei Liu v3: * Minor commit message adjustment. * Drop changes to recalculate_cpuid_policy(). Deferred to a later series. >>> With this dropped, with the title not saying "max/default", and with >>> the description also not mentioning "live" policies at all, I don't >>> think this patch is self-consistent (meaning in particular: leaving >>> aside the fact that there's no way right now to requests e.g. both >>> RSBA and RRSBA for a guest; aiui it is possible for Dom0). >>> >>> As you may imagine I'm also curious why you decided to drop this. >> Because when I tried doing levelling in Xapi, I remembered why I did it >> the way I did in v1, and why the v2 way was wrong. >> >> Xen cannot safely edit what the toolstack provides, so must not. > And this is the part I don't understand: Why can't we correct the > (EIBRS,RSBA,RRSBA) tuple to a combination that is "legal"? At least > as long as ... > >> Instead, failing the set_policy() call is an option, and is what we want >> to do longterm, > ... we aren't there. > >> but also happens to be wrong too in this case. An admin >> may know that a VM isn't using retpoline, and may need to migrate it >> anyway for a number of reasons, so any safety checks need to be in the >> toolstack, and need to be overrideable with something like --force. > Possibly leading to an inconsistent policy exposed to a guest? I > guess this may be the only option when we can't really resolve an > ambiguity, but that isn't the case here, is it? Wrong. Xen does not have any knowledge of other hosts the VM might migrate to. So while Xen can spot problem combinations *on this host*, which way to correct the problem combination depends on where the VM might migrate to. Xen cannot safely correct a problem combination even if you don't wish to allow the admin the ability to override the safety check. > >> I don't really associate "derive policies" with anything other than the >> system policies. Domain construction isn't any kind of derivation - >> it's simply doing what the toolstack asks. > Hmm, I see. To me, since we do certain adjustments, "derive" still > fits there as well. But I'm not going to insist on a subject > adjustment then, given that imo both ways of looking at things make > some sense. It's a problem that Xen ever made adjustments behind the toolstack's back, and this decade of technical debt has been extremely difficult to address. I guess I still view it in terms of the end properties, not the intermediate mess. ~Andrew
Re: [PATCH v3 4/4] x86/cpu-policy: Derive RSBA/RRSBA for guest policies
On 14.06.2023 20:12, Andrew Cooper wrote: > On 13/06/2023 10:59 am, Jan Beulich wrote: >> On 12.06.2023 18:13, Andrew Cooper wrote: >>> The RSBA bit, "RSB Alternative", means that the RSB may use alternative >>> predictors when empty. From a practical point of view, this mean "Retpoline >>> not safe". >>> >>> Enhanced IBRS (officially IBRS_ALL in Intel's docs, previously IBRS_ATT) is >>> a >>> statement that IBRS is implemented in hardware (as opposed to the form >>> retrofitted to existing CPUs in microcode). >>> >>> The RRSBA bit, "Restricted-RSBA", is a combination of RSBA, and the eIBRS >>> property that predictions are tagged with the mode in which they were >>> learnt. >>> Therefore, it means "when eIBRS is active, the RSB may fall back to >>> alternative predictors but restricted to the current prediction mode". As >>> such, it's stronger statement than RSBA, but still means "Retpoline not >>> safe". >>> >>> CPUs are not expected to enumerate both RSBA and RRSBA. >>> >>> Add feature dependencies for EIBRS and RRSBA. While technically they're not >>> linked, absolutely nothing good can come of letting the guest see RRSBA >>> without EIBRS. Nor a guest seeing EIBRS without IBRSB. Furthermore, we use >>> this dependency to simplify the max derivation logic. >>> >>> The max policies gets RSBA and RRSBA unconditionally set (with the EIBRS >>> dependency maybe hiding RRSBA). We can run any VM, even if it has been told >>> "somewhere you might run, Retpoline isn't safe". >>> >>> The default policies are more complicated. A guest shouldn't see both bits, >>> but it needs to see one if the current host suffers from any form of RSBA, >>> and >>> which bit it needs to see depends on whether eIBRS is visible or not. >>> Therefore, the calculation must be performed after sanitise_featureset(). >>> >>> Signed-off-by: Andrew Cooper >>> --- >>> CC: Jan Beulich >>> CC: Roger Pau Monné >>> CC: Wei Liu >>> >>> v3: >>> * Minor commit message adjustment. >>> * Drop changes to recalculate_cpuid_policy(). Deferred to a later series. >> With this dropped, with the title not saying "max/default", and with >> the description also not mentioning "live" policies at all, I don't >> think this patch is self-consistent (meaning in particular: leaving >> aside the fact that there's no way right now to requests e.g. both >> RSBA and RRSBA for a guest; aiui it is possible for Dom0). >> >> As you may imagine I'm also curious why you decided to drop this. > > Because when I tried doing levelling in Xapi, I remembered why I did it > the way I did in v1, and why the v2 way was wrong. > > Xen cannot safely edit what the toolstack provides, so must not. And this is the part I don't understand: Why can't we correct the (EIBRS,RSBA,RRSBA) tuple to a combination that is "legal"? At least as long as ... > Instead, failing the set_policy() call is an option, and is what we want > to do longterm, ... we aren't there. > but also happens to be wrong too in this case. An admin > may know that a VM isn't using retpoline, and may need to migrate it > anyway for a number of reasons, so any safety checks need to be in the > toolstack, and need to be overrideable with something like --force. Possibly leading to an inconsistent policy exposed to a guest? I guess this may be the only option when we can't really resolve an ambiguity, but that isn't the case here, is it? > I don't really associate "derive policies" with anything other than the > system policies. Domain construction isn't any kind of derivation - > it's simply doing what the toolstack asks. Hmm, I see. To me, since we do certain adjustments, "derive" still fits there as well. But I'm not going to insist on a subject adjustment then, given that imo both ways of looking at things make some sense. Jan
Re: [PATCH v3 4/4] x86/cpu-policy: Derive RSBA/RRSBA for guest policies
On 13/06/2023 10:59 am, Jan Beulich wrote: > On 12.06.2023 18:13, Andrew Cooper wrote: >> The RSBA bit, "RSB Alternative", means that the RSB may use alternative >> predictors when empty. From a practical point of view, this mean "Retpoline >> not safe". >> >> Enhanced IBRS (officially IBRS_ALL in Intel's docs, previously IBRS_ATT) is a >> statement that IBRS is implemented in hardware (as opposed to the form >> retrofitted to existing CPUs in microcode). >> >> The RRSBA bit, "Restricted-RSBA", is a combination of RSBA, and the eIBRS >> property that predictions are tagged with the mode in which they were learnt. >> Therefore, it means "when eIBRS is active, the RSB may fall back to >> alternative predictors but restricted to the current prediction mode". As >> such, it's stronger statement than RSBA, but still means "Retpoline not >> safe". >> >> CPUs are not expected to enumerate both RSBA and RRSBA. >> >> Add feature dependencies for EIBRS and RRSBA. While technically they're not >> linked, absolutely nothing good can come of letting the guest see RRSBA >> without EIBRS. Nor a guest seeing EIBRS without IBRSB. Furthermore, we use >> this dependency to simplify the max derivation logic. >> >> The max policies gets RSBA and RRSBA unconditionally set (with the EIBRS >> dependency maybe hiding RRSBA). We can run any VM, even if it has been told >> "somewhere you might run, Retpoline isn't safe". >> >> The default policies are more complicated. A guest shouldn't see both bits, >> but it needs to see one if the current host suffers from any form of RSBA, >> and >> which bit it needs to see depends on whether eIBRS is visible or not. >> Therefore, the calculation must be performed after sanitise_featureset(). >> >> Signed-off-by: Andrew Cooper >> --- >> CC: Jan Beulich >> CC: Roger Pau Monné >> CC: Wei Liu >> >> v3: >> * Minor commit message adjustment. >> * Drop changes to recalculate_cpuid_policy(). Deferred to a later series. > With this dropped, with the title not saying "max/default", and with > the description also not mentioning "live" policies at all, I don't > think this patch is self-consistent (meaning in particular: leaving > aside the fact that there's no way right now to requests e.g. both > RSBA and RRSBA for a guest; aiui it is possible for Dom0). > > As you may imagine I'm also curious why you decided to drop this. Because when I tried doing levelling in Xapi, I remembered why I did it the way I did in v1, and why the v2 way was wrong. Xen cannot safely edit what the toolstack provides, so must not. Instead, failing the set_policy() call is an option, and is what we want to do longterm, but also happens to be wrong too in this case. An admin may know that a VM isn't using retpoline, and may need to migrate it anyway for a number of reasons, so any safety checks need to be in the toolstack, and need to be overrideable with something like --force. I don't really associate "derive policies" with anything other than the system policies. Domain construction isn't any kind of derivation - it's simply doing what the toolstack asks. ~Andrew
Re: [PATCH v3 4/4] x86/cpu-policy: Derive RSBA/RRSBA for guest policies
On 12.06.2023 18:13, Andrew Cooper wrote: > The RSBA bit, "RSB Alternative", means that the RSB may use alternative > predictors when empty. From a practical point of view, this mean "Retpoline > not safe". > > Enhanced IBRS (officially IBRS_ALL in Intel's docs, previously IBRS_ATT) is a > statement that IBRS is implemented in hardware (as opposed to the form > retrofitted to existing CPUs in microcode). > > The RRSBA bit, "Restricted-RSBA", is a combination of RSBA, and the eIBRS > property that predictions are tagged with the mode in which they were learnt. > Therefore, it means "when eIBRS is active, the RSB may fall back to > alternative predictors but restricted to the current prediction mode". As > such, it's stronger statement than RSBA, but still means "Retpoline not safe". > > CPUs are not expected to enumerate both RSBA and RRSBA. > > Add feature dependencies for EIBRS and RRSBA. While technically they're not > linked, absolutely nothing good can come of letting the guest see RRSBA > without EIBRS. Nor a guest seeing EIBRS without IBRSB. Furthermore, we use > this dependency to simplify the max derivation logic. > > The max policies gets RSBA and RRSBA unconditionally set (with the EIBRS > dependency maybe hiding RRSBA). We can run any VM, even if it has been told > "somewhere you might run, Retpoline isn't safe". > > The default policies are more complicated. A guest shouldn't see both bits, > but it needs to see one if the current host suffers from any form of RSBA, and > which bit it needs to see depends on whether eIBRS is visible or not. > Therefore, the calculation must be performed after sanitise_featureset(). > > Signed-off-by: Andrew Cooper > --- > CC: Jan Beulich > CC: Roger Pau Monné > CC: Wei Liu > > v3: > * Minor commit message adjustment. > * Drop changes to recalculate_cpuid_policy(). Deferred to a later series. With this dropped, with the title not saying "max/default", and with the description also not mentioning "live" policies at all, I don't think this patch is self-consistent (meaning in particular: leaving aside the fact that there's no way right now to requests e.g. both RSBA and RRSBA for a guest; aiui it is possible for Dom0). As you may imagine I'm also curious why you decided to drop this. Jan
[PATCH v3 4/4] x86/cpu-policy: Derive RSBA/RRSBA for guest policies
The RSBA bit, "RSB Alternative", means that the RSB may use alternative
predictors when empty. From a practical point of view, this mean "Retpoline
not safe".
Enhanced IBRS (officially IBRS_ALL in Intel's docs, previously IBRS_ATT) is a
statement that IBRS is implemented in hardware (as opposed to the form
retrofitted to existing CPUs in microcode).
The RRSBA bit, "Restricted-RSBA", is a combination of RSBA, and the eIBRS
property that predictions are tagged with the mode in which they were learnt.
Therefore, it means "when eIBRS is active, the RSB may fall back to
alternative predictors but restricted to the current prediction mode". As
such, it's stronger statement than RSBA, but still means "Retpoline not safe".
CPUs are not expected to enumerate both RSBA and RRSBA.
Add feature dependencies for EIBRS and RRSBA. While technically they're not
linked, absolutely nothing good can come of letting the guest see RRSBA
without EIBRS. Nor a guest seeing EIBRS without IBRSB. Furthermore, we use
this dependency to simplify the max derivation logic.
The max policies gets RSBA and RRSBA unconditionally set (with the EIBRS
dependency maybe hiding RRSBA). We can run any VM, even if it has been told
"somewhere you might run, Retpoline isn't safe".
The default policies are more complicated. A guest shouldn't see both bits,
but it needs to see one if the current host suffers from any form of RSBA, and
which bit it needs to see depends on whether eIBRS is visible or not.
Therefore, the calculation must be performed after sanitise_featureset().
Signed-off-by: Andrew Cooper
---
CC: Jan Beulich
CC: Roger Pau Monné
CC: Wei Liu
v3:
* Minor commit message adjustment.
* Drop changes to recalculate_cpuid_policy(). Deferred to a later series.
v2:
* Expand/adjust the comment for the max features.
* Rewrite the default feature derivation in light of new information.
* Fix up in recalculate_cpuid_policy() too.
---
xen/arch/x86/cpu-policy.c | 39 +
xen/include/public/arch-x86/cpufeatureset.h | 4 +--
xen/tools/gen-cpuid.py | 5 ++-
3 files changed, 45 insertions(+), 3 deletions(-)
diff --git a/xen/arch/x86/cpu-policy.c b/xen/arch/x86/cpu-policy.c
index ee256ff5a137..cde7f7605c28 100644
--- a/xen/arch/x86/cpu-policy.c
+++ b/xen/arch/x86/cpu-policy.c
@@ -423,8 +423,17 @@ static void __init
guest_common_max_feature_adjustments(uint32_t *fs)
* Retpoline not safe)", so these need to be visible to a guest in all
* cases, even when it's only some other server in the pool which
* suffers the identified behaviour.
+ *
+ * We can always run any VM which has previously (or will
+ * subsequently) run on hardware where Retpoline is not safe.
+ * Note:
+ * - The dependency logic may hide RRSBA for other reasons.
+ * - The max policy does not contitute a sensible configuration to
+ *run a guest in.
*/
__set_bit(X86_FEATURE_ARCH_CAPS, fs);
+__set_bit(X86_FEATURE_RSBA, fs);
+__set_bit(X86_FEATURE_RRSBA, fs);
}
}
@@ -532,6 +541,21 @@ static void __init calculate_pv_def_policy(void)
guest_common_default_feature_adjustments(fs);
sanitise_featureset(fs);
+
+/*
+ * If the host suffers from RSBA of any form, and the guest can see
+ * MSR_ARCH_CAPS, reflect the appropriate RSBA/RRSBA property to the guest
+ * depending on the visibility of eIBRS.
+ */
+if ( test_bit(X86_FEATURE_ARCH_CAPS, fs) &&
+ (cpu_has_rsba || cpu_has_rrsba) )
+{
+bool eibrs = test_bit(X86_FEATURE_EIBRS, fs);
+
+__set_bit(eibrs ? X86_FEATURE_RRSBA
+: X86_FEATURE_RSBA, fs);
+}
+
x86_cpu_featureset_to_policy(fs, p);
recalculate_xstate(p);
}
@@ -664,6 +688,21 @@ static void __init calculate_hvm_def_policy(void)
__set_bit(X86_FEATURE_VIRT_SSBD, fs);
sanitise_featureset(fs);
+
+/*
+ * If the host suffers from RSBA of any form, and the guest can see
+ * MSR_ARCH_CAPS, reflect the appropriate RSBA/RRSBA property to the guest
+ * depending on the visibility of eIBRS.
+ */
+if ( test_bit(X86_FEATURE_ARCH_CAPS, fs) &&
+ (cpu_has_rsba || cpu_has_rrsba) )
+{
+bool eibrs = test_bit(X86_FEATURE_EIBRS, fs);
+
+__set_bit(eibrs ? X86_FEATURE_RRSBA
+: X86_FEATURE_RSBA, fs);
+}
+
x86_cpu_featureset_to_policy(fs, p);
recalculate_xstate(p);
}
diff --git a/xen/include/public/arch-x86/cpufeatureset.h
b/xen/include/public/arch-x86/cpufeatureset.h
index ea779c29879e..ce7407d6a10c 100644
--- a/xen/include/public/arch-x86/cpufeatureset.h
+++ b/xen/include/public/arch-x86/cpufeatureset.h
@@ -311,7 +311,7 @@ XEN_CPUFEATURE(CET_SSS,15*32+18) /* CET
Supervisor Shadow Stacks s
/* Intel-defined CPU features, MSR_ARCH_CAPS 0x10a.eax, word 16 */
XEN_CPUFEATURE(RDCL_NO,
