Re: [PATCH v4 0/3] Allow user space to restrict and augment MSR emulation
On Wed, Aug 19, 2020 at 3:09 PM Jim Mattson wrote: > > On Wed, Aug 19, 2020 at 2:46 PM Graf (AWS), Alexander wrote: > > > Special MSRs like EFER also irritate me a bit. We can't really trap on them > > - most code paths just know they're handled in kernel. Maybe I'll add some > > sanity checks as well... > > Why can't we intercept EFER? Some MSRs (IA32_GSBASE comes to mind) can't be completely handled by userspace, even if we do intercept RDMSR and WRMSR. The EFER.LMA bit also falls into that category, though the rest of the register isn't a problem (and EFER.LMA is always derivable). Is that what you meant?
Re: [PATCH v4 0/3] Allow user space to restrict and augment MSR emulation
On Wed, Aug 19, 2020 at 2:46 PM Graf (AWS), Alexander wrote: > Special MSRs like EFER also irritate me a bit. We can't really trap on them - > most code paths just know they're handled in kernel. Maybe I'll add some > sanity checks as well... Why can't we intercept EFER?
Re: [PATCH v4 0/3] Allow user space to restrict and augment MSR emulation
> Am 19.08.2020 um 23:27 schrieb Jim Mattson : > >> On Mon, Aug 3, 2020 at 2:14 PM Alexander Graf wrote: >> >> While tying to add support for the MSR_CORE_THREAD_COUNT MSR in KVM, >> I realized that we were still in a world where user space has no control >> over what happens with MSR emulation in KVM. >> >> That is bad for multiple reasons. In my case, I wanted to emulate the >> MSR in user space, because it's a CPU specific register that does not >> exist on older CPUs and that really only contains informational data that >> is on the package level, so it's a natural fit for user space to provide >> it. >> >> However, it is also bad on a platform compatibility level. Currrently, >> KVM has no way to expose different MSRs based on the selected target CPU >> type. >> >> This patch set introduces a way for user space to indicate to KVM which >> MSRs should be handled in kernel space. With that, we can solve part of >> the platform compatibility story. Or at least we can not handle AMD specific >> MSRs on an Intel platform and vice versa. >> >> In addition, it introduces a way for user space to get into the loop >> when an MSR access would generate a #GP fault, such as when KVM finds an >> MSR that is not handled by the in-kernel MSR emulation or when the guest >> is trying to access reserved registers. >> >> In combination with the allow list, the user space trapping allows us >> to emulate arbitrary MSRs in user space, paving the way for target CPU >> specific MSR implementations from user space. > > This is somewhat misleading. If you don't modify the MSR permission > bitmaps, as Aaron has done, you cannot emulate *arbitrary* MSRs in > userspace. You can only emulate MSRs that kvm is going to intercept. > Moreover, since the set of intercepted MSRs evolves over time, this > isn't a stable API. Yeah, I wrote up a patch today to do the passthrough bitmap masking dynamically, partly based on Aaron's patches. I have not verified it yet though and will need to clean up SVM as well. I'll see how far I get with it for the rest of the week. Special MSRs like EFER also irritate me a bit. We can't really trap on them - most code paths just know they're handled in kernel. Maybe I'll add some sanity checks as well... Alex Amazon Development Center Germany GmbH Krausenstr. 38 10117 Berlin Geschaeftsfuehrung: Christian Schlaeger, Jonathan Weiss Eingetragen am Amtsgericht Charlottenburg unter HRB 149173 B Sitz: Berlin Ust-ID: DE 289 237 879
Re: [PATCH v4 0/3] Allow user space to restrict and augment MSR emulation
On Mon, Aug 3, 2020 at 2:14 PM Alexander Graf wrote: > > While tying to add support for the MSR_CORE_THREAD_COUNT MSR in KVM, > I realized that we were still in a world where user space has no control > over what happens with MSR emulation in KVM. > > That is bad for multiple reasons. In my case, I wanted to emulate the > MSR in user space, because it's a CPU specific register that does not > exist on older CPUs and that really only contains informational data that > is on the package level, so it's a natural fit for user space to provide > it. > > However, it is also bad on a platform compatibility level. Currrently, > KVM has no way to expose different MSRs based on the selected target CPU > type. > > This patch set introduces a way for user space to indicate to KVM which > MSRs should be handled in kernel space. With that, we can solve part of > the platform compatibility story. Or at least we can not handle AMD specific > MSRs on an Intel platform and vice versa. > > In addition, it introduces a way for user space to get into the loop > when an MSR access would generate a #GP fault, such as when KVM finds an > MSR that is not handled by the in-kernel MSR emulation or when the guest > is trying to access reserved registers. > > In combination with the allow list, the user space trapping allows us > to emulate arbitrary MSRs in user space, paving the way for target CPU > specific MSR implementations from user space. This is somewhat misleading. If you don't modify the MSR permission bitmaps, as Aaron has done, you cannot emulate *arbitrary* MSRs in userspace. You can only emulate MSRs that kvm is going to intercept. Moreover, since the set of intercepted MSRs evolves over time, this isn't a stable API. > v1 -> v2: > > - s/ETRAP_TO_USER_SPACE/ENOENT/g > - deflect all #GP injection events to user space, not just unknown MSRs. > That was we can also deflect allowlist errors later > - fix emulator case > - new patch: KVM: x86: Introduce allow list for MSR emulation > - new patch: KVM: selftests: Add test for user space MSR handling > > v2 -> v3: > > - return r if r == X86EMUL_IO_NEEDED > - s/KVM_EXIT_RDMSR/KVM_EXIT_X86_RDMSR/g > - s/KVM_EXIT_WRMSR/KVM_EXIT_X86_WRMSR/g > - Use complete_userspace_io logic instead of reply field > - Simplify trapping code > - document flags for KVM_X86_ADD_MSR_ALLOWLIST > - generalize exit path, always unlock when returning > - s/KVM_CAP_ADD_MSR_ALLOWLIST/KVM_CAP_X86_MSR_ALLOWLIST/g > - Add KVM_X86_CLEAR_MSR_ALLOWLIST > - Add test to clear whitelist > - Adjust to reply-less API > - Fix asserts > - Actually trap on MSR_IA32_POWER_CTL writes > > v3 -> v4: > > - Mention exit reasons in re-enter mandatory section of API documentation > - Clear padding bytes > - Generalize get/set deflect functions > - Remove redundant pending_user_msr field > - lock allow check and clearing > - free bitmaps on clear > > Alexander Graf (3): > KVM: x86: Deflect unknown MSR accesses to user space > KVM: x86: Introduce allow list for MSR emulation > KVM: selftests: Add test for user space MSR handling > > Documentation/virt/kvm/api.rst| 157 ++- > arch/x86/include/asm/kvm_host.h | 13 + > arch/x86/include/uapi/asm/kvm.h | 15 + > arch/x86/kvm/emulate.c| 18 +- > arch/x86/kvm/x86.c| 259 +- > include/trace/events/kvm.h| 2 +- > include/uapi/linux/kvm.h | 15 + > tools/testing/selftests/kvm/Makefile | 1 + > .../selftests/kvm/x86_64/user_msr_test.c | 221 +++ > 9 files changed, 692 insertions(+), 9 deletions(-) > create mode 100644 tools/testing/selftests/kvm/x86_64/user_msr_test.c > > -- > 2.17.1 > > > > > Amazon Development Center Germany GmbH > Krausenstr. 38 > 10117 Berlin > Geschaeftsfuehrung: Christian Schlaeger, Jonathan Weiss > Eingetragen am Amtsgericht Charlottenburg unter HRB 149173 B > Sitz: Berlin > Ust-ID: DE 289 237 879 > > >
[PATCH v4 0/3] Allow user space to restrict and augment MSR emulation
While tying to add support for the MSR_CORE_THREAD_COUNT MSR in KVM, I realized that we were still in a world where user space has no control over what happens with MSR emulation in KVM. That is bad for multiple reasons. In my case, I wanted to emulate the MSR in user space, because it's a CPU specific register that does not exist on older CPUs and that really only contains informational data that is on the package level, so it's a natural fit for user space to provide it. However, it is also bad on a platform compatibility level. Currrently, KVM has no way to expose different MSRs based on the selected target CPU type. This patch set introduces a way for user space to indicate to KVM which MSRs should be handled in kernel space. With that, we can solve part of the platform compatibility story. Or at least we can not handle AMD specific MSRs on an Intel platform and vice versa. In addition, it introduces a way for user space to get into the loop when an MSR access would generate a #GP fault, such as when KVM finds an MSR that is not handled by the in-kernel MSR emulation or when the guest is trying to access reserved registers. In combination with the allow list, the user space trapping allows us to emulate arbitrary MSRs in user space, paving the way for target CPU specific MSR implementations from user space. v1 -> v2: - s/ETRAP_TO_USER_SPACE/ENOENT/g - deflect all #GP injection events to user space, not just unknown MSRs. That was we can also deflect allowlist errors later - fix emulator case - new patch: KVM: x86: Introduce allow list for MSR emulation - new patch: KVM: selftests: Add test for user space MSR handling v2 -> v3: - return r if r == X86EMUL_IO_NEEDED - s/KVM_EXIT_RDMSR/KVM_EXIT_X86_RDMSR/g - s/KVM_EXIT_WRMSR/KVM_EXIT_X86_WRMSR/g - Use complete_userspace_io logic instead of reply field - Simplify trapping code - document flags for KVM_X86_ADD_MSR_ALLOWLIST - generalize exit path, always unlock when returning - s/KVM_CAP_ADD_MSR_ALLOWLIST/KVM_CAP_X86_MSR_ALLOWLIST/g - Add KVM_X86_CLEAR_MSR_ALLOWLIST - Add test to clear whitelist - Adjust to reply-less API - Fix asserts - Actually trap on MSR_IA32_POWER_CTL writes v3 -> v4: - Mention exit reasons in re-enter mandatory section of API documentation - Clear padding bytes - Generalize get/set deflect functions - Remove redundant pending_user_msr field - lock allow check and clearing - free bitmaps on clear Alexander Graf (3): KVM: x86: Deflect unknown MSR accesses to user space KVM: x86: Introduce allow list for MSR emulation KVM: selftests: Add test for user space MSR handling Documentation/virt/kvm/api.rst| 157 ++- arch/x86/include/asm/kvm_host.h | 13 + arch/x86/include/uapi/asm/kvm.h | 15 + arch/x86/kvm/emulate.c| 18 +- arch/x86/kvm/x86.c| 259 +- include/trace/events/kvm.h| 2 +- include/uapi/linux/kvm.h | 15 + tools/testing/selftests/kvm/Makefile | 1 + .../selftests/kvm/x86_64/user_msr_test.c | 221 +++ 9 files changed, 692 insertions(+), 9 deletions(-) create mode 100644 tools/testing/selftests/kvm/x86_64/user_msr_test.c -- 2.17.1 Amazon Development Center Germany GmbH Krausenstr. 38 10117 Berlin Geschaeftsfuehrung: Christian Schlaeger, Jonathan Weiss Eingetragen am Amtsgericht Charlottenburg unter HRB 149173 B Sitz: Berlin Ust-ID: DE 289 237 879
