Unmap KVM-created VMAs for KVM-internal memslots when the associated memslot is freed instead of unmapping via x86's internal API. This will allow deleting/freeing all memslots before kvm_arch_destroy_vm() without leaking VMAs, and in general is a net positive as it makes it far less likely that KVM will leak VMAs, e.g. in the unlikely scenario some other flow deletes KVM-internal memslots without going through __x86_set_memory_region().
Free the VMA only when deleting the memslot from the "last" address space, where "last" is the highest numbered address space. KVM currently uses a single VMA for both non-SMM and SMM memslots, and deletes the memslots in ascending numerical order, i.e. deletes the SMM memslot last, if SMM is enabled. Freeing the VMA on the non-SMM memslot would technically regress inhibiting the APIC access page, which is the only case where KVM deletes an internal memslot on a live VM, as accessing the SMM memslot after the VMA is removed would result in KVM_RUN failing with -EFAULT. Add a TODO to call out that unmapping only the "last" memslot is a hack, along with a hint/suggestion for how the hack can be removed. CC: James Houghton <[email protected]> Signed-off-by: Sean Christopherson <[email protected]> --- arch/x86/kvm/x86.c | 39 +++++++++++++++++---------------------- 1 file changed, 17 insertions(+), 22 deletions(-) diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index cb8b60da5b11..489631686ba6 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -10018,7 +10018,7 @@ int kvm_arch_init_vm(struct kvm *kvm, unsigned long type) * @size > 0 to install a new slot, while @size == 0 to uninstall a * slot. The return code can be one of the following: * - * HVA: on success (uninstall will return a bogus HVA) + * HVA: on success (uninstall will return a NULL HVA) * -errno: on error * * The caller should always use IS_ERR() to check the return value @@ -10031,10 +10031,10 @@ int kvm_arch_init_vm(struct kvm *kvm, unsigned long type) void __user * __x86_set_memory_region(struct kvm *kvm, int id, gpa_t gpa, u32 size) { - int i, r; - unsigned long hva, old_npages; struct kvm_memslots *slots = kvm_memslots(kvm); struct kvm_memory_slot *slot; + unsigned long hva; + int i, r; lockdep_assert_held(&kvm->slots_lock); @@ -10058,8 +10058,7 @@ void __user * __x86_set_memory_region(struct kvm *kvm, int id, gpa_t gpa, if (!slot) return NULL; - old_npages = slot->npages; - hva = slot->userspace_addr; + hva = 0; } for (i = 0; i < kvm_arch_nr_memslot_as_ids(kvm); i++) { @@ -10075,9 +10074,6 @@ void __user * __x86_set_memory_region(struct kvm *kvm, int id, gpa_t gpa, return ERR_PTR_USR(r); } - if (!size) - vm_munmap(hva, old_npages * PAGE_SIZE); - return (void __user *)hva; } EXPORT_SYMBOL_FOR_KVM_INTERNAL(__x86_set_memory_region); @@ -10106,20 +10102,6 @@ void kvm_arch_pre_destroy_vm(struct kvm *kvm) void kvm_arch_destroy_vm(struct kvm *kvm) { - if (current->mm == kvm->mm) { - /* - * Free memory regions allocated on behalf of userspace, - * unless the memory map has changed due to process exit - * or fd copying. - */ - mutex_lock(&kvm->slots_lock); - __x86_set_memory_region(kvm, APIC_ACCESS_PAGE_PRIVATE_MEMSLOT, - 0, 0); - __x86_set_memory_region(kvm, IDENTITY_PAGETABLE_PRIVATE_MEMSLOT, - 0, 0); - __x86_set_memory_region(kvm, TSS_PRIVATE_MEMSLOT, 0, 0); - mutex_unlock(&kvm->slots_lock); - } if (kvm->arch.created_mediated_pmu) perf_release_mediated_pmu(); kvm_destroy_vcpus(kvm); @@ -10159,6 +10141,19 @@ void kvm_arch_free_memslot(struct kvm *kvm, struct kvm_memory_slot *slot) } kvm_page_track_free_memslot(slot); + + /* + * Unmap VMAs allocated on behalf of userspace for internal memslots, + * unless the memory map has changed due to process exit or fd copying. + * Leak the mapping on failure, e.g. if the task is killed. Worst case + * scenario, the page(s) will be reclaimed when the process exits. + * + * TODO: Allocate VMAs for each address space and drop the check on the + * as_id, which depends on freeing memslots from 0=>N-1. + */ + if (current->mm == kvm->mm && slot->id >= KVM_USER_MEM_SLOTS && + slot->as_id == (kvm_arch_nr_memslot_as_ids(kvm) - 1)) + vm_munmap(slot->userspace_addr, slot->npages * PAGE_SIZE); } int memslot_rmap_alloc(struct kvm_memory_slot *slot, unsigned long npages) -- 2.56.0.385.gac671c819a-goog
