[PATCH 0/6] Use virtual page class key protection mechanism for speeding up guest page fault
Hi,
With the current code we do an expensive hash page table lookup on every
page fault resulting from a missing hash page table entry. A NO_HPTE
page fault can happen due to the below reasons:
1) Missing hash pte as per guest. This should be forwarded to the guest
2) MMIO hash pte. The address against which the load/store is performed
should be emulated as a MMIO operation.
3) Missing hash pte because host swapped out the guest page.
We want to differentiate (1) from (2) and (3) so that we can speed up
page fault due to (1). Optimizing (1) will help in improving
the overall performance because that covers a large percentage of
the page faults.
To achieve the above we use virtual page calss protection mechanism for
covering (2) and (3). For both the above case we mark the hpte
valid, but associate the page with virtual page class index 30 and 31.
The authority mask register is configured such that class index 30 and 31
will have read/write denied. The above change results in a key fault
for (2) and (3). This allows us to forward a NO_HPTE fault directly to guest
without doing the expensive hash pagetable lookup.
For the test below:
#include unistd.h
#include stdio.h
#include stdlib.h
#include sys/mman.h
#define PAGES (40*1024)
int main()
{
unsigned long size = getpagesize();
unsigned long length = size * PAGES;
unsigned long i, j, k = 0;
for (j = 0; j 10; j++) {
char *c = mmap(NULL, length, PROT_READ|PROT_WRITE,
MAP_PRIVATE|MAP_ANONYMOUS, -1, 0);
if (c == MAP_FAILED) {
perror(mmap);
exit(1);
}
for (i = 0; i length; i += size)
c[i] = 0;
/* flush hptes */
mprotect(c, length, PROT_WRITE);
for (i = 0; i length; i += size)
c[i] = 10;
mprotect(c, length, PROT_READ);
for (i = 0; i length; i += size)
k += c[i];
munmap(c, length);
}
}
Without Fix:
--
[root@qemu-pr-host ~]# time ./pfault
real0m8.438s
user0m0.855s
sys 0m7.540s
[root@qemu-pr-host ~]#
With Fix:
[root@qemu-pr-host ~]# time ./pfault
real0m7.833s
user0m0.782s
sys 0m7.038s
[root@qemu-pr-host ~]#
Aneesh Kumar K.V (6):
KVM: PPC: BOOK3S: HV: Clear hash pte bits from do_h_enter callers
KVM: PPC: BOOK3S: HV: Deny virtual page class key update via h_protect
KVM: PPC: BOOK3S: HV: Remove dead code
KVM: PPC: BOOK3S: HV: Use new functions for mapping/unmapping hpte in
host
KVM: PPC: BOOK3S: Use hpte_update_in_progress to track invalid hpte
during an hpte update
KVM: PPC: BOOK3S: HV: Use virtual page class protection mechanism for
host fault and mmio
arch/powerpc/include/asm/kvm_book3s_64.h | 97 +-
arch/powerpc/include/asm/kvm_host.h | 1 +
arch/powerpc/include/asm/reg.h | 1 +
arch/powerpc/kernel/asm-offsets.c| 1 +
arch/powerpc/kvm/book3s_64_mmu_hv.c | 99 --
arch/powerpc/kvm/book3s_hv.c | 1 +
arch/powerpc/kvm/book3s_hv_rm_mmu.c | 166 +--
arch/powerpc/kvm/book3s_hv_rmhandlers.S | 100 +--
8 files changed, 371 insertions(+), 95 deletions(-)
--
1.9.1
--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/6] Use virtual page class key protection mechanism for speeding up guest page fault
On Sun, 2014-06-29 at 16:47 +0530, Aneesh Kumar K.V wrote: To achieve the above we use virtual page calss protection mechanism for covering (2) and (3). For both the above case we mark the hpte valid, but associate the page with virtual page class index 30 and 31. The authority mask register is configured such that class index 30 and 31 will have read/write denied. The above change results in a key fault for (2) and (3). This allows us to forward a NO_HPTE fault directly to guest without doing the expensive hash pagetable lookup. So we have a measurable performance benefit (about half a second out of 8) but you didn't explain the drawback here which is to essentially make it impossible for guests to exploit virtual page class keys, or did you find a way to still make that possible ? As it-is, it's not a huge issue for Linux but we might have to care with other OSes that do care... Do we have a way in PAPR to signify to the guest that the keys are not available ? Cheers, Ben. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/6] Use virtual page class key protection mechanism for speeding up guest page fault
Benjamin Herrenschmidt b...@kernel.crashing.org writes: On Sun, 2014-06-29 at 16:47 +0530, Aneesh Kumar K.V wrote: To achieve the above we use virtual page calss protection mechanism for covering (2) and (3). For both the above case we mark the hpte valid, but associate the page with virtual page class index 30 and 31. The authority mask register is configured such that class index 30 and 31 will have read/write denied. The above change results in a key fault for (2) and (3). This allows us to forward a NO_HPTE fault directly to guest without doing the expensive hash pagetable lookup. So we have a measurable performance benefit (about half a second out of 8). I was able to measure a performance benefit of 2 seconds earlier. But once i get the below patch applied that got reduced. I am trying to find how the patch is helping the performance. May be it is avoiding some unnecessary invalidation ? http://mid.gmane.org/1403876103-32459-1-git-send-email-aneesh.ku...@linux.vnet.ibm.com I also believe the benefit depends on how much impact a hash table lookup have. I did try to access the addresses linearly so that I can make sure we do take a cache miss for hash page table access. but you didn't explain the drawback here which is to essentially make it impossible for guests to exploit virtual page class keys, or did you find a way to still make that possible ? I am now making PROTFAULT to go to host. That means, ksm sharing is represented as read only page and an attempt to write to it will get to host via PROTFAULT. Now with that we can implement keys for guest if we want to. So irrespective of what restrictions guest has put in, if the host swapout the page, we will deny read/write. Now if the key fault need to go to guest, we will find that out looking at the key index. As it-is, it's not a huge issue for Linux but we might have to care with other OSes that do care... Do we have a way in PAPR to signify to the guest that the keys are not available ? Right now Qemu doesn't provide the device tree node ibm,processor-storage-keys. That means guest cannot use keys. So we are good there. If we want to support guest keys, we need to fill that with the value that indicate how many keys can be used for data and instruction. -aneesh -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 0/6] Use virtual page class key protection mechanism for speeding up guest page fault
Hi,
With the current code we do an expensive hash page table lookup on every
page fault resulting from a missing hash page table entry. A NO_HPTE
page fault can happen due to the below reasons:
1) Missing hash pte as per guest. This should be forwarded to the guest
2) MMIO hash pte. The address against which the load/store is performed
should be emulated as a MMIO operation.
3) Missing hash pte because host swapped out the guest page.
We want to differentiate (1) from (2) and (3) so that we can speed up
page fault due to (1). Optimizing (1) will help in improving
the overall performance because that covers a large percentage of
the page faults.
To achieve the above we use virtual page calss protection mechanism for
covering (2) and (3). For both the above case we mark the hpte
valid, but associate the page with virtual page class index 30 and 31.
The authority mask register is configured such that class index 30 and 31
will have read/write denied. The above change results in a key fault
for (2) and (3). This allows us to forward a NO_HPTE fault directly to guest
without doing the expensive hash pagetable lookup.
For the test below:
#include unistd.h
#include stdio.h
#include stdlib.h
#include sys/mman.h
#define PAGES (40*1024)
int main()
{
unsigned long size = getpagesize();
unsigned long length = size * PAGES;
unsigned long i, j, k = 0;
for (j = 0; j 10; j++) {
char *c = mmap(NULL, length, PROT_READ|PROT_WRITE,
MAP_PRIVATE|MAP_ANONYMOUS, -1, 0);
if (c == MAP_FAILED) {
perror(mmap);
exit(1);
}
for (i = 0; i length; i += size)
c[i] = 0;
/* flush hptes */
mprotect(c, length, PROT_WRITE);
for (i = 0; i length; i += size)
c[i] = 10;
mprotect(c, length, PROT_READ);
for (i = 0; i length; i += size)
k += c[i];
munmap(c, length);
}
}
Without Fix:
--
[root@qemu-pr-host ~]# time ./pfault
real0m8.438s
user0m0.855s
sys 0m7.540s
[root@qemu-pr-host ~]#
With Fix:
[root@qemu-pr-host ~]# time ./pfault
real0m7.833s
user0m0.782s
sys 0m7.038s
[root@qemu-pr-host ~]#
Aneesh Kumar K.V (6):
KVM: PPC: BOOK3S: HV: Clear hash pte bits from do_h_enter callers
KVM: PPC: BOOK3S: HV: Deny virtual page class key update via h_protect
KVM: PPC: BOOK3S: HV: Remove dead code
KVM: PPC: BOOK3S: HV: Use new functions for mapping/unmapping hpte in
host
KVM: PPC: BOOK3S: Use hpte_update_in_progress to track invalid hpte
during an hpte update
KVM: PPC: BOOK3S: HV: Use virtual page class protection mechanism for
host fault and mmio
arch/powerpc/include/asm/kvm_book3s_64.h | 97 +-
arch/powerpc/include/asm/kvm_host.h | 1 +
arch/powerpc/include/asm/reg.h | 1 +
arch/powerpc/kernel/asm-offsets.c| 1 +
arch/powerpc/kvm/book3s_64_mmu_hv.c | 99 --
arch/powerpc/kvm/book3s_hv.c | 1 +
arch/powerpc/kvm/book3s_hv_rm_mmu.c | 166 +--
arch/powerpc/kvm/book3s_hv_rmhandlers.S | 100 +--
8 files changed, 371 insertions(+), 95 deletions(-)
--
1.9.1
--
To unsubscribe from this list: send the line unsubscribe kvm-ppc in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
