Joe's response.. ---
-------- Original Message -------- Subject: Re: [Fwd: Re: [xen-discuss] xvm 32-bit: excessive number of pagefaults after fork() / copy-on-wri] Date: Thu, 24 Apr 2008 12:00:41 -0700 From: Joe B... > > > The hypervisor is told up invalidate the page that contains the > PTE (via HYPERVISOR_update_va_mapping, va cda02000 flags 2), > but the CPU / MMU isn't told that the mapping for the virtual stack address > 8047000 has changed. Isn't it possible that the CPU / MMU / TLB has > cached the information "virtual stack address 8047000 is not valid > address", > after the call to x86pte_inval() ? Nope. No hardware ever works this way. It only caches information in the TLB when the lowest bit (PRESENT) is set in the PTE. Therefore when setting an entry that is zero to a non-zero value, an INVLPG instruction is never needed on hardware. > > htable.c x86pte_set() does a TLB flush when the old PTE > referred to a referenced page, but it doesn't update the TLB when > an empty PTE was replaced with a new translation: > > 2090 /* ... > > > > Btw. I've been experimenting with this change to x86pte_set() > (lines 2103 ... 2111 added): > > 2090 /* ... > 2111 #endif > > > With xpv_page_fault_hack := 0 I get the original code. > > With xpv_page_fault_hack := 2 I try to do an INVALPG on the > new installed translation. But that hasn't fixed the issue... > > But with xpv_page_fault_hack := 1 the entire TLB gets flushed > when installing new stack pages, and now: That would confirm a bug in Xen.. both xen_flush_tlb() and xen_flush_va() should have identical behavior here - and shouldn't be necessary either. > > 1. the libMicro-0.4.0 fork_100 test runs ~ 30x faster in a 32-bit PV > domU !! > 800 seconds -> 28 seconds > 2 ./boot/solaris/bin/create_ramdisk runs ~ 4x faster in a 32-bit PV domU ! > 2 minutes -> 36 seconds > > > So it seems that there is an issue with the TLB in 32-bit xVM PV doms... The bug is probably in TLB flushing management in the Xen code itself. I know they've said in the past that they do all kinds of very crafty optimizations to avoid unnecessary invalidates in the hypervisor. I suspect they've got a bug. _______________________________________________ xen-discuss mailing list [email protected]
