On 2016-07-20 12:05, Marc Zyngier wrote: > On 20/07/16 10:25, Jan Kiszka wrote: > >>> - KVM cleans the cache to the PoC when mapping a page if the guest >>> doesn't have its own caches enabled. Otherwise you will happily run with >>> junk while the data sits pretty in the cache. (see my presentation at >>> KF'15). >> >> Caching/cache flushing vs. guest caches disabled is another topic which >> caused a lot of headache to me in the past weeks (because it seems >> fairly broken in Jailhouse, up to causing hypervisor crashes). As you >> already brought it to that point: >> >> We currently flush the DCache completely before starting a new guest >> (with new code etc.), and also after resetting a guest CPU into >> caches-disabled state. The flushing takes place in the middle of the >> vmexit. Once in a while, I got nasty crashes due to inconsistent states >> of the hypervisor (states that are shared between the CPUs). >> >> My theory for this: After the flush, the CPU picks up some state >> variable again into its local cache, then enters the guest which has >> caches off. While we are in the guest, another CPU changes that shared >> state, but the change is not propagated to the other CPU cache because >> that is off. Now that first CPU leaves the guest, uses its stale cache >> content and ruins our day. Following that theory, I moved the flush >> right before the vmentry, ie. at a point where no more shared state is >> touched. That made the crashes go away, apparently. >> >> However, if that theory is true, I think we would have to flush caches >> before *every* vmentry if the guest has caches off (we don't to that >> right now). Now, what are the real requirements when dealing with guests >> in non-cached mode? > > The real requirement is "don't do it". I mean, really. The architecture > doesn't give us any reasonable way to deal with that, hence the horrible > "trap all the VM regs until M+C are set" code we have on KVM. > > So if you do share state between your guest and your hypervisor, you > have to perform cache maintenance when the guest has its caches off.
The crash was not over state shared between guest and host (though we have that as well - to be addressed as well...), this was about side effects on state shared between CPUs while in HYP mode and trying to perform some cache maintenance. Again my question: What are the requirements regarding cache maintenance when allowing a guest to run with caches off? Jean-Philippe tried to address that in [1], but it's not complete or not fully correct or even both. Jan [1] https://github.com/siemens/jailhouse/commit/add44a7a8431058ec9acb3db328166f8a8c34dcb -- Siemens AG, Corporate Technology, CT RDA ITP SES-DE Corporate Competence Center Embedded Linux -- You received this message because you are subscribed to the Google Groups "Jailhouse" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
