Thanks for the response Ralf, I came across a paper that compares the performance of a few static partitioning hypervisors including Jailhouse. Probably, you are already aware of it, the paper is available here. https://arxiv.org/pdf/2303.11186.
One interesting thing I noticed is that for ARM with direct interrupt injection enabled, the interrupt latency for Jailhouse is around ~200ns (please see the attached image). However, on my x86 system I am getting an average latency of ~800ns (from the APIC demo example). I think the APIC example is also doing the same thing as was done for the experiment in the figure. Do you have any idea why there is such a difference between the values for both cases? is it architecture specific? On Fri, May 3, 2024 at 5:31 PM Ralf Ramsauer < [email protected]> wrote: > Hi, > > On 02/05/2024 15:45, Syed Aftab Rashid wrote: > > [snip] > > > > I just wanted to understand what is the best possible jitter that > > I can > > > get with Jailhouse. We are working on a project that needs cycle > > times > > > between 1-2 microseconds. The bare metal implementation of a > > similar > > > application results in a much lower jitter ~50ns. > > > > So the application does structurally the same like apic-demo? > > (i.e., full IRQ arrival and dispatching, "computational task" (i.e., > > calculating jitter), return). 50ns are ~100 TSC cycles – athletic. > > > > I really doubt those latencies are achievable with IRQs. With active > > waiting for events (i.e., active polling for interruption) - maybe. > > > > > > Yes, it is a similar application implemented in bare metal code directly > > on the hardware. The jitter is even smaller with the max being 30ns. > > Interesting. W/o having the possibility to have a close look at this > issue, I'd say that this comes from the additional need of hardware > lookups through IRQ rerouting tables. I cannot say. > > [snip] > > > > apic-demo doesn't seem to have much impact on the jitter. How is > > cache > > > allocated under Jailhouse house? What does a cache allocation of > > size 2 > > > employ? > > > > Cache allocation is only supported on Intel X86, where we have CAT, > > cache allocation technology. > > > > I don't know precise details of cat (would have to look them up), > > but as > > far as I remember, assigning a size of 2 to a non-root-cell means > that > > 2MiB of L3$ are exclusively assigned to the non-root cell. (Look at > the > > code, I'm not 100% sure). > > > > Why do we do that? Neighbouring noise from other cells. > > > > Just like the non-root cell might "disturb" itself, it's also > possible > > that Linux, running aside of the non-root cell disturbes it: Imagine > > Linux those heavy number crunching while the non-root cell waits for > an > > IRQ. In that case, Linux will mess up caches of the non-root cell. > > > > CAT prevents full cache pollution by ensuring that there is always a > > dedicated, but limited, amount of L3$ for the non-root cell. > > > > > > Thanks for the explanation, I understand the idea of cache pollution. It > > is just that I don't see much impact with or without cache pollution on > > the values I get for the jitter. Ideally, the jitter should be higher if > > I enable cache pollution and lower when I disable it. But, so far I am > > not able to see that. > > > > Also, another question regarding direct interrupt injection, as we saw > > that it is enabled by default. Can you tell me how can I disable it? > > Again, this is just to see how it will impact the jitter if we have > > direct interrupt injection disabled. > > There's no actual support for disabling this, it's used when being > available. When Jailhouse tries to query the capability of IRQ remapping > support during startup, you will have to mask out the bits that show > support for it. I guess that should work. > > Ralf > -- Best Regards, Syed Aftab Rashid -- You received this message because you are subscribed to the Google Groups "Jailhouse" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/jailhouse-dev/CAJtpjXj2vfPc8RP1QzgHHato%2By_qW4-k8xrSDjur2KJkcx_BGw%40mail.gmail.com.
