Adding José Martins in cc (direct gicv2 injection was his idea) as he might be interested in this.
On 02/07/2020, Jan Kiszka <[email protected]> wrote: > On 02.07.20 18:45, Jan Kiszka wrote: >> On 02.07.20 15:31, Angelo Ruocco wrote: >>> On 02/07/2020, 'Nikhil Devshatwar' via Jailhouse >>> <[email protected]> wrote: >>>> On 02/07/20 5:14 pm, Jan Kiszka wrote: >>>>> BTW, regarding direct interrupt delivery on ARM: In >>>>> https://lwn.net/Articles/820830, it is reported that Bao has "found a >>>>> way to map interrupts directly into guests". I didn't find the time >>>>> yet >>>>> to check if that is actually exit-free delivery, and that as a smart >>>>> trick or rather a problematic hack. Or if that sentence is rather a >>>>> misunderstanding. There is also the sentence: "Interrupts [...] have >>>>> to >>>>> be mediated through the hypervisor, which is unfortunate since that >>>>> increases latency." >>>>> >>>> >>>> I found this interesting and tried to read more about this. >>>> I found some commits at [1] which does the direct injection. >>>> Here is my rough understanding: >>>> >>>> * He is not setting the HCR_EL.FMO bit and that way all virtual >>>> interrupts are turned off >>>> * There is a new handler for handling "lower_el_aarch64_fiq" >>>> which ends up being handled by the Hypervisor >>>> * GICD is still being emulated so guests won't be able to route >>>> interrupts to wrong CPUs. Isolation is maintained >>>> * For triggering interrupts from Hypervisor (SGIs, etc) he is using SMC >>>> calls and has a new service implemented in the ATF [2] >>>> * I could not understand how the lower_el_aarch64_fiq exception is >>>> fired >>>> so that the Hypervisor is invoked >>>> >>>> My guess is that most of the code change ihere s to enable interrupts >>>> in >>>> the Hypervisor. Resetting HCR_EL2.FMO would send the interrupts at EL1 >>>> directly. >>> >>> Yup, that's more or less it. >>> >>> Just one clarifications, Bao *does* set the HCR_EL2.FMO, it doesn't >>> set the HCR_EL2.HMO. The HMO bit lets the interrupts through, and they >> >> HCR_EL2.HMO? On which revision of the ARMv8 spec are we looking? Cannot >> find that in mine. Ugh, it's a typo, I meant HCR_EL2.IMO, my bad. >>> go directly to EL1/0. The virtual machines have complete access to the >>> gicc and receive IRQs without overhead. >>> As you have already pointed out, gicd is still emulated, so all the >>> operation of "enable/disable/set_tarrget" are slow as usual, and still >>> under control of the hypervisor. >> >> And so is inner-guest SGI (IPI) submission, I suppose. Just like on >> Intel (so far). Yes, everything that's gicd related is still handled by the hypervisor. The logic though is extremely simple, so the cost becomes almost the same as an empty hypercall. >>> >>> The problem is that IRQs are completely invisible to EL2, so Bao needs >>> to use FIQs for its internal functions. And to do that the only way is >>> to go through the secure monitor in EL3 for each operation, having >>> custom services running in the arm trusted firmware. >> >> Ah, interesting pattern: FIQ becomes the NMI that enables such a trick >> on Intel CPUs. If FIQ triggering architecturally limited to EL3, or is >> that SoC-specific? It's arm specific, FIQs are meant to be only used in the secure world, so EL3 and secure EL1. >> I'm not fully familiar with the ATF/TEE world yet, but your approach to >> model that service as SPD seems to block other use cases, like having a >> full TEE (OP-TEE). Wouldn't it rather make sense to model that as TEE >> app? Yes, it could be done as TEE app, but done like this shouldn't affect other use-cases, that we could of think of anyway... >>> >>> I've even thought of trying something similar in jailhouse, using the >>> linux driver that, running in EL1, could let jailhouse access the IRQs >>> even without the HMO bit set, and therefore allowing direct IRQs >>> access to inmates without the need to put a custom service in the arm >>> trusted firmware, but I found it a bit too intricate. >>> >> >> You can't use the guest to model management interrupts that are there to >> interrupt the guest and inform the hypervisor about a high-prio event >> (like "kill that guest"). True, and you have a lot of other problems having to rely on a guest for hypervisor tasks. > I started to read and think about this more - and then I suddenly found > SDEI [1]. Isn't our problem of having a non-IRQ event already solved by > registering on / sending some event 0? And SDEI is implemented by ATF. > > That would nicely overcome all the problems of the baod patch to ATF > (integration, security, missing gicv3 support, blocking of other > SPDs...). And even if I miss some catch in SDEI, the principle of > synthetic events defined there is what we want for hypervisor management > IPIs. Oh I didn't know about SDEI, it looks promising, but in the current state of the arm developer website I can't find any useful information, it doesn't even seem to be mentioned in the arm architecture reference manual. > The only downside: On a safety critical system, you may rather want to > get the firmware out of the picture during runtime, to reduce the safety > scope to a real minimum. But, IIRC, GICv4 will solve direct interrupt > injection in HW anyway. Yeah, in the GICv4 specifications there's the ability to directly inject the interrupts, but depending on the field it might be too long down the road for someone to wait. -- Angelo > > Jan > > [1] > https://developer.arm.com/architectures/system-architectures/software-standards/sdei > > -- > Siemens AG, Corporate Technology, CT RDA IOT SES-DE > Corporate Competence Center Embedded Linux > -- You received this message because you are subscribed to the Google Groups "Jailhouse" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/jailhouse-dev/CADiTV-1oJBD%2BSkAT475xD5iODeo24wqwBSFGu2JQATxqpF1W0w%40mail.gmail.com.
