Hi Oliver,
On 03/04/2023 14:46, Oliver Schwartz wrote:
Dear all,
I’ve been hunting down some performance issues I had in jailhouse inmates cells
on Arm64 lately. The problem was that some operations driven by interrupts in a
baremetal inmate cell failed to meet the timing requirements depending on
activity in the root cell.
I’ve finally figured out that the reason is the page table operations in the
Linux root cell. In arch/arm64/include/asm/tlbflush.h in the linux kernel all
page table invalidations are broadcasted to all other CPU cores. As far as I
understand, this forces all CPU cores to flush and reload their pagetables,
regardless of whether they are running in the same cell or not. In my case,
this has caused interrupt latencies to increase from under 1 µs to above 20 µs
in the inmate if a Linux application did some heavy memory (de-)allocation.
Apparently this has been addressed on LKML before, but the proposed patch was
rejected:
https://lore.kernel.org/linux-arm-kernel/20191101172851.GC3983@willie-the-truck/T/
Since I only use a single core for Linux I can workaround this issue with local
updates to the page table. However, I was wondering if there is a way to
address this in the hypervisor?
A hypervisor-only solution is possible, but suboptimal if you're not on ARMv8.2+
boards.
You could set HCR_EL2.TTLB to trap TLB maintenance instructions, but this will
trap _all_ TLB maintenance instructions (so, possibly quite a lot). With
FEAT_EVT, you could use HCR_EL2.TTLBIS (TTLBOS) to trap only inner (outer)
shareable updates.
We had similar issues with cache invalidations (where you can use a similar
trick), but since we didn't have a v8.2 board, we needed to implement a patch
for the OS.
Best regards,
Andrea
--
Technische Universität München
Lehrstuhl für Cyber-Physical Systems in Production Engineering
--
You received this message because you are subscribed to the Google Groups
"Jailhouse" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/jailhouse-dev/df4f654b-fcc7-bef7-d385-0841c16e80c5%40tum.de.
