Hi, we have some trouble starting non-root Linux on an AMD box. I already tried to narrow things down, but it raised several questions.
The main problem is, that non-root Linux tries to write to LVT0, and jailhouse crashes with: FATAL: Setting invalid LVT delivery mode (reg 35, value 00000700) Turns out, in comparison to Intel x86, we don't trap on APIC reads, we only intercept APIC write on AMD (cf. svm.c:338). I thought this would be the issue of this bug, as that's an obvious difference between Intel and AMD: on VMX, we do trap xAPIC reads and writes. However, VMX works slightly different in these regards (side note: [1]). xAPIC reads on AMD systems don't trap the hypervisor, so I intercepted reads (by removing the present bit of the XAPIC_PAGE of the guest), and forwarded the traps to the apic dispatcher (adjusted VMEXIT_NPF). I can confirm that we now trap reads as well as writes. But the non-root Linux still crashes with the same error. Digging a bit deeper, I found out that xAPIC reads are directly forwarded to the hardware, if they were intercepted. So this explains why the bug still remains. This raised another question regarding xAPIC handling on Intel: On AMD, we don't intercept xAPIC reads. On Intel, we do, as we follow the strategy mentioned in [1]… But why? Wouldn't it be more performant to just trap on xAPIC writes on Intel? This could be done by switching from APIC_ACCESS interception to simple write-only trap & emulate (page faults). However, back to the initial issue. Looks like the difference between Intel and AMD boot is as follows. AMD: [ 0.325578] Switched APIC routing to physical flat. [ 0.366464] enabled ExtINT on CPU#0 Intel: [ 0.099486] Switched APIC routing to physical flat. [ 0.113000] masked ExtINT on CPU#0 This is why the above-mentioned Jailhouse crash occurs. I tried to find out why Linux takes this decision on AMD. Our victim is in apic.c:1587. On Intel, apic_read(LVT0) & APIC_LVT_MASKED results in 65536, on AMD it is 0. This is why we take a different path. Now the question is simple -- why? :-) Are we just lacking ExtINT delivery mode in Jailhouse, or is anything else odd? Ralf [1] Regarding Intel VMX, why do we actually need the apic_access_page / SECONDARY_EXEC_VIRTUALIZE_APIC_ACCESSES dance? On VMX, we do trap on xAPIC reads AND writes. So why do we need virtualisation? Why don't we simply trap on MMIO and handle these situations under EXIT_REASON_EPT_VIOLATION? Is it just to save the dispatching or is there another reason? -- You received this message because you are subscribed to the Google Groups "Jailhouse" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/jailhouse-dev/c2839a0f-c1b5-9139-43d5-01139341981a%40oth-regensburg.de. For more options, visit https://groups.google.com/d/optout.
