On 17.05.22 18:19, Lad, Prabhakar wrote: > Hi Jan, > > On Tue, May 17, 2022 at 12:44 PM Jan Kiszka <[email protected]> wrote: >> >> On 16.05.22 20:01, Lad, Prabhakar wrote: >>> Hi Jan, >>> >>> On Mon, May 16, 2022 at 6:57 AM Jan Kiszka <[email protected]> wrote: >>>> >>>> On 13.05.22 18:28, Lad, Prabhakar wrote: >>>>> Hi Jan, >>>>> >>>>> On Fri, May 13, 2022 at 4:33 PM Jan Kiszka <[email protected]> wrote: >>>>>> >>>>>> On 13.05.22 17:20, Lad, Prabhakar wrote: >>>>>>> On Fri, May 13, 2022 at 1:18 PM Jan Kiszka <[email protected]> >>>>>>> wrote: >>>>>>>> >>>>>>>> On 13.05.22 13:32, Lad, Prabhakar wrote: >>>>>>>>>> Could it be that arm_dcaches_flush was a red herring? Maybe we are >>>>>>>>>> actually crashing on hvc in arch_entry, and that because of Linux not >>>>>>>>>> starting in HVC mode, thus no hypervisor take-over stub being >>>>>>>>>> installed. >>>>>>>>>> >>>>>>>>>> Could you share the boot log of the kernel? >>>>>>>>>> >>>>>>>>> Attached is the complete log. >>>>>>>>> >>>>>>>> >>>>>>>> "CPU: All CPU(s) started at EL1" >>>>>>>> >>>>>>>> As suspected. Must be EL2 or HYP. Reconfigure your firmware to enable >>>>>>>> that. >>>>>>>> >>>>>>> Argh my bad, I've now enabled it to start in EL2 mode (attached are >>>>>>> the complete logs). Now I see the below panic, >>>>>>> >>>>>>> [ 33.920507] jailhouse: loading out-of-tree module taints kernel. >>>>>>> Reading configuration set: >>>>>>> Root cell: Renesas RZ/V2L SMARC (renesas-r9a07g054l2.cell) >>>>>>> Overlapping memory regions inside cell: None >>>>>>> Overlapping memory regions with hypervisor: None >>>>>>> Missing resource interceptions for architecture arm64: None >>>>>>> [ 34.741847] Bad mode in Synchronous Abort handler detected on CPU0, >>>>>>> code 0x5a000000 -- HVC (AArch64) >>>> >>>> "HVC instruction execution in AArch64 state, when HVC is not disabled", >>>> origin: EL2 (hypervisor mode). >>>> >>>> Is KVM off again when you start Jailhouse here? Does the CPU support VHE >>>> (Virtualization Host Extension)? Is CONFIG_ARM64_VHE enabled? Try to >>>> turn that off. >>>> >>> KVM was OFF, I have enabled both CONFIG_VIRTUALIZATION and CONFIG_KVM >>> (I have attached the complete kernel config). Yes the CPU does have >>> VHE support. I did it by disabling CONFIG_ARM64_VHE which did not make >>> any difference. >>> >>> With the VHE disable now I get the below output: >>> root@smarc-rzg2l:~# dmesg | grep -i kvm >>> [ 0.095927] kvm [1]: IPA Size Limit: 40 bits >>> [ 0.096895] kvm [1]: GICv3: no GICV resource entry >>> [ 0.096916] kvm [1]: disabling GICv2 emulation >>> [ 0.096949] kvm [1]: GIC system register CPU interface enabled >>> [ 0.097045] kvm [1]: vgic interrupt IRQ9 >>> [ 0.097156] kvm [1]: Hyp mode initialized successfully >>> root@smarc-rzg2l:~# >>> >>> Now with the above configs I don't get any output on the console when >>> the root cell is enabled: >>> >>> root@smarc-rzg2l:~# sh -x ./hyp.sh >>> + insmod jailhouse.ko >>> [ 35.379219] jailhouse: loading out-of-tree module taints kernel. >>> + jailhouse config check -a arm64 renesas-r9a07g054l2.cell >>> Reading configuration set: >>> Root cell: Renesas RZ/V2L SMARC (renesas-r9a07g054l2.cell) >>> Overlapping memory regions inside cell: None >>> Overlapping memory regions with hypervisor: None >>> Missing resource interceptions for architecture arm64: None >>> + jailhouse enable renesas-r9a07g054l2.cell >>> [ 36.217465] obcode @arm_dcaches_flush: d53b0024 >>> [ 36.217469] obcode @arm_dcaches_flush: d53b0024 >>> >>> Any pointers on debugging this further, has anyone tried jailhouse on >>> Cortex-A55 before? >> >> Hmm, at least not I so far. Might be the point that we are missing some >> subtle difference here around the way the setup works then. >> > Right. > >> In any case, the fact that at least the previous crash is gone indicates >> that something changed. You could check if you reach now the second hvc >> by adding a ret in front of it. If you, you may end up in hyp mode, >> maybe just not have a working UART yet (is the model you have on the >> board in principle already supported by Jailhouse?), run into a >> different issue, but just don't see Jailhouse report of it. >> > It's not going pass the arm_dcaches_flush statement. > > With the below diff, I don't see any logs on console > option #1 > diff --git a/hypervisor/arch/arm64/caches.S b/hypervisor/arch/arm64/caches.S > index 39dad4af..ce29b8e8 100644 > --- a/hypervisor/arch/arm64/caches.S > +++ b/hypervisor/arch/arm64/caches.S > @@ -40,6 +40,7 @@ > */ > .global arm_dcaches_flush > arm_dcaches_flush: > + ret > dcache_line_size x3, x4 > add x1, x0, x1 > sub x4, x3, #1 > diff --git a/hypervisor/arch/arm64/entry.S b/hypervisor/arch/arm64/entry.S > index a9cabf7f..c8ebf695 100644 > --- a/hypervisor/arch/arm64/entry.S > +++ b/hypervisor/arch/arm64/entry.S > @@ -97,6 +97,7 @@ arch_entry: > ldr x1, [x0, #HEADER_CORE_SIZE] > mov x2, DCACHE_CLEAN_AND_INVALIDATE_ASM > bl arm_dcaches_flush > + ret > > /* install bootstrap_vectors */ > ldr x1, =bootstrap_vectors > >
Forgot: that cannot work. The call of arm_dcaches_flush will overwrite lr, thus the second ret will only return to where the first ret jumped to - endless loop. You would have to restore lr (x30) from x17 in arch_entry first: mov x30, x17 ret Jan -- Siemens AG, Technology Competence Center Embedded Linux -- You received this message because you are subscribed to the Google Groups "Jailhouse" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/jailhouse-dev/35cddc35-a563-ca93-8e29-b9ae06844030%40siemens.com.
