On 26.05.22 17:48, Lad, Prabhakar wrote: > Hi Jan, > > On Tue, May 24, 2022 at 12:55 PM Lad, Prabhakar > <[email protected]> wrote: >> >> On Mon, May 23, 2022 at 4:13 PM Jan Kiszka <[email protected]> wrote: >>> >>> On 23.05.22 15:55, Lad, Prabhakar wrote: >>>> Hi Jan, >>>> >>>> On Fri, May 20, 2022 at 7:08 AM Jan Kiszka <[email protected]> wrote: >>>>> >>>>> On 19.05.22 15:23, Lad, Prabhakar wrote: >>>>>> Hi Jan, >>>>>> >>>>>> On Thu, May 19, 2022 at 12:39 PM Jan Kiszka <[email protected]> >>>>>> wrote: >>>>>>> >>>>>>> On 19.05.22 11:44, Lad, Prabhakar wrote: >>>>>>>> On Thu, May 19, 2022 at 6:30 AM Jan Kiszka <[email protected]> >>>>>>>> wrote: >>>>>>>>> Forgot: that cannot work. The call of arm_dcaches_flush will overwrite >>>>>>>>> lr, thus the second ret will only return to where the first ret jumped >>>>>>>>> to - endless loop. You would have to restore lr (x30) from x17 in >>>>>>>>> arch_entry first: >>>>>>>>> >>>>>>>>> mov x30, x17 >>>>>>>>> ret >>>>>>>>> >>>>>>>> That did the trick thanks! >>>>>>>> >>>>>>>> diff --git a/hypervisor/arch/arm64/entry.S >>>>>>>> b/hypervisor/arch/arm64/entry.S >>>>>>>> index a9cabf7f..7b340bd1 100644 >>>>>>>> --- a/hypervisor/arch/arm64/entry.S >>>>>>>> +++ b/hypervisor/arch/arm64/entry.S >>>>>>>> @@ -109,6 +109,8 @@ arch_entry: >>>>>>>> mov x0, #LINUX_HVC_SET_VECTORS_LEGACY >>>>>>>> 1: >>>>>>>> hvc #0 >>>>>>>> + mov x30, x17 >>>>>>>> + ret >>>>>>>> >>>>>>>> hvc #0 /* bootstrap vectors enter EL2 at el2_entry */ >>>>>>>> b . /* we don't expect to return here */ >>>>>>>> >>>>>>>> >>>>>>>> With the above diff I do get the below: >>>>>>>> >>>>>>>> [ 42.980805] jailhouse: loading out-of-tree module taints kernel. >>>>>>>> Reading configuration set: >>>>>>>> Root cell: Renesas RZ/V2L SMARC (renesas-r9a07g054l2.cell) >>>>>>>> Overlapping memory regions inside cell: None >>>>>>>> Overlapping memory regions with hypervisor: None >>>>>>>> Missing resource interceptions for architecture arm64: None >>>>>>>> [ 46.582588] obcode @arm_dcaches_flush: d53b0024 >>>>>>>> [ 46.582616] obcode @arm_dcaches_flush: d53b0024 >>>>>>>> [ 46.611311] The Jailhouse is opening. >>>>>>>> >>>>>>>> So it looks like something to do with the debug console. This has to >>>>>>>> be poked in the dark or any easy way to debug? >>>>>>> >>>>>>> Well, we do not yet know what goes wrong. We do know that we can call >>>>>>> into the hyp take-over stub and register Jailhouse with it. We do not >>>>>>> know if we will then end up in Jailhouse in hyp mode and just lack >>>>>>> console output or if we crash on entry already. >>>>>>> >>>>>> Right agreed. >>>>>> >>>>>>> To move the uart console out of the picture: Did you already check if >>>>>>> the driver you select in the Jailhouse config is actually one that >>>>>>> should support the UART on your board? Next is to double check if poking >>>>>> The UART on this platform is almost identical to >>>>>> JAILHOUSE_CON_TYPE_SCIFA type, but with some differences which I have >>>>>> patched to work on this platform. >>>>>> >>>>>>> registers in the way the Jailhouse driver will do at the addresses you >>>>>>> configured will work: Pull the code into the kernel module or even into >>>>>>> a userspace application with /dev/mem raw register access and try out if >>>>>>> that works in a "safe" environment (without hypervisor mode). >>>>>>> >>>>>> Sure will give that a shot, any pointers on doing this from userspace? >>>>>> >>>>> >>>>> Something like this may help if you do that the first time: >>>>> https://bakhi.github.io/devmem/ >>>>> >>>> Thanks for the pointer. >>>> I have tried reading/writing from the hypervisor location and that >>>> goes all OK. >>> >>> Means, you were able to generate output on the UART. Hmm, would have >>> been too easy. >>> >> No I meant I was able to read/write into the hypervisor memory which >> is reserved in DTS. >> >>>> To avoid any issue related to debug UART is there any way >>>> I could test this prior to enabling? >>> >>> Not without extra measures: Without JAILHOUSE_BORROW_ROOT_PT, which >>> applies to arm64, the driver will not map the physical UART page. That >>> only happens in init_bootstrap_pt which is run after jumping to EL2. So, >>> we the jump goes wrong, you cannot find out where you are. >>> >> I see. Just to confirm it's not the debug UART the watchdog is enabled >> in Linux and I see the platform reboots after 60 seconds, which is >> hinting we are seeing a kernel freeze. >> >> Just a fyi I tried the queues/jailhouse branch from [0] and still >> seeing the same issue. >> >>> Do you have the chance to get hold of some jtag to find out where the >>> CPUs are? >>> > > X0 FFFF00063F7ECD88 X16 0 ^S+ ^Stack_________+ > X1 0 X17 0 > X2 0 X18 FFFFFFFFFFFFFFFF > X3 FFFF8000112870E8 X19 80 > X4 FFFF00063F7ECD80 X20 FFFF800011179000 > X5 FFFF800011179A48 X21 FFFF80001130BE70 > X6 FFFF80001101E000 X22 FFFF800010DFDED8 > X7 FFFF800011179000 X23 86000004 > X8 FFFF00063F7ECD80 X24 FFFF80001101CB38 > X9 0 X25 FFFF800011308000 > X10 00040000 X26 FFFF80001130C000 > X11 0 X27 FFFF800011182A40 > X12 0 X28 FFFF800011182A40 > X13 FFFFFFFFFFFE0000 X29 FFFF80001130BC40 > X14 FFFF800011192008 X30 FFFF800010B3B464 > X15 FFFF800011192048 PC FFFF8000100D9F78 > -------------------------------------------- > CPSR 80000085 N N I I SS _ > EL1h Z _ F _ IL _ > nsec C _ A _ > V _ D _ > -------------------------------------------- > Current ELx: SP FFFF80001130BC40 > ELR 0 > SPSR 20000085 > -------------------------------------------- > EL0: EL1: > SP FFFF800011182A40 SP FFFF80001130BC40 > ELR 0 > SPSR 20000085 > > EL2: EL3: > SP 0000FF0000011000 SP 4400A500 > ELR FFFF8000104CC8A8 ELR 0000FFFFC02064AC > SPSR 20000085 SPSR 200003C9 > -------------------------------------------- > SPSR_ABT 0 SPSR_SVC 20000085 > SPSR_IRQ 0 SPSR_HYP 20000085 > SPSR_FIQ 0 > SPSR_UND 0 ELR_HYP 104CC8A8 > > > Above is the CPU state, when the kernel freezes. Any hints on what > might have happened?
Can you correlate the PC value with the hypervisor disassembly? Or are we actually back in Linux? Current SP == EL1 SP... If that should be the case, you could use the JTAG to "trace" how far you get: Add an infinite loop at some point the setup should get along, and then check with the debugger if PC points to that address and if EL2 is active. With that, you could also check if the UART print-out is executed. Jan -- Siemens AG, Technology Competence Center Embedded Linux -- You received this message because you are subscribed to the Google Groups "Jailhouse" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/jailhouse-dev/5172e723-49ce-a870-2066-d22f44115da3%40siemens.com.
