Hi Jan,
On Tue, May 24, 2022 at 12:55 PM Lad, Prabhakar
<[email protected]> wrote:
>
> On Mon, May 23, 2022 at 4:13 PM Jan Kiszka <[email protected]> wrote:
> >
> > On 23.05.22 15:55, Lad, Prabhakar wrote:
> > > Hi Jan,
> > >
> > > On Fri, May 20, 2022 at 7:08 AM Jan Kiszka <[email protected]> wrote:
> > >>
> > >> On 19.05.22 15:23, Lad, Prabhakar wrote:
> > >>> Hi Jan,
> > >>>
> > >>> On Thu, May 19, 2022 at 12:39 PM Jan Kiszka <[email protected]>
> > >>> wrote:
> > >>>>
> > >>>> On 19.05.22 11:44, Lad, Prabhakar wrote:
> > >>>>> On Thu, May 19, 2022 at 6:30 AM Jan Kiszka <[email protected]>
> > >>>>> wrote:
> > >>>>>> Forgot: that cannot work. The call of arm_dcaches_flush will
> > >>>>>> overwrite
> > >>>>>> lr, thus the second ret will only return to where the first ret
> > >>>>>> jumped
> > >>>>>> to - endless loop. You would have to restore lr (x30) from x17 in
> > >>>>>> arch_entry first:
> > >>>>>>
> > >>>>>> mov x30, x17
> > >>>>>> ret
> > >>>>>>
> > >>>>> That did the trick thanks!
> > >>>>>
> > >>>>> diff --git a/hypervisor/arch/arm64/entry.S
> > >>>>> b/hypervisor/arch/arm64/entry.S
> > >>>>> index a9cabf7f..7b340bd1 100644
> > >>>>> --- a/hypervisor/arch/arm64/entry.S
> > >>>>> +++ b/hypervisor/arch/arm64/entry.S
> > >>>>> @@ -109,6 +109,8 @@ arch_entry:
> > >>>>> mov x0, #LINUX_HVC_SET_VECTORS_LEGACY
> > >>>>> 1:
> > >>>>> hvc #0
> > >>>>> + mov x30, x17
> > >>>>> + ret
> > >>>>>
> > >>>>> hvc #0 /* bootstrap vectors enter EL2 at el2_entry */
> > >>>>> b . /* we don't expect to return here */
> > >>>>>
> > >>>>>
> > >>>>> With the above diff I do get the below:
> > >>>>>
> > >>>>> [ 42.980805] jailhouse: loading out-of-tree module taints kernel.
> > >>>>> Reading configuration set:
> > >>>>> Root cell: Renesas RZ/V2L SMARC (renesas-r9a07g054l2.cell)
> > >>>>> Overlapping memory regions inside cell: None
> > >>>>> Overlapping memory regions with hypervisor: None
> > >>>>> Missing resource interceptions for architecture arm64: None
> > >>>>> [ 46.582588] obcode @arm_dcaches_flush: d53b0024
> > >>>>> [ 46.582616] obcode @arm_dcaches_flush: d53b0024
> > >>>>> [ 46.611311] The Jailhouse is opening.
> > >>>>>
> > >>>>> So it looks like something to do with the debug console. This has to
> > >>>>> be poked in the dark or any easy way to debug?
> > >>>>
> > >>>> Well, we do not yet know what goes wrong. We do know that we can call
> > >>>> into the hyp take-over stub and register Jailhouse with it. We do not
> > >>>> know if we will then end up in Jailhouse in hyp mode and just lack
> > >>>> console output or if we crash on entry already.
> > >>>>
> > >>> Right agreed.
> > >>>
> > >>>> To move the uart console out of the picture: Did you already check if
> > >>>> the driver you select in the Jailhouse config is actually one that
> > >>>> should support the UART on your board? Next is to double check if
> > >>>> poking
> > >>> The UART on this platform is almost identical to
> > >>> JAILHOUSE_CON_TYPE_SCIFA type, but with some differences which I have
> > >>> patched to work on this platform.
> > >>>
> > >>>> registers in the way the Jailhouse driver will do at the addresses you
> > >>>> configured will work: Pull the code into the kernel module or even into
> > >>>> a userspace application with /dev/mem raw register access and try out
> > >>>> if
> > >>>> that works in a "safe" environment (without hypervisor mode).
> > >>>>
> > >>> Sure will give that a shot, any pointers on doing this from userspace?
> > >>>
> > >>
> > >> Something like this may help if you do that the first time:
> > >> https://bakhi.github.io/devmem/
> > >>
> > > Thanks for the pointer.
> > > I have tried reading/writing from the hypervisor location and that
> > > goes all OK.
> >
> > Means, you were able to generate output on the UART. Hmm, would have
> > been too easy.
> >
> No I meant I was able to read/write into the hypervisor memory which
> is reserved in DTS.
>
> > > To avoid any issue related to debug UART is there any way
> > > I could test this prior to enabling?
> >
> > Not without extra measures: Without JAILHOUSE_BORROW_ROOT_PT, which
> > applies to arm64, the driver will not map the physical UART page. That
> > only happens in init_bootstrap_pt which is run after jumping to EL2. So,
> > we the jump goes wrong, you cannot find out where you are.
> >
> I see. Just to confirm it's not the debug UART the watchdog is enabled
> in Linux and I see the platform reboots after 60 seconds, which is
> hinting we are seeing a kernel freeze.
>
> Just a fyi I tried the queues/jailhouse branch from [0] and still
> seeing the same issue.
>
> > Do you have the chance to get hold of some jtag to find out where the
> > CPUs are?
> >
X0 FFFF00063F7ECD88 X16 0 ^S+ ^Stack_________+
X1 0 X17 0
X2 0 X18 FFFFFFFFFFFFFFFF
X3 FFFF8000112870E8 X19 80
X4 FFFF00063F7ECD80 X20 FFFF800011179000
X5 FFFF800011179A48 X21 FFFF80001130BE70
X6 FFFF80001101E000 X22 FFFF800010DFDED8
X7 FFFF800011179000 X23 86000004
X8 FFFF00063F7ECD80 X24 FFFF80001101CB38
X9 0 X25 FFFF800011308000
X10 00040000 X26 FFFF80001130C000
X11 0 X27 FFFF800011182A40
X12 0 X28 FFFF800011182A40
X13 FFFFFFFFFFFE0000 X29 FFFF80001130BC40
X14 FFFF800011192008 X30 FFFF800010B3B464
X15 FFFF800011192048 PC FFFF8000100D9F78
--------------------------------------------
CPSR 80000085 N N I I SS _
EL1h Z _ F _ IL _
nsec C _ A _
V _ D _
--------------------------------------------
Current ELx: SP FFFF80001130BC40
ELR 0
SPSR 20000085
--------------------------------------------
EL0: EL1:
SP FFFF800011182A40 SP FFFF80001130BC40
ELR 0
SPSR 20000085
EL2: EL3:
SP 0000FF0000011000 SP 4400A500
ELR FFFF8000104CC8A8 ELR 0000FFFFC02064AC
SPSR 20000085 SPSR 200003C9
--------------------------------------------
SPSR_ABT 0 SPSR_SVC 20000085
SPSR_IRQ 0 SPSR_HYP 20000085
SPSR_FIQ 0
SPSR_UND 0 ELR_HYP 104CC8A8
Above is the CPU state, when the kernel freezes. Any hints on what
might have happened?
Cheers,
Prabhakar
--
You received this message because you are subscribed to the Google Groups
"Jailhouse" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/jailhouse-dev/CA%2BV-a8uA%2By-p5GmYavLpc6s1O-TJiRGSkpRHM4-dEA%3DXsqU_mA%40mail.gmail.com.
