Hi Jan,
On Thu, May 12, 2022 at 11:24 AM Jan Kiszka <[email protected]> wrote:
>
> On 12.05.22 09:01, Lad, Prabhakar wrote:
> > Hi Jan,
> >
> > On Thu, May 12, 2022 at 6:45 AM Jan Kiszka <[email protected]> wrote:
> >>
> >> On 11.05.22 19:09, Lad, Prabhakar wrote:
> >>> Hi Jan,
> >>>
> >>> On Wed, May 11, 2022 at 4:11 PM Jan Kiszka <[email protected]> wrote:
> >>>>
> >>>> On 11.05.22 13:20, Prabhakar Lad wrote:
> >>>>> To add further more details:
> >>>>>
> >>>>> I am using jailhouse-enabling/5.10 Linux branch [0] with -next branch
> >>>>> for jailhouse [1].
> >>>>>
> >>>>> I added some debug prints and I see the panic is caused when entry()
> >>>>> function is called (in enter_hypervisor). The entry function lands into
> >>>>> assembly code (entry.S). I dont have a JTAG to see which exact
> >>>>> instruction is causing this issue.
> >>>>
> >>>> So, already the first instruction in the loaded hypervisor binary is not
> >>>> executable? That would explain why we see no hypervisor output at all.
> >>>>
> >>> To clarify when the hypervisor is loaded the output will be via
> >>> debug_console specified in the root cell config?
> >>>
> >>
> >> Correct - if you reach entry() in setup.c.
> >>
> >>>> Was that memory region properly reserved from Linux (via DTB carve-out
> >>>> e.g.)?
> >>>>
> >>> Yes I have the below memory reserved in my dts:
> >>>
> >>> memory@48000000 {
> >>> device_type = "memory";
> >>> /* first 128MB is reserved for secure area. */
> >>> reg = <0x0 0x48000000 0x0 0x78000000>;
> >>> };
> >>>
> >>> reserved-memory {
> >>> #address-cells = <2>;
> >>> #size-cells = <2>;
> >>> ranges;
> >>>
> >>> jh_inmate@a7f00000 {
> >>> status = "okay";
> >>> no-map;
> >>> reg = <0x00 0xa7f00000 0x00 0xf000000>;
> >>> };
> >>>
> >>> jailhouse: jailhouse@b6f00000 {
> >>> status = "okay";
> >>> reg = <0x0 0xb6f00000 0x0 0x1000000>;
> >>> no-map;
> >>> };
> >>> };
> >>>
> >>> Linux does report the hole in RAM:
> >>>
> >>> root@smarc-rzg2l:~# cat /proc/iomem | grep RAM
> >>> 48000000-a7efffff : System RAM
> >>> b7f00000-bfffffff : System RAM
> >>>
> >>> And below is my root cell config related to hypervisor memory:
> >>>
> >>> .hypervisor_memory = {
> >>> .phys_start = 0xb6f00000,
> >>> .size = 0x1000000,
> >>> },
> >>>
> >>> Is there anything obvious I have missed above?
> >>>
> >>
> >> Nothing obvious to me so far.
> >>
> >> So, is this really the first instruction in
> >> hypervisor/arch/arm64/entry.S, arch_entry, that triggers the undefinstr?
> >> Check the reported address by Linux against the disassembly of the
> >> hypervisor. You could also play with adding 'ret' as first instruction,
> >> to check if that returns without a crash.
> >>
> > I did play around with ret, below is my observation:
> >
> > Up until line 98 [0] just before calling arm_dcaches_flush adding ret
> > returned without a crash. Adding a ret at line 99 [1] ie after
> > arm_dcaches_flush it never returned.
> >
> > So I continued with adding ret in arm_dcaches_flush, I added ret as a
> > first statement in arm_dcaches_flush and was able to see the panic!
>
> Which Jailhouse revision are you building? Already disassembled
> hypervisor.o around arch_entry and arm_dcaches_flush? This is what I
> have here for next:
>
I'm using the next branch too.
> 0000ffffc0203efc <arm_dcaches_flush>:
> ffffc0203efc: d53b0024 mrs x4, ctr_el0
> ffffc0203f00: d3504c84 ubfx x4, x4, #16, #4
> ...
>
> 0000ffffc0204800 <arch_entry>:
> ffffc0204800: aa0003f0 mov x16, x0
> ffffc0204804: aa1e03f1 mov x17, x30
> ...
> ffffc0204844: d2800042 mov x2, #0x2
> // #2
> ffffc0204848: 97fffdad bl ffffc0203efc
> <arm_dcaches_flush>
>
> You could check if there is such a relative call in your case as well.
yes it does exist, below is the snippet:
0000ffffc0204000 <arch_entry>:
ffffc0204000: aa0003f0 mov x16, x0
ffffc0204004: aa1e03f1 mov x17, x30
ffffc0204008: 10fdffc0 adr x0, ffffc0200000 <hypervisor_header>
ffffc020400c: f9402412 ldr x18, [x0, #72]
ffffc0204010: 5800fd0f ldr x15, ffffc0205fb0 <sdei_handler+0x2c>
ffffc0204014: 900000e1 adrp x1, ffffc0220000 <__page_pool>
ffffc0204018: 79406002 ldrh w2, [x0, #48]
ffffc020401c: d2880003 mov x3, #0x4000
// #16384
ffffc0204020: 9b030441 madd x1, x2, x3, x1
ffffc0204024: f842c02e ldur x14, [x1, #44]
ffffc0204028: 5800fc8d ldr x13, ffffc0205fb8 <sdei_handler+0x34>
ffffc020402c: f840c02c ldur x12, [x1, #12]
ffffc0204030: cb0d018b sub x11, x12, x13
ffffc0204034: 924051c1 and x1, x14, #0x1fffff
ffffc0204038: 8b0101ef add x15, x15, x1
ffffc020403c: f9001c0f str x15, [x0, #56]
ffffc0204040: f9400401 ldr x1, [x0, #8]
ffffc0204044: d2800042 mov x2, #0x2 // #2
ffffc0204048: 97ffff6c bl ffffc0203df8 <arm_dcaches_flush>
ffffc020404c: 5800fba1 ldr x1, ffffc0205fc0 <sdei_handler+0x3c>
ffffc0204050: 8b0b0021 add x1, x1, x11
ffffc0204054: d2800000 mov x0, #0x0 // #0
ffffc0204058: f100025f cmp x18, #0x0
ffffc020405c: 54000041 b.ne ffffc0204064
<arch_entry+0x64> // b.any
ffffc0204060: d2800020 mov x0, #0x1 // #1
ffffc0204064: d4000002 hvc #0x0
ffffc0204068: d4000002 hvc #0x0
ffffc020406c: 14000000 b ffffc020406c <arch_entry+0x6c>
> Then you could check, before jumping into arch_entry from the
> hypervisor, that the opcode at the actual arm_dcaches_flush address is
> as expected. But maybe already the building injects an issue here.
>
Any pointers on how I could do that?
Cheers,
Prabhakar
--
You received this message because you are subscribed to the Google Groups
"Jailhouse" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/jailhouse-dev/CA%2BV-a8sgPXsMmsKYwE6etP%3DJG8Ef83Qz4RHoWAZKeHdVHhk_Pw%40mail.gmail.com.
