Currently I have a problem when enabling the root cell in QEMU, which I couldn't investigate further yet. Hardware Debugger should be available soon, though.
root@imx8mm:~# jailhouse enable /opt/jailhouse/configs/imx8mm.cell [ 77.686189] Synchronous External Abort: synchronous external abort (0x96000050) at 0xffff00000f800000 [ 77.688767] Internal error: : 96000050 [#1] PREEMPT SMP [ 77.690225] Modules linked in: jailhouse(O) crc32_ce crct10dif_ce ip_tables x_tables [ 77.694196] CPU: 2 PID: 3069 Comm: jailhouse Tainted: G O 4.14.78-05595-g94da7bdc489b #4 [ 77.697023] Hardware name: linux,dummy-virt (DT) [ 77.698221] task: ffff800020543600 task.stack: ffff00000f410000 [ 77.701799] PC is at __memcpy+0x110/0x180 [ 77.703481] LR is at jailhouse_cmd_enable+0x2e0/0x6c8 [jailhouse] [ 77.705048] pc : [<ffff000008f34890>] lr : [<ffff000000fbd898>] pstate: 20000145 [ 77.706912] sp : ffff00000f413b50 [ 77.707701] x29: ffff00000f413b50 x28: ffff800020543600 [ 77.709647] x27: 00000000000003be x26: ffff00000f535000 [ 77.710734] x25: ffff0000097e1188 x24: ffff000000fc2880 [ 77.712065] x23: ffff000000fc2c10 x22: ffff000000fc2070 [ 77.713320] x21: 00000000ffffffea x20: 0000000033b51010 [ 77.714700] x19: 0000000000000004 x18: 0000000000040e4e [ 77.715616] x17: 0000000000413048 x16: ffff000008279c00 [ 77.717309] x15: 000000000000066f x14: 0000000000000000 3: 0000000000000000 x12: 0000000000000000 3 13:.7189728:31 .5].. x1 kernel:[[ 77.723158] x1 1: 0000000000015000 77.688767] Inter x10: 0000na000000004000 l error: : 96000050 [#1] PREEMPT SMP [ 77.726580] x9 : 0000000000004000 x8 : 000000000001d000 [ 77.731462] x7 : 53554f484c49414a x6 : ffff00000f800000 [ 77.731987] x5 : ffff00000fc00000 x4 : 0000000000000000 [ 77.733828] x3 : ffff00000fbfffff x2 : 0000000000015788 [ 77.735039] x1 : ffff00000f535040 x0 : ffff00000f800000 [ 77.736373] Process jailhouse (pid: 3069, stack limit = 0xffff00000f410000) [ 77.737641] Call trace: [ 77.738452] Exception stack(0xffff00000f413a10 to 0xffff00000f413b50) [ 77.739382] 3a00: ffff00000f800000 ffff00000f535040 [ 77.741931] 3a20: 0000000000015788 ffff00000fbfffff 0000000000000000 ffff00000fc00000 [ 77.743681] 3a40: ffff00000f800000 53554f484c49414a 000000000001d000 0000000000004000 [ 77.745714] 3a60: 0000000000004000 0000000000015000 0000000000000000 0000000000000000 [ 77.746888] 3a80: 0000000000000000 000000000000066f ffff000008279c00 0000000000413048 [ 77.747905] 3aa0: 0000000000040e4e 0000000000000004 0000000033b51010 00000000ffffffea [ 77.749358] 3ac0: ffff000000fc2070 ffff000000fc2c10 ffff000000fc2880 ffff0000097e1188 [ 77.750481] 3ae0: ffff00000f535000 00000000000003be ffff800020543600 ffff00000f413b50 [ 77.751226] 3b00: ffff000000fbd898 ffff00000f413b50 ffff000008f34890 0000000020000145 [ 77.752980] 3b20: ffff00000f413b50 ffff000000fbd874 0000ffffffffffff 0000000033b51010 [ 77.754323] 3b40: ffff00000f413b50 ffff000008f34890 [ 77.755333] [<ffff000008f34890>] __memcpy+0x110/0x180 [ 77.756909] [<ffff000000fbdd0c>] jailhouse_ioctl+0x8c/0x118 [jailhouse] [ 77.758048] [<ffff000008291d30>] do_vfs_ioctl+0xb0/0x868 [ 77.758584] [<ffff00000829257c>] SyS_ioctl+0x94/0xa8 [ 77.759050] Exception stack(0xffff00000f413ec0 to 0xffff00000f414000) [ 77.760369] 3ec0: 0000000000000003 0000000040080000 0000000033b51010 0000000000000000 [ 77.761450] 3ee0: 0000000033b51010 00000000000003d1 0000ffff8b4460d0 0000000033b72fff [ 77.762404] 3f00: 000000000000001d 0000ffff8b4469f0 0101010101010101 0000000000000018 [ 77.763412] 3f20: 00000000000003f3 0000000000000000 0000000000000000 000000000000051b [ 77.765363] 3f40: 0000ffff8b3c3150 0000000000413108 0000000000040e4e 0000ffffe934fee3 [ 77.766389] 3f60: 0000000033b51010 0000000000000003 0000000000000000 0000000000000000 [ 77.767384] 3f80: 0000000000000000 0000000000000000 0000000000000000 0000000000000000 [ 77.769682] 3fa0: 0000000000000000 0000ffffe934f420 0000000000401014 0000ffffe934f420 [ 77.770469] 3fc0: 0000ffff8b3c315c 0000000080000000 0000000000000003 000000000000001d [ 77.770915] 3fe0: 0000000000000000 0000000000000000 0000000000000000 0000000000000000 [ 77.771350] [<ffff000008083ac0>] el0_svc_naked+0x34/0x38 [ 77.772570] Code: a8c12027 a8c12829 a8c1302b a8c1382d (a88120c7) [ 77.773509] ---[ end trace b45069ae9849cd95 ]--- Segmentation fault -----Ursprüngliche Nachricht----- Von: Jan Kiszka [mailto:[email protected]] Gesendet: Dienstag, 23. Juli 2019 12:20 An: von Wiarda, Jan; Antonios Motakis (Tony); Mark Rutland Cc: JailhouseMailingListe Betreff: Re: 64 bit Hypervisor crash at 32 bit WFI instruction On 23.07.19 12:14, von Wiarda, Jan wrote: > Hi! > > With > > asm volatile("nop" : : : "memory"); > > instead of > > asm volatile("wfi" : : : "memory"); > > it runs just fine. > >> Is the root cell cpu (CPU 0) specifically crashing with an unexpected >> synchronous exit to Jailhouse? What is the output? > > No, CPU 0 does not crash with any kind of console output, which makes > debugging even more difficult. What I observe is, that after hitting WFI, it > continues to run for a 1-2 seconds and then it stops. Last thing I see from > the instrumented code is a printk() from arch_skip_instruction(), which means > it was handling a SYS64 exit. Maybe interrupts get stalled for the root cell - for whatever reason. Do you have a hardware debugger to analyze the state of the CPUs? Or use QEMU... Jan > >> This is a far shot, but maybe the code generated around the WFI is the >> culprit? > > You might be right, when I place WFI right after inmate_main(), CPU 0 does > not starve. But it's completely strange and undefined behaviour, sometimes it > crashes if I put the WFI right after a printk(), whereas right before the > printk() it doesn't crash. > > Works: > > void inmate_main(void) > { > ... > asm volatile("wfi" : : : "memory"); > printk("IVSHMEM: Done setting up...\n"); > printk("IVSHMEM: waiting for interrupt.\n"); > //asm volatile("wfi" : : : "memory"); > } > > Does not work: > > void inmate_main(void) > { > ... > //asm volatile("wfi" : : : "memory"); > printk("IVSHMEM: Done setting up...\n"); > printk("IVSHMEM: waiting for interrupt.\n"); > asm volatile("wfi" : : : "memory"); > } > > I know this sounds completely strange but I reproduced this multiple times, > compiler is this: > > gcc version 6.3.0 20170516 (Debian 6.3.0-18) > > BR, > Jan > > -----Ursprüngliche Nachricht----- > Von: Antonios Motakis (Tony) [mailto:[email protected]] > Gesendet: Dienstag, 23. Juli 2019 06:40 > An: von Wiarda, Jan; Mark Rutland > Cc: JailhouseMailingListe; Jan Kiszka > Betreff: Re: AW: 64 bit Hypervisor crash at 32 bit WFI instruction > > Hi Jan, > > On 22-Jul-19 7:11 PM, von Wiarda, Jan wrote: >> Hi Mark, >> >> I'm not touching bit 13 or 14 in HCR_EL2, they're both 0. HCR_EL2 is the >> same for 64 bit and 32 bit inmates when the crash happens, except for >> HCR_RW_BIT, obviously. HCR_EL2 value is 0x28001B at crash time. >> > > It's quite an interesting crash that you have there; I wouldn't expect this > to happen. > > The idea with trapping WFI/WFE is to be able to suspend a VM that is just > waiting for something to happen. Since Jailhouse is a partitioning > hypervisor, you shouldn't need to trap it, nor should its use normally > influence the other cores. Yet something is amiss here. > > Is the root cell cpu (CPU 0) specifically crashing with an unexpected > synchronous exit to Jailhouse? What is the output? > > I don't remember what event 0x28001B maps to, I would check the ARM ARM first > to figure out what the unexpected event in CPU 0 was, for a clue to motivate > further investigation. > > Additionally, this WFI code instructs the compiler that memory contents may > change, so ordering of generated instructions, inserted barriers etc, are > influenced. This is a far shot, but maybe the code generated around the WFI > is the culprit? Maybe not, but I would try to rule it out: > (a) First I'd try replacing the WFI with a nop, to observe the behavior > without the WFI but without changing compiler behavior and maintaining any > compiler barriers. > (b) I would also try replacing it with an infinite loop ("b .") to get the > inmate to wait forever at this position, and see what happens. > > Happy debugging :) > > Best regards, > Tony > -- Siemens AG, Corporate Technology, CT RDA IOT SES-DE Corporate Competence Center Embedded Linux -- You received this message because you are subscribed to the Google Groups "Jailhouse" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/jailhouse-dev/95F51F4B902CAC40AF459205F6322F01C4EE0E3E80%40BMK019S01.emtrion.local.
<<attachment: winmail.dat>>
