Le vendredi 24 novembre 2023 à 13:35:29 UTC+1, Ralf Ramsauer a écrit :
On 24/11/2023 09:18, Laurent Corbin wrote:
> Hi,
>
>
> Thanks for your responsiveness. Unfortunately, I have not any output when
I
> apply your patch. With more traces, I can see that only PSCI for CPU
> suspend is
> used. I'm continuing my research to find out more about this kernel
panic.
> Do you have any new suggestions or hypotheses ?
One further thought, see below.
>
> Thanks in advance,
>
> Le mercredi 22 novembre 2023 à 16:34:10 UTC+1, Ralf Ramsauer a écrit :
>
> Hi,
>
> On 22/11/2023 16:14, Laurent Corbin wrote:
> > Hi all,
> >
> > I'm trying to run jailhouse on LX2160ardb (NXP board). To do that,
> > 1. I wrote a cell file for my root cell.
> > 2. I built a Linux 6.1 and a jailhouse (branch lf-6.1.36_2.1.0
> from nxp
> > mirror) with yocto.
> > 3. I loaded the jailhouse driver as a kernel module.
> >
> > But when I try to "enable" jailhouse I get this error :
>
> okay, I think i have seen a similar error before. Would you please try
> to apply this patch, and see if there is some output on the console:
>
> diff --git a/hypervisor/arch/arm-common/smccc.c
> b/hypervisor/arch/arm-common/smccc.c
> index 65639b59..afa26e53 100644
> --- a/hypervisor/arch/arm-common/smccc.c
> +++ b/hypervisor/arch/arm-common/smccc.c
> @@ -83,6 +83,7 @@ static inline long handle_arch_features(u32 id)
> return this_cpu_data()->smccc_feat_workaround_2;
>
> default:
> + printk("Unknown SMCCC!\n");
> return ARM_SMCCC_NOT_SUPPORTED;
> }
> }
> @@ -129,6 +130,7 @@ enum trap_return handle_smc(struct trap_context
> *ctx)
> case ARM_SMCCC_OWNER_SIP:
> stats[JAILHOUSE_CPU_STAT_VMEXITS_SMCCC]++;
> regs[0] = ARM_SMCCC_NOT_SUPPORTED;
> + printk("Unknown SMCCC (SIP)!\n");
> break;
>
> case ARM_SMCCC_OWNER_STANDARD:
>
>
> If you get an output here, then we need to probably forward those
> calls.
> Please get back to me in that case.
>
> Thanks
> Ralf
>
> >
> >
> >
> > root@lx2160ardb-rev2:~# insmod jailhouse.ko
> > root@lx2160ardb-rev2:~#
> > root@lx2160ardb-rev2:~#
> > root@lx2160ardb-rev2:~#
> > root@lx2160ardb-rev2:~# cat /sys/module/jailhouse/sections/.text
> > 0xffffc45e1113e000
> > root@lx2160ardb-rev2:~# jailhouse enable lx2160a-rdb.cell
> >
> > Initializing Jailhouse hypervisor v0.12 (369-g6a87b739) on CPU 0
> > Code location: 0x0000ffffc0200800
> > Page pool usage after early setup: mem 151/993, remap 0/131072
> > Initializing processors:
> > CPU 0... OK
> > CPU 1... OK
> > CPU 5... OK
> > CPU 4... OK
> > CPU 2... OK
> > CPU 3... OK
> > CPU 9... OK
> > CPU 8... OK
> > CPU 15... OK
> > CPU 13... OK
> > CPU 12... OK
> > CPU 7... OK
> > CPU 10... OK
> > CPU 11... OK
> > CPU 6... OK
> > CPU 14... OK
> > Initializing unit: irqchip
> > Initializing unit: ARM SMMU v3
> > Initializing unit: ARM SMMU
> > ARM MMU500 at 0x5000000 with:
> > stream matching with 128 SMR groups
> > 64 context banks (0 stage-2 only)
> > Initializing unit: PVU IOMMU
> > Initializing unit: PCI
> > Adding virtual PCI device 00:00.0 to cell "lx2160a"
> > Adding virtual PCI device 00:01.0 to cell "lx2160a"
> > Page pool usage after late setup: mem 228/993, remap 2576/131072
> > Activating hypervisor
> > [ 173.577969] SError Interrupt on CPU0, code 0x00000000bf000000
> -- SError
SError is a "System Error". See [1] what triggers a SError. I wonder why
this happens behind address 0xbf000000, right after enabling Jailhouse
(which works correctly). What is behind that physical address? Is this
the Jailhouse physical location?
Thanks for the document reference.
For me, 0xbf000000 is not an adress but the value of the register esr_el1
(Exception Syndrome Register EL1) as you can see in linux kernel code :
- arm64_serror_panic(struct pt_regs *regs, unsigned long esr) in
arch/arm64/kernel/traps.c which display this trace,
- el1h_64_error_handler() function in arch/arm64/kernel/entry-common.c
which read the register.
When I decode this register (with https://esr.arm64.dev/?#0xbf000000), I
have
not any additional information about why the SError occurs.
Do you have all Jailhouse patches applied on your kernel? Are memory
regions reserved (via mem= option or device tree)?
I used the NXP linux repo (https://github.com/nxp-qoriq/linux.git
branch=lf-6.1.y) which
contains Jailhouse patches.
About memory reservation, I add this node in my device tree :
reserved-memory {
#address-cells = <0x02>;
#size-cells = <0x02>;
ranges;
hypervisor@0 {
no-map;
reg = <0x00 0xa0000000 0x00 0x10000000>;
};
};
and the config.header in my cell file is :
.header = {
.signature = JAILHOUSE_SYSTEM_SIGNATURE,
.revision = JAILHOUSE_CONFIG_REVISION,
.architecture = JAILHOUSE_ARM64,
.flags = JAILHOUSE_SYS_VIRTUAL_DEBUG_CONSOLE,
.hypervisor_memory = {
.phys_start = 0xa0000000,
.size = 0x00400000,
},
.debug_console = {
.address = 0x21c0000,
.size = 0x1000,
.type = JAILHOUSE_CON_TYPE_PL011,
.flags = JAILHOUSE_CON_ACCESS_MMIO |
JAILHOUSE_CON_REGDIST_4,
},
.platform_info = {
.pci_mmconfig_base = 0xfc000000,
.pci_mmconfig_end_bus = 0,
.pci_is_virtual = 1,
.pci_domain = -1,
.iommu_units = {
{
.type = JAILHOUSE_IOMMU_ARM_MMU500,
.base = 0x5000000,
.size = 0x800000,
},
},
.arm = {
.gic_version = 3,
.gicd_base = 0x6000000,
.gicr_base = 0x6200000,
.gicc_base = 0xc0c0000,
.gich_base = 0xc0d0000,
.gicv_base = 0xc0e0000,
.maintenance_irq = 25,
},
},
.root_cell = {
.name = "lx2160a",
.num_pci_devices = ARRAY_SIZE(config.pci_devices),
.cpu_set_size = sizeof(config.cpus),
.num_memory_regions = ARRAY_SIZE(config.mem_regions),
.num_irqchips = ARRAY_SIZE(config.irqchips),
.vpci_irq_base = 160 - 32,
},
},
Is there an error in my configuration ?
Ralf
[1]
https://developer.arm.com/documentation/102412/0103/Exception-types/Asynchronous-exceptions
> > [ 173.577985] CPU: 0 PID: 0 Comm: swapper/0 Tainted: G O
> > 6.1.22+g66e442bc7fdc #1
> > [ 173.577993] Hardware name: NXP Layerscape LX2160ARDB (DT)
> > [ 173.577996] pstate: 200000c5 (nzCv daIF -PAN -UAO -TCO -DIT -SSBS
> > BTYPE=--)
> > [ 173.578002] pc : enter_hypervisor+0x44/0xa4 [jailhouse]
> > [ 173.578025] lr : enter_hypervisor+0x44/0xa4 [jailhouse]
> > [ 173.578040] sp : ffff800008003f10
> > [ 173.578042] x29: ffff800008003f10 x28: ffffc45e51736c40 x27:
> > 00000000fbd0ea70
> > [ 173.578052] x26: 0000000000000001 x25: 00000028652e0c1e x24:
> > 00000028652b7617
> > [ 173.578060] x23: 0000000000000000 x22: 0000000000000000 x21:
> > 0000000000000000
> > [ 173.578067] x20: 0000000000000000 x19: ffffc45e11144000 x18:
> > 0000000000000000
> > [ 173.578074] x17: 0000000000000000 x16: 0000000000000000 x15:
> > 0000000000000000
> > [ 173.578080] x14: 0000000000000000 x13: 0000000000000000 x12:
> > 0000000000000000
> > [ 173.578086] x11: 0000000000000000 x10: 0000000000000000 x9 :
> > 0000000000000000
> > [ 173.578092] x8 : 0000000000000000 x7 : 0000000000000000 x6 :
> > 0000000000000000
> > [ 173.578098] x5 : 0000000000000000 x4 : 0000000000000000 x3 :
> > 0000000000000000
> > [ 173.578104] x2 : 0000000000000000 x1 : 0000000000000000 x0 :
> > 0000000000000000
> > [ 173.578111] Kernel panic - not syncing: Asynchronous SError
> Interrupt
> > [ 173.578115] CPU: 0 PID: 0 Comm: swapper/0 Tainted: G O
> > 6.1.22+g66e442bc7fdc #1
> > [ 173.578120] Hardware name: NXP Layerscape LX2160ARDB (DT)
> > [ 173.578122] Call trace:
> > [ 173.578124] dump_backtrace.part.0+0xe0/0xf0
> > [ 173.578140] show_stack+0x18/0x30
> > [ 173.578148] dump_stack_lvl+0x64/0x80
> > [ 173.578158] dump_stack+0x18/0x34
> > [ 173.578166] panic+0x188/0x340
> > [ 173.578173] nmi_panic+0xac/0xb0
> > [ 173.578179] arm64_serror_panic+0x6c/0x7c
> > [ 173.578184] do_serror+0x58/0x5c
> > [ 173.578188] el1h_64_error_handler+0x30/0x4c
> > [ 173.578193] el1h_64_error+0x64/0x68
> > [ 173.578198] enter_hypervisor+0x44/0xa4 [jailhouse]
> > [ 173.578213] __flush_smp_call_function_queue+0xd0/0x250
> > [ 173.578223] generic_smp_call_function_single_interrupt+0x14/0x20
> > [ 173.578232] ipi_handler+0x98/0x160
> > [ 173.578239] handle_percpu_devid_irq+0x84/0x130
> > [ 173.578245] generic_handle_domain_irq+0x2c/0x4c
> > [ 173.578251] gic_handle_irq+0x50/0x130
> > [ 173.578258] call_on_irq_stack+0x2c/0x5c
> > [ 173.578263] do_interrupt_handler+0x80/0x84
> > [ 173.578270] el1_interrupt+0x34/0x6c
> > [ 173.578278] el1h_64_irq_handler+0x18/0x2c
> > [ 173.578283] el1h_64_irq+0x64/0x68
> > [ 173.578287] cpuidle_enter_state+0x130/0x2fc
> > [ 173.578296] cpuidle_enter+0x38/0x50
> > [ 173.578302] do_idle+0x22c/0x2c0
> > [ 173.578308] cpu_startup_entry+0x28/0x30
> > [ 173.578313] kernel_init+0x0/0x12c
> > [ 173.578319] arch_post_acpi_subsys_init+0x0/0x18
> > [ 173.578326] start_kernel+0x668/0x6a8
> > [ 173.578331] __primary_switched+0xbc/0xc4
> > [ 173.578337] SMP: stopping secondary CPUs
> > [ 173.578370] Kernel Offset: 0x445e47000000 from 0xffff800008000000
> > [ 173.578373] PHYS_OFFSET: 0xffffd3bc00000000
> > [ 173.578376] CPU features: 0x20000,2012c084,0000421b
> > [ 173.578380] Memory Limit: none
> > [ 173.853682] ---[ end Kernel panic - not syncing: Asynchronous
> SError
> > Interrupt ]---
> >
> >
> > It seems that my Linux panic when it has to jump from the
> hypervisor to the
> > driver (in arch_cpu_activate_vmm function in
> > hypervisor/arch/arm64/setup.c file).
> >
> > Do you have any idea why my Linux panic ?
> >
> > Thanks in advance,
> >
> > --
> > You received this message because you are subscribed to the Google
> > Groups "Jailhouse" group.
> > To unsubscribe from this group and stop receiving emails from it,
> send
> > an email to [email protected]
> > <mailto:[email protected]>.
> > To view this discussion on the web visit
> >
>
https://groups.google.com/d/msgid/jailhouse-dev/18173bdd-0873-4774-a664-ba03be4bd7a3n%40googlegroups.com
<
https://groups.google.com/d/msgid/jailhouse-dev/18173bdd-0873-4774-a664-ba03be4bd7a3n%40googlegroups.com>
<
https://groups.google.com/d/msgid/jailhouse-dev/18173bdd-0873-4774-a664-ba03be4bd7a3n%40googlegroups.com?utm_medium=email&utm_source=footer
<
https://groups.google.com/d/msgid/jailhouse-dev/18173bdd-0873-4774-a664-ba03be4bd7a3n%40googlegroups.com?utm_medium=email&utm_source=footer>>.
>
> --
> You received this message because you are subscribed to the Google
> Groups "Jailhouse" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to [email protected]
> <mailto:[email protected]>.
> To view this discussion on the web visit
>
https://groups.google.com/d/msgid/jailhouse-dev/ce75d57a-5bf7-44b7-95ae-2a494d6a9132n%40googlegroups.com
<
https://groups.google.com/d/msgid/jailhouse-dev/ce75d57a-5bf7-44b7-95ae-2a494d6a9132n%40googlegroups.com?utm_medium=email&utm_source=footer>.
--
You received this message because you are subscribed to the Google Groups
"Jailhouse" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/jailhouse-dev/762f7922-e547-460f-a5d8-4a3ef2dd85e4n%40googlegroups.com.
