Hi,
after changing the virt_start to match the phys_start for that memory
region I got an invalid argument error when starting the cell
Traceback (most recent call last):
File "/usr/local/libexec/jailhouse/jailhouse-cell-linux", line 739, in
<module>
cell.load(arch.kernel_image, arch.kernel_address())
File "/usr/local/lib/python3.10/dist-packages/pyjailhouse/cell.py", line
44, in load
fcntl.ioctl(self.dev, self.JAILHOUSE_CELL_LOAD, load)
OSError: [Errno 22] Invalid argument
Printing the arch.kernel_address() returned 16760320 (should be 0xffbe00),
so I thought that the bzImage was too big for the low ram region.
I increased the size to .size = 0x04000000 and shifted the virt and
phys_start of the other two regions to match the change (virt_start =
0x04000000 for the comm region and virt=phys_start = 0x46000000 for the
third region) .
No issues were reported by config check but after starting the cell i got
no output even using earlyprintk.
I also tried using an initrd but I got a similar issue, invalid argument
when starting the cell, this time it was the arch.ramdisk_address() which
was 107327488 (0x665b000).
So i tried increasing even more the size of the first region to size =
0x0a000000 (comm region now starts at .virt_start = 0x0a000000, and the
third region starts at virt=.phys_start = 0x4c000000 with size =
0x6000000).
Again, no config check issues so I started it but no output again, the
hypervisor only reports:
Created cell "linux-1"
Page pool usage after cell creation: mem 268/32211, remap 16392/131072
Cell "linux-1" can be loaded
CPU 5 received SIPI, vector 100
CPU 4 received SIPI, vector 100
Started cell "linux-1"
My bzImage is 12,9MB and the rootfs.cpio is 23.1MB if that matters.
Thanks,
Michele
Il giorno mercoledì 10 gennaio 2024 alle 20:58:03 UTC+1 Ralf Ramsauer ha
scritto:
> Hi Michele,
>
> On 10/01/2024 17:40, Michele Pescapè wrote:
> > Hi,
> >
> > I'm attaching the configurations I'm currently using. I'll try disabling
> > KASLR.
>
> […]
> /* high RAM */
> {
> .phys_start = 0x42100000,
> .virt_start = 0x00101000,
> .size = 0x8000000,
> […]
>
> Alignment of your high ram is odd. Virt & Phys should have the same
> alignment. Try to use .phys=.virt=0x42100000, I guess that should help.
>
> Thanks,
> Ralf
>
>
> >
> > Thank you for your help,
> > Michele
> >
> > Il giorno mercoledì 10 gennaio 2024 alle 17:24:35 UTC+1 Ralf Ramsauer ha
> > scritto:
> >
> > Hi,
> >
> > On 09/01/2024 19:58, Michele Pescapè wrote:
> > > Hi,
> > >
> > > Just an update, I was able to get the following output by adding -c
> > > "earlyprintk=ttyS0,115200" to the command line:
> > >
> > > Started cell "linux-1"
> > >
> > >
> > > Invalid physical address chosen!
> >
> > could you please share your system and non-root cell configuration?
> >
> > It smells like you have an overlap with the communication region. You
> > can check that with tools/jailhouse-config-check.
> >
> > >
> > >
> > >
> > >
> > > Physical KASLR disabled: no suitable memory region!
> >
> > Disable KASLR in your kernel for the moment. You don't need it while
> > ramping up your system.
> >
> > Thanks,
> > Ralf
> >
> > >
> > > [ 0.000000] Linux version 6.2.0-rc3 (root@mp-LINUX-DESKTOP)
> > > (x86_64-buildroot-linux-gnu-gcc.br_real (Buildroot 2023.11)
> > 12.3.0, GNU
> > > ld (GNU Binutils) 2.40) #2 SMP PREEMPT_DYNAMIC Sun Jan 7
> > 18:35:23 CET 2024
> > > [ 0.000000] Command line: earlyprintk=ttyS0,115200
> > > [ 0.000000] KERNEL supported cpus:
> > > [ 0.000000] Intel GenuineIntel
> > > [ 0.000000] AMD AuthenticAMD
> > > [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87
> > floating
> > > point registers'
> > > [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE
> > registers'
> > > [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX
> > registers'
> > > [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]:
> > 256
> > > [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size
> > is 832
> > > bytes, using 'compacted' format.
> > > [ 0.000000] signal: max sigframe size: 1776
> > > [ 0.000000] BIOS-provided physical RAM map:
> > > [ 0.000000] BIOS-e801: [mem
> > 0x0000000000000000-0x000000000009efff] usable
> > > [ 0.000000] printk: bootconsole [earlyser0] enabled
> > > [ 0.000000] NX (Execute Disable) protection: active
> > > [ 0.000000] extended physical RAM map:
> > > [ 0.000000] reserve setup_data: [mem
> > > 0x0000000000000000-0x0000000000001fff] usable
> > > [ 0.000000] reserve setup_data: [mem
> > > 0x0000000000002000-0x000000000000212b] usable
> > > [ 0.000000] reserve setup_data: [mem
> > > 0x000000000000212c-0x000000000009efff] usable
> > > [ 0.000000] DMI not present or invalid.
> > > [ 0.000000] Hypervisor detected: Jailhouse
> > > [ 0.000000] tsc: Detected 3393.624 MHz processor
> > > [ 0.000019] .text .data .bss are not marked as E820_TYPE_RAM!
> > > [ 0.005751] last_pfn = 0x3630 max_arch_pfn = 0x400000000
> > > [ 0.011039] x86/PAT: Configuration [0-7]: WB WC UC- UC WB
> > WP UC- WT
> > > Memory KASLR using RDRAND RDTSC...
> > > [ 0.032374] Using GB pages for direct mapping
> > > [ 0.036743] Kernel panic - not syncing: alloc_low_pages: can not
> > > alloc memory
> > > [ 0.043712] CPU: 0 PID: 0 Comm: swapper Not tainted 6.2.0-rc3 #2
> > > [ 0.049702] Call Trace:
> > > [ 0.052134] <TASK>
> > > [ 0.054219] dump_stack_lvl+0x3a/0x58
> > > [ 0.057865] dump_stack+0x10/0x18
> > > [ 0.061162] panic+0x15c/0x308
> > > [ 0.064205] alloc_low_pages+0x7d/0x1c0
> > > [ 0.068024] phys_pmd_init+0x1f2/0x2eb
> > > [ 0.071758] phys_pud_init+0x114/0x2d5
> > > [ 0.075493] phys_p4d_init+0x4c/0x269
> > > [ 0.079135] ?
> > __raw_callee_save___native_queued_spin_unlock+0x15/0x30
> > > [ 0.085649] __kernel_physical_mapping_init+0x11e/0x29a
> > > [ 0.090859] kernel_physical_mapping_init+0xf/0x20
> > > [ 0.095631] init_memory_mapping+0x273/0x410
> > > [ 0.099890] init_range_memory_mapping+0xec/0x150
> > > [ 0.104574] init_mem_mapping+0x268/0x2e7
> > > [ 0.108567] setup_arch+0x839/0xdcf
> > > [ 0.112039] ? lockdown_lsm_init+0x21/0x2f
> > > [ 0.116120] start_kernel+0x6e/0x997
> > > [ 0.119680] x86_64_start_reservations+0x24/0x2c
> > > [ 0.124281] x86_64_start_kernel+0xec/0xff
> > > [ 0.128360] secondary_startup_64_no_verify+0xe5/0xeb
> > > [ 0.133400] </TASK>
> > > [ 0.135570] ---[ end Kernel panic - not syncing:
> > alloc_low_pages: can
> > > not alloc memory ]---
> > > Closing cell "linux-1"
> > > Page pool usage after cell destruction: mem 254/32211, remap
> > 16392/131072
> > > CPU 4 received SIPI, vector 96
> > > CPU 5 received SIPI, vector 96
> > >
> > > This happens both with and without the initrd in the command,
> > i'll try
> > > to solve this and see what happens.
> > >
> > > By the way, is it possible that the new cell starts transmitting
> > with a
> > > baud rate different than the root cell? Without specifying the
> > speed in
> > > the command above the serial console stopped receiving anything
> > until I
> > > disabled and re-enabled the hypervisor. If not I may have something
> > > wrong on my end to solve
> > >
> > > Il giorno martedì 9 gennaio 2024 alle 17:37:48 UTC+1 Michele
> > Pescapè ha
> > > scritto:
> > >
> > > I stand corrected, the freeze happens when i do not set
> > > JAILHOUSE_CELL_PASSIVE_COMMREG in the inmate
> > >
> > > Il giorno martedì 9 gennaio 2024 alle 17:26:57 UTC+1 Michele Pescapè
> > > ha scritto:
> > >
> > > Hi, thank you for replying.
> > >
> > > ttyS0 is indeed whitelisted in the inmate and the number of
> > > pio_regions is set accordingly.
> > >
> > > As for the crash after disable/shutdown, I think it was related
> > > to the PCI device, after commenting that and the corresponding
> > > regions out I no longer get that freeze. CONFIG_JAILHOUSE_GUEST and
> > > CONFIG_JAILHOUSE_DBCON were already set in the kernel
> > > configuration I'm using for the non root linux; still, I
> > > recompiled it just in case.
> > >
> > > Michele
> > >
> > > Il giorno martedì 9 gennaio 2024 alle 14:14:33 UTC+1 Ralf
> > > Ramsauer ha scritto:
> > >
> > > Hi,
> > >
> > > On 08/01/2024 23:56, Michele Pescapè wrote:
> > > > Hi all,
> > > >
> > > > I have an AMD Ryzen 2600 on a b450 motherboard. My goal
> > > is to run two
> > > > non root linux cells, however, as of right now, I'm not
> > > able to start
> > > > any non root linux cells.
> > > > First of all I don't receive an output from the cell (I
> > > do get output
> > > > from the apic-demo though), therefore I don't really know
> > > if they're
> > > > even crashing or not; this is the output I get from the
> > > hypervisor after
> > > > enabling the root cell and issuing the following command:
> > > >
> > > > sudo jailhouse cell linux configs/x86/linux_guest1.cell
> > > > ../buildroot-2023.11/output/images/bzImage -i
> > > > ../buildroot-2023.11/output/images/rootfs.cpio
> > >
> > > for the first few tries, simply don't load a ramdisk. Try to
> > > get the
> > > kernel booting. It will crash with "cannot mount rootfs". If
> > > we get so
> > > far, then you can continue specifying the ramdisk. But
> > > first, we have to
> > > get that far.
> > >
> > > >
> > > > Adding PCI device ff:1f.7 to cell "linux-x86-demo"
> > > > Created cell "linux-x86-demo"
> > > > Page pool usage after cell creation: mem 336/32211, remap
> > > 16392/131072
> > > > Cell "linux-x86-demo" can be loaded
> > > > CPU 4 received SIPI, vector 100
> > > > CPU 3 received SIPI, vector 100
> > > > Started cell "linux-x86-demo"
> > > >
> > > > After this the cell appears to be running in cell list,
> > > however I can't
> > > > seem to get any output to confirm that (I even set
> > > > JAILHOUSE_CELL_VIRTUAL_CONSOLE_ACTIVE, but nothing
> > > appears on the vrtual
> > > > console accessed via "jailhouse console -f"). I also
> > > tried adding -c
> > > > "console=ttyS0,115200".
> > >
> > > ttyS0, i.e., x86 Port 0x3f8 is whitelisted in the cell
> > > configuration?
> > > Actually, your cmdline is correct.
> > >
> > > > At this point if I try to disable/shutdown the cell, the
> > > system freezes
> > > > requiring a hard reset.
> > >
> > > Okay, is the guest Linux compiled with Jailhouse guest
> > > support? Please
> > > ensure that non-root Linux is compiled with
> > > CONFIG_JAILHOUSE_GUEST and
> > > CONFIG_JAILHOUSE_DBCON. The first one is mandatory for x86
> > > Linux guests.
> > >
> > > >
> > > > Also, there seems to be a mismatch between the PCI bdf
> > > value in the log
> > > > above and the one in the cell's configuration. In both
> > > root and non root
> > > > cells i set .bdf = (0x0f << 3) for the IVSHMEM net device
> > > and the Root
> > > > cell correctly reports that ("Adding virtual PCI device
> > > 00:0f.0 to cell
> > > > "RootCell"").
> > >
> > > First things first. Let's try to get Linux kicked off before
> > > diving into
> > > PCI. Comment it out for the moment, let's do that later.
> > >
> > > Ralf
> > >
> > > >
> > > > If I understood correctly, once the cell works properly I
> > > won't need to
> > > > set the console anymore as I can assign an ip (-c "ip
> > > x.x.x.x" in the
> > > > cell linux command) and ssh to the cell, right?
> > > >
> > > > I attach the root and non root cells' configurations in
> > > case they're
> > > > needed. I can provide both the root and non root cell's
> > > kernel .conf and
> > > > the buildroot configuration I used if needed.
> > > >
> > > > Thank you for your time,
> > > > Michele
> > > >
> > > > --
> > > > You received this message because you are subscribed to
> > > the Google
> > > > Groups "Jailhouse" group.
> > > > To unsubscribe from this group and stop receiving emails
> > > from it, send
> > > > an email to [email protected]
> > > > <mailto:[email protected]>.
> > > > To view this discussion on the web visit
> > > >
> > >
> >
> https://groups.google.com/d/msgid/jailhouse-dev/824bacc4-b7ae-47c9-878e-7203214b4fc3n%40googlegroups.com
>
> <
> https://groups.google.com/d/msgid/jailhouse-dev/824bacc4-b7ae-47c9-878e-7203214b4fc3n%40googlegroups.com>
>
> <
> https://groups.google.com/d/msgid/jailhouse-dev/824bacc4-b7ae-47c9-878e-7203214b4fc3n%40googlegroups.com
>
> <
> https://groups.google.com/d/msgid/jailhouse-dev/824bacc4-b7ae-47c9-878e-7203214b4fc3n%40googlegroups.com>>
>
> <
> https://groups.google.com/d/msgid/jailhouse-dev/824bacc4-b7ae-47c9-878e-7203214b4fc3n%40googlegroups.com?utm_medium=email&utm_source=footer
>
> <
> https://groups.google.com/d/msgid/jailhouse-dev/824bacc4-b7ae-47c9-878e-7203214b4fc3n%40googlegroups.com?utm_medium=email&utm_source=footer>
>
> <
> https://groups.google.com/d/msgid/jailhouse-dev/824bacc4-b7ae-47c9-878e-7203214b4fc3n%40googlegroups.com?utm_medium=email&utm_source=footer
>
> <
> https://groups.google.com/d/msgid/jailhouse-dev/824bacc4-b7ae-47c9-878e-7203214b4fc3n%40googlegroups.com?utm_medium=email&utm_source=footer
> >>>.
> > >
> > > --
> > > You received this message because you are subscribed to the Google
> > > Groups "Jailhouse" group.
> > > To unsubscribe from this group and stop receiving emails from it,
> > send
> > > an email to [email protected]
> > > <mailto:[email protected]>.
> > > To view this discussion on the web visit
> > >
> >
> https://groups.google.com/d/msgid/jailhouse-dev/2202bf30-5b2c-45ea-9f9f-699daf338843n%40googlegroups.com
>
> <
> https://groups.google.com/d/msgid/jailhouse-dev/2202bf30-5b2c-45ea-9f9f-699daf338843n%40googlegroups.com>
>
> <
> https://groups.google.com/d/msgid/jailhouse-dev/2202bf30-5b2c-45ea-9f9f-699daf338843n%40googlegroups.com?utm_medium=email&utm_source=footer
>
> <
> https://groups.google.com/d/msgid/jailhouse-dev/2202bf30-5b2c-45ea-9f9f-699daf338843n%40googlegroups.com?utm_medium=email&utm_source=footer
> >>.
> >
> > --
> > You received this message because you are subscribed to the Google
> > Groups "Jailhouse" group.
> > To unsubscribe from this group and stop receiving emails from it, send
> > an email to [email protected]
> > <mailto:[email protected]>.
> > To view this discussion on the web visit
> >
> https://groups.google.com/d/msgid/jailhouse-dev/9a021e3f-caca-4ebe-9351-be2c3e999446n%40googlegroups.com
>
> <
> https://groups.google.com/d/msgid/jailhouse-dev/9a021e3f-caca-4ebe-9351-be2c3e999446n%40googlegroups.com?utm_medium=email&utm_source=footer
> >.
>
--
You received this message because you are subscribed to the Google Groups
"Jailhouse" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/jailhouse-dev/23a9c30e-5507-449e-9589-697904becac4n%40googlegroups.com.
