Re: [PATCH] README: drop IRC

[Ralf Ramsauer]

On 27/05/2024 22:25, Josh Soref wrote:

From: Josh Soref <2119212+jso...@users.noreply.github.com>

As noted in 2022 on the ML, irc has not been active:
https://groups.google.com/g/jailhouse-dev/c/qhWeA5n2TWo/m/oiShgDinAQAJ

Neither Freenode nor Libera appear to have a live #jailhouse channel

Signed-off-by: Josh Soref <2119212+jso...@users.noreply.github.com>


Acked-by: Ralf Ramsauer 

Thanks,
  Ralf


---
  README.md | 4 
  1 file changed, 4 deletions(-)

diff --git a/README.md b/README.md
index dc2c3af..ad7aa76 100644
--- a/README.md
+++ b/README.md
@@ -43,10 +43,6 @@ Frequently Asked Questions (FAQ):
  
   - See [FAQ file](FAQ.md)
  
-IRC channel:

-  - Freenode, irc.freenode.net, #jailhouse
-  - [![Webchat](https://img.shields.io/badge/irc-freenode-blue.svg "IRC 
Freenode")](https://webchat.freenode.net/?channels=jailhouse)
-
  Mailing list:
  
- jailhouse-dev@googlegroups.com


--
You received this message because you are subscribed to the Google Groups 
"Jailhouse" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jailhouse-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jailhouse-dev/d0b5f0bc-4209-4e06-b8af-e12471f7c9a7%40oth-regensburg.de.

Re: [EXT] Re: Necessary change to fit mainline kernel.

[Ralf Ramsauer]

Hi,

On 23/05/2024 17:25, Ke Li wrote:

Hi, Ralf,

I've double verified that the UART is working as expected by 
"root@rockpi-s:~# echo "random test text" > /dev/ttyS1" and I DID 
receive corresponding text on work machine, this not only validated the 
dts setup is correct but also the USB-UART converter and the connection 
between them are all as expected.


Alright, ttyS1 in deed is 0xff0b:
[0.946678] ff0b.serial: ttyS1 at MMIO 0xff0b (irq = 28, 
base_baud = 5078125) is a 16550A


Ok, so we don't have an issue here. I'm just looking for common mistakes.



Dmesg is also attached just in case you need to check if anything is 
suspicious.


I checked the mail-list on kernel.org <http://kernel.org> around 
potential issues with the DesignWave based 8250-UART IP from RockChip, 
found this[1]. Another chat behind the difference should be this one [2] 
around the AllWinner difference from RockChip, I may take a deep 
investigation into the detail behind this and let you know.


Tried applying the patch you mentioned, still no text came out - which 
also surprises me.


Ok, crap. Now it's getting difficult. The thing is, we don't know yet 
what is wrong: We can either have an issue with the UART driver, or, 
what is also possible, we might already crash /before/ we initialise the 
UART. Without any possibility of interaction with the device it's hard 
to find out what happens.


Next thing I would try is to try to write to the UART from inside kernel 
space, without starting the hypervisor. Just to know where we hang. 
Would you please try (untested pseudocode, but you should get the idea):



diff --git a/driver/main.c b/driver/main.c
index c8572470..e4482181 100644
--- a/driver/main.c
+++ b/driver/main.c
@@ -585,6 +585,14 @@ static int jailhouse_cmd_enable(struct 
jailhouse_system __user *arg)


error_code = 0;

+   pr("Mapping console %lx\n", config->debug_console.address);
+   console = ioremap(config->debug_console.address, 
config->debug_console.size);

+   if (!console) {
+   pr("Error!\n");
+   } else {
+   writel('X', console + 0x0);
+   }
+
+
preempt_disable();


Can you see the X on the serial line?

  Ralf




[1]https://patchwork.kernel.org/project/linux-rockchip/patch/20170206233000.3021-1-diand...@chromium.org/
 
<https://patchwork.kernel.org/project/linux-rockchip/patch/20170206233000.3021-1-diand...@chromium.org/>
[2]https://groups.google.com/g/linux-sunxi/c/4iYuzbWt79k/m/IFEarUoMDAAJ 
<https://groups.google.com/g/linux-sunxi/c/4iYuzbWt79k/m/IFEarUoMDAAJ>


On Thu, May 23, 2024 at 6:58 PM Ralf Ramsauer 
<mailto:ralf.ramsa...@oth-regensburg.de>> wrote:


Hi,

On 23/05/2024 09:43, Ke Li wrote:
 > Hi, Ralf,
 > I've taken your advice and do the following:
 > 1. Revert the change on JAILHOUSE_BASE to 0xc020. (btw, I
got
 > the idea of modifying it from the talk "Tutorial: Bootstrapping the
 > Partitioning Hypervisor Jailhouse"
 > <https://www.youtube.com/watch?v=7fiJbwmhnRw
<https://www.youtube.com/watch?v=7fiJbwmhnRw>
 > <https://www.youtube.com/watch?v=7fiJbwmhnRw
<https://www.youtube.com/watch?v=7fiJbwmhnRw>>> time stamp 1:28:31)

Outdated. Don't touch the code at the moment, just configuration.

 > 2. Update the RootCellConfig and add whatever memory region I can
find
 > (except gic-v2). Info collected from below, the latest version
can be
 > found here:
 >
https://github.com/siemens/jailhouse/compare/master...cnnblike:jailhouse-rk:working-branch 
<https://github.com/siemens/jailhouse/compare/master...cnnblike:jailhouse-rk:working-branch>
 <https://github.com/siemens/jailhouse/compare/master...cnnblike:jailhouse-rk:working-branch 
<https://github.com/siemens/jailhouse/compare/master...cnnblike:jailhouse-rk:working-branch>>
 >    a. from RK3308 Technical Programming Manual p12-p13
 >

(https://dl.radxa.com/rockpis/docs/hw/datasheets/Rockchip%20RK3308TRM%20V1.1%20Part1-20180810.pdf 
<https://dl.radxa.com/rockpis/docs/hw/datasheets/Rockchip%20RK3308TRM%20V1.1%20Part1-20180810.pdf>
 
<https://dl.radxa.com/rockpis/docs/hw/datasheets/Rockchip%20RK3308TRM%20V1.1%20Part1-20180810.pdf
 
<https://dl.radxa.com/rockpis/docs/hw/datasheets/Rockchip%20RK3308TRM%20V1.1%20Part1-20180810.pdf>>)
 >    b. the live dts I pulled from a working armbian kernel (you
can find
 > it from
 > https://gist.github.com/cnnblike/f758596d59899f4d300eefb55ef5f81e
<https://gist.github.com/cnnblike/f758596d59899f4d300eefb55ef5f81e>
 >
<https://gist.github.com/cnnblike/f758596d59899f4d300eefb55ef5f81e
<https://gist.github.com/cnnblike/f758596d59899f4d300eefb55ef5f81e>>)

Please no links, just simple a

Re: [EXT] Re: Necessary change to fit mainline kernel.

[Ralf Ramsauer]

Hi,

On 23/05/2024 09:43, Ke Li wrote:

Hi, Ralf,
I've taken your advice and do the following:
1. Revert the change on JAILHOUSE_BASE to 0xc020. (btw, I got 
the idea of modifying it from the talk "Tutorial: Bootstrapping the 
Partitioning Hypervisor Jailhouse" 
> time stamp 1:28:31)


Outdated. Don't touch the code at the moment, just configuration.

2. Update the RootCellConfig and add whatever memory region I can find 
(except gic-v2). Info collected from below, the latest version can be 
found here: 
https://github.com/siemens/jailhouse/compare/master...cnnblike:jailhouse-rk:working-branch 
   a. from RK3308 Technical Programming Manual p12-p13 
(https://dl.radxa.com/rockpis/docs/hw/datasheets/Rockchip%20RK3308TRM%20V1.1%20Part1-20180810.pdf )
   b. the live dts I pulled from a working armbian kernel (you can find 
it from 
https://gist.github.com/cnnblike/f758596d59899f4d300eefb55ef5f81e 
)


Please no links, just simple attachments in the future. It's horrible to 
clock-navigate through github to get what i need: raw content.


   c. the iomem result: 
https://gist.github.com/cnnblike/eb6ba4ce958d058edb0b7ae4ddd124e5 



Alright, just a short analysis: Jailhouse reservation is placed 
correctly. iomem confirms the reservation. Hypervisor base addresses in 
the configuration are correct. So there's nothing odd here.


3. I have also done multiple experiments on my side, all leading to "no 
output" result except the following that may help in troubleshooting:
   a. I changed the cpus section from 0b to 0b0111, thinking maybe 
one leftover core could at least allow the kernel to have a chance to 
emit something for diagnosis. It turns out it DID emit something with 
one core left: 


Nonono, don't do that! Jailhouse needs all CPUs. That won't work!

https://gist.github.com/cnnblike/34dd8d241846c8b8cf43f01cc4124dcf 

   b. map the peripheral memory region as a whole 
(0xff00~0xfffd, sram configed same as ram) to physic address, no 
output.

   c. same as b but without gic, no output.


GIC seems to be configured correctly.

With so many experiment, I'm wondering, if it could be some other reason 
behind such weird freeze kernel result.


So the thing is, no matter what mistakes in the memory region you might 
have in your configuration, you should at least see a Hello world from 
the hypervisor on the UART. That's the first thing we need to get 
running. The rest is something for later.


So why doesn't the UART work?

You already confirmed, that echoing to /dev/ttysomething on that line 
works, right? Are you *absolutely* sure that you use the right uart 
line? Please attach output of dmesg.


Anyway, base address and size of the UART are fine... In another 
project, some time ago, I also had some issue with the designware UARTs 
(8250_dw). There, it was the Fifo Control Register for some reason. 
Let's try that. And let's print a early 'X' to see if we get here.


Could you please try the patch below?


  Ralf



diff --git a/hypervisor/uart-8250.c b/hypervisor/uart-8250.c
index e6112820..7b29af0e 100644
--- a/hypervisor/uart-8250.c
+++ b/hypervisor/uart-8250.c
@@ -19,6 +19,7 @@
 #define UART_TX0x0
 #define UART_DLL   0x0
 #define UART_DLM   0x1
+#define UART_FCR   0x2
 #define UART_LCR   0x3
 #define  UART_LCR_8N1  0x03
 #define  UART_LCR_DLAB 0x80
@@ -54,6 +55,10 @@ static void uart_init(struct uart_chip *chip)
chip->reg_in = reg_in_mmio8;
}

+   chip->reg_out(chip, UART_FCR, 0);
+   chip->reg_out(chip, UART_LCR, UART_LCR_8N1);
+   chip->reg_out(chip, UART_TX, 'X');
+
/* only initialise if divider is not zero */
if (!chip->debug_console->divider)
return;

--
You received this message because you are subscribed to the Google Groups 
"Jailhouse" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jailhouse-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jailhouse-dev/af08ae11-7643-4ec4-9b9a-ed8c3e95ef62%40oth-regensburg.de.

Re: [EXT] Re: Direct Interrupt Injection for x86

[Ralf Ramsauer]

Hi Syed,

On 14/05/2024 15:37, Syed Aftab Rashid wrote:

Thanks for the response Ralf,

I came across a paper that compares the performance of a few static 
partitioning hypervisors including Jailhouse. Probably, you are already 
aware of it, the paper is available here. 
https://arxiv.org/pdf/2303.11186 <https://arxiv.org/pdf/2303.11186>.


Yes, the paper from the Portuguese colleagues.



One interesting thing I noticed is that for ARM with direct interrupt 
injection enabled, the interrupt latency for Jailhouse is around ~200ns 
(please see the attached image). However, on my x86 system I am getting 


Yes, and in the above case, jailhouse and bao performs exactly the same, 
as they use the same hardware mechanisms.


an average latency of ~800ns (from the APIC demo example). I think the 
APIC example is also doing the same thing as was done for the experiment 
in the figure. Do you have any idea why there is such a difference 
between the values for both cases? is it architecture specific?


As we see no XApic traps in your case: Yes, it must have to do something 
with /the rest/ of the hardware resp. architecture.


At least we can say, it's not the interrupt delivery system.

What remains? Lots of other architectural specifics, such as:
  - Cache configuration(s)
  - CPU Frequency scaling
  - Power saving mechanisms

If you see lower latencies on your bare-metal configuration, I'd suggest 
do investigate how the bare-metal application tunes and sets up the system.


  Ralf



On Fri, May 3, 2024 at 5:31 PM Ralf Ramsauer 
<mailto:ralf.ramsa...@oth-regensburg.de>> wrote:


Hi,

On 02/05/2024 15:45, Syed Aftab Rashid wrote:

[snip]

 >      > I just wanted to understand what is the best possible
jitter that
 >     I can
 >      > get with Jailhouse. We are working on a project that needs
cycle
 >     times
 >      > between 1-2  microseconds. The bare metal implementation of a
 >     similar
 >      > application results in a much lower jitter ~50ns.
 >
 >     So the application does structurally the same like apic-demo?
 >     (i.e., full IRQ arrival and dispatching, "computational task"
(i.e.,
 >     calculating jitter), return). 50ns are ~100 TSC cycles –
athletic.
 >
 >     I really doubt those latencies are achievable with IRQs. With
active
 >     waiting for events (i.e., active polling for interruption) -
maybe.
 >
 >
 > Yes, it is a similar application implemented in bare metal code
directly
 > on the hardware. The jitter is even smaller with the max being 30ns.

Interesting. W/o having the possibility to have a close look at this
issue, I'd say that this comes from the additional need of hardware
lookups through IRQ rerouting tables. I cannot say.

[snip]

 >      > apic-demo doesn't seem to have much impact on the jitter.
How is
 >     cache
 >      > allocated under Jailhouse house? What does a cache
allocation of
 >     size 2
 >      > employ?
 >
 >     Cache allocation is only supported on Intel X86, where we
have CAT,
 >     cache allocation technology.
 >
 >     I don't know precise details of cat (would have to look them up),
 >     but as
 >     far as I remember, assigning a size of 2 to a non-root-cell
means that
 >     2MiB of L3$ are exclusively assigned to the non-root cell.
(Look at the
 >     code, I'm not 100% sure).
 >
 >     Why do we do that? Neighbouring noise from other cells.
 >
 >     Just like the non-root cell might "disturb" itself, it's also
possible
 >     that Linux, running aside of the non-root cell disturbes it:
Imagine
 >     Linux those heavy number crunching while the non-root cell
waits for an
 >     IRQ. In that case, Linux will mess up caches of the non-root
cell.
 >
 >     CAT prevents full cache pollution by ensuring that there is
always a
 >     dedicated, but limited, amount of L3$ for the non-root cell.
 >
 >
 > Thanks for the explanation, I understand the idea of cache
pollution. It
 > is just that I don't see much impact with or without cache
pollution on
 > the values I get for the jitter. Ideally, the jitter should be
higher if
 > I enable cache pollution and lower when I disable it. But, so far
I am
 > not able to see that.
 >
 > Also, another question regarding direct interrupt injection, as
we saw
 > that it is enabled by default. Can you tell me how can I disable it?
 > Again, this is just to see how it will impact the jitter if we have
 > direct interrupt injection disabled.

There's no act

Re: [EXT] Re: Direct Interrupt Injection for x86

[Ralf Ramsauer]

Hi,

On 02/05/2024 15:45, Syed Aftab Rashid wrote:

[snip]


 > I just wanted to understand what is the best possible jitter that
I can
 > get with Jailhouse. We are working on a project that needs cycle
times
 > between 1-2  microseconds. The bare metal implementation of a
similar
 > application results in a much lower jitter ~50ns.

So the application does structurally the same like apic-demo?
(i.e., full IRQ arrival and dispatching, "computational task" (i.e.,
calculating jitter), return). 50ns are ~100 TSC cycles – athletic.

I really doubt those latencies are achievable with IRQs. With active
waiting for events (i.e., active polling for interruption) - maybe.


Yes, it is a similar application implemented in bare metal code directly 
on the hardware. The jitter is even smaller with the max being 30ns.


Interesting. W/o having the possibility to have a close look at this 
issue, I'd say that this comes from the additional need of hardware 
lookups through IRQ rerouting tables. I cannot say.


[snip]


 > apic-demo doesn't seem to have much impact on the jitter. How is
cache
 > allocated under Jailhouse house? What does a cache allocation of
size 2
 > employ?

Cache allocation is only supported on Intel X86, where we have CAT,
cache allocation technology.

I don't know precise details of cat (would have to look them up),
but as
far as I remember, assigning a size of 2 to a non-root-cell means that
2MiB of L3$ are exclusively assigned to the non-root cell. (Look at the
code, I'm not 100% sure).

Why do we do that? Neighbouring noise from other cells.

Just like the non-root cell might "disturb" itself, it's also possible
that Linux, running aside of the non-root cell disturbes it: Imagine
Linux those heavy number crunching while the non-root cell waits for an
IRQ. In that case, Linux will mess up caches of the non-root cell.

CAT prevents full cache pollution by ensuring that there is always a
dedicated, but limited, amount of L3$ for the non-root cell.


Thanks for the explanation, I understand the idea of cache pollution. It 
is just that I don't see much impact with or without cache pollution on 
the values I get for the jitter. Ideally, the jitter should be higher if 
I enable cache pollution and lower when I disable it. But, so far I am 
not able to see that.


Also, another question regarding direct interrupt injection, as we saw 
that it is enabled by default. Can you tell me how can I disable it? 
Again, this is just to see how it will impact the jitter if we have 
direct interrupt injection disabled.


There's no actual support for disabling this, it's used when being 
available. When Jailhouse tries to query the capability of IRQ remapping 
support during startup, you will have to mask out the bits that show 
support for it. I guess that should work.


  Ralf

--
You received this message because you are subscribed to the Google Groups 
"Jailhouse" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jailhouse-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jailhouse-dev/d30a9764-585f-4189-9f0c-033ddc2e1e55%40oth-regensburg.de.

Re: [EXT] Re: [PATCH] configs: x86: align with recent qemu device model

[Ralf Ramsauer]

On 18/02/2024 12:03, Jan Kiszka wrote:

On 16.02.24 20:36, Ralf Ramsauer wrote:

Devices slightly changed with recent Qemu versions. Adapt changes of
device topology.

Signed-off-by: Ralf Ramsauer 
---

Tested with jailhouse enable, create, disable + apic demo.

  README.md  |  2 +-
  configs/x86/qemu-x86.c | 80 +-
  2 files changed, 41 insertions(+), 41 deletions(-)

diff --git a/README.md b/README.md
index 2ab2a60a..f6fe011b 100644
--- a/README.md
+++ b/README.md
@@ -255,7 +255,7 @@ https://github.com/siemens/jailhouse-images.

  The included system configuration qemu-x86.c can be used to run Jailhouse in
  QEMU/KVM virtual machine on x86 hosts (Intel and AMD are supported). Currently
-it requires Linux 4.4 or newer on the host side. QEMU version 2.8 or newer is
+it requires Linux 4.4 or newer on the host side. QEMU version 8.2 or newer is


I'm reluctant to bump the requirements that high, given how many distros
are on 7.2 or even 7.1. What about starting to use versioned machine
models instead? Say, 'qemu-system-x86_64 -machine pc-q35-7.0'? Similar
for the other archs (virt-x.y).


Turns out we don't need that patch. My x86 Qemu test VM was pretty old, 
and I missed out the qemu option -device pcie-pci-bridge. With that 
option, everything seems to be fine, even with most recent versions of qemu.


  Ralf



Jan


  required.

  You also need a Linux guest image with a recent kernel (tested with >= 3.9) 
and
diff --git a/configs/x86/qemu-x86.c b/configs/x86/qemu-x86.c
index cdd3dd6d..7a585bd7 100644
--- a/configs/x86/qemu-x86.c
+++ b/configs/x86/qemu-x86.c
@@ -179,52 +179,52 @@ struct {
.size = 0x4000,
.flags = JAILHOUSE_MEM_READ | JAILHOUSE_MEM_WRITE,
},
-   /* MemRegion: fea0-fea3 : :00:02.0 */
+   /* MemRegion: feb4-feb7 : :00:02.0 */
{
-   .phys_start = 0xfea0,
-   .virt_start = 0xfea0,
+   .phys_start = 0xfeb4,
+   .virt_start = 0xfeb4,
.size = 0x4,
.flags = JAILHOUSE_MEM_READ | JAILHOUSE_MEM_WRITE,
},
-   /* MemRegion: fea4-fea5 : e1000e */
+   /* MemRegion: feb8-feb9 : e1000e */
{
-   .phys_start = 0xfea4,
-   .virt_start = 0xfea4,
+   .phys_start = 0xfeb8,
+   .virt_start = 0xfeb8,
.size = 0x2,
.flags = JAILHOUSE_MEM_READ | JAILHOUSE_MEM_WRITE,
},
-   /* MemRegion: fea6-fea7 : e1000e */
+   /* MemRegion: feba-febb : e1000e */
{
-   .phys_start = 0xfea6,
-   .virt_start = 0xfea6,
+   .phys_start = 0xfeba,
+   .virt_start = 0xfeba,
.size = 0x2,
.flags = JAILHOUSE_MEM_READ | JAILHOUSE_MEM_WRITE,
},
-   /* MemRegion: fea91000-fea93fff : e1000e */
+   /* MemRegion: febd-febd3fff : e1000e */
{
-   .phys_start = 0xfea91000,
-   .virt_start = 0xfea91000,
+   .phys_start = 0xfebd,
+   .virt_start = 0xfebd,
.size = 0x3000,
.flags = JAILHOUSE_MEM_READ | JAILHOUSE_MEM_WRITE,
},
-   /* MemRegion: fea94000-fea97fff : :00:1b.0 (ICH HD audio) */
+   /* MemRegion: febd4000-febd7fff : :00:1b.0 (ICH HD audio) */
{
-   .phys_start = 0xfea94000,
-   .virt_start = 0xfea94000,
+   .phys_start = 0xfebd4000,
+   .virt_start = 0xfebd4000,
.size = 0x4000,
.flags = JAILHOUSE_MEM_READ | JAILHOUSE_MEM_WRITE,
},
-   /* MemRegion: fea98000-fea98fff : :00:01.0 (vesafd) */
+   /* MemRegion: febd8000-febd8fff : :00:01.0 (vesafd) */
{
-   .phys_start = 0xfea98000,
-   .virt_start = 0xfea98000,
+   .phys_start = 0xfebd8000,
+   .virt_start = 0xfebd8000,
.size = 0x1000,
.flags = JAILHOUSE_MEM_READ | JAILHOUSE_MEM_WRITE,
},
-   /* MemRegion: fea9a000-fea9afff : :00:1f.2 (ahci) */
+   /* MemRegion: febd9000-febd9fff : :00:1f.2 (ahci) */
{
-   .phys_start = 0xfea9a000,
-   .virt_start = 0xfea9a

[PATCH] configs: x86: align with recent qemu device model

[Ralf Ramsauer]

Devices slightly changed with recent Qemu versions. Adapt changes of
device topology.

Signed-off-by: Ralf Ramsauer 
---

Tested with jailhouse enable, create, disable + apic demo.

 README.md  |  2 +-
 configs/x86/qemu-x86.c | 80 +-
 2 files changed, 41 insertions(+), 41 deletions(-)

diff --git a/README.md b/README.md
index 2ab2a60a..f6fe011b 100644
--- a/README.md
+++ b/README.md
@@ -255,7 +255,7 @@ https://github.com/siemens/jailhouse-images.
 
 The included system configuration qemu-x86.c can be used to run Jailhouse in
 QEMU/KVM virtual machine on x86 hosts (Intel and AMD are supported). Currently
-it requires Linux 4.4 or newer on the host side. QEMU version 2.8 or newer is
+it requires Linux 4.4 or newer on the host side. QEMU version 8.2 or newer is
 required.
 
 You also need a Linux guest image with a recent kernel (tested with >= 3.9) and
diff --git a/configs/x86/qemu-x86.c b/configs/x86/qemu-x86.c
index cdd3dd6d..7a585bd7 100644
--- a/configs/x86/qemu-x86.c
+++ b/configs/x86/qemu-x86.c
@@ -179,52 +179,52 @@ struct {
.size = 0x4000,
.flags = JAILHOUSE_MEM_READ | JAILHOUSE_MEM_WRITE,
},
-   /* MemRegion: fea0-fea3 : :00:02.0 */
+   /* MemRegion: feb4-feb7 : :00:02.0 */
{
-   .phys_start = 0xfea0,
-   .virt_start = 0xfea0,
+   .phys_start = 0xfeb4,
+   .virt_start = 0xfeb4,
.size = 0x4,
.flags = JAILHOUSE_MEM_READ | JAILHOUSE_MEM_WRITE,
},
-   /* MemRegion: fea4-fea5 : e1000e */
+   /* MemRegion: feb8-feb9 : e1000e */
{
-   .phys_start = 0xfea4,
-   .virt_start = 0xfea4,
+   .phys_start = 0xfeb8,
+   .virt_start = 0xfeb8,
.size = 0x2,
.flags = JAILHOUSE_MEM_READ | JAILHOUSE_MEM_WRITE,
},
-   /* MemRegion: fea6-fea7 : e1000e */
+   /* MemRegion: feba-febb : e1000e */
{
-   .phys_start = 0xfea6,
-   .virt_start = 0xfea6,
+   .phys_start = 0xfeba,
+   .virt_start = 0xfeba,
.size = 0x2,
.flags = JAILHOUSE_MEM_READ | JAILHOUSE_MEM_WRITE,
},
-   /* MemRegion: fea91000-fea93fff : e1000e */
+   /* MemRegion: febd-febd3fff : e1000e */
{
-   .phys_start = 0xfea91000,
-   .virt_start = 0xfea91000,
+   .phys_start = 0xfebd,
+   .virt_start = 0xfebd,
.size = 0x3000,
.flags = JAILHOUSE_MEM_READ | JAILHOUSE_MEM_WRITE,
},
-   /* MemRegion: fea94000-fea97fff : :00:1b.0 (ICH HD audio) */
+   /* MemRegion: febd4000-febd7fff : :00:1b.0 (ICH HD audio) */
{
-   .phys_start = 0xfea94000,
-   .virt_start = 0xfea94000,
+   .phys_start = 0xfebd4000,
+   .virt_start = 0xfebd4000,
.size = 0x4000,
.flags = JAILHOUSE_MEM_READ | JAILHOUSE_MEM_WRITE,
},
-   /* MemRegion: fea98000-fea98fff : :00:01.0 (vesafd) */
+   /* MemRegion: febd8000-febd8fff : :00:01.0 (vesafd) */
{
-   .phys_start = 0xfea98000,
-   .virt_start = 0xfea98000,
+   .phys_start = 0xfebd8000,
+   .virt_start = 0xfebd8000,
.size = 0x1000,
.flags = JAILHOUSE_MEM_READ | JAILHOUSE_MEM_WRITE,
},
-   /* MemRegion: fea9a000-fea9afff : :00:1f.2 (ahci) */
+   /* MemRegion: febd9000-febd9fff : :00:1f.2 (ahci) */
{
-   .phys_start = 0xfea9a000,
-   .virt_start = 0xfea9a000,
+   .phys_start = 0xfebd9000,
+   .virt_start = 0xfebd9000,
.size = 0x1000,
.flags = JAILHOUSE_MEM_READ | JAILHOUSE_MEM_WRITE,
},
@@ -264,6 +264,24 @@ struct {
},
 
.pci_devices = {
+   /* PCI bridge */
+   {
+   .type = JAILHOUSE_PCI_TYPE_BRIDGE,
+   .domain = 0x,
+   .bdf = 0x,
+   .bar_mask = {
+   0xf

Re: Problem installing Jailhouse on ubuntu VM running on X86

[Ralf Ramsauer]

On 01/02/2024 11:32, Syed Aftab Rashid wrote:

Hi all,

I am trying to install Jailhouse on a Ubuntu VM running using VMware 
inside a Windows 10 host system. I am following the instruction provided 
here


Don't do it. Use Qemu/KVM with Linux as host, as described in the 
README. For building your environment, you can use jailhouse-images.


Thanks,
  Ralf



GitHub - siemens/jailhouse: Linux-based partitioning hypervisor 



However, I am getting the following error when I run "make" in the 
jailhouse folder.


ERROR: modpost: "ioremap_page_range" 
[/home/ubuntu/jailhouse/driver/jailhouse.ko] undefined!
ERROR: modpost: "__get_vm_area_caller" 
[/home/ubuntu/jailhouse/driver/jailhouse.ko] undefined!
ERROR: modpost: "lapic_timer_period" 
[/home/ubuntu/jailhouse/driver/jailhouse.ko] undefined!
make[2]: *** [scripts/Makefile.modpost:144: 
/home/ubuntu/jailhouse/Module.symvers] Error 1

make[1]: *** [Makefile:1989: modpost] Error 2
make: *** [Makefile:40: modules] Error 2

I have modified the /etc/default/grub as follows

GRUB_DEFAULT=0
GRUB_TIMEOUT_STYLE=hidden
GRUB_TIMEOUT=0
GRUB_DISTRIBUTOR=`lsb_release -i -s 2> /dev/null || echo Debian`
GRUB_CMDLINE_LINUX_DEFAULT="quiet splash"
GRUB_CMDLINE_LINUX_DEFAULT="memmap=82M\\\$0x3a00"
GRUB_CMDLINE_LINUX="intel_iommu=off"
GRUB_CMDLINE_LINUX="find_preseed=/preseed.cfg auto noprompt 
priority=critical l


What am I missing here. Unfortunately, I am a bit new to this so I might 
be making a basic mistake. However, I couldn't find any proper 
installation guide.


Any help in this regard is much appreciated.

Thank you

--
You received this message because you are subscribed to the Google 
Groups "Jailhouse" group.
To unsubscribe from this group and stop receiving emails from it, send 
an email to jailhouse-dev+unsubscr...@googlegroups.com 
.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jailhouse-dev/a2cf0e4d-925d-4d67-b619-c07c8bb42657n%40googlegroups.com .


--
You received this message because you are subscribed to the Google Groups 
"Jailhouse" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jailhouse-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jailhouse-dev/338fb8b9-c069-42c7-a0b3-48dfc5bb073e%40oth-regensburg.de.

Re: Unable to start non root linux cell

[Ralf Ramsauer]

Hi,

On 17/01/2024 12:52, Michele Pescapè wrote:

Hi,

I checked again the configs with jailhouse config check and nothing came 
up. I'm attaching the configurations.


Please compare the qemu config with your config *precisely*. Your struct 
pci_devices is at the wrong position in the header of the config. It has 
to come *after* struct pio_regions and before pci_caps. Position is 
important for the internal parsing.


In your case, it the parser will hit some bogus data.

  Ralf



Thanks,
Michele

Il giorno mercoledì 17 gennaio 2024 alle 11:52:18 UTC+1 Ralf Ramsauer ha 
scritto:


Hi Michele,

On 16/01/2024 20:44, Michele Pescapè wrote:
 > Hi,
 >
 > I'm trying to give network access to the non root cell, however
it seems
 > that I'm not able to pass the right PCI device. It always
defaults to
 > taking away the 00:00.0 pci device from the root cell causing a
crash
 > for the root cell.
 >
 > In both the root and non root cell's configs I added the shared
memory
 > regions using the macro JAILHOUSE_SHMEM_NET_REGIONS(0x6000,
0) for
 > the root and using dev_id=1 for the non root. The resulting
regions do
 > not overlap other memory regions, config check does not return
any errors.
 >
 > I also added the following PCI device in both cells, where
 > .shmem_regions_start is assigned to the correct index for the first
 > shared memory region created using the aforementioned macro.
 >
 > /* IVSHMEM: 00:05.0 */
 > {
 > .type = JAILHOUSE_PCI_TYPE_IVSHMEM,
 > .domain = 0x0,
 > .bdf = 0x28,

[…]

 > .bar_mask = JAILHOUSE_IVSHMEM_BAR_MASK_MSIX,
 > .num_msix_vectors = 2,
 > .shmem_regions_start = 0,
 > .shmem_dev_id = 0,   (1 in the non root cell)
 > .shmem_peers = 2,
 > .shmem_protocol = JAILHOUSE_SHMEM_PROTO_VETH,
 > },
 >
 > I tried using different values for the .bdf, none of which were
already
 > used by other devices but none worked. The root cell always sees
the new
 > device correctly when starting the hypervisor, I can also see it
with
 > lspci and ifconfig, however once I issue the cell linux command
to start
 > the non root cell, I get the following output followed by a
streak of
 > errors from the hypervisor, forcing a manual reset.
 >
 > Removing PCI device 00:00.0 from cell "RootCell"
 > Adding PCI device 00:00.0 to cell "linux-1"

.bdf = 0x28 doesn't correspond with 00:00.0. Would you please test your
current configs against jailhouse-config-check, and in case of no
errors, would you please attach the current versions again?

Thanks,
Ralf

 > FATAL: unable to get MMIO instruction
 > FATAL: Invalid MMIO/RAM write, addr: 0x000106d8c020 size: 0
 > RIP: 0xa910ff4c RSP: 0xab85400f7de0 FLAGS: 93
 > RAX: 0x RBX: 0x0002f170 RCX:
0x0040
 > RDX: 0x0414 RSI: 0x8b6000955400 RDI:
0x0001
 > CS: 10 BASE: 0x AR-BYTES: 29b EFER.LMA 1
 > CR0: 0x80050033 CR3: 0x000106d8c000 CR4:
0x003506a0
 > EFER: 0x1d01
 > Parking CPU 4 (Cell: "RootCell")
 > (etc..)
 >
 > I'm not sure what I'm missing here, CONFIG_IVSHMEM_NET is enabled in
 > both the root and non root kernel config, I also enabled
 > CONFIG_PCI_HOST_GENERIC in the root linux as suggested in one of the
 > conversations on this site.
 >
 > Thanks,
 > Michele
 >
 >
 > Il giorno domenica 14 gennaio 2024 alle 15:49:40 UTC+1 Ralf
Ramsauer ha
 > scritto:
 >
 > Hi Michele,
 >
 > On 14/01/2024 02:18, Michele Pescapè wrote:
 > > Hi,
 > >
 > > There were PIO writes to ports 4e,4f,2e and 2f, each of size 1
 > and each
 > > one on subsequent restarts. I'm not sure how to figure out what
they
 > > belong to.
 >
 > Easy and straight forward: Read the jailhouse crash dump.
 >
 > Look at the program counter, and maybe the return address.
Disassemble
 > your kernel (objdump -d vmlinux) and lookup that address. Now you
know
 > the name of the routine in the kernel that caused the crash.
 >
 > Don't simply whitelist ports. Only whitelist them, if you're sure
what
 > you're doing.
 >
 > >
 > > CONFIG_ISA_DMA_API is already disabled.
 > >
 > > I missed the irqchip for the uart. However as of right now I just
 > copied
 > > the whole fragment from the root configuration which means I
 &

Re: Unable to start non root linux cell

[Ralf Ramsauer]

Hi Michele,

On 16/01/2024 20:44, Michele Pescapè wrote:

Hi,

I'm trying to give network access to the non root cell, however it seems 
that I'm not able to pass the right PCI device. It always defaults to 
taking away the 00:00.0 pci device from the root cell causing a crash 
for the root cell.


In both the root and non root cell's configs I added the shared memory 
regions using the macro JAILHOUSE_SHMEM_NET_REGIONS(0x6000, 0) for 
the root and using dev_id=1 for the non root. The resulting regions do 
not overlap other memory regions, config check does not return any errors.


I also added the following PCI device in both cells, where 
.shmem_regions_start is assigned to the correct index for the first 
shared memory region created using the aforementioned macro.


/* IVSHMEM: 00:05.0 */
{
.type = JAILHOUSE_PCI_TYPE_IVSHMEM,
.domain = 0x0,
.bdf = 0x28,


[…]


.bar_mask = JAILHOUSE_IVSHMEM_BAR_MASK_MSIX,
.num_msix_vectors = 2,
.shmem_regions_start = 0,
.shmem_dev_id = 0,   (1 in the non root cell)
.shmem_peers = 2,
.shmem_protocol = JAILHOUSE_SHMEM_PROTO_VETH,
},

I tried using different values for the .bdf, none of which were already 
used by other devices but none worked. The root cell always sees the new 
device correctly when starting the hypervisor, I can also see it with 
lspci and ifconfig, however once I issue the cell linux command to start 
the non root cell, I get the following output followed by a streak of 
errors from the hypervisor, forcing a manual reset.


Removing PCI device 00:00.0 from cell "RootCell"
Adding PCI device 00:00.0 to cell "linux-1"


.bdf = 0x28 doesn't correspond with 00:00.0. Would you please test your 
current configs against jailhouse-config-check, and in case of no 
errors, would you please attach the current versions again?


Thanks,
  Ralf


FATAL: unable to get MMIO instruction
FATAL: Invalid MMIO/RAM write, addr: 0x000106d8c020 size: 0
RIP: 0xa910ff4c RSP: 0xab85400f7de0 FLAGS: 93
RAX: 0x RBX: 0x0002f170 RCX: 0x0040
RDX: 0x0414 RSI: 0x8b6000955400 RDI: 0x0001
CS: 10 BASE: 0x AR-BYTES: 29b EFER.LMA 1
CR0: 0x80050033 CR3: 0x000106d8c000 CR4: 0x003506a0
EFER: 0x1d01
Parking CPU 4 (Cell: "RootCell")
(etc..)

I'm not sure what I'm missing here, CONFIG_IVSHMEM_NET is enabled in 
both the root and non root kernel config, I also enabled 
CONFIG_PCI_HOST_GENERIC in the root linux as suggested in one of the 
conversations on this site.


Thanks,
Michele


Il giorno domenica 14 gennaio 2024 alle 15:49:40 UTC+1 Ralf Ramsauer ha 
scritto:


Hi Michele,

On 14/01/2024 02:18, Michele Pescapè wrote:
 > Hi,
 >
 > There were PIO writes to ports 4e,4f,2e and 2f, each of size 1
and each
 > one on subsequent restarts. I'm not sure how to figure out what they
 > belong to.

Easy and straight forward: Read the jailhouse crash dump.

Look at the program counter, and maybe the return address. Disassemble
your kernel (objdump -d vmlinux) and lookup that address. Now you know
the name of the routine in the kernel that caused the crash.

Don't simply whitelist ports. Only whitelist them, if you're sure what
you're doing.

 >
 > CONFIG_ISA_DMA_API is already disabled.
 >
 > I missed the irqchip for the uart. However as of right now I just
copied
 > the whole fragment from the root configuration which means I
could be
 > taking away other interrupts from the root cell, I still have to
figure
 > out how to tune that
 >
 > /* IOAPIC 13, GSI base 0 */
 > {
 > .address = 0xfec0,
 > .id = 0xa0,
 > .pin_bitmap = {
 > 0x, 0x, 0x, 0x

With this, you take away ALL interrupts from the root cell, which you
don't want to do, you only want to take away the interrupt that
corresponds to ttyS0 (i.e. Port 0x3f8).

See here: https://en.wikipedia.org/wiki/COM_(hardware_interface)
<https://en.wikipedia.org/wiki/COM_(hardware_interface)>

COM1: I/O port 0x3F8, IRQ 4

So you want to only assign IRQ 4 to your non-root cell. See
linux-x86-demo.c to see how that works.

 > },
 > },
 >
 > Still, now I can finally login into the non root linux.

Excellent!

Ralf

     >
 > Thanks,
 > Michele
 >
 > Il giorno sabato 13 gennaio 2024 alle 21:09:28 UTC+1 Ralf
Ramsauer ha
 > scritto:
 >
 > Hi Michele,
 >
 > On 13/01/2024 18:08, Michele Pescapè wrote:
 > > Hi,
 > >
 > > The problem was that the ram regions in the non root configuration
 > > weren't detected as such because of the missing JAILHOUSE_MEM_DMA
 > flag.
 >
 > Yikes, makes sense.

Re: Unable to start non root linux cell

[Ralf Ramsauer]

Hi Michele,

On 14/01/2024 02:18, Michele Pescapè wrote:

Hi,

There were PIO writes to ports 4e,4f,2e and 2f, each of size 1 and each 
one on subsequent restarts. I'm not sure how to figure out what they 
belong to.


Easy and straight forward: Read the jailhouse crash dump.

Look at the program counter, and maybe the return address. Disassemble 
your kernel (objdump -d vmlinux) and lookup that address. Now you know 
the name of the routine in the kernel that caused the crash.


Don't simply whitelist ports. Only whitelist them, if you're sure what 
you're doing.




CONFIG_ISA_DMA_API is already disabled.

I missed the irqchip for the uart. However as of right now I just copied 
the whole fragment from the root configuration which means I could be 
taking away other interrupts from the root cell, I still have to figure 
out how to tune that


/* IOAPIC 13, GSI base 0 */
{
.address = 0xfec0,
.id = 0xa0,
.pin_bitmap = {
0x, 0x, 0x, 0x


With this, you take away ALL interrupts from the root cell, which you 
don't want to do, you only want to take away the interrupt that 
corresponds to ttyS0 (i.e. Port 0x3f8).


See here: https://en.wikipedia.org/wiki/COM_(hardware_interface)

COM1: I/O port 0x3F8, IRQ 4

So you want to only assign IRQ 4 to your non-root cell. See 
linux-x86-demo.c to see how that works.



},
},

Still, now I can finally login into the non root linux.


Excellent!

  Ralf



Thanks,
Michele

Il giorno sabato 13 gennaio 2024 alle 21:09:28 UTC+1 Ralf Ramsauer ha 
scritto:


Hi Michele,

On 13/01/2024 18:08, Michele Pescapè wrote:
 > Hi,
 >
 > The problem was that the ram regions in the non root configuration
 > weren't detected as such because of the missing JAILHOUSE_MEM_DMA
flag.

Yikes, makes sense.

 > After adding that, I also had to add two pio_regions because the non
 > root linux was crashing because of two ports which aren't present in
 > /proc/ioports.

Uhm - which PIO ports? Don't simply assign PIO ports. There must be a
reason for them. What ports did crash?

Did you disable(!) CONFIG_ISA_DMA_API? Please disable it. Therefore,
you
have to activate CONFIG_EXPERT first.

I *bet* it was i8237…

 > I also had to add mce=off to the command line because i had an
unhandled

or disable CONFIG_X86_MCE.

 > MSR error, I'll have to disable that in the kernel config.
 > At this point the non root linux seems to start, I see the boot
log. No

Excellent!

 > way of interacting with it as there is not a login prompt or
anything, I
 > think I need to ssh to the cell at this point, right?

Was the initramdisk loaded correctly?

Did you assign - in your non-root cell config - the irqchip and the
corresponding interrupts of the uart?

 > That means I'll now need to work on those ivshmem net devices.

If you need ivshmem, then this would be the next step.

Ralf

 >
 > Thanks,
 > Michele
 >
 > Il giorno sabato 13 gennaio 2024 alle 15:13:12 UTC+1 Ralf
Ramsauer ha
 > scritto:
 >
 > Hi,
 >
 > On 13/01/2024 12:28, Michele Pescapè wrote:
 > > Hi,
 > >
 > > You are right, I got confused about those addresses, my bad. At
 > least
 > > now I know that config is the correct one and I don't have to
tinker
 > > with it.
 > > I'm back to a kernel panic from the inmate when booting the cell.
 > It's
 > > similar to the one I had earlier, not sure yet of what the
 > problem may be.
 >
 > Great, we're a step further.
 >
 > >
 > > Created cell "linux-2"
 > > Page pool usage after cell creation: mem 406/32211, remap
 > 16392/131072
 > > Cell "linux-2" can be loaded
 > > CPU 9 received SIPI, vector 100
 > > Started cell "linux-2"
 > > CPU 8 received SIPI, vector 100
 > > No EFI environment detected.
 > > early console in extract_kernel
 > > input_data: 0x0275c308
 > > input_len: 0x008b0981
 > > output: 0x0100
 > > output_len: 0x01fccb30
 > > kernel_total_size: 0x01e28000
 > > needed_size: 0x0200
 > > trampoline_32bit: 0x0009d000
 > >
 > > Decompressing Linux... Parsing ELF... done.
 > > Booting the kernel.
 > > [    0.00] Linux version 6.2.0-rc3 (root@mp-LINUX-DESKTOP)
 > > (x86_64-buildroot       -linux-gnu-gcc.br_real (Buildroot 2023.11)
 > > 12.3.0, GNU ld (GNU Binutils) 2.40) #       2 SMP PREEMPT_DYNAMIC
 > Fri
 > > Jan 12 17:36:57 CET 2024
 > > [    0.00] Command line: e

Re: Unable to start non root linux cell

[Ralf Ramsauer]

Hi Michele,

On 13/01/2024 18:08, Michele Pescapè wrote:

Hi,

The problem was that the ram regions in the non root configuration 
weren't detected as such because of the missing JAILHOUSE_MEM_DMA flag. 


Yikes, makes sense.

After adding that, I also had to add two pio_regions because the non 
root linux was crashing because of two ports which aren't present in 
/proc/ioports.


Uhm - which PIO ports? Don't simply assign PIO ports. There must be a 
reason for them. What ports did crash?


Did you disable(!) CONFIG_ISA_DMA_API? Please disable it. Therefore, you 
have to activate CONFIG_EXPERT first.


I *bet* it was i8237…

I also had to add mce=off to the command line because i had an unhandled 


or disable CONFIG_X86_MCE.


MSR error, I'll have to disable that in the kernel config.
At this point the non root linux seems to start, I see the boot log. No 


Excellent!

way of interacting with it as there is not a login prompt or anything, I 
think I need to ssh to the cell at this point, right?


Was the initramdisk loaded correctly?

Did you assign - in your non-root cell config - the irqchip and the 
corresponding interrupts of the uart?



That means I'll now need to work on those ivshmem net devices.


If you need ivshmem, then this would be the next step.

  Ralf



Thanks,
Michele

Il giorno sabato 13 gennaio 2024 alle 15:13:12 UTC+1 Ralf Ramsauer ha 
scritto:


Hi,

On 13/01/2024 12:28, Michele Pescapè wrote:
 > Hi,
 >
 > You are right, I got confused about those addresses, my bad. At
least
 > now I know that config is the correct one and I don't have to tinker
 > with it.
 > I'm back to a kernel panic from the inmate when booting the cell.
It's
 > similar to the one I had earlier, not sure yet of what the
problem may be.

Great, we're a step further.

 >
 > Created cell "linux-2"
 > Page pool usage after cell creation: mem 406/32211, remap
16392/131072
 > Cell "linux-2" can be loaded
 > CPU 9 received SIPI, vector 100
 > Started cell "linux-2"
 > CPU 8 received SIPI, vector 100
 > No EFI environment detected.
 > early console in extract_kernel
 > input_data: 0x0275c308
 > input_len: 0x008b0981
 > output: 0x0100
 > output_len: 0x01fccb30
 > kernel_total_size: 0x01e28000
 > needed_size: 0x0200
 > trampoline_32bit: 0x0009d000
 >
 > Decompressing Linux... Parsing ELF... done.
 > Booting the kernel.
 > [    0.00] Linux version 6.2.0-rc3 (root@mp-LINUX-DESKTOP)
 > (x86_64-buildroot       -linux-gnu-gcc.br_real (Buildroot 2023.11)
 > 12.3.0, GNU ld (GNU Binutils) 2.40) #       2 SMP PREEMPT_DYNAMIC
Fri
 > Jan 12 17:36:57 CET 2024
 > [    0.00] Command line: earlyprintk=ttyS0,115200
 > [    0.00] KERNEL supported cpus:
 > [    0.00]   Intel GenuineIntel
 > [    0.00]   AMD AuthenticAMD
 > [    0.00] x86/fpu: Supporting XSAVE feature 0x001: 'x87
floating
 > point regi       sters'
 > [    0.00] x86/fpu: Supporting XSAVE feature 0x002: 'SSE
registers'
 > [    0.00] x86/fpu: Supporting XSAVE feature 0x004: 'AVX
registers'
 > [    0.00] x86/fpu: xstate_offset[2]:  576, xstate_sizes[2]:
  256
 > [    0.00] x86/fpu: Enabled xstate features 0x7, context size
is 832
 > bytes,        using 'compacted' format.
 > [    0.00] signal: max sigframe size: 1360
 > [    0.00] BIOS-provided physical RAM map:
 > [    0.00] BIOS-e801: [mem
0x-0x0009efff] usable

Okay, here should be all memory regions listed. My non-root Linux in my
Qemu VM, for example, shows here:

[ 0.00] BIOS-provided physical RAM map:
[ 0.00] BIOS-e820: [mem 0x-0x000f]
usable
[ 0.00] BIOS-e820: [mem 0x0010-0x00100fff]
reserved
[ 0.00] BIOS-e820: [mem 0x0020-0x048f]
usable

Are you absolutely sure, that you have no further modifications in
Jailhouse or the Linux loader?

Actually, in your case, there should be e820 as well (instead of e801).
Go to the Linux kernel sources, have a look at
arch/x86/kernel/e820.c:1279

and Jailhouse's jailhouse-cell-linux:638.

jailhouse-cell-linux fills information of all memory regions into the
zero page. Would you please instrument code (Linux/Jailhouse) to see
where those regions are not parsed?

 > [    0.00] printk: bootconsole [earlyser0] enabled
 > [    0.00] NX (Execute Disable) protection: active
 > [    0.00] extended physical RAM map:
 > [    0.00] reserve setup_dat

Re: Unable to start non root linux cell

[Ralf Ramsauer]

w_pages: can 
not all       oc memory ]---



Thank you for your continuous support,
Michele

Il giorno sabato 13 gennaio 2024 alle 00:05:43 UTC+1 Ralf Ramsauer ha 
scritto:


Hi Michele,

On 12/01/2024 14:07, Michele Pescapè wrote:
 > jailhouse cell load linux-2 linux-loader.bin -a 0x0
 > ../buildroot-2023.11/output/images/bzImage -a 0xffbe00 parameters
-a 0x1000
 > jailhouse cell start linux-2
 >
 > I take it the kernel is loaded at 0xffbe00 which is right at the
edge of
 > the low ram region. In fact after issuing the cell load command and
 > adjusting the path for the loader I get JAILHOUSE_CELL_LOAD: Invalid
 > argument.

Just tested cell-linux in a qemu machine, there it works, with pretty
similar addresses, which got me confused.

After double-checking it: 0xffb.e00 is *not* at the edge of low mem:

Low mem is 0x000.000 -- 0x0ff.fff
Comm region is 0x100.000 -- 0x100.fff

0xffb.e00 is (obviously) way above.

Please try to set your high mem's .virt_start to 0x20. Then,
0xffbe00 is offsetted ~16MB inside your highmem, and it should work!

IOW:

/* high RAM */
{
.phys_start = 0x4230,
.virt_start = 0x20,
.size = 0x1100,
.flags = JAILHOUSE_MEM_READ | JAILHOUSE_MEM_WRITE |
JAILHOUSE_MEM_EXECUTE |
JAILHOUSE_MEM_LOADABLE,
},

Thanks
Ralf

--
You received this message because you are subscribed to the Google 
Groups "Jailhouse" group.
To unsubscribe from this group and stop receiving emails from it, send 
an email to jailhouse-dev+unsubscr...@googlegroups.com 
<mailto:jailhouse-dev+unsubscr...@googlegroups.com>.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jailhouse-dev/6712361a-eaef-49cc-8a72-79da2c434169n%40googlegroups.com <https://groups.google.com/d/msgid/jailhouse-dev/6712361a-eaef-49cc-8a72-79da2c434169n%40googlegroups.com?utm_medium=email_source=footer>.


--
You received this message because you are subscribed to the Google Groups 
"Jailhouse" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jailhouse-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jailhouse-dev/b4520b71-86ba-48e7-b9ad-c720c4f6cb42%40oth-regensburg.de.

Re: Unable to start non root linux cell

[Ralf Ramsauer]

Hi Michele,

On 12/01/2024 14:07, Michele Pescapè wrote:
jailhouse cell load linux-2 linux-loader.bin -a 0x0 
../buildroot-2023.11/output/images/bzImage -a 0xffbe00 parameters -a 0x1000

jailhouse cell start linux-2

I take it the kernel is loaded at 0xffbe00 which is right at the edge of 
the low ram region. In fact after issuing the cell load command and 
adjusting the path for the loader I get JAILHOUSE_CELL_LOAD: Invalid 
argument.


Just tested cell-linux in a qemu machine, there it works, with pretty 
similar addresses, which got me confused.


After double-checking it: 0xffb.e00 is *not* at the edge of low mem:

Low mem is 0x000.000 -- 0x0ff.fff
Comm region is 0x100.000 -- 0x100.fff

0xffb.e00 is (obviously) way above.

Please try to set your high mem's .virt_start to 0x20. Then, 
0xffbe00 is offsetted ~16MB inside your highmem, and it should work!


IOW:

/* high RAM */
{
.phys_start = 0x4230,
.virt_start = 0x20,
.size = 0x1100,
.flags = JAILHOUSE_MEM_READ | JAILHOUSE_MEM_WRITE |
 JAILHOUSE_MEM_EXECUTE |
 JAILHOUSE_MEM_LOADABLE,
},

Thanks
  Ralf

--
You received this message because you are subscribed to the Google Groups 
"Jailhouse" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jailhouse-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jailhouse-dev/0aa4c10d-a635-4a37-bf51-2b9863beb2a0%40oth-regensburg.de.

Re: Unable to start non root linux cell

[Ralf Ramsauer]

Hi,

On 12/01/2024 14:07, Michele Pescapè wrote:
I'll attach again the non root config, however it isn't much different 
than the other I sent, i only fixed the weird alignment and changed a 
bit the start addresses of the comm region and the high ram region.


ah, and btw, just had a look at it: Don't move the default location of 
the commregion. Leave it there, afair, it has a fix place on x86.


Memory layout for non-root linux is always (in that order):
  - low mem (1MB)
  - CommRegion
  - high mem

The error must be somewhere inside cell-linux.

Thanks
  Ralf

--
You received this message because you are subscribed to the Google Groups 
"Jailhouse" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jailhouse-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jailhouse-dev/f0dcd19a-231d-4c8d-9872-e962193eda03%40oth-regensburg.de.

Re: Unable to start non root linux cell

[Ralf Ramsauer]

[+Cc'ed Stefan, a colleague of mine who recently used non-root Linux on 
x86 systems]


Hi

On 12/01/2024 14:07, Michele Pescapè wrote:

Hi,

The config is the correct one. Nothing from dmesg except

[ 8254.074890] smpboot: CPU 8 is now offline
[ 8254.206652] smpboot: CPU 9 is now offline
[ 8254.254775] Created Jailhouse cell "linux-2"

The hypervisor only reports:

CPU 9 received SIPI, vector 96         (this is the last line of the 
output from the previous attempt)
Created cell "linux-2"    (first line after issuing the cell linux 
command)

Page pool usage after cell creation: mem 272/32211, remap 16392/131072
Cell "linux-2" can be loaded


I tried using the -w flag, this is what it said when using :

jailhouse cell create configs/x86/linux_guest2.cell
jailhouse cell load linux-2 linux-loader.bin -a 0x0 
../buildroot-2023.11/output/images/bzImage -a 0xffbe00 parameters -a 0x1000

jailhouse cell start linux-2

I take it the kernel is loaded at 0xffbe00 which is right at the edge of 
the low ram region. In fact after issuing the cell load command and 


Exactly, bzImage won't fit in here - that's the culprit. And I don't 
know why cell-linux places it there.


Stefan, CC'ed, recently enabled non-root Linux on one of our test 
machines. Stefan, we finally got non-root Linux running, right?


We had other issues -- but why didn't we have that issue here?

Even if I try to use the qemu-demo config, I basically have the same issue:

$ ./tools/jailhouse-cell-linux \
-w test.params -a x86 ./configs/x86/linux-x86-demo.cell \
bzImage -c "console=foobar"

gives:
jailhouse cell create ./configs/x86/linux-x86-demo.cell
jailhouse cell load linux-x86-demo linux-loader.bin -a 0x0 
/home/ralf/workspace/linux/O/x86-qemu/arch/x86/boot/bzImage -a 0xffc800 
test.params -a 0x1000

jailhouse cell start linux-x86-demo

And same here: 0xffc800 is odd. What does our machine (dodecarya) print 
here?


adjusting the path for the loader I get JAILHOUSE_CELL_LOAD: Invalid 
argument.
Nothing is said from dmesg, the hypervisor only says "Cell "linux-2" can 
be loaded".


At this point I tried pointing the address for the kernel in the cell 
load command to the high ram region by giving -a 0x4230 as the address.


Dont' try to work around, we need to get the loader script working, the 
error is somewhere here. Manually adjusting the load address makes 
things worse, as on x86, the params file contains the so-called 
zero-page, and adresses that are encoded there must fit. That's why we 
have the cell-linux helper script that simplifies things.


We don't have to do that ugly zero-page dance on other architectures.

I'll try to do some testing with our machine when I find some time.

Thanks
  Ralf



I get the following output when starting the cell:

Created cell "linux-2"
Page pool usage after cell creation: mem 270/32211, remap 16392/131072
Cell "linux-2" can be loaded
Started cell "linux-2"
CPU 9 received SIPI, vector 100
CPU 8 received SIPI, vector 100
FATAL: unable to get MMIO instruction
FATAL: Invalid MMIO/RAM read, addr: 0x0100 size: 0
RIP: 0x01000200 RSP: 0x9fe8 FLAGS: 46
RAX: 0x01000200 RBX: 0x427d RCX: 0x00ff
RDX: 0x RSI: 0x1000 RDI: 0x
CS: 10 BASE: 0x AR-BYTES: 29b EFER.LMA 1
CR0: 0x80010011 CR3: 0x5000 CR4: 0x00040220
EFER: 0x1500
Parking CPU 9 (Cell: "linux-2")
FATAL: Unexpected #VMEXIT, exitcode 7f, exitinfo1 0x 
exitinfo2 0x

RIP: 0x000f RSP: 0x9fe8 FLAGS: 2
RAX: 0x01000201 RBX: 0x427d RCX: 0x00ff
RDX: 0x RSI: 0x1000 RDI: 0x
CS: 10 BASE: 0x AR-BYTES: 29b EFER.LMA 1
CR0: 0x01000201 CR3: 0x5000 CR4: 0x00040220
EFER: 0x1500

And then it continues to display the vmexit error without stopping if 
the cell isn't destroyed. I can avoid this by increasing the size of the 
low ram region to 0x0020.


That allows me to proceed and starting the cell with cell start, the 
hypervisor output being:


Created cell "linux-2"
Page pool usage after cell creation: mem 272/32211, remap 16392/131072
Cell "linux-2" can be loaded
Started cell "linux-2"
CPU 9 received SIPI, vector 100
CPU 8 received SIPI, vector 100

but without any other sign.

I still need to decrease the kernel size, i only reduced it by 1MB.
I'll attach again the non root config, however it isn't much different 
than the other I sent, i only fixed the weird alignment and changed a 
bit the start addresses of the comm region and the high ram region.


Thanks,
Michele

Il giorno giovedì 11 gennaio 2024 alle 21:55:47 UTC+1 Ralf Ramsauer ha 
scritto:


Hi,

On 11/01/202

Re: Unable to start non root linux cell

[Ralf Ramsauer]

Hi,

On 11/01/2024 13:57, Michele Pescapè wrote:

Hi,

after changing the virt_start to match the phys_start for that memory 
region I got an invalid argument error when starting the cell


Traceback (most recent call last):
   File "/usr/local/libexec/jailhouse/jailhouse-cell-linux", line 739, 
in 

     cell.load(arch.kernel_image, arch.kernel_address())
   File "/usr/local/lib/python3.10/dist-packages/pyjailhouse/cell.py", 
line 44, in load

     fcntl.ioctl(self.dev, self.JAILHOUSE_CELL_LOAD, load)
OSError: [Errno 22] Invalid argument


Are you sure that you created the cell with the correct config? What 
does dmesg say? Any output on the hypervisor console when this error 
happens?




Printing the arch.kernel_address() returned 16760320 (should be 
0xffbe00), so I thought that the bzImage was too big for the low ram region.


jailhouse-cell-linux won't load the bzImage to the low ram region. You 
can use the -w option to dump a file that prints the commands that 
jailhouse-cell-linux would invoke.


I increased the size to .size = 0x0400 and shifted the virt and 
phys_start of the other two regions to match the change (virt_start = 
0x0400 for the comm region and virt=phys_start = 0x4600 for the 
third region) .
No issues were reported by config check but after starting the cell i 
got no output even using earlyprintk.


I also tried using an initrd but I got a similar issue, invalid argument 
when starting the cell, this time it was the arch.ramdisk_address() 
which was 107327488 (0x665b000).
So i tried increasing even more the size of the first region to size = 
0x0a00 (comm region now starts at .virt_start = 0x0a00, and the 
third region starts at  virt=.phys_start = 0x4c00 with size = 
0x600).
Again, no config check issues so I started it but no output again, the 
hypervisor only reports:


Created cell "linux-1"
Page pool usage after cell creation: mem 268/32211, remap 16392/131072
Cell "linux-1" can be loaded
CPU 5 received SIPI, vector 100
CPU 4 received SIPI, vector 100
Started cell "linux-1"

My bzImage is 12,9MB and the rootfs.cpio is 23.1MB if that matters.


Try starting without the ramdisk. We can later take care of the ramdisk, 
but first the kernel needs to start.


For a compressed image, your kernel is really huge. I bet you can boil 
it down to a few MBs. With a kernel of that size, you might want to play 
around with jailhouse-cell-linux -k.


  Ralf



Thanks,
Michele



Il giorno mercoledì 10 gennaio 2024 alle 20:58:03 UTC+1 Ralf Ramsauer ha 
scritto:


Hi Michele,

On 10/01/2024 17:40, Michele Pescapè wrote:
 > Hi,
 >
 > I'm attaching the configurations I'm currently using. I'll try
disabling
 > KASLR.

[…]
/* high RAM */
{
.phys_start = 0x4210,
.virt_start = 0x00101000,
.size = 0x800,
[…]

Alignment of your high ram is odd. Virt & Phys should have the same
alignment. Try to use .phys=.virt=0x4210, I guess that should help.

Thanks,
Ralf


 >
 > Thank you for your help,
 > Michele
 >
     > Il giorno mercoledì 10 gennaio 2024 alle 17:24:35 UTC+1 Ralf
Ramsauer ha
 > scritto:
 >
 > Hi,
 >
 > On 09/01/2024 19:58, Michele Pescapè wrote:
 > > Hi,
 > >
 > > Just an update, I was able to get the following output by
adding -c
 > > "earlyprintk=ttyS0,115200" to the command line:
 > >
 > > Started cell "linux-1"
 > >
 > >
 > > Invalid physical address chosen!
 >
 > could you please share your system and non-root cell configuration?
 >
 > It smells like you have an overlap with the communication region.
You
 > can check that with tools/jailhouse-config-check.
 >
 > >
 > >
 > >
 > >
 > > Physical KASLR disabled: no suitable memory region!
 >
 > Disable KASLR in your kernel for the moment. You don't need it while
 > ramping up your system.
 >
 > Thanks,
 > Ralf
 >
 > >
 > > [    0.00] Linux version 6.2.0-rc3 (root@mp-LINUX-DESKTOP)
 > > (x86_64-buildroot-linux-gnu-gcc.br_real (Buildroot 2023.11)
 > 12.3.0, GNU
 > > ld (GNU Binutils) 2.40) #2 SMP PREEMPT_DYNAMIC Sun Jan  7
 > 18:35:23 CET 2024
 > > [    0.00] Command line: earlyprintk=ttyS0,115200
 > > [    0.00] KERNEL supported cpus:
 > > [    0.00]   Intel GenuineIntel
 > > [    0.00]   AMD AuthenticAMD
 > > [    0.00] x86/fpu: Supporting XSAVE feature 0x001: 'x87
 > floating
 > > point registers'
 > > [    0.00] x86/fpu: Supporting XSAVE feature 0x002: 'SSE
 &g

Re: Unable to start non root linux cell

[Ralf Ramsauer]

Hi Michele,

On 10/01/2024 17:40, Michele Pescapè wrote:

Hi,

I'm attaching the configurations I'm currently using. I'll try disabling 
KASLR.


[…]
/* high RAM */
{
.phys_start = 0x4210,
.virt_start = 0x00101000,
.size = 0x800,
[…]

Alignment of your high ram is odd. Virt & Phys should have the same 
alignment. Try to use .phys=.virt=0x4210, I guess that should help.


Thanks,
  Ralf




Thank you for your help,
Michele

Il giorno mercoledì 10 gennaio 2024 alle 17:24:35 UTC+1 Ralf Ramsauer ha 
scritto:


Hi,

On 09/01/2024 19:58, Michele Pescapè wrote:
 > Hi,
 >
 > Just an update, I was able to get the following output by adding -c
 > "earlyprintk=ttyS0,115200" to the command line:
 >
 > Started cell "linux-1"
 >
 >
 > Invalid physical address chosen!

could you please share your system and non-root cell configuration?

It smells like you have an overlap with the communication region. You
can check that with tools/jailhouse-config-check.

 >
 >
 >
 >
 > Physical KASLR disabled: no suitable memory region!

Disable KASLR in your kernel for the moment. You don't need it while
ramping up your system.

Thanks,
Ralf

 >
 > [    0.00] Linux version 6.2.0-rc3 (root@mp-LINUX-DESKTOP)
 > (x86_64-buildroot-linux-gnu-gcc.br_real (Buildroot 2023.11)
12.3.0, GNU
 > ld (GNU Binutils) 2.40) #2 SMP PREEMPT_DYNAMIC Sun Jan  7
18:35:23 CET 2024
 > [    0.00] Command line: earlyprintk=ttyS0,115200
 > [    0.00] KERNEL supported cpus:
 > [    0.00]   Intel GenuineIntel
 > [    0.00]   AMD AuthenticAMD
 > [    0.00] x86/fpu: Supporting XSAVE feature 0x001: 'x87
floating
 > point registers'
 > [    0.00] x86/fpu: Supporting XSAVE feature 0x002: 'SSE
registers'
 > [    0.00] x86/fpu: Supporting XSAVE feature 0x004: 'AVX
registers'
 > [    0.00] x86/fpu: xstate_offset[2]:  576, xstate_sizes[2]:
  256
 > [    0.00] x86/fpu: Enabled xstate features 0x7, context size
is 832
 > bytes, using 'compacted' format.
 > [    0.00] signal: max sigframe size: 1776
 > [    0.00] BIOS-provided physical RAM map:
 > [    0.00] BIOS-e801: [mem
0x-0x0009efff] usable
 > [    0.00] printk: bootconsole [earlyser0] enabled
 > [    0.00] NX (Execute Disable) protection: active
 > [    0.00] extended physical RAM map:
 > [    0.00] reserve setup_data: [mem
 > 0x-0x1fff] usable
 > [    0.00] reserve setup_data: [mem
 > 0x2000-0x212b] usable
 > [    0.00] reserve setup_data: [mem
 > 0x212c-0x0009efff] usable
 > [    0.00] DMI not present or invalid.
 > [    0.00] Hypervisor detected: Jailhouse
 > [    0.00] tsc: Detected 3393.624 MHz processor
 > [    0.19] .text .data .bss are not marked as E820_TYPE_RAM!
 > [    0.005751] last_pfn = 0x3630 max_arch_pfn = 0x4
 > [    0.011039] x86/PAT: Configuration [0-7]: WB  WC  UC- UC  WB
  WP  UC- WT
 > Memory KASLR using RDRAND RDTSC...
 > [    0.032374] Using GB pages for direct mapping
 > [    0.036743] Kernel panic - not syncing: alloc_low_pages: can not
 > alloc memory
 > [    0.043712] CPU: 0 PID: 0 Comm: swapper Not tainted 6.2.0-rc3 #2
 > [    0.049702] Call Trace:
 > [    0.052134]  
 > [    0.054219]  dump_stack_lvl+0x3a/0x58
 > [    0.057865]  dump_stack+0x10/0x18
 > [    0.061162]  panic+0x15c/0x308
 > [    0.064205]  alloc_low_pages+0x7d/0x1c0
 > [    0.068024]  phys_pmd_init+0x1f2/0x2eb
 > [    0.071758]  phys_pud_init+0x114/0x2d5
 > [    0.075493]  phys_p4d_init+0x4c/0x269
 > [    0.079135]  ?
__raw_callee_save___native_queued_spin_unlock+0x15/0x30
 > [    0.085649]  __kernel_physical_mapping_init+0x11e/0x29a
 > [    0.090859]  kernel_physical_mapping_init+0xf/0x20
 > [    0.095631]  init_memory_mapping+0x273/0x410
 > [    0.099890]  init_range_memory_mapping+0xec/0x150
 > [    0.104574]  init_mem_mapping+0x268/0x2e7
 > [    0.108567]  setup_arch+0x839/0xdcf
 > [    0.112039]  ? lockdown_lsm_init+0x21/0x2f
 > [    0.116120]  start_kernel+0x6e/0x997
 > [    0.119680]  x86_64_start_reservations+0x24/0x2c
 > [    0.124281]  x86_64_start_kernel+0xec/0xff
 > [    0.128360]  secondary_startup_64_no_verify+0xe5/0xeb
 > [    0.133400]  
 > [    0.135570] ---[ end Kernel panic - not sync

Re: Unable to start non root linux cell

[Ralf Ramsauer]

recompiled it just in case.

Michele

Il giorno martedì 9 gennaio 2024 alle 14:14:33 UTC+1 Ralf
Ramsauer ha scritto:

Hi,

On 08/01/2024 23:56, Michele Pescapè wrote:
 > Hi all,
 >
 > I have an AMD Ryzen 2600 on a b450 motherboard. My goal
is to run two
 > non root linux cells, however, as of right now, I'm not
able to start
 > any non root linux cells.
 > First of all I don't receive an output from the cell (I
do get output
 > from the apic-demo though), therefore I don't really know
if they're
 > even crashing or not; this is the output I get from the
hypervisor after
 > enabling the root cell and issuing the following command:
 >
 > sudo jailhouse cell linux configs/x86/linux_guest1.cell
 > ../buildroot-2023.11/output/images/bzImage -i
 > ../buildroot-2023.11/output/images/rootfs.cpio

for the first few tries, simply don't load a ramdisk. Try to
get the
kernel booting. It will crash with "cannot mount rootfs". If
we get so
far, then you can continue specifying the ramdisk. But
first, we have to
get that far.

 >
 > Adding PCI device ff:1f.7 to cell "linux-x86-demo"
 > Created cell "linux-x86-demo"
 > Page pool usage after cell creation: mem 336/32211, remap
16392/131072
 > Cell "linux-x86-demo" can be loaded
 > CPU 4 received SIPI, vector 100
 > CPU 3 received SIPI, vector 100
 > Started cell "linux-x86-demo"
 >
 > After this the cell appears to be running in cell list,
however I can't
 > seem to get any output to confirm that (I even set
 > JAILHOUSE_CELL_VIRTUAL_CONSOLE_ACTIVE, but nothing
appears on the vrtual
 > console accessed via "jailhouse console -f"). I also
tried adding -c
 > "console=ttyS0,115200".

ttyS0, i.e., x86 Port 0x3f8 is whitelisted in the cell
configuration?
Actually, your cmdline is correct.

 > At this point if I try to disable/shutdown the cell, the
system freezes
 > requiring a hard reset.

Okay, is the guest Linux compiled with Jailhouse guest
support? Please
ensure that non-root Linux is compiled with
CONFIG_JAILHOUSE_GUEST and
CONFIG_JAILHOUSE_DBCON. The first one is mandatory for x86
Linux guests.

 >
 > Also, there seems to be a mismatch between the PCI bdf
value in the log
 > above and the one in the cell's configuration. In both
root and non root
 > cells i set .bdf = (0x0f << 3) for the IVSHMEM net device
and the Root
 > cell correctly reports that ("Adding virtual PCI device
00:0f.0 to cell
 > "RootCell"").

First things first. Let's try to get Linux kicked off before
diving into
PCI. Comment it out for the moment, let's do that later.

Ralf

 >
 > If I understood correctly, once the cell works properly I
won't need to
 > set the console anymore as I can assign an ip (-c "ip
x.x.x.x" in the
 > cell linux command) and ssh to the cell, right?
 >
 > I attach the root and non root cells' configurations in
case they're
 > needed. I can provide both the root and non root cell's
kernel .conf and
 > the buildroot configuration I used if needed.
 >
 > Thank you for your time,
 > Michele
 >
 > --
 > You received this message because you are subscribed to
the Google
 > Groups "Jailhouse" group.
 > To unsubscribe from this group and stop receiving emails
from it, send
 > an email to jailhouse-de...@googlegroups.com
 > <mailto:jailhouse-de...@googlegroups.com>.
 > To view this discussion on the web visit
 >

https://groups.google.com/d/msgid/jailhouse-dev/824bacc4-b7ae-47c9-878e-7203214b4fc3n%40googlegroups.com 
<https://groups.google.com/d/msgid/jailhouse-dev/824bacc4-b7ae-47c9-878e-7203214b4fc3n%40googlegroups.com>

Re: Unable to start non root linux cell

[Ralf Ramsauer]

Hi,

On 08/01/2024 23:56, Michele Pescapè wrote:

Hi all,

I have an AMD Ryzen 2600 on a b450 motherboard. My goal is to run two 
non root linux cells, however, as of right now, I'm not able to start 
any non root linux cells.
First of all I don't receive an output from the cell (I do get output 
from the apic-demo though), therefore I don't really know if they're 
even crashing or not; this is the output I get from the hypervisor after 
enabling the root cell and issuing the following command:


sudo jailhouse cell linux configs/x86/linux_guest1.cell 
../buildroot-2023.11/output/images/bzImage -i 
../buildroot-2023.11/output/images/rootfs.cpio


for the first few tries, simply don't load a ramdisk. Try to get the 
kernel booting. It will crash with "cannot mount rootfs". If we get so 
far, then you can continue specifying the ramdisk. But first, we have to 
get that far.




Adding PCI device ff:1f.7 to cell "linux-x86-demo"
Created cell "linux-x86-demo"
Page pool usage after cell creation: mem 336/32211, remap 16392/131072
Cell "linux-x86-demo" can be loaded
CPU 4 received SIPI, vector 100
CPU 3 received SIPI, vector 100
Started cell "linux-x86-demo"

After this the cell appears to be running in cell list, however I can't 
seem to get any output to confirm that (I even set 
JAILHOUSE_CELL_VIRTUAL_CONSOLE_ACTIVE, but nothing appears on the vrtual 
console accessed via "jailhouse console -f"). I also tried adding -c 
"console=ttyS0,115200".


ttyS0, i.e., x86 Port 0x3f8 is whitelisted in the cell configuration? 
Actually, your cmdline is correct.


At this point if I try to disable/shutdown the cell, the system freezes 
requiring a hard reset.


Okay, is the guest Linux compiled with Jailhouse guest support? Please 
ensure that non-root Linux is compiled with CONFIG_JAILHOUSE_GUEST and 
CONFIG_JAILHOUSE_DBCON. The first one is mandatory for x86 Linux guests.




Also, there seems to be a mismatch between the PCI bdf value in the log 
above and the one in the cell's configuration. In both root and non root 
cells i set .bdf = (0x0f << 3) for the IVSHMEM net device and the Root 
cell correctly reports that ("Adding virtual PCI device 00:0f.0 to cell 
"RootCell"").


First things first. Let's try to get Linux kicked off before diving into 
PCI. Comment it out for the moment, let's do that later.


  Ralf



If I understood correctly, once the cell works properly I won't need to 
set the console anymore as I can assign an ip (-c "ip x.x.x.x" in the 
cell linux command) and ssh to the cell, right?


I attach the root and non root cells' configurations in case they're 
needed. I can provide both the root and non root cell's kernel .conf and 
the buildroot configuration I used if needed.


Thank you for your time,
Michele

--
You received this message because you are subscribed to the Google 
Groups "Jailhouse" group.
To unsubscribe from this group and stop receiving emails from it, send 
an email to jailhouse-dev+unsubscr...@googlegroups.com 
.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jailhouse-dev/824bacc4-b7ae-47c9-878e-7203214b4fc3n%40googlegroups.com .


--
You received this message because you are subscribed to the Google Groups 
"Jailhouse" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jailhouse-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jailhouse-dev/d36e18a4-dd64-4f9c-b2cf-89b02b1a0469%40oth-regensburg.de.

Re: i.MX8MP jailhouse load image stuck at copy_from_user

[Ralf Ramsauer]

Hi,

On 09/01/2024 13:33, wheatfox wrote:
Here is my current linux inmate cell configuration for my OK8MP 
board(imx8mp CPU, 2G RAM + 16G eMMC version), I didn't change the 
imx8mp.c for now.


I don't understand what you're trying to tell.


I edited the RAM region as:
/* RAM */ {
/*
  * We could not use 0x8000 which conflicts with
  * COMM_REGION_BASE


And I don't understand this comment.


  */
.phys_start = 0x6000,
.virt_start = 0x6000,


For non-root cells, host physical and guest physical memory doesn't 
necessarily have to be identity mapped.


  Ralf


.size = 0x1000,
.flags = JAILHOUSE_MEM_READ | JAILHOUSE_MEM_WRITE |
JAILHOUSE_MEM_EXECUTE | JAILHOUSE_MEM_DMA |
JAILHOUSE_MEM_LOADABLE,
},
On Tuesday, January 9, 2024 at 8:19:59 PM UTC+8 wheatfox wrote:

Thanks for your instruction! I tried to edit the linux inmate cell
configuration file (imx8mp-linux-demo.c) and I finally get the linux
inmate to boot, so I think it was mainly a memory region
misconfiguration problem. (However, the boot progress failed due to
VFS fatal error, which I will investigate later...)
The current full output log is attached as attachment.

On Monday, January 8, 2024 at 7:12:46 PM UTC+8 Ralf Ramsauer wrote:

Hi,

On 08/01/2024 09:26, wheatfox wrote:
 > I'm using OK8MP board with i.MX8MP CPU. I have already
started jailhouse
 > with imx8mp.cell configuration, and the example gic-demo
works fine.
 > However, when I try to start a linux inmate using
 > imx8mp-linux-demo.cell, the jailhouse's load progress seems
stuck.
 >
 > After adding some debug print in jailhouse's python script,
firmware
 > code and kernel module code, I found out that the
*copy_from_user* call
 > in *load_image* function (driver/cell.c) doesn't return and
halt forever: >
 > jailhouse commands:
 > ./tools/jailhouse enable ./imx8mp.cell
 > ./tools/jailhouse cell linux \
 > ./imx8mp-linux-demo.cell \
 > ./kernel/Image \
 > -i ./kernel/ramdisk.img \

For testing, try to not load the ramdisk. The kernel should at
least
boot and crash. If that works, we know that there's something
odd with
the ramdisk.

Second, double and triple check addresses where things get loaded.
What's in your non-root cell configuration, and where does the
linux-loader try to load stuff? Does that match?

Thanks,
Ralf

 > -d ./kernel/imx8mp-evk-inmate-wheatfox.dtb \
 > -c "clk_ignore_unused console=ttymxc1,0x3089,115200
 > earlycon=ec_imx6q,0x3089,115200"
 >
 > start linux cell output(with the string 'wheatfox' means it's
my custom
 > debug print):
 > root@OK8MP:/mnt# ./start-linux.sh
 > [wheatfox|python] Jailhouse Linux Cell Boot Helper
 > [wheatfox|python] linux_loader=/m[  129.965933] [wheatfox]
 > (jailhouse_ioctl) ioctl=0x5401 arg=281474720244744
 > nt/tools/../inmates/tools/arm64/l[  129.975611] [wheatfox]
 > (jailhouse_ioctl) ioctl=0x5401 arg=281474720244728
 > inux-loader.bin
 > [  129.988033] [wheatfox] (jailhouse_ioctl) ioctl=0x4012
 > arg=281474720244800
 > [  129.995192] [wheatfox] (jailhouse_ioctl)
JAILHOUSE_CELL_CREATE
 > [  130.001072] [wheatfox] (jailhouse_cmd_cell_create) start
 > [  130.006404] [wheatfox] (jailhouse_cmd_cell_create)
copy_from_user done
 > [  130.012960] [wheatfox] (jailhouse_cmd_cell_create)
copy_from_user done
 > [  130.019507] [wheatfox] (jailhouse_cmd_cell_create)
cell_id.id <http://cell_id.id> = -1
 > [  130.095151] IRQ 6: no longer affine to CPU2
 > [  130.095333] CPU2: shutdown
 > [  130.102251] psci: CPU2 killed (polled 0 ms)
 > [  130.155283] CPU3: shutdown
 > [  130.157999] psci: CPU3 killed (polled 0 ms)
 > [wheatfox] in hypercall, code = 1, arg1 = 2955416576, arg2 = 2
 > [wheatfox] in hypercall, JAILHOUSE_HC_CELL_CREATE
 > Adding virtual PCI device 00:00.0 to cell "linux-inmate-demo"
 > Shared memory connection established, peer cells:
 >  "imx8mp"
 > Adding virtual PCI device 00:01.0 to cell "linux-inmate-demo"
 > Shared memory connection established, peer cells:
 >  "imx8mp"
 > [wheatfox] in resume_cpu, cpu_id = 2
 > [wheatfox] in resume_cpu, target_data->cpu_suspended = 1
 > [wheatfox] in resume_cpu, cpu_id = 3
 > [wheatfox] in resume_cpu, target_data->cpu_suspended = 1
   

Re: i.MX8MP jailhouse load image stuck at copy_from_user

[Ralf Ramsauer]

Hi,

read your log carefully:

[0.444215] Unpacking initramfs...
[0.447384] Initramfs unpacking failed: invalid magic at start of 
compressed archive


Something is odd with your ramdisk. Maybe the wrong size is passed?

Thanks,
  Ralf

On 09/01/2024 13:19, wheatfox wrote:
Thanks for your instruction! I tried to edit the linux inmate cell 
configuration file (imx8mp-linux-demo.c) and I finally get the linux 
inmate to boot, so I think it was mainly a memory region 
misconfiguration problem. (However, the boot progress failed due to VFS 
fatal error, which I will investigate later...)

The current full output log is attached as attachment.

On Monday, January 8, 2024 at 7:12:46 PM UTC+8 Ralf Ramsauer wrote:

Hi,

On 08/01/2024 09:26, wheatfox wrote:
 > I'm using OK8MP board with i.MX8MP CPU. I have already started
jailhouse
 > with imx8mp.cell configuration, and the example gic-demo works fine.
 > However, when I try to start a linux inmate using
 > imx8mp-linux-demo.cell, the jailhouse's load progress seems stuck.
 >
 > After adding some debug print in jailhouse's python script, firmware
 > code and kernel module code, I found out that the
*copy_from_user* call
 > in *load_image* function (driver/cell.c) doesn't return and halt
forever: >
 > jailhouse commands:
 > ./tools/jailhouse enable ./imx8mp.cell
 > ./tools/jailhouse cell linux \
 > ./imx8mp-linux-demo.cell \
 > ./kernel/Image \
 > -i ./kernel/ramdisk.img \

For testing, try to not load the ramdisk. The kernel should at least
boot and crash. If that works, we know that there's something odd with
the ramdisk.

Second, double and triple check addresses where things get loaded.
What's in your non-root cell configuration, and where does the
linux-loader try to load stuff? Does that match?

Thanks,
Ralf

 > -d ./kernel/imx8mp-evk-inmate-wheatfox.dtb \
 > -c "clk_ignore_unused console=ttymxc1,0x3089,115200
 > earlycon=ec_imx6q,0x3089,115200"
 >
 > start linux cell output(with the string 'wheatfox' means it's my
custom
 > debug print):
 > root@OK8MP:/mnt# ./start-linux.sh
 > [wheatfox|python] Jailhouse Linux Cell Boot Helper
 > [wheatfox|python] linux_loader=/m[  129.965933] [wheatfox]
 > (jailhouse_ioctl) ioctl=0x5401 arg=281474720244744
 > nt/tools/../inmates/tools/arm64/l[  129.975611] [wheatfox]
 > (jailhouse_ioctl) ioctl=0x5401 arg=281474720244728
 > inux-loader.bin
 > [  129.988033] [wheatfox] (jailhouse_ioctl) ioctl=0x4012
 > arg=281474720244800
 > [  129.995192] [wheatfox] (jailhouse_ioctl) JAILHOUSE_CELL_CREATE
 > [  130.001072] [wheatfox] (jailhouse_cmd_cell_create) start
 > [  130.006404] [wheatfox] (jailhouse_cmd_cell_create)
copy_from_user done
 > [  130.012960] [wheatfox] (jailhouse_cmd_cell_create)
copy_from_user done
 > [  130.019507] [wheatfox] (jailhouse_cmd_cell_create) cell_id.id
<http://cell_id.id> = -1
 > [  130.095151] IRQ 6: no longer affine to CPU2
 > [  130.095333] CPU2: shutdown
 > [  130.102251] psci: CPU2 killed (polled 0 ms)
 > [  130.155283] CPU3: shutdown
 > [  130.157999] psci: CPU3 killed (polled 0 ms)
 > [wheatfox] in hypercall, code = 1, arg1 = 2955416576, arg2 = 2
 > [wheatfox] in hypercall, JAILHOUSE_HC_CELL_CREATE
 > Adding virtual PCI device 00:00.0 to cell "linux-inmate-demo"
 > Shared memory connection established, peer cells:
 >  "imx8mp"
 > Adding virtual PCI device 00:01.0 to cell "linux-inmate-demo"
 > Shared memory connection established, peer cells:
 >  "imx8mp"
 > [wheatfox] in resume_cpu, cpu_id = 2
 > [wheatfox] in resume_cpu, target_data->cpu_suspended = 1
 > [wheatfox] in resume_cpu, cpu_id = 3
 > [wheatfox] in resume_cpu, target_data->cpu_suspended = 1
 > Created cell "linux-inmate-demo"
 > Page pool usage after cell creation: mem 82/992, remap 144/131072
 > [wheatfox] in resume_cpu, cpu_id = 0
 > [wheatfox] in resume_cpu, target_data->cpu_suspended = 1
 > [  130.231800] [wheatfox] (jailhouse_cmd_cell_create) Created cell
 > "linux-inmate-demo"
 > [wheatfox|python] cell created,
 >

name=b'linux-inmate-demo\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'
 > [wheatf[  130.249264] [wheatfox] (jailhouse_ioctl) ioctl=0x4033
 > arg=281474720245360
 > ox|python] trying to load linux_l[  130.257905] [wheatfox]
 > (jailhouse_ioctl) JAILHOUSE_CELL_LOAD
 > oader.bin into cell, addr=0x0
 > [  130.266684] [wheatfox] (jailhouse_cmd_cell_load) start
 > [  130.27432

Re: i.MX8MP jailhouse load image stuck at copy_from_user

[Ralf Ramsauer]

Hi,

On 08/01/2024 09:26, wheatfox wrote:
I'm using OK8MP board with i.MX8MP CPU. I have already started jailhouse 
with imx8mp.cell configuration, and the example gic-demo works fine. 
However, when I try to start a linux inmate using 
imx8mp-linux-demo.cell, the jailhouse's load progress seems stuck.


After adding some debug print in jailhouse's python script, firmware 
code and kernel module code, I found out that the *copy_from_user* call 
in *load_image* function (driver/cell.c) doesn't return and halt forever: >

jailhouse commands:
./tools/jailhouse enable ./imx8mp.cell
./tools/jailhouse cell linux \
./imx8mp-linux-demo.cell \
./kernel/Image \
-i ./kernel/ramdisk.img \


For testing, try to not load the ramdisk. The kernel should at least 
boot and crash. If that works, we know that there's something odd with 
the ramdisk.


Second, double and triple check addresses where things get loaded. 
What's in your non-root cell configuration, and where does the 
linux-loader try to load stuff? Does that match?


Thanks,
  Ralf


-d ./kernel/imx8mp-evk-inmate-wheatfox.dtb \
-c "clk_ignore_unused console=ttymxc1,0x3089,115200 
earlycon=ec_imx6q,0x3089,115200"


start linux cell output(with the string 'wheatfox' means it's my custom 
debug print):

root@OK8MP:/mnt# ./start-linux.sh
[wheatfox|python] Jailhouse Linux Cell Boot Helper
[wheatfox|python] linux_loader=/m[  129.965933] [wheatfox] 
(jailhouse_ioctl) ioctl=0x5401 arg=281474720244744
nt/tools/../inmates/tools/arm64/l[  129.975611] [wheatfox] 
(jailhouse_ioctl) ioctl=0x5401 arg=281474720244728

inux-loader.bin
[  129.988033] [wheatfox] (jailhouse_ioctl) ioctl=0x4012 
arg=281474720244800

[  129.995192] [wheatfox] (jailhouse_ioctl) JAILHOUSE_CELL_CREATE
[  130.001072] [wheatfox] (jailhouse_cmd_cell_create) start
[  130.006404] [wheatfox] (jailhouse_cmd_cell_create) copy_from_user done
[  130.012960] [wheatfox] (jailhouse_cmd_cell_create) copy_from_user done
[  130.019507] [wheatfox] (jailhouse_cmd_cell_create) cell_id.id = -1
[  130.095151] IRQ 6: no longer affine to CPU2
[  130.095333] CPU2: shutdown
[  130.102251] psci: CPU2 killed (polled 0 ms)
[  130.155283] CPU3: shutdown
[  130.157999] psci: CPU3 killed (polled 0 ms)
[wheatfox] in hypercall, code = 1, arg1 = 2955416576, arg2 = 2
[wheatfox] in hypercall, JAILHOUSE_HC_CELL_CREATE
Adding virtual PCI device 00:00.0 to cell "linux-inmate-demo"
Shared memory connection established, peer cells:
  "imx8mp"
Adding virtual PCI device 00:01.0 to cell "linux-inmate-demo"
Shared memory connection established, peer cells:
  "imx8mp"
[wheatfox] in resume_cpu, cpu_id = 2
[wheatfox] in resume_cpu, target_data->cpu_suspended = 1
[wheatfox] in resume_cpu, cpu_id = 3
[wheatfox] in resume_cpu, target_data->cpu_suspended = 1
Created cell "linux-inmate-demo"
Page pool usage after cell creation: mem 82/992, remap 144/131072
[wheatfox] in resume_cpu, cpu_id = 0
[wheatfox] in resume_cpu, target_data->cpu_suspended = 1
[  130.231800] [wheatfox] (jailhouse_cmd_cell_create) Created cell 
"linux-inmate-demo"
[wheatfox|python] cell created, 
name=b'linux-inmate-demo\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'
[wheatf[  130.249264] [wheatfox] (jailhouse_ioctl) ioctl=0x4033 
arg=281474720245360
ox|python] trying to load linux_l[  130.257905] [wheatfox] 
(jailhouse_ioctl) JAILHOUSE_CELL_LOAD

oader.bin into cell, addr=0x0
[  130.266684] [wheatfox] (jailhouse_cmd_cell_load) start
[  130.274322] [wheatfox] (jailhouse_cmd_cell_load) 
cell_management_prologue(_load.cell_id, ) done

[wheatfox] in hypercall, code = 3, arg1 = 1, arg2 = 4308871775073466112
[wheatfox] in hypercall, JAILHOUSE_HC_CELL_SET_LOADABLE
[wheatfox] in cell_set_loadable, id = 1
[wheatfox] cell_management_prologue finished
[wheatfox] in resume_cpu, cpu_id = 2
[wheatfox] in resume_cpu, target_data->cpu_suspended = 1
[wheatfox] in resume_cpu, cpu_id = 3
[wheatfox] in resume_cpu, target_data->cpu_suspended = 1
[wheatfox] finished arch_park_cpu
[wheatfox] cell->loadable = 0
[wheatfox] not jumped to out_resume
Cell "linux-inmate-demo" can be loaded
[wheatfox] in out_resume
[wheatfox] in resume_cpu, cpu_id = 1
[wheatfox] in resume_cpu, target_data->cpu_suspended = 1
[  130.342417] [wheatfox] (jailhouse_cmd_cell_load) 
jailhouse_call_arg1(JAILHOUSE_HC_CELL_SET_LOADABLE, cell->id) done
[  130.353079] [wheatfox] (jailhouse_cmd_cell_load) 
cell_load.num_preload_images = 1
[  130.360584] [wheatfox] (jailhouse_cmd_cell_load) load_image(cell, 
image) start, n = 1

[  130.368444] [wheatfox] (load_image) start
[  130.372468] [wheatfox] (load_image) copy_from_user done
[  130.377712] [wheatfox] (load_image) image.size = 34b0
[  130.382776] [wheatfox] (load_image) found suitable memory region, 
mem->virt_start = 0, mem->size = 1

[  130.392280] [wheatfox] (load_image) image load mem region found
[  130.398228] [wheatfox] (load_image) phys_start = fdb0
[  130.403653] [wheatfox] (load_image) page_offs = 0
[  130.408373] [wheatfox] (load_image) 

Re: Fail to create sample cell on Intel N6005

[Ralf Ramsauer]

Hi Benoit,

On 02/01/2024 17:03, Benoit wrote:

Hello,
I have an Odroid H3+ (Intel N6005 - 4 cores) with 8GB RAM.
I successfully recompiled the kernel (5.10) with jailhouse support and 
installed jailhouse


Loading the kernel module works fine but the following steps make my 
system unstable and I am unable to create a sample cell (I tried 
tiny-demo and acpi-demo)


 From what I got:
* I have the memory reserved upon kernel startup according to settings 
generated by jailhouse config create.

* enabling jailhouse works "pretty" reliably.


Please be more precise: what means pretty reliable?

* starting a non root cell cause kernel crashes : a core is not 
responding. (as reported by dmesg)


Please attach the corresponding log.


Afterwards, my system ends up hanging.


Sounds like a crash.



I didn't touch anything else and I don't have (yet)  access to the 
serial console (going to look for a 3.3V Serial to USB adapter).


This will very likely be required now. You could also try the 'jailhouse 
console', but it probably won't be very helpful if we have a crash.


Thanks
  Ralf



Did I miss something ?

--
You received this message because you are subscribed to the Google 
Groups "Jailhouse" group.
To unsubscribe from this group and stop receiving emails from it, send 
an email to jailhouse-dev+unsubscr...@googlegroups.com 
.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jailhouse-dev/e06c1b28-3007-421a-960b-a298cfaba7ebn%40googlegroups.com .


--
You received this message because you are subscribed to the Google Groups 
"Jailhouse" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jailhouse-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jailhouse-dev/dc35efeb-302f-48d8-bad7-daa678a4ae75%40oth-regensburg.de.

Re: Can't install jailhouse on linux-6.1 arm system

[Ralf Ramsauer]

On 04/01/2024 11:27, Dave Marples wrote:
The patch is pretty straightforward and attached to the message above, 
but in case you're having trouble getting it here it is as text;


```diff --git a/mm/vmalloc.c b/mm/vmalloc.c index 
52f50fe7db83..7fbfad571968 100644 --- a/mm/vmalloc.c +++ b/mm/vmalloc.c 
@@ -316,7 +316,7 @@ int ioremap_page_range(unsigned long addr, unsigned 
long end, { int err; - err = vmap_range_noflush(addr, end, phys_addr, 
pgprot_nx(prot), + err = vmap_range_noflush(addr, end, phys_addr, prot, 
ioremap_max_page_shift); flush_cache_vmap(addr, end); return err; ```

Good luck!


Oh boy, good luck is really what is required here. ;)

Find all Jailhouse-related kernel patches here:
https://github.com/siemens/linux (see jailhouse-enabling branches)
and here:
https://git.kiszka.org/?p=linux.git;a=shortlog;h=refs/heads/queues/jailhouse

  Ralf


DAVE

--
You received this message because you are subscribed to the Google 
Groups "Jailhouse" group.
To unsubscribe from this group and stop receiving emails from it, send 
an email to jailhouse-dev+unsubscr...@googlegroups.com 
.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jailhouse-dev/c33ce025-503c-4a9e-b17d-c792587adbb8n%40googlegroups.com .


--
You received this message because you are subscribed to the Google Groups 
"Jailhouse" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jailhouse-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jailhouse-dev/6abe410f-8ceb-4af0-b393-10efc03edda3%40oth-regensburg.de.

Re: Jailhouse on OKMX8mq--C , "unable to open an initial console"

[Ralf Ramsauer]

On 25/11/2023 07:57, 杨竣轶 wrote:
I'm using a OKMX8MQ-C which from imx8, now i can boot the root cell and 
some simple demp such as gic-demo. But when boot the non-root linux 
cell, i face some problem.


I only use initramfs to boot non-root linux. Furthermore, my initramfs 
only contains a helloworld and a console device. [I cannot use busybox 
to build initramfs because my toolchain does not allow me to compile 
it]. My ramfs directory is as follows：截屏2023-11-25 14.49.49.png


Then my startup command is as follows:

*cd /home/root/imx_jailhouse *
*insmod driver/jailhouse.ko *
*export PATH=/home/root/imx_jailhouse/tools:$PATH *
*jailhouse enable configs/arm64/imx8mq-veth.cell
*
*jailhouse cell linux configs/arm64/imx8mq-linux-demo.cell \ *
*                                 ~/Image2 -d \ *
*                                 ~/imx8mq-evk-inmate.dtb -i \ *
*                                  ~/rootfs.cpio.gz \ *
*                                 -c "clk_ignore_unused 
console=ttymxc0,3086,115200 earlycon=ec_imx6q,0x3086,115200 
root=/dev/ram ramdisk_size=1000 rdinit=/hello rootwait rw " *


try to add console=tty0 here. Further, did you set stdout-path in yor 
non root's device tree correctly?


> console=ttymxc0,3086,115200
Are you sure that this line is correct? Why do you encode the address here?

Please try:
console=ttymxc0,115200 console=tty0

  Ralf


*
*
Eventually, I got the following output：*
*
root@imx8mqevk:~/imx_jailhouse# jailhouse cell linux 
configs/arm64/imx8mq-linux-demo.cell \
_unused console=ttymxc0,3086,115200 
earlycon=ec_imx6q,0x3086,115200 root=/dev/ram ramdisk_size=1000 
rdinit=/hello rootwait rw "

 >      ~/Image2 -d \
 >     ~/imx8mq-evk-inmate.dtb -i \
 >      ~/rootfs.cpio.gz \
 >  -c "clk_ignore_unused console=ttymxc0,3086,115200 
earlycon=ec_imx6q,0x3086,115200 root=/dev/ram ramdisk_size=1000 
rdinit=/hello rootwait rw "

[  567.018158] Ready to search binary handler...
[  567.023161] Finish search binary handler, ret=0...
[  567.029032] Ready to search binary handler...
[  567.033683] Finish search binary handler, ret=0...
[  567.039395] Ready to search binary handler...
[  567.044092] Finish search binary handler, ret=0...


[  567.365079] IRQ231: set affinity failed(-22).
[  567.365091] IRQ232: set affinity failed(-22).
[  567.365646] CPU2: shutdown
[  567.377126] psci: CPU2 killed.
[  567.424984] IRQ231: set affinity failed(-22).
[  567.424991] IRQ232: set affinity failed(-22).
[  567.425300] CPU3: shutdown
[  567.436789] psci: CPU3 killed.
Adding virtual PCI device 00:00.0 to cell "linux-inmate-demo"
Shared memory connection established: "linux-inmate-demo" <--> "imx8mq"
iommu_config_commit linux-inmate-demo
Created cell "linux-inmate-demo"
Page pool usage after cell creation: mem 111/995, remap 144/131072
[  567.473937] Created Jailhouse cell "linux-inmate-demo"
Cell "linux-inmate-demo" can be loaded

Started cell "linux-inmate-demo"

[    0.00] Booting Linux on physical CPU 0x02 [0x410fd034]
[    0.00] Linux version 5.4.3 (azureuser@junyiyang-test) (gcc 
version 9.2.0 (GCC)) #46 SMP PREEMPT Fri Nov 24 09:06:54 UTC 2023

[    0.00] Machine model: Freescale i.MX8MQ EVK
[    0.00] earlycon: ec_imx6q0 at MMIO 0x3086 (options 
'115200')

[    0.00] printk: bootconsole [ec_imx6q0] enabled
[    0.00] efi: Getting EFI parameters from FDT:
[    0.00] efi: UEFI not found.
[    0.00] cma: Reserved 320 MiB at 0xe400
root@imx8mqevk:~/imx_jailhouse#
root@imx8mqevk:~/imx_jailhouse#
[    0.00] NUMA: No NUMA configuration found
[    0.00] NUMA: Faking a node at [mem 
0xd000-0xf9bf]

[    0.00] NUMA: NODE_DATA [mem 0xf9a97500-0xf9a98fff]
[    0.00] Zone ranges:
[    0.00]   DMA32    [mem 0xd000-0xf9bf]
[    0.00]   Normal   empty
[    0.00] Movable zone start for each node
[    0.00] Early memory node ranges
[    0.00]   node   0: [mem 0xd000-0xf9bf]
[    0.00] Initmem setup node 0 [mem 
0xd000-0xf9bf]

root@imx8mqevk:~/imx_jailhouse#
[    0.00] psci: probing for conduit method from DT.
[    0.00] psci: PSCIv1.1 detected in firmware.
[    0.00] psci: Using standard PSCI v0.2 function IDs
[    0.00] psci: MIGRATE_INFO_TYPE not supported.
[    0.00] psci: SMC Calling Convention v1.1
[    0.00] percpu: Embedded 24 pages/cpu s58968 r8192 d31144 u98304
[    0.00] Detected VIPT I-cache on CPU0
[    0.00] CPU features: detected: ARM erratum 845719
[    0.00] CPU features: detected: GIC system register CPU interface
[    0.00] Built 1 zonelists, mobility grouping on.  Total pages: 168336
[    0.00] Policy zone: DMA32
[    0.00] Kernel command line: clk_ignore_unused 
console=ttymxc0,3086,115200 earlycon=ec_imx6q,0x3086,115200 
root=/dev/ram ramdisk_size=1000 rdinit=/hello rootwait rw
[    0.00] 

Re: Jailhouse on LX2160ardb

[Ralf Ramsauer]

On 24/11/2023 09:18, Laurent Corbin wrote:

Hi,


Thanks for your responsiveness. Unfortunately, I have not any output when I
apply your patch. With more traces, I can see that only PSCI for CPU 
suspend is

used. I'm continuing my research to find out more about this kernel panic.
Do you have any new suggestions or hypotheses ?


One further thought, see below.



Thanks in advance,

Le mercredi 22 novembre 2023 à 16:34:10 UTC+1, Ralf Ramsauer a écrit :

Hi,

On 22/11/2023 16:14, Laurent Corbin wrote:
 > Hi all,
 >
 > I'm trying to run jailhouse on LX2160ardb (NXP board). To do that,
 > 1. I wrote a cell file for my root cell.
 > 2. I built a Linux 6.1 and a jailhouse (branch lf-6.1.36_2.1.0
from nxp
 > mirror) with yocto.
 > 3. I loaded the jailhouse driver as a kernel module.
 >
 > But when I try to "enable" jailhouse I get this error :

okay, I think i have seen a similar error before. Would you please try
to apply this patch, and see if there is some output on the console:

diff --git a/hypervisor/arch/arm-common/smccc.c
b/hypervisor/arch/arm-common/smccc.c
index 65639b59..afa26e53 100644
--- a/hypervisor/arch/arm-common/smccc.c
+++ b/hypervisor/arch/arm-common/smccc.c
@@ -83,6 +83,7 @@ static inline long handle_arch_features(u32 id)
return this_cpu_data()->smccc_feat_workaround_2;

default:
+ printk("Unknown SMCCC!\n");
return ARM_SMCCC_NOT_SUPPORTED;
}
}
@@ -129,6 +130,7 @@ enum trap_return handle_smc(struct trap_context
*ctx)
case ARM_SMCCC_OWNER_SIP:
stats[JAILHOUSE_CPU_STAT_VMEXITS_SMCCC]++;
regs[0] = ARM_SMCCC_NOT_SUPPORTED;
+ printk("Unknown SMCCC (SIP)!\n");
break;

case ARM_SMCCC_OWNER_STANDARD:


If you get an output here, then we need to probably forward those
calls.
Please get back to me in that case.

Thanks
Ralf

 >
 >
 >
 > root@lx2160ardb-rev2:~# insmod jailhouse.ko
 > root@lx2160ardb-rev2:~#
 > root@lx2160ardb-rev2:~#
 > root@lx2160ardb-rev2:~#
 > root@lx2160ardb-rev2:~# cat /sys/module/jailhouse/sections/.text
 > 0xc45e1113e000
 > root@lx2160ardb-rev2:~# jailhouse enable lx2160a-rdb.cell
 >
 > Initializing Jailhouse hypervisor v0.12 (369-g6a87b739) on CPU 0
 > Code location: 0xc0200800
 > Page pool usage after early setup: mem 151/993, remap 0/131072
 > Initializing processors:
 >  CPU 0... OK
 >  CPU 1... OK
 >  CPU 5... OK
 >  CPU 4... OK
 >  CPU 2... OK
 >  CPU 3... OK
 >  CPU 9... OK
 >  CPU 8... OK
 >  CPU 15... OK
 >  CPU 13... OK
 >  CPU 12... OK
 >  CPU 7... OK
 >  CPU 10... OK
 >  CPU 11... OK
 >  CPU 6... OK
 >  CPU 14... OK
 > Initializing unit: irqchip
 > Initializing unit: ARM SMMU v3
 > Initializing unit: ARM SMMU
 > ARM MMU500 at 0x500 with:
 >  stream matching with 128 SMR groups
 >  64 context banks (0 stage-2 only)
 > Initializing unit: PVU IOMMU
 > Initializing unit: PCI
 > Adding virtual PCI device 00:00.0 to cell "lx2160a"
 > Adding virtual PCI device 00:01.0 to cell "lx2160a"
 > Page pool usage after late setup: mem 228/993, remap 2576/131072
 > Activating hypervisor
 > [  173.577969] SError Interrupt on CPU0, code 0xbf00
-- SError


SError is a "System Error". See [1] what triggers a SError. I wonder why 
this happens behind address 0xbf00, right after enabling Jailhouse 
(which works correctly). What is behind that physical address? Is this 
the Jailhouse physical location?


Do you have all Jailhouse patches applied on your kernel? Are memory 
regions reserved (via mem= option or device tree)?


  Ralf

[1] 
https://developer.arm.com/documentation/102412/0103/Exception-types/Asynchronous-exceptions



 > [  173.577985] CPU: 0 PID: 0 Comm: swapper/0 Tainted: G           O
 >   6.1.22+g66e442bc7fdc #1
 > [  173.577993] Hardware name: NXP Layerscape LX2160ARDB (DT)
 > [  173.577996] pstate: 20c5 (nzCv daIF -PAN -UAO -TCO -DIT -SSBS
 > BTYPE=--)
 > [  173.578002] pc : enter_hypervisor+0x44/0xa4 [jailhouse]
 > [  173.578025] lr : enter_hypervisor+0x44/0xa4 [jailhouse]
 > [  173.578040] sp : 88003f10
 > [  173.578042] x29: 88003f10 x28: c45e51736c40 x27:
 > fbd0ea70
 > [  173.578052] x26: 0001 x25: 0028652e0c1e x24:
 > 0028652b7617
 > [  173.578060] x23:  x22:  x21:
 > 
 > [  173.578067] x20:  x19: c45e11144000 x18:
 > 
 > [  1

Re: jailhouse enable linux

[Ralf Ramsauer]

Hi,

On 23/11/2023 05:25, xin zhang wrote:

In none root cell, start command is :
jailhouse cell linux -i /home/file/rootfs.cpio -d 
configs/arm64/dts/AMA0.dtb configs/arm64/linux.cell /home/file/Image -c 
"console=ttyAMA0,115200 root=/dev/ram0  ramdisk_size=0x100 
earlycon=pl011,0x2800"


Okay. Are you absolutely sure, that the device tree matches with the 
cell configuration? I wonder why the kernel tries to access virtual (!) 
address 0x1388, which apparently has no mapping. In case everything is 
configured correctly, then we have here some issue with the kernel, and 
you should debug in this direction.


  Ralf



在2023年11月22日星期三 UTC+8 21:15:15 写道：



On 22/11/2023 07:25, xin zhang wrote:
 > when I run linux image on jailhouse, guset serial port print follow
 > errors: where I can get ways to sovle it？

Please provide more context. Does this happen in the root or
non-root cell?

Ralf

 >
 >
 > [    0.069576][ 0] [    T0] Mount-cache hash table entries: 1024
(order:
 > 1, 8192 bytes, linear)
 > [    0.078110][ 0] [    T0] Mountpoint-cache hash table entries:
1024
 > (order: 1, 8192 bytes, linear)
 > [    0.087451][ 0] [    T2] Unable to handle kernel paging
request at
 > virtual address 1388
 > [    0.096606][ 0] [    T2] Mem abort info:
 > [    0.100564][ 0] [    T2]   ESR = 0x9605
 > [    0.104789][ 0] [    T2]   EC = 0x25: DABT (current EL), IL =
32 bits
 > [    0.111299][ 0] [    T2]   SET = 0, FnV = 0
 > [    0.115522][ 0] [    T2]   EA = 0, S1PTW = 0
 > [    0.119834][ 0] [    T2] Data abort info:
 > [    0.123883][ 0] [    T2]   ISV = 0, ISS = 0x0005
 > [    0.128898][ 0] [    T2]   CM = 0, WnR = 0
 > [    0.133034][ 0] [    T2] [1388] user address but
 > active_mm is swapper
 > [    0.140600][ 0] [    T2] Internal error: Oops: 9605 [#1] SMP
 > [    0.146667][ 0] [    T2] Modules linked in:
 > [    0.150892][ 0] [    T2] CPU: 0 PID: 2 Comm: kthreadd Not tainted
 > 5.4.18-101+ #2
 > [    0.158367][ 0] [    T2] Source Version:
 > e7f24793ad667e777f2cf6dab620d082517f6075
 > [    0.165929][ 0] [    T2] Hardware name: FT-2000/4-D4-DSK
Development
 > Board (DT)
 > [    0.173317][ 0] [    T2] pstate: 8005 (Nzcv daif -PAN -UAO)
 > [    0.179302][ 0] [    T2] pc : __alloc_pages_nodemask+0xf8/0x308
 > [    0.185283][ 0] [    T2] lr : __alloc_pages_nodemask+0xe0/0x308
 > [    0.191263][ 0] [    T2] sp : ff8031867c30
 > [    0.195749][ 0] [    T2] x29: ff8031867c30 x28:
00800700
 > [    0.202258][ 0] [    T2] x27: ff8031823a80 x26:

 > [    0.208766][ 0] [    T2] x25: 00400dc0 x24:
0001
 > [    0.215273][ 0] [    T2] x23:  x22:
ffc0113fa000
 > [    0.221781][ 0] [    T2] x21: 0002 x20:
ffc0113f9000
 > [    0.228289][ 0] [    T2] x19: 00400dc0 x18:
0014
 > [    0.234797][ 0] [    T2] x17: 9242c113 x16:
b8b3e41b
 > [    0.241305][ 0] [    T2] x15: b31b39bd x14:

 > [    0.247817][ 0] [    T2] x13: 0001 x12:
0001
 > [    0.254325][ 0] [    T2] x11: 02e03c00 x10:
05c05800
 > [    0.260833][ 0] [    T2] x9 : 0191 x8 :
0004
 > [    0.267340][ 0] [    T2] x7 : ff8032fe0378 x6 :

 > [    0.273848][ 0] [    T2] x5 : 0003 x4 :

 > [    0.280355][ 0] [    T2] x3 : 1380 x2 :

 > [    0.286863][ 0] [    T2] x1 : 0001 x0 :

 > [    0.293370][ 0] [    T2] Call trace:
 > [    0.296978][ 0] [    T2]  __alloc_pages_nodemask+0xf8/0x308
 > [    0.302609][ 0] [    T2]  copy_process+0x1a4/0x1340
 > [    0.307536][ 0] [    T2]  _do_fork+0x80/0x370
 > [    0.311934][ 0] [    T2]  kernel_thread+0x6c/0x90
 > [    0.316686][ 0] [    T2]  kthreadd+0x1e0/0x270
 > [    0.321173][ 0] [    T2]  ret_from_fork+0x10/0x18
 > [    0.325924][ 0] [    T2] Code: 3901c3a0 a9450ba3 b9406fa1
b5000b62
 > (b9400860)
 > [    0.333224][ 0] [    T2] ---[ end trace 7e9dcf3f049b9227 ]---
 >
 > --
 > You received this message because you are subscribed to the Google
 > Groups "Jailhouse" group.
 > To unsubscribe from this group and stop receiving emails from it,
send
 > an email to jailhouse-de...@googlegroups.com
 > .
 > To view this discussion on the web visit
 >

https://groups.google.com/d/msgid/jailhouse-dev/a3ea29e3-da4c-4352-a328-9a04c86b8b1en%40googlegroups.com 

 

Re: arm cortex A55 support？

[Ralf Ramsauer]

On 23/11/2023 04:33, '张平' via Jailhouse wrote:

 >On 22/11/2023 06:53, '张平' via Jailhouse wrote:
 >> I am trying to transplant jailhouse on the A55 core, but now I have
 >> encountered some problems. After I execute the following command, the
 >> system will freeze.
 >> $ jailhouse enable renesas-r9a07g044l2.cell
 >
 >No error log? Is the debug console configured correctly?

I think there is no problem with the debug console configuration I set up.
Please refer to:
.debug_console = {
      .address = 0x1004b800,
      .size = 0x400,
      .type = JAILHOUSE_CON_TYPE_SCIFA,
      .flags = JAILHOUSE_CON_ACCESS_MMIO |
              JAILHOUSE_CON_REGDIST_4,
},

 >
 >>
 >> What are the possible consequences?
 >>
 >> I made sure I did the following:
 >> 1.Linux kernel version 5.10
 >> 2.Linux starts in EL2
 >> 3. Turn on KVM
 >
 >Do not use KVM in combination with Jailhouse. Disable KVM in your kernel.
 >
I turned off KVM as you said but the problem still exists.
root@smarc-rzg2l:~# dmesg | grep -r kvm
root@smarc-rzg2l:~# insmod jailhouse.ko
[123.760797] jailhouse: loading out-of-tree module taints kernel.
root@smarc-rzg2l:~# jailhouse enable renesas-r9a07g044l2.cell


 >Ralf
 >
 >> 4. Turn off kernel CONFIG_ARM64_VHE

I would also like to add:
I refer to this post and added mov and ret modifications to entry.S to 
output jailhouse is opening.
However, when using the command jailhouse cell list later, I also 
encountered the problem of the serial port being stuck.
Reference post: 
https://groups.google.com/g/jailhouse-dev/c/zwY9bpxq8mg/m/VF44nEC2AwAJ

log:
root@smarc-rzg2l:~# insmod jailhouse.ko
[79.631308] jailhouse: loading out-of-tree module taints kernel.
root@smarc-rzg2l:~# jailhouse enable renesas-r9a07g044l2.cell
[85.716867] pci-host-generic e000.pci: host bridge /pci@0 ranges:
[85.723692] pci-host-generic e000.pci: MEM 
0x00e010..0x00e0103fff -> 0x00e010
[85.733615] pci-host-generic e000.pci: ECAM at [mem 
0xe000-0xe00f] for [bus 00]

[85.742977] pci-host-generic e000.pci: PCI host bridge to bus 0001:00
[85.749828] pci_bus 0001:00: root bus resource [bus 00]
[85.755130] pci_bus 0001:00: root bus resource [mem 0xe010-0xe0103fff]
[85.763626] pci 0001:00:1f.0: [0ff0:c000] type 0f class 0xcf
[85.769669] pci 0001:00:1f.0: unknown header type 0f, ignoring device
[85.776475] The Jailhouse is opening.
root@smarc-rzg2l:~# jailhouse cell list
ID Name

So is this way of modifying entry.S the correct way?


I can not comment on that, I'm not into that issue. If it works for you 
for the moment to do further evaluation - great! You can of course 
contact the original authors of the workaround.


Thanks,
  Ralf



--
You received this message because you are subscribed to the Google 
Groups "Jailhouse" group.
To unsubscribe from this group and stop receiving emails from it, send 
an email to jailhouse-dev+unsubscr...@googlegroups.com 
.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jailhouse-dev/2be7b167-688f-4b75-971f-57242f8b443fn%40googlegroups.com .


--
You received this message because you are subscribed to the Google Groups 
"Jailhouse" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jailhouse-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jailhouse-dev/42e9e931-b628-40a5-8375-cc2a495eff41%40oth-regensburg.de.

Re: Jailhouse on LX2160ardb

[Ralf Ramsauer]

Hi,

On 22/11/2023 16:14, Laurent Corbin wrote:

Hi all,

I'm trying to run jailhouse on LX2160ardb (NXP board). To do that,
1. I wrote a cell file for my root cell.
2. I built a Linux 6.1 and a jailhouse (branch lf-6.1.36_2.1.0 from nxp 
mirror) with yocto.

3. I loaded the jailhouse driver as a kernel module.

But when I try to "enable" jailhouse I get this error :


okay, I think i have seen a similar error before. Would you please try 
to apply this patch, and see if there is some output on the console:


diff --git a/hypervisor/arch/arm-common/smccc.c 
b/hypervisor/arch/arm-common/smccc.c

index 65639b59..afa26e53 100644
--- a/hypervisor/arch/arm-common/smccc.c
+++ b/hypervisor/arch/arm-common/smccc.c
@@ -83,6 +83,7 @@ static inline long handle_arch_features(u32 id)
return this_cpu_data()->smccc_feat_workaround_2;

default:
+   printk("Unknown SMCCC!\n");
return ARM_SMCCC_NOT_SUPPORTED;
}
 }
@@ -129,6 +130,7 @@ enum trap_return handle_smc(struct trap_context *ctx)
case ARM_SMCCC_OWNER_SIP:
stats[JAILHOUSE_CPU_STAT_VMEXITS_SMCCC]++;
regs[0] = ARM_SMCCC_NOT_SUPPORTED;
+   printk("Unknown SMCCC (SIP)!\n");
break;

case ARM_SMCCC_OWNER_STANDARD:


If you get an output here, then we need to probably forward those calls. 
Please get back to me in that case.


Thanks
  Ralf





root@lx2160ardb-rev2:~# insmod jailhouse.ko
root@lx2160ardb-rev2:~#
root@lx2160ardb-rev2:~#
root@lx2160ardb-rev2:~#
root@lx2160ardb-rev2:~# cat /sys/module/jailhouse/sections/.text
0xc45e1113e000
root@lx2160ardb-rev2:~# jailhouse enable lx2160a-rdb.cell

Initializing Jailhouse hypervisor v0.12 (369-g6a87b739) on CPU 0
Code location: 0xc0200800
Page pool usage after early setup: mem 151/993, remap 0/131072
Initializing processors:
  CPU 0... OK
  CPU 1... OK
  CPU 5... OK
  CPU 4... OK
  CPU 2... OK
  CPU 3... OK
  CPU 9... OK
  CPU 8... OK
  CPU 15... OK
  CPU 13... OK
  CPU 12... OK
  CPU 7... OK
  CPU 10... OK
  CPU 11... OK
  CPU 6... OK
  CPU 14... OK
Initializing unit: irqchip
Initializing unit: ARM SMMU v3
Initializing unit: ARM SMMU
ARM MMU500 at 0x500 with:
  stream matching with 128 SMR groups
  64 context banks (0 stage-2 only)
Initializing unit: PVU IOMMU
Initializing unit: PCI
Adding virtual PCI device 00:00.0 to cell "lx2160a"
Adding virtual PCI device 00:01.0 to cell "lx2160a"
Page pool usage after late setup: mem 228/993, remap 2576/131072
Activating hypervisor
[  173.577969] SError Interrupt on CPU0, code 0xbf00 -- SError
[  173.577985] CPU: 0 PID: 0 Comm: swapper/0 Tainted: G           O 
   6.1.22+g66e442bc7fdc #1

[  173.577993] Hardware name: NXP Layerscape LX2160ARDB (DT)
[  173.577996] pstate: 20c5 (nzCv daIF -PAN -UAO -TCO -DIT -SSBS 
BTYPE=--)

[  173.578002] pc : enter_hypervisor+0x44/0xa4 [jailhouse]
[  173.578025] lr : enter_hypervisor+0x44/0xa4 [jailhouse]
[  173.578040] sp : 88003f10
[  173.578042] x29: 88003f10 x28: c45e51736c40 x27: 
fbd0ea70
[  173.578052] x26: 0001 x25: 0028652e0c1e x24: 
0028652b7617
[  173.578060] x23:  x22:  x21: 

[  173.578067] x20:  x19: c45e11144000 x18: 

[  173.578074] x17:  x16:  x15: 

[  173.578080] x14:  x13:  x12: 

[  173.578086] x11:  x10:  x9 : 

[  173.578092] x8 :  x7 :  x6 : 

[  173.578098] x5 :  x4 :  x3 : 

[  173.578104] x2 :  x1 :  x0 : 


[  173.578111] Kernel panic - not syncing: Asynchronous SError Interrupt
[  173.578115] CPU: 0 PID: 0 Comm: swapper/0 Tainted: G           O 
   6.1.22+g66e442bc7fdc #1

[  173.578120] Hardware name: NXP Layerscape LX2160ARDB (DT)
[  173.578122] Call trace:
[  173.578124]  dump_backtrace.part.0+0xe0/0xf0
[  173.578140]  show_stack+0x18/0x30
[  173.578148]  dump_stack_lvl+0x64/0x80
[  173.578158]  dump_stack+0x18/0x34
[  173.578166]  panic+0x188/0x340
[  173.578173]  nmi_panic+0xac/0xb0
[  173.578179]  arm64_serror_panic+0x6c/0x7c
[  173.578184]  do_serror+0x58/0x5c
[  173.578188]  el1h_64_error_handler+0x30/0x4c
[  173.578193]  el1h_64_error+0x64/0x68
[  173.578198]  enter_hypervisor+0x44/0xa4 [jailhouse]
[  173.578213]  __flush_smp_call_function_queue+0xd0/0x250
[  173.578223]  generic_smp_call_function_single_interrupt+0x14/0x20
[  173.578232]  ipi_handler+0x98/0x160
[  173.578239]  handle_percpu_devid_irq+0x84/0x130
[  173.578245]  generic_handle_domain_irq+0x2c/0x4c
[  173.578251]  gic_handle_irq+0x50/0x130
[  173.578258]  call_on_irq_stack+0x2c/0x5c
[  173.578263]  do_interrupt_handler+0x80/0x84
[  173.578270]  

Re: jailhouse enable linux

[Ralf Ramsauer]

On 22/11/2023 07:25, xin zhang wrote:
when I run linux image on jailhouse, guset serial port print follow 
errors: where I can get ways to sovle it？


Please provide more context. Does this happen in the root or non-root cell?

  Ralf




[    0.069576][ 0] [    T0] Mount-cache hash table entries: 1024 (order: 
1, 8192 bytes, linear)
[    0.078110][ 0] [    T0] Mountpoint-cache hash table entries: 1024 
(order: 1, 8192 bytes, linear)
[    0.087451][ 0] [    T2] Unable to handle kernel paging request at 
virtual address 1388

[    0.096606][ 0] [    T2] Mem abort info:
[    0.100564][ 0] [    T2]   ESR = 0x9605
[    0.104789][ 0] [    T2]   EC = 0x25: DABT (current EL), IL = 32 bits
[    0.111299][ 0] [    T2]   SET = 0, FnV = 0
[    0.115522][ 0] [    T2]   EA = 0, S1PTW = 0
[    0.119834][ 0] [    T2] Data abort info:
[    0.123883][ 0] [    T2]   ISV = 0, ISS = 0x0005
[    0.128898][ 0] [    T2]   CM = 0, WnR = 0
[    0.133034][ 0] [    T2] [1388] user address but 
active_mm is swapper

[    0.140600][ 0] [    T2] Internal error: Oops: 9605 [#1] SMP
[    0.146667][ 0] [    T2] Modules linked in:
[    0.150892][ 0] [    T2] CPU: 0 PID: 2 Comm: kthreadd Not tainted 
5.4.18-101+ #2
[    0.158367][ 0] [    T2] Source Version: 
e7f24793ad667e777f2cf6dab620d082517f6075
[    0.165929][ 0] [    T2] Hardware name: FT-2000/4-D4-DSK Development 
Board (DT)

[    0.173317][ 0] [    T2] pstate: 8005 (Nzcv daif -PAN -UAO)
[    0.179302][ 0] [    T2] pc : __alloc_pages_nodemask+0xf8/0x308
[    0.185283][ 0] [    T2] lr : __alloc_pages_nodemask+0xe0/0x308
[    0.191263][ 0] [    T2] sp : ff8031867c30
[    0.195749][ 0] [    T2] x29: ff8031867c30 x28: 00800700
[    0.202258][ 0] [    T2] x27: ff8031823a80 x26: 
[    0.208766][ 0] [    T2] x25: 00400dc0 x24: 0001
[    0.215273][ 0] [    T2] x23:  x22: ffc0113fa000
[    0.221781][ 0] [    T2] x21: 0002 x20: ffc0113f9000
[    0.228289][ 0] [    T2] x19: 00400dc0 x18: 0014
[    0.234797][ 0] [    T2] x17: 9242c113 x16: b8b3e41b
[    0.241305][ 0] [    T2] x15: b31b39bd x14: 
[    0.247817][ 0] [    T2] x13: 0001 x12: 0001
[    0.254325][ 0] [    T2] x11: 02e03c00 x10: 05c05800
[    0.260833][ 0] [    T2] x9 : 0191 x8 : 0004
[    0.267340][ 0] [    T2] x7 : ff8032fe0378 x6 : 
[    0.273848][ 0] [    T2] x5 : 0003 x4 : 
[    0.280355][ 0] [    T2] x3 : 1380 x2 : 
[    0.286863][ 0] [    T2] x1 : 0001 x0 : 
[    0.293370][ 0] [    T2] Call trace:
[    0.296978][ 0] [    T2]  __alloc_pages_nodemask+0xf8/0x308
[    0.302609][ 0] [    T2]  copy_process+0x1a4/0x1340
[    0.307536][ 0] [    T2]  _do_fork+0x80/0x370
[    0.311934][ 0] [    T2]  kernel_thread+0x6c/0x90
[    0.316686][ 0] [    T2]  kthreadd+0x1e0/0x270
[    0.321173][ 0] [    T2]  ret_from_fork+0x10/0x18
[    0.325924][ 0] [    T2] Code: 3901c3a0 a9450ba3 b9406fa1 b5000b62 
(b9400860)

[    0.333224][ 0] [    T2] ---[ end trace 7e9dcf3f049b9227 ]---

--
You received this message because you are subscribed to the Google 
Groups "Jailhouse" group.
To unsubscribe from this group and stop receiving emails from it, send 
an email to jailhouse-dev+unsubscr...@googlegroups.com 
.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jailhouse-dev/a3ea29e3-da4c-4352-a328-9a04c86b8b1en%40googlegroups.com .


--
You received this message because you are subscribed to the Google Groups 
"Jailhouse" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jailhouse-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jailhouse-dev/1024f0c9-f851-46e2-abcb-3f3b40f5c47c%40oth-regensburg.de.

Re: arm cortex A55 support？

[Ralf Ramsauer]

On 22/11/2023 06:53, '张平' via Jailhouse wrote:
I am trying to transplant jailhouse on the A55 core, but now I have 
encountered some problems. After I execute the following command, the 
system will freeze.

$ jailhouse enable renesas-r9a07g044l2.cell


No error log? Is the debug console configured correctly?



What are the possible consequences?

I made sure I did the following:
1.Linux kernel version 5.10
2.Linux starts in EL2
3. Turn on KVM


Do not use KVM in combination with Jailhouse. Disable KVM in your kernel.

  Ralf


4. Turn off kernel CONFIG_ARM64_VHE

Best regards
zhang,

在2023年11月22日星期三 UTC+8 11:04:24 写道：

There is no difference to run jailhouse on A55 cores.

'张平' via Jailhouse  于2023年11月20日周
一 13:47写道：


Hi All

I would like to know, are there any examples that are adapted to
arm cortex A55?
Or are there any SOCs that are arm cortex A55 and have
successfully transplanted the jailhouse program?

I saw that A57 and A53 products have suppressed jailhouse
applications, but I didn’t see A55?

Cheers,
Zhang

-- 
You received this message because you are subscribed to the

Google Groups "Jailhouse" group.
To unsubscribe from this group and stop receiving emails from
it, send an email to jailhouse-de...@googlegroups.com.
To view this discussion on the web visit

https://groups.google.com/d/msgid/jailhouse-dev/ab8484e8-cb54-4c1b-ac78-ef67bb1bf4b3n%40googlegroups.com
 
.



-- 


--
You received this message because you are subscribed to the Google 
Groups "Jailhouse" group.
To unsubscribe from this group and stop receiving emails from it, send 
an email to jailhouse-dev+unsubscr...@googlegroups.com 
.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jailhouse-dev/f23d0d17-ca3c-4807-a7b6-0e2154ccbe75n%40googlegroups.com .


--
You received this message because you are subscribed to the Google Groups 
"Jailhouse" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jailhouse-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jailhouse-dev/184a93de-9ba3-4ad2-9771-f114000eeaf5%40oth-regensburg.de.

Re: Jailhouse enable pci error

[Ralf Ramsauer]

Hi Paresh,

On 21/11/2023 05:30, Paresh Bhagat wrote:

Update. Got the jailhouse enabled but still getting the pci error.

nitializing Jailhouse hypervisor v0.12 (324-ge57d1eff-dirty) on CPU 3
Code location: 0xc0200800
Page pool usage after early setup: mem 39/993, remap 0/131072
Initializing processors:
  CPU 3... OK
  CPU 2... OK
  CPU 0... OK
  CPU 1... OK
Initializing unit: irqchip
Initializing unit: ARM SMMU v3
Initializing unit: ARM SMMU
Initializing unit: PVU IOMMU
Initializing unit: PCI
Adding virtual PCI device 00:00.0 to cell "k3-am62p5-sk"
Adding virtual PCI device 00:01.0 to cell "k3-am62p5-sk"
Page pool usage after late setup: mem 79/993, remap 144/131072
Activating hypervisor
[  332.417442] pci-host-generic 7600.pci: host bridge /pci@0 ranges:
[  332.417476] pci-host-generic 7600.pci:      MEM 
0x007610..0x0076103fff -> 0x007610
[  332.417572] pci-host-generic 7600.pci: ECAM at [mem 
0x7600-0x760f] for [bus 00]

[  332.417732] pci-host-generic 7600.pci: PCI host bridge to bus 0001:00
[  332.417741] pci_bus 0001:00: root bus resource [bus 00]
[  332.417748] pci_bus 0001:00: root bus resource [mem 
0x7610-0x76103fff]

[  332.417793] pci 0001:00:00.0: [110a:4106] type 00 class 0xff


According to your config, 1:0:0.0 ist the first ivshmem device, 1:0:1.0 
is the second one, right?


Could you please try to enable one ivshmem device, so please disable the 
second one for testing. Let's try to get one running, then we can go 
further.


And for the first ivshmem device, would you consider (just for testing), 
to only connect two peers? And, in your config, plase use the 
JAILHOUSE_SHMEM_… macros for defining the ivshmem memregions.


Thanks,
  Ralf


[  332.417819] pci 0001:00:00.0: reg 0x10: [mem 0x-0x]
[  332.418103] pci 0001:00:01.0: [110a:4106] type 00 class 0xff0001
[  332.418127] pci 0001:00:01.0: reg 0x10: [mem 0x-0x]
[  332.420175] pci 0001:00:00.0: BAR 0: no space for [mem size 0x0001]
[  332.420195] pci 0001:00:00.0: BAR 0: failed to assign [mem size 
0x0001]

[  332.420203] pci 0001:00:01.0: BAR 0: no space for [mem size 0x0001]
[  332.420209] pci 0001:00:01.0: BAR 0: failed to assign [mem size 
0x0001]

[  332.420916] The Jailhouse is opening.
[  332.531915] psci: CPU1 killed (polled 0 ms)
[  332.579879] psci: CPU2 killed (polled 0 ms)
[  332.631860] psci: CPU3 killed (polled 0 ms)


On Tuesday, November 21, 2023 at 12:50:49 AM UTC+5:30 Paresh Bhagat wrote:

Any fix for the following error while trying to enable jailhouse ?
something wrong with the memory allocation?

Initializing Jailhouse hypervisor v0.12 (324-ge57d1eff-dirty) on CPU 0
Code location: 0xc0200800
Page pool usage after early setup: mem 39/993, remap 0/131072
Initializing processors:
  CPU 0... OK
  CPU 2... OK
  CPU 1... OK
  CPU 3... OK
Initializing unit: irqchip
Initializing unit: ARM SMMU v3
Initializing unit: ARM SMMU
Initializing unit: PVU IOMMU
Initializing unit: PCI
Adding virtual PCI device 00:00.0 to cell "k3-am62p5-sk"
Adding virtual PCI device 00:01.0 to cell "k3-am62p5-sk"
Page pool usage after late setup: mem 78/993, remap 144/131072
Activating hypervisor
[  122.542841] pci-host-generic 7600.pci: host bridge /pci@0 ranges:
[  122.542873] pci-host-generic 7600.pci:      MEM
0x007610..0x0076103fff -> 0x007610
[  122.542947] pci-host-generic 7600.pci: ECAM at [mem
0x7600-0x760f] for [bus 00]
[  122.543103] pci-host-generic 7600.pci: PCI host bridge to bus
0001:00
[  122.543112] pci_bus 0001:00: root bus resource [bus 00]
[  122.543119] pci_bus 0001:00: root bus resource [mem
0x7610-0x76103fff]
root@am62pxx-evm:/usr/share/jailhouse# [  122.543161] pci
0001:00:00.0: [110a:4106] type 00 class 0xff
[  122.543187] pci 0001:00:00.0: reg 0x10: [mem 0x-0x]
[  122.543455] pci 0001:00:01.0: [110a:4106] type 00 class 0xff0001
[  122.543481] pci 0001:00:01.0: reg 0x10: [mem 0x-0x]
[  122.545437] pci 0001:00:00.0: BAR 0: no space for [mem size
0x0001]
[  122.545458] pci 0001:00:00.0: BAR 0: failed to assign [mem size
0x0001]
[  122.545466] pci 0001:00:01.0: BAR 0: no space for [mem size
0x0001]
[  122.545471] pci 0001:00:01.0: BAR 0: failed to assign [mem size
0x0001]
[  122.545896] The Jailhouse is opening.
Unhandled data read at 0xb00328(4)
FATAL: unhandled trap (exception class 0x24)

i have attached root cell config and output of cat /proc/iomem for
referrence.

Thanks

--
You received this message because you are subscribed to the Google 
Groups "Jailhouse" group.
To unsubscribe from this group and stop receiving emails from it, send 
an email to jailhouse-dev+unsubscr...@googlegroups.com 

Re: Development status

[Ralf Ramsauer]

Hi Chris,

On 20/11/2023 14:37, Chris Paterson wrote:

Hello Jailhouse maintainers,

I was just looking at Jailhouse GitHub repo [0] and it looks like the master 
branch hasn't been updated for a while (10 months).
Is this project still active?


Yes, we're currently still working on the RISC-V Port of Jailhouse, and 
we're almost ready for integration. Therefore, we also have a bunch of 
auxiliary patches in the queue. They'll soon land on the list.




On a side note, are there any plans to merge the latest patches from the next 
branch [1] into master?


@Jan: Ping :)

Thanks,
  Ralf



[0] https://github.com/siemens/jailhouse
[1] https://github.com/siemens/jailhouse/compare/master...next

Kind regards, Chris



--
You received this message because you are subscribed to the Google Groups 
"Jailhouse" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jailhouse-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jailhouse-dev/df468a43-ecdf-46a4-bf02-31681d3ce799%40oth-regensburg.de.

Re: jailhouse enable non-root linux

[Ralf Ramsauer]

On 14/11/2023 09:40, xin zhang wrote:
What is the difference between the kernel image used by Jailhouse to 
start non root Linux and a regular image? Do you need to change the 
configuration? Can the directly compiled kernel image be used? Which 


As always: Depends, but generally yes.

kernel image is used? Image or vmlinuz or something else? Can you give 
an example?


Depends on the architecture. On ARM/ARM64, use non-compressed variants. 
If configured correctly, you can use portable kernels for both, root and 
non-root cells.


Thanks,
  Ralf



--
You received this message because you are subscribed to the Google 
Groups "Jailhouse" group.
To unsubscribe from this group and stop receiving emails from it, send 
an email to jailhouse-dev+unsubscr...@googlegroups.com 
<mailto:jailhouse-dev+unsubscr...@googlegroups.com>.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jailhouse-dev/89d7ee90-8781-4db0-ad76-b045f1844eb2n%40googlegroups.com <https://groups.google.com/d/msgid/jailhouse-dev/89d7ee90-8781-4db0-ad76-b045f1844eb2n%40googlegroups.com?utm_medium=email_source=footer>.


--
Ralf Ramsauer
PGP: 0xC85252CC

--
You received this message because you are subscribed to the Google Groups 
"Jailhouse" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jailhouse-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jailhouse-dev/5eb6becc-4801-4b30-aa7c-7a1826e50aca%40vmexit.de.

Re: jailhouse enable linux kernel error

[Ralf Ramsauer]

On 14/11/2023 07:31, xin zhang wrote:


What is the reason for the following error printed when using Jailhouse 
to start Linux in Linux? Replacing the image is also not feasible

  the error log as follows：

[    0.092570][  0] Unable to handle kernel paging request at virtual 
address 4008


Two things: Did you reserve the memory area for jailhouse via 
device-tree/cmdline? Second, did you apply jailhouse patches to the 
kernel? Please apply those patches:

https://git.kiszka.org/?p=linux.git;a=shortlog;h=refs/heads/queues/jailhouse-5.14

Thanks,
  Ralf


[    0.101012][  0] Mem abort info:
[    0.104266][  0]   ESR = 0x9605
[    0.107785][  0]   Exception class = DABT (current EL), IL = 32 bits
[    0.114204][  0]   SET = 0, FnV = 0
[    0.117723][  0]   EA = 0, S1PTW = 0
[    0.121328][  0] Data abort info:
[    0.124671][  0]   ISV = 0, ISS = 0x0005
[    0.128980][  0]   CM = 0, WnR = 0
[    0.132410][  0] [4008] user address but active_mm is swapper
[    0.139270][  0] Internal error: Oops: 9605 [#1] SMP
[    0.144636][  0] Modules linked in:
[    0.148156][  0] Process kthreadd (pid: 2, stack limit = 
0x(ptrval))

[    0.155281][  0] CPU: 0 PID: 2 Comm: kthreadd Not tainted 4.19.90-69+ #3
[    0.162052][  0] Source Version: a22085976f26303c72a6bc7cc0d4ee20ff890cf9
[    0.168912][  0] Hardware name: FT-2000/4-D4-DSK Development Board (DT)
[    0.175596][  0] pstate: 6085 (nZCv daIf -PAN -UAO)
[    0.180876][  0] pc : get_partial_node.isra.24+0x38/0x298
[    0.186330][  0] lr : ___slab_alloc+0x3b8/0x528
[    0.190903][  0] sp : 8a143b10
[    0.194685][  0] x29: 8a143b10 x28: 4000
[    0.200490][  0] x27: 8a0fbc00 x26: 800023e55de0
[    0.206294][  0] x25:  x24: 0035
[    0.212099][  0] x23: 080edc34 x22: 09539b08
[    0.217903][  0] x21:  x20: 0001
[    0.223707][  0] x19: 007080c0 x18: 0001
[    0.229511][  0] x17:  x16: 
[    0.235315][  0] x15: 89b4cbb7 x14: 0003
[    0.241120][  0] x13: 09b4cbca x12: 
[    0.246924][  0] x11: 0990e2a8 x10: 7fe28520
[    0.252728][  0] x9 : 8a143d90 x8 : 8a126e20
[    0.258532][  0] x7 :  x6 : 80080008
[    0.264337][  0] x5 :  x4 : 7fe28520
[    0.270141][  0] x3 : 007080c0 x2 : 800023e55df0
[    0.275945][  0] x1 : 4000 x0 : 08315028
[    0.281749][  0] Call trace:
[    0.284654][  0]  get_partial_node.isra.24+0x38/0x298
[    0.289755][  0]  ___slab_alloc+0x3b8/0x528
[    0.293978][  0]  __slab_alloc+0x5c/0x78
[    0.297937][  0]  kmem_cache_alloc_node+0xd8/0x220
[    0.302774][  0]  copy_process.isra.8+0x1dc/0x1308
[    0.307611][  0]  _do_fork+0x78/0x370
[    0.311306][  0]  kernel_thread+0x48/0x58
[    0.315352][  0]  kthreadd+0x1c8/0x268
[    0.319135][  0]  ret_from_fork+0x10/0x18
[    0.323182][  0] Code: b90067a3 f90037a2 d503201f b400117c (f9400780)
[    0.329784][  0] ---[ end trace df01b2bd811dec5e ]---
[    0.334885][  0] Kernel panic - not syncing: Fatal exception
[    0.340604][  0] Rebooting in 30 seconds..
[   30.346222][  0] Reboot failed -- System halted



thanks.

--
You received this message because you are subscribed to the Google 
Groups "Jailhouse" group.
To unsubscribe from this group and stop receiving emails from it, send 
an email to jailhouse-dev+unsubscr...@googlegroups.com 
.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jailhouse-dev/8b7c90c6-73bb-4e57-b705-ec1f1b692ee4n%40googlegroups.com .


--
You received this message because you are subscribed to the Google Groups 
"Jailhouse" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jailhouse-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jailhouse-dev/adc02e82-933c-463f-9372-56f43945a2de%40oth-regensburg.de.

Re: How to display xil_printf or printf in jailhouse?

[Ralf Ramsauer]

Hi Tony,

On 06/11/2023 12:06, Tony wrote:

Hello everyone.
I am trying to load a binary as a non-root cell of Jailhouse and run it 
on the Ultrascale ZCU104.  Within this binary there are serial prints 
executed via xil_printf that I do not see in output when I run the 
binary on jailhouse. I think because jailhouse does not support the 
libraries required by xil_printf, such as the printf libraries. In fact, 
the default demos (gic-demo and uart-demo) print via printk. How can I 
solve so that I can be able to see something output on the serial from 
my binary, without necessarily including the printk's within my binary? 
Thank you all in advance.


I guess you're using vitis? There's no compatibility with vitis.

  Ralf



--
You received this message because you are subscribed to the Google 
Groups "Jailhouse" group.
To unsubscribe from this group and stop receiving emails from it, send 
an email to jailhouse-dev+unsubscr...@googlegroups.com 
.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jailhouse-dev/187937f5-9346-485b-b2f1-8a7d78ac2768n%40googlegroups.com .


--
You received this message because you are subscribed to the Google Groups 
"Jailhouse" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jailhouse-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jailhouse-dev/493f2502-4a97-4421-86c3-ce366b00496b%40oth-regensburg.de.

Re: CPU shows offline - Jailhouse Linux inmate

[Ralf Ramsauer]

Hi Bharathiraja,

On 20/10/2023 15:50, Bharathiraja Nallathambi wrote:

Hi Ralf,

Thanks for providing the suggestion, with the boot log, we found the 
below message,


/[    0.045828] psci: no cpu_on method, not booting CPU1/


Hmm.



We went on updating the inmate Linux dts with the following node,

/psci {
compatible = "arm,psci-1.0";
method = "smc";
};/


The device trees are in jailhouse upstream, right? Actually,
'enable-method = "psci";' in the cpu section of the dt was sufficient so 
far. Maybe this changed to cpu-on? It might be that your change is 
required for more recent kernel versions.


If so, patches welcome! :-)

Thanks,
  Ralf




Now we are able to see both the CPUs enabled and working as expected

/[    0.003483] CPU1: Booted secondary processor 0x03 [0x410fd083]/

Thanks for the support.

Thanks and Regards,
Bharathiraja Nallathambi

On Wednesday, 11 October 2023 at 01:26:19 UTC+5:30 Ralf Ramsauer wrote:



On 10/10/2023 17:17, Bharathiraja Nallathambi wrote:
 > Hi,
 >
 > I was trying jailhouse with a linux inmate on an imx8mp-evk board. I
 > have created a yocto build with jailhouse and linux kernel
6.1.22-rt8
 > which is having the patches for real time linux.
 > Booted the board with running run '/jh_mmcboot/' in the uboot
prompt.
 >
 >  Once logged into the device, I have started the jailhouse by 
running,

 >
 > /cd /usr/share/jailhouse/scripts && ./init_jailhouse_env.sh && cp
 > /run/media/boot-mmcblk1p1/Image
/usr/share/jailhouse/inmates/kernel/ &&
 > ../tools/jailhouse enable ../cells/imx8mp.cell; jailhouse cell
list;/
 >
 >  the above command will load the jailhouse.ko and will enable the
jailhouse
 >
 > I got the output of 'jailhouse cell list' as,
 > /root@imx8mp-lpddr4-evk:/usr/share/jailhouse/scripts# jailhouse
cell list
 > ID  Name    State Assigned
 > CPUs   Failed CPUs
 > 0   imx8mp  running   0-3 /
 >
 >  Then I started the linux inmate, the CPU offset in the cell
 > configuration is 0xc which will assign two cores. I am using the
same
 > host kernal image for the inmate also (6.1.22-rt8). Used the below
 > command to start the inmate
 >
 > ../tools/jailhouse cell linux ../cells/imx8mp-linux-demo.cell
 > ../inmates/kernel/Image -d ../inmates/dtb/inmate-imx8mp-evk.dtb -c
 > "clk_ignore_unused console=ttymxc3,115200
 > earlycon=ec_imx6q,0x3089,115200 root=/dev/mmcblk2p2 rootwait
rw";
 > jailhouse cell list;
 >
 >  I got the output of 'jailhouse cell list' as,
 >
 > /root@imx8mp-lpddr4-evk:/usr/share/jailhouse/scripts# jailhouse
cell list
 > ID  Name    State Assigned
 > CPUs   Failed CPUs
 > 0   imx8mp  running   0-1
 > 1   linux-inmate-demo   running   2-3 /
 >
 >  I could login to the inmate using serial console. But the output of
 > 'cat /proc/cpuinfo' is showing only one core.
 >
 > /root@imx8mp-lpddr4-evk:~# cat /proc/cpuinfo
 > processor   : 0
 > BogoMIPS    : 16.66
 > Features    : fp asimd evtstrm aes pmull sha1 sha2 crc32 cpuid
 > CPU implementer : 0x41
 > U architecture: 8
 > CPU variant : 0x0
 > CPU part    : 0xd03
 > CPU resion  : 4/
 >
 > the 'nproc --all' is giving 2
 >
 >  It looks like 2 cores are assigned to the inmate but one of the
core
 > is offline.
 >
 > / root@imx8mp-lpddr4-evk:/usr/share/jailhouse/scripts# cat
 > /sys/devices/system/cpu/cpu0/online /
 >
 > /1
 > root@imx8mp-lpddr4-evk:/usr/share/jailhouse/scripts# cat
 > /sys/devices/system/cpu/cpu1/online
 > 0/
 >
 >  Am I doing something wrong? How can I turn on both the CPUS for the
 > inmate.

What does the non-root Linux kernel console output during boot? Are
there any error messages that might give you a hint why bringing up
secondary CPUs failed?

Ralf

 >
 > Thanks and Regards,
 > Bharathiraja Nallathambi
 >
 > --
 > You received this message because you are subscribed to the Google
 > Groups "Jailhouse" group.
 > To unsubscribe from this group and stop receiving emails from it,
send
 > an email to jailhouse-de...@googlegroups.com
 > <mailto:jailhouse-de...@googlegroups.com>.
 > To view this discussion on the web visit
 >

https://groups.google.com/d/msgid/jailhouse-dev/CAGA%3DGbzmH9eiuBrKjVeeKf-

Re: Can't install jailhouse on linux-6.1 arm system

[Ralf Ramsauer]

On 17/10/2023 19:22, Dave Marples wrote:
Grr, my build suffered User Error. Once I built it properly everthing 
starts to work much better. Thanks for the help!!


Ah - just wanted to mention to make sure that you booted the right 
kernel. Excellent, good to hear!


  Ralf



# jailhouse enable imx8mm.cell
[   57.058121]  jailhouse: firmware: direct-loading firmware jailhouse.bin
[   57.068743] pci-host-generic bb80.pci: host bridge /pci@0 ranges:
[   57.075325] pci-host-generic bb80.pci:      MEM 
0x00bb90..0x00bb903fff -> 0x00bb90
[   57.084068] pci-host-generic bb80.pci: ECAM at [mem 
0xbb80-0xbb8f] for [bus 00]

[   57.092822] pci-host-generic bb80.pci: PCI host bridge to bus 0001:00
[   57.099638] pci_bus 0001:00: root bus resource [bus 00]
[   57.104908] pci_bus 0001:00: root bus resource [mem 
0xbb90-0xbb903fff]

[   57.111833] pci 0001:00:00.0: [110a:4106] type 00 class 0xff
[   57.117874] pci 0001:00:00.0: reg 0x10: [mem 0x-0x0fff]
[   57.124382] pci 0001:00:01.0: [110a:4106] type 00 class 0xff0001
[   57.130430] pci 0001:00:01.0: reg 0x10: [mem 0x-0x0fff]
[   57.137880] pci 0001:00:00.0: BAR 0: assigned [mem 0xbb90-0xbb900fff]
[   57.144709] pci 0001:00:01.0: BAR 0: assigned [mem 0xbb901000-0xbb901fff]
[   57.151711] The Jailhouse is opening.
root@localhost:~/jailhouse/configs/arm64#

On Tuesday, 17 October 2023 at 18:08:08 UTC+1 Dave Marples wrote:




Help still appreciated...Can't help getting the feeling I'm
missing a document somewhere whereby all this lot magically
becomes easy...


Did you also apply the patch that I attached? I'm pretty sure that
it's missing.


Hi Ralf,

Thanks for the reply. That patch was indeed missing but applying it
didn't change my symptoms. Just spinning up a yocto image to see
where the differences are.

Regards

DAVE



--
You received this message because you are subscribed to the Google 
Groups "Jailhouse" group.
To unsubscribe from this group and stop receiving emails from it, send 
an email to jailhouse-dev+unsubscr...@googlegroups.com 
.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jailhouse-dev/bf20ebf0-f447-4e73-8d94-e82b2d8dd836n%40googlegroups.com .


--
You received this message because you are subscribed to the Google Groups 
"Jailhouse" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jailhouse-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jailhouse-dev/626353e4-c280-4026-9fff-3e528c8809f8%40oth-regensburg.de.

Re: Can't install jailhouse on linux-6.1 arm system

[Ralf Ramsauer]

On 17/10/2023 15:04, Dave Marples wrote:

On 17/10/2023 09:48, Dave Marples wrote:
I'm building jailhouse on a arm64 linux 6.1 Debian bookworm system 
(imx8m, if it matters) using gcc 12.2.0...or, more specfically, 
failing to build. I get to the module install stage and then get;



ERROR: modpost: "__hyp_stub_vectors" 
[/root/jailhouse/driver/jailhouse.ko] undefined!
ERROR: modpost: "ioremap_page_range" 
[/root/jailhouse/driver/jailhouse.ko] undefined!
ERROR: modpost: "__get_vm_area_caller" 
[/root/jailhouse/driver/jailhouse.ko] undefined!
make[2]: *** [scripts/Makefile.modpost:126: 
/root/jailhouse/Module.symvers] Error 1

make[1]: *** [Makefile:1989: modpost] Error 2
make: *** [Makefile:40: modules] Error 2


So far as I can tell I have all of the required KALLSYMS set, so 
wondered if anyone else had trouble with recent kernels or if I'm 
missing some magic somewhere?


Just to start the follow-up on this in case it's useful to anyone else 
in future; I figured out that I'd missed the stage of patching the 
kernel ( It might make sense to add a prod for this in the 'Build & 
Installation' instructions ... ) using the patch embedded in 
gen-kernel-build.sh and adjusting to suit my kernel..is there somewhere 
else I'm supposed to get that patch from in a cleaner way?


I'm now at the point where the thing will start, but pretty quickly 
falls over in a big heap;


# jailhouse enable imx8mm.cell
[   62.953888]  jailhouse: firmware: direct-loading firmware jailhouse.bin
[   62.961239] Unable to handle kernel execute from non-executable 
memory at virtual address 8a004000
[   62.961239] Unable to handle kernel execute from non-executable 
memory at virtual address 8a004000
[   62.961239] Unable to handle kernel execute from non-executable 
memory at virtual address 8a004000
[   62.961239] Unable to handle kernel execute from non-executable 
memory at virtual address 8a004000

[   62.961248] Mem abort info:
[   62.961248] Mem abort info:
[   62.961248] Mem abort info:
[   62.961249]   ESR = 0x860e
[   62.961250]   ESR = 0x860e

Help still appreciated...Can't help getting the feeling I'm missing a 
document somewhere whereby all this lot magically becomes easy...


Did you also apply the patch that I attached? I'm pretty sure that it's 
missing.


Thanks,

  Ralf

--
You received this message because you are subscribed to the Google Groups 
"Jailhouse" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jailhouse-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jailhouse-dev/36d21a58-fb8b-47f3-977f-ab179f0ea8b6%40oth-regensburg.de.
From 54ce8bd0b5cc2b9150025e06ab50ee83bfcb62e4 Mon Sep 17 00:00:00 2001
From: Jan Kiszka 
Date: Sat, 2 Oct 2021 11:24:16 +0200
Subject: [PATCH] Revert "mm: don't allow executable ioremap mappings"

This reverts commit 8491502f787c4a902bd4f223b578ef47d3490264.

Jailhouse needs executable ioremapped RAM to start the hypervisor.

Signed-off-by: Jan Kiszka 
---
 mm/vmalloc.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/mm/vmalloc.c b/mm/vmalloc.c
index 52f50fe7db83..7fbfad571968 100644
--- a/mm/vmalloc.c
+++ b/mm/vmalloc.c
@@ -316,7 +316,7 @@ int ioremap_page_range(unsigned long addr, unsigned long end,
 {
 	int err;
 
-	err = vmap_range_noflush(addr, end, phys_addr, pgprot_nx(prot),
+	err = vmap_range_noflush(addr, end, phys_addr, prot,
  ioremap_max_page_shift);
 	flush_cache_vmap(addr, end);
 	return err;
-- 
2.42.0

Re: CPU shows offline - Jailhouse Linux inmate

[Ralf Ramsauer]

On 10/10/2023 17:17, Bharathiraja Nallathambi wrote:

Hi,

I was trying jailhouse with a linux inmate on an imx8mp-evk board. I 
have created a yocto build with jailhouse and linux kernel 6.1.22-rt8 
which is having the patches for real time linux.

Booted the board with running run '/jh_mmcboot/' in the uboot prompt.

  Once logged into the device, I have started the jailhouse by  running,

/cd /usr/share/jailhouse/scripts && ./init_jailhouse_env.sh && cp 
/run/media/boot-mmcblk1p1/Image /usr/share/jailhouse/inmates/kernel/ && 
../tools/jailhouse enable ../cells/imx8mp.cell; jailhouse cell list;/


  the above command will load the jailhouse.ko and will enable the jailhouse

I got the output of 'jailhouse cell list' as,
/root@imx8mp-lpddr4-evk:/usr/share/jailhouse/scripts# jailhouse cell list
ID  Name    State Assigned 
CPUs   Failed CPUs

0   imx8mp  running   0-3 /

  Then I started the linux inmate, the CPU offset in the cell 
configuration is 0xc which will assign two cores. I am using the same 
host kernal image for the inmate also (6.1.22-rt8). Used the below 
command to start the inmate


../tools/jailhouse cell linux ../cells/imx8mp-linux-demo.cell 
../inmates/kernel/Image -d ../inmates/dtb/inmate-imx8mp-evk.dtb -c 
"clk_ignore_unused console=ttymxc3,115200 
earlycon=ec_imx6q,0x3089,115200 root=/dev/mmcblk2p2 rootwait rw"; 
jailhouse cell list;


  I got the output of 'jailhouse cell list' as,

/root@imx8mp-lpddr4-evk:/usr/share/jailhouse/scripts# jailhouse cell list
ID  Name    State Assigned 
CPUs   Failed CPUs

0   imx8mp  running   0-1
1   linux-inmate-demo   running   2-3 /

  I could login to the inmate using serial console. But the output of 
'cat /proc/cpuinfo' is showing only one core.


/root@imx8mp-lpddr4-evk:~# cat /proc/cpuinfo
processor   : 0
BogoMIPS    : 16.66
Features    : fp asimd evtstrm aes pmull sha1 sha2 crc32 cpuid
CPU implementer : 0x41
U architecture: 8
CPU variant : 0x0
CPU part    : 0xd03
CPU resion  : 4/

the 'nproc --all' is giving 2

  It looks like 2 cores are assigned to the inmate but one of the core 
is offline.


/ root@imx8mp-lpddr4-evk:/usr/share/jailhouse/scripts# cat 
/sys/devices/system/cpu/cpu0/online /


/1
root@imx8mp-lpddr4-evk:/usr/share/jailhouse/scripts# cat 
/sys/devices/system/cpu/cpu1/online

0/

  Am I doing something wrong? How can I turn on both the CPUS for the 
inmate.


What does the non-root Linux kernel console output during boot? Are 
there any error messages that might give you a hint why bringing up 
secondary CPUs failed?


  Ralf



Thanks and Regards,
Bharathiraja Nallathambi

--
You received this message because you are subscribed to the Google 
Groups "Jailhouse" group.
To unsubscribe from this group and stop receiving emails from it, send 
an email to jailhouse-dev+unsubscr...@googlegroups.com 
.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jailhouse-dev/CAGA%3DGbzmH9eiuBrKjVeeKf--5fRFVGxwyCe6D20e4MDn66d3tw%40mail.gmail.com .


--
You received this message because you are subscribed to the Google Groups 
"Jailhouse" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jailhouse-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jailhouse-dev/3acdd865-1303-4ba1-8c4c-9fc3ca8d2c67%40oth-regensburg.de.

Re: How to use a TTC timer with jailhouse

[Ralf Ramsauer]

Hi Sara,

On 05/10/2023 11:50, Sara Alonso wrote:

Hi!

I want to create an inmate where I generate a periodic interrupt with a 
TTC timer (I am using a zynq zcu102 board). The address of the TTC is 
0xFF12 and the interrupt number is 71. I am writing the c code for 
the inmate and I have used irq_init(Handler) and irq_enable(71) to 
enable the interrupt, but now I don't know how to configure and start 
the TTC timer. Are there any functions to do that? Or how could I do it?


For Jailhouse: Extend the configuration of the inmate by assigning the 
memory region to the guest. Second, add IRQ 71 to the irqchips member of 
the inmate's configuration. Cf. zynqmp-zcu102-linux-demo.c to see how 
specific interrupts are assigned to the cell.


Have a look at the Zynq's reference manual to see how the timer can be 
used and configured.


  Ralf





--
You received this message because you are subscribed to the Google 
Groups "Jailhouse" group.
To unsubscribe from this group and stop receiving emails from it, send 
an email to jailhouse-dev+unsubscr...@googlegroups.com 
.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jailhouse-dev/7ba359f8-b5ac-4a5b-b0f8-2a4988aae36en%40googlegroups.com .


--
You received this message because you are subscribed to the Google Groups 
"Jailhouse" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jailhouse-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jailhouse-dev/16d9bc11-f820-48e3-955f-6beb04afc6f3%40oth-regensburg.de.

Re: How to set the value of pci_mmconfig_base in rootcell

[Ralf Ramsauer]

On 11/09/2023 07:25, bot crack wrote:

hi,
Now I change my a configuration to:
/.platform_info = {
.pci_mmconfig_base = 0xe000, // virt address
.pci_mmconfig_end_bus = 0,
.pci_is_virtual = 1,
.pci_domain = 1,
/
/./

Then I didn’t see the error log
/[   35.130190] pci-host-generic e000.pci: host bridge /pci@0 ranges:
[   35.130241] pci-host-generic e000.pci:      MEM 
0x00e010..0x00e0101fff -> 0x00e010
[   35.130295] pci-host-generic e000.pci: ECAM at [mem 
0xe000-0xe00f] for [bus 00]

[   35.130457] pci-host-generic e000.pci: PCI host bridge to bus 0001:00
[   35.130470] pci_bus 0001:00: root bus resource [bus 00]
[   35.130480] pci_bus 0001:00: root bus resource [mem 
0xe010-0xe0101fff]

[   35.130519] pci 0001:00:00.0: [110a:4106] type 00 class 0xff
[   35.130545] pci 0001:00:00.0: reg 0x10: [mem 0x-0x0fff]
[   35.137255] pci 0001:00:00.0: BAR 0: assigned [mem 0xe010-0xe0100fff]
/
/[   35.137602] The Jailhouse is opening.
[   35.251548] vcc3v3_lcd1_n: disabling
[   35.285180] psci: CPU3 killed (polled 0 ms)
/


Then I insert uio_ivshmem.ko (build from 
https://github.com/henning-schild-work/ivshmem-guest-code/tree/jailhouse)

/[   35.287908] Created Jailhouse cell "inmate-demo"
[   50.988039] uio_ivshmem 0001:00:00.0: ivshmem_pci_proben
[   50.988085] uio_ivshmem 0001:00:00.0: enabling device ( -> 0002)
[   50.988548] uio_ivshmem 0001:00:00.0: using jailhouse mode/
/
/
/
/
Then run /usr/local/sbin/ivshmem-demo and it will show mmap failure.
ID = 0
Maximum peers = 3
/usr/local/sbin/ivshmem-demo: mmap(state): No such device


I never used uio-ivshmem, so I don't know internals. You could use 
strace to see which mmap fails and why.


  Ralf




Why does mmap fail?
On Saturday, September 9, 2023 at 7:38:59 PM UTC+8 Ralf Ramsauer wrote:



On 09/09/2023 08:04, bot crack wrote:
 > Hi, everybody
 >
 > I encountered some problems when using ivshmem-demo. How should I
set
 > the value of pci_mmconfig_base?

Set it to a region where it does physically not collide with any other
IO/Memory.

 >
 > my rootcell:
 > /.platform_info = {
 > .pci_mmconfig_base = 0x7BC0,
 > .pci_mmconfig_end_bus = 0,
 > .pci_is_virtual = 1,
 > .pci_domain = 1,
 > .arm = {
 > .gic_version = 3,/
 >
 > ...
 >
 >
 > my linux dts:
 > /_memory {
 >
 > jailhouse: jailhouse@0x7000{
 > reg = <0x0 0x7000 0x0 0xBC0>;
 > alignment = <0x1000>;
 > no-map;
 > status = "okay";
 > };
 >
 > pci_reserved: pci@0x7BC0{
 > reg = <0x0 0x7BC0 0x0 0x0020>;
 > alignment = <0x1000>;
 > no-map;
 > status = "okay";
 > };

There's no need to reserve it via device tree, because it isn't backed
with anything in any case.

 >
 > };
 > /
 > Kernel output after inserting jailhouse.ko
 > /[  264.612191] jailhouse: loading out-of-tree module taints kernel.
 > [  264.624958] pci-host-generic 7bc0.pci: host bridge /pci@0
ranges:
 > [  264.625007] pci-host-generic 7bc0.pci:      MEM
 > 0x007bd0..0x007bd01fff -> 0x007bd0
 > [  264.625028] pci-host-generic 7bc0.pci: *resource
collision: [mem
 > 0x7bd0-0x7bd01fff] conflicts with reserved [mem
0x7000-0x7bdf]*

This happens due to your device tree reservation. If there's nothing
behind 0x7bc..., then simply go without reservation.

 > [  264.625051] pci-host-generic: probe of 7bc0.pci failed
with error -12
 > [  264.625475] The Jailhouse is opening.
 > [  264.792429] psci: CPU3 killed (polled 0 ms)
 > [  264.794210] Created Jailhouse cell "inmate-demo"
 > /
 > The operating environment is arm64, linux5.10.160
 >
 > *How should I set the value of pci_mmconfig_base?*

See above, a free (ideally aligned) memory region in your physical
address space.

Ralf

 >
 > --
 > You received this message because you are subscribed to the Google
 > Groups "Jailhouse" group.
 > To unsubscribe from this group and stop receiving emails from it,
send
 > an email to jailhouse-de...@googlegroups.com
 > <mailto:jailhouse-de...@googlegroups.com>.
 > To view this discussion on the web visit
 >

https://groups.google.com/d/msgid/jailhouse-dev/3ffc8cc2-0592-463f-ab65-bd51c2d0ed83n%40googlegroups.com 
<https://groups.google.com/d/msgid/jailhouse-dev/3ffc8cc2-0592-463f-ab65-bd51c2d0ed83n%40googlegroups.com>
 
<https://groups.google.com/d/msgid/jailhouse-dev/3ffc8cc2-0592-463f-ab65-bd51c2d0ed83n%40googlegroups.com?utm_medium=email_source

Re: How to set the value of pci_mmconfig_base in rootcell

[Ralf Ramsauer]

On 09/09/2023 08:04, bot crack wrote:

Hi, everybody

I encountered some problems when using ivshmem-demo. How should I set 
the value of pci_mmconfig_base?


Set it to a region where it does physically not collide with any other 
IO/Memory.




my rootcell:
/.platform_info = {
.pci_mmconfig_base = 0x7BC0,
.pci_mmconfig_end_bus = 0,
.pci_is_virtual = 1,
.pci_domain = 1,
.arm = {
.gic_version = 3,/

...


my linux dts:
/_memory {

jailhouse: jailhouse@0x7000{
reg = <0x0 0x7000 0x0 0xBC0>;
alignment = <0x1000>;
no-map;
status = "okay";
};

pci_reserved: pci@0x7BC0{
reg = <0x0 0x7BC0 0x0 0x0020>;
alignment = <0x1000>;
no-map;
status = "okay";
};


There's no need to reserve it via device tree, because it isn't backed 
with anything in any case.




};
/
Kernel output after inserting jailhouse.ko
/[  264.612191] jailhouse: loading out-of-tree module taints kernel.
[  264.624958] pci-host-generic 7bc0.pci: host bridge /pci@0 ranges:
[  264.625007] pci-host-generic 7bc0.pci:      MEM 
0x007bd0..0x007bd01fff -> 0x007bd0
[  264.625028] pci-host-generic 7bc0.pci: *resource collision: [mem 
0x7bd0-0x7bd01fff] conflicts with reserved [mem 0x7000-0x7bdf]*


This happens due to your device tree reservation. If there's nothing 
behind 0x7bc..., then simply go without reservation.



[  264.625051] pci-host-generic: probe of 7bc0.pci failed with error -12
[  264.625475] The Jailhouse is opening.
[  264.792429] psci: CPU3 killed (polled 0 ms)
[  264.794210] Created Jailhouse cell "inmate-demo"
/
The operating environment is arm64, linux5.10.160

*How should I set the value of pci_mmconfig_base?*


See above, a free (ideally aligned) memory region in your physical 
address space.


  Ralf



--
You received this message because you are subscribed to the Google 
Groups "Jailhouse" group.
To unsubscribe from this group and stop receiving emails from it, send 
an email to jailhouse-dev+unsubscr...@googlegroups.com 
.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jailhouse-dev/3ffc8cc2-0592-463f-ab65-bd51c2d0ed83n%40googlegroups.com .


--
You received this message because you are subscribed to the Google Groups 
"Jailhouse" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jailhouse-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jailhouse-dev/e1577ac3-ab02-4fe4-9641-b56a923d6fb2%40oth-regensburg.de.

Re: How to add new system register macro definition in arm64?

[Ralf Ramsauer]

On 07/09/2023 02:54, bot crack wrote:

Hi,
1.     #define arm_read_sysreg_64(op1, crm, val) \
asm volatile ("mrrc p15, "#op1", %Q0, %R0, "#crm"\n" \
: "=r" ((u64)(val)))   The assembly of this macro should be *"mrrc p15, 
"#op1", %Q0, %R0,"*

*
*2.   But aarch64 does not use the cp15 register, I disassembled 
inmates/lib/arm-common/timing.o(arm_read_sysreg(CNTPCT_EL0, pct64);) and 
the instruction was displayed as *"MRS X0, #3, c14, c0, #1" ,*And the 
instruction "*mrrc p15*" cannot be found in it.


Ah, inmates. Inmates use their own sysregs.h, and right, I looked at the 
wrong include, i took the arm sysregs.h instead of the arm64 one.
See inmates/lib/arm64/include/asm/sysregs.h resp. 
hypervisor/arch/arm64/include/asm/sysregs.h


And yes, on aarch64, cntpct_el0 is understood by the assembler:
https://github.com/bminor/binutils-gdb/blob/33a0b291058120c1294e90b53a5299c3ec62bad9/opcodes/aarch64-opc.c#L4944

  Ralf




On Wednesday, September 6, 2023 at 10:14:30 PM UTC+8 Ralf Ramsauer wrote:



On 06/09/2023 03:40, bot crack wrote:
 > Hi
 >
 > 1. I want to add some register definitions.
 > 2. I want to know why #define CNTPCT_EL0 SYSREG_64(0, c14) in arm64
 > *has only two arguments*, but it can be expanded into assembly "MRS
 > X0, #3, c14, c0, #1"

oh yeah, that's finest macro magic!


#define CNTPCT_EL0 SYSREG_64(0, c14)

#define SYSREG_64(...) 64, __VA_ARGS__

leads to the equivalent:

#define CNTPCT_EL0 64, 0, c14

Usage of CNTPCT_EL0:

arm_read_sysreg(CNTPCT_EL0, pct64);

with

#define arm_read_sysreg(...) _arm_read_sysreg(__VA_ARGS__)

So here, we, in fact, have

_arm_read_sysreg(64, 0, c14, pct64)

Which is defined as:

#define _arm_read_sysreg(size, ...) arm_read_sysreg_ ##
size(__VA_ARGS__)

So we get

arm_read_sysreg_64(0, c14, pct64)

which is defined as:

#define arm_read_sysreg_64(op1, crm, val) \
asm volatile ("mrrc p15, "#op1", %Q0, %R0, "#crm"\n" \
: "=r" ((u64)(val)))

The rest is done by the assembler.



 > 3. I didn’t understand the definition in
 > inmates/lib/arm64/include/asm/sysregs.h because I couldn’t find how
 > to expand the macro definition SYSREG_64

#define SYSREG_64(...) 64, __VA_ARGS__

SYSREG_64(1,2,foo) => 64, 1, 2, foo

These can be used as arguments in further macros.


 > 4. For example, I want to add a new CNTVCT_EL0 (op0=0b11, op1=0b011,
 > CRn=0b1110, CRm=0b0, op2=b010) register. How should I do it?

I don't know, but have a look at the Linux kernel to see how others did
it in a pretty similar way:


https://elixir.bootlin.com/linux/latest/source/arch/arm/include/asm/vdso/cp15.h#L32 
<https://elixir.bootlin.com/linux/latest/source/arch/arm/include/asm/vdso/cp15.h#L32>

HTH,

Ralf

--
You received this message because you are subscribed to the Google 
Groups "Jailhouse" group.
To unsubscribe from this group and stop receiving emails from it, send 
an email to jailhouse-dev+unsubscr...@googlegroups.com 
<mailto:jailhouse-dev+unsubscr...@googlegroups.com>.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jailhouse-dev/c2ec035d-3139-4090-a84d-ca1859706953n%40googlegroups.com <https://groups.google.com/d/msgid/jailhouse-dev/c2ec035d-3139-4090-a84d-ca1859706953n%40googlegroups.com?utm_medium=email_source=footer>.


--
You received this message because you are subscribed to the Google Groups 
"Jailhouse" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jailhouse-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jailhouse-dev/c72b8e4d-c872-4289-a787-a657af2df958%40oth-regensburg.de.

Re: How to add new system register macro definition in arm64?

[Ralf Ramsauer]

On 06/09/2023 03:40, bot crack wrote:

Hi

 1. I want to add some register definitions.
 2. I want to know why #define CNTPCT_EL0 SYSREG_64(0, c14) in arm64
*has only two arguments*, but it can be expanded into assembly "MRS
X0, #3, c14, c0, #1"


oh yeah, that's finest macro magic!


#define CNTPCT_EL0  SYSREG_64(0, c14)

#define SYSREG_64(...)  64, __VA_ARGS__

leads to the equivalent:

#define CNTPCT_EL0  64, 0, c14

Usage of CNTPCT_EL0:

arm_read_sysreg(CNTPCT_EL0, pct64);

with

#define arm_read_sysreg(...) _arm_read_sysreg(__VA_ARGS__)

So here, we, in fact, have

_arm_read_sysreg(64, 0, c14, pct64)

Which is defined as:

#define _arm_read_sysreg(size, ...) arm_read_sysreg_ ## size(__VA_ARGS__)

So we get

arm_read_sysreg_64(0, c14, pct64)

which is defined as:

#define arm_read_sysreg_64(op1, crm, val) \
asm volatile ("mrrc p15, "#op1", %Q0, %R0, "#crm"\n" \
: "=r" ((u64)(val)))

The rest is done by the assembler.




 3. I didn’t understand the definition in
inmates/lib/arm64/include/asm/sysregs.h because I couldn’t find how
to expand the macro definition SYSREG_64


#define SYSREG_64(...) 64, __VA_ARGS__

SYSREG_64(1,2,foo) => 64, 1, 2, foo

These can be used as arguments in further macros.



 4. For example, I want to add a new CNTVCT_EL0 (op0=0b11, op1=0b011,
CRn=0b1110, CRm=0b0, op2=b010) register. How should I do it?


I don't know, but have a look at the Linux kernel to see how others did 
it in a pretty similar way:


https://elixir.bootlin.com/linux/latest/source/arch/arm/include/asm/vdso/cp15.h#L32

HTH,

  Ralf

--
You received this message because you are subscribed to the Google Groups 
"Jailhouse" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jailhouse-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jailhouse-dev/556c7680-b856-4791-a782-e42b05540ae2%40oth-regensburg.de.

Re: when enter_hypervisor on rk3568, system reboot without oops

[Ralf Ramsauer]

Hi,

What does jailhouse config-check report on your system configuration?

  Ralf

On 05/09/2023 10:52, Zhan Zheng wrote:

dmesg in `screen-exchange`

在2023年9月5日星期二 UTC+8 16:00:57 写道：

hello, everyone. I'm porting jailhouse to rk3568 which running open
harmony.
jailhouse.ko, jailhouse.bin, jailhouse were ready, and i wrote a
basic root cell to test it.
but when i `jailhouse enable rk3568.cell`, the system reboot. can
someone help me pls?
thanks

--
You received this message because you are subscribed to the Google 
Groups "Jailhouse" group.
To unsubscribe from this group and stop receiving emails from it, send 
an email to jailhouse-dev+unsubscr...@googlegroups.com 
<mailto:jailhouse-dev+unsubscr...@googlegroups.com>.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jailhouse-dev/a5a7147d-750d-4000-87f5-6ca0cc93fcb2n%40googlegroups.com <https://groups.google.com/d/msgid/jailhouse-dev/a5a7147d-750d-4000-87f5-6ca0cc93fcb2n%40googlegroups.com?utm_medium=email_source=footer>.


--
Mit freundlichen Grüßen

Dr. Ralf Ramsauer
Labor für Digitalisierung
Fakultät für Informatik und Mathematik

Ostbayerische Technische Hochschule Regensburg
Galgenbergstrasse 32
93053 Regensburg

Tel.: +49 (0)941 943-9267
E-Mail: ralf.ramsa...@oth-regensburg.de
http://www.oth-regensburg.de

--
You received this message because you are subscribed to the Google Groups 
"Jailhouse" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jailhouse-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jailhouse-dev/a4c85fc2-4bf4-4c07-830f-67a60a6d40a6%40oth-regensburg.de.

Re: The jailhouse system hangs when running on the 4*A55 board

[Ralf Ramsauer]

On 04/09/2023 10:11, bot crack wrote:

The jailhouse system hangs when running on the 4*A55 board

jailhouse-config-check:
/Reading configuration set:
   Architecture:  arm64
   Root cell:     RootCell (a55-main.cell)
Overlapping memory regions inside cell: None
Overlapping memory regions with hypervisor: None
Missing PCI MMCONFIG interceptions: None
Missing resource interceptions for architecture arm64: None/

jailhouse Started output:/
Initializing Jailhouse hypervisor  on CPU 3
Code location: 0xc0200800
Page pool usage after early setup: mem 39/992, remap 0/131072
Initializing processors:
  CPU 3... OK
  CPU 2... OK
  CPU 0... OK
  CPU 1... OK
Initializing unit: irqchip
Initializing unit: ARM SMMU v3
Initializing unit: ARM SMMU
Initializing unit: PVU IOMMU
Initializing unit: PCI
Adding virtual PCI device 00:00.0 to cell "RootCell"
Page pool usage after late setup: mem 64/992, remap 144/131072
Activating hypervisor/





1. When I run the weston (using openGL ES) program on Linux in the root 
cell, it will cause a CPU to occupy 100%. I use ftrace to track the CPU 
and display(See attachment for full output):

/# tracer: function_graph
#
# CPU  DURATION                  FUNCTION CALLS
# |     |   |                     |   |   |   |
  2)               |            _raw_spin_lock_irqsave() {
  2)   0.583 us    |              do_raw_spin_lock();
  2)   3.500 us    |            }
  2)               |            ktime_get_update_offsets_now() {
  2)   0.583 us    |              arch_counter_read();
  2)   1.750 us    |            }
  2)               |            __hrtimer_run_queues() {
  2)               |              _raw_spin_unlock_irqrestore() {
  2)   0.583 us    |                do_raw_spin_unlock();
  2)   1.750 us    |              }
  2)               |              tick_sched_timer() {
  2)               |                ktime_get() {
  2)   0.875 us    |                  arch_counter_read();
  2)   1.750 us    |                }
  2)               |                tick_sched_do_timer() {
  2)               |                  tick_do_update_jiffies64.part.0() {
  2)               |                    _raw_spin_lock() {
  2)   0.583 us    |                      do_raw_spin_lock();
  2)   1.750 us    |                    }
  2)               |                    do_timer() {
  2)   0.584 us    |                      calc_global_load();
  2)   1.750 us    |                    }
  2)               |                    _raw_spin_unlock() {
  2)   0.584 us    |                      do_raw_spin_unlock();
  2)   1.750 us    |                    }
  2)               |                    update_wall_time() {
  2)               |                      timekeeping_advance() {
  2)               |                        _raw_spin_lock_irqsave() {
  2)   0.584 us    |                          do_raw_spin_lock();
  2)   2.042 us    |                        }
  2)   0.584 us    |                        arch_counter_read();
  2)   0.583 us    |                        ntp_tick_length();
  2)   0.583 us    |                        ntp_tick_length();
  2)               |                        timekeeping_update() {
  2)   0.583 us    |                          ntp_get_next_leap();
  2)   0.875 us    |                          update_vsyscall();
  2)   0.583 us    |                          raw_notifier_call_chain();
  2)   4.375 us    |                        }
  2)               |                        _raw_spin_unlock_irqrestore() {
  2)   0.583 us    |                          do_raw_spin_unlock();
  2)   1.750 us    |                        }
  2) + 14.000 us   |                      }
  2) + 15.166 us   |                    }
  2) + 23.334 us   |                  }
  2) + 24.500 us   |                }
  2)               |                update_process_times() {
  2)               |                  account_process_tick() {
  2)               |                    account_system_time() {
  2)               |                      account_system_index_time() {
  2)               |                        cpuacct_account_field() {
  2)   0.583 us    |                          __rcu_read_lock();
  2)   0.584 us    |                          __rcu_read_unlock();
  2)   3.208 us    |                        }
  2)   0.584 us    |                        __rcu_read_lock();
  2)   0.583 us    |                        __rcu_read_unlock();
  2)               |                        cpufreq_acct_update_power() {
  2)               |                          _raw_spin_lock_irqsave() {
  2)   0.584 us    |                            do_raw_spin_lock();
  2)   1.750 us    |                          }
  2)               | 
  _raw_spin_unlock_irqrestore() {

  2)   0.583 us    |                            do_raw_spin_unlock();
  2)   1.750 us    |                          }
  2)   5.250 us    |                        }
  2) + 12.542 us   |                      }
  2) + 13.708 us   |             

Re: Configuration with 2 guest cells

[Ralf Ramsauer]

On 27/07/2023 10:58, Jan-Marc Stranz wrote:

Hello Ralf!

Thank you very much for the explanations!

I wonder now that if the hypervisor now has to handle a very large page 
table, the timing behavior of an application in the guest cell will be 
negatively affected.


Of course, can be.



Could this then be optimized, e.g. using "huge pages"?


Yes. Depending on the alignment, Jailhouse will automatically try to map 
huge pages, if possible.



Is there a description for this?


See here for x86 paging mechanisms:

https://github.com/siemens/jailhouse/blob/master/hypervisor/arch/x86/paging.c#L134C4-L134C35

Besides 4k pages, you also have 2M and 1G pages. However, 1G pages 
require, that the physical and virtual address are 1G-aligned.


  Ralf




Thanks
Jan.

Ralf Ramsauer schrieb am Mittwoch, 26. Juli 2023 um 15:48:44 UTC+2:



On 26/07/2023 14:41, Jan-Marc Stranz wrote:
 > Thanks for the tip to increase the memory area for the hypervisor!
 >
 > I have increased the memory area for the hypervisor from 6 MiB to
9 MiB
 > and can now successfully start both guest cells with a size of
512 MiB each.
 >
 > I don't know if there is a hint in the documentation for the
hypervisor
 > "Jailhouse" (in the GIT repository).
 > If not, it would certainly be very helpful if this correlation
(guest
 > cell size - required memory for hypervisor) is described
somewhere (e.g.
 > in the file "Documentation/non-root-linux.txt").

Patches welcome! I'd rather recommend to print a proper error
message in
the hypervisor that gives a pointer to this issue.

Just a short explanation what happened: Jailhouse tries to map huge
pages to minimise pressure on TLB. In your case, the area isn't aligned
with huge pages, so Jailhouse must map tons of 4k-Pages to cover that
memory region. Mapping those pages requires memory for page tables, and
apparently, 6MiB weren't sufficient.

    Thanks
    Ralf

 >
 > Jan-Marc.
 >
 > Ralf Ramsauer schrieb am Mittwoch, 26. Juli 2023 um 13:12:02 UTC+2:
 >
 >
 >
 > On 26/07/2023 12:56, Jan-Marc Stranz wrote:
 > > Of course, I checked the configuration for the root cell with
 > "jailhouse
 > > config check".
 > > However, this is not changed at all while I change the
 > configuration for
 > > the guest cells.
 > >
 > > Also, everything is fine as long as I don't set the size for the
 > guest
 > > cell larger than 320 MiB in the guest cell configuration.
 > > This only uses about 62% of the memory area reserved for the
 > guest cells
 > > in the configuration for the root cell.
 > >
 > > I will follow up on the tip to increase the memory area for the
 > > hypervisor ( hypervisor_memory.size).
 > > So far, 6 MiB is reserved for the hypervisor itself.
 > > To what size should I increase this memory area?
 >
 > Just a bit more. Try 10 MiB or so.
 >
 > Ralf
 >
 > >
 > > However, to find the cause in the driver or hypervisor, I lack the
 > > necessary knowledge about the structure and relationships.
 > > So I have to rely on external help.
 > >
 > > Thanks a lot!
 > > Jan-Marc.
 > >
 > > --
 > > You received this message because you are subscribed to the Google
 > > Groups "Jailhouse" group.
 > > To unsubscribe from this group and stop receiving emails from it,
 > send
 > > an email to jailhouse-de...@googlegroups.com
 > > <mailto:jailhouse-de...@googlegroups.com>.
 > > To view this discussion on the web visit
 > >
 >
https://groups.google.com/d/msgid/jailhouse-dev/43f41f9d-0930-416b-a618-1955e237cc87n%40googlegroups.com 
<https://groups.google.com/d/msgid/jailhouse-dev/43f41f9d-0930-416b-a618-1955e237cc87n%40googlegroups.com> 
<https://groups.google.com/d/msgid/jailhouse-dev/43f41f9d-0930-416b-a618-1955e237cc87n%40googlegroups.com 
<https://groups.google.com/d/msgid/jailhouse-dev/43f41f9d-0930-416b-a618-1955e237cc87n%40googlegroups.com>> 
<https://groups.google.com/d/msgid/jailhouse-dev/43f41f9d-0930-416b-a618-1955e237cc87n%40googlegroups.com?utm_medium=email_source=footer
 
<https://groups.google.com/d/msgid/jailhouse-dev/43f41f9d-0930-416b-a618-1955e237cc87n%40googlegroups.com?utm_medium=email_source=footer>
 
<https://groups.google.com/d/msgid/jailhouse-dev/43f41f9d-0930-416b-a618-1955e237cc87n%40googlegroups.com?utm_medium=email_source=footer
 
<https://groups.google.com/d/msgid/jailhouse-dev/43f41f9d-0930-416b-a618-1955e237cc87n%40googlegroups.com?u

Re: Configuration with 2 guest cells

[Ralf Ramsauer]

On 26/07/2023 14:41, Jan-Marc Stranz wrote:

Thanks for the tip to increase the memory area for the hypervisor!

I have increased the memory area for the hypervisor from 6 MiB to 9 MiB 
and can now successfully start both guest cells with a size of 512 MiB each.


I don't know if there is a hint in the documentation for the hypervisor 
"Jailhouse" (in the GIT repository).
If not, it would certainly be very helpful if this correlation (guest 
cell size - required memory for hypervisor) is described somewhere (e.g. 
in the file "Documentation/non-root-linux.txt").


Patches welcome! I'd rather recommend to print a proper error message in 
the hypervisor that gives a pointer to this issue.


Just a short explanation what happened: Jailhouse tries to map huge 
pages to minimise pressure on TLB. In your case, the area isn't aligned 
with huge pages, so Jailhouse must map tons of 4k-Pages to cover that 
memory region. Mapping those pages requires memory for page tables, and 
apparently, 6MiB weren't sufficient.


Thanks
  Ralf



Jan-Marc.

Ralf Ramsauer schrieb am Mittwoch, 26. Juli 2023 um 13:12:02 UTC+2:



On 26/07/2023 12:56, Jan-Marc Stranz wrote:
 > Of course, I checked the configuration for the root cell with
"jailhouse
 > config check".
 > However, this is not changed at all while I change the
configuration for
 > the guest cells.
 >
 > Also, everything is fine as long as I don't set the size for the
guest
 > cell larger than 320 MiB in the guest cell configuration.
 > This only uses about 62% of the memory area reserved for the
guest cells
 > in the configuration for the root cell.
 >
 > I will follow up on the tip to increase the memory area for the
 > hypervisor ( hypervisor_memory.size).
 > So far, 6 MiB is reserved for the hypervisor itself.
 > To what size should I increase this memory area?

Just a bit more. Try 10 MiB or so.

Ralf

 >
 > However, to find the cause in the driver or hypervisor, I lack the
 > necessary knowledge about the structure and relationships.
 > So I have to rely on external help.
 >
 > Thanks a lot!
 > Jan-Marc.
 >
 > --
 > You received this message because you are subscribed to the Google
 > Groups "Jailhouse" group.
 > To unsubscribe from this group and stop receiving emails from it,
send
 > an email to jailhouse-de...@googlegroups.com
 > <mailto:jailhouse-de...@googlegroups.com>.
 > To view this discussion on the web visit
 >

https://groups.google.com/d/msgid/jailhouse-dev/43f41f9d-0930-416b-a618-1955e237cc87n%40googlegroups.com 
<https://groups.google.com/d/msgid/jailhouse-dev/43f41f9d-0930-416b-a618-1955e237cc87n%40googlegroups.com>
 
<https://groups.google.com/d/msgid/jailhouse-dev/43f41f9d-0930-416b-a618-1955e237cc87n%40googlegroups.com?utm_medium=email_source=footer
 
<https://groups.google.com/d/msgid/jailhouse-dev/43f41f9d-0930-416b-a618-1955e237cc87n%40googlegroups.com?utm_medium=email_source=footer>>.

--
You received this message because you are subscribed to the Google 
Groups "Jailhouse" group.
To unsubscribe from this group and stop receiving emails from it, send 
an email to jailhouse-dev+unsubscr...@googlegroups.com 
<mailto:jailhouse-dev+unsubscr...@googlegroups.com>.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jailhouse-dev/57a4477a-2c32-49ef-8ccc-e95e1d9fb815n%40googlegroups.com <https://groups.google.com/d/msgid/jailhouse-dev/57a4477a-2c32-49ef-8ccc-e95e1d9fb815n%40googlegroups.com?utm_medium=email_source=footer>.


--
You received this message because you are subscribed to the Google Groups 
"Jailhouse" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jailhouse-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jailhouse-dev/27f131ff-b744-05fc-0e81-bd2b1267d394%40oth-regensburg.de.

Re: Configuration with 2 guest cells

[Ralf Ramsauer]

On 26/07/2023 12:56, Jan-Marc Stranz wrote:
Of course, I checked the configuration for the root cell with "jailhouse 
config check".
However, this is not changed at all while I change the configuration for 
the guest cells.


Also, everything is fine as long as I don't set the size for the guest 
cell larger than 320 MiB in the guest cell configuration.
This only uses about 62% of the memory area reserved for the guest cells 
in the configuration for the root cell.


I will follow up on the tip to increase the memory area for the 
hypervisor ( hypervisor_memory.size).

So far, 6 MiB is reserved for the hypervisor itself.
To what size should I increase this memory area?


Just a bit more. Try 10 MiB or so.

  Ralf



However, to find the cause in the driver or hypervisor, I lack the 
necessary knowledge about the structure and relationships.

So I have to rely on external help.

Thanks a lot!
Jan-Marc.

--
You received this message because you are subscribed to the Google 
Groups "Jailhouse" group.
To unsubscribe from this group and stop receiving emails from it, send 
an email to jailhouse-dev+unsubscr...@googlegroups.com 
.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jailhouse-dev/43f41f9d-0930-416b-a618-1955e237cc87n%40googlegroups.com .


--
You received this message because you are subscribed to the Google Groups 
"Jailhouse" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jailhouse-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jailhouse-dev/d5ffbbc4-d926-5bc0-e7bb-fa400875039b%40oth-regensburg.de.

Re: Configuration with 2 guest cells

[Ralf Ramsauer]

On 26/07/2023 10:14, Jan-Marc Stranz wrote:

I have a HW target with Intel core i5  (11th generation) and 32 GB RAM.

In the hypervisor configuration for the root cell, 1 GiB (1024 MiB) is 
reserved for guest cells:


         /* MemRegion: 11060-1505f : guest cells (1024 MiB) */
         {
             .phys_start = 0x11060,
             .virt_start = 0x11060,
             .size = 0x4000,
             .flags = JAILHOUSE_MEM_READ | JAILHOUSE_MEM_WRITE,
         },

In the configurations for the guest cells, this area is divided so that 
each guest cell can use a maximum of 512 MiB:


Guest 1:

         /* MemRegion: 11060-1305f : Guest Cell (1) (max. 512 
MiB) */

         /* Low RAM (1 MiB) */
         {
             .phys_start = 0x11060,
             .virt_start = 0,
             .size = 0x10,
             .flags = JAILHOUSE_MEM_READ | JAILHOUSE_MEM_WRITE | 
JAILHOUSE_MEM_EXECUTE | JAILHOUSE_MEM_DMA | JAILHOUSE_MEM_LOADABLE,

         },
         /* Communication region (4 KiB) */
         {
             .virt_start = 0x0010,
             .size = 0x1000,
             .flags = JAILHOUSE_MEM_READ | JAILHOUSE_MEM_WRITE | 
JAILHOUSE_MEM_COMM_REGION,

         },
         /* High RAM (max. 511 MiB) */
         {
             .phys_start = 0x11070,
             .virt_start = 0x0020,
             .size = 0x1ff0,
             .flags = JAILHOUSE_MEM_READ | JAILHOUSE_MEM_WRITE | 
JAILHOUSE_MEM_EXECUTE | JAILHOUSE_MEM_DMA | JAILHOUSE_MEM_LOADABLE,

         },

Guest 2:

         /* MemRegion: 13060-1505f : Guest Cell (2) (max. 512 
MiB) */

         /* Low RAM (1 MiB) */
         {
             .phys_start = 0x13060,
             .virt_start = 0,
             .size = 0x10 ,
             .flags = JAILHOUSE_MEM_READ | JAILHOUSE_MEM_WRITE | 
JAILHOUSE_MEM_EXECUTE | JAILHOUSE_MEM_DMA | JAILHOUSE_MEM_LOADABLE,

         },
         /* Communication region (4 KiB) */
         {
             .virt_start = 0x0010,
             .size = 0x10 ,
             .flags = JAILHOUSE_MEM_READ | JAILHOUSE_MEM_WRITE | 
JAILHOUSE_MEM_COMM_REGION,

         },
         /* High RAM (max. 511 MiB) */
         {
             .phys_start = 0x13070,
             .virt_start = 0x0020,
             .size = 0x1ff0 ,
             .flags = JAILHOUSE_MEM_READ | JAILHOUSE_MEM_WRITE | 
JAILHOUSE_MEM_EXECUTE | JAILHOUSE_MEM_DMA | JAILHOUSE_MEM_LOADABLE,

         },

If only 320 MiB is used in both guest cells, then the guest cells can be 
successfully started one after the other with the command "jailhouse 
cell linux".


However, if more than 320 MiB is used in guest cells (e.g. 336 MiB), 
then only one of the two guest cells can be successfully started.

When the 2nd guest cell is started, an error message is issued:

Traceback (most recent call last):
   File "/usr/libexec/jailhouse/jailhouse-cell-linux", line 737, in 
     cell = JailhouseCell(config)
   File "/usr/lib/python3.8/site-packages/pyjailhouse/cell.py", line 36, 
in __init__

     raise e
   File "/usr/lib/python3.8/site-packages/pyjailhouse/cell.py", line 33, 
in __init__

     fcntl.ioctl(self.dev, JailhouseCell.JAILHOUSE_CELL_CREATE, create)
OSError: [Errno 12] Cannot allocate memory
ERROR: Linux guest cell not started!

I have now run numerous tests with different sizes for the guest cells 
and have come to the limit of 320 MiB, at which both guest cells can 
still be started.


However, I cannot see why both guest cells cannot be started.
Actually, in the configuration for the root cell, enough memory has been 
reserved for both guest cells.


Is there an explanation for this unexpected behavior?


Did you run jailhouse-config-check over the combination of different 
configurations?


The ENOMEM must happen somewhere in the driver or hypervisor. You could 
add printks to boil that down.


What you could try: You could give the hypervisor more memory 
(hypervisor_memory.size). Of course, you have align everything to the 
new layout then. Maybe the HV runs out of memory, as it must keep track 
of more page tables, as the cell gets more memory.


  Ralf



Jan-Marc.

--
You received this message because you are subscribed to the Google 
Groups "Jailhouse" group.
To unsubscribe from this group and stop receiving emails from it, send 
an email to jailhouse-dev+unsubscr...@googlegroups.com 
.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jailhouse-dev/81fa9191-18dd-4003-9cfb-bed496d5723fn%40googlegroups.com .


--
You received this message because you are subscribed to the Google Groups 
"Jailhouse" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jailhouse-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 

Re: Simplification of the hypervisor configuration for the root cell

[Ralf Ramsauer]

Hi,

On 13/07/2023 16:42, Jan-Marc Stranz wrote:
Is there a way and the corresponding rules to simplify the hypervisor 
configuration for the root cell?


I have a hypervisor configuration for the root cell with 99 entries for 
"MemRegion".

I am now trying to merge these regions together.

My idea is to free all areas - except the areas for the hypervisor, for 
ivshmem and the guest cells.
However, this fails: when starting the hypervisor, the whole system 
"freezes".


freezes in terms of no responsiveness without a crash?

Are you sure that you did not make any mistakes while merging the 
regions? Did you decrease the number of memory regions in the 
declaration? Does jailhouse-config-check yell?


  Ralf



Then I tried to group only some areas with the same values for the 
".flags" element.
This is only partially successful and I can't really see a rule when 
areas can be grouped together or not.


How could we simplify the configuration for a root cell?

For our application we don't need the fine grained sharing of memory 
areas (security aspect).
Couldn't we share the whole address range for the root cell - except the 
ranges for the hypervisor, for ivshmem and the guest cells?


Jan-Marc.


--
You received this message because you are subscribed to the Google 
Groups "Jailhouse" group.
To unsubscribe from this group and stop receiving emails from it, send 
an email to jailhouse-dev+unsubscr...@googlegroups.com 
.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jailhouse-dev/d33e6a84-7a1d-4a28-85b0-62165fd20b56n%40googlegroups.com .


--
You received this message because you are subscribed to the Google Groups 
"Jailhouse" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jailhouse-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jailhouse-dev/214dd509-8fa9-82fd-425e-c1b6b9e74542%40oth-regensburg.de.

Re: FATAL: Unhandled HYP exception in Raspberry Pi 4 for jailhouse hypervisor

[Ralf Ramsauer]

On 19/06/2023 16:49, sai krishna Allu wrote:

Hi Ralf,

Sorry, I forgot to load the module, I have loaded now.

ubuntu@ubuntu:~/jailhouse$ sudo insmod driver/jailhouse.ko
ubuntu@ubuntu:~/jailhouse$ ls -l /dev/jailhouse
crw--- 1 root root 10, 59 Jun 19 14:45 /dev/jailhouse
ubuntu@ubuntu:~/jailhouse$ sudo jailhouse enable configs/arm64/rpi4.cell
*[ 4966.214157] jailhouse: request_mem_region failed for hypervisor memory.


Okay. Interesting. I wonder why you didn't hit that error before. 
Jailhouse requires reserved memory where it, as well as inmates, reside. 
Linux must never claim those memory areas, hence you need to reserve 
that memory areas at boot time.


See the comment in arch/arm64/rpi4.c:
 * Reservation via device tree: reg = <0x0 0x2000 0x1000>;

Did you reserve memory in your device tree?

I would recommend building a working image with jailhouse-images. There, 
are required patches and config tweaks are included out of the box.



  Ralf


JAILHOUSE_ENABLE: Invalid argument*
ubuntu@ubuntu:~/jailhouse$


I am getting the above error.

Regards,
Sai Krishna
On Monday, June 19, 2023 at 8:09:37 PM UTC+5:30 Ralf Ramsauer wrote:



On 19/06/2023 16:23, sai krishna Allu wrote:
 > Hi Ralf,
 >
 > I have downloaded the master branch and given make and I have got
the
 > following error, I fixed the error with one of the reference
link, but
 > didn't find the /dev/jailhouse post installation
 >
 > ubuntu@ubuntu:~/jailhouse$ git branch
 > * master
 > ubuntu@ubuntu:~/jailhouse$make
 > ...
 > ...
 > ...
 >   CC      /home/ubuntu/jailhouse/configs/arm64/zynqmp-zcu102.o
 >   OBJCOPY /home/ubuntu/jailhouse/configs/arm64/zynqmp-zcu102.cell
 >   DTC
/home/ubuntu/jailhouse/configs/arm64/dts/inmate-amd-seattle.dtb

 >
*/home/ubuntu/jailhouse/configs/arm64/dts/inmate-amd-seattle.dts:17:54:
 > error: no include path in which to search for
 > dt-bindings/interrupt-controller/arm-gic.h
 >    17 | #include *
 >       |                                                      ^
 > make[3]: *** [scripts/Makefile.lib:286:
 > /home/ubuntu/jailhouse/configs/arm64/dts/inmate-amd-seattle.dtb]
Error 1
 > make[2]: *** [scripts/Makefile.build:519:
 > /home/ubuntu/jailhouse/configs] Error 2
 > make[1]: *** [Makefile:1765: /home/ubuntu/jailhouse] Error 2
 > make: *** [Makefile:40: modules] Error 2
 > ubuntu@ubuntu:~/jailhouse$
 >
 > To fix the above I have followed the link
 > https://bbs.csdn.net/topics/615205697
<https://bbs.csdn.net/topics/615205697> and changed as per instruction,
 > even I followed the earlier build from this link only. but I was not
 > able to build jailhouse-image.

Ok, perfect.

 >
 >
 > DTC_INCLUDE    += /usr/src/linux-headers-5.4.0-1069-raspi/include
added
 > in the file
-->/usr/src/linux-headers-5.4.0-1011-raspi/scripts/Makefile.lib
 >
 > After that I am able to install,
 >
 > ubuntu@ubuntu:~/jailhouse$ sudo make install
 >   Building modules, stage 2.
 >   MODPOST 1 modules
 >   INSTALL /home/ubuntu/jailhouse/driver/jailhouse.ko
 > At main.c:160:
 > - SSL error:02001002:system library:fopen:No such file or directory:
 > ../crypto/bio/bss_file.c:69
 > - SSL error:2006D080:BIO routines:BIO_new_file:no such file:
 > ../crypto/bio/bss_file.c:76
 > sign-file: certs/signing_key.pem: No such file or directory
 >   DEPMOD  5.4.0-1069-raspi
 > Warning: modules_install: missing 'System.map' file. Skipping
depmod.
 > install -m 644 hypervisor/jailhouse*.bin /lib/firmware
 > install -d -m 755 /usr/local/libexec/jailhouse
 > install -m 644 inmates/tools/arm64/*.bin
/usr/local/libexec/jailhouse
 > install jailhouse demos/ivshmem-demo /usr/local/sbin
 > install jailhouse-config-collect jailhouse-cell-linux
 > jailhouse-cell-stats jailhouse-config-create jailhouse-config-check
 > jailhouse-hardware-check /usr/local/libexec/jailhouse
 > install -d -m 755 /usr/local/share/jailhouse
 > install -m 644 jailhouse-config-collect.tmpl root-cell-config.c.tmpl
 > /usr/local/share/jailhouse
 > install -m 644 jailhouse-completion.bash
 > /usr/share/bash-completion/completions/jailhouse
 > install -d -m 755 /usr/local/share/man/man8
 > install -m 644 jailhouse.8 jailhouse-cell.8 jailhouse-enable.8
 > /usr/local/share/man/man8
 > python3 -m pip install --upgrade --force-reinstall  .
 > Processing /home/ubuntu/jailhouse
 > Building wheels for collected packages: pyjailhouse
 >   Building wheel for pyjailhouse (setup.py) ... done
 >   Created wheel for pyjailhouse:
   

Re: FATAL: Unhandled HYP exception in Raspberry Pi 4 for jailhouse hypervisor

[Ralf Ramsauer]

On 19/06/2023 16:23, sai krishna Allu wrote:

Hi Ralf,

I have downloaded the master branch and given make and I have got the 
following error, I fixed the error with one of the reference link, but 
didn't find the /dev/jailhouse post installation


ubuntu@ubuntu:~/jailhouse$ git branch
* master
ubuntu@ubuntu:~/jailhouse$make
...
...
...
   CC      /home/ubuntu/jailhouse/configs/arm64/zynqmp-zcu102.o
   OBJCOPY /home/ubuntu/jailhouse/configs/arm64/zynqmp-zcu102.cell
   DTC     /home/ubuntu/jailhouse/configs/arm64/dts/inmate-amd-seattle.dtb
*/home/ubuntu/jailhouse/configs/arm64/dts/inmate-amd-seattle.dts:17:54: 
error: no include path in which to search for 
dt-bindings/interrupt-controller/arm-gic.h

    17 | #include *
       |                                                      ^
make[3]: *** [scripts/Makefile.lib:286: 
/home/ubuntu/jailhouse/configs/arm64/dts/inmate-amd-seattle.dtb] Error 1
make[2]: *** [scripts/Makefile.build:519: 
/home/ubuntu/jailhouse/configs] Error 2

make[1]: *** [Makefile:1765: /home/ubuntu/jailhouse] Error 2
make: *** [Makefile:40: modules] Error 2
ubuntu@ubuntu:~/jailhouse$

To fix the above I have followed the link 
https://bbs.csdn.net/topics/615205697 and changed as per instruction, 
even I followed the earlier build from this link only. but I was not 
able to build jailhouse-image.


Ok, perfect.




DTC_INCLUDE    += /usr/src/linux-headers-5.4.0-1069-raspi/include added 
in the file -->/usr/src/linux-headers-5.4.0-1011-raspi/scripts/Makefile.lib


After that I am able to install,

ubuntu@ubuntu:~/jailhouse$ sudo make install
   Building modules, stage 2.
   MODPOST 1 modules
   INSTALL /home/ubuntu/jailhouse/driver/jailhouse.ko
At main.c:160:
- SSL error:02001002:system library:fopen:No such file or directory: 
../crypto/bio/bss_file.c:69
- SSL error:2006D080:BIO routines:BIO_new_file:no such file: 
../crypto/bio/bss_file.c:76

sign-file: certs/signing_key.pem: No such file or directory
   DEPMOD  5.4.0-1069-raspi
Warning: modules_install: missing 'System.map' file. Skipping depmod.
install -m 644 hypervisor/jailhouse*.bin /lib/firmware
install -d -m 755 /usr/local/libexec/jailhouse
install -m 644 inmates/tools/arm64/*.bin /usr/local/libexec/jailhouse
install jailhouse demos/ivshmem-demo /usr/local/sbin
install jailhouse-config-collect jailhouse-cell-linux 
jailhouse-cell-stats jailhouse-config-create jailhouse-config-check 
jailhouse-hardware-check /usr/local/libexec/jailhouse

install -d -m 755 /usr/local/share/jailhouse
install -m 644 jailhouse-config-collect.tmpl root-cell-config.c.tmpl 
/usr/local/share/jailhouse
install -m 644 jailhouse-completion.bash 
/usr/share/bash-completion/completions/jailhouse

install -d -m 755 /usr/local/share/man/man8
install -m 644 jailhouse.8 jailhouse-cell.8 jailhouse-enable.8 
/usr/local/share/man/man8

python3 -m pip install --upgrade --force-reinstall  .
Processing /home/ubuntu/jailhouse
Building wheels for collected packages: pyjailhouse
   Building wheel for pyjailhouse (setup.py) ... done
   Created wheel for pyjailhouse: 
filename=pyjailhouse-0.12-py3-none-any.whl size=22795 
sha256=fd9fbf5b1cf127acf8f96d31e8a5b36351a39278db948c6be3bf55021a3853bb
   Stored in directory: 
/tmp/pip-ephem-wheel-cache-bv5mietb/wheels/c9/70/5f/41e4b42a3693d675b8bd9bcdcb021956f8d467789b3be58740

Successfully built pyjailhouse
Installing collected packages: pyjailhouse
Successfully installed pyjailhouse-0.12
ubuntu@ubuntu:~/jailhouse$

*but not able to get jailhouse in dev file.*


Yeah, you need to load the jailhouse module first:

insmod drivers/jailhouse.ko
or
modprobe jailhouse

Thanks
  Ralf



ubuntu@ubuntu:~/jailhouse$*ls -l /dev/jailhouse*
*ls: cannot access '/dev/jailhouse': No such file or directory*
ubuntu@ubuntu:~/jailhouse$

Please suggest the correct way to install or anything else I need to 
change or need to follow different steps to fix this issue to get the 
installation success.


Regards,
Sai Krishna

On Monday, June 19, 2023 at 4:25:09 PM UTC+5:30 Ralf Ramsauer wrote:



On 19/06/2023 12:23, sai krishna Allu wrote:
 > Hi Ralf,
 >
 > I have attached the hypervisor.o and I have taken the dump of
that which
 > is also attached too.
 > I have below command to extract, let me know is it ok or not.
 >
 > ubuntu@ubuntu:~$ aarch64-linux-gnu-objdump -d
 > jailhouse/hypervisor/hypervisor.o|tee hypervisor_dump.txt
 >
 > ubuntu@ubuntu:~$ cat hypervisor_dump.txt -n |grep "c0203820"
 > *3338    c0203820:       f9400282        ldr     x2, [x20]*
 > ubuntu@ubuntu:~$ vi hypervisor_dump.txt +3338
 >       patch+0x218>
 >  3334     c0203810:       f9400282        ldr     x2, [x20]
 >  3335     c0203814:       f9400842        ldr     x2, [x2, #16]
 >  3336     c0203818:       8a130042        and     x2, x2, x19
 >  3337     c020381c:       f90a4002        str     x2, [x0,
#524

Re: FATAL: Unhandled HYP exception in Raspberry Pi 4 for jailhouse hypervisor

[Ralf Ramsauer]

On 19/06/2023 12:23, sai krishna Allu wrote:

Hi Ralf,

I have attached the hypervisor.o and I have taken the dump of that which 
is also attached too.

I have below command to extract, let me know is it ok or not.

ubuntu@ubuntu:~$ aarch64-linux-gnu-objdump -d 
jailhouse/hypervisor/hypervisor.o|tee hypervisor_dump.txt


ubuntu@ubuntu:~$ cat hypervisor_dump.txt -n |grep "c0203820"
*3338    c0203820:       f9400282        ldr     x2, [x20]*
ubuntu@ubuntu:~$ vi hypervisor_dump.txt +3338
       patch+0x218>
  3334     c0203810:       f9400282        ldr     x2, [x20]
  3335     c0203814:       f9400842        ldr     x2, [x2, #16]
  3336     c0203818:       8a130042        and     x2, x2, x19
  3337     c020381c:       f90a4002        str     x2, [x0, #5248]
* 3338     c0203820:       f9400282        ldr     x2, [x20]*
  3339     c0203824:       f9400c42        ldr     x2, [x2, #24]
  3340     c0203828:       8a130053        and     x19, x2, x19
  3341     c020382c:       52800022        mov     w2, #0x1

The instruction at c0203820 means it Load x2 from contents of x20 
register.

Please suggest how to fix this.


Keep calm, I'm still trying to figure out where the problem exactly is.

Now I know the line of code, where the exception exactly happens. It's 
in hypervisor/arch/arm-common/psci.c:39


ldr x2, [x20] <- Here we likely load ctx->regs[2], and the only variant 
how this could fail is that ctx points to somewhere. Here we're inside 
the routine psci_emulate_cpu_on, which shouldn't even be called when 
enabling the hypervisor. I don't know yet what could be the issue.


Before we keep on debugging: I saw that you use Jailhouse v0.12.

Could you please switch to the master and/or next branch of jailhouse, 
and see if the error still occurs there?


Thanks,
  Ralf



Regards,
Sai Krishna

On Saturday, June 17, 2023 at 8:59:56 PM UTC+5:30 Ralf Ramsauer wrote:

Hi,

On 15/06/2023 10:11, sai krishna Allu wrote:
 > Hi Team,
 >
 > I have Raspberry Pi 4, which is installed with Ubuntu 20.04.5 LTS
64 bit.
 > after loading the jailhouse.ko file, when I gave following
command I am
 > getting the exception.
 >
 > ubuntu@ubuntu:~/jailhouse$ sudo jailhouse enable
configs/arm64/rpi4.cell
 >
 > Initializing Jailhouse hypervisor v0.12 (0-g92db71f2) on CPU 0
 > Code location: 0xc0200800
 > Page pool usage after early setup: mem 39/994, remap 0/131072
 > Initializing processors:
 >  CPU 0...
 > FATAL: Unhandled HYP exception: synchronous abort from EL2

Synchronous abort from EL2 means that we have a fatal error inside
Jailhouse (e.g., segfault).

 >  pc: c0203820   lr: c020380c spsr: 83c9     EL2

Could you please disassemble your hypervisor binary, and see where the
exception exactly occurs? Do this with:

aarch64-objdump -d hypervisor/hypervisor.o

And see what code executes behind 0xc0203820. Alternatively,
you
can also attach the hypervisor.o.

Thanks,
Ralf

 >  sp: c021ee40  esr: 00 1 000
 >  x0: 8400   x1: 0008   x2: 80003580
 >  x3: c020e170   x4: 0014   x5: 03ff
 >  x6: 0029   x7: c0219d30   x8: 002a
 >  x9:   x10:   x11: 0001
 > x12: 0015  x13: 0001  x14: c0219000
 > x15: c0015040  x16: c020d1fc  x17: bcae22826688
 > x18: 0001  x19: c021e000  x20: c0219000
 > x21: c020  x22: c0219000  x23: 
 > x24: 0001  x25: c021e000  x26: c021f000
 > x27: c020f000  x28: c0218000  x29: c021ee40
 >
 > Hypervisor stack before exception Stopping CPU 0 (Cell:
"Raspberry-Pi4")
 >
 >
 > Please suggest how to fix this problem.
 >
 > Regards,
 > Sai Krishna
 >
 > --
 > You received this message because you are subscribed to the Google
 > Groups "Jailhouse" group.
 > To unsubscribe from this group and stop receiving emails from it,
send
 > an email to jailhouse-de...@googlegroups.com
 > <mailto:jailhouse-de...@googlegroups.com>.
 > To view this discussion on the web visit
 >

https://groups.google.com/d/msgid/jailhouse-dev/3e6d4f38-5938-4da0-9666-9093a0b76300n%40googlegroups.com 
<https://groups.google.com/d/msgid/jailhouse-dev/3e6d4f38-5938-4da0-9666-9093a0b76300n%40googlegroups.com>
 
<https://groups.google.com/d/msgid/jailhouse-dev/3e6d4f38-5938-4da0-9666-9093a0b76300n%40googlegroups.com?utm_medium=email_source=foo

Re: FATAL: Unhandled HYP exception in Raspberry Pi 4 for jailhouse hypervisor

[Ralf Ramsauer]

Hi,

On 15/06/2023 10:11, sai krishna Allu wrote:

Hi Team,

I have Raspberry Pi 4, which is installed with Ubuntu 20.04.5 LTS 64 bit.
after loading the jailhouse.ko file, when I gave following command I am 
getting the exception.


ubuntu@ubuntu:~/jailhouse$ sudo jailhouse enable configs/arm64/rpi4.cell

Initializing Jailhouse hypervisor v0.12 (0-g92db71f2) on CPU 0
Code location: 0xc0200800
Page pool usage after early setup: mem 39/994, remap 0/131072
Initializing processors:
  CPU 0...
FATAL: Unhandled HYP exception: synchronous abort from EL2


Synchronous abort from EL2 means that we have a fatal error inside 
Jailhouse (e.g., segfault).



  pc: c0203820   lr: c020380c spsr: 83c9     EL2


Could you please disassemble your hypervisor binary, and see where the 
exception exactly occurs? Do this with:


aarch64-objdump -d hypervisor/hypervisor.o

And see what code executes behind 0xc0203820. Alternatively, you 
can also attach the hypervisor.o.


Thanks,
  Ralf


  sp: c021ee40  esr: 00 1 000
  x0: 8400   x1: 0008   x2: 80003580
  x3: c020e170   x4: 0014   x5: 03ff
  x6: 0029   x7: c0219d30   x8: 002a
  x9:   x10:   x11: 0001
x12: 0015  x13: 0001  x14: c0219000
x15: c0015040  x16: c020d1fc  x17: bcae22826688
x18: 0001  x19: c021e000  x20: c0219000
x21: c020  x22: c0219000  x23: 
x24: 0001  x25: c021e000  x26: c021f000
x27: c020f000  x28: c0218000  x29: c021ee40

Hypervisor stack before exception Stopping CPU 0 (Cell: "Raspberry-Pi4")


Please suggest how to fix this problem.

Regards,
Sai Krishna

--
You received this message because you are subscribed to the Google 
Groups "Jailhouse" group.
To unsubscribe from this group and stop receiving emails from it, send 
an email to jailhouse-dev+unsubscr...@googlegroups.com 
.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jailhouse-dev/3e6d4f38-5938-4da0-9666-9093a0b76300n%40googlegroups.com .


--
You received this message because you are subscribed to the Google Groups 
"Jailhouse" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jailhouse-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jailhouse-dev/c7f684bd-ffcd-4ab2-076e-03dac205339e%40oth-regensburg.de.

Re: NXP adoption of jailhouse hypervisor

[Ralf Ramsauer]

Hi Peng,

On 16/06/2023 03:50, Peng Fan wrote:

NXP adoption of jailhouse hypervisor

https://www.nxp.com/design/development-boards/i-mx-evaluation-and-development-boards/harpoon-rtos-on-cortex-a:HARPOON


Uh, very nice! Thanks for the pointer and for the work!

FYI: We just successfully enabled Jailhouse for NXP's S32G{2,3} 
platform. Patches will likely be upstreamed soon.


Thanks,
  Ralf



Regards,
Peng.



--
You received this message because you are subscribed to the Google Groups 
"Jailhouse" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jailhouse-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jailhouse-dev/1bb61848-d373-4eb8-2b7e-1601a5a02b8c%40oth-regensburg.de.

[PATCH v3 73/73] core: riscv: Add support for Zicboz extension

[Ralf Ramsauer]

Signed-off-by: Ralf Ramsauer 
---
 hypervisor/arch/riscv/include/asm/csr64.h | 1 +
 hypervisor/arch/riscv/setup.c | 4 ++--
 2 files changed, 3 insertions(+), 2 deletions(-)

diff --git a/hypervisor/arch/riscv/include/asm/csr64.h 
b/hypervisor/arch/riscv/include/asm/csr64.h
index 8651d0f1..a85d55c4 100644
--- a/hypervisor/arch/riscv/include/asm/csr64.h
+++ b/hypervisor/arch/riscv/include/asm/csr64.h
@@ -35,6 +35,7 @@
 #define CSR_HGEIE  0x607
 #define CSR_HENVCFG0x60a
 #define ENVCFG_STCE(_AC(1, ULL) << 63)
+#define ENVCFG_CBZE(_AC(1, UL) << 7)
 #define CSR_HTVAL  0x643
 #define CSR_HIP0x644
 #define CSR_HVIP   0x645
diff --git a/hypervisor/arch/riscv/setup.c b/hypervisor/arch/riscv/setup.c
index 32d5b4da..cf1dd69a 100644
--- a/hypervisor/arch/riscv/setup.c
+++ b/hypervisor/arch/riscv/setup.c
@@ -159,8 +159,8 @@ void __attribute__ ((noreturn)) arch_cpu_activate_vmm(void)
tmp = csr_read(sip);
csr_write(CSR_HVIP, tmp << VSIP_TO_HVIP_SHIFT); /* reinject pending */
 
-   /* try to enable SSTC extension, if available */
-   csr_write(CSR_HENVCFG, ENVCFG_STCE);
+   /* try to enable SSTC and Zicboz extension, if available */
+   csr_write(CSR_HENVCFG, ENVCFG_STCE | ENVCFG_CBZE);
/* STCE is WARL, check its presence */
has_sstc = !!(csr_read(CSR_HENVCFG) & ENVCFG_STCE);
/*
-- 
2.40.1

-- 
You received this message because you are subscribed to the Google Groups 
"Jailhouse" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jailhouse-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jailhouse-dev/20230519204033.643200-74-ralf.ramsauer%40oth-regensburg.de.

[PATCH v3 68/73] core: riscv: Migrate IMSIC files, if IMSIC is available

[Ralf Ramsauer]

If an IMSIC is available, i.e., when a valid VS-mode File is set in the
configuration, then migrate corresponding CSR registers over to the
target VS-mode file. Undo it, when disabling jailhouse.

Signed-off-by: Ralf Ramsauer 
---
 hypervisor/arch/riscv/control.c|  5 ++
 hypervisor/arch/riscv/include/asm/cell.h   |  2 +
 hypervisor/arch/riscv/include/asm/csr64.h  | 19 ++
 hypervisor/arch/riscv/include/asm/percpu.h | 16 -
 hypervisor/arch/riscv/setup.c  | 78 +-
 5 files changed, 116 insertions(+), 4 deletions(-)

diff --git a/hypervisor/arch/riscv/control.c b/hypervisor/arch/riscv/control.c
index fbd8e81d..c0701abe 100644
--- a/hypervisor/arch/riscv/control.c
+++ b/hypervisor/arch/riscv/control.c
@@ -134,6 +134,9 @@ int arch_cell_create(struct cell *const cell)
cell->arch.mm.root_table =
page_alloc_aligned(_pool, CELL_ROOT_PT_PAGES);
 
+   /* Always take VS file 1 for every cell, if IMSIC is available */
+   cell->arch.vs_file = imsic_base() ? 1 : 0;
+
for_each_cpu(cpu, >cpu_set) {
ppc = public_per_cpu(cpu);
ppc->wait_for_power_on = false;
@@ -260,6 +263,8 @@ void riscv_park_cpu(void)
 {
this_cpu_public()->hsm.state = STOPPED;
 
+   cpu_set_vs_file(this_cell()->arch.vs_file);
+
/*
 * When parking a HART, we let the CPU spin in a wfi loop. To avoid
 * that we busily spin that loop, deactivate interrupts while sitting
diff --git a/hypervisor/arch/riscv/include/asm/cell.h 
b/hypervisor/arch/riscv/include/asm/cell.h
index 2d0002f6..2f1cfbd4 100644
--- a/hypervisor/arch/riscv/include/asm/cell.h
+++ b/hypervisor/arch/riscv/include/asm/cell.h
@@ -37,6 +37,8 @@ struct arch_cell {
unsigned int target[APLIC_MAX_VIRQ];
unsigned int enabled;
} aplic_virq;
+
+   unsigned short vs_file;
 };
 
 #endif /* !_JAILHOUSE_ASM_CELL_H */
diff --git a/hypervisor/arch/riscv/include/asm/csr64.h 
b/hypervisor/arch/riscv/include/asm/csr64.h
index a12f3ffb..8651d0f1 100644
--- a/hypervisor/arch/riscv/include/asm/csr64.h
+++ b/hypervisor/arch/riscv/include/asm/csr64.h
@@ -41,6 +41,24 @@
 #define CSR_HTINST 0x64a
 #define CSR_HGATP  0x680
 
+/* IMSIC CSRs */
+#define CSR_SISELECT   0x150
+#define  CSR_SIREG_EIDELIVERY  0x70
+#define  CSR_SIREG_EITHRESHOLD 0x72
+#define  CSR_SIREG_EIP00x80
+#define  CSR_SIREG_EIE00xc0
+#define CSR_SIREG  0x151
+#define CSR_VSISELECT  0x250
+#define CSR_VSIREG 0x251
+
+#define CSR_STOPEI 0x15c
+#define CSR_VSTOPEI0x25c
+
+#define CSR_STOPI  0xdb0
+#define CSR_VSTOPI 0xeb0
+
+#define TOPI_IID_SHIFT 16
+
 /* Status register flags */
 #define SR_SIE _AC(0x0002, UL) /* Supervisor Interrupt Enable */
 #define SR_MIE _AC(0x0008, UL) /* Machine Interrupt Enable */
@@ -131,6 +149,7 @@
 #define HSTATUS_VTVM_AC(0x0010, UL)
 #define HSTATUS_VGEIN   _AC(0x0003f000, UL)
 #define HSTATUS_VGEIN_SHIFT 12
+#define HSTATUS_VGEIN_MASK BIT_MASK(17, HSTATUS_VGEIN_SHIFT)
 #define HSTATUS_HU  _AC(0x0200, UL)
 #define HSTATUS_SPVP_AC(0x0100, UL)
 #define HSTATUS_SPV _AC(0x0080, UL)
diff --git a/hypervisor/arch/riscv/include/asm/percpu.h 
b/hypervisor/arch/riscv/include/asm/percpu.h
index 4eb88b57..22635a91 100644
--- a/hypervisor/arch/riscv/include/asm/percpu.h
+++ b/hypervisor/arch/riscv/include/asm/percpu.h
@@ -2,7 +2,7 @@
  * Jailhouse, a Linux-based partitioning hypervisor
  *
  * Copyright (c) Siemens AG, 2020
- * Copyright (c) OTH Regensburg, 2022
+ * Copyright (c) OTH Regensburg, 2022-2023
  *
  * Authors:
  *  Konrad Schwarz 
@@ -12,6 +12,7 @@
  * the COPYING file in the top-level directory.
  */
 
+#include 
 #include 
 #include 
 
@@ -27,6 +28,16 @@ enum sbi_hart_state {
RESUME_PENDING  = 6,
 };
 
+static inline void cpu_set_vs_file(unsigned int file)
+{
+   u64 hstatus;
+
+   hstatus = csr_read(CSR_HSTATUS);
+   hstatus &= ~HSTATUS_VGEIN_MASK;
+   hstatus |= file << HSTATUS_VGEIN_SHIFT;
+   csr_write(CSR_HSTATUS, hstatus);
+}
+
 #define ARCH_PUBLIC_PERCPU_FIELDS  \
unsigned long phys_id;  \
enum {  \
@@ -47,6 +58,7 @@ enum sbi_hart_state {
u32 enabled_bitmap[MAX_IRQS / (sizeof(u32) * 8)];   \
u32 pending_bitmap[MAX_IRQS / (sizeof(u32) * 8)];   \
unsigned int aplic_pending; \
-   } virq;
+   } virq; \
+   unsigned long imsic_base;
 
 #define ARCH_PERCPU_FIELDS
diff --git a/hypervisor/arch/riscv/setup.c b/hypervisor/arch/riscv/setup.c
index b1b93d11..94971dd3 100644
--- a/hypervisor/arch/riscv/setup.c

[PATCH v3 72/73] core: riscv: setup: Add Qemu Workaround

[Ralf Ramsauer]

At least on Qemu, we might have the case that external interrupts stuck
pending in hip when enabling the hypervisor. This happens, if a IRQ is
set pending before it got migrated to the guest. This might be a
misbehaviour in Qemu.

Signed-off-by: Ralf Ramsauer 
---
 hypervisor/arch/riscv/setup.c | 6 ++
 1 file changed, 6 insertions(+)

diff --git a/hypervisor/arch/riscv/setup.c b/hypervisor/arch/riscv/setup.c
index 62911055..32d5b4da 100644
--- a/hypervisor/arch/riscv/setup.c
+++ b/hypervisor/arch/riscv/setup.c
@@ -189,6 +189,7 @@ void __attribute__ ((noreturn)) arch_cpu_activate_vmm(void)
 * well as MSIs) are migrated, and we can safely migrate all pending
 * IRQs from the old S-Mode file to the VS-File.
 */
+
 if (csr_read(CSR_HSTATUS) & HSTATUS_VGEIN) {
imsic_migrate_regs(imsic_migrate_to_vs);
imsic_migration_done = true;
@@ -200,6 +201,11 @@ void __attribute__ ((noreturn)) arch_cpu_activate_vmm(void)
ext_disable();
}
 
+   // HACK: Qemu: If VSEIP is set in hip (via hvip) is pending when
+   // enabling the hypervisor, clear it. Otherwise it will remain
+   // indefinetly set (at least in Qemu)
+   csr_clear(CSR_HVIP, VIE_EIE | VIE_TIE);
+
tmp = csr_swap(sscratch, regs->sp);
asm volatile("mv sp, %0\n"
 "j vmreturn\n" : : "r"(tmp));
-- 
2.40.1

-- 
You received this message because you are subscribed to the Google Groups 
"Jailhouse" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jailhouse-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jailhouse-dev/20230519204033.643200-73-ralf.ramsauer%40oth-regensburg.de.

[PATCH v3 71/73] core: riscv: control: fix parking race condition

[Ralf Ramsauer]

When a CPU is parked on RISC-V, we set the per-cpu VS-file in S-Mode.
The VS-file depends on the current cell. So if we park a CPU in case of
destruction of a cell, set the target cell first before parking the
remote CPU, to have the correct target VS-file in place.

This should not affect other architectures.

Signed-off-by: Ralf Ramsauer 
---
 hypervisor/control.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/hypervisor/control.c b/hypervisor/control.c
index 7a5304a0..38b9086c 100644
--- a/hypervisor/control.c
+++ b/hypervisor/control.c
@@ -368,10 +368,10 @@ static void cell_destroy_internal(struct cell *cell)
cell->comm_page.comm_region.cell_state = JAILHOUSE_CELL_SHUT_DOWN;
 
for_each_cpu(cpu, >cpu_set) {
+   public_per_cpu(cpu)->cell = _cell;
arch_park_cpu(cpu);
 
set_bit(cpu, root_cell.cpu_set.bitmap);
-   public_per_cpu(cpu)->cell = _cell;
public_per_cpu(cpu)->failed = false;
memset(public_per_cpu(cpu)->stats, 0,
   sizeof(public_per_cpu(cpu)->stats));
-- 
2.40.1

-- 
You received this message because you are subscribed to the Google Groups 
"Jailhouse" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jailhouse-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jailhouse-dev/20230519204033.643200-72-ralf.ramsauer%40oth-regensburg.de.

[PATCH v3 70/73] core: riscv: pci: Implement MSI-handling logic

[Ralf Ramsauer]

Similar to ARM systems.

Idea is as follows: If the MSI(-X) is masked, then allow any write to
the underlying register. If a MSI(-X) is unmasked, then check its target
address and add the offset of the vs-file.

The original value is stored in the shadow register, and will be
restored when jailhouse is disabled.

Signed-off-by: Ralf Ramsauer 
---
 hypervisor/arch/riscv/pci.c | 86 ++---
 1 file changed, 80 insertions(+), 6 deletions(-)

diff --git a/hypervisor/arch/riscv/pci.c b/hypervisor/arch/riscv/pci.c
index 067a7651..39e1e4cb 100644
--- a/hypervisor/arch/riscv/pci.c
+++ b/hypervisor/arch/riscv/pci.c
@@ -1,16 +1,20 @@
 /*
  * Jailhouse, a Linux-based partitioning hypervisor
  *
- * Copyright (c) Siemens AG, 2020
+ * Copyright (c) OTH Regensburg, 2023
  *
  * Authors:
- *  Jan Kiszka 
+ *  Ralf Ramsauer 
+ *  Stefan Huber 
  *
  * This work is licensed under the terms of the GNU GPL, version 2.  See
  * the COPYING file in the top-level directory.
  */
 
-#include 
+#include 
+#include 
+#include 
+#include 
 #include 
 
 u32 arch_pci_read_config(u16 bdf, u16 address, unsigned int size)
@@ -24,7 +28,7 @@ void arch_pci_write_config(u16 bdf, u16 address, u32 value, 
unsigned int size)
 
 int arch_pci_add_physical_device(struct cell *cell, struct pci_device *device)
 {
-   return -ENOSYS;
+   return 0;
 }
 
 void arch_pci_remove_physical_device(struct pci_device *device)
@@ -40,10 +44,80 @@ void arch_pci_set_suppress_msi(struct pci_device *device,
 int arch_pci_update_msi(struct pci_device *device,
const struct jailhouse_pci_capability *cap)
 {
-   return -ENOSYS;
+   const struct jailhouse_pci_device *info = device->info;
+   union pci_msi_registers target;
+   struct public_per_cpu *ppc;
+   unsigned short vs_file;
+   unsigned int cpu;
+   unsigned int n;
+
+   /* If the MSI is masked, allow to write any address */
+   target = device->msi_registers;
+   if (!device->msi_registers.msg32.enable)
+   goto passthru;
+
+   /* Only allow non-masked access, if vs-file is set */
+   vs_file = this_cell()->arch.vs_file;
+   if (!vs_file)
+   return -EINVAL;
+
+   for_each_cpu(cpu, _cell()->cpu_set) {
+   ppc = public_per_cpu(cpu);
+   /*
+* If the MSI is unmasked, only allow if the address is
+* on the S-Mode file and calculate the VS-mode offset.
+*/
+   if (ppc->imsic_base == device->msi_registers.msg64.address) {
+   target.msg64.address += vs_file * 0x1000;
+   goto passthru;
+   }
+   }
+
+   return -EINVAL;
+
+passthru:
+   for (n = 1; n < (info->msi_64bits ? 4 : 3); n++)
+   pci_write_config(info->bdf, cap->start + n * 4,
+target.raw[n], 4);
+
+   return 0;
 }
 
 int arch_pci_update_msix_vector(struct pci_device *device, unsigned int index)
 {
-   return -ENOSYS;
+   struct public_per_cpu *ppc;
+   unsigned short vs_file;
+   unsigned int cpu;
+   u64 vs_offset = 0;
+
+   vs_file = this_cell()->arch.vs_file;
+
+   /* If the MSI is masked, allow to write any address */
+   if (device->msix_vectors[index].masked)
+   goto passthru;
+
+   /* Only allow non-masked access, if vs-file is set */
+   if (!vs_file)
+   return -EINVAL;
+
+   /*
+* If the MSI is unmasked, only allow if the address is on the S-Mode
+* file and calculate the VS-mode offset.
+*/
+   for_each_cpu(cpu, _cell()->cpu_set) {
+   ppc = public_per_cpu(cpu);
+   if (ppc->imsic_base == device->msix_vectors[index].address) {
+   vs_offset = vs_file * 0x1000;
+   goto passthru;
+   }
+   }
+
+   return -EINVAL;
+
+passthru:
+   mmio_write64_split(>msix_table[index].address,
+  device->msix_vectors[index].address + vs_offset);
+   mmio_write32(>msix_table[index].data,
+device->msix_vectors[index].data);
+   return 0;
 }
-- 
2.40.1

-- 
You received this message because you are subscribed to the Google Groups 
"Jailhouse" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jailhouse-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jailhouse-dev/20230519204033.643200-71-ralf.ramsauer%40oth-regensburg.de.

[PATCH v3 69/73] core: riscv: aplic: Add IMSIC support

[Ralf Ramsauer]

With IMSIC, we achieve zero-traps for interrupts, if we don't have
level-triggered legacy IRQs. Latter one require some special handling
through the APLIC.

With this patch, we also have support for IVSHMEM interrupts: In case of
an IMSIC, we simply inject the pending irq on the target. Only one trap
is required: for sending the IRQ.

Signed-off-by: Ralf Ramsauer 
---
 hypervisor/arch/riscv/aplic.c   | 202 ++--
 hypervisor/arch/riscv/include/asm/cell.h|   1 +
 hypervisor/arch/riscv/include/asm/irqchip.h |   1 +
 hypervisor/arch/riscv/irqchip.c |   2 +
 hypervisor/arch/riscv/plic.c|   5 +
 hypervisor/arch/riscv/setup.c   |   8 +-
 6 files changed, 197 insertions(+), 22 deletions(-)

diff --git a/hypervisor/arch/riscv/aplic.c b/hypervisor/arch/riscv/aplic.c
index 1e6df407..240d6904 100644
--- a/hypervisor/arch/riscv/aplic.c
+++ b/hypervisor/arch/riscv/aplic.c
@@ -33,6 +33,13 @@
 
 #define TARGET_BASE0x3004
 #define TARGET_HART_SHIFT  18
+#define TARGET_GUEST_SHIFT 12
+#define TARGET_GUEST_MSK   BIT_MASK(17, 12)
+#define TARGET_PRIO_MSKBIT_MASK(7, 0)
+#define TARGET_EIID_MSKBIT_MASK(10, 0)
+
+#define IN_CLRIP_START 0x1d00
+#define IN_CLRIP_END   (0x1d7c + IRQCHIP_REG_SZ)
 
 #define SETIE_BASE 0x1e00
 #define SETIENUM   0x1edc
@@ -41,11 +48,15 @@
 #define CLRIE_END  (0x1f7c + IRQCHIP_REG_SZ)
 #define CLRIENUM   0x1fdc
 
+#define SETIPNUM_LE0x2000
+
 /* Per-Hart Interrupt Delivery Control (IDC) */
 #define IDC_BASE   0x4000
 #define IDC_SIZE   0x20
 #define  IDC_CLAIMI0x1c
 
+static void *imsic;
+
 /*
  * When destroying cells, legacy IRQs need to be assigned back to the root
  * cell. Therefore, we need to keep track of their previous state: The target,
@@ -134,7 +145,13 @@ static void aplic_adjust_irq_target(struct cell *cell, 
unsigned int irq)
 
/* We have to adjust the IRQ. Locate it on the first CPU of the cell */
hart_index = public_per_cpu(first_cpu(>cpu_set))->phys_id;
-   target = (hart_index << TARGET_HART_SHIFT) | (target & 0xff);
+
+   target = (hart_index << TARGET_HART_SHIFT);
+   if (imsic)
+   target |= (cell->arch.vs_file << TARGET_GUEST_SHIFT) |
+ (target & TARGET_EIID_MSK);
+   else
+   target |= (target & TARGET_PRIO_MSK); /* preserve priority */
 
aplic_write_target(irq, target);
 }
@@ -155,6 +172,9 @@ static int aplic_claim_irq(void)
unsigned int hart;
u32 claimi;
 
+   if (imsic)
+   return -EINVAL;
+
hart = phys_processor_id();
 
claimi = aplic_read_claimi(hart);
@@ -207,6 +227,9 @@ static inline enum mmio_result aplic_handle_idc(struct 
mmio_access *access)
unsigned long hart;
unsigned int cpu, idc, reg;
 
+   if (imsic)
+   return MMIO_ERROR;
+
idc = (access->address - IDC_BASE) / IDC_SIZE;
reg = access->address % IDC_SIZE;
 
@@ -356,29 +379,38 @@ aplic_handle_virq_target(struct mmio_access *access, 
unsigned int irq)
virq_target = >arch.aplic_virq.target[irq];
 
if (!access->is_write) {
-   access->value = public_per_cpu(*virq_target)->phys_id
-   << TARGET_HART_SHIFT;
+   access->value = public_per_cpu(*virq_target)->phys_id <<
+   TARGET_HART_SHIFT;
+   if (imsic)
+   access->value |= cell->arch.aplic_virq.eiid[irq];
return MMIO_HANDLED;
}
 
target = access->value >> TARGET_HART_SHIFT;
+
for_each_cpu(cpu, >cpu_set) {
hart = public_per_cpu(cpu)->phys_id;
-   if (hart == target) {
-   *virq_target = cpu;
-   return MMIO_HANDLED;
-   }
+   if (hart == target)
+   goto write_allowed;
}
 
return MMIO_ERROR;
+
+write_allowed:
+   *virq_target = cpu;
+   if (imsic) {
+   cell->arch.aplic_virq.eiid[irq] =
+   access->value & TARGET_EIID_MSK;
+   }
+
+   return MMIO_HANDLED;
 }
 
-/* Assumption: We're in direct delivery mode */
 static inline enum mmio_result aplic_handle_target(struct mmio_access *access)
 {
struct cell *cell = this_cell();
unsigned int irq;
-   u32 target;
+   u32 target, value;
 
/* Check if the source IRQ belongs to the cell */
irq = (access->address - TARGET_BASE) / IRQCHIP_REG_SZ + 1;
@@ -393,9 +425,12 @@ static inline enum mmio_result aplic_handle_target(struct 
mmio_access *access)
return MMIO_HANDLED;
}
 
-   /* If read, then pass throug

[PATCH v3 67/73] configs: riscv: add configs for APLIC+IMSIC

[Ralf Ramsauer]

Idea is as follows: Specify the target VS-file in the configuration, and
use simple memory regions to overlay the S-Mode file.

Signed-off-by: Ralf Ramsauer 
---
 .../dts/qemu-linux-inmate-aplic-imsic.dts | 124 ++
 configs/riscv/qemu-aplic-imsic-mc.c   |   5 +
 configs/riscv/qemu-aplic-imsic-uc.c   |   5 +
 configs/riscv/qemu-imsic.h|  46 +++
 configs/riscv/qemu-inmate-demo-aplic-imsic.c  |   3 +
 configs/riscv/qemu-inmate-demo.h  |   1 +
 configs/riscv/qemu-linux-demo-aplic-imsic.c   |   4 +
 configs/riscv/qemu-linux-demo.h   |  15 ++-
 configs/riscv/qemu.h  |  39 +-
 hypervisor/arch/riscv/include/asm/irqchip.h   |  15 +++
 include/jailhouse/cell-config.h   |   5 +
 pyjailhouse/config_parser.py  |   2 +-
 12 files changed, 259 insertions(+), 5 deletions(-)
 create mode 100644 configs/riscv/dts/qemu-linux-inmate-aplic-imsic.dts
 create mode 100644 configs/riscv/qemu-aplic-imsic-mc.c
 create mode 100644 configs/riscv/qemu-aplic-imsic-uc.c
 create mode 100644 configs/riscv/qemu-imsic.h
 create mode 100644 configs/riscv/qemu-inmate-demo-aplic-imsic.c
 create mode 100644 configs/riscv/qemu-linux-demo-aplic-imsic.c

diff --git a/configs/riscv/dts/qemu-linux-inmate-aplic-imsic.dts 
b/configs/riscv/dts/qemu-linux-inmate-aplic-imsic.dts
new file mode 100644
index ..462c784f
--- /dev/null
+++ b/configs/riscv/dts/qemu-linux-inmate-aplic-imsic.dts
@@ -0,0 +1,124 @@
+/*
+ * Jailhouse, a Linux-based partitioning hypervisor
+ *
+ * Copyright (c) OTH Regensburg, 2023
+ *
+ * Authors:
+ *  Ralf Ramsauer 
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2.  See
+ * the COPYING file in the top-level directory.
+ */
+
+/dts-v1/;
+/ {
+   model = "Jailhouse cell on RISC-V";
+
+   #address-cells = <2>;
+   #size-cells = <2>;
+
+   chosen {
+   stdout-path = "/soc/uart@1000:115200n8";
+   bootargs = "console=ttyS0 earlycon=sbi";
+   linux,initrd-start = <0x8200>;
+   linux,initrd-end = <0x8300>;
+   };
+
+   memory@8000 {
+   device_type = "memory";
+   /* 63MiB @ 0x8000 */
+   reg = <0x 0x8000>, <0x 0x800>;
+   };
+
+   cpus {
+   #address-cells = <1>;
+   #size-cells = <0>;
+   timebase-frequency = <0x989680>;
+
+   cpu0: cpu@0 {
+   device_type = "cpu";
+   reg = <0>;
+   status = "okay";
+   compatible = "riscv";
+   riscv,isa = 
"rv64imafdc_zicsr_zifencei_zihintpause_zawrs_zba_zbb_zbc_zbs_smaia_ssaia_sstc";
+   mmu-type = "riscv,sv39";
+
+   cpu0_intc: interrupt-controller {
+   #interrupt-cells = <1>;
+   compatible = "riscv,cpu-intc";
+   interrupt-controller;
+   };
+   };
+
+   cpu1: cpu@1 {
+   device_type = "cpu";
+   reg = <1>;
+   status = "okay";
+   compatible = "riscv";
+   riscv,isa = 
"rv64imafdc_zicsr_zifencei_zihintpause_zawrs_zba_zbb_zbc_zbs_smaia_ssaia_sstc";
+   mmu-type = "riscv,sv39";
+
+   cpu1_intc: interrupt-controller {
+   #interrupt-cells = <1>;
+   compatible = "riscv,cpu-intc";
+   interrupt-controller;
+   };
+   };
+   };
+   soc {
+   #address-cells = <2>;
+   #size-cells = <2>;
+   compatible = "simple-bus";
+   ranges;
+
+   uart@1000 {
+   interrupts = <10 4>;
+   interrupt-parent = <>;
+   clock-frequency = "\08@";
+   reg = <0x00 0x1000 0x00 0x100>;
+   compatible = "ns16550a";
+   status = "okay";
+   };
+
+   /* S-Mode APLIC */
+   aplic0: aplic@d00 {
+   riscv,num-sources = <0x60>;
+   reg = <0x00 0xd00 0x00 0x8000>;
+   msi-parent = <>;
+   interrupt-controller;
+   #interrupt-cells = <0x02>;
+   compatible = "risc

[PATCH v3 65/73] core: riscv: aplic: introduce vIRQ support

[Ralf Ramsauer]

Signed-off-by: Ralf Ramsauer 
---
 hypervisor/arch/riscv/aplic.c  | 147 -
 hypervisor/arch/riscv/include/asm/cell.h   |  12 ++
 hypervisor/arch/riscv/include/asm/percpu.h |   1 +
 3 files changed, 159 insertions(+), 1 deletion(-)

diff --git a/hypervisor/arch/riscv/aplic.c b/hypervisor/arch/riscv/aplic.c
index 8a181a1e..1e6df407 100644
--- a/hypervisor/arch/riscv/aplic.c
+++ b/hypervisor/arch/riscv/aplic.c
@@ -253,13 +253,27 @@ passthru:
 
 static inline enum mmio_result aplic_handle_ienum(struct mmio_access *access)
 {
+   struct cell *cell = this_cell();
+   unsigned int target;
+
/* Spec: A read always returns zero */
if (!access->is_write) {
access->value = 0;
return MMIO_HANDLED;
}
 
-   if (!irqchip_irq_in_cell(this_cell(), access->value))
+   /* Here we are in the write case */
+   if (irqchip_virq_in_cell(cell, access->value)) {
+   target = access->value - cell->config->vpci_irq_base;
+   if (access->address == SETIENUM)
+   cell->arch.aplic_virq.enabled |= (1 << target);
+   else
+   cell->arch.aplic_virq.enabled &= ~(1 << target);
+
+   return MMIO_HANDLED;
+   }
+
+   if (!irqchip_irq_in_cell(cell, access->value))
return MMIO_ERROR;
 
if (access->address == SETIENUM)
@@ -280,6 +294,19 @@ aplic_handle_sourcecfg(struct mmio_access *access)
/* Check if the source IRQ belongs to the cell */
irq = (access->address - SOURCECFG_BASE) / IRQCHIP_REG_SZ + 1;
 
+   if (irqchip_virq_in_cell(this_cell(), irq)) {
+   /*
+* Actually, we don't need sourcecfg for vIRQs at all. Just
+* emulate some 'sane' behaviour.
+*/
+   if (!access->is_write)
+   access->value = 6;
+   else if (access->value == 6 || !access->value)
+   return MMIO_HANDLED;
+
+   return MMIO_ERROR;
+   }
+
/* If not, simply ignore the access. */
if (!irqchip_irq_in_cell(this_cell(), irq)) {
if (!access->is_write)
@@ -317,6 +344,35 @@ static bool hart_in_cell(struct cell *cell, unsigned long 
hart)
return false;
 }
 
+static inline enum mmio_result
+aplic_handle_virq_target(struct mmio_access *access, unsigned int irq)
+{
+   struct cell *cell = this_cell();
+   unsigned int cpu, *virq_target;
+   unsigned long hart;
+   u32 target;
+
+   irq -= cell->config->vpci_irq_base;
+   virq_target = >arch.aplic_virq.target[irq];
+
+   if (!access->is_write) {
+   access->value = public_per_cpu(*virq_target)->phys_id
+   << TARGET_HART_SHIFT;
+   return MMIO_HANDLED;
+   }
+
+   target = access->value >> TARGET_HART_SHIFT;
+   for_each_cpu(cpu, >cpu_set) {
+   hart = public_per_cpu(cpu)->phys_id;
+   if (hart == target) {
+   *virq_target = cpu;
+   return MMIO_HANDLED;
+   }
+   }
+
+   return MMIO_ERROR;
+}
+
 /* Assumption: We're in direct delivery mode */
 static inline enum mmio_result aplic_handle_target(struct mmio_access *access)
 {
@@ -327,6 +383,9 @@ static inline enum mmio_result aplic_handle_target(struct 
mmio_access *access)
/* Check if the source IRQ belongs to the cell */
irq = (access->address - TARGET_BASE) / IRQCHIP_REG_SZ + 1;
 
+   if (irqchip_virq_in_cell(cell, irq))
+   return aplic_handle_virq_target(access, irq);
+
/* If not, simply ignore the access. */
if (!irqchip_irq_in_cell(cell, irq)) {
if (!access->is_write)
@@ -477,9 +536,95 @@ static int aplic_init(void)
return 0;
 }
 
+static void aplic_register_virq(struct cell *cell, unsigned int irq)
+{
+   unsigned int *virq_target;
+   unsigned int index;
+
+   index = irq - cell->config->vpci_irq_base;
+   if (index >= APLIC_MAX_VIRQ) {
+   printk("FATAL: aplic: too much vIRQs\n");
+   panic_stop();
+   }
+
+   spin_lock(>arch.virq_lock);
+   virq_target = >arch.aplic_virq.target[index];
+   if (!cell_owns_cpu(cell, *virq_target))
+   *virq_target = first_cpu(>cpu_set);
+
+   irq_bitmap_set(cell->arch.virq_present_bitmap, irq);
+   spin_unlock(>arch.virq_lock);
+}
+
+static void aplic_unregister_virq(struct cell *cell, unsigned int irq)
+{
+   unsigned int index, cpu;
+
+   index = irq - cell->config->vpci_irq_base;
+
+   spin_lock(>arch.virq_lock);
+   cell->arch.aplic_virq.enabled &= ~(1 << index);
+
+   for_each_cpu(cpu, >cpu_set)
+   publ

[PATCH v3 66/73] configs: riscv: add ivshmem-net devices

[Ralf Ramsauer]

For demonstration purposes only.

Signed-off-by: Ralf Ramsauer 
---
 configs/riscv/qemu-aplic-mc.c |  1 +
 configs/riscv/qemu-aplic-uc.c |  1 +
 configs/riscv/qemu-layout.h   |  1 -
 configs/riscv/qemu-linux-demo-aplic.c |  1 +
 configs/riscv/qemu-linux-demo-plic.c  |  1 +
 configs/riscv/qemu-linux-demo.h   | 32 ++-
 configs/riscv/qemu-plic-mc.c  |  1 +
 configs/riscv/qemu-plic-uc.c  |  1 +
 configs/riscv/qemu.h  | 32 ++-
 9 files changed, 68 insertions(+), 3 deletions(-)

diff --git a/configs/riscv/qemu-aplic-mc.c b/configs/riscv/qemu-aplic-mc.c
index 34c3bc9d..85077f00 100644
--- a/configs/riscv/qemu-aplic-mc.c
+++ b/configs/riscv/qemu-aplic-mc.c
@@ -1,3 +1,4 @@
 #define QEMU_MC
 #define QEMU_APLIC
+#define QEMU_IVSHMEM
 #include "qemu.h"
diff --git a/configs/riscv/qemu-aplic-uc.c b/configs/riscv/qemu-aplic-uc.c
index a26dcaa0..580cfe11 100644
--- a/configs/riscv/qemu-aplic-uc.c
+++ b/configs/riscv/qemu-aplic-uc.c
@@ -1,3 +1,4 @@
 #define QEMU_UC
 #define QEMU_APLIC
+#define QEMU_IVSHMEM
 #include "qemu.h"
diff --git a/configs/riscv/qemu-layout.h b/configs/riscv/qemu-layout.h
index 02e9ba51..6f9e6466 100644
--- a/configs/riscv/qemu-layout.h
+++ b/configs/riscv/qemu-layout.h
@@ -21,7 +21,6 @@
 #define HV_OFFSET  0x20 /* Recent OpenSBI place DTB there */
 #define HV_PHYS(MEM_TOP - HV_OFFSET - HV_SIZE)
 
-/* In preparation for IVSHMEM */
 #define IVSHMEM_NET_PHYS   (HV_PHYS - MIB)
 
 #define INMATE_TINY_SIZE   MIB
diff --git a/configs/riscv/qemu-linux-demo-aplic.c 
b/configs/riscv/qemu-linux-demo-aplic.c
index 81442a13..f8711402 100644
--- a/configs/riscv/qemu-linux-demo-aplic.c
+++ b/configs/riscv/qemu-linux-demo-aplic.c
@@ -1,2 +1,3 @@
 #define QEMU_APLIC
+#define QEMU_IVSHMEM
 #include "qemu-linux-demo.h"
diff --git a/configs/riscv/qemu-linux-demo-plic.c 
b/configs/riscv/qemu-linux-demo-plic.c
index f4c0331a..608efff0 100644
--- a/configs/riscv/qemu-linux-demo-plic.c
+++ b/configs/riscv/qemu-linux-demo-plic.c
@@ -1,2 +1,3 @@
 #define QEMU_PLIC
+#define QEMU_IVSHMEM
 #include "qemu-linux-demo.h"
diff --git a/configs/riscv/qemu-linux-demo.h b/configs/riscv/qemu-linux-demo.h
index 68575dc6..12acc2fe 100644
--- a/configs/riscv/qemu-linux-demo.h
+++ b/configs/riscv/qemu-linux-demo.h
@@ -16,12 +16,26 @@
 #include 
 #include "qemu-layout.h"
 
+#define MEM_REGIONS_BASE   4
+
+#ifdef QEMU_IVSHMEM
+#define IVSHMEM_REGIONS4
+#else
+#define IVSHMEM_REGIONS0
+#endif
+
+#define MEM_REGIONS(MEM_REGIONS_BASE + IVSHMEM_REGIONS)
+
 struct {
struct jailhouse_cell_desc cell;
struct jailhouse_cpu cpus[2];
-   struct jailhouse_memory mem_regions[4];
+   struct jailhouse_memory mem_regions[MEM_REGIONS];
struct jailhouse_irqchip irqchips[1];
+#ifdef QEMU_IVSHMEM
+   struct jailhouse_pci_device pci_devices[1];
+#else
struct jailhouse_pci_device pci_devices[0];
+#endif
 } __attribute__((packed)) config = {
.cell = {
.signature = JAILHOUSE_CELL_DESC_SIGNATURE,
@@ -57,6 +71,10 @@ struct {
},
 
.mem_regions = {
+#ifdef QEMU_IVSHMEM
+   /* IVSHMEM shared memory regions (networking) */
+   JAILHOUSE_SHMEM_NET_REGIONS(IVSHMEM_NET_PHYS, 1),
+#endif
/* RAM low */ {
.phys_start = INMATE_LINUX_LOW_PHYS,
.virt_start = 0x0,
@@ -87,6 +105,18 @@ struct {
},
 
.pci_devices = {
+#ifdef QEMU_IVSHMEM
+   { /* IVSHMEM (networking) */
+   .type = JAILHOUSE_PCI_TYPE_IVSHMEM,
+   .domain = 0x,
+   .bdf = 0x10 << 3,
+   .bar_mask = JAILHOUSE_IVSHMEM_BAR_MASK_INTX,
+   .shmem_regions_start = 0,
+   .shmem_dev_id = 1,
+   .shmem_peers = 2,
+   .shmem_protocol = JAILHOUSE_SHMEM_PROTO_VETH,
+   },
+#endif
},
 
.irqchips = {
diff --git a/configs/riscv/qemu-plic-mc.c b/configs/riscv/qemu-plic-mc.c
index c29be90a..26c807d7 100644
--- a/configs/riscv/qemu-plic-mc.c
+++ b/configs/riscv/qemu-plic-mc.c
@@ -1,3 +1,4 @@
 #define QEMU_MC
 #define QEMU_PLIC
+#define QEMU_IVSHMEM
 #include "qemu.h"
diff --git a/configs/riscv/qemu-plic-uc.c b/configs/riscv/qemu-plic-uc.c
index f9842775..c6fa7339 100644
--- a/configs/riscv/qemu-plic-uc.c
+++ b/configs/riscv/qemu-plic-uc.c
@@ -1,3 +1,4 @@
 #define QEMU_UC
 #define QEMU_PLIC
+#define QEMU_IVSHMEM
 #include "qemu.h"
diff --git a/configs/riscv/qemu.h b/configs/riscv/qemu.h
index ceae015f..6f481769 100644
--- a/configs/riscv/qemu.h
+++ b/configs/riscv/qemu.h
@@ -21,6 +21,16 @@
 #include 
 #include "qemu-layout.h"
 
+#define MEM_REGIONS_BASE   

[PATCH v3 63/73] core: riscv: implement APLIC support

[Ralf Ramsauer]

This adds APLIC support for wired IRQs. For wired IRQs, the situation is
basically the same as with the APLIC: We have to fully moderate access,
due to the memory layout.

Major differences to the PLIC:
  - An IRQ can only be routed to a single HART (PLIC supported HART
target masks)
  - IRQs are only claimed and NOT acknowledged
  - Global IRQ disabling is not supported (via DOMAINCFG). But we don't
really need it.

So let's do the same as with the PLIC and softemulate it.

The situation will get better, once we add support for MSIs: The with
IMSIC-support, the APLIC can present all wired IRQs as MSIs, and also
supports 'virtual MSIs' that can for example be used to model IPIs.

With the IMSIC, situation will get better for us.

Signed-off-by: Ralf Ramsauer 
[since v2: inline single-caller functions, add shadow registers for
enabled, target and sourcecfg]
---
 hypervisor/arch/riscv/Kbuild|   4 +-
 hypervisor/arch/riscv/aplic.c   | 485 
 hypervisor/arch/riscv/irqchip.c |   6 +-
 3 files changed, 492 insertions(+), 3 deletions(-)
 create mode 100644 hypervisor/arch/riscv/aplic.c

diff --git a/hypervisor/arch/riscv/Kbuild b/hypervisor/arch/riscv/Kbuild
index 0e7d3719..e8f0eb9f 100644
--- a/hypervisor/arch/riscv/Kbuild
+++ b/hypervisor/arch/riscv/Kbuild
@@ -14,5 +14,5 @@
 
 always-y := lib.a
 
-lib-y := entry.o exception.o setup.o dbg-write.o control.o ivshmem.o irqchip.o
-lib-y += paging.o pci.o plic.o traps.o lib.o
+lib-y := aplic.o entry.o exception.o setup.o dbg-write.o control.o ivshmem.o
+lib-y += irqchip.o paging.o pci.o plic.o traps.o lib.o
diff --git a/hypervisor/arch/riscv/aplic.c b/hypervisor/arch/riscv/aplic.c
new file mode 100644
index ..8a181a1e
--- /dev/null
+++ b/hypervisor/arch/riscv/aplic.c
@@ -0,0 +1,485 @@
+/*
+ * Jailhouse, a Linux-based partitioning hypervisor
+ *
+ * Copyright (c) OTH Regensburg, 2022-2023
+ *
+ * Authors:
+ *  Ralf Ramsauer 
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2.  See
+ * the COPYING file in the top-level directory.
+ */
+
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+
+#define REGISTER(X) REG_RANGE((X), (X) + IRQCHIP_REG_SZ)
+
+#define DOMAINCFG  0x0
+#define  DOMAINCFG_DM  2
+#define  DOMAINCFG_BE  0
+#define  DOMAINCFG_IE  8
+#define  DOMAINCFG_DEFAULT ((1 << 31) | (1 << DOMAINCFG_IE))
+
+#define SOURCECFG_BASE 0x4
+#define  SOURCECFG_D   10
+#define SOURCECFG_END  0x1000
+
+#define TARGET_BASE0x3004
+#define TARGET_HART_SHIFT  18
+
+#define SETIE_BASE 0x1e00
+#define SETIENUM   0x1edc
+
+#define CLRIE_START0x1f00
+#define CLRIE_END  (0x1f7c + IRQCHIP_REG_SZ)
+#define CLRIENUM   0x1fdc
+
+/* Per-Hart Interrupt Delivery Control (IDC) */
+#define IDC_BASE   0x4000
+#define IDC_SIZE   0x20
+#define  IDC_CLAIMI0x1c
+
+/*
+ * When destroying cells, legacy IRQs need to be assigned back to the root
+ * cell. Therefore, we need to keep track of their previous state: The target,
+ * source configuration and enable state.
+ */
+static struct {
+   u32 sourcecfg[MAX_IRQS];
+   u32 target[MAX_IRQS];
+   u32 enabled[MAX_IRQS / (sizeof(u32) * 8)];
+} shadow;
+
+static inline u32 aplic_read(u32 reg)
+{
+   return mmio_read32(irqchip.base + reg);
+}
+
+static inline void aplic_write(u32 reg, u32 value)
+{
+   mmio_write32(irqchip.base + reg, value);
+}
+
+static inline u32 aplic_read_setie(unsigned short no)
+{
+   return aplic_read(SETIE_BASE + no * IRQCHIP_REG_SZ);
+}
+
+static inline bool aplic_irq_is_enabled(unsigned int irq)
+{
+   u32 en = aplic_read_setie(irq / IRQCHIP_BITS_PER_REG);
+
+   return !!(en & IRQ_MASK(irq));
+}
+
+static inline u32 aplic_read_target(unsigned int irq)
+{
+   return aplic_read(TARGET_BASE + (irq - 1) * IRQCHIP_REG_SZ);
+}
+
+static inline void aplic_write_target(unsigned int irq, u32 val)
+{
+   aplic_write(TARGET_BASE + (irq - 1) * IRQCHIP_REG_SZ, val);
+}
+
+static inline u32 aplic_read_sourcecfg(unsigned int irq)
+{
+   return aplic_read(SOURCECFG_BASE + (irq - 1) * IRQCHIP_REG_SZ);
+}
+
+static inline void aplic_write_sourcecfg(unsigned int irq, u32 val)
+{
+   aplic_write(SOURCECFG_BASE + (irq - 1) * IRQCHIP_REG_SZ, val);
+}
+
+static inline void aplic_write_clrienum(unsigned int irq)
+{
+   aplic_write(CLRIENUM, irq);
+}
+
+static void aplic_adjust_irq_target(struct cell *cell, unsigned int irq)
+{
+   u32 target, hart_index;
+   unsigned int cpu;
+
+   if (cell == _cell) {
+   aplic_write_sourcecfg(irq, shadow.sourcecfg[irq]);
+   aplic_write_target(irq, shadow.target[irq]);
+   if (irq_bitmap_test(shadow.enabled, irq))
+   aplic_write(SETIENUM, irq);
+   else
+   apli

[PATCH v3 64/73] core: riscv: plic: introduce vIRQ support

[Ralf Ramsauer]

For demonstration purposes only.

Signed-off-by: Ralf Ramsauer 
---
 hypervisor/arch/riscv/control.c |   2 +
 hypervisor/arch/riscv/include/asm/cell.h|   3 +
 hypervisor/arch/riscv/include/asm/irqchip.h |  16 +++
 hypervisor/arch/riscv/include/asm/ivshmem.h |   1 +
 hypervisor/arch/riscv/include/asm/percpu.h  |   6 +-
 hypervisor/arch/riscv/irqchip.c |  54 
 hypervisor/arch/riscv/ivshmem.c |  38 +-
 hypervisor/arch/riscv/plic.c| 134 ++--
 hypervisor/arch/riscv/traps.c   |   1 +
 9 files changed, 244 insertions(+), 11 deletions(-)

diff --git a/hypervisor/arch/riscv/control.c b/hypervisor/arch/riscv/control.c
index cc3ebcb2..fbd8e81d 100644
--- a/hypervisor/arch/riscv/control.c
+++ b/hypervisor/arch/riscv/control.c
@@ -141,6 +141,8 @@ int arch_cell_create(struct cell *const cell)
ppc->reset = false;
}
 
+   spin_init(>arch.virq_lock);
+
if (!cell->arch.mm.root_table)
return -ENOMEM;
 
diff --git a/hypervisor/arch/riscv/include/asm/cell.h 
b/hypervisor/arch/riscv/include/asm/cell.h
index 9da228d4..ee3fd892 100644
--- a/hypervisor/arch/riscv/include/asm/cell.h
+++ b/hypervisor/arch/riscv/include/asm/cell.h
@@ -17,11 +17,14 @@
 
 #include 
 #include 
+#include 
 
 struct arch_cell {
struct paging_structures mm;
 
u32 irq_bitmap[MAX_IRQS / (sizeof(u32) * 8)];
+   u32 virq_present_bitmap[MAX_IRQS / (sizeof(u32) * 8)];
+   spinlock_t virq_lock;
 };
 
 #endif /* !_JAILHOUSE_ASM_CELL_H */
diff --git a/hypervisor/arch/riscv/include/asm/irqchip.h 
b/hypervisor/arch/riscv/include/asm/irqchip.h
index 5183f5de..6afdac5c 100644
--- a/hypervisor/arch/riscv/include/asm/irqchip.h
+++ b/hypervisor/arch/riscv/include/asm/irqchip.h
@@ -29,6 +29,11 @@ struct irqchip {
int (*claim_irq)(void);
void (*adjust_irq_target)(struct cell *cell, unsigned int irq);
 
+   void (*send_virq)(struct cell *cell, unsigned int irq);
+   void (*register_virq)(struct cell *cell, unsigned int irq);
+   void (*unregister_virq)(struct cell *cell, unsigned int irq);
+   bool (*inject_pending_virqs)(void);
+
void *base;
unsigned long pending[MAX_CPUS];
 };
@@ -82,6 +87,17 @@ static inline bool irqchip_irq_in_cell(struct cell *cell, 
unsigned int irq)
return irq_bitmap_test(cell->arch.irq_bitmap, irq);
 }
 
+static inline bool irqchip_virq_in_cell(struct cell *cell, unsigned int irq)
+{
+   return irq_bitmap_test(cell->arch.virq_present_bitmap, irq);
+}
+
 int irqchip_set_pending(void);
 
+void irqchip_register_virq(unsigned int irq);
+void irqchip_unregister_virq(unsigned int irq);
+void irqchip_send_virq(struct cell *cell, unsigned int irq);
+void irqchip_process_pending_virqs(void);
+bool irqchip_inject_pending_virqs(void);
+
 #endif /* __ASSEMBLY__ */
diff --git a/hypervisor/arch/riscv/include/asm/ivshmem.h 
b/hypervisor/arch/riscv/include/asm/ivshmem.h
index 03251590..8b193947 100644
--- a/hypervisor/arch/riscv/include/asm/ivshmem.h
+++ b/hypervisor/arch/riscv/include/asm/ivshmem.h
@@ -11,4 +11,5 @@
  */
 
 struct arch_ivshmem_irq_cache {
+   u16 id[IVSHMEM_MSIX_VECTORS];
 };
diff --git a/hypervisor/arch/riscv/include/asm/percpu.h 
b/hypervisor/arch/riscv/include/asm/percpu.h
index f57e6d57..c6dd8cb2 100644
--- a/hypervisor/arch/riscv/include/asm/percpu.h
+++ b/hypervisor/arch/riscv/include/asm/percpu.h
@@ -42,6 +42,10 @@ enum sbi_hart_state {
} hsm;  \
bool wait_for_power_on; \
bool reset; \
-   bool park;
+   bool park;  \
+   struct {\
+   u32 enabled_bitmap[MAX_IRQS / (sizeof(u32) * 8)];   \
+   u32 pending_bitmap[MAX_IRQS / (sizeof(u32) * 8)];   \
+   } virq;
 
 #define ARCH_PERCPU_FIELDS
diff --git a/hypervisor/arch/riscv/irqchip.c b/hypervisor/arch/riscv/irqchip.c
index 693c37bf..9834dcee 100644
--- a/hypervisor/arch/riscv/irqchip.c
+++ b/hypervisor/arch/riscv/irqchip.c
@@ -67,6 +67,8 @@ static int irqchip_cell_init(struct cell *cell)
 irqchip_mmio, cell);
 
memset(cell->arch.irq_bitmap, 0, sizeof(cell->arch.irq_bitmap));
+   memset(cell->arch.virq_present_bitmap, 0,
+  sizeof(cell->arch.virq_present_bitmap));
 
for_each_irqchip(chip, cell->config, n) {
/* Only support one single PLIC at the moment */
@@ -188,4 +190,56 @@ static void irqchip_cell_exit(struct cell *cell)
chip->pin_bitmap[pos];
 }
 
+void irqchip_send_virq(struct cell *cell, unsigned int irq)
+{
+   //printk("sending vIRQ %u from %s to %s\n", irq, 
th

[PATCH v3 60/73] core: riscv: PLIC: remove workaround

[Ralf Ramsauer]

This bug went away with linux 6.0

Signed-off-by: Ralf Ramsauer 
---
 hypervisor/arch/riscv/plic.c | 32 ++--
 1 file changed, 6 insertions(+), 26 deletions(-)

diff --git a/hypervisor/arch/riscv/plic.c b/hypervisor/arch/riscv/plic.c
index a65de82c..4f4b64fc 100644
--- a/hypervisor/arch/riscv/plic.c
+++ b/hypervisor/arch/riscv/plic.c
@@ -224,8 +224,13 @@ static enum mmio_result plic_handle_prio(struct 
mmio_access *access)
 
irq = access->address / IRQCHIP_REG_SZ;
 
+   /*
+* When booting non-root Linux, it will set priorities of all IRQs.
+* Hence, simply ignore non-allowed writes instead of crashing the
+* cell.
+*/
if (!irqchip_irq_in_cell(this_cell(), irq))
-   return MMIO_ERROR;
+   return MMIO_HANDLED;
 
/*
 * Maybe we can abandon this check. The cell should know the max
@@ -254,32 +259,7 @@ static enum mmio_result plic_handle_enable(struct 
mmio_access *access)
goto allowed;
}
 
-   /*
-* FIXME: Why does Linux read non-allowed ctxs? This seems to be an
-* actual bug in Linux. When we remove a CPU from Linux, and we later
-* change the affinity of the IRQ, then Linux will try to access
-* Contexts which it is not in charge of any longer. While Linux
-* disables IRQs, it does not adjust smp_affinities when removing CPUs.
-*
-* For the moment, and as a workaround, simply report any read as 0,
-* and forbid writes != 0.
-*
-* ... Okay, we really have a Linux bug here.
-*  (a) Linux doesn't remove the affinity from removed CPUs
-*  (b) Linux allows to set affinity to non-present CPUs
-*
-* Actually, we should always return MMIO_ERROR here.
-*/
-
-#if 1
-   if (!access->is_write) {
-   access->value = 0;
-   } else if (access->value != 0)
-   return MMIO_ERROR;
-   return MMIO_HANDLED;
-#else
return MMIO_ERROR;
-#endif
 
 allowed:
/*
-- 
2.40.1

-- 
You received this message because you are subscribed to the Google Groups 
"Jailhouse" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jailhouse-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jailhouse-dev/20230519204033.643200-61-ralf.ramsauer%40oth-regensburg.de.

[PATCH v3 62/73] configs: riscv: Add configuration parameters for APLIC

[Ralf Ramsauer]

From: Stefan Huber 

With this, introduce the APLIC irqchip type in cell-config.h.

Signed-off-by: Ralf Ramsauer 
---
 configs/riscv/dts/qemu-linux-inmate-aplic.dts | 111 ++
 configs/riscv/qemu-aplic-mc.c |   3 +
 configs/riscv/qemu-aplic-uc.c |   3 +
 configs/riscv/qemu-linux-demo-aplic.c |   2 +
 configs/riscv/qemu-linux-demo.h   |   4 +-
 configs/riscv/qemu-plic-mc.c  |   1 +
 configs/riscv/qemu-plic-uc.c  |   1 +
 configs/riscv/qemu.h  |  17 ++-
 include/jailhouse/cell-config.h   |   1 +
 9 files changed, 140 insertions(+), 3 deletions(-)
 create mode 100644 configs/riscv/dts/qemu-linux-inmate-aplic.dts
 create mode 100644 configs/riscv/qemu-aplic-mc.c
 create mode 100644 configs/riscv/qemu-aplic-uc.c
 create mode 100644 configs/riscv/qemu-linux-demo-aplic.c

diff --git a/configs/riscv/dts/qemu-linux-inmate-aplic.dts 
b/configs/riscv/dts/qemu-linux-inmate-aplic.dts
new file mode 100644
index ..49eb3993
--- /dev/null
+++ b/configs/riscv/dts/qemu-linux-inmate-aplic.dts
@@ -0,0 +1,111 @@
+/*
+ * Jailhouse, a Linux-based partitioning hypervisor
+ *
+ * Copyright (c) OTH Regensburg, 2022
+ *
+ * Authors:
+ *  Ralf Ramsauer 
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2.  See
+ * the COPYING file in the top-level directory.
+ */
+
+/dts-v1/;
+/ {
+   model = "Jailhouse cell on RISC-V";
+
+   #address-cells = <2>;
+   #size-cells = <2>;
+
+   chosen {
+   stdout-path = "/soc/uart@1000:115200n8";
+   bootargs = "console=ttyS0 earlycon=sbi";
+   linux,initrd-start = <0x8200>;
+   linux,initrd-end = <0x8300>;
+   };
+
+   memory@8000 {
+   device_type = "memory";
+   /* 63MiB @ 0x8000 */
+   reg = <0x 0x8000>, <0x 0x800>;
+   };
+
+   cpus {
+   #address-cells = <1>;
+   #size-cells = <0>;
+   timebase-frequency = <0x989680>;
+
+   cpu0: cpu@0 {
+   device_type = "cpu";
+   reg = <0>;
+   status = "okay";
+   compatible = "riscv";
+   riscv,isa = 
"rv64imafdc_zicsr_zifencei_zihintpause_zawrs_zba_zbb_zbc_zbs_sstc";
+   mmu-type = "riscv,sv39";
+
+   cpu0_intc: interrupt-controller {
+   #interrupt-cells = <1>;
+   compatible = "riscv,cpu-intc";
+   interrupt-controller;
+   };
+   };
+
+   cpu1: cpu@1 {
+   device_type = "cpu";
+   reg = <1>;
+   status = "okay";
+   compatible = "riscv";
+   riscv,isa = 
"rv64imafdc_zicsr_zifencei_zihintpause_zawrs_zba_zbb_zbc_zbs_sstc";
+   mmu-type = "riscv,sv39";
+
+   cpu1_intc: interrupt-controller {
+   #interrupt-cells = <1>;
+   compatible = "riscv,cpu-intc";
+   interrupt-controller;
+   };
+   };
+   };
+   soc {
+   #address-cells = <2>;
+   #size-cells = <2>;
+   compatible = "simple-bus";
+   ranges;
+
+   uart@1000 {
+   interrupts = <10 4>;
+   interrupt-parent = <>;
+   clock-frequency = "\08@";
+   reg = <0x00 0x1000 0x00 0x100>;
+   compatible = "ns16550a";
+   status = "okay";
+   };
+
+   /* S-Mode APLIC */
+   aplic0: aplic@d00 {
+   riscv,num-sources = <0x60>;
+   reg = <0x00 0xd00 0x00 0x8000>;
+   interrupts-extended = <_intc 0x09>,
+ <_intc 0x09>;
+   interrupt-controller;
+   #interrupt-cells = <0x02>;
+   compatible = "riscv,aplic";
+   };
+
+   pci@3000 {
+   compatible = "pci-host-ecam-generic";
+   device_type = "pci";
+   bus-range = <0 0>;
+   #interrupt-cells = <1>;
+  

[PATCH v3 61/73] driver: riscv: Add vPCI support

[Ralf Ramsauer]

For demonstration purposes only.

Signed-off-by: Ralf Ramsauer 
---
 driver/pci.c | 33 +
 1 file changed, 21 insertions(+), 12 deletions(-)

diff --git a/driver/pci.c b/driver/pci.c
index da516479..4f52eba8 100644
--- a/driver/pci.c
+++ b/driver/pci.c
@@ -292,19 +292,21 @@ static unsigned int count_ivshmem_devices(struct cell 
*cell)
return count;
 }
 
-static const struct of_device_id gic_of_match[] = {
+static const struct of_device_id irqchip_of_match[] = {
{ .compatible = "arm,cortex-a15-gic", },
{ .compatible = "arm,cortex-a7-gic", },
{ .compatible = "arm,gic-400", },
{ .compatible = "arm,gic-v3", },
+   { .compatible = "riscv,plic0", },
+   { .compatible = "sifive,plic-1.0.0", },
{},
 };
 
 static bool create_vpci_of_overlay(struct jailhouse_system *config)
 {
-   u32 address_cells, size_cells, gic_address_cells, gic_phandle;
+   u32 address_cells, size_cells, irqchip_address_cells, irqchip_phandle;
struct device_node *vpci_node = NULL;
-   struct device_node *root, *gic;
+   struct device_node *root, *irqchip;
struct property *prop = NULL;
unsigned int n, cell;
u64 base_addr;
@@ -322,15 +324,16 @@ static bool create_vpci_of_overlay(struct 
jailhouse_system *config)
 
of_node_put(root);
 
-   gic = of_find_matching_node(NULL, gic_of_match);
-   if (!gic)
+   irqchip= of_find_matching_node(NULL, irqchip_of_match);
+   if (!irqchip)
return false;
 
-   if (of_property_read_u32(gic, "#address-cells", _address_cells) < 0)
-   gic_address_cells = 0;
-   gic_phandle = gic->phandle;
+   if (of_property_read_u32(irqchip, "#address-cells",
+_address_cells) < 0)
+   irqchip_address_cells = 0;
+   irqchip_phandle = irqchip->phandle;
 
-   of_node_put(gic);
+   of_node_put(irqchip);
 
 #if LINUX_VERSION_CODE >= KERNEL_VERSION(4,17,0)
if (of_overlay_fdt_apply(__dtb_vpci_template_begin,
@@ -386,7 +389,7 @@ static bool create_vpci_of_overlay(struct jailhouse_system 
*config)
}
 
prop = alloc_prop("interrupt-map",
- sizeof(u32) * (8 + gic_address_cells) * 4);
+ sizeof(u32) * (8 + irqchip_address_cells) * 4);
if (!prop)
goto out;
 
@@ -394,12 +397,18 @@ static bool create_vpci_of_overlay(struct 
jailhouse_system *config)
for (n = 0, cell = 0; n < 4; n++) {
cell += 3;  /* match addr (0) */
prop_val[cell++] = cpu_to_be32(n + 1);  /* match addr */
-   prop_val[cell++] = cpu_to_be32(gic_phandle);
-   cell += gic_address_cells;  /* parent addr (0) */
+   prop_val[cell++] = cpu_to_be32(irqchip_phandle);
+   cell += irqchip_address_cells;  /* parent addr (0) */
+#if defined(CONFIG_ARM) || defined(CONFIG_ARM64)
prop_val[cell++] = cpu_to_be32(GIC_SPI);
prop_val[cell++] =
cpu_to_be32(config->root_cell.vpci_irq_base + n);
prop_val[cell++] = cpu_to_be32(IRQ_TYPE_EDGE_RISING);
+#elif defined(CONFIG_RISCV)
+   prop_val[cell++] =
+   cpu_to_be32(config->root_cell.vpci_irq_base + n);
+   prop_val[cell++] = 0;
+#endif
}
 
if (of_changeset_add_property(_changeset, vpci_node, prop) < 0)
-- 
2.40.1

-- 
You received this message because you are subscribed to the Google Groups 
"Jailhouse" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jailhouse-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jailhouse-dev/20230519204033.643200-62-ralf.ramsauer%40oth-regensburg.de.

[PATCH v3 52/73] inmates: riscv: implement tiny-demo RISC-V

[Ralf Ramsauer]

From: Stefan Huber 

add necessary files for developing tiny-demo inmate. Consists of
modified copies from x86/arm.

Signed-off-by: Stefan Huber 
Signed-off-by: Ralf Ramsauer 
---
 .gitignore  |   1 +
 inmates/demos/riscv/Makefile|  19 
 inmates/demos/riscv/tiny-demo.c |  19 
 inmates/lib/include/inmate_common.h |   1 +
 inmates/lib/riscv/Makefile  |  46 
 inmates/lib/riscv/Makefile.lib  |  64 
 inmates/lib/riscv/header.S  |  44 
 inmates/lib/riscv/include/inmate.h  | 157 
 inmates/lib/riscv/inmate.lds.S  |  74 +
 inmates/lib/riscv/printk.c  |  61 +++
 inmates/lib/riscv/setup.c   |  43 
 inmates/lib/riscv/uart.c|  47 +
 12 files changed, 576 insertions(+)
 create mode 100644 inmates/demos/riscv/tiny-demo.c
 create mode 100644 inmates/lib/riscv/Makefile.lib
 create mode 100644 inmates/lib/riscv/header.S
 create mode 100644 inmates/lib/riscv/include/inmate.h
 create mode 100644 inmates/lib/riscv/inmate.lds.S
 create mode 100644 inmates/lib/riscv/printk.c
 create mode 100644 inmates/lib/riscv/setup.c
 create mode 100644 inmates/lib/riscv/uart.c

diff --git a/.gitignore b/.gitignore
index 245733cb..80224289 100644
--- a/.gitignore
+++ b/.gitignore
@@ -20,6 +20,7 @@ include/jailhouse/config.h
 hypervisor/hypervisor.lds
 inmates/lib/arm/inmate.lds
 inmates/lib/arm64/inmate.lds
+inmates/lib/riscv/inmate.lds
 pyjailhouse/pci_defs.py
 tools/demos/cache-timings
 tools/demos/ivshmem-demo
diff --git a/inmates/demos/riscv/Makefile b/inmates/demos/riscv/Makefile
index e69de29b..2f0bac84 100644
--- a/inmates/demos/riscv/Makefile
+++ b/inmates/demos/riscv/Makefile
@@ -0,0 +1,19 @@
+#
+# Jailhouse, a Linux-based partitioning hypervisor
+#
+# Copyright (c) Siemens AG, 2013, 2014
+#
+# Authors:
+#  Jan Kiszka 
+#
+# This work is licensed under the terms of the GNU GPL, version 2.  See
+# the COPYING file in the top-level directory.
+#
+
+include $(INMATES_LIB)/Makefile.lib
+
+INMATES := tiny-demo.bin
+
+tiny-demo-y:= tiny-demo.o
+
+$(eval $(call DECLARE_TARGETS,$(INMATES)))
diff --git a/inmates/demos/riscv/tiny-demo.c b/inmates/demos/riscv/tiny-demo.c
new file mode 100644
index ..e4af2642
--- /dev/null
+++ b/inmates/demos/riscv/tiny-demo.c
@@ -0,0 +1,19 @@
+/*
+ * Jailhouse, a Linux-based partitioning hypervisor
+ *
+ * Copyright (c) OTH Regensburg, 2022
+ *
+ * Authors:
+ *  Ralf Ramsauer 
+ *  Stefan Huber 
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2.  See
+ * the COPYING file in the top-level directory.
+ */
+
+#include 
+
+void inmate_main(void)
+{
+   printk("Hello from this tiny cell!\n");
+}
diff --git a/inmates/lib/include/inmate_common.h 
b/inmates/lib/include/inmate_common.h
index 1c20a0af..8f474264 100644
--- a/inmates/lib/include/inmate_common.h
+++ b/inmates/lib/include/inmate_common.h
@@ -72,6 +72,7 @@
 #define MSIX_CTRL_FMASK0x4000
 
 #ifndef __ASSEMBLY__
+
 typedef s8 __s8;
 typedef u8 __u8;
 
diff --git a/inmates/lib/riscv/Makefile b/inmates/lib/riscv/Makefile
index e69de29b..9ff57721 100644
--- a/inmates/lib/riscv/Makefile
+++ b/inmates/lib/riscv/Makefile
@@ -0,0 +1,46 @@
+#
+# Jailhouse, a Linux-based partitioning hypervisor
+#
+# Copyright (c) Siemens AG, 2015, 2016
+#
+# Authors:
+#  Jan Kiszka 
+#
+# This work is licensed under the terms of the GNU GPL, version 2.  See
+# the COPYING file in the top-level directory.
+#
+# Alternatively, you can use or redistribute this file under the following
+# BSD license:
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions
+# are met:
+#
+# 1. Redistributions of source code must retain the above copyright
+#notice, this list of conditions and the following disclaimer.
+#
+# 2. Redistributions in binary form must reproduce the above copyright
+#notice, this list of conditions and the following disclaimer in the
+#documentation and/or other materials provided with the distribution.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
+# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+# ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE
+# LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+# CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+# SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+# INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+# CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+# ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF
+# THE POSSIBILITY OF SUCH DAMAGE.
+#
+
+include $(INMATES_LIB)/Makefile.lib

[PATCH v3 59/73] core: riscv: add SSTC support

[Ralf Ramsauer]

SSTC (Supervisor-level timecmp) is an extension to bypass SBI calls to
control the platform's timer. If SSTC is available, the supervisor can
directly write the next timeout into the stimecmp CSR, instead of
conducting a SBI call. Furthermore, platform timer IRQs will directly
arrive at S-Mode resp. VS-Mode w/o M-Mode reinjection, which saves time
up on timer arrivals.

SSTC supports virtualisation: VS-Mode's stimecmp is shadowed by
vstimecmp. This means, if SSTC is available, we can save up to two
context switches for arming timers.

So we have drastically fewer exits with SSTC.

Linux (since v6.0) will discover and enable SSTC during the
initialisation of the platform timer, and from then on use the stimecmp
CSR for arming the timer instead of calling SBI. Actually pretty easy.

Some important notes from the specification [1]:

  - Discovery of SSTC is done via misa CSR, or via device tree. From the
hypervisor perspective, we can also discover it via the HENVCFG CSR
register, as the STCE Enable bit is a WARL bit. That makes discovery
easy.

  - Spec says that the S-Mode hypervisor should write to vstimecmp if
the VS_Mode guest decides to csr_write(stimecmp), as this saves a
detour through SBI.

  - However, SBI calls are of course still supported

All we have to do is to discover the availability of SSTC, and the
migrate stimecmp to vstimecmp, and disable stimecmp. When disabling
jailhouse, we simply migrate vstimecmp back to stimecmp.

[1] https://drive.google.com/file/d/1f4DyxZMzl3yH7KGKXJFZ_iUY_AU9az_K/view

Signed-off-by: Ralf Ramsauer 
[since RFC v2: If we have STCE, no need to forward the SBI call to
firmware. Simply 'inject' the timer to our guest via stimecmp. Add QEMU
workaround.]
---
 hypervisor/arch/riscv/include/asm/csr64.h |  4 +++
 hypervisor/arch/riscv/include/asm/processor.h |  5 
 hypervisor/arch/riscv/include/asm/setup.h |  2 ++
 hypervisor/arch/riscv/setup.c | 26 +++
 hypervisor/arch/riscv/traps.c | 15 +++
 5 files changed, 52 insertions(+)

diff --git a/hypervisor/arch/riscv/include/asm/csr64.h 
b/hypervisor/arch/riscv/include/asm/csr64.h
index d0f39e8a..a12f3ffb 100644
--- a/hypervisor/arch/riscv/include/asm/csr64.h
+++ b/hypervisor/arch/riscv/include/asm/csr64.h
@@ -15,6 +15,8 @@
 #define _AT(T,X)   ((T)(X))
 #endif
 
+#define CSR_STIMECMP   0x14D
+#define CSR_VSTIMECMP  0x24D
 #define CSR_SATP   0x180
 #define CSR_VSSTATUS   0x200
 #define CSR_VSIE   0x204
@@ -31,6 +33,8 @@
 #define CSR_HTIMEDELTA 0x605
 #define CSR_HCOUNTEREN 0x606
 #define CSR_HGEIE  0x607
+#define CSR_HENVCFG0x60a
+#define ENVCFG_STCE(_AC(1, ULL) << 63)
 #define CSR_HTVAL  0x643
 #define CSR_HIP0x644
 #define CSR_HVIP   0x645
diff --git a/hypervisor/arch/riscv/include/asm/processor.h 
b/hypervisor/arch/riscv/include/asm/processor.h
index ed0d9a36..77ada05a 100644
--- a/hypervisor/arch/riscv/include/asm/processor.h
+++ b/hypervisor/arch/riscv/include/asm/processor.h
@@ -87,6 +87,11 @@ static inline void guest_clear_ext(void)
csr_clear(CSR_HVIP, (1 << IRQ_S_EXT) << VSIP_TO_HVIP_SHIFT);
 }
 
+static inline void timer_disable(void)
+{
+   csr_clear(sie, IE_TIE);
+}
+
 static inline void ext_disable(void)
 {
csr_clear(sie, IE_EIE);
diff --git a/hypervisor/arch/riscv/include/asm/setup.h 
b/hypervisor/arch/riscv/include/asm/setup.h
index d8ba88db..8a0e6c98 100644
--- a/hypervisor/arch/riscv/include/asm/setup.h
+++ b/hypervisor/arch/riscv/include/asm/setup.h
@@ -10,5 +10,7 @@
  * the COPYING file in the top-level directory.
  */
 
+extern bool has_sstc;
+
 void __attribute__((noreturn))
 riscv_deactivate_vmm(union registers *regs, int errcode, bool from_ecall);
diff --git a/hypervisor/arch/riscv/setup.c b/hypervisor/arch/riscv/setup.c
index f3cdfc06..b1b93d11 100644
--- a/hypervisor/arch/riscv/setup.c
+++ b/hypervisor/arch/riscv/setup.c
@@ -24,6 +24,8 @@ void riscv_park_loop(void);
 void __attribute((noreturn))
 __riscv_deactivate_vmm(union registers *regs, int errcode, bool from_ecall);
 
+bool has_sstc;
+
 int arch_init_early(void)
 {
int err;
@@ -98,6 +100,22 @@ void __attribute__ ((noreturn)) arch_cpu_activate_vmm(void)
tmp = csr_read(sip);
csr_write(CSR_HVIP, tmp << VSIP_TO_HVIP_SHIFT); /* reinject pending */
 
+   /* try to enable SSTC extension, if available */
+   csr_write(CSR_HENVCFG, ENVCFG_STCE);
+   /* STCE is WARL, check its presence */
+   has_sstc = !!(csr_read(CSR_HENVCFG) & ENVCFG_STCE);
+   /*
+* If we discovered SSTC, then disable the S-Mode Timer and migrate it
+* to VS-Mode. Even if the guest doesn't use SSTC this is okay, as the
+* Timer will arrive regularly.
+*/
+   if (has_sstc) {
+   tmp = csr_read(CSR_STIMECMP);
+   csr_write(CSR_VSTIMECMP, tmp);
+   csr_write(CSR_STIMECMP, -1);
+

[PATCH v3 56/73] configs: riscv: qemu: add system configs

[Ralf Ramsauer]

Signed-off-by: Ralf Ramsauer 
---
 configs/riscv/qemu-layout.h  |  22 
 configs/riscv/qemu-plic-mc.c |   2 +
 configs/riscv/qemu-plic-uc.c |   2 +
 configs/riscv/qemu-riscv64.c |  65 --
 configs/riscv/qemu.h | 244 +++
 5 files changed, 270 insertions(+), 65 deletions(-)
 create mode 100644 configs/riscv/qemu-layout.h
 create mode 100644 configs/riscv/qemu-plic-mc.c
 create mode 100644 configs/riscv/qemu-plic-uc.c
 delete mode 100644 configs/riscv/qemu-riscv64.c
 create mode 100644 configs/riscv/qemu.h

diff --git a/configs/riscv/qemu-layout.h b/configs/riscv/qemu-layout.h
new file mode 100644
index ..5ec52198
--- /dev/null
+++ b/configs/riscv/qemu-layout.h
@@ -0,0 +1,22 @@
+/*
+ * Jailhouse, a Linux-based partitioning hypervisor
+ *
+ * Configuration for QEMU riscv64 virtual target, 1G RAM, 6 cores
+ *
+ * Copyright (c) OTH Regensburg, 2023
+ *
+ * Authors:
+ *  Ralf Ramsauer 
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2.  See
+ * the COPYING file in the top-level directory.
+ */
+
+#define MIB(1024UL * 1024UL)
+#define GIB(1024UL * MIB)
+
+#define MEM_TOP(0x8000 + 1 * GIB)
+
+#define HV_SIZE0x60 /* 6MiB HV Size */
+#define HV_OFFSET  0x20 /* Recent OpenSBI place DTB there */
+#define HV_PHYS(MEM_TOP - HV_OFFSET - HV_SIZE)
diff --git a/configs/riscv/qemu-plic-mc.c b/configs/riscv/qemu-plic-mc.c
new file mode 100644
index ..c70371d2
--- /dev/null
+++ b/configs/riscv/qemu-plic-mc.c
@@ -0,0 +1,2 @@
+#define QEMU_MC
+#include "qemu.h"
diff --git a/configs/riscv/qemu-plic-uc.c b/configs/riscv/qemu-plic-uc.c
new file mode 100644
index ..1439aa95
--- /dev/null
+++ b/configs/riscv/qemu-plic-uc.c
@@ -0,0 +1,2 @@
+#define QEMU_UC
+#include "qemu.h"
diff --git a/configs/riscv/qemu-riscv64.c b/configs/riscv/qemu-riscv64.c
deleted file mode 100644
index 3c68ad71..
--- a/configs/riscv/qemu-riscv64.c
+++ /dev/null
@@ -1,65 +0,0 @@
-/*
- * Jailhouse, a Linux-based partitioning hypervisor
- *
- * Configuration for QEMU riscv64 virtual target, 1G RAM, 4 cores
- *
- * Copyright (c) Siemens AG, 2020
- *
- * Authors:
- *  Jan Kiszka 
- *
- * This work is licensed under the terms of the GNU GPL, version 2.  See
- * the COPYING file in the top-level directory.
- *
- * NOTE: Add "mem=768M" to the kernel command line.
- */
-
-#include 
-#include 
-
-struct {
-   struct jailhouse_system header;
-   struct jailhouse_cpu cpus[4];
-   struct jailhouse_memory mem_regions[1];
-} __attribute__((packed)) config = {
-   .header = {
-   .signature = JAILHOUSE_SYSTEM_SIGNATURE,
-   .revision = JAILHOUSE_CONFIG_REVISION,
-   .flags = JAILHOUSE_SYS_VIRTUAL_DEBUG_CONSOLE,
-   .hypervisor_memory = {
-   .phys_start = 0xbfc0,
-   .size =   0x0040,
-   },
-   .root_cell = {
-   .name = "qemu-riscv64",
-
-   .num_cpus = ARRAY_SIZE(config.cpus),
-   .num_memory_regions = ARRAY_SIZE(config.mem_regions),
-   },
-   },
-
-   .cpus = {
-   {
-   .phys_id = 0,
-   },
-   {
-   .phys_id = 1,
-   },
-   {
-   .phys_id = 2,
-   },
-   {
-   .phys_id = 3,
-   },
-   },
-
-   .mem_regions = {
-   /* RAM */ {
-   .phys_start = 0x8000,
-   .virt_start = 0x8000,
-   .size = 0x3fa1,
-   .flags = JAILHOUSE_MEM_READ | JAILHOUSE_MEM_WRITE |
-   JAILHOUSE_MEM_EXECUTE,
-   },
-   }
-};
diff --git a/configs/riscv/qemu.h b/configs/riscv/qemu.h
new file mode 100644
index ..940efb3f
--- /dev/null
+++ b/configs/riscv/qemu.h
@@ -0,0 +1,244 @@
+/*
+ * Jailhouse, a Linux-based partitioning hypervisor
+ *
+ * Configuration for QEMU riscv64 virtual target, 1G RAM, 6 cores
+ *
+ * Copyright (c) Siemens AG, 2020
+ * Copyright (c) OTH Regensburg, 2022
+ *
+ * Authors:
+ *  Jan Kiszka 
+ *  Konrad Schwarz 
+ *  Ralf Ramsauer 
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2.  See
+ * the COPYING file in the top-level directory.
+ *
+ * NOTE: Add "mem=768M" to the kernel command line.
+ */
+
+#include 
+#include 
+#include "qemu-layout.h"
+
+struct {
+   struct jailhouse_system header;
+#ifdef QEMU_UC
+   struct jailhouse_cpu cpus[1];
+#elif defined(QEMU_MC)
+   struct jailhouse_cpu cpus[6];
+#endif
+   struct jailhouse_memory mem_regions[9];
+   struct jailhouse_irqchip ir

[PATCH v3 57/73] configs: riscv: qemu: add inmate demo config

[Ralf Ramsauer]

Signed-off-by: Ralf Ramsauer 
---
 configs/riscv/qemu-inmate-demo.c |  1 +
 configs/riscv/qemu-inmate-demo.h | 65 
 configs/riscv/qemu-layout.h  |  6 +++
 3 files changed, 72 insertions(+)
 create mode 100644 configs/riscv/qemu-inmate-demo.c
 create mode 100644 configs/riscv/qemu-inmate-demo.h

diff --git a/configs/riscv/qemu-inmate-demo.c b/configs/riscv/qemu-inmate-demo.c
new file mode 100644
index ..f0a3db6c
--- /dev/null
+++ b/configs/riscv/qemu-inmate-demo.c
@@ -0,0 +1 @@
+#include "qemu-inmate-demo.h"
diff --git a/configs/riscv/qemu-inmate-demo.h b/configs/riscv/qemu-inmate-demo.h
new file mode 100644
index ..9d3e7abe
--- /dev/null
+++ b/configs/riscv/qemu-inmate-demo.h
@@ -0,0 +1,65 @@
+/*
+ * Jailhouse, a Linux-based partitioning hypervisor
+ *
+ * Minimal configuration for RISC-V demo inmate, 1 CPU, 1MiB RAM
+ *
+ * Copyright (c) OTH Regensburg 2022
+ *
+ * Authors:
+ *  Ralf Ramsauer 
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2.  See
+ * the COPYING file in the top-level directory.
+ */
+
+#include 
+#include 
+#include "qemu-layout.h"
+
+struct {
+   struct jailhouse_cell_desc cell;
+   struct jailhouse_cpu cpus[1];
+   struct jailhouse_memory mem_regions[2];
+} __attribute__((packed)) config = {
+   .cell = {
+   .signature = JAILHOUSE_CELL_DESC_SIGNATURE,
+   .revision = JAILHOUSE_CONFIG_REVISION,
+   .architecture = JAILHOUSE_RISCV64,
+   .name = "qemu-inmate-demo",
+   .flags = JAILHOUSE_CELL_PASSIVE_COMMREG |
+   JAILHOUSE_CELL_TEST_DEVICE |
+   JAILHOUSE_CELL_VIRTUAL_CONSOLE_PERMITTED |
+   JAILHOUSE_CELL_VIRTUAL_CONSOLE_ACTIVE,
+
+   .num_cpus = ARRAY_SIZE(config.cpus),
+   .num_memory_regions = ARRAY_SIZE(config.mem_regions),
+   .num_irqchips = 0,
+   .num_pci_devices = 0,
+
+   .console = {
+   .type = JAILHOUSE_CON_TYPE_RISCV_SBI,
+   },
+   },
+
+   .cpus = {
+   {
+   .phys_id = 2,
+   },
+   },
+
+   .mem_regions = {
+   /* RAM */ {
+   .phys_start = INMATE_TINY_PHYS,
+   .virt_start = 0x0,
+   .size = INMATE_TINY_SIZE,
+   .flags = JAILHOUSE_MEM_READ | JAILHOUSE_MEM_WRITE |
+   JAILHOUSE_MEM_EXECUTE | JAILHOUSE_MEM_LOADABLE,
+   },
+   /* communication region */ {
+   .virt_start = 0x0010,
+   .size = 0x1000,
+   .flags = JAILHOUSE_MEM_READ | JAILHOUSE_MEM_WRITE |
+   JAILHOUSE_MEM_COMM_REGION,
+   },
+   },
+};
diff --git a/configs/riscv/qemu-layout.h b/configs/riscv/qemu-layout.h
index 5ec52198..8f4d4554 100644
--- a/configs/riscv/qemu-layout.h
+++ b/configs/riscv/qemu-layout.h
@@ -20,3 +20,9 @@
 #define HV_SIZE0x60 /* 6MiB HV Size */
 #define HV_OFFSET  0x20 /* Recent OpenSBI place DTB there */
 #define HV_PHYS(MEM_TOP - HV_OFFSET - HV_SIZE)
+
+/* In preparation for IVSHMEM */
+#define IVSHMEM_NET_PHYS   (HV_PHYS - MIB)
+
+#define INMATE_TINY_SIZE   MIB
+#define INMATE_TINY_PHYS   (IVSHMEM_NET_PHYS - INMATE_TINY_SIZE)
-- 
2.40.1

-- 
You received this message because you are subscribed to the Google Groups 
"Jailhouse" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jailhouse-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jailhouse-dev/20230519204033.643200-58-ralf.ramsauer%40oth-regensburg.de.

[PATCH v3 53/73] inmates: riscv: add timer demo

[Ralf Ramsauer]

Same demo as for other architectures.

Signed-off-by: Ralf Ramsauer 
---
 inmates/demos/riscv/Makefile   |   3 +-
 inmates/demos/riscv/timer-demo.c   |  57 +++
 inmates/lib/riscv/Makefile |   2 +-
 inmates/lib/riscv/header.S |  29 
 inmates/lib/riscv/include/inmate.h |  27 +++-
 inmates/lib/riscv/inmate.lds.S |   3 +
 inmates/lib/riscv/irq.c| 107 +
 7 files changed, 225 insertions(+), 3 deletions(-)
 create mode 100644 inmates/demos/riscv/timer-demo.c
 create mode 100644 inmates/lib/riscv/irq.c

diff --git a/inmates/demos/riscv/Makefile b/inmates/demos/riscv/Makefile
index 2f0bac84..e371a29b 100644
--- a/inmates/demos/riscv/Makefile
+++ b/inmates/demos/riscv/Makefile
@@ -12,8 +12,9 @@
 
 include $(INMATES_LIB)/Makefile.lib
 
-INMATES := tiny-demo.bin
+INMATES := tiny-demo.bin timer-demo.bin
 
 tiny-demo-y:= tiny-demo.o
+timer-demo-y   := timer-demo.o
 
 $(eval $(call DECLARE_TARGETS,$(INMATES)))
diff --git a/inmates/demos/riscv/timer-demo.c b/inmates/demos/riscv/timer-demo.c
new file mode 100644
index ..748219b7
--- /dev/null
+++ b/inmates/demos/riscv/timer-demo.c
@@ -0,0 +1,57 @@
+/*
+ * Jailhouse, a Linux-based partitioning hypervisor
+ *
+ * Copyright (c) OTH Regensburg, 2022
+ *
+ * Authors:
+ *  Ralf Ramsauer 
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2.  See
+ * the COPYING file in the top-level directory.
+ */
+
+#include 
+
+#include 
+
+#define TIMEBASE_FREQ  0x989680
+#define TIMER_DELTA(TIMEBASE_FREQ / 6)
+
+unsigned long timer_next;
+
+static inline u64 timer_ticks_to_ns(u64 ticks)
+{
+   return (1000UL * 1000UL * 1000UL * ticks) / TIMEBASE_FREQ;
+}
+
+unsigned long timer_handler(void)
+{
+   static u64 min_delta = ~0ULL, max_delta = 0;
+   unsigned long delta;
+
+   delta = get_cycles() - timer_next;
+   if (delta < min_delta)
+   min_delta = delta;
+   if (delta > max_delta)
+   max_delta = delta;
+
+   printk("Timer fired, jitter: %6ld ns, min: %6ld ns, max: %6ld ns\n",
+   (long)timer_ticks_to_ns(delta),
+   (long)timer_ticks_to_ns(min_delta),
+   (long)timer_ticks_to_ns(max_delta));
+
+   timer_next += TIMER_DELTA;
+
+   return timer_next;
+}
+
+void inmate_main(void)
+{
+   printk("Initializing the timer...\n");
+   timer_next = get_cycles() + 0x40;
+   sbi_set_timer(timer_next);
+   timer_enable();
+   enable_irqs();
+
+   halt();
+}
diff --git a/inmates/lib/riscv/Makefile b/inmates/lib/riscv/Makefile
index 9ff57721..3b796dc3 100644
--- a/inmates/lib/riscv/Makefile
+++ b/inmates/lib/riscv/Makefile
@@ -41,6 +41,6 @@ include $(INMATES_LIB)/Makefile.lib
 always-y := lib.a inmate.lds
 
 lib-y := $(common-objs-y)
-lib-y += header.o printk.o setup.o uart.o
+lib-y += header.o irq.o printk.o setup.o uart.o
 
 lib-y += ../cmdline.o ../printk.o ../setup.o ../string.o ../uart-8250.o
diff --git a/inmates/lib/riscv/header.S b/inmates/lib/riscv/header.S
index 3a12f51f..5ddffe79 100644
--- a/inmates/lib/riscv/header.S
+++ b/inmates/lib/riscv/header.S
@@ -36,9 +36,38 @@
  * THE POSSIBILITY OF SUCH DAMAGE.
  */
 
+#define STRUCT_REGISTERS_SIZE (32 * 8)
+
+.macro context cmd
+.irp reg 1, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, \
+19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31
+\cmdx\reg, 8*(\reg - 1)(sp)
+.endr
+.endm
+
 .section ".boot", "ax"
 
.globl __reset_entry
 __reset_entry:
la  sp, __stack_top
+   addisp, sp, -STRUCT_REGISTERS_SIZE
+
+   la  a5 , __ex_stack_top
+   csrwsscratch, a5
+
+   la  a5, exception_handler
+   csrwstvec, a5
+
j   c_entry
+
+.text
+.align 4
+exception_handler:
+   csrrw   sp, sscratch, sp
+   context sd
+
+   jal arch_handle_trap
+
+   context ld
+   csrrw   sp, sscratch, sp
+   sret
diff --git a/inmates/lib/riscv/include/inmate.h 
b/inmates/lib/riscv/include/inmate.h
index dac9f146..57338d25 100644
--- a/inmates/lib/riscv/include/inmate.h
+++ b/inmates/lib/riscv/include/inmate.h
@@ -1,7 +1,7 @@
 /*
  * Jailhouse, a Linux-based partitioning hypervisor
  *
- * Copyright (c) OTH Regensburg
+ * Copyright (c) OTH Regensburg, 2022
  *
  * Authors:
  *  Ralf Ramsauer 
@@ -60,6 +60,9 @@ typedef unsigned long long u64;
 
 #define SR_SIE 0x0002UL
 
+#define IRQ_S_TIMER5
+#define IE_TIE (0x1UL << IRQ_S_TIMER)
+
 #define csr_read(csr)   \
 ({  \
register unsigned long __v; \
@@ -93,11 +96,31 @@ typedef unsigned long long u64;
  : "memory");  \
 })
 
+static inline unsigned long get_cycles(void)
+{
+   ret

[PATCH v3 55/73] inmates: riscv: implement Linux Loader

[Ralf Ramsauer]

The boot protocol is simple:
 - a0 hold the hart id
 - a1 hold the DTB

That's it.

Signed-off-by: Ralf Ramsauer 
---
 inmates/tools/riscv/Makefile   | 19 +++
 inmates/tools/riscv/linux-loader.c | 25 +
 2 files changed, 44 insertions(+)
 create mode 100644 inmates/tools/riscv/linux-loader.c

diff --git a/inmates/tools/riscv/Makefile b/inmates/tools/riscv/Makefile
index e69de29b..4a722777 100644
--- a/inmates/tools/riscv/Makefile
+++ b/inmates/tools/riscv/Makefile
@@ -0,0 +1,19 @@
+#
+# Jailhouse, a Linux-based partitioning hypervisor
+#
+# Copyright (c) Siemens AG, 2013-2015
+#
+# Authors:
+#  Jan Kiszka 
+#
+# This work is licensed under the terms of the GNU GPL, version 2.  See
+# the COPYING file in the top-level directory.
+#
+
+include $(INMATES_LIB)/Makefile.lib
+
+INMATES := linux-loader.bin
+
+linux-loader-y := linux-loader.o
+
+$(eval $(call DECLARE_TARGETS,$(INMATES)))
diff --git a/inmates/tools/riscv/linux-loader.c 
b/inmates/tools/riscv/linux-loader.c
new file mode 100644
index ..1434cb0a
--- /dev/null
+++ b/inmates/tools/riscv/linux-loader.c
@@ -0,0 +1,25 @@
+/*
+ * Jailhouse, a Linux-based partitioning hypervisor
+ *
+ * Copyright (c) OTH Regensburg, 2022
+ *
+ * Authors:
+ *  Ralf Ramsauer 
+ *  Stefan Huber 
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2.  See
+ * the COPYING file in the top-level directory.
+ */
+
+#include 
+
+void inmate_main(void)
+{
+   void (*entry)(u64 hartid, u64 dtb);
+   unsigned long dtb;
+
+   entry = (void *)cmdline_parse_int("kernel", 0);
+   dtb = cmdline_parse_int("dtb", 0);
+
+   entry(hart_id, dtb);
+}
-- 
2.40.1

-- 
You received this message because you are subscribed to the Google Groups 
"Jailhouse" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jailhouse-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jailhouse-dev/20230519204033.643200-56-ralf.ramsauer%40oth-regensburg.de.

[PATCH v3 58/73] configs: riscv: qemu: Add linux-demo

[Ralf Ramsauer]

Signed-off-by: Ralf Ramsauer 
[RFC v2: fix typo]
---
 configs/riscv/dts/qemu-linux-inmate-plic.dts | 112 +++
 configs/riscv/qemu-layout.h  |   6 +
 configs/riscv/qemu-linux-demo-plic.c |   2 +
 configs/riscv/qemu-linux-demo.h  | 103 +
 4 files changed, 223 insertions(+)
 create mode 100644 configs/riscv/dts/qemu-linux-inmate-plic.dts
 create mode 100644 configs/riscv/qemu-linux-demo-plic.c
 create mode 100644 configs/riscv/qemu-linux-demo.h

diff --git a/configs/riscv/dts/qemu-linux-inmate-plic.dts 
b/configs/riscv/dts/qemu-linux-inmate-plic.dts
new file mode 100644
index ..4a47766d
--- /dev/null
+++ b/configs/riscv/dts/qemu-linux-inmate-plic.dts
@@ -0,0 +1,112 @@
+/*
+ * Jailhouse, a Linux-based partitioning hypervisor
+ *
+ * Copyright (c) OTH Regensburg, 2022
+ *
+ * Authors:
+ *  Ralf Ramsauer 
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2.  See
+ * the COPYING file in the top-level directory.
+ */
+
+/dts-v1/;
+/ {
+   model = "Jailhouse cell on RISC-V";
+
+   #address-cells = <2>;
+   #size-cells = <2>;
+
+   chosen {
+   stdout-path = "/soc/uart@1000:115200n8";
+   bootargs = "console=ttyS0 earlycon=sbi";
+   linux,initrd-start = <0x8200>;
+   linux,initrd-end = <0x8300>;
+   };
+
+   memory@8000 {
+   device_type = "memory";
+   /* 63MiB @ 0x8000 */
+   reg = <0x 0x8000>, <0x 0x800>;
+   };
+
+   cpus {
+   #address-cells = <1>;
+   #size-cells = <0>;
+   timebase-frequency = <0x989680>;
+
+   cpu0: cpu@0 {
+   device_type = "cpu";
+   reg = <0>;
+   status = "okay";
+   compatible = "riscv";
+   riscv,isa = 
"rv64imafdc_zicsr_zifencei_zihintpause_zawrs_zba_zbb_zbc_zbs_sstc";
+   mmu-type = "riscv,sv39";
+
+   cpu0_intc: interrupt-controller {
+   #interrupt-cells = <1>;
+   compatible = "riscv,cpu-intc";
+   interrupt-controller;
+   };
+   };
+
+   cpu1: cpu@1 {
+   device_type = "cpu";
+   reg = <1>;
+   status = "okay";
+   compatible = "riscv";
+   riscv,isa = 
"rv64imafdc_zicsr_zifencei_zihintpause_zawrs_zba_zbb_zbc_zbs_sstc";
+   mmu-type = "riscv,sv39";
+
+   cpu1_intc: interrupt-controller {
+   #interrupt-cells = <1>;
+   compatible = "riscv,cpu-intc";
+   interrupt-controller;
+   };
+   };
+   };
+   soc {
+   #address-cells = <2>;
+   #size-cells = <2>;
+   compatible = "simple-bus";
+   ranges;
+
+   uart@1000 {
+   interrupts = <10>;
+   interrupt-parent = <>;
+   clock-frequency = "\08@";
+   reg = <0x00 0x1000 0x00 0x100>;
+   compatible = "ns16550a";
+   status = "okay";
+   };
+
+   plic0: plic@c00 {
+   compatible = "sifive,plic-1.0.0\0riscv,plic0";
+   reg = <0x00 0xc00 0x00 0x21>;
+   #address-cells = <0>;
+   #interrupt-cells = <1>;
+   interrupts-extended =
+   <_intc 11>, <_intc 9>,
+   <_intc 11>, <_intc 9>;
+   interrupt-controller;
+   riscv,ndev = <0x35>;
+   };
+
+   pci@3000 {
+   compatible = "pci-host-ecam-generic";
+   device_type = "pci";
+   bus-range = <0 0>;
+   #interrupt-cells = <1>;
+   reg = <0x0 0x3000 0x0 0x1000>;
+   #address-cells = <3>;
+   #size-cells = <2>;
+   ranges = <
+   0x100 0x00 0x00 0x00 0x300 0x00 0x1
+   0x200 0x00 0x4000 0

[PATCH v3 51/73] core: riscv: implement trap handlers

[Ralf Ramsauer]

Signed-off-by: Ralf Ramsauer 
[since RFC v1: Add more fine granular SBI accounting, simplify usage of hlvx]
[since RFC v2: inline single-caller functions, improve rfence handler]
---
 hypervisor/arch/riscv/traps.c | 793 +-
 1 file changed, 788 insertions(+), 5 deletions(-)

diff --git a/hypervisor/arch/riscv/traps.c b/hypervisor/arch/riscv/traps.c
index f61dfc1a..b3bbe413 100644
--- a/hypervisor/arch/riscv/traps.c
+++ b/hypervisor/arch/riscv/traps.c
@@ -10,19 +10,802 @@
  * the COPYING file in the top-level directory.
  */
 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
 #include 
+#include 
+#include 
 
 void arch_handle_trap(union registers *regs);
 void arch_handle_fault(union registers *regs);
 
-void arch_handle_trap(union registers *regs)
+static const char *causes[] = {
+   [EXC_INST_MISALIGNED]   = "Instruction Address Misaligned",
+   [EXC_INST_ACCESS]   = "Instruction Address Fault",
+   [EXC_INST_ILLEGAL]  = "Illegal Instruction",
+   [EXC_BREAKPOINT]= "Breakpoint",
+   [EXC_LOAD_ACCESS_MISALIGNED]= "Load Accesss Misaligned",
+   [EXC_LOAD_ACCESS]   = "Load Access Fault",
+   [EXC_AMO_ADDRESS_MISALIGNED]= "AMO Address Misaligned",
+   [EXC_STORE_ACCESS]  = "Store Access Fault",
+   [EXC_SYSCALL]   = "Env Call From U-Mode",
+   [EXC_HYPERVISOR_SYSCALL]= "Env Call From S-Mode",
+   [EXC_SUPERVISOR_SYSCALL]= "Env Call From VS-Mode",
+   [11]= "Env Call From M-Mode",
+   [EXC_INST_PAGE_FAULT]   = "Instruction Pagefault",
+   [EXC_LOAD_PAGE_FAULT]   = "Load Pagefault",
+   [14]= "Reserved",
+   [EXC_STORE_PAGE_FAULT]  = "Store Pagefault",
+   [16 ... 19] = "Reserved",
+   [EXC_INST_GUEST_PAGE_FAULT] = "Inst Guest Pagefault",
+   [EXC_LOAD_GUEST_PAGE_FAULT] = "Load Guest Pagefault",
+   [EXC_VIRTUAL_INST_FAULT]= "Virtual Instruction Fault",
+   [EXC_STORE_GUEST_PAGE_FAULT]= "Store Guest Pagefault",
+};
+
+static inline bool is_irq(u64 cause)
+{
+   return !!(cause & CAUSE_IRQ_FLAG);
+}
+
+static inline unsigned long to_irq(unsigned long cause)
+{
+   return cause & ~CAUSE_IRQ_FLAG;
+}
+
+static void dump_regs(union registers *r)
+{
+   struct public_per_cpu *pc = this_cpu_public();
+   unsigned long scause = csr_read(scause);
+   const char *cause_str = "UNKNOWN";
+
+   /* We might want to later exchange it with panic_printk, but for the
+* moment of development, printk is more helpful. */
+   void (*printer)(const char *, ...) = panic_printk;
+
+   if (scause <= 23)
+   cause_str = causes[scause];
+   else if (is_irq(scause))
+   cause_str = "IRQ";
+
+   printer("Fatal: Exception on CPU %u (HART %u). Cause: %lu (%s)\n"
+   "scause: 0x%016lx, stval: 0x%016lx, htval<<2: 0x%016lx\n"
+   " PC: 0x%016llx RA: 0x%016llx  SP: 0x%016llx\n"
+   " GP: 0x%016llx TP: 0x%016llx  T0: 0x%016llx\n"
+   " T1: 0x%016llx T2: 0x%016llx  S0: 0x%016llx\n"
+   " S1: 0x%016llx A0: 0x%016llx  A1: 0x%016llx\n"
+   " A2: 0x%016llx A3: 0x%016llx  A4: 0x%016llx\n"
+   " A5: 0x%016llx A6: 0x%016llx  A7: 0x%016llx\n"
+   " S2: 0x%016llx S3: 0x%016llx  S4: 0x%016llx\n"
+   " S5: 0x%016llx S6: 0x%016llx  S7: 0x%016llx\n"
+   " S8: 0x%016llx S9: 0x%016llx S10: 0x%016llx\n"
+   "S11: 0x%016llx T3: 0x%016llx  T4: 0x%016llx\n"
+   " T5: 0x%016llx T6: 0x%016llx\n",
+   pc->cpu_id, pc->phys_id, to_irq(scause), cause_str,
+   scause, csr_read(stval), csr_read(CSR_HTVAL) << 2,
+   r->pc, r->ra, r->sp,
+   r->gp, r->tp, r->t0,
+   r->t1, r->t2, r->s0,
+   r->s1, r->a0, r->a1,
+   r->a2, r->a3, r->a4,
+   r->a5, r->a6, r->a7,
+   r->s2, r->s3, r->s4,
+   r->s5, r->s6, r->s7,
+   r->s8, r->s9, r->s10,
+   r->s11, r->t3, r->t4,
+   r->t5, r->t6);
+}
+
+static inline int handle_timer(void)
+{
+   this_cpu_public()->stats[JAILHOUSE_CPU_STAT_VMEXITS_TIMER]++;
+   sbi_set_timer(-1);
+
+   /* inject

[PATCH v3 54/73] inmates: riscv: propagate hartid

[Ralf Ramsauer]

We will need the hartid for the linux-loader later.

Signed-off-by: Ralf Ramsauer 
---
 inmates/lib/riscv/header.S | 3 +++
 inmates/lib/riscv/include/inmate.h | 2 ++
 inmates/lib/riscv/setup.c  | 2 ++
 3 files changed, 7 insertions(+)

diff --git a/inmates/lib/riscv/header.S b/inmates/lib/riscv/header.S
index 5ddffe79..e4b21ce5 100644
--- a/inmates/lib/riscv/header.S
+++ b/inmates/lib/riscv/header.S
@@ -58,6 +58,9 @@ __reset_entry:
la  a5, exception_handler
csrwstvec, a5
 
+   la  a5, hart_id
+   sd  a0, 0(a5)
+
j   c_entry
 
 .text
diff --git a/inmates/lib/riscv/include/inmate.h 
b/inmates/lib/riscv/include/inmate.h
index 57338d25..468d2218 100644
--- a/inmates/lib/riscv/include/inmate.h
+++ b/inmates/lib/riscv/include/inmate.h
@@ -58,6 +58,8 @@ typedef unsigned int u32;
 typedef signed long long s64;
 typedef unsigned long long u64;
 
+extern unsigned long hart_id;
+
 #define SR_SIE 0x0002UL
 
 #define IRQ_S_TIMER5
diff --git a/inmates/lib/riscv/setup.c b/inmates/lib/riscv/setup.c
index 705c5630..a87e8c50 100644
--- a/inmates/lib/riscv/setup.c
+++ b/inmates/lib/riscv/setup.c
@@ -38,6 +38,8 @@
 
 #include 
 
+unsigned long hart_id;
+
 void arch_init_early(void)
 {
 }
-- 
2.40.1

-- 
You received this message because you are subscribed to the Google Groups 
"Jailhouse" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jailhouse-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jailhouse-dev/20230519204033.643200-55-ralf.ramsauer%40oth-regensburg.de.

[PATCH v3 50/73] core: riscv: Add PLIC implementation

[Ralf Ramsauer]

The design of the PLIC is poor: We basically need to trap & moderate
each access.

The strategy is as follows:

On IRQ arrival in S-Mode, we directly acknowledge the IRQ, save it in a
shadow register, and reinject it to the VS-Mode guest. Now disable IRQs
for S-Mode, until the guest has claimed and acknowledged the IRQ.

After the guest acknowledged the IRQ, reenable IRQs in S-Mode again.

Signed-off-by: Ralf Ramsauer 
---
 hypervisor/arch/riscv/Kbuild|   2 +-
 hypervisor/arch/riscv/irqchip.c |   7 +-
 hypervisor/arch/riscv/plic.c| 392 
 3 files changed, 399 insertions(+), 2 deletions(-)
 create mode 100644 hypervisor/arch/riscv/plic.c

diff --git a/hypervisor/arch/riscv/Kbuild b/hypervisor/arch/riscv/Kbuild
index 57c52854..0e7d3719 100644
--- a/hypervisor/arch/riscv/Kbuild
+++ b/hypervisor/arch/riscv/Kbuild
@@ -15,4 +15,4 @@
 always-y := lib.a
 
 lib-y := entry.o exception.o setup.o dbg-write.o control.o ivshmem.o irqchip.o
-lib-y += paging.o pci.o traps.o lib.o
+lib-y += paging.o pci.o plic.o traps.o lib.o
diff --git a/hypervisor/arch/riscv/irqchip.c b/hypervisor/arch/riscv/irqchip.c
index 880f1fab..4d8ade0b 100644
--- a/hypervisor/arch/riscv/irqchip.c
+++ b/hypervisor/arch/riscv/irqchip.c
@@ -23,6 +23,8 @@
 #define IRQ_BITMAP_PINS \
(sizeof(((struct cell *)0)->arch.irq_bitmap) * 8)
 
+extern const struct irqchip irqchip_plic;
+
 struct irqchip irqchip;
 
 static unsigned int irqchip_mmio_count_regions(struct cell *cell)
@@ -131,8 +133,11 @@ static int irqchip_init(void)
 {
int err;
 
-   /* We don't have a working irqchip yet */
switch (irqchip_type()) {
+   case JAILHOUSE_RISCV_PLIC:
+   irqchip = irqchip_plic;
+   break;
+
default:
return trace_error(-ENOSYS);
}
diff --git a/hypervisor/arch/riscv/plic.c b/hypervisor/arch/riscv/plic.c
new file mode 100644
index ..a65de82c
--- /dev/null
+++ b/hypervisor/arch/riscv/plic.c
@@ -0,0 +1,392 @@
+/*
+ * Jailhouse, a Linux-based partitioning hypervisor
+ *
+ * Copyright (c) OTH Regensburg, 2022-2023
+ *
+ * Authors:
+ *  Ralf Ramsauer 
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2.  See
+ * the COPYING file in the top-level directory.
+ */
+
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+
+#define PLIC_PRIO_BASE 0x0
+#define PLIC_PENDING_BASE  0x1000
+#define PLIC_ENABLE_BASE   0x2000
+#define PLIC_ENABLE_OFF0x80
+#define PLIC_ENABLE_END0x1f2000
+#define PLIC_CTX_BASE  0x20
+#define PLIC_CTX_PRIO_TH   0x0
+#define PLIC_CTX_CLAIM 0x4
+#define PLIC_CTX_SZ0x1000
+#define PLIC_CTX_END   0x400
+
+static inline s16 hart_to_context(unsigned int hartid)
+{
+   if (hartid > ARRAY_SIZE(SYSCONFIG_IRQCHIP.hart_to_context))
+   return -1;
+
+   return SYSCONFIG_IRQCHIP.hart_to_context[hartid];
+}
+
+static inline u16 plic_max_priority(void)
+{
+   return SYSCONFIG_IRQCHIP.max_priority;
+}
+
+static inline unsigned int plic_size(void)
+{
+   return SYSCONFIG_IRQCHIP.size;
+}
+
+static inline u32 plic_read(u32 reg)
+{
+   return mmio_read32(irqchip.base + reg);
+}
+
+static inline void plic_write(u32 reg, u32 value)
+{
+   mmio_write32(irqchip.base + reg, value);
+}
+
+static inline u32 plic_read_context(u32 context, u32 off)
+{
+   return plic_read(PLIC_CTX_BASE + context * PLIC_CTX_SZ + off);
+}
+
+static inline u16 plic_read_claim(u32 context)
+{
+   return plic_read_context(context, PLIC_CTX_CLAIM);
+}
+
+static inline u32 plic_en_reg(s16 context, unsigned int irq)
+{
+   u32 reg;
+
+   reg = PLIC_ENABLE_BASE + (context * PLIC_ENABLE_OFF) +
+ (irq / IRQCHIP_BITS_PER_REG) * IRQCHIP_REG_SZ;
+
+   return reg;
+}
+
+static inline u32 plic_read_enabled(s16 context, unsigned int irq)
+{
+   return plic_read(plic_en_reg(context, irq));
+}
+
+static inline void plic_write_enabled(s16 context, unsigned int irq, u32 val)
+{
+   plic_write(plic_en_reg(context, irq), val);
+}
+
+static inline bool plic_irq_is_enabled(s16 context, unsigned int irq)
+{
+   u32 en = plic_read_enabled(context, irq);
+
+   return !!(en & IRQ_MASK(irq));
+}
+
+static inline void plic_enable_irq(s16 context, unsigned int irq)
+{
+   u32 val;
+
+   val = plic_read_enabled(context, irq) | IRQ_MASK(irq);
+   plic_write_enabled(context, irq, val);
+}
+
+static inline void plic_disable_irq_ctx(s16 context, unsigned int irq)
+{
+   u32 val;
+
+   val = plic_read_enabled(context, irq) & ~IRQ_MASK(irq);
+   plic_write_enabled(context, irq, val);
+}
+
+static int plic_claim_irq(void)
+{
+   unsigned long hart;
+   unsigned short irq;
+   int my_context;
+
+   hart = phys_processor_id();
+
+   /* Assume that phys_processor_id always retu

[PATCH v3 49/73] core: riscv: Add irqchip abstraction layer

[Ralf Ramsauer]

Later, we will add support for PLIC and APLIC. This irqchip layer adds a
level of abstraction for both irqchips, and will hold code that is
shared between both.

With this commit, fill irqchip.c with life, and and the common code for
both.

Signed-off-by: Ralf Ramsauer 
---
 hypervisor/arch/riscv/Kbuild|   4 +-
 hypervisor/arch/riscv/irqchip.c | 182 
 2 files changed, 184 insertions(+), 2 deletions(-)
 create mode 100644 hypervisor/arch/riscv/irqchip.c

diff --git a/hypervisor/arch/riscv/Kbuild b/hypervisor/arch/riscv/Kbuild
index 7809007c..57c52854 100644
--- a/hypervisor/arch/riscv/Kbuild
+++ b/hypervisor/arch/riscv/Kbuild
@@ -14,5 +14,5 @@
 
 always-y := lib.a
 
-lib-y := entry.o exception.o setup.o dbg-write.o control.o ivshmem.o paging.o
-lib-y += pci.o traps.o lib.o
+lib-y := entry.o exception.o setup.o dbg-write.o control.o ivshmem.o irqchip.o
+lib-y += paging.o pci.o traps.o lib.o
diff --git a/hypervisor/arch/riscv/irqchip.c b/hypervisor/arch/riscv/irqchip.c
new file mode 100644
index ..880f1fab
--- /dev/null
+++ b/hypervisor/arch/riscv/irqchip.c
@@ -0,0 +1,182 @@
+/*
+ * Jailhouse, a Linux-based partitioning hypervisor
+ *
+ * Copyright (c) OTH Regensburg, 2023
+ *
+ * Authors:
+ *  Ralf Ramsauer 
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2.  See
+ * the COPYING file in the top-level directory.
+ */
+
+#include 
+#include 
+#include 
+#include 
+#include 
+
+/* Could also be used for arm-common/irqchip.c */
+#define IRQCHIP_PINS \
+   (sizeof(((struct jailhouse_irqchip *)0)->pin_bitmap) * 8)
+
+#define IRQ_BITMAP_PINS \
+   (sizeof(((struct cell *)0)->arch.irq_bitmap) * 8)
+
+struct irqchip irqchip;
+
+static unsigned int irqchip_mmio_count_regions(struct cell *cell)
+{
+   return 1;
+}
+
+/* Borrowed from ARM */
+static void irqchip_config_commit(struct cell *cell)
+{
+   unsigned int n;
+
+   if (!cell)
+   return;
+
+   for (n = 1; n < MAX_IRQS; n++) {
+   if (irqchip_irq_in_cell(cell, n) && cell != _cell)
+   irqchip.adjust_irq_target(cell, n);
+
+   if (irqchip_irq_in_cell(_cell, n))
+   irqchip.adjust_irq_target(_cell, n);
+   }
+}
+
+static enum mmio_result irqchip_mmio(void *arg, struct mmio_access *access)
+{
+   /* only allow 32bit access */
+   if (access->size != IRQCHIP_REG_SZ)
+   return MMIO_ERROR;
+
+   return irqchip.mmio_handler(arg, access);
+}
+
+static int irqchip_cell_init(struct cell *cell)
+{
+   const struct jailhouse_irqchip *chip;
+   unsigned int n, pos;
+
+   mmio_region_register(cell, irqchip_phys(), irqchip_size(),
+irqchip_mmio, cell);
+
+   memset(cell->arch.irq_bitmap, 0, sizeof(cell->arch.irq_bitmap));
+
+   for_each_irqchip(chip, cell->config, n) {
+   /* Only support one single PLIC at the moment */
+   if (chip->address !=
+   system_config->platform_info.riscv.irqchip.base_address)
+   return trace_error(-EINVAL);
+
+   if (chip->pin_base % 32 != 0 ||
+   chip->pin_base + IRQCHIP_PINS > IRQ_BITMAP_PINS)
+   return trace_error(-EINVAL);
+
+   for (pos = 0; pos < ARRAY_SIZE(chip->pin_bitmap); pos++)
+   cell->arch.irq_bitmap[chip->pin_base / 32 + pos] |=
+   chip->pin_bitmap[pos];
+   }
+
+   /* This logic is shared with arm-common */
+   if (cell == _cell)
+   return 0;
+
+   for_each_irqchip(chip, cell->config, n)
+   for (pos = 0; pos < ARRAY_SIZE(chip->pin_bitmap); pos++)
+   root_cell.arch.irq_bitmap[chip->pin_base / 32 + pos] &=
+   ~chip->pin_bitmap[pos];
+
+   return 0;
+}
+
+static inline bool guest_ext_pending(void)
+{
+   return !!(csr_read(CSR_HVIP) &
+   ((1 << IRQ_S_EXT) << VSIP_TO_HVIP_SHIFT));
+}
+
+int irqchip_set_pending(void)
+{
+   int err;
+
+   this_cpu_public()->stats[JAILHOUSE_CPU_STAT_VMEXITS_VIRQ]++;
+
+   err = irqchip.claim_irq();
+   if (err)
+   return err;
+
+   /*
+* We can directly inject the IRQ into the guest if the IRQ is not
+* pending, because we know that the IRQ is enabled, otherwise we
+* wouldn't have received it
+*/
+   guest_inject_ext();
+
+   /*
+* Don't claim complete! This must be done by the guest. We will handle
+* that in plic_handler(). In the meanwhile we simply deactivate S-Mode
+* External IRQs, and reenable them when the guest claims it. In this
+* way, we only need to store one pending IRQ per hart.
+*/
+   ext_disable();
+
+   return 0;
+}
+
+static int irqchip_

[PATCH v3 48/73] core: riscv: implement setup.c

[Ralf Ramsauer]

And do some heavy liftig.

Signed-off-by: Ralf Ramsauer 
[RFC v2: allow access to cycle counter]
[RFC v2: get vmm deactivation under control (didn't work with -O2)]
---
 hypervisor/arch/riscv/entry.S |  65 ++
 hypervisor/arch/riscv/include/asm/csr64.h |   1 +
 hypervisor/arch/riscv/setup.c | 137 --
 3 files changed, 195 insertions(+), 8 deletions(-)

diff --git a/hypervisor/arch/riscv/entry.S b/hypervisor/arch/riscv/entry.S
index 524270cf..a7f1ce6e 100644
--- a/hypervisor/arch/riscv/entry.S
+++ b/hypervisor/arch/riscv/entry.S
@@ -353,4 +353,69 @@ virtual_arch_entry:
 
mv  a1, a0
mv  a0, sp
+   li  a2, 0 /* We don't come from an ecall */
j   riscv_deactivate_vmm
+
+
+.global __riscv_deactivate_vmm
+__riscv_deactivate_vmm:
+   /*
+* From now on, we can safely access stack variables, but we must not
+* use any absolute addresses. We may clobber a0-a7, use them as
+* scratch.
+*/
+   csr_from_csrsatp, CSR_VSATP, a5;
+   sfence.vma
+   /* From here on, Linux's paging is active. */
+
+   /* Restore S-mode CSRs from VS-mode */
+   csr_from_csrstval, CSR_VSTVAL, a5
+   csr_from_csrscause, CSR_VSCAUSE, a5
+   csr_from_csrsscratch, CSR_VSSCRATCH, a5
+   csr_from_csrsie, CSR_VSIE, a5
+   csr_from_csrstvec, CSR_VSTVEC, a5
+   csr_from_csrsstatus, CSR_VSSTATUS, a5
+
+   /* If we didn't come here from en ecall, skip sepc setup */
+   beq a2, x0, 1f
+   /* scratch setup */
+   csrra3, sepc
+   addia3, a3, 4 /* ecall */
+
+   /* restore original sepc */
+   csrr a4, CSR_VSEPC
+   csrw sepc, a4
+
+1:
+   /* restore registers */
+   ld ra, 8(a0)
+   ld sp, 16(a0)
+   ld gp, 24(a0)
+   ld tp, 32(a0)
+   ld t0, 40(a0)
+   ld t1, 48(a0)
+   ld t2, 56(a0)
+   ld s0, 64(a0)
+   ld s1, 72(a0)
+   /* Skip clobbers a0 - a7 */
+   ld s2, 144(a0)
+   ld s3, 152(a0)
+   ld s4, 160(a0)
+   ld s5, 168(a0)
+   ld s6, 176(a0)
+   ld s7, 184(a0)
+   ld s8, 192(a0)
+   ld s9, 200(a0)
+   ld s10, 208(a0)
+   ld s11, 216(a0)
+   ld t3, 224(a0)
+   ld t4, 232(a0)
+   ld t5, 240(a0)
+   ld t6, 248(a0)
+   mv a0, a1
+
+   /* And we're done. */
+   beq a2, x0, 1f
+   jalr zero, a3, 0
+1:
+   ret
diff --git a/hypervisor/arch/riscv/include/asm/csr64.h 
b/hypervisor/arch/riscv/include/asm/csr64.h
index 3c4fa00f..d0f39e8a 100644
--- a/hypervisor/arch/riscv/include/asm/csr64.h
+++ b/hypervisor/arch/riscv/include/asm/csr64.h
@@ -136,6 +136,7 @@
 #define HGATP_VMID_SHIFT   22
 #define HGATP_VMID_WIDTH   7
 
+#define SCOUNTEREN_CY  0x0001
 #define SCOUNTEREN_TM  0x0002
 
 #ifndef __ASSEMBLY__
diff --git a/hypervisor/arch/riscv/setup.c b/hypervisor/arch/riscv/setup.c
index 7b4b6a9e..f3cdfc06 100644
--- a/hypervisor/arch/riscv/setup.c
+++ b/hypervisor/arch/riscv/setup.c
@@ -2,37 +2,158 @@
  * Jailhouse, a Linux-based partitioning hypervisor
  *
  * Copyright (c) Siemens AG, 2020
+ * Copyright (c) OTH Regensburg, 2022
  *
  * Authors:
  *  Konrad Schwarz 
+ *  Ralf Ramsauer 
  *
  * This work is licensed under the terms of the GNU GPL, version 2.  See
  * the COPYING file in the top-level directory.
  */
 
+#include 
 #include 
-#include 
+#include 
+#include 
 #include 
 
+extern unsigned long bt_tbl_l0[PAGE_SIZE / sizeof(unsigned long)];
+
+void riscv_park_loop(void);
+void __attribute((noreturn))
+__riscv_deactivate_vmm(union registers *regs, int errcode, bool from_ecall);
+
 int arch_init_early(void)
 {
-   return -ENOSYS;
+   int err;
+
+   err = riscv_paging_cell_init(_cell);
+   if (err)
+   return err;
+
+   parking_pt.root_paging = root_cell.arch.mm.root_paging;
+
+   err = paging_create(_pt, paging_hvirt2phys(riscv_park_loop),
+   PAGE_SIZE, 0, PAGE_DEFAULT_FLAGS | RISCV_PTE_FLAG(G) |
+   RISCV_PTE_FLAG(U), PAGING_COHERENT | PAGING_NO_HUGE);
+
+   return 0;
 }
 
 int arch_cpu_init(struct per_cpu *cpu_data)
 {
-   return -ENOSYS;
+   struct public_per_cpu *ppc = _data->public;
+   unsigned long final_pt;
+
+   spin_init(>control_lock);
+
+   ppc->reset = false;
+   ppc->park = false;
+   ppc->wait_for_power_on = false;
+
+   ppc->phys_id =
+   jailhouse_cell_cpus(root_cell.config)[cpu_data->public.cpu_id]
+   .phys_id;
+   ppc->hsm.state = STARTED;
+
+   final_pt = paging_hvirt2phys(>root_table_page);
+   enable_mmu_satp(hv_atp_mode, final_pt);
+
+   csr_write(CSR_HSTATUS, 0);
+
+   return 0;
 }
 
-void __attribute__((noreturn))
-riscv_deactivate_vmm(union registers *regs, int errcode, bool from_ecall)
+void __attribute__ ((noreturn)) arch_cpu_activate_vmm(void)
 {
-   while (

[PATCH v3 47/73] core: forward original HV memory location

[Ralf Ramsauer]

We need this for RISC-V. Makes life easier.

Signed-off-by: Ralf Ramsauer 
---
 driver/main.c | 1 +
 hypervisor/include/jailhouse/header.h | 2 ++
 2 files changed, 3 insertions(+)

diff --git a/driver/main.c b/driver/main.c
index 51be11e2..373b77b2 100644
--- a/driver/main.c
+++ b/driver/main.c
@@ -517,6 +517,7 @@ static int jailhouse_cmd_enable(struct jailhouse_system 
__user *arg)
 
header = (struct jailhouse_header *)hypervisor_mem;
header->max_cpus = config_header.root_cell.num_cpus;
+   header->initial_load_address = (unsigned long)hypervisor_mem;
 
 #if defined(CONFIG_ARM) || defined(CONFIG_ARM64)
header->arm_linux_hyp_vectors = virt_to_phys(*__hyp_stub_vectors_sym);
diff --git a/hypervisor/include/jailhouse/header.h 
b/hypervisor/include/jailhouse/header.h
index c5285262..574aae3f 100644
--- a/hypervisor/include/jailhouse/header.h
+++ b/hypervisor/include/jailhouse/header.h
@@ -77,6 +77,8 @@ struct jailhouse_header {
 * @note Filled at build time */
void *gcov_info_head;
 
+   unsigned long initial_load_address;
+
/** Configured maximum logical CPU ID + 1.
 * @note Filled by Linux loader driver before entry. */
unsigned int max_cpus;
-- 
2.40.1

-- 
You received this message because you are subscribed to the Google Groups 
"Jailhouse" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jailhouse-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jailhouse-dev/20230519204033.643200-48-ralf.ramsauer%40oth-regensburg.de.

[PATCH v3 43/73] Documentation: Add some documentation

[Ralf Ramsauer]

Specifically on RISC-V, and fix some typos.

Signed-off-by: Ralf Ramsauer 
---
 Documentation/hypervisor-interfaces.txt | 16 ++--
 Documentation/memory-layout.txt | 20 
 2 files changed, 26 insertions(+), 10 deletions(-)

diff --git a/Documentation/hypervisor-interfaces.txt 
b/Documentation/hypervisor-interfaces.txt
index d888dab7..fe493e46 100644
--- a/Documentation/hypervisor-interfaces.txt
+++ b/Documentation/hypervisor-interfaces.txt
@@ -13,8 +13,8 @@ Detection
 -
 
 This interface is useful for cell code that should work not only inside a
-Jailhouse cell. The ABI is architecture specific. So far, it is only available
-for x86.
+Jailhouse cell. The ABI is architecture specific and not available for
+all processor architectures.
 
 
 x86 ABI
@@ -53,6 +53,18 @@ Result in EAX:  0
   EDX:  0
 
 
+RISC-V ABI
+- - - - - -
+
+Jailhouse adhers to the Supervisor Binary Interface (SBI) defined in
+https://github.com/riscv/riscv-sbi-doc/blob/master/riscv-sbi.adoc
+That is, it provides the SBI to guests (as well as using the system
+provided SBI).
+
+The sbi_get_impl_id() function returns 'JHOU', 0x4a484f55.
+The get_sbi_impl_version() function returns the version of the Jailhouse SBI.
+
+
 Hypercalls
 --
 
diff --git a/Documentation/memory-layout.txt b/Documentation/memory-layout.txt
index 82029f17..7e8281af 100644
--- a/Documentation/memory-layout.txt
+++ b/Documentation/memory-layout.txt
@@ -6,10 +6,10 @@ visible hypervisor memory, globally visible I/O memory 
remappings and CPU-
 specific remappings of hypervisor memory as well as cell memory pages.
 
 Cells are not mapped as a whole into the hypervisor address space. Only
-explicitely shared pages and pages that are temporarily mapped, e.g. during
-MMIO instruction parsing, are visible by the hypervisor during runtime.
-Furthermore, the visibility is limited to the CPU that create it because they
-are only added to the CPU-specifc mapping.
+explicitly shared pages and pages that are temporarily mapped, e.g. during
+MMIO instruction parsing, are visible to the hypervisor during runtime.
+Furthermore, their visibility is limited to the CPU that creates the mapping
+because they are only added to the CPU-specific mapping.
 
 
 Common memory region
@@ -23,9 +23,9 @@ Prior to enabling the hypervisor and after disabling it, this 
region is also
 mapped into the address space of Linux that acts as root cell. The virtual
 address of this mapping is identical to the one used by the hypervisor when the
 architecture set JAILHOUSE_BORROW_ROOT_PT (currently x86 and ARM), on other,
-this addressed can differ.
+this address can differ.
 
-The commom memory region contains an array of per-CPU data structures, one for
+The common memory region contains an array of per-CPU data structures, one for
 each configured CPU. Each per-CPU data structure consists of a private part and
 a public part. The public part will remain visible for all CPUs throughout the
 hypervisor operation. The private part, however, is only visible during setup
@@ -112,15 +112,19 @@ Size: PAGE_SIZE * NUM_REMAP_BITMAP_PAGES * PAGE_SIZE * 8
 |  |
 +--+ - higher address
 
+The architecture-independent code assumes that the I/O mememory remapping
+region is described by the same top-level PTE as the common memory region
+(JAILHOUSE_BASE).
+
 
 CPU-specific remapping region
 -
 
-This region is differently mapped for each CPU. It consistes of a virtual
+This region is mapped differently for each CPU. It consists of a virtual
 address range that is used for temporarily mapping individual pages of the cell
 that runs on the same CPU.
 
-Futhermore, the private per-CPU data which is hidden from the common memory
+Furthermore, the private per-CPU data which is hidden from the common memory
 region is made available at fixed virtual address here. This allows to
 dereference CPU local data quickly and generically. Moreover, it hides cell-
 private data that the hypervisor may collect on VM exit, e.g. register content,
-- 
2.40.1

-- 
You received this message because you are subscribed to the Google Groups 
"Jailhouse" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jailhouse-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jailhouse-dev/20230519204033.643200-44-ralf.ramsauer%40oth-regensburg.de.

[PATCH v3 46/73] core: riscv: implement control.c

[Ralf Ramsauer]

Signed-off-by: Ralf Ramsauer 
[since RFC v1: fix incorrect hsm state on cell reset]
[since RFC v2: fix parking state]
---
 hypervisor/arch/riscv/control.c | 200 ++--
 hypervisor/arch/riscv/include/asm/control.h |  13 ++
 2 files changed, 196 insertions(+), 17 deletions(-)

diff --git a/hypervisor/arch/riscv/control.c b/hypervisor/arch/riscv/control.c
index 7d4c31be..cc3ebcb2 100644
--- a/hypervisor/arch/riscv/control.c
+++ b/hypervisor/arch/riscv/control.c
@@ -2,70 +2,221 @@
  * Jailhouse, a Linux-based partitioning hypervisor
  *
  * Copyright (c) Siemens AG, 2020
+ * Copyright (c) OTH Regensburg, 2022-2023
  *
  * Authors:
- *  Jan Kiszka 
+ *  Konrad Schwarz 
+ *  Ralf Ramsauer 
  *
  * This work is licensed under the terms of the GNU GPL, version 2.  See
  * the COPYING file in the top-level directory.
  */
 
 #include 
+#include 
 #include 
+#include 
+#include 
+#include 
 #include 
 
-int arch_cell_create(struct cell *cell)
+extern void __attribute__((noreturn)) riscv_park_loop(void);
+
+static void riscv_cpu_reset(unsigned long pc, unsigned long a0,
+   unsigned long a1)
 {
-   return -ENOSYS;
+   union registers *regs;
+
+   regs = _cpu_data()->guest_regs;
+
+   memset(regs, 0, sizeof(union registers));
+   regs->pc = pc;
+   regs->a0 = a0;
+   regs->a1 = a1;
+
+   csr_write(sepc, regs->pc);
+   csr_write(CSR_VSATP, 0);
+   csr_write(CSR_HIE, 0);
+   csr_write(CSR_HIP, 0);
+   csr_write(CSR_VSSTATUS, 0);
+
+   csr_set(sstatus, SR_SPP); /* Return to VS-Mode */
+}
+
+void arch_check_events(void)
+{
+   struct public_per_cpu *pcpu;
+
+   pcpu = this_cpu_public();
+
+   spin_lock(>control_lock);
+
+   if (pcpu->suspend_cpu) {
+   pcpu->cpu_suspended = true;
+
+   spin_unlock(>control_lock);
+
+   while (pcpu->suspend_cpu)
+   cpu_relax();
+
+   spin_lock(>control_lock);
+   }
+
+   pcpu->cpu_suspended = false;
+
+   if (pcpu->wait_for_power_on) {
+   pcpu->wait_for_power_on = false;
+   goto out;
+   }
+
+   if (pcpu->hsm.state == START_PENDING) {
+   pcpu->reset = false;
+   riscv_cpu_reset(pcpu->hsm.start_addr, pcpu->phys_id,
+   pcpu->hsm.opaque);
+   riscv_paging_vcpu_init(_cell()->arch.mm);
+   pcpu->hsm.state = STARTED;
+   } else if (pcpu->park) {
+   riscv_park_cpu();
+   }
+
+   if (pcpu->reset) {
+   pcpu->reset = false;
+   riscv_cpu_reset(0x0, pcpu->phys_id, 0);
+   riscv_paging_vcpu_init(_cell()->arch.mm);
+   }
+
+out:
+   pcpu->park = false;
+   spin_unlock(>control_lock);
 }
 
 int arch_map_memory_region(struct cell *cell,
   const struct jailhouse_memory *mem)
 {
-   return -ENOSYS;
+   return paging_create(
+   >arch.mm,
+   mem->flags & JAILHOUSE_MEM_COMM_REGION ?
+   paging_hvirt2phys (>comm_page) : mem->phys_start,
+   mem->size, mem->virt_start,
+   PAGE_GUEST_PRESENT_FLAGS |
+   (mem->flags & JAILHOUSE_MEM_READ ? RISCV_PTE_FLAG(R) : 0) |
+   (mem->flags & JAILHOUSE_MEM_WRITE ? RISCV_PTE_FLAG(W) : 0) |
+   (mem->flags & JAILHOUSE_MEM_EXECUTE ? RISCV_PTE_FLAG(X) : 0),
+   PAGING_COHERENT |
+   (mem->flags & JAILHOUSE_MEM_NO_HUGEPAGES ? 0 : PAGING_HUGE));
 }
 
-int arch_unmap_memory_region(struct cell *cell,
-const struct jailhouse_memory *mem)
+int arch_unmap_memory_region(struct cell *const cell,
+const struct jailhouse_memory *mem_original)
 {
-   return -ENOSYS;
+   return paging_destroy(>arch.mm, mem_original->virt_start,
+ mem_original->size, PAGING_COHERENT);
 }
 
-void arch_check_events(void)
+void arch_flush_cell_vcpu_caches(struct cell *const cell)
 {
+   /*
+* The necessary TLB invalidation has already been performed in the
+* map/unmap routines. Doing it here would require the entire cell's
+* TLB to be flushed, because this function does not receive
+* information about the memory segment to invalidate. That would be
+* overkill (although the current unmap_region implementation does this
+* nonetheless, owing to other API shortcomings).
+*/
 }
 
-void arch_flush_cell_vcpu_caches(struct cell *cell)
+int arch_cell_create(struct cell *const cell)
 {
+   struct public_per_cpu *ppc;
+   unsigned int cpu;
+
+   cell->arch.mm.root_paging = riscv_Sv39x4;
+   cell->arch.mm.root_table =
+   page_alloc_aligned(_pool, C

[PATCH v3 45/73] core: riscv: introduce irqchip.h

[Ralf Ramsauer]

Which will contained shared definitions that are used by both, PLIC and
APLIC. The irqchip abstraction layer provides abstraction for both irq
controller variants.

Signed-off-by: Ralf Ramsauer 
---
 hypervisor/arch/riscv/include/asm/irqchip.h | 87 +
 1 file changed, 87 insertions(+)
 create mode 100644 hypervisor/arch/riscv/include/asm/irqchip.h

diff --git a/hypervisor/arch/riscv/include/asm/irqchip.h 
b/hypervisor/arch/riscv/include/asm/irqchip.h
new file mode 100644
index ..5183f5de
--- /dev/null
+++ b/hypervisor/arch/riscv/include/asm/irqchip.h
@@ -0,0 +1,87 @@
+/*
+ * Jailhouse, a Linux-based partitioning hypervisor
+ *
+ * Copyright (c) OTH Regensburg, 2022-2023
+ *
+ * Authors:
+ *  Ralf Ramsauer 
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2.  See
+ * the COPYING file in the top-level directory.
+ */
+
+/* Shared definitions for (A)PLIC */
+#define IRQCHIP_REG_SZ 4
+#define IRQCHIP_BITS_PER_REG   (IRQCHIP_REG_SZ * 8)
+
+#define IRQ_BIT(irq)   ((irq) % IRQCHIP_BITS_PER_REG)
+#define IRQ_MASK(irq)  (1 << IRQ_BIT(irq))
+
+#ifndef __ASSEMBLY__
+
+#define REG_RANGE(A, B)(A)...((B) - IRQCHIP_REG_SZ)
+
+#define SYSCONFIG_IRQCHIP  system_config->platform_info.riscv.irqchip
+
+struct irqchip {
+   int (*init)(void);
+   enum mmio_result (*mmio_handler)(void *arg, struct mmio_access *access);
+   int (*claim_irq)(void);
+   void (*adjust_irq_target)(struct cell *cell, unsigned int irq);
+
+   void *base;
+   unsigned long pending[MAX_CPUS];
+};
+
+extern struct irqchip irqchip;
+
+static inline unsigned long irqchip_type(void)
+{
+   return SYSCONFIG_IRQCHIP.type;
+}
+
+static inline unsigned long irqchip_phys(void)
+{
+   return SYSCONFIG_IRQCHIP.base_address;
+}
+
+static inline unsigned long irqchip_size(void)
+{
+   return SYSCONFIG_IRQCHIP.size;
+}
+
+static inline short irqchip_max_irq(void)
+{
+   return SYSCONFIG_IRQCHIP.max_irq;
+}
+
+static inline void irq_bitmap_set(u32 *bitmap, unsigned int irq)
+{
+   bitmap[irq / IRQCHIP_BITS_PER_REG] |= IRQ_MASK(irq);
+}
+
+static inline void irq_bitmap_clear(u32 *bitmap, unsigned int irq)
+{
+   bitmap[irq / IRQCHIP_BITS_PER_REG] &= ~IRQ_MASK(irq);
+}
+
+static inline bool irq_bitmap_test(u32 *bitmap, unsigned short irq)
+{
+   u32 val;
+
+   if (irq >= irqchip_max_irq())
+   return false;
+
+   val = bitmap[irq / IRQCHIP_BITS_PER_REG];
+
+   return !!(val & IRQ_MASK(irq));
+}
+
+static inline bool irqchip_irq_in_cell(struct cell *cell, unsigned int irq)
+{
+   return irq_bitmap_test(cell->arch.irq_bitmap, irq);
+}
+
+int irqchip_set_pending(void);
+
+#endif /* __ASSEMBLY__ */
-- 
2.40.1

-- 
You received this message because you are subscribed to the Google Groups 
"Jailhouse" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jailhouse-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jailhouse-dev/20230519204033.643200-46-ralf.ramsauer%40oth-regensburg.de.

[PATCH v3 41/73] core: riscv: implement arch_send_event

[Ralf Ramsauer]

On RISC-V, there's no possibility to parameterise IPIs resp. enrich them
with some userdata. Great. We need to do it in software...

We differentiate between two IPI types in software: IPI_CAUSE_MGMT, and
IPI_CAUSE_GUEST.

When a guest sends an IPI to a HART within the cell, the HV will trap,
and we forward the IPI call to the SBI. Before, we set the cause to
IPI_CAUSE_GUEST.

Follow the same logic for management IPIs accordingly.

Signed-off-by: Ralf Ramsauer 
---
 hypervisor/arch/riscv/control.c | 33 +
 1 file changed, 33 insertions(+)

diff --git a/hypervisor/arch/riscv/control.c b/hypervisor/arch/riscv/control.c
index b848994b..7d4c31be 100644
--- a/hypervisor/arch/riscv/control.c
+++ b/hypervisor/arch/riscv/control.c
@@ -11,6 +11,8 @@
  */
 
 #include 
+#include 
+#include 
 
 int arch_cell_create(struct cell *cell)
 {
@@ -68,4 +70,35 @@ void arch_park_cpu(unsigned int cpu_id)
 
 void arch_send_event(struct public_per_cpu *target_data)
 {
+   struct sbiret result;
+
+   /*
+* For sending an event to a remote HART, we need two conditions to be
+* set:
+*   1. We have to grab the CPU's control lock. Having the control lock
+*  ensures that the CPU is currently not processing any other IPI.
+*  If the CPU is processing an IPI, then we have to busy wait for
+*  completion.
+*   2. target_data->ipi_cause must be NONE at the moment of grabbing
+*  the lock. This ensures that no other CPU enqueued an IPI
+*  before.
+*/
+retry:
+   spin_lock(_data->control_lock);
+   if (target_data->ipi_cause != IPI_CAUSE_NONE) {
+   spin_unlock(_data->control_lock);
+   goto retry;
+   }
+
+   target_data->ipi_cause = IPI_CAUSE_MGMT;
+   memory_barrier();
+   spin_unlock(_data->control_lock);
+
+   result = sbi_send_ipi(1UL << (target_data->phys_id % BITS_PER_LONG),
+ target_data->phys_id / BITS_PER_LONG);
+   if (result.error != SBI_SUCCESS) {
+   printk("IPI send to HART %lu failed: %ld\n",
+  target_data->phys_id, result.error);
+   panic_stop();
+   }
 }
-- 
2.40.1

-- 
You received this message because you are subscribed to the Google Groups 
"Jailhouse" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jailhouse-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jailhouse-dev/20230519204033.643200-42-ralf.ramsauer%40oth-regensburg.de.

[PATCH v3 39/73] core: riscv: Hypervisor entry code

[Ralf Ramsauer]

Now we have to think of Jailhouse's memory layout.

RISC-V has a bunch of MMU variants SVxx (SV39, SV48, SV57), while xx
denotes the number of virtually adressable bits. And there's an
exceptional case: If bit xx-1 is set in a vaddr, then all bits >= XX
must be 1.

For the virtual HV location, we need a virtual address that is valid for
each MMU type. Let's virtually place the hypervisor at the topmost 16
MiB of a SV39. This results in address 0x_ffd0__.

On RISC-V, Linux could map the hypervisor to any virtual location, so we
need to setup bootstrap page tables that help us to jump to our link
location. Now this gets funny, as we need to support ALL MMU types for
the trampoline, as Linux might map us to a very high address.

The position-independent startup/trampoline code sets up a trampoline
that consists of two entries:
  - Virtual Linux Load Location -> physical HV memory
  - 0x_ffd0__   -> physical HV memory

The startup code will create and activate this table, and jump to the
destination address where it was linked. From now on, we can use
position-dependant code.

Later, those trampoline page tables will not be destroyed, as we can
reuse them when disabling the hypervisor, or when errors occur during
startup.

Signed-off-by: Ralf Ramsauer 
---
 hypervisor/arch/riscv/Kbuild  |   4 +-
 hypervisor/arch/riscv/entry.S | 349 +-
 hypervisor/arch/riscv/exception.S |  91 +
 .../arch/riscv/include/asm/jailhouse_header.h |  24 +-
 hypervisor/arch/riscv/include/asm/setup.h |  14 +
 hypervisor/arch/riscv/setup.c |  10 +-
 6 files changed, 481 insertions(+), 11 deletions(-)
 create mode 100644 hypervisor/arch/riscv/exception.S
 create mode 100644 hypervisor/arch/riscv/include/asm/setup.h

diff --git a/hypervisor/arch/riscv/Kbuild b/hypervisor/arch/riscv/Kbuild
index 75252531..7809007c 100644
--- a/hypervisor/arch/riscv/Kbuild
+++ b/hypervisor/arch/riscv/Kbuild
@@ -14,5 +14,5 @@
 
 always-y := lib.a
 
-lib-y := entry.o setup.o dbg-write.o control.o ivshmem.o paging.o pci.o traps.o
-lib-y += lib.o
+lib-y := entry.o exception.o setup.o dbg-write.o control.o ivshmem.o paging.o
+lib-y += pci.o traps.o lib.o
diff --git a/hypervisor/arch/riscv/entry.S b/hypervisor/arch/riscv/entry.S
index def55516..524270cf 100644
--- a/hypervisor/arch/riscv/entry.S
+++ b/hypervisor/arch/riscv/entry.S
@@ -2,16 +2,355 @@
  * Jailhouse, a Linux-based partitioning hypervisor
  *
  * Copyright (c) Siemens AG, 2020
+ * Copyright (c) OTH Regensburg, 2022
  *
  * Authors:
- *  Jan Kiszka 
+ *  Konrad Schwarz 
+ *  Ralf Ramsauer 
  *
  * This work is licensed under the terms of the GNU GPL, version 2.  See
  * the COPYING file in the top-level directory.
  */
 
-   .text
-   .globl arch_entry
+#include 
+#include 
+#include 
+
+#define SSTATUS_INITIAL (SR_SPIE | SR_SPP | SR_FS_DIRTY | \
+SR_VS_DIRTY | SR_XS_DIRTY | SR_UXL_64)
+
+#define PAGING_FLAGS_NEXT_LEVELRISCV_PTE_FLAG(G) | RISCV_PTE_FLAG(V)
+#define PAGING_FLAGS_BOOTSTRAP PAGE_DEFAULT_FLAGS | RISCV_PTE_FLAG(G)
+
+/*
+ * hideleg needs to be set early, otherwise vsip/vsie are forced to 0. Bits set
+ * here causes aliasing of VSIP/HIP and VSIE/HIE
+ */
+#define INITIAL_HIDELEG \
+   ((IE_SIE << VSIP_TO_HVIP_SHIFT) | \
+   (IE_TIE << VSIP_TO_HVIP_SHIFT) | \
+   (IE_EIE << VSIP_TO_HVIP_SHIFT))
+
+#define INITIAL_SIE (IE_SIE | IE_TIE | IE_EIE)
+
+.macro csr_from_csr vsreg, sreg, t
+   csrr\t, \sreg
+   csrw\vsreg, \t
+.endm
+
+.macro load_csrcsr, temp, value
+   li  \temp, \value
+   csrw\csr, \temp
+.endm
+
+.macro access_config op, reg, offset
+   \op \reg, \offset (t4)
+.endm
+
+.macro ld_constant reg, constant
+   ld  \reg, %lo (pool_\constant - constant_pool) (t6)
+.endm
+
+.macro bootstrap_page macro
+   .irp page,\
+   bt_tbl_l0,\
+   bt_tbl_l1_0,\
+   bt_tbl_l2_0,\
+   bt_tbl_l3_0,\
+   bt_tbl_l1_1,\
+   bt_tbl_l2_1,\
+   bt_tbl_l3_1
+   \macro  \page
+   .endr
+.endm
+.globl bt_tbl_l0
+
+.macro define_page page
+   .p2align RISCV_PAGE_WIDTH
+   \page:
+   .skip (1 << RISCV_PAGE_WIDTH)
+.endm
+
+.macro pool_constant symbol
+   .p2align 3
+   pool_\symbol:
+   .8byte  \symbol
+.endm
+
+/*
+ * level: 0 for 4 KiB leaves
+ * virt: virtual address to map
+ * pte: pte contents: phys addr for leaves, sub node otherwise; clobbered
+ * table: node to write pte to
+ * temp: clobberable register
+ * flags: PTE flag bits
+ */
+.macro riscv_pt_write_pte level, virt, pte, table, temp, flags
+   ori \pte, \pte, \flags
+   /* RISC-V uses 12-bit _signed_ immediates, so extracting the
+* least-significant 12 bits can't be done in a single instruction of
+* the base ABI
+  

[PATCH v3 40/73] core: riscv: implement paging structures

[Ralf Ramsauer]

From: Konrad Schwarz 

While we need support for SV39, SV48 AND SV57 in the bootstrapping
pagetables, we can later grade down to SV39.

For hypervisor paging, I chose to use SV39, as it comes with less levels
than the other ones.

For G-Stage (guest) paging, I chose SV39 as well: It is very unlikely
that we need a GP address that is very very high. Independant of the
G-Stage paging, the guest may use any available paging method that it
wants. This means, Linux may enable SV57 paging on top of a SV39
G-stage.

Signed-off-by: Ralf Ramsauer 
---
 hypervisor/arch/riscv/paging.c| 261 +-
 hypervisor/include/jailhouse/paging.h |   2 +-
 2 files changed, 259 insertions(+), 4 deletions(-)

diff --git a/hypervisor/arch/riscv/paging.c b/hypervisor/arch/riscv/paging.c
index 294ea958..e6d32246 100644
--- a/hypervisor/arch/riscv/paging.c
+++ b/hypervisor/arch/riscv/paging.c
@@ -2,22 +2,277 @@
  * Jailhouse, a Linux-based partitioning hypervisor
  *
  * Copyright (c) Siemens AG, 2020
+ * Copyright (c) OTH Regensburg, 2022
  *
  * Authors:
- *  Jan Kiszka 
+ *  Konrad Schwarz 
+ *  Ralf Ramsauer 
  *
  * This work is licensed under the terms of the GNU GPL, version 2.  See
  * the COPYING file in the top-level directory.
  */
 
+#include 
 #include 
+#include 
+
+#define HV_PAGING  riscv_Sv39
+#define HV_ATP_MODEATP_MODE_SV39
+#define CELL_PAGINGriscv_Sv39x4
+#define CELL_ATP_MODE  ATP_MODE_SV39
+
+#definePAGE_BITS   12
+#defineWORD_BITS   3 /* 1 << WORD_BITS == sizeof (void *) */
+#defineMAX_FLAG10
+#define FLAG_MASK  ((1 << MAX_FLAG) - 1)
+
+#definePAGE_LEVEL_BITS (PAGE_BITS - WORD_BITS)
+#define PAGE_LEVEL_MASK(ROOT)  ((1 << (PAGE_LEVEL_BITS + 2 * !!(ROOT))) - 1)
+
+#defineUNTRANSLATED_BITS(LEVEL) \
+   ((LEVEL) * PAGE_LEVEL_BITS + PAGE_BITS)
+
+#definePAGE_TERMINAL_FLAGS \
+   (RISCV_PTE_FLAG(R) | RISCV_PTE_FLAG(W) | RISCV_PTE_FLAG(X))
+
+unsigned char hv_atp_mode;
+static unsigned char cell_atp_mode;
+
+static inline unsigned long pte2phys(unsigned long pte)
+{
+   return (pte & ~FLAG_MASK) << (PAGE_BITS - MAX_FLAG);
+}
+
+static inline unsigned long phys2pte(unsigned long phys)
+{
+   return phys >> (PAGE_BITS - MAX_FLAG);
+}
+
+#define DEF_GET_ENTRY(NAME, LEVEL, ROOT)   \
+static pt_entry_t  \
+sv## NAME ##_vpn## LEVEL ##_get_entry(page_table_t pt, \
+   unsigned long virt) \
+{  \
+   return pt + ((virt >> UNTRANSLATED_BITS(LEVEL)) &   \
+   PAGE_LEVEL_MASK(ROOT)); \
+}
+
+DEF_GET_ENTRY(X, 0, false)
+DEF_GET_ENTRY(X, 1, false)
+DEF_GET_ENTRY(X, 2, false)
+DEF_GET_ENTRY(X, 3, false)
+
+static bool svX_entry_valid(pt_entry_t pte, unsigned long flags)
+{
+   /*
+* We must not check flags, we only need to check for the V-bit. PTEs
+* are valid, if V is set. If R/W/X is set, then it means that the PTE
+* is a leaf.
+*/
+   return !!(*pte & RISCV_PTE_FLAG(V));
+}
+
+#define DEF_SET_TERMINAL(NAME, FLAGS)  \
+static void sv## NAME ##_vpnX_set_terminal(pt_entry_t pte, \
+  unsigned long phys,  \
+  unsigned long flags) \
+{  \
+   /*  \
+* set A and D flas pre-emptively, to avoid page-faults \
+* exceptions when the hardware does not set A and D by itself  \
+*/ \
+   *pte = FLAGS | RISCV_PTE_FLAG(D) | RISCV_PTE_FLAG(A) | flags |  \
+   phys2pte(phys); \
+}
+
+DEF_SET_TERMINAL(X, 0)
+
+#define DEF_GET_PHYS(LEVEL)\
+static unsigned long   \
+svX_vpn## LEVEL ##_get_phys (pt_entry_t pte, unsigned long virt)\
+{  \
+   unsigned long entry = *pte; \
+   if (!(RISCV_PTE_FLAG(V) & entry) || \
+   !(PAGE_TERMINAL_FLAGS & (entry)))   \
+   return INVALID_PHYS_ADDR;   \
+   return pte2phys(entry) |\
+  (((1UL << UNTRANSLATED_BITS(LEVEL)) - 1) & virt);\
+}
+
+DEF_GET_PHYS(0)
+DEF_GET_PHYS(1)
+DEF_GET_PHYS(2)
+DEF_GET_PHYS(3)
+
+static unsigned long svX_get_flags(pt_entry_t pte)
+{
+   return *pte & FLAG_MASK;
+}
+
+#define DEF_SET

[PATCH v3 42/73] core: riscv: Add spinlock primitives

[Ralf Ramsauer]

Signed-off-by: Ralf Ramsauer 
[RFC v2: fix incorrect asm specifiers]
---
 hypervisor/arch/riscv/include/asm/spinlock.h | 52 +++-
 1 file changed, 50 insertions(+), 2 deletions(-)

diff --git a/hypervisor/arch/riscv/include/asm/spinlock.h 
b/hypervisor/arch/riscv/include/asm/spinlock.h
index a5e707d7..4ce2f282 100644
--- a/hypervisor/arch/riscv/include/asm/spinlock.h
+++ b/hypervisor/arch/riscv/include/asm/spinlock.h
@@ -4,7 +4,7 @@
  * Copyright (c) Siemens AG, 2020
  *
  * Authors:
- *  Jan Kiszka 
+ *  Konrad Schwarz 
  *
  * This work is licensed under the terms of the GNU GPL, version 2.  See
  * the COPYING file in the top-level directory.
@@ -13,15 +13,63 @@
 #ifndef _JAILHOUSE_ASM_SPINLOCK_H
 #define _JAILHOUSE_ASM_SPINLOCK_H
 
+#defineRISCV_USE_LR_SC 1
+
 typedef struct {
+   int unsigned spin; /* has to have offset 0 */
 } spinlock_t;
 
-static inline void spin_lock(spinlock_t *lock)
+static inline void spin_init(spinlock_t *lock)
 {
+   lock->spin = 0;
 }
 
 static inline void spin_unlock(spinlock_t *lock)
 {
+   __asm__ ("\n\
+   .if %[use_lr_sc]\n\
+   fence   rw, w\n\
+   sw  zero, %[spin]\n\
+   .else\n\
+   amoswap.w.rlx0, x0, %[spin]\n\
+   .endif\n"
+   : [spin] "=" (lock->spin):
+   [use_lr_sc] "n" (RISCV_USE_LR_SC):
+   "memory");
+}
+
+static inline void spin_lock(spinlock_t *lock)
+{
+   /* test and test and set */
+   __asm__ ("\n\
+   .if %[use_lr_sc]\n\
+\n\
+   la  t2, 1\n\
+\n\
+1: lw  t1, %[spin]\n\
+   bnezt1, 1b\n\
+\n\
+2: lr.w.aq t1, %[spin]\n\
+   bnezt1, 1b\n\
+   sc.wt1, t2, %[spin]\n\
+   bnezt1, 1b\n\
+\n\
+   .else\n\
+\n\
+   # see figure 8.2 in the RISC-V Unprivileged ISA\n\
+   li  t2, 1\n\
+\n\
+3: lw  t1, %[spin]\n\
+   bnezt1, 3b\n\
+\n\
+   amoswap.w.aqt1, t2, %[spin]\n\
+   bnezt1, 3b\n\
+\n\
+   .endif\n\
+"  :
+   [spin] "=" (lock->spin):
+   [use_lr_sc] "n" (RISCV_USE_LR_SC):
+   "t1", "t2", "memory");
 }
 
 #endif /* !_JAILHOUSE_ASM_SPINLOCK_H */
-- 
2.40.1

-- 
You received this message because you are subscribed to the Google Groups 
"Jailhouse" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jailhouse-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jailhouse-dev/20230519204033.643200-43-ralf.ramsauer%40oth-regensburg.de.

[PATCH v3 44/73] arm-common: export for_each_irqchip-macro

[Ralf Ramsauer]

We need this macro on RISC-V.

Signed-off-by: Ralf Ramsauer 
---
 hypervisor/arch/arm-common/irqchip.c   | 5 -
 hypervisor/include/jailhouse/control.h | 5 +
 2 files changed, 5 insertions(+), 5 deletions(-)

diff --git a/hypervisor/arch/arm-common/irqchip.c 
b/hypervisor/arch/arm-common/irqchip.c
index 59e5dd53..c93117d1 100644
--- a/hypervisor/arch/arm-common/irqchip.c
+++ b/hypervisor/arch/arm-common/irqchip.c
@@ -24,11 +24,6 @@
 #include 
 #include 
 
-#define for_each_irqchip(chip, config, counter)
\
-   for ((chip) = jailhouse_cell_irqchips(config), (counter) = 0;   \
-(counter) < (config)->num_irqchips;\
-(chip)++, (counter)++)
-
 spinlock_t dist_lock;
 
 void *gicd_base;
diff --git a/hypervisor/include/jailhouse/control.h 
b/hypervisor/include/jailhouse/control.h
index 2e2446f0..ddb9d039 100644
--- a/hypervisor/include/jailhouse/control.h
+++ b/hypervisor/include/jailhouse/control.h
@@ -106,6 +106,11 @@ unsigned int next_cpu(unsigned int cpu, struct cpu_set 
*cpu_set,
 (counter) < (config)->num_memory_regions;  \
 (mem)++, (counter)++)
 
+#define for_each_irqchip(chip, config, counter)
\
+   for ((chip) = jailhouse_cell_irqchips(config), (counter) = 0;   \
+(counter) < (config)->num_irqchips;\
+(chip)++, (counter)++)
+
 /**
  * Check if the CPU is assigned to the specified cell.
  * @param cell Cell the CPU may belong to.
-- 
2.40.1

-- 
You received this message because you are subscribed to the Google Groups 
"Jailhouse" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jailhouse-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jailhouse-dev/20230519204033.643200-45-ralf.ramsauer%40oth-regensburg.de.

[PATCH v3 32/73] core: riscv: implement the hypercall interface

[Ralf Ramsauer]

Nothing special here.

SBIs are parameterised bei an "Extension ID", which is typically 3-4 ASCII
characters, 'TIME', 'HSM', 'IPI', 'RFNC', …

Let's pray that no one else will implement 'JHOU' as official EID.

Then there are FIDs within the EID - Function IDs. For Jailhouse
Hypercalls, use FID = 'JHOU' and EID shall represent the hypercall
number.

Signed-off-by: Ralf Ramsauer 
---
 include/arch/riscv/asm/jailhouse_hypercall.h | 61 
 1 file changed, 49 insertions(+), 12 deletions(-)

diff --git a/include/arch/riscv/asm/jailhouse_hypercall.h 
b/include/arch/riscv/asm/jailhouse_hypercall.h
index 107d25e5..456eb6c8 100644
--- a/include/arch/riscv/asm/jailhouse_hypercall.h
+++ b/include/arch/riscv/asm/jailhouse_hypercall.h
@@ -1,10 +1,10 @@
 /*
  * Jailhouse, a Linux-based partitioning hypervisor
  *
- * Copyright (c) Siemens AG, 2020
+ * Copyright (c) OTH Regensburg, 2022
  *
  * Authors:
- *  Jan Kiszka 
+ *  Ralf Ramsauer 
  *
  * This work is licensed under the terms of the GNU GPL, version 2.  See
  * the COPYING file in the top-level directory.
@@ -36,6 +36,8 @@
  * THE POSSIBILITY OF SUCH DAMAGE.
  */
 
+#include "sbi_ecall.h"
+
 /* CPU statistics, RISC-V-specific part */
 #define JAILHOUSE_CPU_STAT_VMEXITS_SBI_IPI JAILHOUSE_GENERIC_CPU_STATS
 #define JAILHOUSE_CPU_STAT_VMEXITS_SBI_RFENCE  JAILHOUSE_GENERIC_CPU_STATS + 1
@@ -46,31 +48,66 @@
 #define JAILHOUSE_CPU_STAT_VMEXITS_VIRQ
JAILHOUSE_GENERIC_CPU_STATS + 6
 #define JAILHOUSE_NUM_CPU_STATS
JAILHOUSE_GENERIC_CPU_STATS + 7
 
+#define ASCII32(C, POS)(C << ((POS) * 8))
+#define JAILHOUSE_EID  \
+   ASCII32('J', 3) |   \
+   ASCII32('H', 2) |   \
+   ASCII32('O', 1) |   \
+   ASCII32('U', 0)
+
 #ifndef __ASSEMBLY__
 
-static inline __u32 jailhouse_call(__u32 num)
+struct jailhouse_comm_region {
+   COMM_REGION_GENERIC_HEADER;
+} __attribute__((packed));
+
+static inline unsigned long jailhouse_call(unsigned long num)
 {
-   return -ENOSYS;
+   struct sbiret ret;
+
+   ret = sbi_ecall(JAILHOUSE_EID, num, 0, 0, 0, 0, 0, 0);
+
+   return ret.error;
 }
 
-static inline __u32 jailhouse_call_arg1(__u32 num, __u32 arg1)
+static inline unsigned long
+jailhouse_call_arg1(unsigned long num, unsigned long arg1)
 {
-   return -ENOSYS;
+   struct sbiret ret;
+
+   ret = sbi_ecall(JAILHOUSE_EID, num, arg1, 0, 0, 0, 0, 0);
+
+   return ret.error;
 }
 
-static inline __u32 jailhouse_call_arg2(__u32 num, __u32 arg1, __u32 arg2)
+static inline unsigned long
+jailhouse_call_arg2(unsigned long num, unsigned long arg1, unsigned long arg2)
 {
-   return -ENOSYS;
-}
+   struct sbiret ret;
 
-struct jailhouse_comm_region {
-COMM_REGION_GENERIC_HEADER;
-} __attribute__((packed));
+   ret = sbi_ecall(JAILHOUSE_EID, num, arg1, arg2, 0, 0, 0, 0);
+
+   return ret.error;
+}
 
 static inline void
 jailhouse_send_msg_to_cell(struct jailhouse_comm_region *comm_region,
   __u32 msg)
 {
+   comm_region->reply_from_cell = JAILHOUSE_MSG_NONE;
+   /* ensure reply was cleared before sending new message */
+   asm volatile("fence rw, rw" : : : "memory");
+   comm_region->msg_to_cell = msg;
+}
+
+static inline void
+jailhouse_send_reply_from_cell(struct jailhouse_comm_region *comm_region,
+  __u32 reply)
+{
+   comm_region->msg_to_cell = JAILHOUSE_MSG_NONE;
+   /* ensure message was cleared before sending reply */
+   asm volatile("fence rw, rw" : : : "memory");
+   comm_region->reply_from_cell = reply;
 }
 
 #endif /* !__ASSEMBLY__ */
-- 
2.40.1

-- 
You received this message because you are subscribed to the Google Groups 
"Jailhouse" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jailhouse-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jailhouse-dev/20230519204033.643200-33-ralf.ramsauer%40oth-regensburg.de.

[PATCH v3 37/73] core: riscv: add asm-defines.c

[Ralf Ramsauer]

Export offsets that we will later need in assembly.

Signed-off-by: Ralf Ramsauer 
---
 hypervisor/arch/riscv/asm-defines.c| 27 ++
 hypervisor/include/jailhouse/gen-defines.h |  8 ++-
 2 files changed, 34 insertions(+), 1 deletion(-)

diff --git a/hypervisor/arch/riscv/asm-defines.c 
b/hypervisor/arch/riscv/asm-defines.c
index 17c2f256..0143d5c9 100644
--- a/hypervisor/arch/riscv/asm-defines.c
+++ b/hypervisor/arch/riscv/asm-defines.c
@@ -11,9 +11,36 @@
  */
 
 #include 
+#include 
 
 void common(void);
 
 void common(void)
 {
+   OFFSETU(REG_SP, registers, sp);
+   OFFSETU(REG_RA, registers, ra);
+   OFFSETU(REG_T0, registers, t0);
+   OFFSETU(REG_TP, registers, tp);
+   OFFSETU(REG_GP, registers, gp);
+   OFFSETU(REG_S0, registers, s0);
+   OFFSETU(REG_S1, registers, s1);
+   OFFSETU(REG_S2, registers, s2);
+   OFFSETU(REG_S3, registers, s3);
+   OFFSETU(REG_S4, registers, s4);
+   OFFSETU(REG_S5, registers, s5);
+   OFFSETU(REG_S6, registers, s6);
+   OFFSETU(REG_S7, registers, s7);
+   OFFSETU(REG_S8, registers, s8);
+   OFFSETU(REG_S9, registers, s9);
+   OFFSETU(REG_S10, registers, s10);
+   OFFSETU(REG_S11, registers, s11);
+
+   OFFSET(HEADER_MAX_CPUS, jailhouse_header, max_cpus);
+   OFFSET(HEADER_PERCPU_SIZE, jailhouse_header, percpu_size);
+
+   OFFSET(CFG_PHYS_START, jailhouse_system, hypervisor_memory.phys_start);
+
+   OFFSET(PCPU_GUEST_REGS, per_cpu, guest_regs);
+
+   DEFINE(REGISTERS_SIZE, sizeof(union registers));
 }
diff --git a/hypervisor/include/jailhouse/gen-defines.h 
b/hypervisor/include/jailhouse/gen-defines.h
index d42fe255..7b348761 100644
--- a/hypervisor/include/jailhouse/gen-defines.h
+++ b/hypervisor/include/jailhouse/gen-defines.h
@@ -22,8 +22,14 @@
 
 #define BLANK() asm volatile("\n=>" : : )
 
+#define OFF(type, sym, str, mem) \
+   DEFINE(sym, __builtin_offsetof(type str, mem))
+
 #define OFFSET(sym, str, mem) \
-   DEFINE(sym, __builtin_offsetof(struct str, mem))
+   OFF(struct, sym, str, mem)
+
+#define OFFSETU(sym, str, mem) \
+   OFF(union , sym, str, mem)
 
 #define COMMENT(x) \
asm volatile("\n=>#" x)
-- 
2.40.1

-- 
You received this message because you are subscribed to the Google Groups 
"Jailhouse" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jailhouse-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jailhouse-dev/20230519204033.643200-38-ralf.ramsauer%40oth-regensburg.de.

[PATCH v3 33/73] core: riscv: introduce cell-specific fields

[Ralf Ramsauer]

Add RISC-V cell-specific fields.

Unlike ARM, for example, the PLIC doesn't support hardware injection of
external IRQs. So we need to do it in software, sigh...

Prepare the fields that are needed for software injection. The
irq_bitmap, just like on ARM, denotes physical IRQs that the cell is
allowed to manage.

Signed-off-by: Ralf Ramsauer 
---
 hypervisor/arch/riscv/include/asm/cell.h  | 11 +++
 hypervisor/arch/riscv/include/asm/types.h |  1 +
 2 files changed, 8 insertions(+), 4 deletions(-)

diff --git a/hypervisor/arch/riscv/include/asm/cell.h 
b/hypervisor/arch/riscv/include/asm/cell.h
index 63ba43ce..9da228d4 100644
--- a/hypervisor/arch/riscv/include/asm/cell.h
+++ b/hypervisor/arch/riscv/include/asm/cell.h
@@ -2,9 +2,11 @@
  * Jailhouse, a Linux-based partitioning hypervisor
  *
  * Copyright (c) Siemens AG, 2020
+ * Copyright (c) OTH Regensburg, 2022
  *
  * Authors:
- *  Jan Kiszka 
+ *  Konrad Schwarz 
+ *  Ralf Ramsauer 
  *
  * This work is licensed under the terms of the GNU GPL, version 2.  See
  * the COPYING file in the top-level directory.
@@ -13,12 +15,13 @@
 #ifndef _JAILHOUSE_ASM_CELL_H
 #define _JAILHOUSE_ASM_CELL_H
 
+#include 
 #include 
 
-// this shouldn't be here
-#include 
-
 struct arch_cell {
+   struct paging_structures mm;
+
+   u32 irq_bitmap[MAX_IRQS / (sizeof(u32) * 8)];
 };
 
 #endif /* !_JAILHOUSE_ASM_CELL_H */
diff --git a/hypervisor/arch/riscv/include/asm/types.h 
b/hypervisor/arch/riscv/include/asm/types.h
index 120859e0..adf58120 100644
--- a/hypervisor/arch/riscv/include/asm/types.h
+++ b/hypervisor/arch/riscv/include/asm/types.h
@@ -11,4 +11,5 @@
  */
 
 #define MAX_CPUS   __riscv_xlen
+#define MAX_IRQS   1024
 #define BITS_PER_LONG  64
-- 
2.40.1

-- 
You received this message because you are subscribed to the Google Groups 
"Jailhouse" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jailhouse-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jailhouse-dev/20230519204033.643200-34-ralf.ramsauer%40oth-regensburg.de.

[PATCH v3 35/73] core: riscv: implement library routines

[Ralf Ramsauer]

On RISC-V, the logical CPU ID should be the same as Linux's ID. A
logical CPU ID is mapped to a HART, the physical ID (phys_id).

All communication with the SBI interface require specification of
physical HART IDs.

Signed-off-by: Ralf Ramsauer 
---
 hypervisor/arch/riscv/control.c |  7 ---
 hypervisor/arch/riscv/lib.c | 25 +
 2 files changed, 21 insertions(+), 11 deletions(-)

diff --git a/hypervisor/arch/riscv/control.c b/hypervisor/arch/riscv/control.c
index ab0bb449..b848994b 100644
--- a/hypervisor/arch/riscv/control.c
+++ b/hypervisor/arch/riscv/control.c
@@ -69,10 +69,3 @@ void arch_park_cpu(unsigned int cpu_id)
 void arch_send_event(struct public_per_cpu *target_data)
 {
 }
-
-// Likely misplaced
-#include 
-unsigned int cpu_by_phys_processor_id(u64 phys_id)
-{
-   return -1;
-}
diff --git a/hypervisor/arch/riscv/lib.c b/hypervisor/arch/riscv/lib.c
index 5fe45bc7..2ea1d5b6 100644
--- a/hypervisor/arch/riscv/lib.c
+++ b/hypervisor/arch/riscv/lib.c
@@ -1,19 +1,36 @@
 /*
  * Jailhouse, a Linux-based partitioning hypervisor
  *
- * Copyright (c) Siemens AG, 2020
+ * Copyright (c) OTH Regensburg, 2022
  *
  * Authors:
- *  Jan Kiszka 
+ *  Ralf Ramsauer 
  *
  * This work is licensed under the terms of the GNU GPL, version 2.  See
  * the COPYING file in the top-level directory.
  */
 
+#include 
+#include 
 #include 
+#include 
 
-// Check if it be moved elsewhere
 unsigned long phys_processor_id(void)
 {
-   return 0;
+   return this_cpu_public()->phys_id;
+}
+
+unsigned int cpu_by_phys_processor_id(u64 phys)
+{
+   const struct jailhouse_cpu *map = jailhouse_cell_cpus(root_cell.config);
+   unsigned int num_cpus = root_cell.config->num_cpus;
+   unsigned int cpu;
+
+   for (cpu = 0; cpu < num_cpus; cpu++, map++)
+   if (map->phys_id == phys)
+   return cpu;
+
+   /* this should never happen */
+   panic_printk("FATAL: unknown HART ID %llu\n", phys);
+   panic_stop();
 }
-- 
2.40.1

-- 
You received this message because you are subscribed to the Google Groups 
"Jailhouse" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jailhouse-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jailhouse-dev/20230519204033.643200-36-ralf.ramsauer%40oth-regensburg.de.

[PATCH v3 38/73] core: riscv: introduce traps.c

[Ralf Ramsauer]

This commit add the stubs for two exception handlers: arch_handle_trap
and arch_handle_fault.

Rationale:

On RISC-V, there's a scratch register (CSR), which is typically user by
the supervisor to store its stack pointer. On traps, the guest/user's SP
is atomically exchanged with the scratch register, to set up the
supervisor's stack.

Now, in case of faults when we're inside Jailhouse, we would have to do
the same trick, but, in case of double faults we're doomed anyway.

So instead of preparing a new scratch register when entering the
Hypervisor, I chose to simply redirect the trap vector and reuse the
stack, while the hypervisor is active.

So if we're trapping the hypervisor, we will later end up in
arch_handle_trap. If we get a fault while we're inside the hypervisor,
we will end up in arch_handle_fault.

These routines will be called from assembly.

Signed-off-by: Ralf Ramsauer 
---
 hypervisor/arch/riscv/Kbuild  |  3 ++-
 hypervisor/arch/riscv/traps.c | 28 
 2 files changed, 30 insertions(+), 1 deletion(-)
 create mode 100644 hypervisor/arch/riscv/traps.c

diff --git a/hypervisor/arch/riscv/Kbuild b/hypervisor/arch/riscv/Kbuild
index dd114a6c..75252531 100644
--- a/hypervisor/arch/riscv/Kbuild
+++ b/hypervisor/arch/riscv/Kbuild
@@ -14,4 +14,5 @@
 
 always-y := lib.a
 
-lib-y := entry.o setup.o dbg-write.o control.o ivshmem.o paging.o pci.o lib.o
+lib-y := entry.o setup.o dbg-write.o control.o ivshmem.o paging.o pci.o traps.o
+lib-y += lib.o
diff --git a/hypervisor/arch/riscv/traps.c b/hypervisor/arch/riscv/traps.c
new file mode 100644
index ..f61dfc1a
--- /dev/null
+++ b/hypervisor/arch/riscv/traps.c
@@ -0,0 +1,28 @@
+/*
+ * Jailhouse, a Linux-based partitioning hypervisor
+ *
+ * Copyright (c) OTH Regensburg, 2022
+ *
+ * Authors:
+ *  Ralf Ramsauer 
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2.  See
+ * the COPYING file in the top-level directory.
+ */
+
+#include 
+
+void arch_handle_trap(union registers *regs);
+void arch_handle_fault(union registers *regs);
+
+void arch_handle_trap(union registers *regs)
+{
+   for (;;)
+   cpu_relax();
+}
+
+void arch_handle_fault(union registers *regs)
+{
+   for (;;)
+   cpu_relax();
+}
-- 
2.40.1

-- 
You received this message because you are subscribed to the Google Groups 
"Jailhouse" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jailhouse-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jailhouse-dev/20230519204033.643200-39-ralf.ramsauer%40oth-regensburg.de.

[PATCH v3 36/73] core: riscv: paging: add headers

[Ralf Ramsauer]

Signed-off-by: Ralf Ramsauer 
---
 hypervisor/arch/riscv/include/asm/paging.h| 106 --
 .../arch/riscv/include/asm/paging_modes.h |  18 +++
 hypervisor/include/jailhouse/header.h |   5 +
 3 files changed, 118 insertions(+), 11 deletions(-)

diff --git a/hypervisor/arch/riscv/include/asm/paging.h 
b/hypervisor/arch/riscv/include/asm/paging.h
index 866f9a3f..b34173e8 100644
--- a/hypervisor/arch/riscv/include/asm/paging.h
+++ b/hypervisor/arch/riscv/include/asm/paging.h
@@ -4,7 +4,7 @@
  * Copyright (c) Siemens AG, 2020
  *
  * Authors:
- *  Jan Kiszka 
+ *  Konrad Schwarz 
  *
  * This work is licensed under the terms of the GNU GPL, version 2.  See
  * the COPYING file in the top-level directory.
@@ -14,41 +14,125 @@
 #define _JAILHOUSE_ASM_PAGING_H
 
 #include 
+#include 
+#include 
 
 #define PAGE_SHIFT 12
 
-#define MAX_PAGE_TABLE_LEVELS  3
+#define RISCV_PAGE_SV_DEPTH39
+#define RISCV_HV_PAGE_SV_DEPTH 39
 
+/* SV32 requires 2, SV39 3, SV48 4. Once we support SV57, this should be 5. */
+#define MAX_PAGE_TABLE_LEVELS  4
 #define PAGE_FLAG_FRAMEBUFFER  0
+
+/*
+ * RISC-V does not insert caching/prefetching/... information into page tables.
+ * Instead, this is done via `Physical Memory Attributes'. However, the ISA
+ * does not define a concrete form of PMAs.  E.g., they could be hardcoded by
+ * physical address in a SoC.
+ */
 #define PAGE_FLAG_DEVICE   0
 
-#define PAGE_DEFAULT_FLAGS 0
-#define PAGE_READONLY_FLAGS0
-#define PAGE_PRESENT_FLAGS 0
+#define RISCV_PAGE_WIDTH   12
+#define RISCV_PTE_SIZE 3
+#define RISCV_PTES_PER_NODE(RISCV_PAGE_WIDTH - RISCV_PTE_SIZE)
+
+#define RISCV_PTE_V0
+#define RISCV_PTE_R1
+#define RISCV_PTE_W2
+#define RISCV_PTE_X3
+#define RISCV_PTE_U4
+#define RISCV_PTE_G5
+#define RISCV_PTE_A6
+#define RISCV_PTE_D7
+
+#define RISCV_PTE_FLAG(FLAG)   (1 << RISCV_PTE_ ## FLAG)
+
+
+#define PAGE_PRESENT_FLAGS (RISCV_PTE_FLAG(V))
 #define PAGE_NONPRESENT_FLAGS  0
+#define PAGE_GUEST_PRESENT_FLAGS   (PAGE_PRESENT_FLAGS | RISCV_PTE_FLAG(U))
+
+#definePAGE_DEFAULT_FLAGS  (PAGE_PRESENT_FLAGS | RISCV_PTE_FLAG(R) 
| \
+RISCV_PTE_FLAG(W) | RISCV_PTE_FLAG(X))
+
+#define PAGE_READONLY_FLAGS(PAGE_PRESENT_FLAGS | RISCV_PTE_FLAG(R))
+
 
 #define INVALID_PHYS_ADDR  (~0UL)
 
-#define TEMPORARY_MAPPING_BASE 0x0080UL
+#define TEMPORARY_MAPPING_BASE (~0UL << (RISCV_PAGE_SV_DEPTH - 1))
 #define NUM_TEMPORARY_PAGES16
 
-#define REMAP_BASE 0xff80UL
 #define NUM_REMAP_BITMAP_PAGES 4
+/*
+ * REMAP_BASE hast to be described by the same top-level PTE as JAILHOUSE_BASE,
+ * otherwise it won't be added to non-CPU 0 page tables automatically
+ */
+#define REMAP_BASE\
+   (JAILHOUSE_BASE & ~0UL << (RISCV_HV_PAGE_SV_DEPTH - 9))
 
-#define CELL_ROOT_PT_PAGES 1
+#if REMAP_BASE + (NUM_REMAP_BITMAP_PAGES * 1UL << /*
+   */   12 /* ld bytes/page
+   */ +  3 /* ld bits/byte
+   */ + 12 /* each bit represents a page
+   */) > JAILHOUSE_BASE
+
+# error Overlap between REMAP area and JAILHOUSE_BASE!
+#endif
+
+#define CELL_ROOT_PT_PAGES (1 << 2)
+
+#define ATP_MODE_SHIFT 60
+#define ATP_MODE_SV39  0x8
+#define ATP_MODE_SV48  0x9
+#define ATP_MODE_SV57  0xa
 
 #ifndef __ASSEMBLY__
 
-typedef u64 *pt_entry_t;
+#include 
 
-static inline void arch_paging_flush_page_tlbs(unsigned long page_addr)
+typedef size_t *pt_entry_t;
+
+/* MMU mode for Jailhouse (S-Mode) */
+extern unsigned char hv_atp_mode;
+
+static inline void arch_paging_flush_page_tlbs(size_t page_addr)
 {
+   asm volatile("sfence.vma /* rd, */ zero, %[addr]" :
+: [addr] "r" (page_addr));
 }
 
-static inline void arch_paging_flush_cpu_caches(void *addr, long size)
+/*
+ * In RISC-V, the MMU accesses page tables through the caches (the MMU is a
+ * coherent agent)
+ */
+static inline void arch_paging_flush_cpu_caches(void const *addr, size_t size)
 {
 }
 
+#define ENABLE_MMU(NAME, REG)  \
+static inline void enable_mmu_##NAME(u8 mode, unsigned long pt)
\
+{  \
+   u64 atp;\
+   \
+   atp = (u64)mode << ATP_MODE_SHIFT | (u64)pt >> PAGE_SHIFT;  \
+   asm volatile("sfence.vma\n" \
+"csrw %0, %1\n"\
+"sfence.vma\n" \
+: : "i"(REG), &q

[PATCH v3 27/73] core: riscv: add processor.h

[Ralf Ramsauer]

Add processor.h. We have 32x 64-Bit registers, x0-x32. x0 is the zero
register. In our case, let's place the program counter at its location.

In cpu_relax, simply do a dummy division, like Linux does. Though there
is a pause instruction in the Zihintpause extensions available, we would
have (a) to discover it and (b) statically patch the instruction to the
routine. Simply don't use it for those reasons...

Signed-off-by: Ralf Ramsauer 
[since RFC v1: use div for cpu_relax()]
[since RFC v2: add interrupt control routines]
---
 hypervisor/arch/riscv/include/asm/processor.h | 71 ++-
 1 file changed, 69 insertions(+), 2 deletions(-)

diff --git a/hypervisor/arch/riscv/include/asm/processor.h 
b/hypervisor/arch/riscv/include/asm/processor.h
index 8d4b1c60..ed0d9a36 100644
--- a/hypervisor/arch/riscv/include/asm/processor.h
+++ b/hypervisor/arch/riscv/include/asm/processor.h
@@ -2,9 +2,11 @@
  * Jailhouse, a Linux-based partitioning hypervisor
  *
  * Copyright (c) Siemens AG, 2020
+ * Copyright (c) OTH Regensburg, 2022
  *
  * Authors:
- *  Jan Kiszka 
+ *  Konrad Schwarz 
+ *  Ralf Ramsauer 
  *
  * This work is licensed under the terms of the GNU GPL, version 2.  See
  * the COPYING file in the top-level directory.
@@ -14,20 +16,85 @@
 #define _JAILHOUSE_ASM_PROCESSOR_H
 
 #include 
+#include 
 
 union registers {
-};
+   struct {
+   unsigned long pc; /* x0 is zero register, in our case it's PC */
+   unsigned long ra;
+   unsigned long sp;
+   unsigned long gp;
+   unsigned long tp;
+   unsigned long t0;
+   unsigned long t1;
+   unsigned long t2;
+   unsigned long s0;
+   unsigned long s1;
+   unsigned long a0;
+   unsigned long a1;
+   unsigned long a2;
+   unsigned long a3;
+   unsigned long a4;
+   unsigned long a5;
+   unsigned long a6;
+   unsigned long a7;
+   unsigned long s2;
+   unsigned long s3;
+   unsigned long s4;
+   unsigned long s5;
+   unsigned long s6;
+   unsigned long s7;
+   unsigned long s8;
+   unsigned long s9;
+   unsigned long s10;
+   unsigned long s11;
+   unsigned long t3;
+   unsigned long t4;
+   unsigned long t5;
+   unsigned long t6;
+   };
+   struct riscv_raw_registers {
+   unsigned long x[32];
+   } raw;
+} __attribute__ ((__aligned__ (1 << (7 - 3) /* bits/byte */)));
+/* RISC-V ABI requires 128-bit stack alignment */
 
 static inline void cpu_relax(void)
 {
+   int dummy;
+
+   /* In lieu of a halt instruction, induce a long-latency stall. */
+   asm volatile("div %0, %0, zero" : "=r" (dummy));
 }
 
 static inline void memory_barrier(void)
 {
+   asm volatile("fence rw, rw" : : : "memory");
 }
 
 static inline void memory_load_barrier(void)
 {
+   asm volatile("fence r, rw" : : : "memory");
+}
+
+static inline void guest_inject_ext(void)
+{
+   csr_set(CSR_HVIP, (1 << IRQ_S_EXT) << VSIP_TO_HVIP_SHIFT);
+}
+
+static inline void guest_clear_ext(void)
+{
+   csr_clear(CSR_HVIP, (1 << IRQ_S_EXT) << VSIP_TO_HVIP_SHIFT);
+}
+
+static inline void ext_disable(void)
+{
+   csr_clear(sie, IE_EIE);
+}
+
+static inline void ext_enable(void)
+{
+   csr_set(sie, IE_EIE);
 }
 
 #endif /* !_JAILHOUSE_ASM_PROCESSOR_H */
-- 
2.40.1

-- 
You received this message because you are subscribed to the Google Groups 
"Jailhouse" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jailhouse-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jailhouse-dev/20230519204033.643200-28-ralf.ramsauer%40oth-regensburg.de.