[RFC v1 21/42] core: riscv: implement arch_send_event

On RISC-V, there's no possibility to parameterise IPIs resp. enrich them with some userdata. Great. We need to do it in software... We differentiate between two IPI types in software: IPI_CAUSE_MGMT, and IPI_CAUSE_GUEST. When a guest sends an IPI to a HART within the cell, the HV will trap, and

[RFC v1 24/42] core: riscv: implement control.c

--- hypervisor/arch/riscv/control.c | 213 ++-- hypervisor/arch/riscv/include/asm/control.h | 13 ++ 2 files changed, 209 insertions(+), 17 deletions(-) diff --git a/hypervisor/arch/riscv/control.c b/hypervisor/arch/riscv/control.c index 263708a7..fa9921e6 100644 ---

[RFC v1 16/42] core: riscv: paging: add headers

Signed-off-by: Ralf Ramsauer --- hypervisor/arch/riscv/include/asm/paging.h| 119 -- .../arch/riscv/include/asm/paging_modes.h | 18 +++ hypervisor/include/jailhouse/header.h | 5 + 3 files changed, 130 insertions(+), 12 deletions(-) diff --git

Re: [RFC v1 40/42] core: riscv: plic: introduce vIRQ support

On 27.06.22 15:29, Ralf Ramsauer wrote: > diff --git a/hypervisor/arch/riscv/include/asm/percpu.h > b/hypervisor/arch/riscv/include/asm/percpu.h > index 4eda15b6..bcafff51 100644 > --- a/hypervisor/arch/riscv/include/asm/percpu.h > +++ b/hypervisor/arch/riscv/include/asm/percpu.h > @@ -43,6 +43,8

Re: [RFC v1 34/42] configs: riscv: qemu: add system configs

On 27.06.22 15:28, Ralf Ramsauer wrote: > Signed-off-by: Ralf Ramsauer > --- > configs/riscv/qemu-riscv64.c | 218 +-- > include/jailhouse/console.h | 1 + > 2 files changed, 211 insertions(+), 8 deletions(-) > ... > diff --git a/include/jailhouse/console.h

[RFC v1 33/42] riscv: inmates: implement Linux Loader

The boot protocol is simple: - a0 hold the hart id - a1 hold the DTB That's it. Signed-off-by: Ralf Ramsauer --- inmates/tools/riscv/Makefile | 19 +++ inmates/tools/riscv/linux-loader.c | 25 + 2 files changed, 44 insertions(+) create mode

[RFC v1 31/42] inmates: riscv: add timer demo

Same demo as for other architectures. Signed-off-by: Ralf Ramsauer --- inmates/demos/riscv/Makefile | 3 +- inmates/demos/riscv/timer-demo.c | 57 +++ inmates/lib/riscv/Makefile | 2 +- inmates/lib/riscv/header.S | 29

[RFC v1 17/42] core: riscv: add asm-defines.c

Export offsets that we will later need in assembly. Signed-off-by: Ralf Ramsauer --- hypervisor/arch/riscv/asm-defines.c| 27 ++ hypervisor/include/jailhouse/gen-defines.h | 8 ++- 2 files changed, 34 insertions(+), 1 deletion(-) diff --git

[RFC v1 38/42] configs: riscv: noelv: add tiny demo

--- configs/riscv/noelv-tiny-demo.c | 63 + 1 file changed, 63 insertions(+) create mode 100644 configs/riscv/noelv-tiny-demo.c diff --git a/configs/riscv/noelv-tiny-demo.c b/configs/riscv/noelv-tiny-demo.c new file mode 100644 index ..4eedbed2 ---

Re: [EXT] Re: [RFC v1 03/42] core: riscv: define MMIO accessors

On 27/06/2022 15:43, Jan Kiszka wrote: On 27.06.22 15:28, Ralf Ramsauer wrote: GCC may emit 4x 1 Byte reads in case of our regular mmio_read32 accessor, when used in combination with -Os. Yes, I've seen it. Define safe handlers to overcome this issue. The kernel does something similar,

[RFC v1 01/42] driver: RISC-V: check for availability of h-extensions

Like on all other architectures, check for the availability of h-extensions in the driver to early prevent damage, if h-extensions are not available. Signed-off-by: Ralf Ramsauer --- driver/main.c | 7 +++ 1 file changed, 7 insertions(+) diff --git a/driver/main.c b/driver/main.c index

[RFC v1 14/42] core: riscv: implement library routines

On RISC-V, the logical CPU ID should be the same as Linux's ID. A logical CPU ID is mapped to a HART, the physical ID (phys_id). All communication with the SBI interface require specification of physical HART IDs. Signed-off-by: Ralf Ramsauer --- hypervisor/arch/riscv/control.c | 7 ---

[RFC v1 02/42] driver: sysfs: prepare for exit accounting on RISC-V

Besides generic traps, we will have a bunch of RISC-V specific traps: Platform handling: - VMEXITS_SBI: Calls from the guest (VS-Mode) to the SBI firmware IRQ handling: - VMEXITS_TIMER: Timers will arrive in S-Mode - VMEXITS_IPI: IPIs arrive in S-Mode - VMEXITS_VIRQ: external IRQs arrive in

[RFC v1 13/42] core: riscv: add dbg-write helper

For the moment, only support the fast output path that is always available: SBI. No UARTs needed for the hypervisor, SBI suffices atm. Signed-off-by: Ralf Ramsauer --- hypervisor/arch/riscv/dbg-write.c | 14 -- 1 file changed, 12 insertions(+), 2 deletions(-) diff --git

[RFC v1 20/42] core: riscv: implement paging structures

From: Konrad Schwarz While we need support for SV39, SV48 AND SV57 in the bootstrapping pagetables, we can later grade down to SV39. For hypervisor paging, I chose to use SV39, as it comes with less levels than the other ones. For G-Stage (guest) paging, I chose SV39 as well: It is very

[RFC v1 18/42] core: riscv: introduce traps.c

This commit add the stubs for two exception handlers: arch_handle_trap and arch_handle_fault. Rationale: On RISC-V, there's a scratch register (CSR), which is typically user by the supervisor to store its stack pointer. On traps, the guest/user's SP is atomically exchanged with the scratch

[RFC v1 29/42] core: riscv: implement trap handlers

Signed-off-by: Ralf Ramsauer --- hypervisor/arch/riscv/traps.c | 784 +- 1 file changed, 779 insertions(+), 5 deletions(-) diff --git a/hypervisor/arch/riscv/traps.c b/hypervisor/arch/riscv/traps.c index f61dfc1a..8f59a675 100644 ---

[RFC v1 26/42] core: riscv: implement setup.c

And do some heavy liftig. Signed-off-by: Ralf Ramsauer --- hypervisor/arch/riscv/setup.c | 198 -- 1 file changed, 190 insertions(+), 8 deletions(-) diff --git a/hypervisor/arch/riscv/setup.c b/hypervisor/arch/riscv/setup.c index 985f27e2..dadffc70 100644 ---

[RFC v1 28/42] riscv: Add plic implementation

The design of the PLIC is poor: We basically need to trap & moderate each access. The strategy is as follows: On IRQ arrival in S-Mode, we directly acknowledge the IRQ, save it in a shadow register, and reinject it to the VS-Mode guest. Now disable IRQs for S-Mode, until the guest has claimed

[RFC v1 09/42] core: riscv: introduce asm/sbi.h

Add helpers for straight-forward human-readable SBI calls. Signed-off-by: Ralf Ramsauer --- hypervisor/arch/riscv/include/asm/sbi.h | 84 + 1 file changed, 84 insertions(+) create mode 100644 hypervisor/arch/riscv/include/asm/sbi.h diff --git

[RFC v1 30/42] riscv: inmates: implement tiny-demo RISC-V

From: Stefan Huber add necessary files for developing tiny-demo inmate. Consists of modified copies from x86/arm. Signed-off-by: Stefan Huber Signed-off-by: Ralf Ramsauer --- .gitignore | 1 + inmates/demos/riscv/Makefile| 19

[RFC v1 36/42] configs: riscv: qemu: Add linux-demo

Signed-off-by: Ralf Ramsauer --- configs/riscv/dts/qemu-linux-inmate.dts | 135 configs/riscv/qemu-linux-demo.c | 101 ++ 2 files changed, 236 insertions(+) create mode 100644 configs/riscv/dts/qemu-linux-inmate.dts create mode 100644

[RFC v1 41/42] configs: riscv: add ivshmem-net devices

For demonstration purposes only. Signed-off-by: Ralf Ramsauer --- configs/riscv/dts/noelv-linux-inmate.dts | 17 + configs/riscv/noelv-linux-demo.c | 12 +++- configs/riscv/noelv.c| 12 +++- configs/riscv/qemu-linux-demo.c |

[RFC v1 32/42] riscv: inmates: propagate hartid

We will need the hartid for the linux-loader later. Signed-off-by: Ralf Ramsauer --- inmates/lib/riscv/header.S | 3 +++ inmates/lib/riscv/include/inmate.h | 2 ++ inmates/lib/riscv/setup.c | 2 ++ 3 files changed, 7 insertions(+) diff --git a/inmates/lib/riscv/header.S

[RFC v1 40/42] core: riscv: plic: introduce vIRQ support

For demonstration purposes only. Signed-off-by: Ralf Ramsauer --- hypervisor/arch/riscv/include/asm/cell.h| 1 + hypervisor/arch/riscv/include/asm/ivshmem.h | 1 + hypervisor/arch/riscv/include/asm/percpu.h | 4 +- hypervisor/arch/riscv/include/asm/plic.h| 7 +

[RFC v1 10/42] core: riscv: add platform information

It's pretty easy on RISC-V: All platform specific confi information that we need is the PLIC. Base/size fields are obvious, max_irq defines the highest possible IRQ on the controller. max_priority the highest possible priority of an IRQ. hart_to_context is a map that maps physical HART IDs to

[RFC v1 34/42] configs: riscv: qemu: add system configs

Signed-off-by: Ralf Ramsauer --- configs/riscv/qemu-riscv64.c | 218 +-- include/jailhouse/console.h | 1 + 2 files changed, 211 insertions(+), 8 deletions(-) diff --git a/configs/riscv/qemu-riscv64.c b/configs/riscv/qemu-riscv64.c index 3c68ad71..1ace554b

[RFC v1 25/42] core: forward original HV memory location

We need this for RISC-V. Makes life easier. Signed-off-by: Ralf Ramsauer --- driver/main.c | 1 + hypervisor/include/jailhouse/header.h | 2 ++ 2 files changed, 3 insertions(+) diff --git a/driver/main.c b/driver/main.c index ef1c2a90..d5c823f2 100644 ---

[RFC v1 22/42] core: riscv: Add spinlock primitives

Signed-off-by: Ralf Ramsauer --- hypervisor/arch/riscv/include/asm/spinlock.h | 52 +++- 1 file changed, 50 insertions(+), 2 deletions(-) diff --git a/hypervisor/arch/riscv/include/asm/spinlock.h b/hypervisor/arch/riscv/include/asm/spinlock.h index a5e707d7..db9b33bc 100644 ---

[RFC v1 03/42] core: riscv: define MMIO accessors

GCC may emit 4x 1 Byte reads in case of our regular mmio_read32 accessor, when used in combination with -Os. Yes, I've seen it. Define safe handlers to overcome this issue. Signed-off-by: Ralf Ramsauer --- hypervisor/arch/riscv/include/asm/mmio.h | 67

[RFC v1 07/42] core: riscv: Define percpu fields

Nothing special, but two important things: the ipi_cause and the hsm field. There's only one single IPI, and we can't differentiate between different IPI causes. That makes things hard: if an IPI arrives at a HART, we don't know if it is a management IPI or if the IPI is for our guest. Hence,

[RFC v1 04/42] core: riscv: Add CSR definitions

RISC-V has a bunch of Control and Status Registers (CSRs). Add definitions and accessors for those registers. We will need them later. These definitions are copied over from the Linux kernel. Signed-off-by: Ralf Ramsauer --- hypervisor/arch/riscv/include/asm/csr64.h | 196

[RFC v1 06/42] core: riscv: add processor.h

Add processor.h. We have 32x 64-Bit registers, x0-x32. x0 is the zero register. In our case, let's place the program counter at its location. Signed-off-by: Ralf Ramsauer --- hypervisor/arch/riscv/include/asm/processor.h | 47 ++- 1 file changed, 45 insertions(+), 2 deletions(-)

[RFC v1 00/42] RISC-V support for Jailhouse

tl;dr: this is a first approach towards RISC-V support in Jailhouse. It works, but there are still some open architectural discussions that have to be addressed. Hi, this is the first RFC series for RISC-V support in Jailhouse. The series was initially developed by Konrad Schwarz at SIEMENS in

[RFC v1 05/42] core: riscv: introduce asm/bitops.h

Signed-off-by: Ralf Ramsauer --- hypervisor/arch/riscv/include/asm/bitops.h | 54 ++ 1 file changed, 46 insertions(+), 8 deletions(-) diff --git a/hypervisor/arch/riscv/include/asm/bitops.h b/hypervisor/arch/riscv/include/asm/bitops.h index 4b7e31b1..f62f382a 100644 ---

[RFC v1 15/42] core: riscv: implement PCI support

Basically the same like on arm systems. Signed-off-by: Ralf Ramsauer --- hypervisor/arch/riscv/pci.c | 25 + 1 file changed, 21 insertions(+), 4 deletions(-) diff --git a/hypervisor/arch/riscv/pci.c b/hypervisor/arch/riscv/pci.c index 067a7651..fc6ed5fa 100644 ---

[RFC v1 42/42] driver: riscv: Add vPCI support

For demonstration purposes only. Signed-off-by: Ralf Ramsauer --- driver/pci.c | 33 + 1 file changed, 21 insertions(+), 12 deletions(-) diff --git a/driver/pci.c b/driver/pci.c index da516479..4f52eba8 100644 --- a/driver/pci.c +++ b/driver/pci.c @@ -292,19

[RFC v1 39/42] configs: riscv: noelv: Add linux inmate config

Signed-off-by: Ralf Ramsauer --- configs/riscv/dts/noelv-linux-inmate.dts | 130 +++ configs/riscv/noelv-linux-demo.c | 101 ++ 2 files changed, 231 insertions(+) create mode 100644 configs/riscv/dts/noelv-linux-inmate.dts create mode 100644

[RFC v1 08/42] core: riscv: introduce global sbi_ecall.h

This routines will later also be shared with inmates, so implement it globally. Signed-off-by: Ralf Ramsauer --- include/arch/riscv/asm/sbi_ecall.h | 96 ++ 1 file changed, 96 insertions(+) create mode 100644 include/arch/riscv/asm/sbi_ecall.h diff --git

[RFC v1 23/42] Documentation: Add some documentation

Specifically on RISC-V, and fix some typos. Signed-off-by: Ralf Ramsauer --- Documentation/hypervisor-interfaces.txt | 16 ++-- Documentation/memory-layout.txt | 20 2 files changed, 26 insertions(+), 10 deletions(-) diff --git

[RFC v1 27/42] arm-common: export for_each_irqchip-macro

We need this macro on RISC-V. Signed-off-by: Ralf Ramsauer --- hypervisor/arch/arm-common/irqchip.c | 5 - hypervisor/include/jailhouse/control.h | 5 + 2 files changed, 5 insertions(+), 5 deletions(-) diff --git a/hypervisor/arch/arm-common/irqchip.c

[RFC v1 35/42] configs: riscv: qemu: add tiny demo config

Signed-off-by: Ralf Ramsauer --- configs/riscv/qemu-riscv64-tiny-demo.c | 63 ++ 1 file changed, 63 insertions(+) create mode 100644 configs/riscv/qemu-riscv64-tiny-demo.c diff --git a/configs/riscv/qemu-riscv64-tiny-demo.c b/configs/riscv/qemu-riscv64-tiny-demo.c new

[RFC v1 06/20] arm64: Update unit link order comment and save a line

From: Jan Kiszka Signed-off-by: Jan Kiszka --- hypervisor/arch/arm64/Kbuild | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/hypervisor/arch/arm64/Kbuild b/hypervisor/arch/arm64/Kbuild index 59102226..fd56a4da 100644 --- a/hypervisor/arch/arm64/Kbuild +++

[RFC v1 03/20] x86: Do not kick targets of vcpu cache flushes

From: Jan Kiszka All CPUs != current are supposed to be suspended when this function is called. They will process the flush request when being resumed. So drop the superfluous kick. Signed-off-by: Jan Kiszka --- hypervisor/arch/x86/control.c | 9 - 1 file changed, 4 insertions(+), 5

[RFC v1 01/20] core: Add internal event check before hypervisor activation

From: Jan Kiszka If setup code triggers any event that is to be handled on all CPUs, this should be processed right before VMM activation. Adding this will consume the superfluous and to-be-removed flush_vcpu_caches. More importantly, it will enable handling of update_cat properly on Intel x86.

[RFC v1 12/20] core: Change CPU configuration to an array of structures

From: Jan Kiszka Switch from a bitmap-based way to an array of structures for describing cell CPUs. This has the advantage that we can decouple Jailhouse from CPU enumeration of Linux and pass the physical CPUs to the hypervisor. Furthermore, the structure can be extended later on to associate

[RFC v1 08/20] x86: Rename CPU_ID_INVALID to APIC_INVALID_CPU

From: Jan Kiszka We already have INVALID_CPU_ID, so better make this 8-bit version of it apic-specific in its name to reduce confusion. No functional change. Signed-off-by: Jan Kiszka --- hypervisor/arch/x86/apic.c | 12 ++-- hypervisor/arch/x86/include/asm/apic.h | 4

[RFC v1 04/20] core: Clarify invocation context of arch_flush_cell_vcpu_caches

From: Jan Kiszka This service only works for cells that are suspended. Add this to the common function description. Also drop the redundant note from the ARM implementation. Signed-off-by: Jan Kiszka --- hypervisor/arch/arm-common/control.c | 1 - hypervisor/include/jailhouse/control.h | 2

[RFC v1 17/20] configs: arm: Convert to new CPU configuration format

From: Jan Kiszka At this chance, move the jetson-tk1-inmate-demo from CPU 3 to 1, avoiding the overlap with its linux-demo and aligning it with other systems with 4 or more cores. Signed-off-by: Jan Kiszka --- configs/arm/bananapi-inmate-demo.c | 10

[RFC v1 14/20] core: Convert to new CPU configuration format

From: Jan Kiszka These are the necessary changes to adapt the hypervisor core to the new struct-array CPU configuration format. Some things become easier: cpu_id_valid() is now a simple inline bounds check, counting CPUs is already done in the config. Other things become more complex: An

[RFC v1 05/20] core: Convert config_commit into a unit callback

From: Jan Kiszka This simplifies the code as it obsoletes arch_config_commit. The programmatic order of commit functions called by config_commit and arch_config_commit were already in line with the unit order. Signed-off-by: Jan Kiszka --- hypervisor/arch/arm-common/control.c | 6

[RFC v1 00/20] new-cpu-config + RISC-V Cornerstone

Hi, this is a rebase of "wip/new-cpu-config" branch of Jan. We will need these patches for RISC-V support, which will land on the list soon. Basically, this series adds support for both: logical and physical CPU IDs in Jailhouse. Thanks Ralf Jan Kiszka (20): core: Add internal event check

[RFC v1 09/20] core: Beautify for_each_cpu helper

From: Jan Kiszka Avoid -1 as impossible exception value, rather use the equally impossible INVALID_CPU_ID. Signed-off-by: Jan Kiszka --- hypervisor/include/jailhouse/control.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/hypervisor/include/jailhouse/control.h

[RFC v1 19/20] core: Optimize parsing of large CPU sets

From: Jan Kiszka Currently, the for_each_cpu iterators scan the whole given CPU set bitwise. Since we use static CPU set sizes, we scan even more bits needlessly for most workloads. This is particularly relevant for sending IPIs that go out to a set of CPUs. But the common case of having a

[RFC v1 13/20] driver: Convert to new CPU configuration format

From: Jan Kiszka The new format requires the driver to identify a CPU by its physical ID and then pass its config array index to the hypervisor, both on enabling as well as during runtime (status queries). For that purpose, two per-cpu variables are initialized while enabling Jailhouse:

[RFC v1 16/20] configs: x86: Convert to new CPU configuration format

From: Jan Kiszka Signed-off-by: Jan Kiszka --- configs/x86/apic-demo.c | 10 ++ configs/x86/e1000-demo.c | 10 ++ configs/x86/f2a88xm-hd3.c| 19 +++ configs/x86/imb-a180.c | 19 +++ configs/x86/ioapic-demo.c| 10 ++

[RFC v1 10/20] core: Use statically sized cpu sets

From: Jan Kiszka Given that we do not face excessive amounts of CPUs so far, are in fact limited to 254 on x86 e.g., there is no point in the complex bitmap allocation algorithm used for cpu_set. Introduce a per-arch upper limit of CPUs (MAX_CPUS) and lay out struct cpu_set accordingly.

[RFC v1 15/20] tools: Convert to new CPU configuration format

From: Jan Kiszka This updates the config and sysfs parser as well as the config generator to the new format. For generating x86 configs, we are parsing /proc/cpuinfo for the initial APIC ID and use that - in Linux order - to fill out the config template. Signed-off-by: Jan Kiszka ---

[RFC v1 18/20] configs: arm64: Convert to new CPU configuration format

From: Jan Kiszka At this chance, the inmate demo is moved to the second core for jetson-tx1, miriac, ultra96 and zynqmp-zcu102. Signed-off-by: Jan Kiszka [ralf: rebase to next, catch missing configs] Signed-off-by: Ralf Ramsauer --- configs/arm64/amd-seattle-inmate-demo.c | 8 ++-

[RFC v1 07/20] arm, arm64: Make GICv3 arm64-only

From: Jan Kiszka There is no 32-bit ARM board that we support and that has a GICv3. Very likely, such a board does not even exist and never will. So move the GICv3 support in Jailhouse to arm64, removing no longer needed abstractions. Rather than stubbing a GICv3 for irqchip_cpu_init, use

[RFC v1 02/20] core: Skip arch_flush_cell_vcpu_caches during setup

From: Jan Kiszka There is no need to kick off a vcpu cache flush right after setup. We just installed the new mapping and didn't use it yet, thus nothing can be cached. Actually, this didn't fully trigger a flush on ARM and ARM64 anyway because those archs didn't kick all CPUs but rather

[RFC v1 11/42] core: riscv: implement the hypercall interface

Nothing special here. SBIs are parameterised bei an "Extension ID", which is typically 3-4 ASCII characters, 'TIME', 'HSM', 'IPI', 'RFNC', … Let's pray that no one else will implement 'JHOU' as official EID. Then there are FIDs within the EID - Function IDs. For Jailhouse Hypercalls, use FID =

[RFC v1 11/20] core: Move cell_init() of root cell from init_early to init_late

From: Jan Kiszka This is possible because no one needs the initialization results of cell_init() early. These results are more precisely the cpu_set bitmap and the mmio subsystem setup for the root cell. At the a same time, cell_init() will later on benefit from having all CPUs initialized.

[RFC v1 20/20] RISC-V cornerstone

From: Jan Kiszka This allows to build Jailhouse for RISC-V, using QEMU as a first target. The assembly entry routine can be called and returns a funky error for now. All the rest are stubs. Signed-off-by: Jan Kiszka [ralf: rebase to next, add stubs for missing routines] Signed-off-by: Ralf

Re: [RFC v1 03/42] core: riscv: define MMIO accessors

On 27.06.22 15:28, Ralf Ramsauer wrote: > GCC may emit 4x 1 Byte reads in case of our regular mmio_read32 > accessor, when used in combination with -Os. Yes, I've seen it. > > Define safe handlers to overcome this issue. > The kernel does something similar, right? Worth to point that out then.

Re: [RFC v1 34/42] configs: riscv: qemu: add system configs

On 27/06/2022 15:36, Jan Kiszka wrote: On 27.06.22 15:28, Ralf Ramsauer wrote: Signed-off-by: Ralf Ramsauer --- configs/riscv/qemu-riscv64.c | 218 +-- include/jailhouse/console.h | 1 + 2 files changed, 211 insertions(+), 8 deletions(-) ... diff