tl;dr: this is a first approach towards RISC-V support in Jailhouse. It works,
but there are still some open architectural discussions that have to be
addressed.
Hi,
this is the first RFC series for RISC-V support in Jailhouse. The series was
initially developed by Konrad Schwarz at SIEMENS in context of the SELENE
project [0], and later extended by Stefan an me (see Contributors below) at OTH
Regensburg.
The series is not yet ready to be integrated, but I think it's a good starting
point to have first discussions about the architectural aspects and directions
of the RISC-V port.
The series adds support for:
- "wip/new-cpu-config" (by Jan, rebased and ported by Ralf)
- RISC-V h-extensions (RV64 only)
- MMU support: SV39, SV48, SV57
- PLIC interrupt controller
- Multi-Core non-root cells
- Basic demo inmates
- Non-Root Linux support
- IVSHMEM (IRQs are implemented by virtualising the PLIC, sigh…)
Target platforms:
- Qemu virtual targets
- NOEL-V [1]
Architectural limitations:
- IRQs arrive in S-Mode, and must be reinjected to VS-Mode. This applies for
IPIs, Timers, and physical IRQs at the PLIC.
- The PLIC needs full virtualisation, as its design is uncapable for
virtualisation. Cf.: [2].
- IPIs are sent via SBI firmware. This requires double trapping (Jailhouse +
SBI), and manual reinjection on receiver site.
- IPIs can not be parameterised: there's only one single IPI. We have to
differentiate between management and guest IPIs in software. Sigh…
What is missing and should be discussed:
- APLIC support (and MSI support)
The PLIC does not support to be virtualised at all, and we should
probably add support for the APLIC soon. The question is, if Jailhouse
should still offer support for the PLIC: At the moment, only Qemu
supports the APLIC, and we don't have a real hardware target.
- Minor: missing support for jailhouse-config-check, support for
jailhouse-linux
- Prioritisation of vIRQs: they have lowest prio.
vIRQ support is only suppported for testing / demonstration at the
moment. Full virtualisation of the PLIC comes at a high cost, as
priorities of the vIRQs need to be respectes as well. As this level of
virtualisation is contrary to the concepts of Jailhouse, I deliberately
did not implement prioritisation of vIRQs.
- Stable CPU enumeration:
On Qemu, for example, the mapping from logical CPU ID to physical HART ID
differs each boot. Hence, Jailhouse's static enumeration may drift. This
is only a problem for jailhouse cell stats, as erroneous CPUs will be
addressed. We need to stabilise CPU enumeration. The driver will drop a
warning if CPU mapping differs.
- Some spots need better locking and testing. E.g., IPI handling needs better
locking.
- There are some overlaps with arm-common/arm64; see irq handling, bitops.
Should we factor those bits out?
Linux support:
- We need this single downstream patch [3]. Patches can be found on the branch
ralf/riscv-jailhouse/V10.
Contributors:
- Konrad Schwarz, Siemens (base support, first draft)
- Ralf Ramsauer, OTH (exception handling, control flow, reworked PLIC support)
- Stefan Huber, OTH (inmate support)
- Jan Kiszka, Siemes (wip/new-cpu-config)
Besides this patch series, there's a testbed for jailhouse on RISC-V that eases
debugging and development [4].
The series can also be found here [5].
Ralf
[0] https://www.selene-project.eu/
[1] https://www.gaisler.com/index.php/products/processors/noel-v
[2] https://github.com/riscv/riscv-plic-spec/blob/master/riscv-plic.adoc
[3]
https://github.com/lfd/linux/commit/e4c207ca00395c0fec578ae9b79b6f6a615fddeb
[4] https://github.com/lfd/jailv-testbed
[5] https://github.com/lfd/jailhouse/tree/ralf/riscv/V8
Konrad Schwarz (1):
core: riscv: implement paging structures
Ralf Ramsauer (40):
driver: RISC-V: check for availability of h-extensions
driver: sysfs: prepare for exit accounting on RISC-V
core: riscv: define MMIO accessors
core: riscv: Add CSR definitions
core: riscv: introduce asm/bitops.h
core: riscv: add processor.h
core: riscv: Define percpu fields
core: riscv: introduce global sbi_ecall.h
core: riscv: introduce asm/sbi.h
core: riscv: add platform information
core: riscv: implement the hypercall interface
core: riscv: introduce cell-specific fields
core: riscv: add dbg-write helper
core: riscv: implement library routines
core: riscv: implement PCI support
core: riscv: paging: add headers
core: riscv: add asm-defines.c
core: riscv: introduce traps.c
core: riscv: Hypervisor entry code
core: riscv: implement arch_send_event
core: riscv: Add spinlock primitives
Documentation: Add some documentation
core: riscv: implement control.c
core: forward original HV memory location
core: riscv: implement setup.c
arm-common: export for_each_irqchip-macro
riscv: Add plic implementation
core: riscv: implement trap handlers
inmates: riscv: add timer demo
riscv: inmates: propagate hartid
riscv: inmates: implement Linux Loader
configs: riscv: qemu: add system configs
configs: riscv: qemu: add tiny demo config
configs: riscv: qemu: Add linux-demo
configs: riscv: noelv: Add system configs
configs: riscv: noelv: add tiny demo
configs: riscv: noelv: Add linux inmate config
core: riscv: plic: introduce vIRQ support
configs: riscv: add ivshmem-net devices
driver: riscv: Add vPCI support
Stefan Huber (1):
riscv: inmates: implement tiny-demo RISC-V
.gitignore | 1 +
Documentation/hypervisor-interfaces.txt | 16 +-
Documentation/memory-layout.txt | 20 +-
configs/riscv/dts/noelv-linux-inmate.dts | 147 ++++
configs/riscv/dts/qemu-linux-inmate.dts | 135 +++
configs/riscv/noelv-linux-demo.c | 111 +++
configs/riscv/noelv-tiny-demo.c | 63 ++
configs/riscv/noelv.c | 153 ++++
configs/riscv/qemu-linux-demo.c | 111 +++
configs/riscv/qemu-riscv64-tiny-demo.c | 63 ++
configs/riscv/qemu-riscv64.c | 228 ++++-
driver/main.c | 8 +
driver/pci.c | 33 +-
driver/sysfs.c | 15 +
hypervisor/arch/arm-common/irqchip.c | 5 -
hypervisor/arch/riscv/Kbuild | 3 +-
hypervisor/arch/riscv/asm-defines.c | 27 +
hypervisor/arch/riscv/control.c | 228 ++++-
hypervisor/arch/riscv/dbg-write.c | 14 +-
hypervisor/arch/riscv/entry.S | 349 +++++++-
hypervisor/arch/riscv/exception.S | 91 ++
hypervisor/arch/riscv/include/asm/bitops.h | 54 +-
hypervisor/arch/riscv/include/asm/cell.h | 13 +-
hypervisor/arch/riscv/include/asm/control.h | 13 +
hypervisor/arch/riscv/include/asm/csr64.h | 196 +++++
hypervisor/arch/riscv/include/asm/ivshmem.h | 1 +
.../arch/riscv/include/asm/jailhouse_header.h | 24 +-
hypervisor/arch/riscv/include/asm/mmio.h | 67 ++
hypervisor/arch/riscv/include/asm/paging.h | 119 ++-
.../arch/riscv/include/asm/paging_modes.h | 18 +
hypervisor/arch/riscv/include/asm/percpu.h | 39 +-
hypervisor/arch/riscv/include/asm/plic.h | 28 +
hypervisor/arch/riscv/include/asm/processor.h | 47 +-
hypervisor/arch/riscv/include/asm/sbi.h | 84 ++
hypervisor/arch/riscv/include/asm/setup.h | 14 +
hypervisor/arch/riscv/include/asm/spinlock.h | 52 +-
hypervisor/arch/riscv/ivshmem.c | 36 +-
hypervisor/arch/riscv/lib.c | 25 +-
hypervisor/arch/riscv/paging.c | 262 +++++-
hypervisor/arch/riscv/pci.c | 25 +-
hypervisor/arch/riscv/plic.c | 706 +++++++++++++++
hypervisor/arch/riscv/setup.c | 200 ++++-
hypervisor/arch/riscv/traps.c | 803 ++++++++++++++++++
hypervisor/include/jailhouse/control.h | 5 +
hypervisor/include/jailhouse/gen-defines.h | 8 +-
hypervisor/include/jailhouse/header.h | 7 +
hypervisor/include/jailhouse/mmio.h | 4 +
hypervisor/include/jailhouse/paging.h | 2 +-
include/arch/riscv/asm/jailhouse_hypercall.h | 68 +-
include/arch/riscv/asm/sbi_ecall.h | 96 +++
include/jailhouse/cell-config.h | 11 +-
include/jailhouse/console.h | 1 +
inmates/demos/riscv/Makefile | 20 +
inmates/demos/riscv/timer-demo.c | 57 ++
inmates/demos/riscv/tiny-demo.c | 19 +
inmates/lib/include/inmate_common.h | 1 +
inmates/lib/riscv/Makefile | 46 +
inmates/lib/riscv/Makefile.lib | 64 ++
inmates/lib/riscv/header.S | 76 ++
inmates/lib/riscv/include/inmate.h | 184 ++++
inmates/lib/riscv/inmate.lds.S | 77 ++
inmates/lib/riscv/irq.c | 107 +++
inmates/lib/riscv/printk.c | 61 ++
inmates/lib/riscv/setup.c | 45 +
inmates/lib/riscv/uart.c | 47 +
inmates/tools/riscv/Makefile | 19 +
inmates/tools/riscv/linux-loader.c | 25 +
pyjailhouse/config_parser.py | 17 +-
tools/jailhouse-config-check | 9 +-
69 files changed, 5584 insertions(+), 139 deletions(-)
create mode 100644 configs/riscv/dts/noelv-linux-inmate.dts
create mode 100644 configs/riscv/dts/qemu-linux-inmate.dts
create mode 100644 configs/riscv/noelv-linux-demo.c
create mode 100644 configs/riscv/noelv-tiny-demo.c
create mode 100644 configs/riscv/noelv.c
create mode 100644 configs/riscv/qemu-linux-demo.c
create mode 100644 configs/riscv/qemu-riscv64-tiny-demo.c
create mode 100644 hypervisor/arch/riscv/exception.S
create mode 100644 hypervisor/arch/riscv/include/asm/csr64.h
create mode 100644 hypervisor/arch/riscv/include/asm/plic.h
create mode 100644 hypervisor/arch/riscv/include/asm/sbi.h
create mode 100644 hypervisor/arch/riscv/include/asm/setup.h
create mode 100644 hypervisor/arch/riscv/plic.c
create mode 100644 hypervisor/arch/riscv/traps.c
create mode 100644 include/arch/riscv/asm/sbi_ecall.h
create mode 100644 inmates/demos/riscv/timer-demo.c
create mode 100644 inmates/demos/riscv/tiny-demo.c
create mode 100644 inmates/lib/riscv/Makefile.lib
create mode 100644 inmates/lib/riscv/header.S
create mode 100644 inmates/lib/riscv/include/inmate.h
create mode 100644 inmates/lib/riscv/inmate.lds.S
create mode 100644 inmates/lib/riscv/irq.c
create mode 100644 inmates/lib/riscv/printk.c
create mode 100644 inmates/lib/riscv/setup.c
create mode 100644 inmates/lib/riscv/uart.c
create mode 100644 inmates/tools/riscv/linux-loader.c
--
2.36.1
--
You received this message because you are subscribed to the Google Groups
"Jailhouse" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/jailhouse-dev/20220627132905.4338-1-ralf.ramsauer%40oth-regensburg.de.
