On Tue, May 3, 2022 at 5:57 PM Atish Patra <ati...@atishpatra.org> wrote: > > On Tue, Apr 19, 2022 at 5:26 PM Atish Patra <ati...@atishpatra.org> wrote: > > > > On Tue, Apr 19, 2022 at 9:51 AM Daniel P. Berrangé <berra...@redhat.com> > > wrote: > > > > > > On Mon, Apr 11, 2022 at 07:10:06PM -0700, Atish Patra wrote: > > > > > > > > The RISC-V virt machine has helped RISC-V software eco system to evolve > > > > at a > > > > rapid pace even in absense of the real hardware. It is definitely > > > > commendable. > > > > However, the number of devices & commandline options keeps growing as a > > > > result > > > > of that as well. That adds flexibility but will also become bit > > > > difficult > > > > to manage in the future as more extension support will be added. As it > > > > is the > > > > most commonly used qemu machine, it needs to support all kinds of > > > > device and > > > > interrupts as well. Moreover, virt machine has limitations on the > > > > maximum > > > > number of harts it can support because of all the MMIO devices it has > > > > to support. > > > > > > > > The RISC-V IMSIC specification allows to develop machines completely > > > > relying > > > > on MSI and don't care about the wired interrupts at all. It just > > > > requires > > > > all the devices to be present behind a PCI bus or present themselves as > > > > platform > > > > MSI device. The former is a more common scenario in x86 world where most > > > > of the devices are behind PCI bus. As there is very limited MMIO device > > > > support, it can also scale to very large number of harts. > > > > > > > > That's why, this patch series introduces a minimalistic yet very > > > > extensible > > > > forward looking machine called as "RISC-V Mini Computer" or "minic". The > > > > idea is to build PC or server like systems with this machine. The > > > > machine can > > > > work with or without virtio framework. The current implementation only > > > > supports RV64. I am not sure if building a RV32 machine would be of > > > > interest > > > > for such machines. The only mmio device it requires is clint to emulate > > > > the mtimecmp. > > > > > Any other thoughts ?
I don't *love* this idea. I think the virt machine is useful, but I'm not convinced we need a second one. This feels a little bit more like a "none" machine, as it contains just the bare minimum to work. > > > > I would ask what you see as the long term future usage for 'virt' vs > > > 'minic' machine types ? Would you expect all existing users of 'virt' > > > to ultimately switch to 'minic', or are there distinct non-overlapping > > > use cases for 'virt' vs 'minic' such that both end up widely used ? > > > > > > > Nope. I don't expect existing 'virt' users to switch to 'minic' as > > they aim to cater to different users. > > > > Here are the major differences > > 1. virt machine supports MMIO devices & wired interrupts. Minic doesn't This seems like the main difference > > 2. virt machine doesn't support the MSI only option yet (can be added > > though[1]). Minic does. This could be fixed > > 3. Number of cpu supported by virt machine are limited because of the > > MMIO devices. Minic can scale to very > > large numbers of cpu. Similar to 1 > > 4. 'Minic' only supports PCI based MSI capable devices. Thus, MSI is a > > mandatory requirement for 'minic' while > > it is optional for 'virt'. I'm not fully convinced we need this, but it also doesn't seem to cost us a lot in terms of maintenance. It would be beneficial if we could share a bit more of the code. Can we share the socket creation code as well? I don't like the name minic though. What about something like `virt-hpc`, `virt-pcie-minimal` or something like that? That way we indicate it's still a virt board Alistair > > > > 'Minic' aims towards the users who want to create virtual machines > > that are MSI based and don't care about > > a million options that virt machines provide. Virt machine is more > > complex so that it can be flexible in terms of > > what it supports. Minic is a minimalistic machine which doesn't need > > to be expanded a lot in the future given that > > most of the devices can be behind PCI. > > > > [1] https://github.com/atishp04/qemu/tree/virt_imsic_only > > > > > Is 'minic' intended to be able to mimic real physical hardware at all, > > > or is it still intended as a purely virtual machine, like a 'virt-ng' ? > > > > > > > Any future hardware that relies only on PCI-MSI based devices, they > > can be created on top of minic. > > At that point, minic will provide a useful abstract for all those > > machines as well. minic doesn't need a virtio framework. > > Thus, it can closely emulate such hardware as well. > > > > > Essentially 'virt' was positioned as the standard machine to use if > > > you want to run a virtual machine, without any particular match to > > > physical hardware. It feels like 'minic' is creating a second machine > > > type to fill the same purpose, so how do users decide which to use ? > > > > > > > I envision 'minic' to be a standard machine for a specific set of user > > requirements (x86 style PCI based > > machines). Virt machine will continue to be a standard machine for > > more generic use cases with MMIO devices. > > > > > > "Naming is hard". I am not too attached with the name "minic". > > > > I just chose least bad one out of the few on my mind :). I am definitely > > > > open to any other name as well. > > > > > > > > The other alternative to provide MSI only option to aia in the > > > > existing virt machine to build MSI only machines. This is certainly > > > > doable > > > > and here is the patch that supports that kind of setup. > > > > > > > > https://github.com/atishp04/qemu/tree/virt_imsic_only > > > > > > > > However, it even complicates the virt machine even further with > > > > additional > > > > command line option, branches in the code. I believe virt machine will > > > > become > > > > very complex if we continue this path. I am interested to learn what > > > > everyone > > > > else think. > > > > > > > > It is needless to say that the current version of minic machine > > > > is inspired from virt machine and tries to reuse as much as code > > > > possible. > > > > The first patch in this series adds MSI support for serial-pci device so > > > > console can work on such a machine. The 2nd patch moves some common > > > > functions > > > > between minic and the virt machine to a helper file. The PATCH3 actually > > > > implements the new minic machine. > > > > > > > > I have not added the fw-cfg/flash support. We probably should add those > > > > but I just wanted to start small and get the feedback first. > > > > This is a work in progress and have few more TODO items before becoming > > > > the > > > > new world order :) > > > > > > > > 1. OpenSBI doesn't have PCI support. Thus, no console support for > > > > OpenSBI > > > > for now. > > > > 2. The ns16550 driver in OpenSBI also need to support MSI/MSI-X. > > > > 3. Add MSI-X support for serial-pci device. > > > > > > > > This series can boot Linux distros with the minic machine with or > > > > without virtio > > > > devices with out-of-tree Linux kernel patches[1]. Here is an example > > > > commandline > > > > > > > > Without virtio devices (nvme, serial-pci & e1000e): > > > > ===================================================== > > > > /scratch/workspace/qemu/build/qemu-system-riscv64 -cpu rv64 -M minic -m > > > > 1G -smp 4 -nographic -nodefaults \ > > > > -display none -bios > > > > /scratch/workspace/opensbi/build/platform/generic/firmware/fw_dynamic.elf > > > > \ > > > > -kernel /scratch/workspace/linux/arch/riscv/boot/Image \ > > > > -chardev stdio,mux=on,signal=off,id=charconsole0 \ > > > > -mon chardev=charconsole0,mode=readline \ > > > > -device pci-serial,msi=true,chardev=charconsole0 \ > > > > -drive > > > > id=disk3,file=/scratch/workspace/rootfs_images//fedora/Fedora-Developer-Rawhide-20211110.n.0-sda.raw,format=raw,if=none,id=drive-system-disk,cache=none,format=raw > > > > \ > > > > -device nvme,serial=deadbeef,drive=disk3 \ > > > > -netdev user,id=usernet,hostfwd=tcp::10000-:22 -device > > > > e1000e,netdev=usernet,bus=pcie.0 \ > > > > -append 'root=/dev/nvme0n1p2 rw loglevel=8 memblock=debug console=ttyS0 > > > > earlycon' -d in_asm -D log.txt -s > > > > > > > > With virtio devices (virtio-scsi-pci, serial-pci & virtio-net-pci) > > > > ================================================================== > > > > /scratch/workspace/qemu/build/qemu-system-riscv64 -cpu rv64 -M minic -m > > > > 1G -smp 4 -nographic -nodefaults \ > > > > -display none -bios > > > > /scratch/workspace/opensbi/build/platform/generic/firmware/fw_dynamic.elf > > > > \ > > > > -kernel /scratch/workspace/linux/arch/riscv/boot/Image \ > > > > -chardev stdio,mux=on,signal=off,id=charconsole0 \ > > > > -mon chardev=charconsole0,mode=readline \ > > > > -device pci-serial,msi=true,chardev=charconsole0 \ > > > > -drive > > > > file=/scratch/workspace/rootfs_images//fedora/Fedora-Developer-Rawhide-20211110.n.0-sda.raw,format=raw,if=none,id=drive-system-disk,cache=none > > > > \ > > > > -device virtio-scsi-pci,id=scsi0 -device > > > > scsi-hd,bus=scsi0.0,drive=drive-system-disk,id=system-disk,bootindex=1 \ > > > > -netdev user,id=n1,hostfwd=tcp::10000-:22 -device > > > > virtio-net-pci,netdev=n1 \ > > > > -append 'root=/dev/sda2 rw loglevel=8 memblock=debug console=ttyS0 > > > > earlycon' > > > > > > > > The objective of this series is to engage the community to solve this > > > > problem. > > > > Please suggest if you have another alternatve solution. > > > > > > > > [1] https://github.com/atishp04/linux/tree/msi_only_console > > > > > > > > Atish Patra (3): > > > > serial: Enable MSI capablity and option > > > > hw/riscv: virt: Move common functions to a separate helper file > > > > hw/riscv: Create a new qemu machine for RISC-V > > > > > > > > configs/devices/riscv64-softmmu/default.mak | 1 + > > > > hw/char/serial-pci.c | 36 +- > > > > hw/riscv/Kconfig | 11 + > > > > hw/riscv/machine_helper.c | 417 +++++++++++++++++++ > > > > hw/riscv/meson.build | 2 + > > > > hw/riscv/minic.c | 438 ++++++++++++++++++++ > > > > hw/riscv/virt.c | 403 ++---------------- > > > > include/hw/riscv/machine_helper.h | 87 ++++ > > > > include/hw/riscv/minic.h | 65 +++ > > > > include/hw/riscv/virt.h | 13 - > > > > 10 files changed, 1090 insertions(+), 383 deletions(-) > > > > create mode 100644 hw/riscv/machine_helper.c > > > > create mode 100644 hw/riscv/minic.c > > > > create mode 100644 include/hw/riscv/machine_helper.h > > > > create mode 100644 include/hw/riscv/minic.h > > > > > > > > -- > > > > 2.25.1 > > > > > > > > > > > > > > With regards, > > > Daniel > > > -- > > > |: https://berrange.com -o- > > > https://www.flickr.com/photos/dberrange :| > > > |: https://libvirt.org -o- > > > https://fstop138.berrange.com :| > > > |: https://entangle-photo.org -o- > > > https://www.instagram.com/dberrange :| > > > > > > > > > > > > -- > > Regards, > > Atish > > > > -- > Regards, > Atish >
