Re: [RFC v1 00/25] Add Realm support to QEMU-VMM

2026-07-22 Thread Gavin Shan

Hi Mathieu,

On 7/8/26 8:42 AM, Mathieu Poirier wrote:

This patchset provides minimal functionality to start a Realm VM
from an Arm RME cabable host using the following command line:

qemu-system-aarch64 -M confidential-guest-support=rme0 -object 
rme-guest,id=rme0 \
 -cpu host -M virt -enable-kvm -M gic-version=3,its=on -nodefaults ..

This set is a refactoring of Jean-Philippe Brucker's initial work dating from
a while back.  It is compatible with Steven Price's v14 revision [1] of his work
adding CCA support to KVM.

The RME capable host it was tested on is the QEMU SBSA machine.  For 
convenicence
a repository is hosted here [2], along with the TF-A [3], RMM [4] and Linux
kernel [5] for the SBSA machine (all compatible with Steven's v14 patchset).

Device Assignment is not included.

 From hereon I will keep this set up to date with new releases from Steven.

Thanks,
Mathieu

[1]. 
https://lore.kernel.org/kvm/[email protected]/T/#m06dd14216aaf76acab65b0a76fb84653141ea64f
[2]. https://gitlab.com/Linaro/cca-public/qemu/-/tree/upstream-v1?ref_type=heads
[3]. 
https://gitlab.com/Linaro/cca-public/tf-a/trusted-firmware-a/-/tree/cca/v13?ref_type=heads
[4]. https://gitlab.com/Linaro/cca-public/rmm/-/tree/cca/v14?ref_type=heads
[5]. https://gitlab.com/Linaro/cca-public/linux/-/tree/cca/v14?ref_type=heads


[...]

Gave this series + (v14) series a try and found the following issues so far. 
Note
some of the HMP commands like those relevant to migration have been skipped 
since
we know the migration isn't supported. I list the issues for record and we 
eventually
need to figure them out one by one.

(1) qemu is terminiated on exeute qmp command 'dump-guest-memory'

(qemu) dump-guest-memory -p "/tmp/mem" 0x4000 0x1000
**
ERROR:../system/memory_mapping.c:215:guest_phys_block_add_section: assertion failed: 
(predecessor->target_end <= target_start)
Bail out! ERROR:../system/memory_mapping.c:215:guest_phys_block_add_section: 
assertion failed: (predecessor->target_end <= target_start)
Aborted

(2) qemu emulation can't be stopped and resumed

(qemu) stop
(qemu) cont
Failed to set KVM_REG_ARM_TIMER_CNT

(3) qemu is terminiated on qmp command 'system_reset'

(qemu) system_reset
cpus are not resettable, terminating

(4) Incompatible to virtio-iommu-pci. The realm guest doesn't boot up with
a virtio-iommu-pci device

host$ qemu-system-aarch64 -enable-kvm -object rme-guest,id=rme0\
-machine virt,gic-version=3,confidential-guest-support=rme0\
-m 2G -cpu host -smp maxcpus=4,cpus=4,sockets=1,clusters=1,cores=2,threads=2   \
-object memory-backend-ram,id=mem0,size=2G,share=yes   \
-numa node,nodeid=0,cpus=0-3,memdev=mem0   \
-serial mon:stdio -monitor none -nographic -nodefaults \
-bios /mnt/edk2/Build/ArmVirtQemu-AARCH64/RELEASE_GCC5/FV/QEMU_EFI.fd  \
-kernel /mnt/linux/arch/arm64/boot/Image   \
-initrd /mnt/buildroot/output/images/rootfs.cpio.xz\
-append earlycon=pl011,mmio,0x1000900  \
-device pcie-root-port,bus=pcie.0,chassis=1,id=pcie.1  \
-device pcie-root-port,bus=pcie.0,chassis=2,id=pcie.2  \
-device virtio-iommu-pci,bus=pcie.0\
-netdev tap,id=tap1,vhost=on,script=/etc/qemu-ifup,downscript=/etc/qemu-ifdown \
-device virtio-net-pci,bus=pcie.2,netdev=tap1,mac=b8:3f:d2:1d:3e:c0
:
UEFI firmware (version  built at 00:09:11 on Jun 30 2026)
:
[0.00] Booting Linux on physical CPU 0x00 [0x000f0510]
[0.00] Linux version 7.2.0-rc4-gavin-g1590cf032971 
([email protected]) (gcc (GCC) 14.3.1 
20251022 (Red Hat 14.3.1-4), GNU ld version 2.41-65.el10) #46 SMP PREEMPT Mon 
Jul 20 06:56:20 EDT 2026
[0.00] KASLR enabled
[0.00] earlycon: pl11 at MMIO 0x01000900 (options '')
[0.00] printk: legacy bootconsole [pl11] enabled
[0.00] efi: EFI v2.7 by EDK II
[0.00] efi: SMBIOS 3.0=0xbff6 ACPI 2.0=0xbdee2018 INITRD=0xbdd94f18 
RNG=0xbdee2a18 MEMRESERVE=0xbdd94f98
:
[   18.374507] virtio-pci :00:03.0: enabling device ( -> 0002)
[   18.594890] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
[   18.651840] Serial: AMBA driver
[   19.669576] software IO TLB: Memory encryption is active and system is using 
DMA bounce buffers
[   35.021315] Freeing initrd memory: 5772K
[   40.984997] rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
[   41.076752] rcu: 3-...0: (77 ticks this GP) 
idle=aed4/1/0x4000 softirq=83/97 fqs=2365
[   41.210687] rcu: (detected by 2, t=5307 jiffies, g=-1079, q=352 ncpus=4)
[   41.311646] Sending NMI from CPU 2 to CPUs 3:
[   51.631564] After 10 seconds, these CPUS still haven't responde

[RFC v1 00/25] Add Realm support to QEMU-VMM

2026-07-07 Thread Mathieu Poirier
This patchset provides minimal functionality to start a Realm VM
from an Arm RME cabable host using the following command line:

qemu-system-aarch64 -M confidential-guest-support=rme0 -object 
rme-guest,id=rme0 \
-cpu host -M virt -enable-kvm -M gic-version=3,its=on -nodefaults ..

This set is a refactoring of Jean-Philippe Brucker's initial work dating from
a while back.  It is compatible with Steven Price's v14 revision [1] of his work
adding CCA support to KVM.  

The RME capable host it was tested on is the QEMU SBSA machine.  For 
convenicence
a repository is hosted here [2], along with the TF-A [3], RMM [4] and Linux
kernel [5] for the SBSA machine (all compatible with Steven's v14 patchset).

Device Assignment is not included.

>From hereon I will keep this set up to date with new releases from Steven.

Thanks,
Mathieu 

[1]. 
https://lore.kernel.org/kvm/[email protected]/T/#m06dd14216aaf76acab65b0a76fb84653141ea64f
[2]. https://gitlab.com/Linaro/cca-public/qemu/-/tree/upstream-v1?ref_type=heads
[3]. 
https://gitlab.com/Linaro/cca-public/tf-a/trusted-firmware-a/-/tree/cca/v13?ref_type=heads
[4]. https://gitlab.com/Linaro/cca-public/rmm/-/tree/cca/v14?ref_type=heads
[5]. https://gitlab.com/Linaro/cca-public/linux/-/tree/cca/v14?ref_type=heads


Jean-Philippe Brucker (25):
  linux-headers: Add RME related definitions
  target/arm/kvm: Return immediately on error in kvm_arch_init()
  target/arm: Add confidential guest support
  target/arm/kvm-rme: Add mechanic to initialize realms
  target/arm/kvm: Split kvm_arch_get/put_registers
  target/arm/kvm-rme: Initialize vCPU
  target/arm/kvm: Create scratch Realm VM when requested
  target/arm/kvm: Use kvm_vm_check_extension() where necessary
  target/arm/kvm-rme: Initialise Realm Initial Address space
  hw/core/loader: Add a ROM loader notifier
  target/arm/kvm-rme: Keep track of images loaded in Realm memory
  target/arm/kvm-rme: Populate Realm with runtime images
  target/arm/cpu: Set number of breakpoints and watchpoints in KVM
  target/arm/cpu: Set number of PMU counters in KVM
  target/arm/cpu: Don't read Realm registers
  hw/arm/virt: Set proper conduit method for Realms
  hw/arm/virt: Embed Realm VM type with IPA address space
  hw/arm/virt: Reserve one bit of guest physical address for RME
  hw/arm/virt: Disable DTB randomness for confidential VMs
  hw/arm/virt: Move virt_flash_create() to machvirt_init()
  hw/arm/virt: Use RAM instead of flash for confidential guest firmware
  target/arm/kvm-rme: Add DMA remapping for the shared memory region
  docs/interop/firmware.json: Add arm-rme firmware feature
  hw/arm/boot: Load DTB as is for confidential VMs
  hw/arm/boot: Skip bootloader for confidential guests

 docs/interop/firmware.json |   5 +-
 docs/system/arm/virt.rst   |   9 +-
 docs/system/confidential-guest-support.rst |   1 +
 hw/arm/boot.c  |  74 -
 hw/arm/virt.c  | 138 +++--
 hw/core/loader.c   |  15 +
 include/hw/arm/boot.h  |   9 +
 include/hw/arm/virt.h  |   2 +-
 include/hw/core/loader.h   |  17 ++
 linux-headers/asm-arm64/kvm.h  |   9 +
 linux-headers/linux/kvm.h  |  21 +-
 qapi/qom.json  |   1 +
 target/arm/arm-qmp-cmds.c  |   1 +
 target/arm/cpu.c   |   5 +
 target/arm/cpu.h   |  10 +
 target/arm/cpu64.c | 122 
 target/arm/kvm-rme.c   | 335 +
 target/arm/kvm-stub.c  |  23 ++
 target/arm/kvm.c   | 194 +++-
 target/arm/kvm_arm.h   |  47 +++
 target/arm/meson.build |   5 +-
 21 files changed, 990 insertions(+), 53 deletions(-)
 create mode 100644 target/arm/kvm-rme.c

-- 
2.43.0