On Tue, Aug 23, 2022 at 10:36:00PM +0000, Martin Oliveira wrote: > Hello, > > I'm trying to use the QEMU NVMe userspace driver and I'm hitting an error > when trying to use more than one device from an IOMMU group: > > Failed to open VFIO group file: /dev/vfio/39: Device or resource busy > > If devices belong to different IOMMU groups, then it works as expected. > > For each device, I bind it to vfio-pci and then use something like this: > > -drive file=nvme://0000:26:00.0,if=none,id=drive0,format=raw > -device virtio-blk,drive=drive0,id=virtio0,serial=nvme0 > > Using the file-based protocol (file=/dev/nvme0n1) works with multiple devices > from the same group. > > My host is running a 5.19 kernel and QEMU is the latest upstream > (a8cc5842b5cb).
First, multiple QEMU instances cannot access nvme:// devices sharing the same IOMMU group. I don't think this will ever be possible because it opens a backdoor around process memory isolation. However, a single QEMU (or qemu-storage-daemon) instance should be able to access multiple nvme:// devices in the same IOMMU group. Unfortunately the code currently doesn't support that. util/vfio-helpers.c:qemu_vfio_init_pci() has no logic for sharing groups/containers. Opening the group fails with EBUSY because the kernel only allows the file to be opened once at any given time. It's possible to extend the util/vfio-helpers.c code to reuse VFIO groups (and share VFIO containers), but I'm not aware of anyone who is currently working on that. Stefan
signature.asc
Description: PGP signature
