I have updated the bug description according to SRU template. I can help with testing the changes on Jammy (in my environment) and Noble using the reproducer I added.
** Description changed: This is a bug in libvirt which has been fixed in upstream: https://gitlab.com/libvirt/libvirt/-/issues/790 > When using qemu-pr-helper with multipath devices, the target devices are not added to the domain's namespace as well as cgroups. As a result, the sg_persist calls from the guest VM fail. And the fix will be available in the next release (v11.6.0), but needs to be backported to 22.04 and 24.04 as those are on 8.0.0 and 10.0.0 respectively. I'm working on patch for 22.04 followed by 24.04 + + + --- + + [ Impact ] + + When using qemu-pr-helper with multipath devices, the target devices are + not added to the domain's namespace as well as cgroups. As a result, the + sg_persist calls from the guest VM fail. These are mostly used by + clustered applications like Windows Cluster. + + [ Test Plan ] + + These steps help to reproduce the issue with the scsi_debug kernel + module. We create a fake multipath device with 4 target disks, without + the fix we can only see the device mapper device in qemu-pr-helper's + mount namespace. + + 1. Get the scsi_debug kernel module and verify it is available. + + sudo apt install linux-modules-extra-$(uname -r) + + $ find /lib/modules/ -name "*scsi*" | grep debug + /lib/modules/6.8.0-79-generic/kernel/drivers/scsi/scsi_debug.ko.zst + + 2. Ensure multipath-tools in installed on the system and multipathd + service is running. + + sudo apt install multipath-tools + sudo systemctl status multipathd.service + + 3. Create a multipath device with 4 target disks using scsi_debug + + sudo modprobe scsi_debug dev_size_mb=100 num_tgts=1 vpd_use_hostno=0 + add_host=4 delay=20 max_luns=2 no_lun_0=1 + + $ sudo multipath -ll + mpatha (1Linux_scsi_debug_2001) dm-0 Linux,scsi_debug + size=100M features='0' hwhandler='1 alua' wp=rw + |-+- policy='service-time 0' prio=1 status=enabled + | `- 2:0:0:1 sda 8:0 active ready running + |-+- policy='service-time 0' prio=1 status=enabled + | `- 5:0:0:1 sdd 8:48 active ready running + |-+- policy='service-time 0' prio=50 status=active + | `- 3:0:0:1 sdb 8:16 active ready running + `-+- policy='service-time 0' prio=50 status=enabled + `- 4:0:0:1 sdc 8:32 active ready running + + Notice that the name is 1Linux_scsi_debug_2001 and it has 4 disks, + sda to sdd. + + We will use the following path of the multipath device mapper disk + when creating the VM, this points to dm-0. + + /dev/disk/by-id/dm-uuid-mpath-1Linux_scsi_debug_2001 + + $ ls -l /dev/disk/by-id/dm-uuid-mpath-1Linux_scsi_debug_2001 + … /dev/disk/by-id/dm-uuid-mpath-1Linux_scsi_debug_2001 -> ../../dm-0 + + 4. Create a new VM with reservations (starts qemu-pr-helper along with + the qemu process for the VM). Use domain.xml from attachment. + + Install libvirt first: + + sudo apt install libvirt-daemon libvirt-daemon-system libvirt-clients + sudo systemctl start libvirtd + + Create the VM: + + sudo virsh define --file domain.xml --validate + sudo virsh start libvirt-lp2117467 + + Verify it is in running state. + + sudo virsh list --all + + 5. Find the mount namespace of the VM, enter the namespace and verify + the disks are added there. + + Find the PID of qemu-pr-helper or qemu-system-x86 + + $ pgrep qemu-pr-helper + 734148 + + Enter the mount namespace of this process to verify all the target + disks are present. In this example, we don't see the sda to sdd as + libvirt is not patched. + + $ sudo nsenter --target 734148 --mount bash + root@bhavin-build:/# ls /dev/ + disk dm-0 full hugepages mapper mqueue null ptmx pts random shm urandom zero + + When the patch is applied, we should be able to see the target + devices as well. + + [ Where problems could occur ] + + This is a bugfix backport, not able to think any potential problem as of + now. ** Attachment added: "domain.xml" https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/2117467/+attachment/5910958/+files/domain.xml ** Description changed: This is a bug in libvirt which has been fixed in upstream: https://gitlab.com/libvirt/libvirt/-/issues/790 > When using qemu-pr-helper with multipath devices, the target devices are not added to the domain's namespace as well as cgroups. As a result, the sg_persist calls from the guest VM fail. And the fix will be available in the next release (v11.6.0), but needs to be backported to 22.04 and 24.04 as those are on 8.0.0 and 10.0.0 respectively. I'm working on patch for 22.04 followed by 24.04 - --- [ Impact ] When using qemu-pr-helper with multipath devices, the target devices are not added to the domain's namespace as well as cgroups. As a result, the sg_persist calls from the guest VM fail. These are mostly used by clustered applications like Windows Cluster. [ Test Plan ] These steps help to reproduce the issue with the scsi_debug kernel module. We create a fake multipath device with 4 target disks, without the fix we can only see the device mapper device in qemu-pr-helper's mount namespace. 1. Get the scsi_debug kernel module and verify it is available. - sudo apt install linux-modules-extra-$(uname -r) + sudo apt install linux-modules-extra-$(uname -r) - $ find /lib/modules/ -name "*scsi*" | grep debug - /lib/modules/6.8.0-79-generic/kernel/drivers/scsi/scsi_debug.ko.zst + $ find /lib/modules/ -name "*scsi*" | grep debug + /lib/modules/6.8.0-79-generic/kernel/drivers/scsi/scsi_debug.ko.zst 2. Ensure multipath-tools in installed on the system and multipathd service is running. - sudo apt install multipath-tools - sudo systemctl status multipathd.service + sudo apt install multipath-tools + sudo systemctl status multipathd.service 3. Create a multipath device with 4 target disks using scsi_debug - sudo modprobe scsi_debug dev_size_mb=100 num_tgts=1 vpd_use_hostno=0 + sudo modprobe scsi_debug dev_size_mb=100 num_tgts=1 vpd_use_hostno=0 add_host=4 delay=20 max_luns=2 no_lun_0=1 - $ sudo multipath -ll - mpatha (1Linux_scsi_debug_2001) dm-0 Linux,scsi_debug - size=100M features='0' hwhandler='1 alua' wp=rw - |-+- policy='service-time 0' prio=1 status=enabled - | `- 2:0:0:1 sda 8:0 active ready running - |-+- policy='service-time 0' prio=1 status=enabled - | `- 5:0:0:1 sdd 8:48 active ready running - |-+- policy='service-time 0' prio=50 status=active - | `- 3:0:0:1 sdb 8:16 active ready running - `-+- policy='service-time 0' prio=50 status=enabled - `- 4:0:0:1 sdc 8:32 active ready running + $ sudo multipath -ll + mpatha (1Linux_scsi_debug_2001) dm-0 Linux,scsi_debug + size=100M features='0' hwhandler='1 alua' wp=rw + |-+- policy='service-time 0' prio=1 status=enabled + | `- 2:0:0:1 sda 8:0 active ready running + |-+- policy='service-time 0' prio=1 status=enabled + | `- 5:0:0:1 sdd 8:48 active ready running + |-+- policy='service-time 0' prio=50 status=active + | `- 3:0:0:1 sdb 8:16 active ready running + `-+- policy='service-time 0' prio=50 status=enabled + `- 4:0:0:1 sdc 8:32 active ready running - Notice that the name is 1Linux_scsi_debug_2001 and it has 4 disks, + Notice that the name is 1Linux_scsi_debug_2001 and it has 4 disks, sda to sdd. - We will use the following path of the multipath device mapper disk + We will use the following path of the multipath device mapper disk when creating the VM, this points to dm-0. - /dev/disk/by-id/dm-uuid-mpath-1Linux_scsi_debug_2001 + /dev/disk/by-id/dm-uuid-mpath-1Linux_scsi_debug_2001 - $ ls -l /dev/disk/by-id/dm-uuid-mpath-1Linux_scsi_debug_2001 - … /dev/disk/by-id/dm-uuid-mpath-1Linux_scsi_debug_2001 -> ../../dm-0 + $ ls -l /dev/disk/by-id/dm-uuid-mpath-1Linux_scsi_debug_2001 + … /dev/disk/by-id/dm-uuid-mpath-1Linux_scsi_debug_2001 -> ../../dm-0 4. Create a new VM with reservations (starts qemu-pr-helper along with - the qemu process for the VM). Use domain.xml from attachment. + the qemu process for the VM). Use domain.xml from attachment: + https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/2117467/+attachment/5910958/+files/domain.xml - Install libvirt first: + Install libvirt first: - sudo apt install libvirt-daemon libvirt-daemon-system libvirt-clients - sudo systemctl start libvirtd + sudo apt install libvirt-daemon libvirt-daemon-system libvirt-clients + sudo systemctl start libvirtd - Create the VM: + Create the VM: - sudo virsh define --file domain.xml --validate - sudo virsh start libvirt-lp2117467 + sudo virsh define --file domain.xml --validate + sudo virsh start libvirt-lp2117467 - Verify it is in running state. + Verify it is in running state. - sudo virsh list --all + sudo virsh list --all 5. Find the mount namespace of the VM, enter the namespace and verify the disks are added there. - Find the PID of qemu-pr-helper or qemu-system-x86 + Find the PID of qemu-pr-helper or qemu-system-x86 - $ pgrep qemu-pr-helper - 734148 + $ pgrep qemu-pr-helper + 734148 - Enter the mount namespace of this process to verify all the target + Enter the mount namespace of this process to verify all the target disks are present. In this example, we don't see the sda to sdd as libvirt is not patched. - $ sudo nsenter --target 734148 --mount bash - root@bhavin-build:/# ls /dev/ - disk dm-0 full hugepages mapper mqueue null ptmx pts random shm urandom zero + $ sudo nsenter --target 734148 --mount bash + root@bhavin-build:/# ls /dev/ + disk dm-0 full hugepages mapper mqueue null ptmx pts random shm urandom zero - When the patch is applied, we should be able to see the target + When the patch is applied, we should be able to see the target devices as well. [ Where problems could occur ] This is a bugfix backport, not able to think any potential problem as of now. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/2117467 Title: Multipath device's targets are not added to domain namespace/cgroup To manage notifications about this bug go to: https://bugs.launchpad.net/libvirt/+bug/2117467/+subscriptions -- ubuntu-bugs mailing list [email protected] https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
