>-----Original Message-----
>From: Jason Gunthorpe <j...@nvidia.com>
>Sent: Wednesday, November 8, 2023 8:48 PM
>Subject: Re: [PATCH v4 28/41] vfio/iommufd: Implement the iommufd backend
>
>On Wed, Nov 08, 2023 at 07:16:52AM +0000, Duan, Zhenzhong wrote:
>
>> >> + ret = iommufd_backend_alloc_hwpt(iommufd, vbasedev->devid,
>> >> + container->ioas_id, &hwpt_id);
>> >> +
>> >> + if (ret) {
>> >> + error_setg_errno(errp, errno, "error alloc shadow hwpt");
>> >> + return ret;
>> >> + }
>> >
>> >The above alloc_hwpt fails for mdevs (at least, it fails for me attempting
>> >to
>use
>> >iommufd backend with vfio-ccw and vfio-ap on s390). The ioctl is failing
>> >in the
>> >kernel because it can't find an IOMMUFD_OBJ_DEVICE.
>> >
>> >AFAIU that's because the mdevs are meant to instead use kernel access via
>> >vfio_iommufd_emulated_attach_ioas, not hwpt. That's how mdevs behave
>when
>> >looking at the kernel vfio compat container.
>> >
>> >As a test, I was able to get vfio-ccw and vfio-ap working using the iommufd
>> >backend by just skipping this alloc_hwpt above and instead passing
>> >container-
>> >>ioas_id into the iommufd_cdev_attach_hwpt below. That triggers the
>> >vfio_iommufd_emulated_attach_ioas call in the kernel.
>>
>> Thanks for help test and investigation.
>> I was only focusing on real device and missed the mdev particularity, sorry.
>> You are right, there is no hwpt support for mdev, not even an emulated hwpt.
>> I'll digging into this and see how to distinguish mdev with real device in
>> this low level function.
>
>I was expecting that hwpt manipulation would be done exclusively
>inside the device-specific vIOMMU userspace driver. Generic code paths
>that don't have that knowledge should use the IOAS for everything
Yes, this way we don't need to distinguish between mdev and real device,
just attach to IOAS. But lose the benefit that same hwpt could be passed
into vIOMMU to be used as S2 hwpt in nesting.
If you don't have a strong opinion to use IOAS for everything, I'm thinking
about adding a bool variable is_mdev in VFIODevice, checking this bool
to decide if attach to manually allocated hwpt or IOAS.
For vfio-ap and vfio-ccw, is_mdev is set to true, for vfio-pci, we check
"/sys/bus/mdev" from vbasedev->sysfsdev to decide if it's true.
Another choice is to add VFIO_DEVICE_FLAGS_MDEV in vfio_device_info.flags
on kernel side, qemu can know if this device is mdev by checking the flag from
kernel, this works even in fd passing case.
Thanks
Zhenzhong
