On 10/11/2023 03:15, Duan, Zhenzhong wrote: > Hi Jason, Joao, > >> -----Original Message----- >> From: Jason Gunthorpe <j...@nvidia.com> >> Sent: Thursday, November 9, 2023 10:35 PM >> Subject: Re: [PATCH v4 28/41] vfio/iommufd: Implement the iommufd backend >> >> On Thu, Nov 09, 2023 at 01:21:59PM +0000, Joao Martins wrote: >>> On 09/11/2023 13:09, Jason Gunthorpe wrote: >>>> On Thu, Nov 09, 2023 at 01:03:02PM +0000, Joao Martins wrote: >>>> >>>>>> I am not talking about mdevs; but rather the regular (non mdev) case not >> being >>>>>> able to use dirty tracking with autodomains hwpt allocation. >>>>> >>>>> ... without any vIOMMU. >>>> >>>> Ah, well, that is troublesome isn't it.. >>>> >>>> So do we teach autodomains to be more featured in the kernel or do we >>>> teach the generic qemu code to effectively implement autodomains in >>>> userspace? >>> >>> The latter is actually what we have been doing. Well I wouldn't call >> autodomains >>> in qemu, but rather just allocate a hwpt, instead of attaching the IOAS >>> directly. But well mdevs don't have domains and we overlooked that. I would >> turn >>> the exception into an exception rather than making the norm, doesn't look to >> be >>> much complexity added? >> >> Autodomains are complex because of things like mdev and iommu >> non-uniformity's. Qemu can't just allocate a single HWPT, it needs to >> be annoyingly managed. >> >>> What I last re-collect is that autodomains represents the 'simple users' >>> that >>> don't care much beyond the basics of IOMMU features (I recall the example >> was >>> DPDK apps and the like). You could say that for current needs IOMMU >> autodomains >>> suffices for qemu. >> >> Yes, that was my intention. Aside from that it primarily exists to >> support vfio compatibility >> >>> Connecting autodomains to this enforcing on the hwpt is relatively simple >>> btw, >>> it just needs to connect the dirty tracking flag with same semantic of >>> hwpt-alloc equivalent and pass the hwpt flags into the domain allocation. >> >> Yes >> >>> It's more of what of a question should be the expectations to the user when >>> using ATTACH_HWPT with an IOAS_ID versus direct manipulation of HWPT. I am >>> wondering if dirty tracking is alone here or whether there's more features >>> that >>> start to mud the simplicity of autodomains that would approximate of hwpt- >> alloc. >> >> This is why I had been thinking of a pure HWPT based scheme >> >> So it seems we cannot have a simple model where the generic qmeu layer >> just works in IOAS :( It might as well always work in HWPT and >> understand all the auto domains complexity itself. > > Let me know if there is anything I can do in this series to facilitate > future qemu dirty tracking support of iommufd. Not clear if I should > restore to the manual HWPT_ALLOC method in v4.
If we want to have the closest support as type1-iommu, from what we have been discussing... it sounds like IOAS is the easiest first step to get barebones iommufd support. Which sort of makes sense since this is the introduction of iommufd and it already requires a lot of churn & refactoring to get there. For the new iommufd-only features (nesting/dirty-tracking) we will need the auto domains done by Qemu IIUC -- unless nesting is meant to coexist with autodomains with its own hwpts somehow (?) Right now I don't have the autodomains QEMU equivalent structure in mind to suggest a path in alternative to v5; Looking at the kernel autodomains path, aside from mdev I am not sure yet what annoyances the autodomains path in qemu is going to generate: more worringly whether we have enough information to tackle the non-uniformity e.g. if we are talking about features or whether different devices are behind different IOMMUs.
