Re: [PATCH kernel] powerpc/iommu: Add simple iommu_ops to report capabilities

2022-07-05 Thread Jason Gunthorpe via iommu
On Tue, Jul 05, 2022 at 04:22:35PM +1000, Alexey Kardashevskiy wrote: > I have not looked into the domains for ages, what is missing here? With this > on top of 5.19-rc1 VFIO works again on my POWER9 box. Thanks, Does this solve all the problems or just coherency? It seems like it should solve ev

Re: [PATCH v3 4/4] vfio: Require that devices support DMA cache coherence

2022-07-04 Thread Jason Gunthorpe via iommu
On Mon, Jul 04, 2022 at 09:21:26PM +0800, chenxiang (M) wrote: > Hi, > > We encounter a issue with the patch: our platform is ARM64, and we run DPDK > with smmu disable on VM (without iommu=smmuv3 etc), > > so we use noiommu mode with enable_unsafe_noiommu_mode=1 to passthrough the > device to VM

Re: [PATCH v3 2/2] vfio: Use device_iommu_capable()

2022-06-24 Thread Jason Gunthorpe via iommu
On Fri, Jun 24, 2022 at 06:59:35PM +0100, Robin Murphy wrote: > Use the new interface to check the capabilities for our device > specifically. > > Reviewed-by: Lu Baolu > Signed-off-by: Robin Murphy > --- > drivers/vfio/vfio.c | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) Reviewed-by

Re: [PATCH v3 1/2] vfio/type1: Simplify bus_type determination

2022-06-24 Thread Jason Gunthorpe via iommu
On Fri, Jun 24, 2022 at 06:51:44PM +0100, Robin Murphy wrote: > Since IOMMU groups are mandatory for drivers to support, it stands to > reason that any device which has been successfully added to a group > must be on a bus supported by that IOMMU driver, and therefore a domain > viable for any devi

Re: [PATCH v3 3/5] vfio/iommu_type1: Remove the domain->ops comparison

2022-06-24 Thread Jason Gunthorpe via iommu
On Fri, Jun 24, 2022 at 03:28:33PM -0300, Jason Gunthorpe wrote: > On Thu, Jun 23, 2022 at 01:00:27PM -0700, Nicolin Chen wrote: > > The domain->ops validation was added, as a precaution, for mixed-driver > > systems. > > > > Per Robin's remarks, > > * While bus_set_iommu() still exists, the core

Re: [PATCH v2 3/5] vfio/iommu_type1: Remove the domain->ops comparison

2022-06-24 Thread Jason Gunthorpe via iommu
On Fri, Jun 24, 2022 at 07:31:47PM +0100, Robin Murphy wrote: > > > Oh, physical platforms with mixed IOMMUs definitely exist already. The > > > main > > > point is that while bus_set_iommu still exists, the core code effectively > > > *does* prevent multiple drivers from registering - even in em

Re: [PATCH v3 3/5] vfio/iommu_type1: Remove the domain->ops comparison

2022-06-24 Thread Jason Gunthorpe via iommu
On Thu, Jun 23, 2022 at 01:00:27PM -0700, Nicolin Chen wrote: > The domain->ops validation was added, as a precaution, for mixed-driver > systems. > > Per Robin's remarks, > * While bus_set_iommu() still exists, the core code prevents multiple > drivers from registering, so we can't really run i

Re: [PATCH v3 1/5] iommu: Return -EMEDIUMTYPE for incompatible domain and device/group

2022-06-24 Thread Jason Gunthorpe via iommu
On Fri, Jun 24, 2022 at 06:35:49PM +0800, Yong Wu wrote: > > > It's not used in VFIO context. "return 0" just satisfy the iommu > > > framework to go ahead. and yes, here we only allow the shared > > > "mapping-domain" (All the devices share a domain created > > > internally). What part of the io

Re: [PATCH v2 1/2] vfio/type1: Simplify bus_type determination

2022-06-24 Thread Jason Gunthorpe via iommu
On Fri, Jun 24, 2022 at 08:28:31AM -0600, Alex Williamson wrote: > > > That's essentially what I'm suggesting, the vfio_group is passed as an > > > opaque pointer which type1 can use for a > > > vfio_group_for_each_vfio_device() type call. Thanks, > > > > I don't want to add a whole vfio_group

Re: [PATCH v2 1/2] vfio/type1: Simplify bus_type determination

2022-06-24 Thread Jason Gunthorpe via iommu
On Fri, Jun 24, 2022 at 08:11:59AM -0600, Alex Williamson wrote: > On Thu, 23 Jun 2022 22:50:30 -0300 > Jason Gunthorpe wrote: > > > On Thu, Jun 23, 2022 at 05:00:44PM -0600, Alex Williamson wrote: > > > > > > >> +struct vfio_device *vfio_device_get_from_iommu(struct iommu_group > > > > >> *iom

Re: [PATCH 1/2] vfio/type1: Simplify bus_type determination

2022-06-24 Thread Jason Gunthorpe via iommu
On Tue, Jun 21, 2022 at 08:09:20PM +0100, Robin Murphy wrote: > Quick consensus then: does anyone have a particular preference between > changing the .attach_group signature vs. adding a helper based on > vfio_group_get_from_iommu() for type1 to call from within its callback? They > seem about equ

Re: [PATCH 1/3] iommu: Add Visconti5 IOMMU driver

2022-06-24 Thread Jason Gunthorpe via iommu
On Mon, Jun 20, 2022 at 05:49:13AM +, nobuhiro1.iwama...@toshiba.co.jp wrote: > Hi, > > Thanks for your review. > > > -Original Message- > > From: Jason Gunthorpe > > Sent: Thursday, May 26, 2022 3:27 AM > > To: Baolu Lu > > Cc: iwamatsu nobuhiro(岩松 信洋 □SWC◯ACT) > > ; Joerg Roedel

Re: [PATCH v2 3/5] vfio/iommu_type1: Remove the domain->ops comparison

2022-06-24 Thread Jason Gunthorpe via iommu
On Wed, Jun 22, 2022 at 08:54:45AM +0100, Robin Murphy wrote: > On 2022-06-16 23:23, Nicolin Chen wrote: > > On Thu, Jun 16, 2022 at 06:40:14AM +, Tian, Kevin wrote: > > > > > > The domain->ops validation was added, as a precaution, for mixed-driver > > > > systems. However, at this moment onl

Re: [PATCH v2 1/2] vfio/type1: Simplify bus_type determination

2022-06-23 Thread Jason Gunthorpe via iommu
On Wed, Jun 22, 2022 at 01:04:11PM +0100, Robin Murphy wrote: > +struct vfio_device *vfio_device_get_from_iommu(struct iommu_group > *iommu_group) > +{ > + struct vfio_group *group = vfio_group_get_from_iommu(iommu_group); > + struct vfio_device *device; > + > + mutex_lock(&group->dev

Re: [PATCH v2 1/2] vfio/type1: Simplify bus_type determination

2022-06-23 Thread Jason Gunthorpe via iommu
On Thu, Jun 23, 2022 at 05:00:44PM -0600, Alex Williamson wrote: > > >> +struct vfio_device *vfio_device_get_from_iommu(struct iommu_group > > >> *iommu_group) > > >> +{ > > >> +struct vfio_group *group = > > >> vfio_group_get_from_iommu(iommu_group); > > >> +struct vfio_device *

Re: [PATCH v2 1/2] vfio/type1: Simplify bus_type determination

2022-06-23 Thread Jason Gunthorpe via iommu
On Thu, Jun 23, 2022 at 01:23:05PM +0100, Robin Murphy wrote: > So yes, technically we could implement an iommu_group_capable() and an > iommu_group_domain_alloc(), which would still just internally resolve the > IOMMU ops and instance data from a member device to perform the driver-level > call,

Re: New IOMMU mailing list coming

2022-06-23 Thread Jason Gunthorpe via iommu
On Tue, Jun 14, 2022 at 10:30:20AM +0200, Jörg Rödel wrote: > Hi all, > > after several problems with the current IOMMU mailing list (no DKIM > support, poor b4 interoperability) we have decided to move the IOMMU > development discussions to a new list hosted at lists.linux.dev. > > The new list

Re: [RFC PATCHES 1/2] iommu: Add RCU-protected page free support

2022-06-19 Thread Jason Gunthorpe via iommu
On Thu, Jun 16, 2022 at 10:27:02AM +0800, Baolu Lu wrote: > On 2022/6/15 23:40, Jason Gunthorpe wrote: > > On Fri, Jun 10, 2022 at 01:37:20PM +0800, Baolu Lu wrote: > > > On 2022/6/9 20:49, Jason Gunthorpe wrote: > > > > > +void iommu_free_pgtbl_pages(struct iommu_domain *domain, > > > > > +

Re: [PATCH v2 5/5] vfio/iommu_type1: Simplify group attachment

2022-06-19 Thread Jason Gunthorpe via iommu
On Fri, Jun 17, 2022 at 04:07:20PM -0700, Nicolin Chen wrote: > > > > > + vfio_iommu_aper_expand(iommu, &iova_copy); > > > > > > > > but now it's done for every group detach. The aperture is decided > > > > by domain geometry which is not affected by attached groups. > > > > > > Yea, I've noti

Re: [RFC PATCHES 1/2] iommu: Add RCU-protected page free support

2022-06-15 Thread Jason Gunthorpe via iommu
On Fri, Jun 10, 2022 at 01:37:20PM +0800, Baolu Lu wrote: > On 2022/6/9 20:49, Jason Gunthorpe wrote: > > > +void iommu_free_pgtbl_pages(struct iommu_domain *domain, > > > + struct list_head *pages) > > > +{ > > > + struct page *page, *next; > > > + > > > + if (!domain->concurre

Re: [PATCH 1/2] vfio/type1: Simplify bus_type determination

2022-06-09 Thread Jason Gunthorpe via iommu
On Wed, Jun 08, 2022 at 03:25:49PM +0100, Robin Murphy wrote: > Since IOMMU groups are mandatory for drivers to support, it stands to > reason that any device which has been successfully be added to a group > must be on a bus supported by that IOMMU driver, and therefore a domain > viable for any d

Re: [PATCH v8 01/11] iommu: Add max_pasids field in struct iommu_device

2022-06-09 Thread Jason Gunthorpe via iommu
On Thu, Jun 09, 2022 at 05:25:42PM +, Raj, Ashok wrote: > > On Tue, Jun 07, 2022 at 09:49:32AM +0800, Lu Baolu wrote: > > Use this field to keep the number of supported PASIDs that an IOMMU > > hardware is able to support. This is a generic attribute of an IOMMU > > and lifting it into the per

Re: [RFC PATCHES 1/2] iommu: Add RCU-protected page free support

2022-06-09 Thread Jason Gunthorpe via iommu
On Thu, Jun 09, 2022 at 02:19:06PM +0100, Robin Murphy wrote: > Is there a significant benefit to keeping both paths, or could we get away > with just always using RCU? Realistically, pagetable pages aren't likely to > be freed all that frequently, except perhaps at domain teardown, but that > sho

Re: [RFC PATCHES 1/2] iommu: Add RCU-protected page free support

2022-06-09 Thread Jason Gunthorpe via iommu
On Thu, Jun 09, 2022 at 03:08:10PM +0800, Lu Baolu wrote: > The IOMMU page tables are updated using iommu_map/unmap() interfaces. > Currently, there is no mandatory requirement for drivers to use locks > to ensure concurrent updates to page tables, because it's assumed that > overlapping IOVA range

Re: [PATCH 3/5] vfio/iommu_type1: Prefer to reuse domains vs match enforced cache coherency

2022-06-08 Thread Jason Gunthorpe via iommu
On Wed, Jun 08, 2022 at 08:28:03AM +, Tian, Kevin wrote: > > From: Nicolin Chen > > Sent: Monday, June 6, 2022 2:19 PM > > > > From: Jason Gunthorpe > > > > The KVM mechanism for controlling wbinvd is only triggered during > > kvm_vfio_group_add(), meaning it is a one-shot test done once the

Re: [PATCH 0/5] Simplify vfio_iommu_type1 attach/detach routine

2022-06-07 Thread Jason Gunthorpe via iommu
On Tue, Jun 07, 2022 at 03:44:43PM +0800, Baolu Lu wrote: > On 2022/6/6 14:19, Nicolin Chen wrote: > > Worths mentioning the exact match for enforce_cache_coherency is removed > > with this series, since there's very less value in doing that since KVM > > won't be able to take advantage of it -- th

Re: [PATCH 2/5] iommu: Ensure device has the same iommu_ops as the domain

2022-06-06 Thread Jason Gunthorpe via iommu
On Mon, Jun 06, 2022 at 11:28:49AM -0700, Nicolin Chen wrote: > > Well, as before I'd prefer to make the code match the commit message - > > if I really need to spell it out, see below - since I can't imagine that > > we should ever have need to identify a set of iommu_domain_ops in > > isolation,

Re: [PATCH 01/12] iommu/vt-d: Use iommu_get_domain_for_dev() in debugfs

2022-06-01 Thread Jason Gunthorpe via iommu
On Wed, Jun 01, 2022 at 02:52:05PM +0100, Joao Martins wrote: > On 6/1/22 13:33, Jason Gunthorpe wrote: > > On Wed, Jun 01, 2022 at 01:18:52PM +0100, Joao Martins wrote: > > > >>> So having safe racy reading in the kernel is probably best, and so RCU > >>> would be good here too. > >> > >> Reading

Re: [PATCH 01/12] iommu/vt-d: Use iommu_get_domain_for_dev() in debugfs

2022-06-01 Thread Jason Gunthorpe via iommu
On Wed, Jun 01, 2022 at 01:18:52PM +0100, Joao Martins wrote: > > So having safe racy reading in the kernel is probably best, and so RCU > > would be good here too. > > Reading dirties ought to be similar to map/unmap but slightly simpler as > I supposedly don't need to care about the pte changin

Re: [PATCH 01/12] iommu/vt-d: Use iommu_get_domain_for_dev() in debugfs

2022-05-31 Thread Jason Gunthorpe via iommu
On Tue, May 31, 2022 at 10:22:32PM +0100, Robin Murphy wrote: > There are only 3 instances where we'll free a table while the domain is > live. The first is the one legitimate race condition, where two map requests > targeting relatively nearby PTEs both go to fill in an intermediate level of > ta

Re: [PATCH v4 1/6] iommu: Add a per domain PASID for DMA API

2022-05-31 Thread Jason Gunthorpe via iommu
On Tue, May 31, 2022 at 10:29:55AM -0700, Jacob Pan wrote: > The reason why I store PASID at IOMMU domain is for IOTLB flush within the > domain. Device driver is not aware of domain level IOTLB flush. We also > have iova_cookie for each domain which essentially is for RIDPASID. You need to make

Re: [PATCH 01/12] iommu/vt-d: Use iommu_get_domain_for_dev() in debugfs

2022-05-31 Thread Jason Gunthorpe via iommu
On Tue, May 31, 2022 at 07:07:32PM +0100, Robin Murphy wrote: > > And we expect the iommu driver to be unable to free page table levels > > that have IOVA boundaries in them? > > I'm not entirely sure what you mean there, but in general an unmap request > is expected to match some previous map re

Re: [PATCH 01/12] iommu/vt-d: Use iommu_get_domain_for_dev() in debugfs

2022-05-31 Thread Jason Gunthorpe via iommu
On Tue, May 31, 2022 at 05:01:46PM +0100, Robin Murphy wrote: > The DMA API doesn't need locking, partly since it can trust itself not to do > stupid things, and mostly because it's DMA API performance that's > fundamentally incompatible with serialisation anyway. Why do you think we > have a comp

Re: [PATCH v4 1/6] iommu: Add a per domain PASID for DMA API

2022-05-31 Thread Jason Gunthorpe via iommu
On Tue, May 31, 2022 at 08:45:28PM +0800, Baolu Lu wrote: > My understanding is that device driver owns its PASID policy. I'm not sure this is actually useful, the core code should provide the allocator as I can't think of any device that actually needs a special allocator. > If ENQCMD is suppor

Re: [PATCH 01/12] iommu/vt-d: Use iommu_get_domain_for_dev() in debugfs

2022-05-31 Thread Jason Gunthorpe via iommu
On Tue, May 31, 2022 at 02:52:28PM +0100, Robin Murphy wrote: > > +    break; > > +    pgtable_walk_level(m, phys_to_virt(phys_addr), > > Also, obligatory reminder that pfn_valid() only means that pfn_to_page() > gets you a valid struct page. Whether that page is direct-mapped

Re: [PATCH 01/12] iommu/vt-d: Use iommu_get_domain_for_dev() in debugfs

2022-05-31 Thread Jason Gunthorpe via iommu
On Tue, May 31, 2022 at 04:01:04PM +0100, Robin Murphy wrote: > On 2022-05-31 15:53, Jason Gunthorpe wrote: > > On Tue, May 31, 2022 at 10:11:18PM +0800, Baolu Lu wrote: > > > On 2022/5/31 21:10, Jason Gunthorpe wrote: > > > > On Tue, May 31, 2022 at 11:02:06AM +0800, Baolu Lu wrote: > > > > > > >

Re: [PATCH 01/12] iommu/vt-d: Use iommu_get_domain_for_dev() in debugfs

2022-05-31 Thread Jason Gunthorpe via iommu
On Tue, May 31, 2022 at 10:11:18PM +0800, Baolu Lu wrote: > On 2022/5/31 21:10, Jason Gunthorpe wrote: > > On Tue, May 31, 2022 at 11:02:06AM +0800, Baolu Lu wrote: > > > > > For case 2, it is a bit weird. I tried to add a rwsem lock to make the > > > iommu_unmap() and dumping tables in debugfs ex

Re: [PATCH 01/12] iommu/vt-d: Use iommu_get_domain_for_dev() in debugfs

2022-05-31 Thread Jason Gunthorpe via iommu
On Tue, May 31, 2022 at 11:02:06AM +0800, Baolu Lu wrote: > For case 2, it is a bit weird. I tried to add a rwsem lock to make the > iommu_unmap() and dumping tables in debugfs exclusive. This does not > work because debugfs may depend on the DMA of the devices to work. It > seems that what we can

Re: [PATCH v4 1/6] iommu: Add a per domain PASID for DMA API

2022-05-30 Thread Jason Gunthorpe via iommu
On Tue, May 24, 2022 at 08:17:27AM -0700, Jacob Pan wrote: > Hi Jason, > > On Tue, 24 May 2022 10:50:34 -0300, Jason Gunthorpe wrote: > > > On Wed, May 18, 2022 at 11:21:15AM -0700, Jacob Pan wrote: > > > DMA requests tagged with PASID can target individual IOMMU domains. > > > Introduce a domai

Re: [PATCH 01/12] iommu/vt-d: Use iommu_get_domain_for_dev() in debugfs

2022-05-30 Thread Jason Gunthorpe via iommu
On Sun, May 29, 2022 at 01:14:46PM +0800, Baolu Lu wrote: > From 1e87b5df40c6ce9414cdd03988c3b52bfb17af5f Mon Sep 17 00:00:00 2001 > From: Lu Baolu > Date: Sun, 29 May 2022 10:18:56 +0800 > Subject: [PATCH 1/1] iommu/vt-d: debugfs: Remove device_domain_lock usage > > The domain_translation_struc

Re: [PATCH 09/12] iommu/vt-d: Check device list of domain in domain free path

2022-05-27 Thread Jason Gunthorpe via iommu
On Fri, May 27, 2022 at 02:30:16PM +0800, Lu Baolu wrote: > When the IOMMU domain is about to be freed, it should not be set on any > device. Instead of silently dealing with some bug cases, it's better to > trigger a warning to report and fix any potential bugs at the first time. > > Signed-off-b

Re: [PATCH 04/12] iommu/vt-d: Use pci_get_domain_bus_and_slot() in pgtable_walk()

2022-05-27 Thread Jason Gunthorpe via iommu
On Fri, May 27, 2022 at 02:30:11PM +0800, Lu Baolu wrote: > Use pci_get_domain_bus_and_slot() instead of searching the global list > to retrieve the pci device pointer. This removes device_domain_list > global list as there are no consumers anymore. > > Signed-off-by: Lu Baolu > --- > drivers/io

Re: [PATCH 03/12] iommu/vt-d: Remove clearing translation data in disable_dmar_iommu()

2022-05-27 Thread Jason Gunthorpe via iommu
On Fri, May 27, 2022 at 02:30:10PM +0800, Lu Baolu wrote: > The disable_dmar_iommu() is called when IOMMU initialzation fails or > the IOMMU is hot-removed from the system. In both cases, there is no > need to clear the IOMMU translation data structures for devices. > > On the initialization path,

Re: [PATCH 02/12] iommu/vt-d: Remove for_each_device_domain()

2022-05-27 Thread Jason Gunthorpe via iommu
On Fri, May 27, 2022 at 02:30:09PM +0800, Lu Baolu wrote: > The per-device device_domain_info data could be retrieved from the > device itself. There's no need to search a global list. > > Signed-off-by: Lu Baolu > --- > drivers/iommu/intel/iommu.h | 2 -- > drivers/iommu/intel/iommu.c | 25 ---

Re: [PATCH 01/12] iommu/vt-d: Use iommu_get_domain_for_dev() in debugfs

2022-05-27 Thread Jason Gunthorpe via iommu
On Fri, May 27, 2022 at 02:30:08PM +0800, Lu Baolu wrote: > Retrieve the attached domain for a device through the generic interface > exposed by the iommu core. This also makes device_domain_lock static. > > Signed-off-by: Lu Baolu > drivers/iommu/intel/iommu.h | 1 - > drivers/iommu/intel/de

Re: [PATCH] drivers: iommu: Directly use ida_alloc()/free()

2022-05-27 Thread Jason Gunthorpe via iommu
On Fri, May 27, 2022 at 07:03:07AM +, keliu wrote: > Use ida_alloc()/ida_free() instead of deprecated > ida_simple_get()/ida_simple_remove() . > > Signed-off-by: keliu > --- > drivers/iommu/iommu.c | 6 +++--- > 1 file changed, 3 insertions(+), 3 deletions(-) Reviewed-by: Jason Gunthorpe

Re: [PATCH 1/3] iommu: Add Visconti5 IOMMU driver

2022-05-25 Thread Jason Gunthorpe via iommu
On Thu, May 26, 2022 at 12:43:25AM +, Tian, Kevin wrote: > iiuc then this suggests there should be only one iommu group per atu, > instead of using generic_device_group() to create one group per > device. Sounds like it > > rejected then this driver should have a IOMMU_DOMAIN_BLOCKING and >

Re: [PATCH 1/3] iommu: Add Visconti5 IOMMU driver

2022-05-25 Thread Jason Gunthorpe via iommu
On Wed, May 25, 2022 at 02:26:37PM +0800, Baolu Lu wrote: > On 2022/5/25 09:31, Nobuhiro Iwamatsu wrote: > > +static const struct iommu_ops visconti_atu_ops = { > > + .domain_alloc = visconti_atu_domain_alloc, > > + .probe_device = visconti_atu_probe_device, > > + .release_device = visconti_a

Re: [PATCH v7 03/10] iommu/sva: Add iommu_sva_domain support

2022-05-25 Thread Jason Gunthorpe via iommu
On Wed, May 25, 2022 at 01:19:08PM +0800, Baolu Lu wrote: > Hi Jason, > > On 2022/5/24 21:44, Jason Gunthorpe wrote: > > > diff --git a/drivers/iommu/iommu-sva-lib.c b/drivers/iommu/iommu-sva-lib.c > > > index 106506143896..210c376f6043 100644 > > > +++ b/drivers/iommu/iommu-sva-lib.c > > > @@ -69

Re: [PATCH v4 3/6] iommu/vt-d: Implement domain ops for attach_dev_pasid

2022-05-24 Thread Jason Gunthorpe via iommu
On Tue, May 24, 2022 at 01:45:26PM -0700, Jacob Pan wrote: > > The idea that there is only one PASID per domain per device is not > > right. > > > Got you, I was under the impression that there is no use case yet for > multiple PASIDs per device-domain based on our early discussion. The key word

Re: [PATCH v4 3/6] iommu/vt-d: Implement domain ops for attach_dev_pasid

2022-05-24 Thread Jason Gunthorpe via iommu
On Tue, May 24, 2022 at 09:12:35AM -0700, Jacob Pan wrote: > Hi Jason, > > On Tue, 24 May 2022 10:51:35 -0300, Jason Gunthorpe wrote: > > > On Wed, May 18, 2022 at 11:21:17AM -0700, Jacob Pan wrote: > > > On VT-d platforms with scalable mode enabled, devices issue DMA requests > > > with PASID n

Re: [PATCH v4 3/6] iommu/vt-d: Implement domain ops for attach_dev_pasid

2022-05-24 Thread Jason Gunthorpe via iommu
On Wed, May 18, 2022 at 11:21:17AM -0700, Jacob Pan wrote: > On VT-d platforms with scalable mode enabled, devices issue DMA requests > with PASID need to attach PASIDs to given IOMMU domains. The attach > operation involves the following: > - Programming the PASID into the device's PASID table > -

Re: [PATCH v4 1/6] iommu: Add a per domain PASID for DMA API

2022-05-24 Thread Jason Gunthorpe via iommu
On Wed, May 18, 2022 at 11:21:15AM -0700, Jacob Pan wrote: > DMA requests tagged with PASID can target individual IOMMU domains. > Introduce a domain-wide PASID for DMA API, it will be used on the same > mapping as legacy DMA without PASID. Let it be IOVA or PA in case of > identity domain. Huh? I

Re: [PATCH v7 03/10] iommu/sva: Add iommu_sva_domain support

2022-05-24 Thread Jason Gunthorpe via iommu
On Thu, May 19, 2022 at 03:20:40PM +0800, Lu Baolu wrote: > The iommu_sva_domain represents a hardware pagetable that the IOMMU > hardware could use for SVA translation. This adds some infrastructure > to support SVA domain in the iommu common layer. It includes: > > - Add a new struct iommu_sva_d

Re: [PATCH v7 03/10] iommu/sva: Add iommu_sva_domain support

2022-05-24 Thread Jason Gunthorpe via iommu
On Tue, May 24, 2022 at 09:39:52AM +, Tian, Kevin wrote: > > From: Lu Baolu > > Sent: Thursday, May 19, 2022 3:21 PM > > > > The iommu_sva_domain represents a hardware pagetable that the IOMMU > > hardware could use for SVA translation. This adds some infrastructure > > to support SVA domain

Re: [PATCH RFC 11/12] iommufd: vfio container FD ioctl compatibility

2022-05-24 Thread Jason Gunthorpe via iommu
On Mon, May 23, 2022 at 04:02:22PM +1000, Alexey Kardashevskiy wrote: > Which means the guest RAM does not need to be all mapped in that base IOAS > suggested down this thread as that would mean all memory is pinned and > powervm won't be able to swap it out (yeah, it can do such thing now!). Not

Re: [PATCH] vfio: Remove VFIO_TYPE1_NESTING_IOMMU

2022-05-20 Thread Jason Gunthorpe via iommu
On Fri, May 20, 2022 at 01:00:32PM +0100, Will Deacon wrote: > On Fri, May 20, 2022 at 12:02:23PM +0100, Robin Murphy wrote: > > On 2022-05-10 17:55, Jason Gunthorpe via iommu wrote: > > > This control causes the ARM SMMU drivers to choose a stage 2 > > > implementati

Re: [PATCH] vfio: Remove VFIO_TYPE1_NESTING_IOMMU

2022-05-20 Thread Jason Gunthorpe via iommu
On Fri, May 20, 2022 at 12:02:23PM +0100, Robin Murphy wrote: > On 2022-05-10 17:55, Jason Gunthorpe via iommu wrote: > > This control causes the ARM SMMU drivers to choose a stage 2 > > implementation for the IO pagetable (vs the stage 1 usual default), > > however this

Re: [PATCH] iommu/s390: tolerate repeat attach_dev calls

2022-05-19 Thread Jason Gunthorpe via iommu
On Thu, May 19, 2022 at 02:29:29PM -0400, Matthew Rosato wrote: > Since commit 0286300e6045 ("iommu: iommu_group_claim_dma_owner() must > always assign a domain") s390-iommu will get called to allocate multiple > unmanaged iommu domains for a vfio-pci device -- however the current > s390-iommu logi

[PATCH] vfio: Do not manipulate iommu dma_owner for fake iommu groups

2022-05-19 Thread Jason Gunthorpe via iommu
Since asserting dma ownership now causes the group to have its DMA blocked the iommu layer requires a working iommu. This means the dma_owner APIs cannot be used on the fake groups that VFIO creates. Test for this and avoid calling them. Otherwise asserting dma ownership will fail for VFIO mdev de

Re: [PATCH v3] iommu: iommu_group_claim_dma_owner() must always assign a domain

2022-05-19 Thread Jason Gunthorpe via iommu
On Thu, May 19, 2022 at 10:51:47AM -0600, Alex Williamson wrote: > On Thu, 19 May 2022 09:32:05 +0200 > Joerg Roedel wrote: > > > On Wed, May 18, 2022 at 04:14:46PM -0300, Jason Gunthorpe wrote: > > > Fixes: 0286300e6045 ("iommu: iommu_group_claim_dma_owner() must always > > > assign a domain")

Re: [PATCH RFC 10/12] iommufd: Add kAPI toward external drivers

2022-05-19 Thread Jason Gunthorpe via iommu
On Thu, May 19, 2022 at 05:45:06PM +0800, Yi Liu wrote: > Hi Jason, > > On 2022/3/19 01:27, Jason Gunthorpe wrote: > > > +/** > > + * iommufd_device_attach - Connect a device to an iommu_domain > > + * @idev: device to attach > > + * @pt_id: Input a IOMMUFD_OBJ_IOAS, or IOMMUFD_OBJ_HW_PAGETABLE >

Re: [PATCH v3] iommu: iommu_group_claim_dma_owner() must always assign a domain

2022-05-18 Thread Jason Gunthorpe via iommu
On Wed, May 18, 2022 at 02:50:36PM -0400, Eric Farman wrote: > I got a heads up from Matt about the s390 KVM vfio- variants failing on > linux-next. > > For vfio-ap and vfio-ccw, they fail on the above error. Both calls to > __iommu_domain_alloc fail because while dev->dev->bus is non-NULL (it >

Re: [PATCH v3 1/4] iommu/vt-d: Implement domain ops for attach_dev_pasid

2022-05-18 Thread Jason Gunthorpe via iommu
On Wed, May 18, 2022 at 11:42:04AM -0700, Jacob Pan wrote: > > Yes.. It seems inefficient to iterate over that xarray multiple times > > on the flush hot path, but maybe there is little choice. Try to use > > use the xas iterators under the xa_lock spinlock.. > > > xas_for_each takes a max range,

Re: [PATCH 2/5] iommu: Add blocking_domain_ops field in iommu_ops

2022-05-17 Thread Jason Gunthorpe via iommu
On Tue, May 17, 2022 at 01:43:03PM +0100, Robin Murphy wrote: > FWIW from my point of view I'm happy with having a .detach_dev_pasid op > meaning implicitly-blocked access for now. If this is the path then lets not call it attach/detach please. 'set_dev_pasid' and 'set_dev_blocking_pasid' are cl

Re: [PATCH 2/5] iommu: Add blocking_domain_ops field in iommu_ops

2022-05-17 Thread Jason Gunthorpe via iommu
On Tue, May 17, 2022 at 10:37:55AM +0800, Baolu Lu wrote: > Hi Jason, > > On 2022/5/16 21:57, Jason Gunthorpe wrote: > > On Mon, May 16, 2022 at 12:22:08PM +0100, Robin Murphy wrote: > > > On 2022-05-16 02:57, Lu Baolu wrote: > > > > Each IOMMU driver must provide a blocking domain ops. If the har

Re: [PATCH 6/7] x86/boot/tboot: Move tboot_force_iommu() to Intel IOMMU

2022-05-17 Thread Jason Gunthorpe via iommu
On Tue, May 17, 2022 at 10:05:43AM +0800, Baolu Lu wrote: > Hi Jason, > > On 2022/5/17 02:06, Jason Gunthorpe wrote: > > > +static __init int tboot_force_iommu(void) > > > +{ > > > + if (!tboot_enabled()) > > > + return 0; > > > + > > > + if (no_iommu || dmar_disabled) > > > + pr_w

Re: [PATCH 7/7] iommu/vt-d: Move include/linux/intel_iommu.h under iommu

2022-05-16 Thread Jason Gunthorpe via iommu
On Sat, May 14, 2022 at 09:43:22AM +0800, Lu Baolu wrote: > This header file is private to the Intel IOMMU driver. Move it to the > driver folder. > > Signed-off-by: Lu Baolu > --- > include/linux/intel-iommu.h => drivers/iommu/intel/iommu.h | 0 > drivers/iommu/intel/trace.h

Re: [PATCH 6/7] x86/boot/tboot: Move tboot_force_iommu() to Intel IOMMU

2022-05-16 Thread Jason Gunthorpe via iommu
On Sat, May 14, 2022 at 09:43:21AM +0800, Lu Baolu wrote: > tboot_force_iommu() is only called by the Intel IOMMU driver. Move the > helper into that driver. No functional change intended. > > Signed-off-by: Lu Baolu > --- > include/linux/tboot.h | 2 -- > arch/x86/kernel/tboot.c | 15

Re: [PATCH 5/7] KVM: x86: Remove unnecessary include

2022-05-16 Thread Jason Gunthorpe via iommu
On Sat, May 14, 2022 at 09:43:20AM +0800, Lu Baolu wrote: > intel-iommu.h is not needed in kvm/x86 anymore. Remove its include. > > Signed-off-by: Lu Baolu > --- > arch/x86/kvm/x86.c | 1 - > 1 file changed, 1 deletion(-) Reviewed-by: Jason Gunthorpe Jason

Re: [PATCH 4/7] drm/i915: Remove unnecessary include

2022-05-16 Thread Jason Gunthorpe via iommu
On Sat, May 14, 2022 at 09:43:19AM +0800, Lu Baolu wrote: > intel-iommu.h is not needed in drm/i915 anymore. Remove its include. > > Signed-off-by: Lu Baolu > --- > drivers/gpu/drm/i915/i915_drv.h| 1 - > drivers/gpu/drm/i915/display/intel_display.c | 1 - > drivers/gpu/drm/i91

Re: [PATCH 3/7] iommu/vt-d: Remove unnecessary exported symbol

2022-05-16 Thread Jason Gunthorpe via iommu
On Sat, May 14, 2022 at 09:43:18AM +0800, Lu Baolu wrote: > The exported symbol intel_iommu_gfx_mapped is not used anywhere in the > tree. Remove it to avoid dead code. > > Signed-off-by: Lu Baolu > --- > include/linux/intel-iommu.h | 1 - > drivers/iommu/intel/iommu.c | 6 -- > 2 files chan

Re: [PATCH 2/7] agp/intel: Use per device iommu check

2022-05-16 Thread Jason Gunthorpe via iommu
On Sat, May 14, 2022 at 09:43:17AM +0800, Lu Baolu wrote: > The IOMMU subsystem has already provided an interface to query whether > the IOMMU hardware is enabled for a specific device. This changes the > check from Intel specific intel_iommu_gfx_mapped (globally exported by > the Intel IOMMU drive

Re: [PATCH 1/7] iommu/vt-d: Move trace/events/intel_iommu.h under iommu

2022-05-16 Thread Jason Gunthorpe via iommu
On Sat, May 14, 2022 at 09:43:16AM +0800, Lu Baolu wrote: > This header file is private to the Intel IOMMU driver. Move it to the > driver folder. > > Signed-off-by: Lu Baolu > --- > .../trace/events/intel_iommu.h => drivers/iommu/intel/trace.h | 4 > drivers/iommu/intel/dmar.c

Re: [PATCH 2/5] iommu: Add blocking_domain_ops field in iommu_ops

2022-05-16 Thread Jason Gunthorpe via iommu
On Mon, May 16, 2022 at 12:22:08PM +0100, Robin Murphy wrote: > On 2022-05-16 02:57, Lu Baolu wrote: > > Each IOMMU driver must provide a blocking domain ops. If the hardware > > supports detaching domain from device, setting blocking domain equals > > detaching the existing domain from the deivce.

Re: [PATCH 2/5] iommu: Add blocking_domain_ops field in iommu_ops

2022-05-16 Thread Jason Gunthorpe via iommu
On Mon, May 16, 2022 at 12:27:41AM -0700, Christoph Hellwig wrote: > On Mon, May 16, 2022 at 09:57:56AM +0800, Lu Baolu wrote: > > Each IOMMU driver must provide a blocking domain ops. If the hardware > > supports detaching domain from device, setting blocking domain equals > > detaching the existi

Re: [RESEND PATCH v8 00/11] Fix BUG_ON in vfio_iommu_group_notifier()

2022-05-13 Thread Jason Gunthorpe via iommu
On Fri, May 13, 2022 at 10:25:48AM -0600, Alex Williamson wrote: > On Fri, 13 May 2022 17:49:44 +0200 > Joerg Roedel wrote: > > > Hi Alex, > > > > On Wed, May 04, 2022 at 10:29:56AM -0600, Alex Williamson wrote: > > > Done, and thanks for the heads-up. Please try to cc me when the > > > vfio-no

Re: [PATCH] iommu/vt-d: Try info->iommu in device_to_iommu()

2022-05-13 Thread Jason Gunthorpe via iommu
On Fri, May 13, 2022 at 11:32:11AM +0800, Baolu Lu wrote: > On 2022/5/13 08:32, Nicolin Chen wrote: > > Local boot test and VFIO sanity test show that info->iommu can be > > used in device_to_iommu() as a fast path. So this patch adds it. > > > > Signed-off-by: Nicolin Chen > > drivers/iommu/in

Re: [PATCH v6 08/12] iommu/sva: Use attach/detach_pasid_dev in SVA interfaces

2022-05-12 Thread Jason Gunthorpe via iommu
On Thu, May 12, 2022 at 07:59:41PM +0800, Baolu Lu wrote: > On 2022/5/12 19:48, Jason Gunthorpe wrote: > > On Thu, May 12, 2022 at 01:17:08PM +0800, Baolu Lu wrote: > > > On 2022/5/12 13:01, Tian, Kevin wrote: > > > > > From: Baolu Lu > > > > > Sent: Thursday, May 12, 2022 11:03 AM > > > > > > >

Re: [PATCH] vfio: Remove VFIO_TYPE1_NESTING_IOMMU

2022-05-12 Thread Jason Gunthorpe via iommu
On Thu, May 12, 2022 at 07:57:26PM +0800, zhangfei@foxmail.com wrote: > > > On 2022/5/12 下午7:32, Jason Gunthorpe via iommu wrote: > > On Thu, May 12, 2022 at 03:07:09PM +0800, Zhangfei Gao wrote: > > > > > I can't help feeling a little wary about removi

Re: [PATCH v3 1/4] iommu/vt-d: Implement domain ops for attach_dev_pasid

2022-05-12 Thread Jason Gunthorpe via iommu
On Thu, May 12, 2022 at 02:22:03PM +0800, Baolu Lu wrote: > Hi Jason, > > On 2022/5/12 01:00, Jason Gunthorpe wrote: > > > Consolidate pasid programming into dev_set_pasid() then called by both > > > intel_svm_attach_dev_pasid() and intel_iommu_attach_dev_pasid(), right? > > I was only suggesting

Re: [PATCH v6 08/12] iommu/sva: Use attach/detach_pasid_dev in SVA interfaces

2022-05-12 Thread Jason Gunthorpe via iommu
On Thu, May 12, 2022 at 11:02:39AM +0800, Baolu Lu wrote: > > > + mutex_lock(&group->mutex); > > > + domain = xa_load(&group->pasid_array, pasid); > > > + if (domain && domain->type != type) > > > + domain = NULL; > > > + mutex_unlock(&group->mutex); > > > +

Re: [PATCH v6 03/12] iommu: Add attach/detach_dev_pasid domain ops

2022-05-12 Thread Jason Gunthorpe via iommu
On Thu, May 12, 2022 at 08:00:59AM +0100, Jean-Philippe Brucker wrote: > > It is not "missing" it is just renamed to > > blocking_domain->ops->set_dev_pasid() > > > > The implementation of that function would be identical to > > detach_dev_pasid. > > attach(dev, pasid, sva_domain) > detach(

Re: [PATCH v6 08/12] iommu/sva: Use attach/detach_pasid_dev in SVA interfaces

2022-05-12 Thread Jason Gunthorpe via iommu
On Thu, May 12, 2022 at 01:17:08PM +0800, Baolu Lu wrote: > On 2022/5/12 13:01, Tian, Kevin wrote: > > > From: Baolu Lu > > > Sent: Thursday, May 12, 2022 11:03 AM > > > > > > On 2022/5/11 22:53, Jason Gunthorpe wrote: > > > > > > Also, given the current arrangement it might make sense to have a

Re: [PATCH] vfio: Remove VFIO_TYPE1_NESTING_IOMMU

2022-05-12 Thread Jason Gunthorpe via iommu
On Thu, May 12, 2022 at 03:07:09PM +0800, Zhangfei Gao wrote: > > > I can't help feeling a little wary about removing this until IOMMUFD > > > can actually offer a functional replacement - is it in the way of > > > anything upcoming? > > From an upstream perspective if someone has a patched kernel

Re: [PATCH v3 1/4] iommu/vt-d: Implement domain ops for attach_dev_pasid

2022-05-11 Thread Jason Gunthorpe via iommu
On Wed, May 11, 2022 at 10:25:21AM -0700, Jacob Pan wrote: > Hi Jason, > > On Wed, 11 May 2022 14:00:25 -0300, Jason Gunthorpe wrote: > > > On Wed, May 11, 2022 at 10:02:16AM -0700, Jacob Pan wrote: > > > > > If not global, perhaps we could have a list of pasids (e.g. xarray) > > > > > attached

Re: [PATCH v3 1/4] iommu/vt-d: Implement domain ops for attach_dev_pasid

2022-05-11 Thread Jason Gunthorpe via iommu
On Wed, May 11, 2022 at 10:02:16AM -0700, Jacob Pan wrote: > > > If not global, perhaps we could have a list of pasids (e.g. xarray) > > > attached to the device_domain_info. The TLB flush logic would just go > > > through the list w/o caring what the PASIDs are for. Does it make sense > > > to you

Re: [PATCH RFC 11/12] iommufd: vfio container FD ioctl compatibility

2022-05-11 Thread Jason Gunthorpe via iommu
On Wed, May 11, 2022 at 03:15:22AM +, Tian, Kevin wrote: > > From: Jason Gunthorpe > > Sent: Wednesday, May 11, 2022 3:00 AM > > > > On Tue, May 10, 2022 at 05:12:04PM +1000, David Gibson wrote: > > > Ok... here's a revised version of my proposal which I think addresses > > > your concerns an

Re: [PATCH v3 1/4] iommu/vt-d: Implement domain ops for attach_dev_pasid

2022-05-11 Thread Jason Gunthorpe via iommu
On Wed, May 11, 2022 at 08:35:18AM -0700, Jacob Pan wrote: > > Huh? The intel driver shares the same ops between UNMANAGED and DMA - > > and in general I do not think we should be putting special knowledge > > about the DMA domains in the drivers. Drivers should continue to treat > > them identica

Re: [PATCH v6 08/12] iommu/sva: Use attach/detach_pasid_dev in SVA interfaces

2022-05-11 Thread Jason Gunthorpe via iommu
On Wed, May 11, 2022 at 03:21:31PM +0800, Baolu Lu wrote: > On 2022/5/10 23:23, Jason Gunthorpe wrote: > > On Tue, May 10, 2022 at 02:17:34PM +0800, Lu Baolu wrote: > > > > > +/** > > > + * iommu_sva_bind_device() - Bind a process address space to a device > > > + * @dev: the device > > > + * @mm:

Re: [PATCH v6 03/12] iommu: Add attach/detach_dev_pasid domain ops

2022-05-11 Thread Jason Gunthorpe via iommu
On Wed, May 11, 2022 at 08:54:39AM +0100, Jean-Philippe Brucker wrote: > > > > Then 'detach pasid' is: > > > > > > > > iommu_ops->blocking_domain->ops->attach_dev_pasid(domain, dev, > > > pasid); > > > > > > > > And we move away from the notion of 'detach' and in the direction that > > > > everythi

Re: [PATCH v6 02/12] iommu: Add pasid_bits field in struct dev_iommu

2022-05-11 Thread Jason Gunthorpe via iommu
On Wed, May 11, 2022 at 09:00:50AM +0100, Jean-Philippe Brucker wrote: > > /** > > * struct iommu_device - IOMMU core representation of one IOMMU hardware > > * instance > > * @list: Used by the iommu-core to keep a list of registered iommus > > * @ops: iommu-ops for talk

Re: [PATCH v3 1/4] iommu/vt-d: Implement domain ops for attach_dev_pasid

2022-05-11 Thread Jason Gunthorpe via iommu
On Tue, May 10, 2022 at 05:23:09PM -0700, Jacob Pan wrote: > > > diff --git a/include/linux/intel-iommu.h b/include/linux/intel-iommu.h > > > index 5af24befc9f1..55845a8c4f4d 100644 > > > +++ b/include/linux/intel-iommu.h > > > @@ -627,6 +627,7 @@ struct device_domain_info { > > > struct intel_i

Re: [PATCH v3 2/4] iommu: Add PASID support for DMA mapping API users

2022-05-10 Thread Jason Gunthorpe via iommu
On Tue, May 10, 2022 at 02:07:02PM -0700, Jacob Pan wrote: > DMA mapping API is the de facto standard for in-kernel DMA. It operates > on a per device/RID basis which is not PASID-aware. > > Some modern devices such as Intel Data Streaming Accelerator, PASID is > required for certain work submissi

Re: [PATCH v3 1/4] iommu/vt-d: Implement domain ops for attach_dev_pasid

2022-05-10 Thread Jason Gunthorpe via iommu
On Tue, May 10, 2022 at 02:07:01PM -0700, Jacob Pan wrote: > +static int intel_iommu_attach_dev_pasid(struct iommu_domain *domain, > + struct device *dev, > + ioasid_t pasid) > +{ > + struct device_domain_info *info = dev_i

Re: [PATCH RFC 11/12] iommufd: vfio container FD ioctl compatibility

2022-05-10 Thread Jason Gunthorpe via iommu
On Tue, May 10, 2022 at 05:12:04PM +1000, David Gibson wrote: > On Mon, May 09, 2022 at 11:00:41AM -0300, Jason Gunthorpe wrote: > > On Mon, May 09, 2022 at 04:01:52PM +1000, David Gibson wrote: > > > > > > The default iommu_domain that the iommu driver creates will be used > > > > here, it is up

Re: [PATCH] vfio: Remove VFIO_TYPE1_NESTING_IOMMU

2022-05-10 Thread Jason Gunthorpe via iommu
On Tue, May 10, 2022 at 06:52:06PM +0100, Robin Murphy wrote: > On 2022-05-10 17:55, Jason Gunthorpe via iommu wrote: > > This control causes the ARM SMMU drivers to choose a stage 2 > > implementation for the IO pagetable (vs the stage 1 usual default), > > however this

[PATCH] vfio: Remove VFIO_TYPE1_NESTING_IOMMU

2022-05-10 Thread Jason Gunthorpe via iommu
This control causes the ARM SMMU drivers to choose a stage 2 implementation for the IO pagetable (vs the stage 1 usual default), however this choice has no visible impact to the VFIO user. Further qemu never implemented this and no other userspace user is known. The original description in commit

Re: [PATCH v6 08/12] iommu/sva: Use attach/detach_pasid_dev in SVA interfaces

2022-05-10 Thread Jason Gunthorpe via iommu
On Tue, May 10, 2022 at 02:17:34PM +0800, Lu Baolu wrote: > +/** > + * iommu_sva_bind_device() - Bind a process address space to a device > + * @dev: the device > + * @mm: the mm to bind, caller must hold a reference to mm_users > + * @drvdata: opaque data pointer to pass to bind callback > + * >

Re: [PATCH v6 05/12] iommu/vt-d: Remove SVM_FLAG_SUPERVISOR_MODE support

2022-05-10 Thread Jason Gunthorpe via iommu
On Tue, May 10, 2022 at 02:17:31PM +0800, Lu Baolu wrote: > The current kernel DMA with PASID support is based on the SVA with a flag > SVM_FLAG_SUPERVISOR_MODE. The IOMMU driver binds the kernel memory address > space to a PASID of the device. The device driver programs the device with > kernel vi

  1   2   3   4   5   6   >