Re: [PATCH v3 19/20] PCI/P2PDMA: introduce pci_mmap_p2pmem()

2021-10-01 Thread Jason Gunthorpe
On Fri, Oct 01, 2021 at 11:01:49AM -0600, Logan Gunthorpe wrote: > In device-dax, the refcount is only used to prevent the device, and > therefore the pages, from going away on device unbind. Pages cannot be > recycled, as you say, as they are mapped linearly within the device. The > address

Re: [PATCH v3 19/20] PCI/P2PDMA: introduce pci_mmap_p2pmem()

2021-10-01 Thread Jason Gunthorpe
On Wed, Sep 29, 2021 at 09:36:52PM -0300, Jason Gunthorpe wrote: > Why would DAX want to do this in the first place?? This means the > address space zap is much more important that just speeding up > destruction, it is essential for correctness since the PTEs are not > holding refcoun

Re: [RFC 07/20] iommu/iommufd: Add iommufd_[un]bind_device()

2021-10-01 Thread Jason Gunthorpe via iommu
On Thu, Sep 30, 2021 at 01:10:29PM +1000, David Gibson wrote: > On Wed, Sep 29, 2021 at 09:24:57AM -0300, Jason Gunthorpe wrote: > 65;6402;1c> On Wed, Sep 29, 2021 at 03:25:54PM +1000, David Gibson wrote: > > > > > > +struct iommufd_device { > > > > +

Re: [RFC 0/7] Support in-kernel DMA with PASID and SVA

2021-10-01 Thread Jason Gunthorpe via iommu
On Sat, Oct 02, 2021 at 01:24:54AM +1300, Barry Song wrote: > I assume KVA mode can avoid this iotlb flush as the device is using > the page table of the kernel and sharing the whole kernel space. But > will users be glad to accept this mode? You can avoid the lock be identity mapping the

Re: [RFC 11/20] iommu/iommufd: Add IOMMU_IOASID_ALLOC/FREE

2021-10-01 Thread Jason Gunthorpe via iommu
On Fri, Oct 01, 2021 at 04:19:22PM +1000, da...@gibson.dropbear.id.au wrote: > On Wed, Sep 22, 2021 at 11:09:11AM -0300, Jason Gunthorpe wrote: > > On Wed, Sep 22, 2021 at 03:40:25AM +, Tian, Kevin wrote: > > > > From: Jason Gunthorpe > > > > Sent: Wedn

Re: [RFC 11/20] iommu/iommufd: Add IOMMU_IOASID_ALLOC/FREE

2021-10-01 Thread Jason Gunthorpe via iommu
On Fri, Oct 01, 2021 at 04:13:58PM +1000, David Gibson wrote: > On Tue, Sep 21, 2021 at 02:44:38PM -0300, Jason Gunthorpe wrote: > > On Sun, Sep 19, 2021 at 02:38:39PM +0800, Liu Yi L wrote: > > > This patch adds IOASID allocation/free interface per iommufd. When > >

Re: [RFC 06/20] iommu: Add iommu_device_init[exit]_user_dma interfaces

2021-09-30 Thread Jason Gunthorpe via iommu
On Thu, Sep 30, 2021 at 01:09:22PM +1000, David Gibson wrote: > > The *admin* the one responsible to understand the groups, not the > > applications. The admin has no idea what a group FD is - they should > > be looking at the sysfs and seeing the iommu_group directories. > > Not just the admin.

Re: [RFC 10/20] iommu/iommufd: Add IOMMU_DEVICE_GET_INFO

2021-09-30 Thread Jason Gunthorpe via iommu
On Thu, Sep 30, 2021 at 09:35:45AM +, Tian, Kevin wrote: > > The Intel functional issue is that Intel blocks the cache maintaince > > ops from the VM and the VM has no way to self-discover that the cache > > maintaince ops don't work. > > the VM doesn't need to know whether the maintenance

Re: [RFC 10/20] iommu/iommufd: Add IOMMU_DEVICE_GET_INFO

2021-09-30 Thread Jason Gunthorpe via iommu
On Thu, Sep 30, 2021 at 08:49:03AM +, Tian, Kevin wrote: > > From: Jason Gunthorpe > > Sent: Thursday, September 23, 2021 8:22 PM > > > > > > These are different things and need different bits. Since the ARM path > > > > has a lot more code supp

Re: [PATCH v3 19/20] PCI/P2PDMA: introduce pci_mmap_p2pmem()

2021-09-29 Thread Jason Gunthorpe
On Wed, Sep 29, 2021 at 05:49:36PM -0600, Logan Gunthorpe wrote: > Some of this seems out of date. Pretty sure the pages are not refcounted > with vmf_insert_mixed() and vmf_insert_mixed() is currently the only way > to use VM_MIXEDMAP mappings. Hum. vmf_insert_mixed() boils down to

Re: [RFC 0/7] Support in-kernel DMA with PASID and SVA

2021-09-29 Thread Jason Gunthorpe via iommu
On Wed, Sep 29, 2021 at 03:57:20PM -0700, Jacob Pan wrote: > Hi Jason, > > On Wed, 29 Sep 2021 16:39:53 -0300, Jason Gunthorpe wrote: > > > On Wed, Sep 29, 2021 at 12:37:19PM -0700, Jacob Pan wrote: > > > > > For #2, it seems we can store

Re: [PATCH v3 4/20] PCI/P2PDMA: introduce helpers for dma_map_sg implementations

2021-09-29 Thread Jason Gunthorpe via iommu
On Wed, Sep 29, 2021 at 05:00:43PM -0600, Logan Gunthorpe wrote: > > > > On 2021-09-29 4:46 p.m., Jason Gunthorpe wrote: > > On Wed, Sep 29, 2021 at 03:30:42PM -0600, Logan Gunthorpe wrote: > >> On 2021-09-28 4:05 p.m., Jason Gunthorpe wrote: > >> No, that

Re: [PATCH v3 00/20] Userspace P2PDMA with O_DIRECT NVMe devices

2021-09-29 Thread Jason Gunthorpe
On Wed, Sep 29, 2021 at 05:28:38PM -0600, Logan Gunthorpe wrote: > > > On 2021-09-29 5:21 p.m., Jason Gunthorpe wrote: > > On Wed, Sep 29, 2021 at 03:50:02PM -0600, Logan Gunthorpe wrote: > >> > >> > >> On 2021-09-28 2:02 p.m., Jason Gunthorpe wrote

Re: [PATCH v3 19/20] PCI/P2PDMA: introduce pci_mmap_p2pmem()

2021-09-29 Thread Jason Gunthorpe
On Wed, Sep 29, 2021 at 05:27:22PM -0600, Logan Gunthorpe wrote: > > finish_fault() should set the pte_devmap - eg by passing the > > PFN_DEV|PFN_MAP somehow through the vma->vm_page_prot to mk_pte() or > > otherwise signaling do_set_pte() that it should set those PTE bits > > when it creates the

Re: [PATCH v3 00/20] Userspace P2PDMA with O_DIRECT NVMe devices

2021-09-29 Thread Jason Gunthorpe
On Wed, Sep 29, 2021 at 03:50:02PM -0600, Logan Gunthorpe wrote: > > > On 2021-09-28 2:02 p.m., Jason Gunthorpe wrote: > > On Thu, Sep 16, 2021 at 05:40:40PM -0600, Logan Gunthorpe wrote: > >> Hi, > >> > >> This patchset continues my work to add usersp

Re: [PATCH v3 19/20] PCI/P2PDMA: introduce pci_mmap_p2pmem()

2021-09-29 Thread Jason Gunthorpe
On Wed, Sep 29, 2021 at 03:42:00PM -0600, Logan Gunthorpe wrote: > The main reason is probably this: if we don't use VM_MIXEDMAP, then we > can't set pte_devmap(). I think that is an API limitation in the fault routines.. finish_fault() should set the pte_devmap - eg by passing the

Re: [PATCH v3 14/20] mm: introduce FOLL_PCI_P2PDMA to gate getting PCI P2PDMA pages

2021-09-29 Thread Jason Gunthorpe
On Wed, Sep 29, 2021 at 03:34:22PM -0600, Logan Gunthorpe wrote: > > > > On 2021-09-28 1:47 p.m., Jason Gunthorpe wrote: > > On Thu, Sep 16, 2021 at 05:40:54PM -0600, Logan Gunthorpe wrote: > >> Callers that expect PCI P2PDMA pages can now set FOLL_PCI_P2PDMA to

Re: [PATCH v3 4/20] PCI/P2PDMA: introduce helpers for dma_map_sg implementations

2021-09-29 Thread Jason Gunthorpe via iommu
On Wed, Sep 29, 2021 at 03:30:42PM -0600, Logan Gunthorpe wrote: > > > > On 2021-09-28 4:05 p.m., Jason Gunthorpe wrote: > > On Thu, Sep 16, 2021 at 05:40:44PM -0600, Logan Gunthorpe wrote: > > > >> +enum pci_p2pdma_map_type > >> +pci_p2pdma_map_

Re: [RFC 0/7] Support in-kernel DMA with PASID and SVA

2021-09-29 Thread Jason Gunthorpe via iommu
On Wed, Sep 29, 2021 at 12:37:19PM -0700, Jacob Pan wrote: > For #2, it seems we can store the kernel PASID in struct device. This will > preserve the DMA API interface while making it PASID capable. Essentially, > each PASID capable device would have two special global PASIDs: > - PASID

Re: [RFC 06/20] iommu: Add iommu_device_init[exit]_user_dma interfaces

2021-09-29 Thread Jason Gunthorpe via iommu
On Wed, Sep 29, 2021 at 12:38:35AM +, Tian, Kevin wrote: > /* If set the driver must call iommu_XX as the first action in probe() or > * before it attempts to do DMA > */ > bool suppress_dma_owner:1; It is not "attempts to do DMA" but more "operates the physical device in any away" Not

Re: [RFC 06/20] iommu: Add iommu_device_init[exit]_user_dma interfaces

2021-09-29 Thread Jason Gunthorpe via iommu
On Wed, Sep 29, 2021 at 04:35:19PM +1000, David Gibson wrote: > Yes, exactly. And with a group interface it's obvious it has to > understand it. With the non-group interface, you can get to this > stage in ignorance of groups. It will even work as long as you are > lucky enough only to try

Re: [RFC 10/20] iommu/iommufd: Add IOMMU_DEVICE_GET_INFO

2021-09-29 Thread Jason Gunthorpe via iommu
On Wed, Sep 29, 2021 at 08:48:28AM +, Tian, Kevin wrote: > ARM: > - set to snoop format if IOMMU_CACHE > - set to nonsnoop format if !IOMMU_CACHE > (in both cases TLP snoop bit is ignored?) Where do you see this? I couldn't even find this functionality in the ARM HW manual?? What I

Re: [RFC 17/20] iommu/iommufd: Report iova range to userspace

2021-09-29 Thread Jason Gunthorpe via iommu
On Wed, Sep 29, 2021 at 01:07:56PM +0100, Jean-Philippe Brucker wrote: > Yes, so the guest knows the size of GPA it can write into the page table. > For Arm SMMU the GPA size is determined by both the SMMU implementation > and the host kernel configuration. But maybe that could also be >

Re: [RFC 08/20] vfio/pci: Add VFIO_DEVICE_BIND_IOMMUFD

2021-09-29 Thread Jason Gunthorpe via iommu
On Wed, Sep 29, 2021 at 06:41:00AM +, Tian, Kevin wrote: > > From: David Gibson > > Sent: Wednesday, September 29, 2021 2:01 PM > > > > On Sun, Sep 19, 2021 at 02:38:36PM +0800, Liu Yi L wrote: > > > This patch adds VFIO_DEVICE_BIND_IOMMUFD for userspace to bind the > > vfio > > > device to

Re: [RFC 07/20] iommu/iommufd: Add iommufd_[un]bind_device()

2021-09-29 Thread Jason Gunthorpe via iommu
On Wed, Sep 29, 2021 at 03:25:54PM +1000, David Gibson wrote: > > +struct iommufd_device { > > + unsigned int id; > > + struct iommufd_ctx *ictx; > > + struct device *dev; /* always be the physical device */ > > + u64 dev_cookie; > > Why do you need both an 'id' and a 'dev_cookie'?

Re: [RFC 03/20] vfio: Add vfio_[un]register_device()

2021-09-29 Thread Jason Gunthorpe via iommu
On Wed, Sep 29, 2021 at 12:46:14PM +1000, da...@gibson.dropbear.id.au wrote: > On Tue, Sep 21, 2021 at 10:00:14PM -0300, Jason Gunthorpe wrote: > > On Wed, Sep 22, 2021 at 12:54:02AM +, Tian, Kevin wrote: > > > > From: Jason Gunthorpe > > > > Sent: Wedne

Re: [RFC 03/20] vfio: Add vfio_[un]register_device()

2021-09-29 Thread Jason Gunthorpe via iommu
On Wed, Sep 29, 2021 at 09:08:25AM +0200, Cornelia Huck wrote: > On Wed, Sep 29 2021, "Tian, Kevin" wrote: > > >> From: David Gibson > >> Sent: Wednesday, September 29, 2021 10:44 AM > >> > >> > One alternative option is to arrange device nodes in sub-directories > >> > based > >> > on the

Re: [PATCH v3 4/20] PCI/P2PDMA: introduce helpers for dma_map_sg implementations

2021-09-28 Thread Jason Gunthorpe via iommu
On Thu, Sep 16, 2021 at 05:40:44PM -0600, Logan Gunthorpe wrote: > +enum pci_p2pdma_map_type > +pci_p2pdma_map_segment(struct pci_p2pdma_map_state *state, struct device > *dev, > +struct scatterlist *sg) > +{ > + if (state->pgmap != sg_page(sg)->pgmap) { > +

Re: [PATCH v3 19/20] PCI/P2PDMA: introduce pci_mmap_p2pmem()

2021-09-28 Thread Jason Gunthorpe
On Thu, Sep 16, 2021 at 05:40:59PM -0600, Logan Gunthorpe wrote: > +static void pci_p2pdma_unmap_mappings(void *data) > +{ > + struct pci_dev *pdev = data; > + struct pci_p2pdma *p2pdma = rcu_dereference_protected(pdev->p2pdma, 1); > + > + p2pdma->active = false; > +

Re: [PATCH v3 00/20] Userspace P2PDMA with O_DIRECT NVMe devices

2021-09-28 Thread Jason Gunthorpe
On Thu, Sep 16, 2021 at 05:40:40PM -0600, Logan Gunthorpe wrote: > Hi, > > This patchset continues my work to add userspace P2PDMA access using > O_DIRECT NVMe devices. My last posting[1] just included the first 13 > patches in this series, but the early P2PDMA cleanup and map_sg error > changes

Re: [PATCH v3 19/20] PCI/P2PDMA: introduce pci_mmap_p2pmem()

2021-09-28 Thread Jason Gunthorpe
On Thu, Sep 16, 2021 at 05:40:59PM -0600, Logan Gunthorpe wrote: > +int pci_mmap_p2pmem(struct pci_dev *pdev, struct vm_area_struct *vma) > +{ > + struct pci_p2pdma_map *pmap; > + struct pci_p2pdma *p2pdma; > + int ret; > + > + /* prevent private mappings from being established */

Re: [PATCH v3 14/20] mm: introduce FOLL_PCI_P2PDMA to gate getting PCI P2PDMA pages

2021-09-28 Thread Jason Gunthorpe
On Thu, Sep 16, 2021 at 05:40:54PM -0600, Logan Gunthorpe wrote: > Callers that expect PCI P2PDMA pages can now set FOLL_PCI_P2PDMA to > allow obtaining P2PDMA pages. If a caller does not set this flag > and tries to map P2PDMA pages it will fail. > > This is implemented by adding a flag and a

Re: [PATCH v3 13/20] PCI/P2PDMA: remove pci_p2pdma_[un]map_sg()

2021-09-28 Thread Jason Gunthorpe
2pdma.c | 65 -- > include/linux/pci-p2pdma.h | 27 > 2 files changed, 92 deletions(-) Reviewed-by: Jason Gunthorpe Good riddance :) Jason ___ iommu mailing list iommu@lists.li

Re: [PATCH v3 12/20] RDMA/rw: use dma_map_sgtable()

2021-09-28 Thread Jason Gunthorpe
atch to do it. But as it is, this looks fine Reviewed-by: Jason Gunthorpe Jason ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu

Re: [PATCH v3 11/20] RDMA/core: introduce ib_dma_pci_p2p_dma_supported()

2021-09-28 Thread Jason Gunthorpe
arget/rdma.c | 2 +- > include/rdma/ib_verbs.h| 11 +++ > 2 files changed, 12 insertions(+), 1 deletion(-) Reviewed-by: Jason Gunthorpe > +/* > + * Check if a IB device's underlying DMA mapping supports P2PDMA transfers. > + */ > +static inline b

Re: [PATCH v3 08/20] iommu/dma: support PCI P2PDMA pages in dma-iommu map_sg

2021-09-28 Thread Jason Gunthorpe
t; > Signed-off-by: Logan Gunthorpe > --- > drivers/iommu/dma-iommu.c | 68 +++---- > 1 file changed, 61 insertions(+), 7 deletions(-) Reviewed-by: Jason Gunthorpe Jason ___ iommu mailing list iommu@lists

Re: [PATCH v3 07/20] dma-mapping: add flags to dma_map_ops to indicate PCI P2PDMA support

2021-09-28 Thread Jason Gunthorpe
> include/linux/dma-map-ops.h | 10 ++ > include/linux/dma-mapping.h | 5 + > kernel/dma/mapping.c| 18 ++ > 3 files changed, 33 insertions(+) Reviewed-by: Jason Gunthorpe Jason ___ iommu mailing list

Re: [PATCH v3 06/20] dma-direct: support PCI P2PDMA pages in dma-direct map_sg

2021-09-28 Thread Jason Gunthorpe
On Thu, Sep 16, 2021 at 05:40:46PM -0600, Logan Gunthorpe wrote: > Add PCI P2PDMA support for dma_direct_map_sg() so that it can map > PCI P2PDMA pages directly without a hack in the callers. This allows > for heterogeneous SGLs that contain both P2PDMA and regular pages. > > A P2PDMA page may

Re: [PATCH v3 05/20] dma-mapping: allow EREMOTEIO return code for P2PDMA transfers

2021-09-28 Thread Jason Gunthorpe
--- > kernel/dma/mapping.c | 16 +--- > 1 file changed, 9 insertions(+), 7 deletions(-) Reviewed-by: Jason Gunthorpe Jason ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu

Re: [PATCH v3 04/20] PCI/P2PDMA: introduce helpers for dma_map_sg implementations

2021-09-28 Thread Jason Gunthorpe
On Thu, Sep 16, 2021 at 05:40:44PM -0600, Logan Gunthorpe wrote: > Add pci_p2pdma_map_segment() as a helper for simple dma_map_sg() > implementations. It takes an scatterlist segment that must point to a > pci_p2pdma struct page and will map it if the mapping requires a bus > address. > > The

Re: [PATCH v3 03/20] PCI/P2PDMA: make pci_p2pdma_map_type() non-static

2021-09-28 Thread Jason Gunthorpe
uld be used to program the DMA engine. > + */ > + PCI_P2PDMA_MAP_THRU_HOST_BRIDGE, > +}; Reviewed-by: Jason Gunthorpe Jason ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu

Re: [PATCH v3 01/20] lib/scatterlist: add flag for indicating P2PDMA segments in an SGL

2021-09-28 Thread Jason Gunthorpe
ssign a given page to an SG entry > @@ -86,13 +103,13 @@ struct sg_append_table { > **/ > static inline void sg_assign_page(struct scatterlist *sg, struct page *page) > { > - unsigned long page_link = sg->page_link & (SG_CHAIN | SG_END); > +

Re: [RFC 06/20] iommu: Add iommu_device_init[exit]_user_dma interfaces

2021-09-28 Thread Jason Gunthorpe via iommu
On Tue, Sep 28, 2021 at 09:35:05PM +0800, Lu Baolu wrote: > Another issue is, when putting a device into user-dma mode, all devices > belonging to the same iommu group shouldn't be bound with a kernel-dma > driver. Kevin's prototype checks this by READ_ONCE(dev->driver). This is > not lock safe as

Re: [RFC 06/20] iommu: Add iommu_device_init[exit]_user_dma interfaces

2021-09-28 Thread Jason Gunthorpe via iommu
On Tue, Sep 28, 2021 at 07:30:41AM +, Tian, Kevin wrote: > > Also, don't call it "hint", there is nothing hinty about this, it has > > definitive functional impacts. > > possibly dma_mode (too broad?) or dma_usage You just need a flag to specify if the driver manages DMA ownership itself,

Re: [RFC 06/20] iommu: Add iommu_device_init[exit]_user_dma interfaces

2021-09-27 Thread Jason Gunthorpe via iommu
On Mon, Sep 27, 2021 at 09:42:58AM +, Tian, Kevin wrote: > +static int iommu_dev_viable(struct device *dev, void *data) > +{ > + enum dma_hint hint = *data; > + struct device_driver *drv = READ_ONCE(dev->driver); > + > + /* no conflict if the new device doesn't do DMA */ > +

Re: [RFC 06/20] iommu: Add iommu_device_init[exit]_user_dma interfaces

2021-09-27 Thread Jason Gunthorpe via iommu
On Mon, Sep 27, 2021 at 01:00:08PM +, Tian, Kevin wrote: > > I think for such a narrow usage you should not change the struct > > device_driver. Just have pci_stub call a function to flip back to user > > mode. > > Here we want to ensure that kernel dma should be blocked > if the group is

Re: [RFC 06/20] iommu: Add iommu_device_init[exit]_user_dma interfaces

2021-09-27 Thread Jason Gunthorpe via iommu
On Mon, Sep 27, 2021 at 09:42:58AM +, Tian, Kevin wrote: > > From: Jason Gunthorpe > > Sent: Wednesday, September 22, 2021 8:40 PM > > > > > > Ie the basic flow would see the driver core doing some: > > > > > > Just double confirm. Is there

Re: [RFC 11/20] iommu/iommufd: Add IOMMU_IOASID_ALLOC/FREE

2021-09-23 Thread Jason Gunthorpe via iommu
On Thu, Sep 23, 2021 at 01:20:55PM +, Tian, Kevin wrote: > > > this is not a flow for mdev. It's also required for pdev on Intel > > > platform, > > > because the pasid table is in HPA space thus must be managed by host > > > kernel. Even no translation we still need the user to provide the

Re: [RFC 11/20] iommu/iommufd: Add IOMMU_IOASID_ALLOC/FREE

2021-09-23 Thread Jason Gunthorpe via iommu
On Thu, Sep 23, 2021 at 12:45:17PM +, Tian, Kevin wrote: > > From: Jason Gunthorpe > > Sent: Thursday, September 23, 2021 8:31 PM > > > > On Thu, Sep 23, 2021 at 12:22:23PM +, Tian, Kevin wrote: > > > > From: Jason Gunthorpe > > >

Re: [RFC 11/20] iommu/iommufd: Add IOMMU_IOASID_ALLOC/FREE

2021-09-23 Thread Jason Gunthorpe via iommu
On Thu, Sep 23, 2021 at 12:22:23PM +, Tian, Kevin wrote: > > From: Jason Gunthorpe > > Sent: Thursday, September 23, 2021 8:07 PM > > > > On Thu, Sep 23, 2021 at 09:14:58AM +, Tian, Kevin wrote: > > > > > currently the type is aimed to different

Re: [RFC 10/20] iommu/iommufd: Add IOMMU_DEVICE_GET_INFO

2021-09-23 Thread Jason Gunthorpe via iommu
On Thu, Sep 23, 2021 at 12:05:29PM +, Tian, Kevin wrote: > > From: Jason Gunthorpe > > Sent: Thursday, September 23, 2021 7:27 PM > > > > On Thu, Sep 23, 2021 at 11:15:24AM +0100, Jean-Philippe Brucker wrote: > > > > > So we can only tell userspace &

Re: [RFC 11/20] iommu/iommufd: Add IOMMU_IOASID_ALLOC/FREE

2021-09-23 Thread Jason Gunthorpe via iommu
On Thu, Sep 23, 2021 at 09:14:58AM +, Tian, Kevin wrote: > currently the type is aimed to differentiate three usages: > > - kernel-managed I/O page table > - user-managed I/O page table > - shared I/O page table (e.g. with mm, or ept) Creating a shared ios is something that should probably

Re: [RFC 03/20] vfio: Add vfio_[un]register_device()

2021-09-23 Thread Jason Gunthorpe via iommu
On Thu, Sep 23, 2021 at 09:25:27AM +0200, Eric Auger wrote: > Hi, > > On 9/22/21 3:00 AM, Jason Gunthorpe wrote: > > On Wed, Sep 22, 2021 at 12:54:02AM +, Tian, Kevin wrote: > >>> From: Jason Gunthorpe > >>> Sent: Wednesday, September 22, 2021 12

Re: [RFC 10/20] iommu/iommufd: Add IOMMU_DEVICE_GET_INFO

2021-09-23 Thread Jason Gunthorpe via iommu
On Thu, Sep 23, 2021 at 03:38:10AM +, Tian, Kevin wrote: > > From: Tian, Kevin > > Sent: Thursday, September 23, 2021 11:11 AM > > > > > > > > The required behavior for iommufd is to have the IOMMU ignore the > > > no-snoop bit so that Intel HW can disable wbinvd. This bit should be > > >

Re: [RFC 10/20] iommu/iommufd: Add IOMMU_DEVICE_GET_INFO

2021-09-23 Thread Jason Gunthorpe via iommu
On Thu, Sep 23, 2021 at 03:10:47AM +, Tian, Kevin wrote: > Disabling wbinvd is one purpose. imo the more important intention > is that iommu vendor uses different PTE formats between snoop and > !snoop. The PTE format for userspace is communicated through the format input, not through

Re: [RFC 10/20] iommu/iommufd: Add IOMMU_DEVICE_GET_INFO

2021-09-23 Thread Jason Gunthorpe via iommu
On Thu, Sep 23, 2021 at 11:15:24AM +0100, Jean-Philippe Brucker wrote: > So we can only tell userspace "No_snoop is not supported" (provided we > even want to allow them to enable No_snoop). Users in control of stage-1 > tables can create non-cacheable mappings through MAIR attributes. My point

Re: [RFC 03/20] vfio: Add vfio_[un]register_device()

2021-09-22 Thread Jason Gunthorpe via iommu
On Wed, Sep 22, 2021 at 02:10:36PM -0600, Alex Williamson wrote: > But why would we create vfio device interface files at all if they > can't work? I'm not really on board with creating a try-and-fail > interface for a mechanism that cannot work for a given device. The > existence of the device

Re: [RFC 03/20] vfio: Add vfio_[un]register_device()

2021-09-22 Thread Jason Gunthorpe via iommu
On Wed, Sep 22, 2021 at 11:45:33PM +, Tian, Kevin wrote: > > From: Alex Williamson > btw I realized another related piece regarding to the new layout that > Jason suggested, which have sys device node include a link to the vfio > devnode: > >

Re: [RFC 10/20] iommu/iommufd: Add IOMMU_DEVICE_GET_INFO

2021-09-22 Thread Jason Gunthorpe via iommu
On Wed, Sep 22, 2021 at 03:24:07PM -0600, Alex Williamson wrote: > On Sun, 19 Sep 2021 14:38:38 +0800 > Liu Yi L wrote: > > > +struct iommu_device_info { > > + __u32 argsz; > > + __u32 flags; > > +#define IOMMU_DEVICE_INFO_ENFORCE_SNOOP(1 << 0) /* IOMMU enforced > > snoop */ > > Is

Re: [RFC 08/20] vfio/pci: Add VFIO_DEVICE_BIND_IOMMUFD

2021-09-22 Thread Jason Gunthorpe via iommu
On Wed, Sep 22, 2021 at 03:01:01PM -0600, Alex Williamson wrote: > On Tue, 21 Sep 2021 14:29:39 -0300 > Jason Gunthorpe wrote: > > > On Sun, Sep 19, 2021 at 02:38:36PM +0800, Liu Yi L wrote: > > > +struct vfio_device_iommu_bind_data { > > > + __u32 argsz; >

Re: [RFC 0/7] Support in-kernel DMA with PASID and SVA

2021-09-22 Thread Jason Gunthorpe via iommu
On Tue, Sep 21, 2021 at 01:29:34PM -0700, Jacob Pan wrote: > Hi Joerg/Jason/Christoph et all, > > The current in-kernel supervisor PASID support is based on the SVM/SVA > machinery in sva-lib. Kernel SVA is achieved by extending a special flag > to indicate the binding of the device and a page

Re: [RFC 01/20] iommu/iommufd: Add /dev/iommu core

2021-09-22 Thread Jason Gunthorpe via iommu
On Wed, Sep 22, 2021 at 01:59:39PM +, Tian, Kevin wrote: > > From: Jason Gunthorpe > > Sent: Wednesday, September 22, 2021 8:41 PM > > > > On Wed, Sep 22, 2021 at 01:51:03AM +, Tian, Kevin wrote: > > > > From: Jason Gunthorpe > > >

Re: [RFC 11/20] iommu/iommufd: Add IOMMU_IOASID_ALLOC/FREE

2021-09-22 Thread Jason Gunthorpe via iommu
On Wed, Sep 22, 2021 at 03:40:25AM +, Tian, Kevin wrote: > > From: Jason Gunthorpe > > Sent: Wednesday, September 22, 2021 1:45 AM > > > > On Sun, Sep 19, 2021 at 02:38:39PM +0800, Liu Yi L wrote: > > > This patch adds IOASID allocation/free interface p

Re: [RFC 11/20] iommu/iommufd: Add IOMMU_IOASID_ALLOC/FREE

2021-09-22 Thread Jason Gunthorpe via iommu
On Wed, Sep 22, 2021 at 12:51:38PM +, Liu, Yi L wrote: > > From: Jason Gunthorpe > > Sent: Wednesday, September 22, 2021 1:45 AM > > > [...] > > > diff --git a/drivers/iommu/iommufd/iommufd.c > > b/drivers/iommu/iommufd/iommufd.c > > > index

Re: [RFC 15/20] vfio/pci: Add VFIO_DEVICE_[DE]ATTACH_IOASID

2021-09-22 Thread Jason Gunthorpe via iommu
On Wed, Sep 22, 2021 at 03:56:18AM +, Tian, Kevin wrote: > > From: Jason Gunthorpe > > Sent: Wednesday, September 22, 2021 2:04 AM > > > > On Sun, Sep 19, 2021 at 02:38:43PM +0800, Liu Yi L wrote: > > > This patch adds interface for userspace to attach

Re: [RFC 14/20] iommu/iommufd: Add iommufd_device_[de]attach_ioasid()

2021-09-22 Thread Jason Gunthorpe via iommu
On Wed, Sep 22, 2021 at 03:53:52AM +, Tian, Kevin wrote: > Actually this was one open we closed in previous design proposal, but > looks you have a different thought now. > > vfio maintains one ioas per container. Devices in the container > can be attached to different domains (e.g. due to

Re: [RFC 12/20] iommu/iommufd: Add IOMMU_CHECK_EXTENSION

2021-09-22 Thread Jason Gunthorpe via iommu
On Wed, Sep 22, 2021 at 03:41:50AM +, Tian, Kevin wrote: > > From: Jason Gunthorpe > > Sent: Wednesday, September 22, 2021 1:47 AM > > > > On Sun, Sep 19, 2021 at 02:38:40PM +0800, Liu Yi L wrote: > > > As aforementioned, userspace should check ext

Re: [RFC 02/20] vfio: Add device class for /dev/vfio/devices

2021-09-22 Thread Jason Gunthorpe via iommu
On Wed, Sep 22, 2021 at 03:22:42AM +, Tian, Kevin wrote: > > From: Tian, Kevin > > Sent: Wednesday, September 22, 2021 9:07 AM > > > > > From: Jason Gunthorpe > > > Sent: Wednesday, September 22, 2021 8:55 AM > > > > > > On Tu

Re: [RFC 10/20] iommu/iommufd: Add IOMMU_DEVICE_GET_INFO

2021-09-22 Thread Jason Gunthorpe via iommu
On Wed, Sep 22, 2021 at 03:30:09AM +, Tian, Kevin wrote: > > From: Jason Gunthorpe > > Sent: Wednesday, September 22, 2021 1:41 AM > > > > On Sun, Sep 19, 2021 at 02:38:38PM +0800, Liu Yi L wrote: > > > After a device is bound to the iommufd, userspace can

Re: [RFC 01/20] iommu/iommufd: Add /dev/iommu core

2021-09-22 Thread Jason Gunthorpe via iommu
On Wed, Sep 22, 2021 at 01:51:03AM +, Tian, Kevin wrote: > > From: Jason Gunthorpe > > Sent: Tuesday, September 21, 2021 11:42 PM > > > > - Delete the iommufd_ctx->lock. Use RCU to protect load, erase/alloc does > >not need locking (order it prope

Re: [RFC 06/20] iommu: Add iommu_device_init[exit]_user_dma interfaces

2021-09-22 Thread Jason Gunthorpe via iommu
On Wed, Sep 22, 2021 at 01:47:05AM +, Tian, Kevin wrote: > > IIRC in VFIO the container is the IOAS and when the group goes to > > create the device fd it should simply do the > > iommu_device_init_user_dma() followed immediately by a call to bind > > the container IOAS as your #3. > > a

Re: [RFC 02/20] vfio: Add device class for /dev/vfio/devices

2021-09-22 Thread Jason Gunthorpe via iommu
On Wed, Sep 22, 2021 at 01:07:11AM +, Tian, Kevin wrote: > > From: Jason Gunthorpe > > Sent: Wednesday, September 22, 2021 8:55 AM > > > > On Tue, Sep 21, 2021 at 11:56:06PM +, Tian, Kevin wrote: > > > > The opened atomic is aweful. A newly create

Re: [RFC 03/20] vfio: Add vfio_[un]register_device()

2021-09-22 Thread Jason Gunthorpe via iommu
On Wed, Sep 22, 2021 at 09:23:34AM +, Tian, Kevin wrote: > > Providing an ioctl to bind to a normal VFIO container or group might > > allow a reasonable fallback in userspace.. > > I didn't get this point though. An error in binding already allows the > user to fall back to the group path.

Re: [RFC 03/20] vfio: Add vfio_[un]register_device()

2021-09-21 Thread Jason Gunthorpe via iommu
On Wed, Sep 22, 2021 at 12:54:02AM +, Tian, Kevin wrote: > > From: Jason Gunthorpe > > Sent: Wednesday, September 22, 2021 12:01 AM > > > > > One open about how to organize the device nodes under > > /dev/vfio/devices/. > > > This RFC adopt

Re: [RFC 02/20] vfio: Add device class for /dev/vfio/devices

2021-09-21 Thread Jason Gunthorpe via iommu
On Tue, Sep 21, 2021 at 11:56:06PM +, Tian, Kevin wrote: > > The opened atomic is aweful. A newly created fd should start in a > > state where it has a disabled fops > > > > The only thing the disabled fops can do is register the device to the > > iommu fd. When successfully registered the

Re: [RFC 03/20] vfio: Add vfio_[un]register_device()

2021-09-21 Thread Jason Gunthorpe via iommu
On Tue, Sep 21, 2021 at 11:10:15PM +, Tian, Kevin wrote: > > From: Jason Gunthorpe > > Sent: Wednesday, September 22, 2021 12:01 AM > > > > On Sun, Sep 19, 2021 at 02:38:31PM +0800, Liu Yi L wrote: > > > With /dev/vfio/devices introduced, now a vfio devi

Re: [RFC 05/20] vfio/pci: Register device to /dev/vfio/devices

2021-09-21 Thread Jason Gunthorpe via iommu
On Tue, Sep 21, 2021 at 03:09:29PM -0600, Alex Williamson wrote: > the iommufd uAPI at all. Isn't part of that work to understand how KVM > will be told about non-coherent devices rather than "meh, skip it in the > kernel"? Also let's not forget that vfio is not only for KVM. vfio is not only

Re: [RFC 16/20] vfio/type1: Export symbols for dma [un]map code sharing

2021-09-21 Thread Jason Gunthorpe via iommu
On Sun, Sep 19, 2021 at 02:38:44PM +0800, Liu Yi L wrote: > [HACK. will fix in v2] > > There are two options to impelement vfio type1v2 mapping semantics in > /dev/iommu. > > One is to duplicate the related code from vfio as the starting point, > and then merge with vfio type1 at a later time.

Re: [RFC 15/20] vfio/pci: Add VFIO_DEVICE_[DE]ATTACH_IOASID

2021-09-21 Thread Jason Gunthorpe via iommu
On Sun, Sep 19, 2021 at 02:38:43PM +0800, Liu Yi L wrote: > This patch adds interface for userspace to attach device to specified > IOASID. > > Note: > One device can only be attached to one IOASID in this version. This is > on par with what vfio provides today. In the future this restriction can

Re: [RFC 14/20] iommu/iommufd: Add iommufd_device_[de]attach_ioasid()

2021-09-21 Thread Jason Gunthorpe via iommu
On Sun, Sep 19, 2021 at 02:38:42PM +0800, Liu Yi L wrote: > An I/O address space takes effect in the iommu only after it's attached > by a device. This patch provides iommufd_device_[de/at]tach_ioasid() > helpers for this purpose. One device can be only attached to one ioasid > at this point, but

Re: [RFC 12/20] iommu/iommufd: Add IOMMU_CHECK_EXTENSION

2021-09-21 Thread Jason Gunthorpe via iommu
On Sun, Sep 19, 2021 at 02:38:40PM +0800, Liu Yi L wrote: > As aforementioned, userspace should check extension for what formats > can be specified when allocating an IOASID. This patch adds such > interface for userspace. In this RFC, iommufd reports EXT_MAP_TYPE1V2 > support and no no-snoop

Re: [RFC 11/20] iommu/iommufd: Add IOMMU_IOASID_ALLOC/FREE

2021-09-21 Thread Jason Gunthorpe via iommu
On Sun, Sep 19, 2021 at 02:38:39PM +0800, Liu Yi L wrote: > This patch adds IOASID allocation/free interface per iommufd. When > allocating an IOASID, userspace is expected to specify the type and > format information for the target I/O page table. > > This RFC supports only one type

Re: [RFC 10/20] iommu/iommufd: Add IOMMU_DEVICE_GET_INFO

2021-09-21 Thread Jason Gunthorpe via iommu
On Sun, Sep 19, 2021 at 02:38:38PM +0800, Liu Yi L wrote: > After a device is bound to the iommufd, userspace can use this interface > to query the underlying iommu capability and format info for this device. > Based on this information the user then creates I/O address space in a > compatible

Re: [RFC 08/20] vfio/pci: Add VFIO_DEVICE_BIND_IOMMUFD

2021-09-21 Thread Jason Gunthorpe via iommu
On Sun, Sep 19, 2021 at 02:38:36PM +0800, Liu Yi L wrote: > This patch adds VFIO_DEVICE_BIND_IOMMUFD for userspace to bind the vfio > device to an iommufd. No VFIO_DEVICE_UNBIND_IOMMUFD interface is provided > because it's implicitly done when the device fd is closed. > > In concept a vfio device

Re: [RFC 07/20] iommu/iommufd: Add iommufd_[un]bind_device()

2021-09-21 Thread Jason Gunthorpe via iommu
On Sun, Sep 19, 2021 at 02:38:35PM +0800, Liu Yi L wrote: > +/* > + * A iommufd_device object represents the binding relationship > + * between iommufd and device. It is created per a successful > + * binding request from device driver. The bound device must be > + * a physical device so far.

Re: [RFC 06/20] iommu: Add iommu_device_init[exit]_user_dma interfaces

2021-09-21 Thread Jason Gunthorpe via iommu
On Sun, Sep 19, 2021 at 02:38:34PM +0800, Liu Yi L wrote: > From: Lu Baolu > > This extends iommu core to manage security context for passthrough > devices. Please bear a long explanation for how we reach this design > instead of managing it solely in iommufd like what vfio does today. > >

Re: [RFC 05/20] vfio/pci: Register device to /dev/vfio/devices

2021-09-21 Thread Jason Gunthorpe via iommu
On Sun, Sep 19, 2021 at 02:38:33PM +0800, Liu Yi L wrote: > This patch exposes the device-centric interface for vfio-pci devices. To > be compatiable with existing users, vfio-pci exposes both legacy group > interface and device-centric interface. > > As explained in last patch, this change

Re: [RFC 04/20] iommu: Add iommu_device_get_info interface

2021-09-21 Thread Jason Gunthorpe via iommu
On Sun, Sep 19, 2021 at 02:38:32PM +0800, Liu Yi L wrote: > From: Lu Baolu > > This provides an interface for upper layers to get the per-device iommu > attributes. > > int iommu_device_get_info(struct device *dev, > enum iommu_devattr attr, void *data); Can't

Re: [RFC 03/20] vfio: Add vfio_[un]register_device()

2021-09-21 Thread Jason Gunthorpe via iommu
On Sun, Sep 19, 2021 at 02:38:31PM +0800, Liu Yi L wrote: > With /dev/vfio/devices introduced, now a vfio device driver has three > options to expose its device to userspace: > > a) only legacy group interface, for devices which haven't been moved to > iommufd (e.g. platform devices, sw

Re: [RFC 02/20] vfio: Add device class for /dev/vfio/devices

2021-09-21 Thread Jason Gunthorpe via iommu
On Sun, Sep 19, 2021 at 02:38:30PM +0800, Liu Yi L wrote: > This patch introduces a new interface (/dev/vfio/devices/$DEVICE) for > userspace to directly open a vfio device w/o relying on container/group > (/dev/vfio/$GROUP). Anything related to group is now hidden behind > iommufd (more

Re: [RFC 01/20] iommu/iommufd: Add /dev/iommu core

2021-09-21 Thread Jason Gunthorpe via iommu
On Sun, Sep 19, 2021 at 02:38:29PM +0800, Liu Yi L wrote: > /dev/iommu aims to provide a unified interface for managing I/O address > spaces for devices assigned to userspace. This patch adds the initial > framework to create a /dev/iommu node. Each open of this node returns an > iommufd. And this

Re: [RFC 00/20] Introduce /dev/iommu for userspace I/O address space management

2021-09-21 Thread Jason Gunthorpe via iommu
On Sun, Sep 19, 2021 at 02:38:28PM +0800, Liu Yi L wrote: > Linux now includes multiple device-passthrough frameworks (e.g. VFIO and > vDPA) to manage secure device access from the userspace. One critical task > of those frameworks is to put the assigned device in a secure, IOMMU- > protected

Re: [RFC][PATCH v2 00/13] iommu/arm-smmu-v3: Add NVIDIA implementation

2021-09-02 Thread Jason Gunthorpe via iommu
On Wed, Sep 01, 2021 at 06:55:55AM +, Tian, Kevin wrote: > > From: Alex Williamson > > Sent: Wednesday, September 1, 2021 12:16 AM > > > > On Mon, 30 Aug 2021 19:59:10 -0700 > > Nicolin Chen wrote: > > > > > The SMMUv3 devices implemented in the Grace SoC support NVIDIA's > > custom > > >

Re: [PATCH v4 14/14] tpm: Allow locality 2 to be set when initializing the TPM for Secure Launch

2021-08-27 Thread Jason Gunthorpe
On Fri, Aug 27, 2021 at 09:28:37AM -0400, Ross Philipson wrote: > The Secure Launch MLE environment uses PCRs that are only accessible from > the DRTM locality 2. By default the TPM drivers always initialize the > locality to 0. When a Secure Launch is in progress, initialize the > locality to 2.

Re: [RFC v2] /dev/iommu uAPI proposal

2021-08-06 Thread Jason Gunthorpe via iommu
On Fri, Aug 06, 2021 at 02:45:26PM +1000, David Gibson wrote: > Well, that's kind of what I'm doing. PCI currently has the notion of > "default" address space for a RID, but there's no guarantee that other > buses (or even future PCI extensions) will. The idea is that > "endpoint" means exactly

Re: [RFC v2] /dev/iommu uAPI proposal

2021-08-05 Thread Jason Gunthorpe via iommu
On Wed, Aug 04, 2021 at 10:59:21PM +, Tian, Kevin wrote: > > From: Jason Gunthorpe > > Sent: Wednesday, August 4, 2021 10:05 PM > > > > On Mon, Aug 02, 2021 at 02:49:44AM +, Tian, Kevin wrote: > > > > > Can you elaborate? IMO the user only cares ab

Re: [RFC v2] /dev/iommu uAPI proposal

2021-08-04 Thread Jason Gunthorpe via iommu
On Tue, Aug 03, 2021 at 11:58:54AM +1000, David Gibson wrote: > > I'd rather deduce the endpoint from a collection of devices than the > > other way around... > > Which I think is confusing, and in any case doesn't cover the case of > one "device" with multiple endpoints. Well they are both

Re: [RFC v2] /dev/iommu uAPI proposal

2021-08-04 Thread Jason Gunthorpe via iommu
On Mon, Aug 02, 2021 at 02:49:44AM +, Tian, Kevin wrote: > Can you elaborate? IMO the user only cares about the label (device cookie > plus optional vPASID) which is generated by itself when doing the attaching > call, and expects this virtual label being used in various spots >

Re: [RFC v2] /dev/iommu uAPI proposal

2021-07-30 Thread Jason Gunthorpe via iommu
On Mon, Jul 26, 2021 at 02:50:48PM +1000, David Gibson wrote: > That said, I'm still finding the various ways a device can attach to > an ioasid pretty confusing. Here are some thoughts on some extra > concepts that might make it easier to handle [note, I haven't thought > this all the way

Re: [RFC v2] /dev/iommu uAPI proposal

2021-07-22 Thread Jason Gunthorpe via iommu
On Wed, Jul 21, 2021 at 02:13:23AM +, Tian, Kevin wrote: > > From: Shenming Lu > > Sent: Friday, July 16, 2021 8:20 PM > > > > On 2021/7/16 9:20, Tian, Kevin wrote: > > > To summarize, for vIOMMU we can work with the spec owner to > > > define a proper interface to feedback such restriction

<    1   2   3   4   5   6   7   8   9   10   >