Re: [PATCH for-next 2/4] RDMA/hns: Add IOMMU enable support in hip08

2017-11-07 Thread Jason Gunthorpe
On Tue, Nov 07, 2017 at 10:45:29AM +0800, Wei Hu (Xavier) wrote: > We reconstruct the code as below: > It replaces dma_alloc_coherent with __get_free_pages and > dma_map_single functions. So, we can vmap serveral ptrs returned by > __get_free_pages, right? Can't you just use

Re: [PATCH for-next 2/4] RDMA/hns: Add IOMMU enable support in hip08

2017-11-07 Thread Jason Gunthorpe
On Tue, Nov 07, 2017 at 07:58:05AM -0800, Christoph Hellwig wrote: > On Tue, Nov 07, 2017 at 08:48:38AM -0700, Jason Gunthorpe wrote: > > Can't you just use vmalloc and dma_map that? Other drivers follow that > > approach.. > > You can't easily due to the flushing require

Re: [RFC PATCH 3/5] mm/vma: add support for peer to peer to device vma

2019-01-29 Thread Jason Gunthorpe
On Tue, Jan 29, 2019 at 02:11:23PM -0500, Jerome Glisse wrote: > On Tue, Jan 29, 2019 at 11:36:29AM -0700, Logan Gunthorpe wrote: > > > > > > On 2019-01-29 10:47 a.m., jgli...@redhat.com wrote: > > > > > + /* > > > + * Optional for device driver that want to allow peer to peer (p2p) > > > + *

Re: [RFC PATCH 3/5] mm/vma: add support for peer to peer to device vma

2019-01-29 Thread Jason Gunthorpe
On Tue, Jan 29, 2019 at 03:44:00PM -0500, Jerome Glisse wrote: > > But this API doesn't seem to offer any control - I thought that > > control was all coming from the mm/hmm notifiers triggering p2p_unmaps? > > The control is within the driver implementation of those callbacks. Seems like what

Re: [RFC PATCH 3/5] mm/vma: add support for peer to peer to device vma

2019-01-29 Thread Jason Gunthorpe
On Tue, Jan 29, 2019 at 02:50:55PM -0500, Jerome Glisse wrote: > GPU driver do want more control :) GPU driver are moving things around > all the time and they have more memory than bar space (on newer platform > AMD GPU do resize the bar but it is not the rule for all GPUs). So > GPU driver do

Re: [RFC PATCH 3/5] mm/vma: add support for peer to peer to device vma

2019-01-29 Thread Jason Gunthorpe
On Tue, Jan 29, 2019 at 01:39:49PM -0700, Logan Gunthorpe wrote: > implement the mapping. And I don't think we should have 'special' vma's > for this (though we may need something to ensure we don't get mapping > requests mixed with different types of pages...). I think Jerome explained the

Re: [RFC PATCH 3/5] mm/vma: add support for peer to peer to device vma

2019-01-29 Thread Jason Gunthorpe
On Tue, Jan 29, 2019 at 07:08:06PM -0500, Jerome Glisse wrote: > On Tue, Jan 29, 2019 at 11:02:25PM +0000, Jason Gunthorpe wrote: > > On Tue, Jan 29, 2019 at 03:44:00PM -0500, Jerome Glisse wrote: > > > > > > But this API doesn't seem to offer any control - I thought t

Re: [RFC PATCH 3/5] mm/vma: add support for peer to peer to device vma

2019-01-29 Thread Jason Gunthorpe
On Tue, Jan 29, 2019 at 06:17:43PM -0700, Logan Gunthorpe wrote: > This isn't answering my question at all... I specifically asked what is > backing the VMA when we are *not* using HMM. At least for RDMA what backs the VMA today is non-struct-page BAR memory filled in with io_remap_pfn. And we

Re: [RFC PATCH 3/5] mm/vma: add support for peer to peer to device vma

2019-01-30 Thread Jason Gunthorpe
On Wed, Jan 30, 2019 at 06:26:53PM +0100, Christoph Hellwig wrote: > On Wed, Jan 30, 2019 at 10:55:43AM -0500, Jerome Glisse wrote: > > Even outside GPU driver, device driver like RDMA just want to share their > > doorbell to other device and they do not want to see those doorbell page > > use in

Re: [RFC PATCH 3/5] mm/vma: add support for peer to peer to device vma

2019-01-30 Thread Jason Gunthorpe
On Wed, Jan 30, 2019 at 09:02:08AM +0100, Christoph Hellwig wrote: > On Tue, Jan 29, 2019 at 08:58:35PM +0000, Jason Gunthorpe wrote: > > On Tue, Jan 29, 2019 at 01:39:49PM -0700, Logan Gunthorpe wrote: > > > > > implement the mapping. And I don't think we sho

Re: [RFC PATCH 3/5] mm/vma: add support for peer to peer to device vma

2019-01-30 Thread Jason Gunthorpe
On Wed, Jan 30, 2019 at 02:01:35PM -0700, Logan Gunthorpe wrote: > And I feel the GUP->SGL->DMA flow should still be what we are aiming > for. Even if we need a special GUP for special pages, and a special DMA > map; and the SGL still has to be homogenous *shrug* so what if the special GUP

Re: [RFC PATCH 3/5] mm/vma: add support for peer to peer to device vma

2019-01-30 Thread Jason Gunthorpe
On Wed, Jan 30, 2019 at 10:17:27AM -0700, Logan Gunthorpe wrote: > > > On 2019-01-29 9:18 p.m., Jason Gunthorpe wrote: > > Every attempt to give BAR memory to struct page has run into major > > trouble, IMHO, so I like that this approach avoids that. > > > >

Re: [RFC PATCH 3/5] mm/vma: add support for peer to peer to device vma

2019-01-30 Thread Jason Gunthorpe
On Wed, Jan 30, 2019 at 09:00:06AM +0100, Christoph Hellwig wrote: > On Wed, Jan 30, 2019 at 04:18:48AM +0000, Jason Gunthorpe wrote: > > Every attempt to give BAR memory to struct page has run into major > > trouble, IMHO, so I like that this approach avoids that. > > W

Re: [RFC PATCH 3/5] mm/vma: add support for peer to peer to device vma

2019-01-30 Thread Jason Gunthorpe
On Wed, Jan 30, 2019 at 11:13:11AM -0700, Logan Gunthorpe wrote: > > > On 2019-01-30 10:44 a.m., Jason Gunthorpe wrote: > > I don't see why a special case with a VMA is really that different. > > Well one *really* big difference is the VMA changes necessarily exp

Re: [RFC PATCH 3/5] mm/vma: add support for peer to peer to device vma

2019-01-30 Thread Jason Gunthorpe
On Wed, Jan 30, 2019 at 03:43:32PM -0500, Jerome Glisse wrote: > On Wed, Jan 30, 2019 at 08:11:19PM +0000, Jason Gunthorpe wrote: > > On Wed, Jan 30, 2019 at 01:00:02PM -0700, Logan Gunthorpe wrote: > > > > > We never changed SGLs. We still use them to pass p2pdma p

Re: [RFC PATCH 3/5] mm/vma: add support for peer to peer to device vma

2019-01-30 Thread Jason Gunthorpe
On Wed, Jan 30, 2019 at 01:00:02PM -0700, Logan Gunthorpe wrote: > We never changed SGLs. We still use them to pass p2pdma pages, only we > need to be a bit careful where we send the entire SGL. I see no reason > why we can't continue to be careful once their in userspace if there's > something

Re: [RFC PATCH 3/5] mm/vma: add support for peer to peer to device vma

2019-01-30 Thread Jason Gunthorpe
On Wed, Jan 30, 2019 at 12:48:33PM -0700, Logan Gunthorpe wrote: > > > On 2019-01-30 12:19 p.m., Jason Gunthorpe wrote: > > On Wed, Jan 30, 2019 at 11:13:11AM -0700, Logan Gunthorpe wrote: > >> > >> > >> On 2019-01-30 10:44 a.m., Jason Gunthorpe w

Re: [RFC PATCH 3/5] mm/vma: add support for peer to peer to device vma

2019-01-30 Thread Jason Gunthorpe
On Wed, Jan 30, 2019 at 02:22:34PM -0500, Jerome Glisse wrote: > For GPU it would not work, GPU might want to use main memory (because > it is running out of BAR space) it is a lot easier if the p2p_map > callback calls the right dma map function (for page or io) rather than > having to define

Re: [RFC PATCH 3/5] mm/vma: add support for peer to peer to device vma

2019-01-30 Thread Jason Gunthorpe
On Wed, Jan 30, 2019 at 12:45:46PM -0700, Logan Gunthorpe wrote: > > > On 2019-01-30 12:06 p.m., Jason Gunthorpe wrote: > >> Way less problems than not having struct page for doing anything > >> non-trivial. If you map the BAR to userspace with remap_pfn_range &

Re: [RFC PATCH 3/5] mm/vma: add support for peer to peer to device vma

2019-01-30 Thread Jason Gunthorpe
On Wed, Jan 30, 2019 at 04:45:25PM -0500, Jerome Glisse wrote: > On Wed, Jan 30, 2019 at 08:50:00PM +0000, Jason Gunthorpe wrote: > > On Wed, Jan 30, 2019 at 03:43:32PM -0500, Jerome Glisse wrote: > > > On Wed, Jan 30, 2019 at 08:11:19PM +0000, Jason Gunthorpe wrote: > >

Re: [RFC PATCH 3/5] mm/vma: add support for peer to peer to device vma

2019-01-30 Thread Jason Gunthorpe
On Wed, Jan 30, 2019 at 05:30:27PM -0500, Jerome Glisse wrote: > > What is the problem in the HMM mirror that it needs this restriction? > > No restriction at all here. I think i just wasn't understood. Are you are talking about from the exporting side - where the thing creating the VMA can

Re: [RFC PATCH 3/5] mm/vma: add support for peer to peer to device vma

2019-01-31 Thread Jason Gunthorpe
On Thu, Jan 31, 2019 at 09:13:55AM +0100, Christoph Hellwig wrote: > On Wed, Jan 30, 2019 at 03:52:13PM -0700, Logan Gunthorpe wrote: > > > *shrug* so what if the special GUP called a VMA op instead of > > > traversing the VMA PTEs today? Why does it really matter? It could > > > easily change to

Re: [RFC PATCH 3/5] mm/vma: add support for peer to peer to device vma

2019-01-31 Thread Jason Gunthorpe
On Thu, Jan 31, 2019 at 02:35:14PM -0500, Jerome Glisse wrote: > > Basically invert the API flow - the DMA map would be done close to > > GUP, not buried in the driver. This absolutely doesn't work for every > > flow we have, but it does enable the ones that people seem to care > > about when

Re: [RFC PATCH 3/5] mm/vma: add support for peer to peer to device vma

2019-01-30 Thread Jason Gunthorpe
On Wed, Jan 30, 2019 at 03:52:13PM -0700, Logan Gunthorpe wrote: > > > On 2019-01-30 2:50 p.m., Jason Gunthorpe wrote: > > On Wed, Jan 30, 2019 at 02:01:35PM -0700, Logan Gunthorpe wrote: > > > >> And I feel the GUP->SGL->DMA flow should still be what we

Re: use exact allocation for dma coherent memory

2019-06-19 Thread Jason Gunthorpe
On Mon, Jun 17, 2019 at 10:33:42AM +0200, Christoph Hellwig wrote: > > drivers/infiniband/hw/cxgb4/qp.c > >129 static int alloc_host_sq(struct c4iw_rdev *rdev, struct t4_sq *sq) > >130 { > >131 sq->queue = dma_alloc_coherent(&(rdev->lldi.pdev->dev), > > sq->memsize, > >

Re: [PATCH v3 hmm 00/11] Add mmu_notifier_get/put for managing mmu notifier registrations

2019-08-16 Thread Jason Gunthorpe
On Tue, Aug 06, 2019 at 08:15:37PM -0300, Jason Gunthorpe wrote: > This series is already entangled with patches in the hmm & RDMA tree and > will require some git topic branches for the RDMA ODP stuff. I intend for > it to go through the hmm tree. > Jason Gunthorpe (11): >

Re: [PATCH v3 hmm 03/11] mm/mmu_notifiers: add a get/put scheme for the registration

2019-08-14 Thread Jason Gunthorpe
On Wed, Aug 14, 2019 at 02:20:31PM -0700, Ralph Campbell wrote: > > On 8/6/19 4:15 PM, Jason Gunthorpe wrote: > > From: Jason Gunthorpe > > > > Many places in the kernel have a flow where userspace will create some > > object and that object will need

Re: [PATCH v3 hmm 08/11] drm/radeon: use mmu_notifier_get/put for struct radeon_mn

2019-08-15 Thread Jason Gunthorpe
On Thu, Aug 15, 2019 at 10:28:21AM +0200, Christian K├Ânig wrote: > Am 07.08.19 um 01:15 schrieb Jason Gunthorpe: > > From: Jason Gunthorpe > > > > radeon is using a device global hash table to track what mmu_notifiers > > have been registered on struct mm. This is

Re: [PATCH v3 hmm 04/11] misc/sgi-gru: use mmu_notifier_get/put for struct gru_mm_struct

2019-08-14 Thread Jason Gunthorpe
On Thu, Aug 08, 2019 at 12:25:56PM +0200, Christoph Hellwig wrote: > Looks good, > > Reviewed-by: Christoph Hellwig Dimitri, are you OK with this patch? Thanks, Jason ___ iommu mailing list iommu@lists.linux-foundation.org

Re: [PATCH v3 hmm 08/11] drm/radeon: use mmu_notifier_get/put for struct radeon_mn

2019-08-14 Thread Jason Gunthorpe
On Tue, Aug 06, 2019 at 08:15:45PM -0300, Jason Gunthorpe wrote: > From: Jason Gunthorpe > > radeon is using a device global hash table to track what mmu_notifiers > have been registered on struct mm. This is better served with the new > get/put scheme instead. > > r

Re: [PATCH v3 hmm 00/11] Add mmu_notifier_get/put for managing mmu notifier registrations

2019-08-21 Thread Jason Gunthorpe
On Tue, Aug 06, 2019 at 08:15:37PM -0300, Jason Gunthorpe wrote: > This series is already entangled with patches in the hmm & RDMA tree and > will require some git topic branches for the RDMA ODP stuff. I intend for > it to go through the hmm tree. The RDMA related patches have

Re: [PATCH v3 hmm 10/11] drm/amdkfd: use mmu_notifier_put

2019-08-07 Thread Jason Gunthorpe
On Tue, Aug 06, 2019 at 11:47:44PM +, Kuehling, Felix wrote: > On 2019-08-06 19:15, Jason Gunthorpe wrote: > > From: Jason Gunthorpe > > > > The sequence of mmu_notifier_unregister_no_release(), > > mmu_notifier_call_srcu() is identical to mmu_notifier_put()

[PATCH v3 hmm 08/11] drm/radeon: use mmu_notifier_get/put for struct radeon_mn

2019-08-06 Thread Jason Gunthorpe
From: Jason Gunthorpe radeon is using a device global hash table to track what mmu_notifiers have been registered on struct mm. This is better served with the new get/put scheme instead. radeon has a bug where it was not blocking notifier release() until all the BO's had been invalidated

[PATCH v3 hmm 00/11] Add mmu_notifier_get/put for managing mmu notifier registrations

2019-08-06 Thread Jason Gunthorpe
From: Jason Gunthorpe This series introduces a new registration flow for mmu_notifiers based on the idea that the user would like to get a single refcounted piece of memory for a mm, keyed to its use. For instance many users of mmu_notifiers use an interval tree or similar to dispatch

[PATCH v3 hmm 10/11] drm/amdkfd: use mmu_notifier_put

2019-08-06 Thread Jason Gunthorpe
From: Jason Gunthorpe The sequence of mmu_notifier_unregister_no_release(), mmu_notifier_call_srcu() is identical to mmu_notifier_put() with the free_notifier callback. As this is the last user of those APIs, converting it means we can drop them. Signed-off-by: Jason Gunthorpe --- drivers

[PATCH v3 hmm 09/11] drm/amdkfd: fix a use after free race with mmu_notifer unregister

2019-08-06 Thread Jason Gunthorpe
From: Jason Gunthorpe When using mmu_notifer_unregister_no_release() the caller must ensure there is a SRCU synchronize before the mn memory is freed, otherwise use after free races are possible, for instance: CPU0 CPU1

[PATCH v3 hmm 05/11] hmm: use mmu_notifier_get/put for 'struct hmm'

2019-08-06 Thread Jason Gunthorpe
From: Jason Gunthorpe This is a significant simplification, it eliminates all the remaining 'hmm' stuff in mm_struct, eliminates krefing along the critical notifier paths, and takes away all the ugly locking and abuse of page_table_lock. mmu_notifier_get() provides the single struct hmm per

[PATCH v3 hmm 03/11] mm/mmu_notifiers: add a get/put scheme for the registration

2019-08-06 Thread Jason Gunthorpe
From: Jason Gunthorpe Many places in the kernel have a flow where userspace will create some object and that object will need to connect to the subsystem's mmu_notifier subscription for the duration of its lifetime. In this case the subsystem is usually tracking multiple mm_structs

[PATCH v3 hmm 07/11] RDMA/odp: remove ib_ucontext from ib_umem

2019-08-06 Thread Jason Gunthorpe
From: Jason Gunthorpe At this point the ucontext is only being stored to access the ib_device, so just store the ib_device directly instead. This is more natural and logical as the umem has nothing to do with the ucontext. Signed-off-by: Jason Gunthorpe --- drivers/infiniband/core/umem.c

[PATCH v3 hmm 06/11] RDMA/odp: use mmu_notifier_get/put for 'struct ib_ucontext_per_mm'

2019-08-06 Thread Jason Gunthorpe
From: Jason Gunthorpe This is a significant simplification, no extra list is kept per FD, and the interval tree is now shared between all the ucontexts, reducing overhead if there are multiple ucontexts active. Signed-off-by: Jason Gunthorpe --- drivers/infiniband/core/umem_odp.c| 170

[PATCH v3 hmm 02/11] mm/mmu_notifiers: do not speculatively allocate a mmu_notifier_mm

2019-08-06 Thread Jason Gunthorpe
From: Jason Gunthorpe A prior commit e0f3c3f78da2 ("mm/mmu_notifier: init notifier if necessary") made an attempt at doing this, but had to be reverted as calling the GFP_KERNEL allocator under the i_mmap_mutex causes deadlock, see commit 35cfa2b0b491 ("mm/mmu_notifier: alloca

[PATCH v3 hmm 11/11] mm/mmu_notifiers: remove unregister_no_release

2019-08-06 Thread Jason Gunthorpe
From: Jason Gunthorpe mmu_notifier_unregister_no_release() and mmu_notifier_call_srcu() no longer have any users, they have all been converted to use mmu_notifier_put(). So delete this difficult to use interface. Signed-off-by: Jason Gunthorpe --- include/linux/mmu_notifier.h | 5 - mm

[PATCH v3 hmm 01/11] mm/mmu_notifiers: hoist do_mmu_notifier_register down_write to the caller

2019-08-06 Thread Jason Gunthorpe
From: Jason Gunthorpe This simplifies the code to not have so many one line functions and extra logic. __mmu_notifier_register() simply becomes the entry point to register the notifier, and the other one calls it under lock. Also add a lockdep_assert to check that the callers are holding

[PATCH v3 hmm 04/11] misc/sgi-gru: use mmu_notifier_get/put for struct gru_mm_struct

2019-08-06 Thread Jason Gunthorpe
From: Jason Gunthorpe GRU is already using almost the same algorithm as get/put, it even helpfully has a 10 year old comment to make this algorithm common, which is finally happening. There are a few differences and fixes from this conversion: - GRU used rcu not srcu to read the hlist - Unclear

Re: remove DMA_ATTR_WRITE_BARRIER

2019-11-13 Thread Jason Gunthorpe
On Wed, Nov 13, 2019 at 08:32:12AM +0100, Christoph Hellwig wrote: > There is no implementation of the DMA_ATTR_WRITE_BARRIER flag left > now that the ia64 sn2 code has been removed. Drop the flag and > the calling convention to set it in the RDMA code. This looks OK, do you want it to go

Re: remove DMA_ATTR_WRITE_BARRIER

2019-11-14 Thread Jason Gunthorpe
On Wed, Nov 13, 2019 at 08:32:12AM +0100, Christoph Hellwig wrote: > There is no implementation of the DMA_ATTR_WRITE_BARRIER flag left > now that the ia64 sn2 code has been removed. Drop the flag and the > calling convention to set it in the RDMA code. Applied to rdma.git for-next There were

Re: [PATCH v4 01/26] mm/mmu_notifiers: pass private data down to alloc_notifier()

2020-02-24 Thread Jason Gunthorpe
On Mon, Feb 24, 2020 at 07:23:36PM +0100, Jean-Philippe Brucker wrote: > The new allocation scheme introduced by 2c7933f53f6b ("mm/mmu_notifiers: > add a get/put scheme for the registration") provides a convenient way > for users to attach notifier data to an mm. However, it would be even > better

Re: [PATCH v4 01/26] mm/mmu_notifiers: pass private data down to alloc_notifier()

2020-02-28 Thread Jason Gunthorpe
On Fri, Feb 28, 2020 at 03:39:35PM +0100, Jean-Philippe Brucker wrote: > > > + list_for_each_entry_rcu(bond, _mm->devices, mm_head) { > > > + /* > > > + * To ensure that we observe the initialization of io_mm fields > > > + * by io_mm_finalize() before the registration of

Re: [PATCH v4 01/26] mm/mmu_notifiers: pass private data down to alloc_notifier()

2020-02-28 Thread Jason Gunthorpe
On Fri, Feb 28, 2020 at 04:04:27PM +0100, Jean-Philippe Brucker wrote: > On Fri, Feb 28, 2020 at 10:48:44AM -0400, Jason Gunthorpe wrote: > > On Fri, Feb 28, 2020 at 03:39:35PM +0100, Jean-Philippe Brucker wrote: > > > > > + list_for_each_entry_rcu(bon

Re: [PATCH v4 02/26] iommu/sva: Manage process address spaces

2020-02-28 Thread Jason Gunthorpe
On Fri, Feb 28, 2020 at 03:40:07PM +0100, Jean-Philippe Brucker wrote: > > > Device > > > + * 00:00.0 accesses address spaces X and Y, each corresponding to an > > > mm_struct. > > > + * Devices 00:01.* only access address space Y. In addition each > > > + * IOMMU_DOMAIN_DMA domain has a private

Re: [PATCH v4 01/26] mm/mmu_notifiers: pass private data down to alloc_notifier()

2020-03-06 Thread Jason Gunthorpe
On Fri, Mar 06, 2020 at 10:56:14AM +0100, Jean-Philippe Brucker wrote: > I tried to keep it simple like that: normally mmu_notifier_get() is called > in bind(), and mmu_notifier_put() is called in unbind(). > > Multiple device drivers may call bind() with the same mm. Each bind() > calls

Re: [PATCH v4 01/26] mm/mmu_notifiers: pass private data down to alloc_notifier()

2020-03-06 Thread Jason Gunthorpe
On Fri, Mar 06, 2020 at 03:35:56PM +0100, Jean-Philippe Brucker wrote: > On Fri, Mar 06, 2020 at 09:09:19AM -0400, Jason Gunthorpe wrote: > > On Fri, Mar 06, 2020 at 10:56:14AM +0100, Jean-Philippe Brucker wrote: > > > I tried to keep it simple like that: normally mmu_notifi

Re: [PATCH v4 01/26] mm/mmu_notifiers: pass private data down to alloc_notifier()

2020-02-25 Thread Jason Gunthorpe
On Tue, Feb 25, 2020 at 10:24:39AM +0100, Jean-Philippe Brucker wrote: > On Mon, Feb 24, 2020 at 03:00:56PM -0400, Jason Gunthorpe wrote: > > On Mon, Feb 24, 2020 at 07:23:36PM +0100, Jean-Philippe Brucker wrote: > > > The new allocation scheme introduced by 2c7933f53f6b

Re: [PATCH v4 01/26] mm/mmu_notifiers: pass private data down to alloc_notifier()

2020-03-06 Thread Jason Gunthorpe
On Fri, Mar 06, 2020 at 05:15:19PM +0100, Jean-Philippe Brucker wrote: > On Fri, Mar 06, 2020 at 10:52:45AM -0400, Jason Gunthorpe wrote: > > On Fri, Mar 06, 2020 at 03:35:56PM +0100, Jean-Philippe Brucker wrote: > > > On Fri, Mar 06, 2020 at 09:09:19AM -0400, Jason Gunthorpe wro

Re: [PATCH 0/2] iommu: Remove iommu_sva_ops::mm_exit()

2020-04-09 Thread Jason Gunthorpe
On Thu, Apr 09, 2020 at 09:21:34AM -0700, Jacob Pan wrote: > On Thu, 9 Apr 2020 11:25:19 -0300 > Jason Gunthorpe wrote: > > > On Thu, Apr 09, 2020 at 07:14:24AM -0700, Jacob Pan wrote: > > > > When the process is killed, mm release can happen before fds are &g

Re: [PATCH 0/2] iommu: Remove iommu_sva_ops::mm_exit()

2020-04-08 Thread Jason Gunthorpe
On Wed, Apr 08, 2020 at 04:04:25PM +0200, Jean-Philippe Brucker wrote: > The IOMMU SVA API currently requires device drivers to implement an > mm_exit() callback, which stops device jobs that do DMA. This function > is called in the release() MMU notifier, when an address space that is > shared

Re: [PATCH 0/2] iommu: Remove iommu_sva_ops::mm_exit()

2020-04-08 Thread Jason Gunthorpe
On Wed, Apr 08, 2020 at 11:35:52AM -0700, Jacob Pan wrote: > Hi Jean, > > On Wed, 8 Apr 2020 16:04:25 +0200 > Jean-Philippe Brucker wrote: > > > The IOMMU SVA API currently requires device drivers to implement an > > mm_exit() callback, which stops device jobs that do DMA. This function > > is

Re: [PATCH 0/2] iommu: Remove iommu_sva_ops::mm_exit()

2020-04-08 Thread Jason Gunthorpe
On Wed, Apr 08, 2020 at 02:35:52PM -0700, Jacob Pan wrote: > > On Wed, Apr 08, 2020 at 11:35:52AM -0700, Jacob Pan wrote: > > > Hi Jean, > > > > > > On Wed, 8 Apr 2020 16:04:25 +0200 > > > Jean-Philippe Brucker wrote: > > > > > > > The IOMMU SVA API currently requires device drivers to

Re: [PATCH v5 01/25] mm/mmu_notifiers: pass private data down to alloc_notifier()

2020-04-14 Thread Jason Gunthorpe
ed by not having A->ctx be allocated memory, but inlined into the notifier struct But I can't think of a down side to not add a params either. Reviewed-by: Jason Gunthorpe Regards, Jason ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu

Re: [PATCH v5 02/25] iommu/sva: Manage process address spaces

2020-04-20 Thread Jason Gunthorpe
On Mon, Apr 20, 2020 at 09:42:13AM +0200, Jean-Philippe Brucker wrote: > On Thu, Apr 16, 2020 at 05:13:31AM -0700, Christoph Hellwig wrote: > > On Thu, Apr 16, 2020 at 10:54:02AM +0200, Jean-Philippe Brucker wrote: > > > On Thu, Apr 16, 2020 at 12:28:52AM -0700, Christoph Hellwig wrote: > > > > >

Re: [PATCH 0/2] iommu: Remove iommu_sva_ops::mm_exit()

2020-04-09 Thread Jason Gunthorpe
On Wed, Apr 08, 2020 at 04:48:02PM -0700, Jacob Pan wrote: > > Yes, this is the proper way, when the DMA is stopped and no use of the > > PASID remains then you can drop the mmu notifier and release the PASID > > entirely. If that is linked to the lifetime of the FD then forget > > completely

Re: [PATCH 0/2] iommu: Remove iommu_sva_ops::mm_exit()

2020-04-09 Thread Jason Gunthorpe
On Wed, Apr 08, 2020 at 04:49:01PM -0700, Fenghua Yu wrote: > > > Isn't the idea of mmu_notifier is to avoid holding the mm reference and > > > rely on the notifier to tell us when mm is going away? > > > > The notifier only holds a mmgrab(), not a mmget() - this allows > > exit_mmap to proceed,

Re: [PATCH 0/2] iommu: Remove iommu_sva_ops::mm_exit()

2020-04-09 Thread Jason Gunthorpe
On Thu, Apr 09, 2020 at 07:14:24AM -0700, Jacob Pan wrote: > > When the process is killed, mm release can happen before fds are > > released. If you look at do_exit() in kernel/exit.c: > > > > exit_mm() > > mmput() > >-> mmu release notifier > > ... > > exit_files() >

Re: [PATCH v4 01/26] mm/mmu_notifiers: pass private data down to alloc_notifier()

2020-03-13 Thread Jason Gunthorpe
On Fri, Mar 13, 2020 at 07:49:29PM +0100, Jean-Philippe Brucker wrote: > On Fri, Mar 06, 2020 at 01:42:39PM -0400, Jason Gunthorpe wrote: > > On Fri, Mar 06, 2020 at 05:15:19PM +0100, Jean-Philippe Brucker wrote: > > > On Fri, Mar 06, 2020 at 10:52:45AM -0400, Jason Gunthorpe wro

Re: [PATCH v6 17/25] iommu/arm-smmu-v3: Implement iommu_sva_bind/unbind()

2020-05-01 Thread Jason Gunthorpe
On Fri, May 01, 2020 at 05:15:52AM -0700, Christoph Hellwig wrote: > > @@ -432,6 +432,7 @@ config ARM_SMMU_V3 > > tristate "ARM Ltd. System MMU Version 3 (SMMUv3) Support" > > depends on ARM64 > > select IOMMU_API > > + select IOMMU_SVA > > select IOMMU_IO_PGTABLE_LPAE > >

Re: [PATCH v4 01/26] mm/mmu_notifiers: pass private data down to alloc_notifier()

2020-03-17 Thread Jason Gunthorpe
On Mon, Mar 16, 2020 at 08:46:59AM -0700, Christoph Hellwig wrote: > On Fri, Mar 06, 2020 at 09:09:19AM -0400, Jason Gunthorpe wrote: > > This is why release must do invalidate all - but it doesn't need to do > > any more - as no SPTE can be established without a mmget()

Re: [trivial PATCH] treewide: Convert switch/case fallthrough; to break;

2020-09-09 Thread Jason Gunthorpe
On Wed, Sep 09, 2020 at 01:06:39PM -0700, Joe Perches wrote: > fallthrough to a separate case/default label break; isn't very readable. > > Convert pseudo-keyword fallthrough; statements to a simple break; when > the next label is case or default and the only statement in the next > label block

Re: [PATCH v7 00/16] vfio: expose virtual Shared Virtual Addressing to VMs

2020-09-15 Thread Jason Gunthorpe
On Mon, Sep 14, 2020 at 03:44:38PM -0700, Raj, Ashok wrote: > Hi Jason, > > I thought we discussed this at LPC, but still seems to be going in > circles :-(. We discused mdev at LPC, not PASID. PASID applies widely to many device and needs to be introduced with a wide community agreement so all

Re: [PATCH v7 00/16] vfio: expose virtual Shared Virtual Addressing to VMs

2020-09-15 Thread Jason Gunthorpe
On Mon, Sep 14, 2020 at 04:33:10PM -0600, Alex Williamson wrote: > Can you explain that further, or spit-ball what you think this /dev/sva > interface looks like and how a user might interact between vfio and > this new interface? When you open it you get some container, inside the container

Re: [PATCH v7 00/16] vfio: expose virtual Shared Virtual Addressing to VMs

2020-09-15 Thread Jason Gunthorpe
On Tue, Sep 15, 2020 at 11:11:54AM -0700, Raj, Ashok wrote: > > PASID applies widely to many device and needs to be introduced with a > > wide community agreement so all scenarios will be supportable. > > True, reading some of the earlier replies I was clearly confused as I > thought you were

Re: [PATCH v7 00/16] vfio: expose virtual Shared Virtual Addressing to VMs

2020-09-15 Thread Jason Gunthorpe
On Tue, Sep 15, 2020 at 12:26:32PM -0700, Raj, Ashok wrote: > > Yes, there is. There is a limited pool of HW PASID's. If one user fork > > bombs it can easially claim an unreasonable number from that pool as > > each process will claim a PASID. That can DOS the rest of the system. > > Not sure

Re: [PATCH v7 00/16] vfio: expose virtual Shared Virtual Addressing to VMs

2020-09-15 Thread Jason Gunthorpe
On Tue, Sep 15, 2020 at 03:08:51PM -0700, Jacob Pan wrote: > > A PASID vIOMMU solution sharable with VDPA and VFIO, based on a PASID > > control char dev (eg /dev/sva, or maybe /dev/iommu) seems like a > > reasonable starting point for discussion. > > I am not sure what can really be consolidated

Re: [PATCH v7 00/16] vfio: expose virtual Shared Virtual Addressing to VMs

2020-09-14 Thread Jason Gunthorpe
On Mon, Sep 14, 2020 at 03:31:13PM +0200, Jean-Philippe Brucker wrote: > > Jason suggest something like /dev/sva. There will be a lot of other > > subsystems that could benefit from this (e.g vDPA). > > Do you have a more precise idea of the interface /dev/sva would provide, > how it would

Re: [PATCH v7 00/16] vfio: expose virtual Shared Virtual Addressing to VMs

2020-09-14 Thread Jason Gunthorpe
On Mon, Sep 14, 2020 at 10:38:10AM +, Tian, Kevin wrote: > is widely used thus can better help verify the core logic with > many existing devices. For vSVA, vDPA support has not be started > while VFIO support is close to be accepted. It doesn't make much > sense by blocking the VFIO part

Re: [PATCH v7 00/16] vfio: expose virtual Shared Virtual Addressing to VMs

2020-09-14 Thread Jason Gunthorpe
On Mon, Sep 14, 2020 at 09:22:47AM -0700, Raj, Ashok wrote: > Hi Jason, > > On Mon, Sep 14, 2020 at 10:47:38AM -0300, Jason Gunthorpe wrote: > > On Mon, Sep 14, 2020 at 03:31:13PM +0200, Jean-Philippe Brucker wrote: > > > > > > Jason suggest something like /dev/

Re: [PATCH v7 00/16] vfio: expose virtual Shared Virtual Addressing to VMs

2020-09-14 Thread Jason Gunthorpe
On Mon, Sep 14, 2020 at 10:58:57AM -0600, Alex Williamson wrote: > "its own special way" is arguable, VFIO is just making use of what's > being proposed as the uapi via its existing IOMMU interface. I mean, if we have a /dev/sva then it makes no sense to extend the VFIO interfaces with the same

Re: [PATCH v7 00/16] vfio: expose virtual Shared Virtual Addressing to VMs

2020-09-14 Thread Jason Gunthorpe
On Mon, Sep 14, 2020 at 12:23:28PM -0600, Alex Williamson wrote: > On Mon, 14 Sep 2020 14:41:21 -0300 > Jason Gunthorpe wrote: > > > On Mon, Sep 14, 2020 at 10:58:57AM -0600, Alex Williamson wrote: > > > > > "its own special way" is arguable, VFIO

Re: [patch V2 24/46] PCI: vmd: Mark VMD irqdomain with DOMAIN_BUS_VMD_MSI

2020-08-31 Thread Jason Gunthorpe
On Wed, Aug 26, 2020 at 01:16:52PM +0200, Thomas Gleixner wrote: > From: Thomas Gleixner > > Devices on the VMD bus use their own MSI irq domain, but it is not > distinguishable from regular PCI/MSI irq domains. This is required > to exclude VMD devices from getting the irq domain pointer set by

Re: [patch V2 46/46] irqchip: Add IMS (Interrupt Message Storm) driver - NOT FOR MERGING

2020-08-31 Thread Jason Gunthorpe
On Wed, Aug 26, 2020 at 01:17:14PM +0200, Thomas Gleixner wrote: > + * ims_queue_info - Information to create an IMS queue domain > + * @queue_lock: Callback which informs the device driver that > + * an interrupt management operation starts. > + *

Re: (proposal) RE: [PATCH v7 00/16] vfio: expose virtual Shared Virtual Addressing to VMs

2020-10-16 Thread Jason Gunthorpe
On Wed, Oct 14, 2020 at 03:16:22AM +, Tian, Kevin wrote: > Hi, Alex and Jason (G), > > How about your opinion for this new proposal? For now looks both > Jason (W) and Jean are OK with this direction and more discussions > are possibly required for the new /dev/ioasid interface. Internally >

Re: (proposal) RE: [PATCH v7 00/16] vfio: expose virtual Shared Virtual Addressing to VMs

2020-10-19 Thread Jason Gunthorpe
On Mon, Oct 19, 2020 at 08:39:03AM +, Liu, Yi L wrote: > Hi Jason, > > Good to see your response. Ah, I was away > > > > Second, IOMMU nested translation is a per IOMMU domain > > > > capability. Since IOMMU domains are managed by VFIO/VDPA > > > > (alloc/free domain, attach/detach device,

Re: (proposal) RE: [PATCH v7 00/16] vfio: expose virtual Shared Virtual Addressing to VMs

2020-10-20 Thread Jason Gunthorpe
On Tue, Oct 20, 2020 at 09:40:14AM +, Liu, Yi L wrote: > > See previous discussion with Kevin. If I understand correctly, you expect a > > shared > > L2 table if vDPA and VFIO device are using the same PASID. > > L2 table sharing is not mandatory. The mapping is the same, but no need to >

Re: (proposal) RE: [PATCH v7 00/16] vfio: expose virtual Shared Virtual Addressing to VMs

2020-10-20 Thread Jason Gunthorpe
On Tue, Oct 20, 2020 at 10:21:41AM +, Liu, Yi L wrote: > > I'm sure there will be some > > weird overlaps because we can't delete any of the existing VFIO APIs, but > > that > > should not be a blocker. > > but the weird thing is what we should consider. And it perhaps not just > overlap, it

Re: (proposal) RE: [PATCH v7 00/16] vfio: expose virtual Shared Virtual Addressing to VMs

2020-10-20 Thread Jason Gunthorpe
On Tue, Oct 20, 2020 at 02:00:31PM +, Liu, Yi L wrote: > > From: Jason Gunthorpe > > Sent: Tuesday, October 20, 2020 9:55 PM > > > > On Tue, Oct 20, 2020 at 09:40:14AM +, Liu, Yi L wrote: > > > > > > See previous discussion with Kevin. If I

Re: [patch V2 34/46] PCI/MSI: Make arch_.*_msi_irq[s] fallbacks selectable

2020-08-28 Thread Jason Gunthorpe
On Fri, Aug 28, 2020 at 12:21:42PM +0100, Lorenzo Pieralisi wrote: > On Thu, Aug 27, 2020 at 01:20:40PM -0500, Bjorn Helgaas wrote: > > [...] > > > And I can't figure out what's special about tegra, rcar, and xilinx > > that makes them need it as well. Is there something I could grep for > > to

Re: [patch V2 34/46] PCI/MSI: Make arch_.*_msi_irq[s] fallbacks selectable

2020-08-28 Thread Jason Gunthorpe
On Fri, Aug 28, 2020 at 01:47:59PM +0100, Marc Zyngier wrote: > > So the arch_setup_msi_irq/etc is not really an arch hook, but some > > infrastructure to support those 4 PCI root port drivers. > > I happen to have a *really old* patch addressing Tegra [1], which > I was never able to test (no

Re: [patch RFC 38/38] irqchip: Add IMS array driver - NOT FOR MERGING

2020-08-21 Thread Jason Gunthorpe
On Fri, Aug 21, 2020 at 02:25:02AM +0200, Thomas Gleixner wrote: > +static void ims_mask_irq(struct irq_data *data) > +{ > + struct msi_desc *desc = irq_data_get_msi_desc(data); > + struct ims_array_slot __iomem *slot = desc->device_msi.priv_iomem; > + u32 __iomem *ctrl = >ctrl; > + >

Re: [patch RFC 38/38] irqchip: Add IMS array driver - NOT FOR MERGING

2020-08-21 Thread Jason Gunthorpe
On Fri, Aug 21, 2020 at 09:47:43PM +0200, Thomas Gleixner wrote: > On Fri, Aug 21 2020 at 09:45, Jason Gunthorpe wrote: > > On Fri, Aug 21, 2020 at 02:25:02AM +0200, Thomas Gleixner wrote: > >> +static void ims_mask_irq(struct irq_data *data) > >> +{ &g

Re: [patch RFC 38/38] irqchip: Add IMS array driver - NOT FOR MERGING

2020-08-21 Thread Jason Gunthorpe
On Sat, Aug 22, 2020 at 01:47:12AM +0200, Thomas Gleixner wrote: > On Fri, Aug 21 2020 at 17:17, Jason Gunthorpe wrote: > > On Fri, Aug 21, 2020 at 09:47:43PM +0200, Thomas Gleixner wrote: > >> So if I understand correctly then the queue memory where the MSI > >>

Re: [patch RFC 38/38] irqchip: Add IMS array driver - NOT FOR MERGING

2020-08-22 Thread Jason Gunthorpe
On Sat, Aug 22, 2020 at 03:34:45AM +0200, Thomas Gleixner wrote: > >> One question is whether the device can see partial updates to that > >> memory due to the async 'swap' of context from the device CPU. > > > > It is worse than just partial updates.. The device operation is much > > more like

Re: [patch V2 00/46] x86, PCI, XEN, genirq ...: Prepare for device MSI

2020-09-30 Thread Jason Gunthorpe
On Wed, Sep 30, 2020 at 08:41:48AM +0200, Thomas Gleixner wrote: > On Tue, Sep 29 2020 at 16:03, Megha Dey wrote: > > On 8/26/2020 4:16 AM, Thomas Gleixner wrote: > >> #9 is obviously just for the folks interested in IMS > >> > > > > I see that the tip tree (as of 9/29) has most of these patches

Re: [patch V2 24/46] PCI: vmd: Mark VMD irqdomain with DOMAIN_BUS_VMD_MSI

2020-09-30 Thread Jason Gunthorpe
On Wed, Sep 30, 2020 at 12:45:30PM +, Derrick, Jonathan wrote: > Hi Jason > > On Mon, 2020-08-31 at 11:39 -0300, Jason Gunthorpe wrote: > > On Wed, Aug 26, 2020 at 01:16:52PM +0200, Thomas Gleixner wrote: > > > From: Thomas Gleixner > > > > > > De

Re: [patch V2 24/46] PCI: vmd: Mark VMD irqdomain with DOMAIN_BUS_VMD_MSI

2020-09-30 Thread Jason Gunthorpe
On Wed, Sep 30, 2020 at 01:08:27PM +, Derrick, Jonathan wrote: > +Megha > > On Wed, 2020-09-30 at 09:57 -0300, Jason Gunthorpe wrote: > > On Wed, Sep 30, 2020 at 12:45:30PM +, Derrick, Jonathan wrote: > > > Hi Jason > > > > > > On Mon, 2020-0

Re: [PATCH v7 00/16] vfio: expose virtual Shared Virtual Addressing to VMs

2020-09-17 Thread Jason Gunthorpe
On Thu, Sep 17, 2020 at 11:53:49AM +0800, Jason Wang wrote: > > > When VDPA is used by qemu it makes sense that the PASID will be an > > > arbitary IOVA map constructed to be 1:1 with the guest vCPU physical > > > map. /dev/sva allows a single uAPI to do this kind of setup, and qemu > > > can

Re: [PATCH v7 00/16] vfio: expose virtual Shared Virtual Addressing to VMs

2020-09-16 Thread Jason Gunthorpe
On Wed, Sep 16, 2020 at 10:32:17AM +0200, Jean-Philippe Brucker wrote: > And this is the only PASID model for Arm SMMU (and AMD IOMMU, I believe): > the PASID space of a PCI function cannot be shared between host and guest, > so we assign the whole PASID table along with the RID. Since we need the

Re: [PATCH v7 00/16] vfio: expose virtual Shared Virtual Addressing to VMs

2020-09-16 Thread Jason Gunthorpe
On Tue, Sep 15, 2020 at 05:22:26PM -0700, Jacob Pan (Jun) wrote: > > If user space wants to bind page tables, create the PASID with > > /dev/sva, use ioctls there to setup the page table the way it wants, > > then pass the now configured PASID to a driver that can use it. > > Are we talking

Re: [PATCH v7 00/16] vfio: expose virtual Shared Virtual Addressing to VMs

2020-09-16 Thread Jason Gunthorpe
On Wed, Sep 16, 2020 at 01:19:18AM +, Tian, Kevin wrote: > > From: Jason Gunthorpe > > Sent: Tuesday, September 15, 2020 10:29 PM > > > > > Do they need a device at all? It's not clear to me why RID based > > > IOMMU management fits withi

Re: [PATCH v7 00/16] vfio: expose virtual Shared Virtual Addressing to VMs

2020-09-16 Thread Jason Gunthorpe
On Wed, Sep 16, 2020 at 09:33:43AM -0700, Raj, Ashok wrote: > On Wed, Sep 16, 2020 at 12:07:54PM -0300, Jason Gunthorpe wrote: > > On Tue, Sep 15, 2020 at 05:22:26PM -0700, Jacob Pan (Jun) wrote: > > > > If user space wants to bind page tables, create the PASID with > &g

Re: [PATCH v7 00/16] vfio: expose virtual Shared Virtual Addressing to VMs

2020-09-16 Thread Jason Gunthorpe
On Wed, Sep 16, 2020 at 11:21:10AM -0700, Jacob Pan (Jun) wrote: > Hi Jason, > On Wed, 16 Sep 2020 14:01:13 -0300, Jason Gunthorpe > wrote: > > > On Wed, Sep 16, 2020 at 09:33:43AM -0700, Raj, Ashok wrote: > > > On Wed, Sep 16, 2020 at 12:07:54PM -0300, Jason Guntho

Re: [PATCH v7 00/16] vfio: expose virtual Shared Virtual Addressing to VMs

2020-09-16 Thread Jason Gunthorpe
On Wed, Sep 16, 2020 at 06:20:52PM +0200, Jean-Philippe Brucker wrote: > On Wed, Sep 16, 2020 at 11:51:48AM -0300, Jason Gunthorpe wrote: > > On Wed, Sep 16, 2020 at 10:32:17AM +0200, Jean-Philippe Brucker wrote: > > > And this is the only PASID model for Arm SMMU (and AM

  1   2   3   4   >