Re: [PATCH v2 1/3] mm/gup: Introduce pin_user_pages_fd() for pinning shmem/hugetlbfs file pages (v2)

2023-11-06 Thread Jason Gunthorpe
> Cc: Hugh Dickins > Cc: Peter Xu > Cc: Gerd Hoffmann > Cc: Dongwon Kim > Cc: Junxiao Chang > Suggested-by: Jason Gunthorpe > Signed-off-by: Vivek Kasireddy > --- > include/linux/mm.h | 2 + > mm/gup.c | 99 ++++++ > 2 files changed, 101 insertions(+) Reviewed-by: Jason Gunthorpe Jason

Re: [PATCH v1 1/3] mm/gup: Introduce pin_user_pages_fd() for pinning shmem/hugetlbfs file pages

2023-10-10 Thread Jason Gunthorpe
On Tue, Oct 03, 2023 at 12:44:45AM -0700, Vivek Kasireddy wrote: > +/** > + * pin_user_pages_fd() - pin user pages associated with a file > + * @fd: the fd whose pages are to be pinned > + * @start: starting file offset > + * @nr_pages: number of pages from start to pin > + *

Re: [PATCH v1 1/3] mm/gup: Introduce pin_user_pages_fd() for pinning shmem/hugetlbfs file pages

2023-10-06 Thread Jason Gunthorpe
On Fri, Oct 06, 2023 at 10:03:33AM +0200, David Hildenbrand wrote: > > + * > > + * Returns number of pages pinned. This would be equal to the number of > > + * pages requested. > > + * If nr_pages is 0 or negative, returns 0. If no pages were pinned, > > returns > > + * -errno. > > + */ > > +long

Re: [PATCH v7 1/1] vfio/nvgpu: Add vfio pci variant module for grace hopper

2023-08-31 Thread Jason Gunthorpe
On Thu, Aug 31, 2023 at 12:21:50PM -0600, Alex Williamson wrote: > I assume the above driver understands how to access and make use of > this coherent memory whether running bare-metal or virtualized, so > potentially we have some understanding of how it's used by the driver, > which can't be

Re: [PATCH v1 0/3] udmabuf: Add support for page migration out of movable zone or CMA

2023-08-30 Thread Jason Gunthorpe
On Sun, Aug 27, 2023 at 07:05:59PM +, Kasireddy, Vivek wrote: > Hi Jason, David, > > > > > Sure, we can simply always fail when we detect ZONE_MOVABLE or > > > MIGRATE_CMA. > > > > Maybe that keeps at least some use cases working. > > > > > > That seems fairly reasonable > > AFAICS, failing

Re: [RFC v1 1/3] mm/mmu_notifier: Add a new notifier for mapping updates (new pages)

2023-08-30 Thread Jason Gunthorpe
On Mon, Aug 28, 2023 at 04:38:01AM +, Kasireddy, Vivek wrote: > Turns out, calling hmm_range_fault() from the invalidate callback was indeed > a problem and the reason why new pages were not faulted-in. In other words, > it looks like the invalidate callback is not the right place to invoke

Re: [PATCH v7 1/1] vfio/nvgpu: Add vfio pci variant module for grace hopper

2023-08-30 Thread Jason Gunthorpe
On Wed, Aug 30, 2023 at 06:50:32AM -0700, Christoph Hellwig wrote: > I know I'm chiming in a bit late, but what ultimate user space is going > to use this? We should not add anything to the kernel that can't > be used without fully open user space. qemu will get the matching VFIO userspace

Re: [PATCH v1 0/3] udmabuf: Add support for page migration out of movable zone or CMA

2023-08-25 Thread Jason Gunthorpe
On Thu, Aug 24, 2023 at 08:33:09PM +0200, David Hildenbrand wrote: > Sure, we can simply always fail when we detect ZONE_MOVABLE or MIGRATE_CMA. > Maybe that keeps at least some use cases working. That seems fairly reasonable Jason

Re: [PATCH v1 0/3] udmabuf: Add support for page migration out of movable zone or CMA

2023-08-24 Thread Jason Gunthorpe
On Thu, Aug 24, 2023 at 08:30:17PM +0200, David Hildenbrand wrote: > On 24.08.23 08:31, Kasireddy, Vivek wrote: > > Hi David, > > > > > > > > > > - Add a new API to the backing store/allocator to longterm-pin the > > > > > page. > > > > > For example, something along the lines of > > >

Re: [PATCH v1 0/3] udmabuf: Add support for page migration out of movable zone or CMA

2023-08-22 Thread Jason Gunthorpe
On Tue, Aug 22, 2023 at 05:36:56AM +, Kasireddy, Vivek wrote: > Hi Jason, > > > > This patch series adds support for migrating pages associated with > > > a udmabuf out of the movable zone or CMA to avoid breaking features > > > such as memory hotunplug. > > > > > > The first patch exports

Re: [PATCH] iommu: Remove the device_lock_assert() from __iommu_probe_device()

2023-08-21 Thread Jason Gunthorpe
On Mon, Aug 21, 2023 at 04:36:49PM +0200, Joerg Roedel wrote: > On Mon, Aug 21, 2023 at 09:51:13AM -0300, Jason Gunthorpe wrote: > > So now that Joerg has dropped it - what is your big idea to make the > > locking actually work right? > > I am not opposed to the general idea.

Re: [PATCH] iommu: Remove the device_lock_assert() from __iommu_probe_device()

2023-08-21 Thread Jason Gunthorpe
On Mon, Aug 21, 2023 at 12:06:40PM +0100, Robin Murphy wrote: > On 2023-08-18 22:32, Jason Gunthorpe wrote: > > It is subtle and was never documented or enforced, but there has always > > been an assumption that of_dma_configure_id() is not concurrent. It makes > > several

Re: [PATCH] iommu: Remove the device_lock_assert() from __iommu_probe_device()

2023-08-21 Thread Jason Gunthorpe
On Mon, Aug 21, 2023 at 02:27:47PM +0200, Joerg Roedel wrote: > On Mon, Aug 21, 2023 at 12:06:40PM +0100, Robin Murphy wrote: > > The solution is to drop those locking patches entirely and rethink the whole > > thing. > > Agreed, that was exactly what I thought when looking at this patch. > > I

[PATCH] iommu: Remove the device_lock_assert() from __iommu_probe_device()

2023-08-18 Thread Jason Gunthorpe
probing") Reported-by: Marek Szyprowski Closes: https://lore.kernel.org/all/d090f196-a5c2-b188-31bf-e42553d8d...@samsung.com/ Signed-off-by: Jason Gunthorpe --- drivers/bcma/main.c | 4 drivers/dma/qcom/hidma_mgmt.c| 4 drivers/g

Re: [PATCH v1 0/3] udmabuf: Add support for page migration out of movable zone or CMA

2023-08-17 Thread Jason Gunthorpe
On Wed, Aug 16, 2023 at 11:49:31PM -0700, Vivek Kasireddy wrote: > This patch series adds support for migrating pages associated with > a udmabuf out of the movable zone or CMA to avoid breaking features > such as memory hotunplug. > > The first patch exports check_and_migrate_movable_pages()

Re: [RFC PATCH v2 00/11] Device Memory TCP

2023-08-10 Thread Jason Gunthorpe
On Thu, Aug 10, 2023 at 11:44:53AM -0700, Mina Almasry wrote: > Someone will correct me if I'm wrong but I'm not sure netlink itself > will do (sufficient) access control. However I meant for the netlink > API to bind dma-bufs to be a CAP_NET_ADMIN API, and I forgot to add > this check in this

Re: [RFC PATCH v2 00/11] Device Memory TCP

2023-08-10 Thread Jason Gunthorpe
On Thu, Aug 10, 2023 at 12:29:08PM +0200, Christian König wrote: > Am 10.08.23 um 03:57 schrieb Mina Almasry: > > Changes in RFC v2: > > -- > > > > The sticking point in RFC v1[1] was the dma-buf pages approach we used to > > deliver the device memory to the TCP stack. RFC v2 is a

Re: [RFC v1 1/3] mm/mmu_notifier: Add a new notifier for mapping updates (new pages)

2023-08-08 Thread Jason Gunthorpe
On Tue, Aug 08, 2023 at 07:37:19AM +, Kasireddy, Vivek wrote: > Hi Jason, > > > > > > No, adding HMM_PFN_REQ_WRITE still doesn't help in fixing the issue. > > > Although, I do not have THP enabled (or built-in), shmem does not evict > > > the pages after hole punch as noted in the comment in

Re: [RFC v1 1/3] mm/mmu_notifier: Add a new notifier for mapping updates (new pages)

2023-08-04 Thread Jason Gunthorpe
On Fri, Aug 04, 2023 at 06:39:22AM +, Kasireddy, Vivek wrote: > No, adding HMM_PFN_REQ_WRITE still doesn't help in fixing the issue. > Although, I do not have THP enabled (or built-in), shmem does not evict > the pages after hole punch as noted in the comment in shmem_fallocate(): This is

Re: [RFC v1 1/3] mm/mmu_notifier: Add a new notifier for mapping updates (new pages)

2023-08-03 Thread Jason Gunthorpe
On Thu, Aug 03, 2023 at 07:35:51AM +, Kasireddy, Vivek wrote: > Hi Jason, > > > > > Right, the "the zero pages are changed into writable pages" in your > > > > above comment just might not apply, because there won't be any page > > > > replacement (hopefully :) ). > > > > > If the page

Re: [RFC v1 1/3] mm/mmu_notifier: Add a new notifier for mapping updates (new pages)

2023-08-01 Thread Jason Gunthorpe
On Tue, Aug 01, 2023 at 05:53:32PM +, Kasireddy, Vivek wrote: > > Right, the "the zero pages are changed into writable pages" in your > > above comment just might not apply, because there won't be any page > > replacement (hopefully :) ). > If the page replacement does not happen when there

Re: [RFC v1 1/3] mm/mmu_notifier: Add a new notifier for mapping updates (new pages)

2023-08-01 Thread Jason Gunthorpe
On Tue, Aug 01, 2023 at 02:26:03PM +0200, David Hildenbrand wrote: > On 01.08.23 14:23, Jason Gunthorpe wrote: > > On Tue, Aug 01, 2023 at 02:22:12PM +0200, David Hildenbrand wrote: > > > On 01.08.23 14:19, Jason Gunthorpe wrote: > > > > On Tue, Aug 01, 2023 at 05:3

Re: [RFC v1 1/3] mm/mmu_notifier: Add a new notifier for mapping updates (new pages)

2023-08-01 Thread Jason Gunthorpe
On Tue, Aug 01, 2023 at 02:22:12PM +0200, David Hildenbrand wrote: > On 01.08.23 14:19, Jason Gunthorpe wrote: > > On Tue, Aug 01, 2023 at 05:32:38AM +, Kasireddy, Vivek wrote: > > > > > > You get another invalidate because the memfd removes the zero pages &

Re: [RFC v1 1/3] mm/mmu_notifier: Add a new notifier for mapping updates (new pages)

2023-08-01 Thread Jason Gunthorpe
On Tue, Aug 01, 2023 at 05:32:38AM +, Kasireddy, Vivek wrote: > > You get another invalidate because the memfd removes the zero pages > > that hmm_range_fault installed in the PTEs before replacing them with > > actual writable pages. Then you do the move, and another > > hmm_range_fault, and

Re: [RFC v1 1/3] mm/mmu_notifier: Add a new notifier for mapping updates (new pages)

2023-07-30 Thread Jason Gunthorpe
On Sat, Jul 29, 2023 at 12:46:59AM +, Kasireddy, Vivek wrote: > > Later the importer decides it needs the memory again so it again asks > > for the dmabuf to be present, which does hmm_range_fault and gets > > whatever is appropriate at the time. > Unless I am missing something, I think just

Re: [RFC v1 1/3] mm/mmu_notifier: Add a new notifier for mapping updates (new pages)

2023-07-27 Thread Jason Gunthorpe
On Thu, Jul 27, 2023 at 07:34:30AM +, Kasireddy, Vivek wrote: > Hi Jason, > > > > > On Tue, Jul 25, 2023 at 10:44:09PM +, Kasireddy, Vivek wrote: > > > > If you still need the memory mapped then you re-call hmm_range_fault > > > > and re-obtain it. hmm_range_fault will resolve all the

Re: [RFC v1 1/3] mm/mmu_notifier: Add a new notifier for mapping updates (new pages)

2023-07-25 Thread Jason Gunthorpe
On Tue, Jul 25, 2023 at 10:44:09PM +, Kasireddy, Vivek wrote: > > If you still need the memory mapped then you re-call hmm_range_fault > > and re-obtain it. hmm_range_fault will resolve all the races and you > > get new pages. > IIUC, for my udmabuf use-case, it looks like calling

Re: [RFC v1 1/3] mm/mmu_notifier: Add a new notifier for mapping updates (new pages)

2023-07-25 Thread Jason Gunthorpe
On Mon, Jul 24, 2023 at 08:32:45PM +, Kasireddy, Vivek wrote: > Hi Jason, > > > > > On Mon, Jul 24, 2023 at 07:54:38AM +, Kasireddy, Vivek wrote: > > > > > > I'm not at all familiar with the udmabuf use case but that sounds > > > > brittle and effectively makes this notifier udmabuf

Re: [RFC v1 1/3] mm/mmu_notifier: Add a new notifier for mapping updates (new pages)

2023-07-24 Thread Jason Gunthorpe
On Mon, Jul 24, 2023 at 11:36:47PM +1000, Alistair Popple wrote: > My primary issue with this patch is the notifier is called without the > PTL while providing a PTE value. Right, this is no-go. The PTE value must be protected by the PTLs at all times. We've made enough bugs already by being

Re: [RFC v1 1/3] mm/mmu_notifier: Add a new notifier for mapping updates (new pages)

2023-07-24 Thread Jason Gunthorpe
On Mon, Jul 24, 2023 at 07:54:38AM +, Kasireddy, Vivek wrote: > > I'm not at all familiar with the udmabuf use case but that sounds > > brittle and effectively makes this notifier udmabuf specific right? > Oh, Qemu uses the udmabuf driver to provide Host Graphics components > (such as Spice,

Re: [RFC PATCH 00/10] Device Memory TCP

2023-07-19 Thread Jason Gunthorpe
On Wed, Jul 19, 2023 at 10:57:11AM -0700, Stephen Hemminger wrote: > Naive idea. > Would it be possible for process to use mmap() on the GPU memory and then > do zero copy TCP receive some how? Or is this what is being proposed. It could be possible, but currently there is no API to recover the

Re: [RFC v1 1/3] mm/mmu_notifier: Add a new notifier for mapping updates (new pages)

2023-07-18 Thread Jason Gunthorpe
On Wed, Jul 19, 2023 at 12:05:29AM +, Kasireddy, Vivek wrote: > > If there is no change to the PTEs then it is hard to see why this > > would be part of a mmu_notifier. > IIUC, the PTEs do get changed but only when a new page is faulted in. > For shmem, it looks like the PTEs are updated in

Re: [RFC PATCH 00/10] Device Memory TCP

2023-07-18 Thread Jason Gunthorpe
On Tue, Jul 18, 2023 at 10:36:52AM -0700, Mina Almasry wrote: > That is specific to this proposal, and will likely be very different > in future ones. I thought the dma-buf pages approach was extensible > and the uapi belonged somewhere in dma-buf. Clearly not. The next > proposal, I think, will

Re: [PATCH 0/2] eventfd: simplify signal helpers

2023-07-18 Thread Jason Gunthorpe
On Mon, Jul 17, 2023 at 04:52:03PM -0600, Alex Williamson wrote: > On Mon, 17 Jul 2023 19:12:16 -0300 > Jason Gunthorpe wrote: > > > On Mon, Jul 17, 2023 at 01:08:31PM -0600, Alex Williamson wrote: > > > > > What would that mechanism be? We've been iterating on g

Re: [RFC v1 1/3] mm/mmu_notifier: Add a new notifier for mapping updates (new pages)

2023-07-18 Thread Jason Gunthorpe
On Tue, Jul 18, 2023 at 01:28:56AM -0700, Vivek Kasireddy wrote: > Currently, there does not appear to be any mechanism for letting > drivers or other kernel entities know about updates made in a > mapping particularly when a new page is faulted in. Providing > notifications for such situations is

Re: [PATCH 0/2] eventfd: simplify signal helpers

2023-07-17 Thread Jason Gunthorpe
On Mon, Jul 17, 2023 at 01:08:31PM -0600, Alex Williamson wrote: > What would that mechanism be? We've been iterating on getting the > serialization and buffering correct, but I don't know of another means > that combines the notification with a value, so we'd likely end up with > an eventfd

Re: [PATCH 0/2] eventfd: simplify signal helpers

2023-07-14 Thread Jason Gunthorpe
On Fri, Jul 14, 2023 at 09:05:21AM +0200, Christian Brauner wrote: > I have no skin in the game aside from having to drop this conversion > which I'm fine to do if there are actually users for this btu really, > that looks a lot like abusing an api that really wasn't designed for > this. Yeah, I

Re: [PATCH v1 0/2] udmabuf: Add back support for mapping hugetlb pages

2023-06-27 Thread Jason Gunthorpe
On Tue, Jun 27, 2023 at 12:00:38PM -0400, Peter Xu wrote: > On Tue, Jun 27, 2023 at 12:52:34PM -0300, Jason Gunthorpe wrote: > > On Mon, Jun 26, 2023 at 03:04:21PM -0400, Peter Xu wrote: > > > On Mon, Jun 26, 2023 at 03:18:48PM -0300, Jason Gunthorpe wrote: > > > >

Re: [PATCH v1 0/2] udmabuf: Add back support for mapping hugetlb pages

2023-06-27 Thread Jason Gunthorpe
On Mon, Jun 26, 2023 at 03:04:21PM -0400, Peter Xu wrote: > On Mon, Jun 26, 2023 at 03:18:48PM -0300, Jason Gunthorpe wrote: > > On Mon, Jun 26, 2023 at 08:14:27PM +0200, David Hildenbrand wrote: > > > > > So we might have to implement the same page migration as gup doe

Re: [PATCH v1 0/2] udmabuf: Add back support for mapping hugetlb pages

2023-06-26 Thread Jason Gunthorpe
On Mon, Jun 26, 2023 at 08:14:27PM +0200, David Hildenbrand wrote: > So we might have to implement the same page migration as gup does on > FOLL_LONGTERM here ... maybe there are more such cases/drivers that actually > require that handling when simply taking pages out of the memfd, believing >

Re: [PATCH v1 0/2] udmabuf: Add back support for mapping hugetlb pages

2023-06-26 Thread Jason Gunthorpe
On Fri, Jun 23, 2023 at 01:28:24PM -0400, Peter Xu wrote: > On Fri, Jun 23, 2023 at 01:37:58PM -0300, Jason Gunthorpe wrote: > > On Fri, Jun 23, 2023 at 12:35:45PM -0400, Peter Xu wrote: > > > > > It seems the previous concern on us

Re: [PATCH v1 0/2] udmabuf: Add back support for mapping hugetlb pages

2023-06-23 Thread Jason Gunthorpe
On Fri, Jun 23, 2023 at 12:35:45PM -0400, Peter Xu wrote: > It seems the previous concern on using gup was majorly fork(), if this is it: > > https://patchwork.freedesktop.org/patch/210992/?series=39879=2#comment_414213 Fork and GUP have been fixed since that comment anyhow there is no longer a

Re: [PATCH 1/3] iommu/dma: Clean up Kconfig

2023-05-05 Thread Jason Gunthorpe
On Fri, May 05, 2023 at 03:53:54PM +0100, Robin Murphy wrote: > > > diff --git a/drivers/iommu/Kconfig b/drivers/iommu/Kconfig > > > index 5c5cb5bee8b6..1d99c2d984fb 100644 > > > --- a/drivers/iommu/Kconfig > > > +++ b/drivers/iommu/Kconfig > > > @@ -137,7 +137,7 @@ config OF_IOMMU > > > #

Re: [PATCH 1/3] iommu/dma: Clean up Kconfig

2023-05-05 Thread Jason Gunthorpe
On Tue, Aug 16, 2022 at 06:28:03PM +0100, Robin Murphy wrote: > Although iommu-dma is a per-architecture chonce, that is currently > implemented in a rather haphazard way. Selecting from the arch Kconfig > was the original logical approach, but is complicated by having to > manage dependencies;

Re: [LSF/MM/BPF proposal]: Physr discussion

2023-04-17 Thread Jason Gunthorpe
On Tue, Feb 28, 2023 at 12:59:41PM -0800, T.J. Mercier wrote: > On Sat, Jan 21, 2023 at 7:03 AM Jason Gunthorpe wrote: > > > > I would like to have a session at LSF to talk about Matthew's > > physr discussion starter: > > > > https://lore.

Re: [PATCH 1/7] mm/gup: remove unused vmas parameter from get_user_pages()

2023-04-17 Thread Jason Gunthorpe
.c | 4 ++-- > 7 files changed, 11 insertions(+), 16 deletions(-) Reviewed-by: Jason Gunthorpe Jason

Re: [Lsf-pc] [LSF/MM/BPF proposal]: Physr discussion

2023-01-26 Thread Jason Gunthorpe
On Mon, Jan 23, 2023 at 12:50:52PM -0800, Dan Williams wrote: > Matthew Wilcox wrote: > > On Mon, Jan 23, 2023 at 11:36:51AM -0800, Dan Williams wrote: > > > Jason Gunthorpe via Lsf-pc wrote: > > > > I would like to have a session at LSF to talk about Matthew's &g

[PATCH v3 03/10] iommu: Add a gfp parameter to iommu_map_sg()

2023-01-23 Thread Jason Gunthorpe
Follow the pattern for iommu_map() and remove iommu_map_sg_atomic(). This allows __iommu_dma_alloc_noncontiguous() to use a GFP_KERNEL allocation here, based on the provided gfp flags. Reviewed-by: Kevin Tian Signed-off-by: Jason Gunthorpe --- drivers/iommu/dma-iommu.c | 5 +++-- drivers

[PATCH v3 09/10] iommu/s390: Push the gfp parameter to the kmem_cache_alloc()'s

2023-01-23 Thread Jason Gunthorpe
Signed-off-by: Jason Gunthorpe --- arch/s390/include/asm/pci_dma.h | 5 +++-- arch/s390/pci/pci_dma.c | 31 +-- drivers/iommu/s390-iommu.c | 15 +-- 3 files changed, 29 insertions(+), 22 deletions(-) diff --git a/arch/s390/include/asm

[PATCH v3 10/10] iommu/s390: Use GFP_KERNEL in sleepable contexts

2023-01-23 Thread Jason Gunthorpe
These contexts are sleepable, so use the proper annotation. The GFP_ATOMIC was added mechanically in the prior patches. Reviewed-by: Niklas Schnelle Reviewed-by: Matthew Rosato Signed-off-by: Jason Gunthorpe --- arch/s390/pci/pci_dma.c| 2 +- drivers/iommu/s390-iommu.c | 2 +- 2 files

[PATCH v3 01/10] iommu: Add a gfp parameter to iommu_map()

2023-01-23 Thread Jason Gunthorpe
The internal mechanisms support this, but instead of exposting the gfp to the caller it wrappers it into iommu_map() and iommu_map_atomic() Fix this instead of adding more variants for GFP_KERNEL_ACCOUNT. Reviewed-by: Kevin Tian Signed-off-by: Jason Gunthorpe --- arch/arm/mm/dma-mapping.c

[PATCH v3 08/10] iommu/intel: Use GFP_KERNEL in sleepable contexts

2023-01-23 Thread Jason Gunthorpe
These contexts are sleepable, so use the proper annotation. The GFP_ATOMIC was added mechanically in the prior patches. Reviewed-by: Lu Baolu Reviewed-by: Kevin Tian Signed-off-by: Jason Gunthorpe --- drivers/iommu/intel/iommu.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff

[PATCH v3 02/10] iommu: Remove iommu_map_atomic()

2023-01-23 Thread Jason Gunthorpe
There is only one call site and it can now just pass the GFP_ATOMIC to the normal iommu_map(). Reviewed-by: Kevin Tian Signed-off-by: Jason Gunthorpe --- drivers/iommu/dma-iommu.c | 2 +- drivers/iommu/iommu.c | 7 --- include/linux/iommu.h | 9 - 3 files changed, 1

[PATCH v3 05/10] iommufd: Use GFP_KERNEL_ACCOUNT for iommu_map()

2023-01-23 Thread Jason Gunthorpe
the iommu_domain. Many drivers will allocate these at iommu_map() time and will trivially do the right thing if we pass in GFP_KERNEL_ACCOUNT. Reviewed-by: Kevin Tian Signed-off-by: Jason Gunthorpe --- drivers/iommu/iommufd/pages.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git

[PATCH v3 06/10] iommu/intel: Add a gfp parameter to alloc_pgtable_page()

2023-01-23 Thread Jason Gunthorpe
This is eventually called by iommufd through intel_iommu_map_pages() and it should not be forced to atomic. Push the GFP_ATOMIC to all callers. Reviewed-by: Kevin Tian Reviewed-by: Lu Baolu Signed-off-by: Jason Gunthorpe --- drivers/iommu/intel/iommu.c | 14 +++--- drivers/iommu/intel

[PATCH v3 07/10] iommu/intel: Support the gfp argument to the map_pages op

2023-01-23 Thread Jason Gunthorpe
Flow it down to alloc_pgtable_page() via pfn_to_dma_pte() and __domain_mapping(). Reviewed-by: Kevin Tian Reviewed-by: Lu Baolu Signed-off-by: Jason Gunthorpe --- drivers/iommu/intel/iommu.c | 24 +++- 1 file changed, 15 insertions(+), 9 deletions(-) diff --git a/drivers

[PATCH v3 00/10] Let iommufd charge IOPTE allocations to the memory cgroup

2023-01-23 Thread Jason Gunthorpe
https://lore.kernel.org/r/0-v1-6e8b3997c46d+89e-iommu_map_gfp_...@nvidia.com Jason Gunthorpe (10): iommu: Add a gfp parameter to iommu_map() iommu: Remove iommu_map_atomic() iommu: Add a gfp parameter to iommu_map_sg() iommu/dma: Use the gfp parameter in __iommu_dma_alloc_

[PATCH v3 04/10] iommu/dma: Use the gfp parameter in __iommu_dma_alloc_noncontiguous()

2023-01-23 Thread Jason Gunthorpe
and policy bits that are only relevant for the buffer allocation before re-using them for internal allocations. Auditing says this is never called from an atomic context, so the GFP_ATOMIC is the incorrect flag. Reviewed-by: Kevin Tian Signed-off-by: Jason Gunthorpe --- drivers/iommu/dma-iommu.c

Re: [PATCH v2 04/10] iommu/dma: Use the gfp parameter in __iommu_dma_alloc_noncontiguous()

2023-01-23 Thread Jason Gunthorpe
On Fri, Jan 20, 2023 at 07:28:19PM +, Robin Murphy wrote: > Overall I'm starting to wonder if it might not be better to stick a "use > GFP_KERNEL_ACCOUNT if you allocate" flag in the domain for any level of the > API internals to pick up as appropriate, rather than propagate per-call gfp >

Re: [LSF/MM/BPF proposal]: Physr discussion

2023-01-23 Thread Jason Gunthorpe
On Mon, Jan 23, 2023 at 04:36:25AM +, Matthew Wilcox wrote: > > I've been working on an implementation and hope to have something > > draft to show on the lists in a few weeks. It is pretty clear there > > are several interesting decisions to make that I think will benefit > > from a live

[LSF/MM/BPF proposal]: Physr discussion

2023-01-21 Thread Jason Gunthorpe
I would like to have a session at LSF to talk about Matthew's physr discussion starter: https://lore.kernel.org/linux-mm/ydykweu0htv8m...@casper.infradead.org/ I have become interested in this with some immediacy because of IOMMUFD and this other discussion with Christoph:

Re: [PATCH v2 04/10] iommu/dma: Use the gfp parameter in __iommu_dma_alloc_noncontiguous()

2023-01-20 Thread Jason Gunthorpe
On Fri, Jan 20, 2023 at 07:28:19PM +, Robin Murphy wrote: > On 2023-01-18 18:00, Jason Gunthorpe wrote: > > Change the sg_alloc_table_from_pages() allocation that was hardwired to > > GFP_KERNEL to use the gfp parameter like the other allocations in this > > function. &

Re: [PATCH 1/8] iommu: Add a gfp parameter to iommu_map()

2023-01-20 Thread Jason Gunthorpe
On Fri, Jan 20, 2023 at 10:24:55AM +0100, Joerg Roedel wrote: > On Fri, Jan 06, 2023 at 01:24:11PM -0400, Jason Gunthorpe wrote: > > I think it is just better to follow kernel convention and have > > allocation functions include the GFP because it is a clear signal

[PATCH v2 04/10] iommu/dma: Use the gfp parameter in __iommu_dma_alloc_noncontiguous()

2023-01-18 Thread Jason Gunthorpe
Change the sg_alloc_table_from_pages() allocation that was hardwired to GFP_KERNEL to use the gfp parameter like the other allocations in this function. Auditing says this is never called from an atomic context, so it is safe as is, but reads wrong. Signed-off-by: Jason Gunthorpe --- drivers

[PATCH v2 09/10] iommu/s390: Push the gfp parameter to the kmem_cache_alloc()'s

2023-01-18 Thread Jason Gunthorpe
dma_alloc_cpu_table() and dma_alloc_page_table() are eventually called by iommufd through s390_iommu_map_pages() and it should not be forced to atomic. Thread the gfp parameter through the call chain starting from s390_iommu_map_pages(). Reviewed-by: Niklas Schnelle Signed-off-by: Jason

[PATCH v2 10/10] iommu/s390: Use GFP_KERNEL in sleepable contexts

2023-01-18 Thread Jason Gunthorpe
These contexts are sleepable, so use the proper annotation. The GFP_ATOMIC was added mechanically in the prior patches. Reviewed-by: Niklas Schnelle Signed-off-by: Jason Gunthorpe --- arch/s390/pci/pci_dma.c| 2 +- drivers/iommu/s390-iommu.c | 2 +- 2 files changed, 2 insertions(+), 2

[PATCH v2 00/10] Let iommufd charge IOPTE allocations to the memory cgroup

2023-01-18 Thread Jason Gunthorpe
the iommu wrappers - Split out the new GFP_KERNEL usages into dedicated patches so it is easier to check. No code change after the full series v1: https://lore.kernel.org/r/0-v1-6e8b3997c46d+89e-iommu_map_gfp_...@nvidia.com Jason Gunthorpe (10): iommu: Add a gfp parameter to iommu_map() iom

[PATCH v2 06/10] iommu/intel: Add a gfp parameter to alloc_pgtable_page()

2023-01-18 Thread Jason Gunthorpe
This is eventually called by iommufd through intel_iommu_map_pages() and it should not be forced to atomic. Push the GFP_ATOMIC to all callers. Signed-off-by: Jason Gunthorpe --- drivers/iommu/intel/iommu.c | 14 +++--- drivers/iommu/intel/iommu.h | 2 +- drivers/iommu/intel/pasid.c

[PATCH v2 07/10] iommu/intel: Support the gfp argument to the map_pages op

2023-01-18 Thread Jason Gunthorpe
Flow it down to alloc_pgtable_page() via pfn_to_dma_pte() and __domain_mapping(). Signed-off-by: Jason Gunthorpe --- drivers/iommu/intel/iommu.c | 24 +++- 1 file changed, 15 insertions(+), 9 deletions(-) diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel

[PATCH v2 05/10] iommufd: Use GFP_KERNEL_ACCOUNT for iommu_map()

2023-01-18 Thread Jason Gunthorpe
the iommu_domain. Many drivers will allocate these at iommu_map() time and will trivially do the right thing if we pass in GFP_KERNEL_ACCOUNT. Signed-off-by: Jason Gunthorpe --- drivers/iommu/iommufd/pages.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/drivers/iommu/iommufd

[PATCH v2 01/10] iommu: Add a gfp parameter to iommu_map()

2023-01-18 Thread Jason Gunthorpe
The internal mechanisms support this, but instead of exposting the gfp to the caller it wrappers it into iommu_map() and iommu_map_atomic() Fix this instead of adding more variants for GFP_KERNEL_ACCOUNT. Signed-off-by: Jason Gunthorpe --- arch/arm/mm/dma-mapping.c | 11

[PATCH v2 08/10] iommu/intel: Use GFP_KERNEL in sleepable contexts

2023-01-18 Thread Jason Gunthorpe
These contexts are sleepable, so use the proper annotation. The GFP_ATOMIC was added mechanically in the prior patches. Signed-off-by: Jason Gunthorpe --- drivers/iommu/intel/iommu.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/drivers/iommu/intel/iommu.c b/drivers

[PATCH v2 02/10] iommu: Remove iommu_map_atomic()

2023-01-18 Thread Jason Gunthorpe
There is only one call site and it can now just pass the GFP_ATOMIC to the normal iommu_map(). Signed-off-by: Jason Gunthorpe --- drivers/iommu/dma-iommu.c | 2 +- drivers/iommu/iommu.c | 7 --- include/linux/iommu.h | 9 - 3 files changed, 1 insertion(+), 17 deletions

[PATCH v2 03/10] iommu: Add a gfp parameter to iommu_map_sg()

2023-01-18 Thread Jason Gunthorpe
Follow the pattern for iommu_map() and remove iommu_map_sg_atomic(). This allows __iommu_dma_alloc_noncontiguous() to use a GFP_KERNEL allocation here, based on the provided gfp flags. Signed-off-by: Jason Gunthorpe --- drivers/iommu/dma-iommu.c | 5 +++-- drivers/iommu/iommu.c | 26

Re: [PATCH 6/8] iommu/intel: Add a gfp parameter to alloc_pgtable_page()

2023-01-18 Thread Jason Gunthorpe
On Wed, Jan 18, 2023 at 01:18:18AM +, Tian, Kevin wrote: > > From: Jason Gunthorpe > > Sent: Tuesday, January 17, 2023 9:30 PM > > > > On Tue, Jan 17, 2023 at 03:35:08AM +, Tian, Kevin wrote: > > > > From: Jason Gunthorpe > > &g

Re: [PATCH 6/8] iommu/intel: Add a gfp parameter to alloc_pgtable_page()

2023-01-17 Thread Jason Gunthorpe
On Tue, Jan 17, 2023 at 03:35:08AM +, Tian, Kevin wrote: > > From: Jason Gunthorpe > > Sent: Saturday, January 7, 2023 12:43 AM > > > > @@ -2676,7 +2676,7 @@ static int copy_context_table(struct intel_iommu > > *iommu, > > if (!old_ce)

Re: [PATCH 7/8] iommu/intel: Support the gfp argument to the map_pages op

2023-01-17 Thread Jason Gunthorpe
On Tue, Jan 17, 2023 at 03:38:51AM +, Tian, Kevin wrote: > > From: Jason Gunthorpe > > Sent: Saturday, January 7, 2023 12:43 AM > > > > @@ -2368,7 +2372,7 @@ static int iommu_domain_identity_map(struct > > dmar_domain *domain, > > > >

Re: [PATCH 1/8] iommu: Add a gfp parameter to iommu_map()

2023-01-16 Thread Jason Gunthorpe
On Fri, Jan 06, 2023 at 05:15:28PM +, Robin Murphy wrote: > However, echoing the recent activity over on the DMA API side of things, I > think it's still worth proactively constraining the set of permissible > flags, lest we end up with more weird problems if stuff that doesn't really > make

Re: [PATCH 1/8] iommu: Add a gfp parameter to iommu_map()

2023-01-06 Thread Jason Gunthorpe
On Fri, Jan 06, 2023 at 05:15:28PM +, Robin Murphy wrote: > On 2023-01-06 16:42, Jason Gunthorpe wrote: > > The internal mechanisms support this, but instead of exposting the gfp to > > the caller it wrappers it into iommu_map() and iommu_map_atomic() > > > > Fix

[PATCH 2/8] iommu: Remove iommu_map_atomic()

2023-01-06 Thread Jason Gunthorpe
There is only one call site and it can now just pass the GFP_ATOMIC to the normal iommu_map(). Signed-off-by: Jason Gunthorpe --- drivers/iommu/dma-iommu.c | 2 +- drivers/iommu/iommu.c | 7 --- include/linux/iommu.h | 9 - 3 files changed, 1 insertion(+), 17 deletions

[PATCH 7/8] iommu/intel: Support the gfp argument to the map_pages op

2023-01-06 Thread Jason Gunthorpe
Flow it down to alloc_pgtable_page() via pfn_to_dma_pte() and __domain_mapping(). Signed-off-by: Jason Gunthorpe --- drivers/iommu/intel/iommu.c | 24 +++- 1 file changed, 15 insertions(+), 9 deletions(-) diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel

[PATCH 6/8] iommu/intel: Add a gfp parameter to alloc_pgtable_page()

2023-01-06 Thread Jason Gunthorpe
This is eventually called by iommufd through intel_iommu_map_pages() and it should not be forced to atomic. Push the GFP_ATOMIC to all callers. Signed-off-by: Jason Gunthorpe --- drivers/iommu/intel/iommu.c | 14 +++--- drivers/iommu/intel/iommu.h | 2 +- drivers/iommu/intel/pasid.c

[PATCH 1/8] iommu: Add a gfp parameter to iommu_map()

2023-01-06 Thread Jason Gunthorpe
The internal mechanisms support this, but instead of exposting the gfp to the caller it wrappers it into iommu_map() and iommu_map_atomic() Fix this instead of adding more variants for GFP_KERNEL_ACCOUNT. Signed-off-by: Jason Gunthorpe --- arch/arm/mm/dma-mapping.c | 11

[PATCH 8/8] iommu/s390: Push the gfp parameter to the kmem_cache_alloc()'s

2023-01-06 Thread Jason Gunthorpe
dma_alloc_cpu_table() and dma_alloc_page_table() are eventually called by iommufd through s390_iommu_map_pages() and it should not be forced to atomic. Thread the gfp parameter through the call chain starting from s390_iommu_map_pages(). Signed-off-by: Jason Gunthorpe --- arch/s390/include/asm

[PATCH 4/8] iommu/dma: Use the gfp parameter in __iommu_dma_alloc_noncontiguous()

2023-01-06 Thread Jason Gunthorpe
Change the sg_alloc_table_from_pages() allocation that was hardwired to GFP_KERNEL to use the gfp parameter like the other allocations in this function. Auditing says this is never called from an atomic context, so it is safe as is, but reads wrong. Signed-off-by: Jason Gunthorpe --- drivers

[PATCH 3/8] iommu: Add a gfp parameter to iommu_map_sg()

2023-01-06 Thread Jason Gunthorpe
Follow the pattern for iommu_map() and remove iommu_map_sg_atomic(). This allows __iommu_dma_alloc_noncontiguous() to use a GFP_KERNEL allocation here, based on the provided gfp flags. Signed-off-by: Jason Gunthorpe --- drivers/iommu/dma-iommu.c | 5 +++-- drivers/iommu/iommu.c | 21

[PATCH 0/8] Let iommufd charge IOPTE allocations to the memory cgroup

2023-01-06 Thread Jason Gunthorpe
P argument and always use GFP_ATOMIC. This is problematic for iommufd anyhow, so fix it. AMD and ARM SMMUv2/3 are already correct. A follow up series will be needed to capture the allocations made when the iommu_domain itself is allocated, which will complete the job. Jason Gunthorpe (8): iommu:

[PATCH 5/8] iommufd: Use GFP_KERNEL_ACCOUNT for iommu_map()

2023-01-06 Thread Jason Gunthorpe
the iommu_domain. Many drivers will allocate these at iommu_map() time and will trivially do the right thing if we pass in GFP_KERNEL_ACCOUNT. Signed-off-by: Jason Gunthorpe --- drivers/iommu/iommufd/pages.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/drivers/iommu/iommufd

Re: [Linaro-mm-sig] Re: [PATCH] dma-buf: Require VM_PFNMAP vma for mmap

2022-11-23 Thread Jason Gunthorpe
On Wed, Nov 23, 2022 at 04:15:05PM +0100, Christian König wrote: > I have not the slightest idea how people got this impression, but I have > heard it so many time from so many different sources that there must be some > common cause to this. Is the maybe some book or tutorial how to sophisticate

Re: [Linaro-mm-sig] Re: [PATCH] dma-buf: Require VM_PFNMAP vma for mmap

2022-11-23 Thread Jason Gunthorpe
On Wed, Nov 23, 2022 at 03:34:54PM +0100, Daniel Vetter wrote: > > diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c > > index 1376a47fedeedb..4161241fc3228c 100644 > > --- a/virt/kvm/kvm_main.c > > +++ b/virt/kvm/kvm_main.c > > @@ -2598,6 +2598,19 @@ static int hva_to_pfn_remapped(struct

Re: [Linaro-mm-sig] Re: [PATCH] dma-buf: Require VM_PFNMAP vma for mmap

2022-11-23 Thread Jason Gunthorpe
On Wed, Nov 23, 2022 at 03:28:27PM +0100, Daniel Vetter wrote: > > This patch is known to be broken in so many ways. It also has a major > > security hole that it ignores the PTE flags making the page > > RO. Ignoring the special bit is somehow not surprising :( > > > > This probably doesn't

Re: [Linaro-mm-sig] Re: [PATCH] dma-buf: Require VM_PFNMAP vma for mmap

2022-11-23 Thread Jason Gunthorpe
On Wed, Nov 23, 2022 at 02:12:25PM +0100, Christian König wrote: > Am 23.11.22 um 13:53 schrieb Jason Gunthorpe: > > On Wed, Nov 23, 2022 at 01:49:41PM +0100, Christian König wrote: > > > Am 23.11.22 um 13:46 schrieb Jason Gunthorpe: > > > > On Wed, Nov 23, 2022 at 1

Re: [Linaro-mm-sig] Re: [PATCH] dma-buf: Require VM_PFNMAP vma for mmap

2022-11-23 Thread Jason Gunthorpe
On Wed, Nov 23, 2022 at 01:49:41PM +0100, Christian König wrote: > Am 23.11.22 um 13:46 schrieb Jason Gunthorpe: > > On Wed, Nov 23, 2022 at 11:06:55AM +0100, Daniel Vetter wrote: > > > > > > Maybe a GFP flag to set the page reference count to zero or something >

Re: [Linaro-mm-sig] Re: [PATCH] dma-buf: Require VM_PFNMAP vma for mmap

2022-11-23 Thread Jason Gunthorpe
On Wed, Nov 23, 2022 at 11:06:55AM +0100, Daniel Vetter wrote: > > Maybe a GFP flag to set the page reference count to zero or something > > like this? > > Hm yeah that might work. I'm not sure what it will all break though? > And we'd need to make sure that underflowing the page refcount dies

Re: [PATCH] dma-buf: Require VM_PFNMAP vma for mmap

2022-11-22 Thread Jason Gunthorpe
On Tue, Nov 22, 2022 at 08:29:05PM +0100, Daniel Vetter wrote: > You nuke all the ptes. Drivers that move have slightly more than a > bare struct file, they also have a struct address_space so that > invalidate_mapping_range() works. Okay, this is one of the ways that this can be made to work

Re: [PATCH] dma-buf: Require VM_PFNMAP vma for mmap

2022-11-22 Thread Jason Gunthorpe
On Tue, Nov 22, 2022 at 07:08:25PM +0100, Daniel Vetter wrote: > On Tue, 22 Nov 2022 at 19:04, Jason Gunthorpe wrote: > > > > On Tue, Nov 22, 2022 at 06:08:00PM +0100, Daniel Vetter wrote: > > > tldr; DMA buffers aren't normal memory, expecting that you can use > >

Re: [PATCH] dma-buf: Require VM_PFNMAP vma for mmap

2022-11-22 Thread Jason Gunthorpe
On Tue, Nov 22, 2022 at 06:08:00PM +0100, Daniel Vetter wrote: > tldr; DMA buffers aren't normal memory, expecting that you can use > them like that (like calling get_user_pages works, or that they're > accounting like any other normal memory) cannot be guaranteed. > > Since some userspace only

Re: [PATCH mm-unstable v1 12/20] RDMA/siw: remove FOLL_FORCE usage

2022-11-16 Thread Jason Gunthorpe
for ptrace access. > > Cc: Bernard Metzler > Cc: Jason Gunthorpe > Cc: Leon Romanovsky > Signed-off-by: David Hildenbrand > --- > drivers/infiniband/sw/siw/siw_mem.c | 9 - > 1 file changed, 4 insertions(+), 5 deletions(-) Reviewed-by: Jason Gunthorpe Jason

Re: [PATCH mm-unstable v1 11/20] RDMA/usnic: remove FOLL_FORCE usage

2022-11-16 Thread Jason Gunthorpe
for ptrace access. > > Cc: Christian Benvenuti > Cc: Nelson Escobar > Cc: Jason Gunthorpe > Cc: Leon Romanovsky > Signed-off-by: David Hildenbrand > --- > drivers/infiniband/hw/usnic/usnic_uiom.c | 9 - > 1 file changed, 4 insertions(+), 5 deletions(-) Reviewed-by: Jason Gunthorpe Jason

Re: [PATCH mm-unstable v1 10/20] RDMA/umem: remove FOLL_FORCE usage

2022-11-16 Thread Jason Gunthorpe
ptrace access. > > Tested-by: Leon Romanovsky # Over mlx4 and mlx5. > Cc: Jason Gunthorpe > Cc: Leon Romanovsky > Signed-off-by: David Hildenbrand > --- > drivers/infiniband/core/umem.c | 8 > 1 file changed, 4 insertions(+), 4 deletions(-) Reviewed-by: Jason Gunthorpe Jason

<    1   2   3   4   5   6   7   8   9   10   >