Re: [PATCH 2/2] mm: set PG_dma_pinned on get_user_pages*()

2018-06-19 Thread John Hubbard
On 06/19/2018 06:57 PM, Dan Williams wrote: > On Tue, Jun 19, 2018 at 6:34 PM, John Hubbard wrote: >> On 06/19/2018 06:24 PM, Dan Williams wrote: >>> On Tue, Jun 19, 2018 at 11:11 AM, John Hubbard wrote: >>>> On 06/19/2018 03:41 AM, Jan Kara wrote: >>>>

[PATCH v2 0/6] mm/fs: gup: don't unmap or drop filesystem buffers

2018-07-01 Thread john . hubbard
From: John Hubbard This fixes a few problems that came up when using devices (NICs, GPUs, for example) that want to have direct access to a chunk of system (CPU) memory, so that they can DMA to/from that memory. Problems [1] come up if that memory is backed by persistence storage; for example

[PATCH v2 3/6] mm: introduce zone_gup_lock, for dma-pinned pages

2018-07-01 Thread john . hubbard
From: John Hubbard The page->dma_pinned_flags and _count fields require lock protection. A lock at approximately the granularity of the zone_lru_lock is called for, but adding to the locking contention of zone_lru_lock is undesirable, because that is a pre-existing hot spot. Fortunat

[PATCH v2 5/6] mm: track gup pages with page->dma_pinned_* fields

2018-07-01 Thread john . hubbard
From: John Hubbard This patch sets and restores the new page->dma_pinned_flags and page->dma_pinned_count fields, but does not actually use them for anything yet. In order to use these fields at all, the page must be removed from any LRU list that it's on. The patch also adds some preca

[PATCH v2 1/6] mm: get_user_pages: consolidate error handling

2018-07-01 Thread john . hubbard
From: John Hubbard An upcoming patch requires a way to operate on each page that any of the get_user_pages_*() variants returns. In preparation for that, consolidate the error handling for __get_user_pages(). This provides a single location (the "out:" label) for operating on the col

[PATCH v2 6/6] mm: page_mkclean, ttu: handle pinned pages

2018-07-01 Thread john . hubbard
From: John Hubbard Update page_mkclean(), page_mkclean's callers, and try_to_unmap(), so that there is a choice: in some cases, skipped dma-pinned pages. In other cases (sync_mode == WB_SYNC_ALL), wait for those pages to become unpinned. This fixes some problems that came up when using devices

[PATCH v2 4/6] mm/fs: add a sync_mode param for clear_page_dirty_for_io()

2018-07-01 Thread john . hubbard
From: John Hubbard Add a sync_mode parameter to clear_page_dirty_for_io(), to specify the writeback sync mode, and also pass in the appropriate value (WB_SYNC_NONE or WB_SYNC_ALL), from each filesystem location that calls it. This will be used in subsequent patches, to allow page_mkclean

[PATCH v2 2/6] mm: introduce page->dma_pinned_flags, _count

2018-07-01 Thread john . hubbard
From: John Hubbard Add two struct page fields that, combined, are unioned with struct page->lru. There is no change in the size of struct page. These new fields are for type safety and clarity. Also add page flag accessors to test, set and clear the new page->dma_pinned_flags field. Th

Re: [PATCH v2 4/6] mm/fs: add a sync_mode param for clear_page_dirty_for_io()

2018-07-01 Thread John Hubbard
ed to the wrong git tree, please drop us a note to > help improve the system] > > url: > https://github.com/0day-ci/linux/commits/john-hubbard-gmail-com/mm-fs-gup-don-t-unmap-or-drop-filesystem-buffers/20180702-090125 > config: x86_64-randconfig-x010-201826 (attached as .config) &g

Re: [PATCH v2 4/6] mm/fs: add a sync_mode param for clear_page_dirty_for_io()

2018-07-01 Thread John Hubbard
ng git tree, please drop us a note to > help improve the system] > > url: > https://github.com/0day-ci/linux/commits/john-hubbard-gmail-com/mm-fs-gup-don-t-unmap-or-drop-filesystem-buffers/20180702-090125 > config: i386-randconfig-x075-201826 (attached as .config) > compiler:

Re: [PATCH v2 5/6] mm: track gup pages with page->dma_pinned_* fields

2018-07-01 Thread John Hubbard
ed to the wrong git tree, please drop us a note to > help improve the system] > > url: > https://github.com/0day-ci/linux/commits/john-hubbard-gmail-com/mm-fs-gup-don-t-unmap-or-drop-filesystem-buffers/20180702-090125 > config: x86_64-randconfig-x010-201826 (attached as .config) &g

Re: [PATCH v2 0/6] mm/fs: gup: don't unmap or drop filesystem buffers

2018-07-01 Thread John Hubbard
On 07/01/2018 05:56 PM, john.hubb...@gmail.com wrote: > From: John Hubbard > There were some typos in patches #4 and #5, which I've fixed locally. Let me know if anyone would like me to repost with those right away, otherwise I'll wait for other review besides the kbuild test robot. Mea

Re: [PATCH 2/2] mm: set PG_dma_pinned on get_user_pages*()

2018-07-02 Thread John Hubbard
On 07/01/2018 10:52 PM, Leon Romanovsky wrote: > On Thu, Jun 28, 2018 at 11:17:43AM +0200, Jan Kara wrote: >> On Wed 27-06-18 19:42:01, John Hubbard wrote: >>> On 06/27/2018 10:02 AM, Jan Kara wrote: >>>> On Wed 27-06-18 08:57:18, Jason Gunthorpe wrote: >>&g

Re: [PATCH 2/2] mm: set PG_dma_pinned on get_user_pages*()

2018-07-02 Thread John Hubbard
On 07/01/2018 11:34 PM, Leon Romanovsky wrote: > On Sun, Jul 01, 2018 at 11:10:04PM -0700, John Hubbard wrote: >> On 07/01/2018 10:52 PM, Leon Romanovsky wrote: >>> On Thu, Jun 28, 2018 at 11:17:43AM +0200, Jan Kara wrote: >>>> On Wed 27-06-18 19:42:01, John Hubbard

Re: [PATCH v2 5/6] mm: track gup pages with page->dma_pinned_* fields

2018-07-02 Thread John Hubbard
On 07/02/2018 02:53 AM, Jan Kara wrote: > On Sun 01-07-18 17:56:53, john.hubb...@gmail.com wrote: >> From: John Hubbard >> > ... > >> @@ -904,12 +907,24 @@ static inline void get_page(struct page *page) >> */ >> VM_BUG_ON_PAGE(page_ref_count(

Re: [PATCH v2 6/6] mm: page_mkclean, ttu: handle pinned pages

2018-07-02 Thread John Hubbard
ed in? Also the > locking is IMHO going to hurt a lot and we need to avoid it. > > What I think needs to happen is that in page_mkclean(), after you've > cleared all the page tables, you check PageDmaPinned() and wait if needed. > Page cannot be faulted in again as we hold page loc

Re: [PATCH v2 1/6] mm: get_user_pages: consolidate error handling

2018-07-02 Thread John Hubbard
On 07/02/2018 03:17 AM, Jan Kara wrote: > On Sun 01-07-18 17:56:49, john.hubb...@gmail.com wrote: >> From: John Hubbard >> >> An upcoming patch requires a way to operate on each page that >> any of the get_user_pages_*() variants returns. >> >> In prep

Re: [PATCH v2 5/6] mm: track gup pages with page->dma_pinned_* fields

2018-07-02 Thread John Hubbard
On 07/02/2018 05:08 PM, Christopher Lameter wrote: > On Mon, 2 Jul 2018, John Hubbard wrote: > >>> >>> These two are just wrong. You cannot make any page reference for >>> PageDmaPinned() account against a pin count. First, it is just conceptually >>>

Re: [PATCH v2 0/6] RFC: gup+dma: tracking dma-pinned pages

2018-11-20 Thread John Hubbard
's my main concern: > Hi Tom, Thanks again for looking at this! > On 11/10/2018 3:50 AM, john.hubb...@gmail.com wrote: >> From: John Hubbard >> ... >> -- >> WITHOUT the patch: >> --

[PATCH 0/1] mm/gup: finish consolidating error handling

2018-11-21 Thread john . hubbard
From: John Hubbard Hi, Keith Busch and Dan Williams noticed that this patch (which was part of my RFC[1] for the get_user_pages + DMA fix) also fixes a bug. Accordingly, I'm adjusting the changelog and posting this as it's own patch. [1] https://lkml.kernel.org/r/20181110085041.10071-1-jhubb

[PATCH] mm/gup: finish consolidating error handling

2018-11-21 Thread john . hubbard
From: John Hubbard Commit df06b37ffe5a4 ("mm/gup: cache dev_pagemap while pinning pages") attempted to operate on each page that get_user_pages had retrieved. In order to do that, it created a common exit point from the routine. However, one case was missed, which this patch fixes

Re: [PATCH v2 0/6] RFC: gup+dma: tracking dma-pinned pages

2018-11-21 Thread John Hubbard
On 11/21/18 8:49 AM, Tom Talpey wrote: > On 11/21/2018 1:09 AM, John Hubbard wrote: >> On 11/19/18 10:57 AM, Tom Talpey wrote: >>> ~14000 4KB read IOPS is really, really low for an NVMe disk. >> >> Yes, but Jan Kara's original config file for fio is *intended* to h

Re: [PATCH] mm/gup: finish consolidating error handling

2018-11-21 Thread John Hubbard
his call", rather than the generic case of crossing a vma boundary. (I think there's a fine point that I must be overlooking.) But it's still a valid case, either way. -- thanks, John Hubbard NVIDIA

Re: [PATCH 1/2] mm: introduce put_user_page*(), placeholder versions

2018-12-14 Thread John Hubbard
rs really are complex) really seems worth the extra work, so that's a big benefit. Next steps: I want to go try this dynamic_page approach out right away. If there are pieces such as page_to_pfn and related, that are already in progress, I'd definitely like to work on top of that. Also, any up front advice or pitfalls to avoid is always welcome, of course. :) thanks, -- John Hubbard NVIDIA

Re: [PATCH v2 0/6] RFC: gup+dma: tracking dma-pinned pages

2018-11-27 Thread John Hubbard
On 11/27/18 5:21 PM, Tom Talpey wrote: > On 11/21/2018 5:06 PM, John Hubbard wrote: >> On 11/21/18 8:49 AM, Tom Talpey wrote: >>> On 11/21/2018 1:09 AM, John Hubbard wrote: >>>> On 11/19/18 10:57 AM, Tom Talpey wrote: [...] >>> >>> What I'd real

Re: [PATCH v2 0/6] RFC: gup+dma: tracking dma-pinned pages

2018-11-29 Thread John Hubbard
On 11/28/18 5:59 AM, Tom Talpey wrote: > On 11/27/2018 9:52 PM, John Hubbard wrote: >> On 11/27/18 5:21 PM, Tom Talpey wrote: >>> On 11/21/2018 5:06 PM, John Hubbard wrote: >>>> On 11/21/18 8:49 AM, Tom Talpey wrote: >>>>> On 11/21/2018 1:09 AM, John Hu

Re: [PATCH v2 0/6] RFC: gup+dma: tracking dma-pinned pages

2018-11-29 Thread John Hubbard
On 11/29/18 6:18 PM, Tom Talpey wrote: > On 11/29/2018 8:39 PM, John Hubbard wrote: >> On 11/28/18 5:59 AM, Tom Talpey wrote: >>> On 11/27/2018 9:52 PM, John Hubbard wrote: >>>> On 11/27/18 5:21 PM, Tom Talpey wrote: >>>>> On 11/21/2018 5:06 PM, John H

Re: [PATCH v2 0/6] RFC: gup+dma: tracking dma-pinned pages

2018-11-29 Thread John Hubbard
On 11/29/18 6:30 PM, Tom Talpey wrote: > On 11/29/2018 9:21 PM, John Hubbard wrote: >> On 11/29/18 6:18 PM, Tom Talpey wrote: >>> On 11/29/2018 8:39 PM, John Hubbard wrote: >>>> On 11/28/18 5:59 AM, Tom Talpey wrote: >>>>> On 11/27/2018 9:52 PM, John H

Re: [PATCH 1/2] mm: introduce put_user_page*(), placeholder versions

2018-12-12 Thread John Hubbard
nned dax pages, see >>> dax_layout_busy_page(). As much as possible I want to eliminate the >>> concept of "dax pages" as a special case that gets sprinkled >>> throughout the mm. >>> >>>> For [O1] and [O2] i believe a solution with mapcount

Re: [PATCH 1/2] mm: introduce put_user_page*(), placeholder versions

2018-12-12 Thread John Hubbard
On 12/7/18 9:18 PM, Matthew Wilcox wrote: > On Fri, Dec 07, 2018 at 04:52:42PM -0800, John Hubbard wrote: >> I see. OK, HMM has done an efficient job of mopping up unused fields, and >> now we are >> completely out of space. At this point, after thinking about it carefull

Re: [PATCH 1/2] mm: introduce put_user_page*(), placeholder versions

2018-12-12 Thread John Hubbard
about it then trying to make it pass under the radar. > > This will put the burden on broken user and allow you to properly > recycle your DAX page. > > Think of it as revoke through mmu notifier. > > So patchset would be: > enum mmu_notifier_event { > + MMU_NOTI

Re: [PATCH 1/2] mm: introduce put_user_page*(), placeholder versions

2018-12-12 Thread John Hubbard
On 12/12/18 2:04 PM, Jerome Glisse wrote: > On Wed, Dec 12, 2018 at 01:56:00PM -0800, John Hubbard wrote: >> On 12/12/18 1:30 PM, Jerome Glisse wrote: >>> On Wed, Dec 12, 2018 at 08:27:35AM -0800, Dan Williams wrote: >>>> On Wed, Dec 12, 2018 at 7:03 AM Jerome Gli

Re: [PATCH 1/2] mm: introduce put_user_page*(), placeholder versions

2018-12-12 Thread John Hubbard
On 12/12/18 2:14 PM, Jerome Glisse wrote: > On Wed, Dec 12, 2018 at 02:11:58PM -0800, John Hubbard wrote: >> On 12/12/18 2:04 PM, Jerome Glisse wrote: >>> On Wed, Dec 12, 2018 at 01:56:00PM -0800, John Hubbard wrote: >>>> On 12/12/18 1:30 PM, Jerome Glisse wrote: &g

Re: [PATCH 1/2] mm: introduce put_user_page*(), placeholder versions

2018-12-12 Thread John Hubbard
t can replay page faults, in many cases. I think as long as we specify that the acceptable consequence of doing, say, umount on a filesystem that has active DMA happening is that the associated processes get killed, then we're going to be OK. What would worry me is if there was an expectation that processes could continue working properly after such a scenario. thanks, -- John Hubbard NVIDIA

Re: [PATCH 1/2] mm: introduce put_user_page*(), placeholder versions

2018-12-13 Thread John Hubbard
an't call ->page_mkwrite() from >>> put_user_page(), so I don't think this is workable at all. >> >> Hu why ? i can not think of any reason whike you could not. User of > > It's not a fault path, you can't safely lock pages, you can't take > fault-path only locks in the IO path (mmap_sem inversion problems), > etc. > Yes, I looked closer at ->page_mkwrite (ext4_page_mkwrite, for example), and it's clearly doing lock_page(), so it does seem like this particular detail (calling page_mkwrite from put_user_page) is dead. > /me has a nagging feeling this was all explained in a previous > discussions of this patchset... > Yes, lots of related discussion definitely happened already, for example this October thread covered page_mkwrite and interactions with gup: https://lore.kernel.org/r/20181001061127.GQ31060@dastard ...but so far, this is the first time I recall seeing a proposal to call page_mkwrite from put_user_page. thanks, -- John Hubbard NVIDIA

Re: [PATCH 1/2] mm: introduce put_user_page*(), placeholder versions

2018-12-13 Thread John Hubbard
On 12/13/18 9:21 PM, Dan Williams wrote: > On Thu, Dec 13, 2018 at 7:53 PM John Hubbard wrote: >> >> On 12/12/18 4:51 PM, Dave Chinner wrote: >>> On Wed, Dec 12, 2018 at 04:59:31PM -0500, Jerome Glisse wrote: >>>> On Thu, Dec 13, 2018 at 08:46:41AM +1100, Dave

Re: [PATCH 1/2] mm: introduce put_user_page*(), placeholder versions

2018-12-18 Thread John Hubbard
n that it is the lightest weight solution for that. So as I understand it, this would use page->_mapcount to store both the real mapcount, and the dma pinned count (simply added together), but only do so for file-backed (non-anonymous) pages: __get_user_pages() { ... get_page(page); if (!PageAnon) atomic_inc(page->_mapcount); ... } put_user_page(struct page *page) { ... if (!PageAnon) atomic_dec(>_mapcount); put_page(page); ... } ...and then in the various consumers of the DMA pinned count, we use page_mapped(page) to see if any mapcount remains, and if so, we treat it as DMA pinned. Is that what you had in mind? -- thanks, John Hubbard NVIDIA

Re: [PATCH 1/2] mm: introduce put_user_page*(), placeholder versions

2018-12-20 Thread John Hubbard
On 12/19/18 3:08 AM, Jan Kara wrote: > On Tue 18-12-18 21:07:24, Jerome Glisse wrote: >> On Tue, Dec 18, 2018 at 03:29:34PM -0800, John Hubbard wrote: >>> OK, so let's take another look at Jerome's _mapcount idea all by itself >>> (using >>> *only* the

Re: [PATCH 1/2] mm: introduce put_user_page*(), placeholder versions

2018-12-04 Thread John Hubbard
On 12/4/18 12:28 PM, Dan Williams wrote: > On Mon, Dec 3, 2018 at 4:17 PM wrote: >> >> From: John Hubbard >> >> Introduces put_user_page(), which simply calls put_page(). >> This provides a way to update all get_user_pages*() callers, >> so that they ca

Re: [PATCH 1/2] mm: introduce put_user_page*(), placeholder versions

2018-12-04 Thread John Hubbard
On 12/4/18 3:03 PM, Dan Williams wrote: > On Tue, Dec 4, 2018 at 1:56 PM John Hubbard wrote: >> >> On 12/4/18 12:28 PM, Dan Williams wrote: >>> On Mon, Dec 3, 2018 at 4:17 PM wrote: >>>> >>>> From: John Hubbard >>>>

Re: [PATCH 1/2] mm: introduce put_user_page*(), placeholder versions

2018-12-04 Thread John Hubbard
On 12/4/18 4:40 PM, Dan Williams wrote: > On Tue, Dec 4, 2018 at 4:37 PM Jerome Glisse wrote: >> >> On Tue, Dec 04, 2018 at 03:03:02PM -0800, Dan Williams wrote: >>> On Tue, Dec 4, 2018 at 1:56 PM John Hubbard wrote: >>>> >>>> On 12/4/18 12:28 P

Re: [PATCH 0/2] put_user_page*(): start converting the call sites

2018-12-04 Thread John Hubbard
hich I should have added in this cover letter. Here's a start: https://lore.kernel.org/r/20181110085041.10071-1-jhubb...@nvidia.com ...and it looks like this small patch series is not going to work out--I'm going to have to fall back to another RFC spin. So I'll be sure to include you and everyone on that. Hope that helps. thanks, -- John Hubbard NVIDIA

Re: [PATCH 1/2] mm: introduce put_user_page*(), placeholder versions

2018-12-04 Thread John Hubbard
On 12/3/18 11:53 PM, Mike Rapoport wrote: > Hi John, > > Thanks for having documentation as a part of the patch. Some kernel-doc > nits below. > > On Mon, Dec 03, 2018 at 04:17:19PM -0800, john.hubb...@gmail.com wrote: >> From: John Hubbard >> >> Introduces

Re: [PATCH 1/2] mm: introduce put_user_page*(), placeholder versions

2018-12-04 Thread John Hubbard
On 12/4/18 5:44 PM, Jerome Glisse wrote: > On Tue, Dec 04, 2018 at 05:15:19PM -0800, Matthew Wilcox wrote: >> On Tue, Dec 04, 2018 at 04:58:01PM -0800, John Hubbard wrote: >>> On 12/4/18 3:03 PM, Dan Williams wrote: >>>> Except the LRU fields are already in us

[PATCH] mei: fix use-after-free in mei_cl_write

2018-08-22 Thread john . hubbard
From: John Hubbard KASAN reports a use-after-free during startup, in mei_cl_write: BUG: KASAN: use-after-free in mei_cl_write+0x601/0x870 [mei] (drivers/misc/mei/client.c:1770) This is caused by commit 98e70866aacb ("mei: add support for variable length mei headers."), whi

Re: [PATCH 4/6] mm: introduce page->dma_pinned_flags, _count

2018-10-16 Thread John Hubbard
On 10/16/18 1:51 AM, Jan Kara wrote: > On Sun 14-10-18 10:01:24, Dave Chinner wrote: >> On Sat, Oct 13, 2018 at 12:34:12AM -0700, John Hubbard wrote: >>> On 10/12/18 8:55 PM, Dave Chinner wrote: >>>> On Thu, Oct 11, 2018 at 11:00:12PM -0700, john.hubb...@gmail.com w

Re: [PATCH 4/6] mm: introduce page->dma_pinned_flags, _count

2018-10-17 Thread John Hubbard
On 10/17/18 4:09 AM, Michal Hocko wrote: > On Tue 16-10-18 18:48:23, John Hubbard wrote: > [...] >> It's hard to say exactly what the active/inactive/unevictable list should >> be when DMA is done and put_user_page*() is called, because we don't know >> if some device rea

Re: [PATCH 3/4] infiniband/mm: convert to the new put_user_page() call

2018-10-02 Thread John Hubbard
On 10/1/18 7:35 AM, Dennis Dalessandro wrote: > On 9/28/2018 11:12 PM, John Hubbard wrote: >> On 9/28/18 8:39 AM, Jason Gunthorpe wrote: >>> On Thu, Sep 27, 2018 at 10:39:47PM -0700, john.hubb...@gmail.com wrote: >>>> From: John Hubbard >> [...] >>&g

Re: [PATCH 3/4] infiniband/mm: convert to the new put_user_page() call

2018-10-03 Thread John Hubbard
On 10/3/18 9:27 AM, Jan Kara wrote: > On Fri 28-09-18 20:12:33, John Hubbard wrote: >> static inline void release_user_pages(struct page **pages, >> - unsigned long npages) >> +

Re: [PATCH 2/4] mm: introduce put_user_page(), placeholder version

2018-10-03 Thread John Hubbard
On 10/3/18 9:22 AM, Jan Kara wrote: > On Thu 27-09-18 22:39:48, john.hubb...@gmail.com wrote: >> From: John Hubbard >> >> Introduces put_user_page(), which simply calls put_page(). >> This provides a way to update all get_user_pages*() callers, >> so that t

[PATCH v2 1/3] mm: get_user_pages: consolidate error handling

2018-10-04 Thread john . hubbard
From: John Hubbard An upcoming patch requires a way to operate on each page that any of the get_user_pages_*() variants returns. In preparation for that, consolidate the error handling for __get_user_pages(). This provides a single location (the "out:" label) for operating on the col

[PATCH v2 0/3] get_user_pages*() and RDMA: first steps

2018-10-04 Thread john . hubbard
From: John Hubbard Changes since v1: -- Renamed release_user_pages*() to put_user_pages*(), from Jan's feedback. -- Removed the goldfish.c changes, and instead, only included a single user (infiniband) of the new functions. That is because goldfish.c no longer has a name collision

[PATCH v2 3/3] infiniband/mm: convert to the new put_user_page[s]() calls

2018-10-04 Thread john . hubbard
From: John Hubbard For code that retains pages via get_user_pages*(), release those pages via the new put_user_page(), instead of put_page(). This prepares for eventually fixing the problem described in [1], and is following a plan listed in [2], [3], [4]. [1] https://lwn.net/Articles/753027

[PATCH v2 2/3] mm: introduce put_user_page[s](), placeholder versions

2018-10-04 Thread john . hubbard
From: John Hubbard Introduces put_user_page(), which simply calls put_page(). This provides a way to update all get_user_pages*() callers, so that they call put_user_page(), instead of put_page(). Also introduces put_user_pages(), and a few dirty/locked variations, as a replacement

Re: [PATCH v2 2/3] mm: introduce put_user_page[s](), placeholder versions

2018-10-05 Thread John Hubbard
On 10/5/18 8:17 AM, Jason Gunthorpe wrote: > On Thu, Oct 04, 2018 at 09:02:24PM -0700, john.hubb...@gmail.com wrote: >> From: John Hubbard >> >> Introduces put_user_page(), which simply calls put_page(). >> This provides a way to update all get_user_pages*() ca

Re: [PATCH 2/6] mm: introduce put_user_page*(), placeholder versions

2018-10-12 Thread John Hubbard
On 10/12/18 12:35 AM, Balbir Singh wrote: On Thu, Oct 11, 2018 at 11:00:10PM -0700, john.hubb...@gmail.com wrote: From: John Hubbard [...]>> +/* + * put_user_pages_dirty() - for each page in the @pages array, make + * that page (or its head page, if a compound page)

Re: [PATCH 1/6] mm: get_user_pages: consolidate error handling

2018-10-12 Thread John Hubbard
On 10/11/18 11:30 PM, Balbir Singh wrote: On Thu, Oct 11, 2018 at 11:00:09PM -0700, john.hubb...@gmail.com wrote: From: John Hubbard An upcoming patch requires a way to operate on each page that any of the get_user_pages_*() variants returns. In preparation for that, consolidate the error

Re: [PATCH 4/6] mm: introduce page->dma_pinned_flags, _count

2018-10-12 Thread John Hubbard
On 10/12/18 3:56 AM, Balbir Singh wrote: > On Thu, Oct 11, 2018 at 11:00:12PM -0700, john.hubb...@gmail.com wrote: >> From: John Hubbard [...] >> + * Because page->dma_pinned_flags is unioned with page->lru, any page that >> + * uses these flags must NOT be on an

Re: [PATCH 6/6] mm: track gup pages with page->dma_pinned_* fields

2018-10-12 Thread John Hubbard
On 10/12/18 4:07 AM, Balbir Singh wrote: > On Thu, Oct 11, 2018 at 11:00:14PM -0700, john.hubb...@gmail.com wrote: >> From: John Hubbard [...] >> +static int pin_page_for_dma(struct page *page) >> +{ >> +int ret = 0; >> +struct zone *zone; >&

Re: [PATCH 4/6] mm: introduce page->dma_pinned_flags, _count

2018-10-13 Thread John Hubbard
On 10/12/18 8:55 PM, Dave Chinner wrote: > On Thu, Oct 11, 2018 at 11:00:12PM -0700, john.hubb...@gmail.com wrote: >> From: John Hubbard [...] >> diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h >> index 5ed8f6292a53..017ab82e36ca 100644 >> --- a/includ

Re: [PATCH 4/6] mm: introduce page->dma_pinned_flags, _count

2018-10-13 Thread John Hubbard
On 10/13/18 9:47 AM, Christoph Hellwig wrote: > On Sat, Oct 13, 2018 at 12:34:12AM -0700, John Hubbard wrote: >> In patch 6/6, pin_page_for_dma(), which is called at the end of >> get_user_pages(), >> unceremoniously rips the pages out of the LRU, as a prerequisite to using

[PATCH 0/4] get_user_pages*() and RDMA: first steps

2018-09-27 Thread john . hubbard
From: John Hubbard Hi, This short series prepares for eventually fixing the problem described in [1], and is following a plan listed in [2]. I'd like to get the first two patches into the -mm tree. Patch 1, although not technically critical to do now, is still nice to have, because it's

[PATCH 1/4] mm: get_user_pages: consolidate error handling

2018-09-27 Thread john . hubbard
From: John Hubbard An upcoming patch requires a way to operate on each page that any of the get_user_pages_*() variants returns. In preparation for that, consolidate the error handling for __get_user_pages(). This provides a single location (the "out:" label) for operating on the col

[PATCH 4/4] goldfish_pipe/mm: convert to the new release_user_pages() call

2018-09-27 Thread john . hubbard
From: John Hubbard For code that retains pages via get_user_pages*(), release those pages via the new release_user_pages(), instead of calling put_page(). This prepares for eventually fixing the problem described in [1], and is following a plan listed in [2]. [1] https://lwn.net/Articles

[PATCH 2/4] mm: introduce put_user_page(), placeholder version

2018-09-27 Thread john . hubbard
From: John Hubbard Introduces put_user_page(), which simply calls put_page(). This provides a way to update all get_user_pages*() callers, so that they call put_user_page(), instead of put_page(). Also adds release_user_pages(), a drop-in replacement for release_pages(). This is intended

[PATCH 3/4] infiniband/mm: convert to the new put_user_page() call

2018-09-27 Thread john . hubbard
From: John Hubbard For code that retains pages via get_user_pages*(), release those pages via the new put_user_page(), instead of put_page(). This prepares for eventually fixing the problem described in [1], and is following a plan listed in [2]. [1] https://lwn.net/Articles/753027

Re: [PATCH 0/4] get_user_pages*() and RDMA: first steps

2018-09-28 Thread John Hubbard
On 9/28/18 8:29 AM, Jerome Glisse wrote: > On Thu, Sep 27, 2018 at 10:39:45PM -0700, john.hubb...@gmail.com wrote: >> From: John Hubbard >> >> Hi, >> >> This short series prepares for eventually fixing the problem described >> in [1], and is following a pl

Re: [PATCH 0/4] get_user_pages*() and RDMA: first steps

2018-09-28 Thread John Hubbard
On 9/28/18 2:49 PM, Jerome Glisse wrote: > On Fri, Sep 28, 2018 at 12:06:12PM -0700, John Hubbard wrote: >> On 9/28/18 8:29 AM, Jerome Glisse wrote: >>> On Thu, Sep 27, 2018 at 10:39:45PM -0700, john.hubb...@gmail.com wrote: >>>> From: John Hubbard [...] >>&g

Re: [PATCH 3/4] infiniband/mm: convert to the new put_user_page() call

2018-09-28 Thread John Hubbard
On 9/28/18 8:39 AM, Jason Gunthorpe wrote: > On Thu, Sep 27, 2018 at 10:39:47PM -0700, john.hubb...@gmail.com wrote: >> From: John Hubbard [...] >> >> diff --git a/drivers/infiniband/core/umem.c b/drivers/infiniband/core/umem.c >> index a41792dbae1f..9430d697c

[PATCH] lockdep: fix warning: print_lock_trace defined but not used

2019-05-19 Thread John Hubbard
E. McKenney Cc: Peter Zijlstra Cc: Thomas Gleixner Cc: Will Deacon Signed-off-by: John Hubbard --- kernel/locking/lockdep.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/kernel/locking/lockdep.c b/kernel/locking/lockdep.c index d06190fa5082..3065dc36c27a 100644 --- a/kernel/locking/

[PATCH 0/1] lockdep: fix warning: print_lock_trace defined but not used

2019-05-19 Thread John Hubbard
nyway. Cc: Frederic Weisbecker Cc: Andrew Morton Cc: Linus Torvalds Cc: Paul E. McKenney Cc: Peter Zijlstra Cc: Thomas Gleixner Cc: Will Deacon John Hubbard (1): lockdep: fix warning: print_lock_trace defined but not used kernel/locking/lockdep.c | 3 +++ 1 file changed, 3 insertions(+) -- 2.21.0

Re: [PATCH v3 0/1] mm: introduce put_user_page*(), placeholder versions

2019-03-14 Thread John Hubbard
(or, we just haven't been in communication with anyone who hit it). thanks, -- John Hubbard NVIDIA It's always good to know who might be doing this and why if for no other reason than as something to keep in mind when designing future interfaces.

Re: [PATCH v3 0/1] mm: introduce put_user_page*(), placeholder versions

2019-03-18 Thread John Hubbard
r > pagecache pages as that would regress existing applications. > > Honza > Christopher L, Are you OK with this approach now? If so, I'd like to collect any additional ACKs people are willing to provide, and ask Andrew to consider this first patch for 5.2, so we can get started. thanks, -- John Hubbard NVIDIA

Re: [PATCH v4 1/1] mm: introduce put_user_page*(), placeholder versions

2019-03-19 Thread John Hubbard
0300, Kirill A. Shutemov wrote: >>>>> On Fri, Mar 08, 2019 at 01:36:33PM -0800, john.hubb...@gmail.com wrote: >>>>>> From: John Hubbard >>>> [...] >>> Forgot to mention one thing, we had a discussion with Andrea and Jan >>> about set_page_d

Re: [PATCH v4 1/1] mm: introduce put_user_page*(), placeholder versions

2019-03-19 Thread John Hubbard
On 3/19/19 6:47 AM, Jerome Glisse wrote: > On Tue, Mar 19, 2019 at 03:04:17PM +0300, Kirill A. Shutemov wrote: >> On Fri, Mar 08, 2019 at 01:36:33PM -0800, john.hubb...@gmail.com wrote: >>> From: John Hubbard > > [...] >>> +void put_user_pages_dirty(struct p

Re: [PATCH v4 0/1] mm: introduce put_user_page*(), placeholder versions

2019-03-19 Thread John Hubbard
on. > > So lets get this merged. Whatever the solution will be, we will need this > markup. > Sounds good. Do you care to promote that thought into a formal ACK for me? :) thanks, -- John Hubbard NVIDIA

Re: [PATCH v4 1/1] mm: introduce put_user_page*(), placeholder versions

2019-03-19 Thread John Hubbard
On 3/19/19 7:06 AM, Kirill A. Shutemov wrote: > On Tue, Mar 19, 2019 at 09:47:24AM -0400, Jerome Glisse wrote: >> On Tue, Mar 19, 2019 at 03:04:17PM +0300, Kirill A. Shutemov wrote: >>> On Fri, Mar 08, 2019 at 01:36:33PM -0800, john.hubb...@gmail.com wrote: >&

Re: [PATCH v4 1/1] mm: introduce put_user_page*(), placeholder versions

2019-03-19 Thread John Hubbard
9:47:24AM -0400, Jerome Glisse wrote: >>>>> On Tue, Mar 19, 2019 at 03:04:17PM +0300, Kirill A. Shutemov wrote: >>>>>> On Fri, Mar 08, 2019 at 01:36:33PM -0800, john.hubb...@gmail.com wrote: >>>>>>> From: John Hubbard >>>>> [...]

Re: [PATCH v4 0/1] mm: introduce put_user_page*(), placeholder versions

2019-03-19 Thread John Hubbard
On 3/19/19 6:09 PM, Christopher Lameter wrote: > On Tue, 19 Mar 2019, John Hubbard wrote: > >>> >>> My concerns do not affect this patchset which just marks the get/put for >>> the pagecache. The problem was that the description was making claims that >

Re: [PATCH v4 1/1] mm: introduce put_user_page*(), placeholder versions

2019-03-19 Thread John Hubbard
0:14:16AM -0400, Jerome Glisse wrote: >>>>> On Tue, Mar 19, 2019 at 09:47:24AM -0400, Jerome Glisse wrote: >>>>>> On Tue, Mar 19, 2019 at 03:04:17PM +0300, Kirill A. Shutemov wrote: >>>>>>> On Fri, Mar 08, 2019 at 01:36:33PM -0800, john.hubb...@gm

[PATCH v5 0/1] mm: introduce put_user_page*(), placeholder versions

2019-03-26 Thread john . hubbard
From: John Hubbard Hi Andrew and all, Can we please apply this for Linux 5.2? I'm adding a Reviewed-by from Christopher Lameter (thanks!), and an updated comment about a race condition (from Kirill's review feedback), and can now make the following statement more confidently: We seem to have

[PATCH v5 1/1] mm: introduce put_user_page*(), placeholder versions

2019-03-26 Thread john . hubbard
From: John Hubbard Introduces put_user_page(), which simply calls put_page(). This provides a way to update all get_user_pages*() callers, so that they call put_user_page(), instead of put_page(). Also introduces put_user_pages(), and a few dirty/locked variations, as a replacement

Re: [PATCH v2 2/4] mm: remove zone_lru_lock() function access ->lru_lock directly

2019-02-28 Thread John Hubbard
ine spinlock_t *zone_lru_lock(struct zone *zone) -{ - return >zone_pgdat->lru_lock; -} - static inline struct lruvec *node_lruvec(struct pglist_data *pgdat) { return >lruvec; Like it? thanks, -- John Hubbard NVIDIA

Re: [PATCH v2 2/4] mm: remove zone_lru_lock() function access ->lru_lock directly

2019-02-28 Thread John Hubbard
On 2/28/19 1:56 PM, Vlastimil Babka wrote: > On 2/28/2019 10:44 PM, John Hubbard wrote: >> Instead of removing that function, let's change it, and add another >> (since you have two cases: either a page* or a pgdat* is available), >> and move it to where it can compile, lik

Re: [PATCH v2 01/11] mm/hmm: select mmu notifier when selecting HMM

2019-03-28 Thread John Hubbard
érôme Glisse > Acked-by: Balbir Singh > Cc: Ralph Campbell > Cc: Andrew Morton > Cc: John Hubbard > Cc: Dan Williams > --- > mm/Kconfig | 1 + > 1 file changed, 1 insertion(+) > > diff --git a/mm/Kconfig b/mm/Kconfig > index 25c71eb8a7db..0d2944278d80 100644 &g

Re: [PATCH v2 02/11] mm/hmm: use reference counting for HMM struct v2

2019-03-28 Thread John Hubbard
ting. >>> >>> Changes since v1: >>> - removed bunch of useless check (if API is use with bogus argument >>> better to fail loudly so user fix their code) >>> - s/hmm_get/mm_get_hmm/ >>> >>> Signed-off-by: Jérôme Glisse >

Re: [PATCH v2 10/11] mm/hmm: add helpers for driver to safely take the mmap_sem v2

2019-03-28 Thread John Hubbard
s since v1: > - removed bunch of useless check (if API is use with bogus argument > better to fail loudly so user fix their code) > > Signed-off-by: Jérôme Glisse > Reviewed-by: Ralph Campbell > Cc: Andrew Morton > Cc: John Hubbard

Re: [PATCH v2 10/11] mm/hmm: add helpers for driver to safely take the mmap_sem v2

2019-03-28 Thread John Hubbard
On 3/28/19 2:30 PM, Jerome Glisse wrote: > On Thu, Mar 28, 2019 at 01:54:01PM -0700, John Hubbard wrote: >> On 3/25/19 7:40 AM, jgli...@redhat.com wrote: >>> From: Jérôme Glisse >>> >>> The device driver context which holds reference to mirror and thus to >

Re: [PATCH v2 07/11] mm/hmm: add default fault flags to avoid the need to pre-fill pfns arrays.

2019-03-28 Thread John Hubbard
; > Signed-off-by: Jérôme Glisse > Reviewed-by: Ralph Campbell > Cc: Andrew Morton > Cc: John Hubbard > Cc: Dan Williams > --- > include/linux/hmm.h | 7 +++ > mm/hmm.c| 12 > 2 files changed, 19 insertions(+) > > diff --gi

Re: [PATCH v2 07/11] mm/hmm: add default fault flags to avoid the need to pre-fill pfns arrays.

2019-03-28 Thread John Hubbard
On 3/28/19 3:12 PM, Jerome Glisse wrote: > On Thu, Mar 28, 2019 at 02:59:50PM -0700, John Hubbard wrote: >> On 3/25/19 7:40 AM, jgli...@redhat.com wrote: >>> From: Jérôme Glisse >>> >>> The HMM mirror API can be use in two fashions. The first one where the

Re: [PATCH v2 10/11] mm/hmm: add helpers for driver to safely take the mmap_sem v2

2019-03-28 Thread John Hubbard
On 3/28/19 3:08 PM, Jerome Glisse wrote: > On Thu, Mar 28, 2019 at 02:41:02PM -0700, John Hubbard wrote: >> On 3/28/19 2:30 PM, Jerome Glisse wrote: >>> On Thu, Mar 28, 2019 at 01:54:01PM -0700, John Hubbard wrote: >>>> On 3/25/19 7:40 AM, jgli...@redhat.com wr

Re: [PATCH v2 07/11] mm/hmm: add default fault flags to avoid the need to pre-fill pfns arrays.

2019-03-28 Thread John Hubbard
On 3/28/19 3:31 PM, Jerome Glisse wrote: > On Thu, Mar 28, 2019 at 03:19:06PM -0700, John Hubbard wrote: >> On 3/28/19 3:12 PM, Jerome Glisse wrote: >>> On Thu, Mar 28, 2019 at 02:59:50PM -0700, John Hubbard wrote: >>>> On 3/25/19 7:40 AM, jgli...@redhat.com wr

Re: [PATCH v2 10/11] mm/hmm: add helpers for driver to safely take the mmap_sem v2

2019-03-28 Thread John Hubbard
On 3/28/19 3:40 PM, Jerome Glisse wrote: > On Thu, Mar 28, 2019 at 03:25:39PM -0700, John Hubbard wrote: >> On 3/28/19 3:08 PM, Jerome Glisse wrote: >>> On Thu, Mar 28, 2019 at 02:41:02PM -0700, John Hubbard wrote: >>>> On 3/28/19 2:30 PM, Jerome Glisse wrote: >&

Re: [PATCH v2 10/11] mm/hmm: add helpers for driver to safely take the mmap_sem v2

2019-03-28 Thread John Hubbard
On 3/28/19 4:05 PM, Jerome Glisse wrote: > On Thu, Mar 28, 2019 at 03:43:33PM -0700, John Hubbard wrote: >> On 3/28/19 3:40 PM, Jerome Glisse wrote: >>> On Thu, Mar 28, 2019 at 03:25:39PM -0700, John Hubbard wrote: >>>> On 3/28/19 3:08 PM, Jerome Glisse wrote: >&

Re: [PATCH v2 07/11] mm/hmm: add default fault flags to avoid the need to pre-fill pfns arrays.

2019-03-28 Thread John Hubbard
On 3/28/19 4:21 PM, Jerome Glisse wrote: > On Thu, Mar 28, 2019 at 03:40:42PM -0700, John Hubbard wrote: >> On 3/28/19 3:31 PM, Jerome Glisse wrote: >>> On Thu, Mar 28, 2019 at 03:19:06PM -0700, John Hubbard wrote: >>>> On 3/28/19 3:12 PM, Jerome Glisse wrote: >&

Re: [PATCH v2 10/11] mm/hmm: add helpers for driver to safely take the mmap_sem v2

2019-03-28 Thread John Hubbard
On 3/28/19 4:24 PM, Jerome Glisse wrote: > On Thu, Mar 28, 2019 at 04:20:37PM -0700, John Hubbard wrote: >> On 3/28/19 4:05 PM, Jerome Glisse wrote: >>> On Thu, Mar 28, 2019 at 03:43:33PM -0700, John Hubbard wrote: >>>> On 3/28/19 3:40 PM, Jerome Glisse wrote: >&

Re: [PATCH v2 02/11] mm/hmm: use reference counting for HMM struct v2

2019-03-28 Thread John Hubbard
On 3/28/19 2:21 PM, Jerome Glisse wrote: > On Thu, Mar 28, 2019 at 01:43:13PM -0700, John Hubbard wrote: >> On 3/28/19 12:11 PM, Jerome Glisse wrote: >>> On Thu, Mar 28, 2019 at 04:07:20AM -0700, Ira Weiny wrote: >>>> On Mon, Mar 25, 2019 at 10:40:02AM -0400, J

Re: [PATCH v2 02/11] mm/hmm: use reference counting for HMM struct v2

2019-03-28 Thread John Hubbard
On 3/28/19 6:00 PM, Jerome Glisse wrote: > On Thu, Mar 28, 2019 at 09:57:09AM -0700, Ira Weiny wrote: >> On Thu, Mar 28, 2019 at 05:39:26PM -0700, John Hubbard wrote: >>> On 3/28/19 2:21 PM, Jerome Glisse wrote: >>>> On Thu, Mar 28, 2019 at 01:43:13PM -0700, John Hubb

Re: [PATCH v2 07/11] mm/hmm: add default fault flags to avoid the need to pre-fill pfns arrays.

2019-03-28 Thread John Hubbard
On 3/28/19 6:17 PM, Jerome Glisse wrote: > On Thu, Mar 28, 2019 at 09:42:31AM -0700, Ira Weiny wrote: >> On Thu, Mar 28, 2019 at 04:28:47PM -0700, John Hubbard wrote: >>> On 3/28/19 4:21 PM, Jerome Glisse wrote: >>>> On Thu, Mar 28, 2019 at 03:40:42PM -0700, John Hub

Re: [PATCH v2 07/11] mm/hmm: add default fault flags to avoid the need to pre-fill pfns arrays.

2019-03-28 Thread John Hubbard
t pone this whole patchset on that new bit. To avoid post-poning >> RDMA and bunch of other patchset that build on top of that i rather get >> this patchset in and then do more changes in the next cycle. >> >> This is just a capacity thing. > > Also for clarity changes to API i am doing in this patchset is to make > the ODP convertion easier and thus they bring a real hard value. Renaming > those function is esthetic, i am not saying it is useless, i am saying it > does not have the same value as those other changes and i would rather not > miss another merge window just for esthetic changes. > Agreed, that this minor point should not hold up this patch. thanks, -- John Hubbard NVIDIA

<    2   3   4   5   6   7   8   9   10   11   >