Re: [PATCH v5 05/15] mm/frame-vector: Use FOLL_LONGTERM

2020-11-09 Thread Jason Gunthorpe
On Mon, Nov 09, 2020 at 09:44:02AM +0100, Thomas Hellström wrote: > static inline bool vma_is_special_huge(const struct vm_area_struct > *vma) > { > return vma_is_dax(vma) || (vma->vm_file && > (vma->vm_flags & (VM_PFNMAP | VM_MIXEDMAP))); > } That is

Re: [PATCH v5 05/15] mm/frame-vector: Use FOLL_LONGTERM

2020-11-09 Thread Thomas Hellström
On Fri, 2020-11-06 at 08:55 -0400, Jason Gunthorpe wrote: > On Fri, Nov 06, 2020 at 11:27:59AM +0100, Daniel Vetter wrote: > > On Fri, Nov 6, 2020 at 11:01 AM Daniel Vetter > > wrote: > > > On Fri, Nov 6, 2020 at 5:08 AM John Hubbard > > > wrote: > > > > On 11/5/20 4:49 AM, Jason Gunthorpe

Re: [PATCH v5 05/15] mm/frame-vector: Use FOLL_LONGTERM

2020-11-06 Thread Jason Gunthorpe
On Fri, Nov 06, 2020 at 11:01:57AM +0100, Daniel Vetter wrote: > gpu drivers also tend to use vmf_insert_pfn* directly, so we can do > on-demand paging and move buffers around. From what I glanced for > lowest level we to the pte_mkspecial correctly (I think I convinced > myself that

Re: [PATCH v5 05/15] mm/frame-vector: Use FOLL_LONGTERM

2020-11-06 Thread Jason Gunthorpe
On Fri, Nov 06, 2020 at 11:27:59AM +0100, Daniel Vetter wrote: > On Fri, Nov 6, 2020 at 11:01 AM Daniel Vetter wrote: > > > > On Fri, Nov 6, 2020 at 5:08 AM John Hubbard wrote: > > > > > > On 11/5/20 4:49 AM, Jason Gunthorpe wrote: > > > > On Thu, Nov 05, 2020 at 10:25:24AM +0100, Daniel Vetter

Re: [PATCH v5 05/15] mm/frame-vector: Use FOLL_LONGTERM

2020-11-06 Thread Daniel Vetter
On Fri, Nov 6, 2020 at 11:01 AM Daniel Vetter wrote: > > On Fri, Nov 6, 2020 at 5:08 AM John Hubbard wrote: > > > > On 11/5/20 4:49 AM, Jason Gunthorpe wrote: > > > On Thu, Nov 05, 2020 at 10:25:24AM +0100, Daniel Vetter wrote: > > >>> /* > > >>> * If we can't determine whether or not a pte is

Re: [PATCH v5 05/15] mm/frame-vector: Use FOLL_LONGTERM

2020-11-06 Thread Daniel Vetter
On Fri, Nov 6, 2020 at 5:08 AM John Hubbard wrote: > > On 11/5/20 4:49 AM, Jason Gunthorpe wrote: > > On Thu, Nov 05, 2020 at 10:25:24AM +0100, Daniel Vetter wrote: > >>> /* > >>> * If we can't determine whether or not a pte is special, then fail > >>> immediately > >>> * for ptes. Note, we

Re: [PATCH v5 05/15] mm/frame-vector: Use FOLL_LONGTERM

2020-11-05 Thread John Hubbard
On 11/5/20 4:49 AM, Jason Gunthorpe wrote: On Thu, Nov 05, 2020 at 10:25:24AM +0100, Daniel Vetter wrote: /* * If we can't determine whether or not a pte is special, then fail immediately * for ptes. Note, we can still pin HugeTLB and THP as these are guaranteed not * to be special. *

Re: [PATCH v5 05/15] mm/frame-vector: Use FOLL_LONGTERM

2020-11-05 Thread Jason Gunthorpe
On Thu, Nov 05, 2020 at 10:25:24AM +0100, Daniel Vetter wrote: > > /* > > * If we can't determine whether or not a pte is special, then fail > > immediately > > * for ptes. Note, we can still pin HugeTLB and THP as these are guaranteed > > not > > * to be special. > > * > > * For a futex to

Re: [PATCH v5 05/15] mm/frame-vector: Use FOLL_LONGTERM

2020-11-05 Thread Daniel Vetter
On Wed, Nov 04, 2020 at 10:44:56AM -0800, John Hubbard wrote: > On 11/4/20 10:17 AM, Jason Gunthorpe wrote: > > On Wed, Nov 04, 2020 at 04:41:19PM +, Christoph Hellwig wrote: > > > On Wed, Nov 04, 2020 at 04:37:58PM +, Christoph Hellwig wrote: > > > > On Wed, Nov 04, 2020 at 05:26:58PM

Re: [PATCH v5 05/15] mm/frame-vector: Use FOLL_LONGTERM

2020-11-04 Thread Christoph Hellwig
On Wed, Nov 04, 2020 at 03:02:14PM -0400, Jason Gunthorpe wrote: > I saw that once and I really couldn't make sense of it.. > What use is having futex's that only work on THP pages? Confused > > CH said there was no case of HAVE_FAST_GUP !ARCH_HAS_PTE_SPECIAL, is > one hidden someplace then?

Re: [PATCH v5 05/15] mm/frame-vector: Use FOLL_LONGTERM

2020-11-04 Thread Jason Gunthorpe
On Wed, Nov 04, 2020 at 10:44:56AM -0800, John Hubbard wrote: > On 11/4/20 10:17 AM, Jason Gunthorpe wrote: > > On Wed, Nov 04, 2020 at 04:41:19PM +, Christoph Hellwig wrote: > > > On Wed, Nov 04, 2020 at 04:37:58PM +, Christoph Hellwig wrote: > > > > On Wed, Nov 04, 2020 at 05:26:58PM

Re: [PATCH v5 05/15] mm/frame-vector: Use FOLL_LONGTERM

2020-11-04 Thread John Hubbard
On 11/4/20 10:17 AM, Jason Gunthorpe wrote: On Wed, Nov 04, 2020 at 04:41:19PM +, Christoph Hellwig wrote: On Wed, Nov 04, 2020 at 04:37:58PM +, Christoph Hellwig wrote: On Wed, Nov 04, 2020 at 05:26:58PM +0100, Daniel Vetter wrote: What we're discussing is whether gup_fast and

Re: [PATCH v5 05/15] mm/frame-vector: Use FOLL_LONGTERM

2020-11-04 Thread Jason Gunthorpe
On Wed, Nov 04, 2020 at 04:41:19PM +, Christoph Hellwig wrote: > On Wed, Nov 04, 2020 at 04:37:58PM +, Christoph Hellwig wrote: > > On Wed, Nov 04, 2020 at 05:26:58PM +0100, Daniel Vetter wrote: > > > What we're discussing is whether gup_fast and pup_fast also obey this, > > > or fall over

Re: [PATCH v5 05/15] mm/frame-vector: Use FOLL_LONGTERM

2020-11-04 Thread Christoph Hellwig
On Wed, Nov 04, 2020 at 04:37:58PM +, Christoph Hellwig wrote: > On Wed, Nov 04, 2020 at 05:26:58PM +0100, Daniel Vetter wrote: > > What we're discussing is whether gup_fast and pup_fast also obey this, > > or fall over and can give you the struct page that's backing the > > dma_mmap_* memory.

Re: [PATCH v5 05/15] mm/frame-vector: Use FOLL_LONGTERM

2020-11-04 Thread Christoph Hellwig
On Wed, Nov 04, 2020 at 05:26:58PM +0100, Daniel Vetter wrote: > What we're discussing is whether gup_fast and pup_fast also obey this, > or fall over and can give you the struct page that's backing the > dma_mmap_* memory. Since the _fast variant doesn't check for > vma->vm_flags, and afaict

Re: [PATCH v5 05/15] mm/frame-vector: Use FOLL_LONGTERM

2020-11-04 Thread Daniel Vetter
On Wed, Nov 4, 2020 at 5:21 PM Christoph Hellwig wrote: > > On Wed, Nov 04, 2020 at 04:54:19PM +0100, Daniel Vetter wrote: > > I don't really have a box here, but dma_mmap_attrs() and friends to > > mmap dma_alloc_coherent memory is set up as VM_IO | VM_PFNMAP (it's > > actually enforced since

Re: [PATCH v5 05/15] mm/frame-vector: Use FOLL_LONGTERM

2020-11-04 Thread Christoph Hellwig
On Wed, Nov 04, 2020 at 04:54:19PM +0100, Daniel Vetter wrote: > I don't really have a box here, but dma_mmap_attrs() and friends to > mmap dma_alloc_coherent memory is set up as VM_IO | VM_PFNMAP (it's > actually enforced since underneath it uses remap_pfn_range), and > usually (except if it's

Re: [PATCH v5 05/15] mm/frame-vector: Use FOLL_LONGTERM

2020-11-04 Thread Daniel Vetter
On Wed, Nov 4, 2020 at 3:00 PM Jason Gunthorpe wrote: > > On Sun, Nov 01, 2020 at 11:50:39PM +0100, Daniel Vetter wrote: > > > It's not device drivers, but everyone else. At least my understanding > > is that VM_IO | VM_PFNMAP means "even if it happens to be backed by a > > struct page, do not

Re: [PATCH v5 05/15] mm/frame-vector: Use FOLL_LONGTERM

2020-11-04 Thread Jason Gunthorpe
On Sun, Nov 01, 2020 at 11:50:39PM +0100, Daniel Vetter wrote: > It's not device drivers, but everyone else. At least my understanding > is that VM_IO | VM_PFNMAP means "even if it happens to be backed by a > struct page, do not treat it like normal memory". And gup/pup_fast > happily break that.

Re: [PATCH v5 05/15] mm/frame-vector: Use FOLL_LONGTERM

2020-11-02 Thread Tomasz Figa
On Fri, Oct 30, 2020 at 3:38 PM Daniel Vetter wrote: > > On Fri, Oct 30, 2020 at 3:11 PM Tomasz Figa wrote: > > > > On Fri, Oct 30, 2020 at 11:08 AM Daniel Vetter > > wrote: > > > > > > This is used by media/videbuf2 for persistent dma mappings, not just > > > for a single dma operation and

Re: [PATCH v5 05/15] mm/frame-vector: Use FOLL_LONGTERM

2020-11-01 Thread Daniel Vetter
On Sun, Nov 1, 2020 at 10:13 PM John Hubbard wrote: > > On 11/1/20 2:30 AM, Daniel Vetter wrote: > > On Sun, Nov 1, 2020 at 6:22 AM John Hubbard wrote: > >> > >> On 10/31/20 7:45 AM, Daniel Vetter wrote: > >>> On Sat, Oct 31, 2020 at 3:55 AM John Hubbard wrote: > On 10/30/20 3:08 AM,

Re: [PATCH v5 05/15] mm/frame-vector: Use FOLL_LONGTERM

2020-11-01 Thread John Hubbard
On 11/1/20 2:30 AM, Daniel Vetter wrote: On Sun, Nov 1, 2020 at 6:22 AM John Hubbard wrote: On 10/31/20 7:45 AM, Daniel Vetter wrote: On Sat, Oct 31, 2020 at 3:55 AM John Hubbard wrote: On 10/30/20 3:08 AM, Daniel Vetter wrote: ... By removing this check from this location, and changing

Re: [PATCH v5 05/15] mm/frame-vector: Use FOLL_LONGTERM

2020-11-01 Thread Daniel Vetter
On Sun, Nov 1, 2020 at 6:22 AM John Hubbard wrote: > > On 10/31/20 7:45 AM, Daniel Vetter wrote: > > On Sat, Oct 31, 2020 at 3:55 AM John Hubbard wrote: > >> On 10/30/20 3:08 AM, Daniel Vetter wrote: > ... > >> By removing this check from this location, and changing from > >>

Re: [PATCH v5 05/15] mm/frame-vector: Use FOLL_LONGTERM

2020-10-31 Thread John Hubbard
On 10/31/20 7:45 AM, Daniel Vetter wrote: On Sat, Oct 31, 2020 at 3:55 AM John Hubbard wrote: On 10/30/20 3:08 AM, Daniel Vetter wrote: ... By removing this check from this location, and changing from pin_user_pages_locked() to pin_user_pages_fast(), I *think* we end up losing the check

Re: [PATCH v5 05/15] mm/frame-vector: Use FOLL_LONGTERM

2020-10-31 Thread Daniel Vetter
On Sat, Oct 31, 2020 at 3:55 AM John Hubbard wrote: > > On 10/30/20 3:08 AM, Daniel Vetter wrote: > > This is used by media/videbuf2 for persistent dma mappings, not just > > for a single dma operation and then freed again, so needs > > FOLL_LONGTERM. > > > > Unfortunately current pup_locked

Re: [PATCH v5 05/15] mm/frame-vector: Use FOLL_LONGTERM

2020-10-30 Thread John Hubbard
On 10/30/20 3:08 AM, Daniel Vetter wrote: This is used by media/videbuf2 for persistent dma mappings, not just for a single dma operation and then freed again, so needs FOLL_LONGTERM. Unfortunately current pup_locked doesn't support FOLL_LONGTERM due to locking issues. Rework the code to pull

Re: [PATCH v5 05/15] mm/frame-vector: Use FOLL_LONGTERM

2020-10-30 Thread Daniel Vetter
On Fri, Oct 30, 2020 at 3:11 PM Tomasz Figa wrote: > > On Fri, Oct 30, 2020 at 11:08 AM Daniel Vetter wrote: > > > > This is used by media/videbuf2 for persistent dma mappings, not just > > for a single dma operation and then freed again, so needs > > FOLL_LONGTERM. > > > > Unfortunately current

Re: [PATCH v5 05/15] mm/frame-vector: Use FOLL_LONGTERM

2020-10-30 Thread Tomasz Figa
On Fri, Oct 30, 2020 at 11:08 AM Daniel Vetter wrote: > > This is used by media/videbuf2 for persistent dma mappings, not just > for a single dma operation and then freed again, so needs > FOLL_LONGTERM. > > Unfortunately current pup_locked doesn't support FOLL_LONGTERM due to > locking issues.

[PATCH v5 05/15] mm/frame-vector: Use FOLL_LONGTERM

2020-10-30 Thread Daniel Vetter
This is used by media/videbuf2 for persistent dma mappings, not just for a single dma operation and then freed again, so needs FOLL_LONGTERM. Unfortunately current pup_locked doesn't support FOLL_LONGTERM due to locking issues. Rework the code to pull the pup path out from the mmap_sem critical