[Cluster-devel] [PATCH v7 03/24] mm: Ignore return value of ->readpages

2020-02-19 Thread Matthew Wilcox
From: "Matthew Wilcox (Oracle)" We used to assign the return value to a variable, which we then ignored. Remove the pretence of caring. Signed-off-by: Matthew Wilcox (Oracle) Reviewed-by: Christoph Hellwig Reviewed-by: Dave Chinner Reviewed-by: John Hubbard --- mm/reada

[Cluster-devel] [PATCH v7 05/24] mm: Use readahead_control to pass arguments

2020-02-19 Thread Matthew Wilcox
From: "Matthew Wilcox (Oracle)" In this patch, only between __do_page_cache_readahead() and read_pages(), but it will be extended in upcoming patches. Also add the readahead_count() accessor. The read_pages() function becomes aops centric, as this makes the most sense

[Cluster-devel] [PATCH v7 12/24] mm: Add page_cache_readahead_unbounded

2020-02-19 Thread Matthew Wilcox
From: "Matthew Wilcox (Oracle)" ext4 and f2fs have duplicated the guts of the readahead code so they can read past i_size. Instead, separate out the guts of the readahead code so they can call it directly. Signed-off-by: Matthew Wilcox (Oracle) --- fs/ext4/verity.c

[Cluster-devel] [PATCH v7 02/24] mm: Return void from various readahead functions

2020-02-19 Thread Matthew Wilcox
From: "Matthew Wilcox (Oracle)" ondemand_readahead has two callers, neither of which use the return value. That means that both ra_submit and __do_page_cache_readahead() can return void, and we don't need to worry that a present page in the readahead window causes us to return a smalle

[Cluster-devel] [PATCH v7 17/24] ext4: Convert from readpages to readahead

2020-02-19 Thread Matthew Wilcox
From: "Matthew Wilcox (Oracle)" Use the new readahead operation in ext4 Signed-off-by: Matthew Wilcox (Oracle) --- fs/ext4/ext4.h | 3 +-- fs/ext4/inode.c| 21 + fs/ext4/readpage.c | 22 -- 3 files changed, 18 insertions(+), 28

[Cluster-devel] [PATCH v7 11/24] mm: Move end_index check out of readahead loop

2020-02-19 Thread Matthew Wilcox
From: "Matthew Wilcox (Oracle)" By reducing nr_to_read, we can eliminate this check from inside the loop. Signed-off-by: Matthew Wilcox (Oracle) --- mm/readahead.c | 15 +-- 1 file changed, 9 insertions(+), 6 deletions(-) diff --git a/mm/readahead.c b/mm/readahe

[Cluster-devel] [PATCH v7 08/24] mm: Remove 'page_offset' from readahead loop

2020-02-19 Thread Matthew Wilcox
From: "Matthew Wilcox (Oracle)" Replace the page_offset variable with 'index + i'. Signed-off-by: Matthew Wilcox (Oracle) --- mm/readahead.c | 8 +++- 1 file changed, 3 insertions(+), 5 deletions(-) diff --git a/mm/readahead.c b/mm/readahead.c index 8a25fc7e2bf2..83df5c061

[Cluster-devel] [PATCH v7 15/24] erofs: Convert uncompressed files from readpages to readahead

2020-02-19 Thread Matthew Wilcox
From: "Matthew Wilcox (Oracle)" Use the new readahead operation in erofs Signed-off-by: Matthew Wilcox (Oracle) Acked-by: Gao Xiang --- fs/erofs/data.c | 39 +--- fs/erofs/zdata.c | 2 +- include/trace/events/erofs.h | 6

[Cluster-devel] [PATCH v7 24/24] mm: Use memalloc_nofs_save in readahead path

2020-02-19 Thread Matthew Wilcox
From: "Matthew Wilcox (Oracle)" Ensure that memory allocations in the readahead path do not attempt to reclaim file-backed pages, which could lead to a deadlock. It is possible, though unlikely this is the root cause of a problem observed by Cong Wang. Signed-off-by: Matthew Wilc

[Cluster-devel] [PATCH v7 09/24] mm: Put readahead pages in cache earlier

2020-02-19 Thread Matthew Wilcox
From: "Matthew Wilcox (Oracle)" When populating the page cache for readahead, mappings that use ->readpages must populate the page cache themselves as the pages are passed on a linked list which would normally be used for the page cache's LRU. For mappings that use ->readpage

[Cluster-devel] [PATCH v7 16/24] erofs: Convert compressed files from readpages to readahead

2020-02-19 Thread Matthew Wilcox
From: "Matthew Wilcox (Oracle)" Use the new readahead operation in erofs. Signed-off-by: Matthew Wilcox (Oracle) Acked-by: Gao Xiang Reviewed-by: Dave Chinner --- fs/erofs/zdata.c | 29 + 1 file changed, 9 insertions(+), 20 deletions(-) diff --git

[Cluster-devel] [PATCH v7 23/24] mm: Document why we don't set PageReadahead

2020-02-19 Thread Matthew Wilcox
From: "Matthew Wilcox (Oracle)" If the page is already in cache, we don't set PageReadahead on it. Signed-off-by: Matthew Wilcox (Oracle) --- mm/readahead.c | 9 ++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/mm/readahead.c b/mm/readahead.c index 45

[Cluster-devel] [PATCH v7 07/24] mm: rename readahead loop variable to 'i'

2020-02-19 Thread Matthew Wilcox
From: "Matthew Wilcox (Oracle)" Change the type of page_idx to unsigned long, and rename it -- it's just a loop counter, not a page index. Suggested-by: John Hubbard Signed-off-by: Matthew Wilcox (Oracle) Reviewed-by: Dave Chinner --- mm/readahead.c | 8 1 file

Re: [Cluster-devel] [PATCH v6 17/19] iomap: Restructure iomap_readpages_actor

2020-02-19 Thread Matthew Wilcox
On Wed, Feb 19, 2020 at 05:40:05PM +1100, Dave Chinner wrote: > Ok, that's what the ctx.cur_page_in_bio check is used to detect i.e. > if we've got a page that the readahead cursor points at, and we > haven't actually added it to a bio, then we can leave it to the > read_pages() to unlock and

Re: [Cluster-devel] [PATCH v6 08/19] mm: Add readahead address space operation

2020-02-19 Thread Matthew Wilcox
On Tue, Feb 18, 2020 at 07:10:44PM -0800, Eric Biggers wrote: > > +``readahead`` > > + Called by the VM to read pages associated with the address_space > > + object. The pages are consecutive in the page cache and are > > + locked. The implementation should decrement the page refcount > >

Re: [Cluster-devel] [PATCH v6 07/19] mm: Put readahead pages in cache earlier

2020-02-19 Thread Matthew Wilcox
On Wed, Feb 19, 2020 at 06:52:46AM -0800, Christoph Hellwig wrote: > On Wed, Feb 19, 2020 at 06:41:17AM -0800, Matthew Wilcox wrote: > > #define readahead_for_each(rac, page) \ > > while ((page = readahead_page(rac))) > > >

Re: [Cluster-devel] [PATCH v6 07/19] mm: Put readahead pages in cache earlier

2020-02-19 Thread Matthew Wilcox
On Tue, Feb 18, 2020 at 04:01:43PM -0800, John Hubbard wrote: > How about this instead? It uses the "for" loop fully and more naturally, > and is easier to read. And it does the same thing: > > static inline struct page *readahead_page(struct readahead_control *rac) > { > struct page *page;

Re: [Cluster-devel] [PATCH v6 17/19] iomap: Restructure iomap_readpages_actor

2020-02-18 Thread Matthew Wilcox
On Wed, Feb 19, 2020 at 02:29:00PM +1100, Dave Chinner wrote: > On Mon, Feb 17, 2020 at 10:46:11AM -0800, Matthew Wilcox wrote: > > @@ -418,6 +412,15 @@ iomap_readpages_actor(struct inode *inode, loff_t pos, > > loff_t length, > > } > > ret =

Re: [Cluster-devel] [PATCH v6 17/19] iomap: Restructure iomap_readpages_actor

2020-02-18 Thread Matthew Wilcox
On Tue, Feb 18, 2020 at 07:17:18PM -0800, John Hubbard wrote: > > - for (done = 0; done < length; done += ret) { > > nit: this "for" loop was perfect just the way it was. :) I'd vote here for > reverting > the change to a "while" loop. Because with this change, now the code has to >

Re: [Cluster-devel] [PATCH v6 19/19] mm: Use memalloc_nofs_save in readahead path

2020-02-18 Thread Matthew Wilcox
On Wed, Feb 19, 2020 at 02:43:24PM +1100, Dave Chinner wrote: > On Mon, Feb 17, 2020 at 10:46:13AM -0800, Matthew Wilcox wrote: > > From: "Matthew Wilcox (Oracle)" > > > > Ensure that memory allocations in the readahead path do not attempt to > > reclai

Re: [Cluster-devel] [PATCH v6 00/19] Change readahead API

2020-02-18 Thread Matthew Wilcox
On Wed, Feb 19, 2020 at 02:45:25PM +1100, Dave Chinner wrote: > On Wed, Feb 19, 2020 at 08:26:52AM +1100, Dave Chinner wrote: > > On Tue, Feb 18, 2020 at 05:42:30AM -0800, Matthew Wilcox wrote: > > > On Tue, Feb 18, 2020 at 03:56:33PM +1100, Dave Chinner wrote: > > > &

Re: [Cluster-devel] [PATCH v6 10/19] fs: Convert mpage_readpages to mpage_readahead

2020-02-18 Thread Matthew Wilcox
On Tue, Feb 18, 2020 at 07:28:26PM -0800, Eric Biggers wrote: > On Mon, Feb 17, 2020 at 10:45:58AM -0800, Matthew Wilcox wrote: > > diff --git a/include/linux/mpage.h b/include/linux/mpage.h > > index 001f1fcf9836..f4f5e90a6844 100644 > > --- a/include/linux/mpage.h > >

Re: [Cluster-devel] [PATCH v6 09/19] mm: Add page_cache_readahead_limit

2020-02-18 Thread Matthew Wilcox
On Tue, Feb 18, 2020 at 05:32:31PM -0800, John Hubbard wrote: > > + page_cache_readahead_limit(inode->i_mapping, NULL, > > + index, LONG_MAX, num_ra_pages, 0); > > > LONG_MAX seems bold at first, but then again I can't think of anything >

Re: [Cluster-devel] [PATCH v6 07/19] mm: Put readahead pages in cache earlier

2020-02-18 Thread Matthew Wilcox
On Tue, Feb 18, 2020 at 04:01:43PM -0800, John Hubbard wrote: > How about this instead? It uses the "for" loop fully and more naturally, > and is easier to read. And it does the same thing: > > static inline struct page *readahead_page(struct readahead_control *rac) > { > struct page *page;

Re: [Cluster-devel] [PATCH v6 03/19] mm: Use readahead_control to pass arguments

2020-02-18 Thread Matthew Wilcox
On Wed, Feb 19, 2020 at 09:46:10AM +1100, Dave Chinner wrote: > On Tue, Feb 18, 2020 at 05:56:18AM -0800, Matthew Wilcox wrote: > > On Tue, Feb 18, 2020 at 04:03:00PM +1100, Dave Chinner wrote: > > > On Mon, Feb 17, 2020 at 10:45:44AM -0800, Matthew Wilcox wrote: > > >

Re: [Cluster-devel] [PATCH v6 01/19] mm: Return void from various readahead functions

2020-02-18 Thread Matthew Wilcox
On Tue, Feb 18, 2020 at 01:05:29PM -0800, John Hubbard wrote: > This is an easy review and obviously correct, so: > > Reviewed-by: John Hubbard Thanks > Thoughts for the future of the API: > > I will add that I could envision another patchset that went in the > opposite direction, and

Re: [Cluster-devel] [PATCH v6 11/19] btrfs: Convert from readpages to readahead

2020-02-18 Thread Matthew Wilcox
On Tue, Feb 18, 2020 at 05:57:58PM +1100, Dave Chinner wrote: > On Mon, Feb 17, 2020 at 10:45:59AM -0800, Matthew Wilcox wrote: > > From: "Matthew Wilcox (Oracle)" > > > > Use the new readahead operation in btrfs. Add a > > readahead_for_each_batch() iterat

Re: [Cluster-devel] [PATCH v6 09/19] mm: Add page_cache_readahead_limit

2020-02-18 Thread Matthew Wilcox
On Tue, Feb 18, 2020 at 05:31:10PM +1100, Dave Chinner wrote: > On Mon, Feb 17, 2020 at 10:45:56AM -0800, Matthew Wilcox wrote: > > From: "Matthew Wilcox (Oracle)" > > > > ext4 and f2fs have duplicated the guts of the readahead code so > > they can rea

Re: [Cluster-devel] [PATCH v6 08/19] mm: Add readahead address space operation

2020-02-18 Thread Matthew Wilcox
On Tue, Feb 18, 2020 at 05:21:47PM +1100, Dave Chinner wrote: > On Mon, Feb 17, 2020 at 10:45:54AM -0800, Matthew Wilcox wrote: > > From: "Matthew Wilcox (Oracle)" > > > > This replaces ->readpages with a saner interface: > > - Return void instea

Re: [Cluster-devel] [PATCH v6 07/19] mm: Put readahead pages in cache earlier

2020-02-18 Thread Matthew Wilcox
On Tue, Feb 18, 2020 at 05:14:59PM +1100, Dave Chinner wrote: > On Mon, Feb 17, 2020 at 10:45:52AM -0800, Matthew Wilcox wrote: > > From: "Matthew Wilcox (Oracle)" > > > > At allocation time, put the pages in the cache unless we're using > > ->readpages

Re: [Cluster-devel] [PATCH v6 04/19] mm: Rearrange readahead loop

2020-02-18 Thread Matthew Wilcox
On Tue, Feb 18, 2020 at 04:08:24PM +1100, Dave Chinner wrote: > On Mon, Feb 17, 2020 at 10:45:45AM -0800, Matthew Wilcox wrote: > > From: "Matthew Wilcox (Oracle)" > > > > Move the declaration of 'page' to inside the loop and move the 'kick > > off a fresh b

Re: [Cluster-devel] [PATCH v6 03/19] mm: Use readahead_control to pass arguments

2020-02-18 Thread Matthew Wilcox
On Tue, Feb 18, 2020 at 04:03:00PM +1100, Dave Chinner wrote: > On Mon, Feb 17, 2020 at 10:45:44AM -0800, Matthew Wilcox wrote: > > +static void read_pages(struct readahead_control *rac, struct list_head > > *pages, > > + gfp_t gfp) > > { > > + cons

Re: [Cluster-devel] [PATCH v6 00/19] Change readahead API

2020-02-18 Thread Matthew Wilcox
On Tue, Feb 18, 2020 at 03:56:33PM +1100, Dave Chinner wrote: > Latest version in your git tree: > > $ ▶ glo -n 5 willy/readahead > 4be497096c04 mm: Use memalloc_nofs_save in readahead path > ff63497fcb98 iomap: Convert from readpages to readahead > 26aee60e89b5 iomap: Restructure

Re: [Cluster-devel] [PATCH 00/44] Manually convert filesystem FS documents to ReST

2020-02-17 Thread Matthew Wilcox
On Mon, Feb 17, 2020 at 05:11:46PM +0100, Mauro Carvalho Chehab wrote: > There are lots of plain text documents under Documentation/filesystems. > > Manually convert several of those to ReST and add them to the index file. I think at least 90% of this material needs to be in admin-guide. Should

Re: [Cluster-devel] [PATCH v6 00/19] Change readahead API

2020-02-17 Thread Matthew Wilcox
On Mon, Feb 17, 2020 at 10:45:41AM -0800, Matthew Wilcox wrote: > This series adds a readahead address_space operation to eventually *sigh*. Clearly I forgot to rm -rf an earlier version. Please disregard any patches labelled n/16. I can send a v7 if this is too much hassle.

[Cluster-devel] [PATCH v6 01/19] mm: Return void from various readahead functions

2020-02-17 Thread Matthew Wilcox
From: "Matthew Wilcox (Oracle)" ondemand_readahead has two callers, neither of which use the return value. That means that both ra_submit and __do_page_cache_readahead() can return void, and we don't need to worry that a present page in the readahead window causes us to return a smalle

[Cluster-devel] [PATCH v6 04/19] mm: Rearrange readahead loop

2020-02-17 Thread Matthew Wilcox
From: "Matthew Wilcox (Oracle)" Move the declaration of 'page' to inside the loop and move the 'kick off a fresh batch' code to the end of the function for easier use in subsequent patches. Signed-off-by: Matthew Wilcox (Oracle) --- mm/readahead.c | 21 + 1 fi

[Cluster-devel] [PATCH v6 16/16] mm: Use memalloc_nofs_save in readahead path

2020-02-17 Thread Matthew Wilcox
From: "Matthew Wilcox (Oracle)" Ensure that memory allocations in the readahead path do not attempt to reclaim file-backed pages, which could lead to a deadlock. It is possible, though unlikely this is the root cause of a problem observed by Cong Wang. Signed-off-by: Matthew Wilc

[Cluster-devel] [PATCH v6 16/19] fuse: Convert from readpages to readahead

2020-02-17 Thread Matthew Wilcox
From: "Matthew Wilcox (Oracle)" Use the new readahead operation in fuse. Switching away from the read_cache_pages() helper gets rid of an implicit call to put_page(), so we can get rid of the get_page() call in fuse_readpages_fill(). Signed-off-by: Matthew Wilcox (Oracle) --- fs/f

[Cluster-devel] [PATCH v6 12/19] erofs: Convert uncompressed files from readpages to readahead

2020-02-17 Thread Matthew Wilcox
From: "Matthew Wilcox (Oracle)" Use the new readahead operation in erofs Signed-off-by: Matthew Wilcox (Oracle) --- fs/erofs/data.c | 39 +--- fs/erofs/zdata.c | 2 +- include/trace/events/erofs.h | 6 +++--- 3 files c

[Cluster-devel] [PATCH v6 10/16] erofs: Convert uncompressed files from readpages to readahead

2020-02-17 Thread Matthew Wilcox
From: "Matthew Wilcox (Oracle)" Use the new readahead operation in erofs Signed-off-by: Matthew Wilcox (Oracle) --- fs/erofs/data.c | 39 +--- fs/erofs/zdata.c | 2 +- include/trace/events/erofs.h | 6 +++--- 3 files c

[Cluster-devel] [PATCH v6 11/19] btrfs: Convert from readpages to readahead

2020-02-17 Thread Matthew Wilcox
From: "Matthew Wilcox (Oracle)" Use the new readahead operation in btrfs. Add a readahead_for_each_batch() iterator to optimise the loop in the XArray. Signed-off-by: Matthew Wilcox (Oracle) --- fs/btrfs/extent_io.c| 46 + fs/btrfs/e

[Cluster-devel] [PATCH v6 00/19] Change readahead API

2020-02-17 Thread Matthew Wilcox
From: "Matthew Wilcox (Oracle)" This series adds a readahead address_space operation to eventually replace the readpages operation. The key difference is that pages are added to the page cache as they are allocated (and then looked up by the filesystem) instead of passing them

[Cluster-devel] [PATCH v6 04/16] mm: Tweak readahead loop slightly

2020-02-17 Thread Matthew Wilcox
From: "Matthew Wilcox (Oracle)" Eliminate the page_offset variable which was just confusing; record the start of each consecutive run of pages in the readahead_control, and move the 'kick off a fresh batch' code to the end of the function for easier use in the next patch. Signed-off-b

[Cluster-devel] [PATCH v6 08/16] fs: Convert mpage_readpages to mpage_readahead

2020-02-17 Thread Matthew Wilcox
From: "Matthew Wilcox (Oracle)" Implement the new readahead aop and convert all callers (block_dev, exfat, ext2, fat, gfs2, hpfs, isofs, jfs, nilfs2, ocfs2, omfs, qnx6, reiserfs & udf). The callers are all trivial except for GFS2 & OCFS2. Signed-off-by: Matthew Wilcox

[Cluster-devel] [PATCH v6 06/16] mm: Add readahead address space operation

2020-02-17 Thread Matthew Wilcox
From: "Matthew Wilcox (Oracle)" This replaces ->readpages with a saner interface: - Return void instead of an ignored error code. - Pages are already in the page cache when ->readahead is called. - Implementation looks up the pages in the page cache instead of ha

[Cluster-devel] [PATCH v6 17/19] iomap: Restructure iomap_readpages_actor

2020-02-17 Thread Matthew Wilcox
From: "Matthew Wilcox (Oracle)" By putting the 'have we reached the end of the page' condition at the end of the loop instead of the beginning, we can remove the 'submit the last page' code from iomap_readpages(). Also check that iomap_readpage_actor() didn't return 0, which

[Cluster-devel] [PATCH v6 07/19] mm: Put readahead pages in cache earlier

2020-02-17 Thread Matthew Wilcox
From: "Matthew Wilcox (Oracle)" At allocation time, put the pages in the cache unless we're using ->readpages. Add the readahead_for_each() iterator for the benefit of the ->readpage fallback. This iterator supports huge pages, even though none of the filesystems to be

[Cluster-devel] [PATCH v6 06/19] mm: rename readahead loop variable to 'i'

2020-02-17 Thread Matthew Wilcox
From: "Matthew Wilcox (Oracle)" Change the type of page_idx to unsigned long, and rename it -- it's just a loop counter, not a page index. Suggested-by: John Hubbard Signed-off-by: Matthew Wilcox (Oracle) --- mm/readahead.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletion

[Cluster-devel] [PATCH v6 12/16] ext4: Convert from readpages to readahead

2020-02-17 Thread Matthew Wilcox
From: "Matthew Wilcox (Oracle)" Use the new readahead operation in ext4 Signed-off-by: Matthew Wilcox (Oracle) --- fs/ext4/ext4.h | 3 +-- fs/ext4/inode.c| 23 ++- fs/ext4/readpage.c | 22 -- 3 files changed, 19 insertions(+), 29

[Cluster-devel] [PATCH v6 05/19] mm: Remove 'page_offset' from readahead loop

2020-02-17 Thread Matthew Wilcox
From: "Matthew Wilcox (Oracle)" Eliminate the page_offset variable which was confusing with the 'offset' parameter and record the start of each consecutive run of pages in the readahead_control. Signed-off-by: Matthew Wilcox (Oracle) --- mm/readahead.c | 10 ++ 1 file

[Cluster-devel] [PATCH v6 02/19] mm: Ignore return value of ->readpages

2020-02-17 Thread Matthew Wilcox
From: "Matthew Wilcox (Oracle)" We used to assign the return value to a variable, which we then ignored. Remove the pretence of caring. Signed-off-by: Matthew Wilcox (Oracle) Reviewed-by: Christoph Hellwig --- mm/readahead.c | 8 ++-- 1 file changed, 2 insertions(+), 6 deletion

[Cluster-devel] [PATCH v6 09/19] mm: Add page_cache_readahead_limit

2020-02-17 Thread Matthew Wilcox
From: "Matthew Wilcox (Oracle)" ext4 and f2fs have duplicated the guts of the readahead code so they can read past i_size. Instead, separate out the guts of the readahead code so they can call it directly. Signed-off-by: Matthew Wilcox (Oracle) --- fs/ext4/verity.c

[Cluster-devel] [PATCH v6 10/19] fs: Convert mpage_readpages to mpage_readahead

2020-02-17 Thread Matthew Wilcox
From: "Matthew Wilcox (Oracle)" Implement the new readahead aop and convert all callers (block_dev, exfat, ext2, fat, gfs2, hpfs, isofs, jfs, nilfs2, ocfs2, omfs, qnx6, reiserfs & udf). The callers are all trivial except for GFS2 & OCFS2. Signed-off-by: Matthew Wilcox

[Cluster-devel] [PATCH v6 05/16] mm: Put readahead pages in cache earlier

2020-02-17 Thread Matthew Wilcox
From: "Matthew Wilcox (Oracle)" At allocation time, put the pages in the cache unless we're using ->readpages. Add the readahead_for_each() iterator for the benefit of the ->readpage fallback. This iterator supports huge pages, even though none of the filesystems to be

[Cluster-devel] [PATCH v6 11/16] erofs: Convert compressed files from readpages to readahead

2020-02-17 Thread Matthew Wilcox
From: "Matthew Wilcox (Oracle)" Use the new readahead operation in erofs. Signed-off-by: Matthew Wilcox (Oracle) --- fs/erofs/zdata.c | 29 + 1 file changed, 9 insertions(+), 20 deletions(-) diff --git a/fs/erofs/zdata.c b/fs/erofs/zdata.c index 17

[Cluster-devel] [PATCH v6 08/19] mm: Add readahead address space operation

2020-02-17 Thread Matthew Wilcox
From: "Matthew Wilcox (Oracle)" This replaces ->readpages with a saner interface: - Return void instead of an ignored error code. - Pages are already in the page cache when ->readahead is called. - Implementation looks up the pages in the page cache instead of ha

[Cluster-devel] [PATCH v6 15/19] f2fs: Convert from readpages to readahead

2020-02-17 Thread Matthew Wilcox
From: "Matthew Wilcox (Oracle)" Use the new readahead operation in f2fs Signed-off-by: Matthew Wilcox (Oracle) --- fs/f2fs/data.c | 50 +++-- fs/f2fs/f2fs.h | 5 ++-- include/trace/events/f2fs.h | 6 ++--- 3 files c

[Cluster-devel] [PATCH v6 19/19] mm: Use memalloc_nofs_save in readahead path

2020-02-17 Thread Matthew Wilcox
From: "Matthew Wilcox (Oracle)" Ensure that memory allocations in the readahead path do not attempt to reclaim file-backed pages, which could lead to a deadlock. It is possible, though unlikely this is the root cause of a problem observed by Cong Wang. Signed-off-by: Matthew Wilc

[Cluster-devel] [PATCH v6 09/16] btrfs: Convert from readpages to readahead

2020-02-17 Thread Matthew Wilcox
From: "Matthew Wilcox (Oracle)" Use the new readahead operation in btrfs. Add a readahead_for_each_batch() iterator to optimise the loop in the XArray. Signed-off-by: Matthew Wilcox (Oracle) --- fs/btrfs/extent_io.c| 48 ++--- fs/btrfs/e

[Cluster-devel] [PATCH v6 14/19] ext4: Convert from readpages to readahead

2020-02-17 Thread Matthew Wilcox
From: "Matthew Wilcox (Oracle)" Use the new readahead operation in ext4 Signed-off-by: Matthew Wilcox (Oracle) --- fs/ext4/ext4.h | 3 +-- fs/ext4/inode.c| 23 ++- fs/ext4/readpage.c | 22 -- 3 files changed, 19 insertions(+), 29

[Cluster-devel] [PATCH v6 13/16] f2fs: Convert from readpages to readahead

2020-02-17 Thread Matthew Wilcox
From: "Matthew Wilcox (Oracle)" Use the new readahead operation in f2fs Signed-off-by: Matthew Wilcox (Oracle) --- fs/f2fs/data.c | 50 +++-- fs/f2fs/f2fs.h | 5 ++-- include/trace/events/f2fs.h | 6 ++--- 3 files c

[Cluster-devel] [PATCH v6 07/16] mm: Add page_cache_readahead_limit

2020-02-17 Thread Matthew Wilcox
From: "Matthew Wilcox (Oracle)" ext4 and f2fs have duplicated the guts of the readahead code so they can read past i_size. Instead, separate out the guts of the readahead code so they can call it directly. Signed-off-by: Matthew Wilcox (Oracle) --- fs/ext4/verity.c

[Cluster-devel] [PATCH v6 14/16] fuse: Convert from readpages to readahead

2020-02-17 Thread Matthew Wilcox
From: "Matthew Wilcox (Oracle)" Use the new readahead operation in fuse. Switching away from the read_cache_pages() helper gets rid of an implicit call to put_page(), so we can get rid of the get_page() call in fuse_readpages_fill(). Signed-off-by: Matthew Wilcox (Oracle) --- fs/f

[Cluster-devel] [PATCH v6 13/19] erofs: Convert compressed files from readpages to readahead

2020-02-17 Thread Matthew Wilcox
From: "Matthew Wilcox (Oracle)" Use the new readahead operation in erofs. Signed-off-by: Matthew Wilcox (Oracle) --- fs/erofs/zdata.c | 29 + 1 file changed, 9 insertions(+), 20 deletions(-) diff --git a/fs/erofs/zdata.c b/fs/erofs/zdata.c index 17

[Cluster-devel] [PATCH v6 18/19] iomap: Convert from readpages to readahead

2020-02-17 Thread Matthew Wilcox
From: "Matthew Wilcox (Oracle)" Use the new readahead operation in iomap. Convert XFS and ZoneFS to use it. Signed-off-by: Matthew Wilcox (Oracle) --- fs/iomap/buffered-io.c | 91 +++--- fs/iomap/trace.h | 2 +- fs/xfs/xfs_aops.c

[Cluster-devel] [PATCH v6 03/19] mm: Use readahead_control to pass arguments

2020-02-17 Thread Matthew Wilcox
From: "Matthew Wilcox (Oracle)" In this patch, only between __do_page_cache_readahead() and read_pages(), but it will be extended in upcoming patches. Also add the readahead_count() accessor. Signed-off-by: Matthew Wilcox (Oracle) --- include/linux/pagemap.h | 17 +++

[Cluster-devel] [PATCH v6 15/16] iomap: Convert from readpages to readahead

2020-02-17 Thread Matthew Wilcox
From: "Matthew Wilcox (Oracle)" Use the new readahead operation in iomap. Convert XFS and ZoneFS to use it. Signed-off-by: Matthew Wilcox (Oracle) --- fs/iomap/buffered-io.c | 116 - fs/iomap/trace.h | 2 +- fs/xfs/xfs_aops.c

Re: [Cluster-devel] [PATCH v5 04/13] mm: Add readahead address space operation

2020-02-14 Thread Matthew Wilcox
On Thu, Feb 13, 2020 at 09:36:25PM -0800, John Hubbard wrote: > > +static inline struct page *readahead_page(struct readahead_control *rac) > > +{ > > + struct page *page; > > + > > + if (!rac->nr_pages) > > + return NULL; > > + > > + page = xa_load(>mapping->i_pages, rac->start);

Re: [Cluster-devel] [PATCH v5 03/13] mm: Put readahead pages in cache earlier

2020-02-14 Thread Matthew Wilcox
On Thu, Feb 13, 2020 at 07:36:38PM -0800, John Hubbard wrote: > I see two distinct things happening in this patch, and I think they want to > each be > in their own patch: > > 1) A significant refactoring of the page loop, and > > 2) Changing the place where the page is added to the page cache.

Re: [Cluster-devel] [PATCH v5 01/13] mm: Fix the return type of __do_page_cache_readahead

2020-02-14 Thread Matthew Wilcox
On Mon, Feb 10, 2020 at 05:03:36PM -0800, Matthew Wilcox wrote: > From: "Matthew Wilcox (Oracle)" > > ra_submit() which is a wrapper around __do_page_cache_readahead() already > returns an unsigned long, and the 'nr_to_read' parameter is an unsigned > long, so fix

Re: [Cluster-devel] [PATCH v5 01/13] mm: Fix the return type of __do_page_cache_readahead

2020-02-13 Thread Matthew Wilcox
On Thu, Feb 13, 2020 at 07:19:53PM -0800, John Hubbard wrote: > On 2/10/20 5:03 PM, Matthew Wilcox wrote: > > @@ -161,7 +161,7 @@ unsigned int __do_page_cache_readahead(struct > > address_space *mapping, > > unsigned long end_index;/* The last page we want to read

Re: [Cluster-devel] [PATCH 00/12] Change readahead API

2020-02-13 Thread Matthew Wilcox
On Wed, Feb 12, 2020 at 08:38:52PM -0800, Andrew Morton wrote: > On Fri, 24 Jan 2020 17:35:41 -0800 Matthew Wilcox wrote: > > > From: "Matthew Wilcox (Oracle)" > > > > This series adds a readahead address_space operation to eventually > > replace the

Re: [Cluster-devel] [PATCH v5 04/13] mm: Add readahead address space operation

2020-02-11 Thread Matthew Wilcox
On Tue, Feb 11, 2020 at 03:52:30PM +1100, Dave Chinner wrote: > > +struct readahead_control { > > + struct file *file; > > + struct address_space *mapping; > > +/* private: use the readahead_* accessors instead */ > > + pgoff_t start; > > + unsigned int nr_pages; > > + unsigned int

Re: [Cluster-devel] [PATCH v5 01/13] mm: Fix the return type of __do_page_cache_readahead

2020-02-11 Thread Matthew Wilcox
On Tue, Feb 11, 2020 at 08:19:14AM +, Johannes Thumshirn wrote: > On 11/02/2020 02:05, Matthew Wilcox wrote: > > even though I'm pretty sure we're not going to readahead more than 2^32 > > pages ever. > > And 640K is more memory than anyone will ever need on a

[Cluster-devel] [PATCH v5 05/13] mm: Add page_cache_readahead_limit

2020-02-10 Thread Matthew Wilcox
From: "Matthew Wilcox (Oracle)" ext4 and f2fs have duplicated the guts of the readahead code so they can read past i_size. Instead, separate out the guts of the readahead code so they can call it directly. Signed-off-by: Matthew Wilcox (Oracle) --- fs/ext4/verity.c

[Cluster-devel] [PATCH v5 11/13] f2fs: Convert from readpages to readahead

2020-02-10 Thread Matthew Wilcox
From: "Matthew Wilcox (Oracle)" Use the new readahead operation in f2fs Signed-off-by: Matthew Wilcox (Oracle) --- fs/f2fs/data.c | 50 +++-- fs/f2fs/f2fs.h | 5 ++-- include/trace/events/f2fs.h | 6 ++--- 3 files c

[Cluster-devel] [PATCH v5 00/13] Change readahead API

2020-02-10 Thread Matthew Wilcox
From: "Matthew Wilcox (Oracle)" This series adds a readahead address_space operation to eventually replace the readpages operation. The key difference is that pages are added to the page cache as they are allocated (and then looked up by the filesystem) instead of passing them

[Cluster-devel] [PATCH v5 13/13] iomap: Convert from readpages to readahead

2020-02-10 Thread Matthew Wilcox
From: "Matthew Wilcox (Oracle)" Use the new readahead operation in iomap. Convert XFS and ZoneFS to use it. Signed-off-by: Matthew Wilcox (Oracle) --- fs/iomap/buffered-io.c | 101 - fs/iomap/trace.h | 2 +- fs/xfs/xfs_aops.c

[Cluster-devel] [PATCH v5 10/13] ext4: Convert from readpages to readahead

2020-02-10 Thread Matthew Wilcox
From: "Matthew Wilcox (Oracle)" Use the new readahead operation in ext4 Signed-off-by: Matthew Wilcox (Oracle) --- fs/ext4/ext4.h | 3 +-- fs/ext4/inode.c| 23 ++- fs/ext4/readpage.c | 22 -- 3 files changed, 19 insertions(+), 29

[Cluster-devel] [PATCH v5 04/13] mm: Add readahead address space operation

2020-02-10 Thread Matthew Wilcox
From: "Matthew Wilcox (Oracle)" This replaces ->readpages with a saner interface: - Return void instead of an ignored error code. - Pages are already in the page cache when ->readahead is called. - Implementation looks up the pages in the page cache instead of ha

[Cluster-devel] [PATCH v5 07/13] btrfs: Convert from readpages to readahead

2020-02-10 Thread Matthew Wilcox
From: "Matthew Wilcox (Oracle)" Use the new readahead operation in btrfs. Add a readahead_for_each_batch() iterator to optimise the loop in the XArray. Signed-off-by: Matthew Wilcox (Oracle) --- fs/btrfs/extent_io.c| 48 ++--- fs/btrfs/e

[Cluster-devel] [PATCH v5 09/13] erofs: Convert compressed files from readpages to readahead

2020-02-10 Thread Matthew Wilcox
From: "Matthew Wilcox (Oracle)" Use the new readahead operation in erofs. Signed-off-by: Matthew Wilcox (Oracle) --- fs/erofs/zdata.c | 29 + 1 file changed, 9 insertions(+), 20 deletions(-) diff --git a/fs/erofs/zdata.c b/fs/erofs/zdata.c index 17

[Cluster-devel] [PATCH v5 12/13] fuse: Convert from readpages to readahead

2020-02-10 Thread Matthew Wilcox
From: "Matthew Wilcox (Oracle)" Use the new readahead operation in fuse. Switching away from the read_cache_pages() helper gets rid of an implicit call to put_page(), so we can get rid of the get_page() call in fuse_readpages_fill(). Signed-off-by: Matthew Wilcox (Oracle) --- fs/f

[Cluster-devel] [PATCH v5 08/13] erofs: Convert uncompressed files from readpages to readahead

2020-02-10 Thread Matthew Wilcox
From: "Matthew Wilcox (Oracle)" Use the new readahead operation in erofs Signed-off-by: Matthew Wilcox (Oracle) --- fs/erofs/data.c | 39 +--- fs/erofs/zdata.c | 2 +- include/trace/events/erofs.h | 6 +++--- 3 files c

[Cluster-devel] [PATCH v5 03/13] mm: Put readahead pages in cache earlier

2020-02-10 Thread Matthew Wilcox
From: "Matthew Wilcox (Oracle)" At allocation time, put the pages in the cache unless we're using ->readpages. Signed-off-by: Matthew Wilcox (Oracle) --- mm/readahead.c | 66 -- 1 file changed, 42 insertions(+), 24 deletions(-) di

[Cluster-devel] [PATCH v5 06/13] fs: Convert mpage_readpages to mpage_readahead

2020-02-10 Thread Matthew Wilcox
From: "Matthew Wilcox (Oracle)" Implement the new readahead aop and convert all callers (block_dev, exfat, ext2, fat, gfs2, hpfs, isofs, jfs, nilfs2, ocfs2, omfs, qnx6, reiserfs & udf). The callers are all trivial except for GFS2 & OCFS2. Signed-off-by: Matthew Wilcox (Ora

[Cluster-devel] [PATCH v5 01/13] mm: Fix the return type of __do_page_cache_readahead

2020-02-10 Thread Matthew Wilcox
From: "Matthew Wilcox (Oracle)" ra_submit() which is a wrapper around __do_page_cache_readahead() already returns an unsigned long, and the 'nr_to_read' parameter is an unsigned long, so fix __do_page_cache_readahead() to return an unsigned long, even though I'm pretty sure we're

[Cluster-devel] [PATCH v5 02/13] mm: Ignore return value of ->readpages

2020-02-10 Thread Matthew Wilcox
From: "Matthew Wilcox (Oracle)" We used to assign the return value to a variable, which we then ignored. Remove the pretence of caring. Signed-off-by: Matthew Wilcox (Oracle) --- mm/readahead.c | 8 ++-- 1 file changed, 2 insertions(+), 6 deletions(-) diff --git a/mm/readah

Re: [Cluster-devel] [PATCH v4 00/12] Change readahead API

2020-02-04 Thread Matthew Wilcox
On Tue, Feb 04, 2020 at 04:32:27PM +0100, David Sterba wrote: > On Sat, Feb 01, 2020 at 07:12:28AM -0800, Matthew Wilcox wrote: > > From: "Matthew Wilcox (Oracle)" > > > > I would particularly value feedback on this from the gfs2 and ocfs2 > > mainta

[Cluster-devel] [PATCH v4 04/12] mm: Add readahead address space operation

2020-02-01 Thread Matthew Wilcox
From: "Matthew Wilcox (Oracle)" This replaces ->readpages with a saner interface: - Return the number of pages not read instead of an ignored error code. - Pages are already in the page cache when ->readahead is called. - Implementation looks up the pages in the pa

[Cluster-devel] [PATCH v4 05/12] fs: Convert mpage_readpages to mpage_readahead

2020-02-01 Thread Matthew Wilcox
From: "Matthew Wilcox (Oracle)" Implement the new readahead aop and convert all callers (block_dev, exfat, ext2, fat, gfs2, hpfs, isofs, jfs, nilfs2, ocfs2, omfs, qnx6, reiserfs & udf). The callers are all trivial except for GFS2 & OCFS2. Signed-off-by: Matthew Wilcox (Or

[Cluster-devel] [PATCH v4 00/12] Change readahead API

2020-02-01 Thread Matthew Wilcox
From: "Matthew Wilcox (Oracle)" I would particularly value feedback on this from the gfs2 and ocfs2 maintainers. They have non-trivial changes, and a review on patch 5 would be greatly appreciated. This series adds a readahead address_space operation to eventually replace the

[Cluster-devel] [PATCH v4 03/12] readahead: Put pages in cache earlier

2020-02-01 Thread Matthew Wilcox
From: "Matthew Wilcox (Oracle)" At allocation time, put the pages in the cache unless we're using ->readpages. Signed-off-by: Matthew Wilcox (Oracle) Cc: linux-bt...@vger.kernel.org Cc: linux-er...@lists.ozlabs.org Cc: linux-e...@vger.kernel.org Cc: linux-f2fs-de...@lists.sourc

Re: [Cluster-devel] [PATCH 04/12] mm: Add readahead address space operation

2020-01-31 Thread Matthew Wilcox
On Fri, Jan 24, 2020 at 07:57:40PM -0800, Randy Dunlap wrote: > > +``readahead`` > > + called by the VM to read pages associated with the address_space > > + object. The pages are consecutive in the page cache and are > > +locked. The implementation should decrement the page refcount

Re: [Cluster-devel] [PATCH 04/12] mm: Add readahead address space operation

2020-01-30 Thread Matthew Wilcox
On Wed, Jan 29, 2020 at 11:24:56AM +1100, Dave Chinner wrote: > On Fri, Jan 24, 2020 at 05:35:45PM -0800, Matthew Wilcox wrote: > > From: "Matthew Wilcox (Oracle)" > > > > This replaces ->readpages with a saner interface: > > - Return the number of pa

Re: [Cluster-devel] [PATCH 03/12] readahead: Put pages in cache earlier

2020-01-25 Thread Matthew Wilcox
On Fri, Jan 24, 2020 at 05:35:44PM -0800, Matthew Wilcox wrote: > @@ -192,8 +194,18 @@ unsigned long __do_page_cache_readahead(struct > address_space *mapping, > page = __page_cache_alloc(gfp_mask); > if (!page) > break; >

[Cluster-devel] [PATCH 05/12] fs: Convert mpage_readpages to mpage_readahead

2020-01-24 Thread Matthew Wilcox
From: "Matthew Wilcox (Oracle)" Implement the new readahead aop and convert all callers (block_dev, exfat, ext2, fat, gfs2, hpfs, isofs, jfs, nilfs2, ocfs2, omfs, qnx6, reiserfs & udf). The callers are all trivial except for GFS2 & OCFS2. Signed-off-by: Matthew Wilcox (Or

[Cluster-devel] [PATCH 03/12] readahead: Put pages in cache earlier

2020-01-24 Thread Matthew Wilcox
From: "Matthew Wilcox (Oracle)" At allocation time, put the pages in the cache unless we're using ->readpages. Signed-off-by: Matthew Wilcox (Oracle) Cc: linux-bt...@vger.kernel.org Cc: linux-er...@lists.ozlabs.org Cc: linux-e...@vger.kernel.org Cc: linux-f2fs-de...@lists.sourc

[Cluster-devel] [PATCH 04/12] mm: Add readahead address space operation

2020-01-24 Thread Matthew Wilcox
From: "Matthew Wilcox (Oracle)" This replaces ->readpages with a saner interface: - Return the number of pages not read instead of an ignored error code. - Pages are already in the page cache when ->readahead is called. - Implementation looks up the pages in the pa

<    1   2   3   4   5   >