[PATCH v2 14/19] xfs: convert to new i_version API

2017-12-16 Thread Jeff Layton
From: Jeff Layton <jlay...@redhat.com> Signed-off-by: Jeff Layton <jlay...@redhat.com> --- fs/xfs/libxfs/xfs_inode_buf.c | 5 +++-- fs/xfs/xfs_icache.c | 4 ++-- fs/xfs/xfs_inode.c| 2 +- fs/xfs/xfs_inode_item.c | 2 +- fs/xfs/xfs_trans_inode.c | 2

[PATCH v2 15/19] IMA: switch IMA over to new i_version API

2017-12-16 Thread Jeff Layton
From: Jeff Layton <jlay...@redhat.com> Signed-off-by: Jeff Layton <jlay...@redhat.com> --- security/integrity/ima/ima_api.c | 2 +- security/integrity/ima/ima_main.c | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/security/integrity/ima/ima_api.c b/security/i

[PATCH v2 16/19] fs: only set S_VERSION when updating times if necessary

2017-12-16 Thread Jeff Layton
From: Jeff Layton <jlay...@redhat.com> We only really need to update i_version if someone has queried for it since we last incremented it. By doing that, we can avoid having to update the inode if the times haven't changed. If the times have changed, then we go ahead and forcibly inc

[PATCH v2 12/19] ocfs2: convert to new i_version API

2017-12-16 Thread Jeff Layton
From: Jeff Layton <jlay...@redhat.com> Signed-off-by: Jeff Layton <jlay...@redhat.com> --- fs/ocfs2/dir.c | 14 +++--- fs/ocfs2/inode.c| 2 +- fs/ocfs2/namei.c| 2 +- fs/ocfs2/quota_global.c | 2 +- 4 files changed, 10 insertions(+), 10 deleti

[PATCH v2 19/19] fs: handle inode->i_version more efficiently

2017-12-16 Thread Jeff Layton
From: Jeff Layton <jlay...@redhat.com> Since i_version is mostly treated as an opaque value, we can exploit that fact to avoid incrementing it when no one is watching. With that change, we can avoid incrementing the counter on writes, unless someone has queried for it since it wa

[PATCH v2 17/19] xfs: avoid setting XFS_ILOG_CORE if i_version doesn't need incrementing

2017-12-16 Thread Jeff Layton
From: Jeff Layton <jlay...@redhat.com> If XFS_ILOG_CORE is already set then go ahead and increment it. Signed-off-by: Jeff Layton <jlay...@redhat.com> --- fs/xfs/xfs_trans_inode.c | 14 -- 1 file changed, 8 insertions(+), 6 deletions(-) diff --git a/fs/xfs/xfs_trans_

[PATCH v2 18/19] btrfs: only dirty the inode in btrfs_update_time if something was changed

2017-12-16 Thread Jeff Layton
From: Jeff Layton <jlay...@redhat.com> At this point, we know that "now" and the file times may differ, and we suspect that the i_version has been flagged to be bumped. Attempt to bump the i_version, and only mark the inode dirty if that actually occurred or if one of the t

[PATCH v2 08/19] ext2: convert to new i_version API

2017-12-16 Thread Jeff Layton
From: Jeff Layton <jlay...@redhat.com> Signed-off-by: Jeff Layton <jlay...@redhat.com> --- fs/ext2/dir.c | 8 fs/ext2/super.c | 4 ++-- 2 files changed, 6 insertions(+), 6 deletions(-) diff --git a/fs/ext2/dir.c b/fs/ext2/dir.c index 987647986f47..c99f14fec3f3 100644 -

[PATCH v2 00/19] fs: rework and optimize i_version handling in filesystems

2017-12-16 Thread Jeff Layton
From: Jeff Layton <jlay...@redhat.com> v2: - xfs should use inode_peek_iversion instead of inode_peek_iversion_raw - rework file_update_time patch - don't dirty inode when only S_ATIME is set and SB_LAZYTIME is enabled - better comments and documentation tl;dr: I think we can greatly

Re: [RFC PATCH v1 00/30] fs: inode->i_version rework and optimization

2017-10-30 Thread Jeff Layton
, link, unlink, and rename). The post code does do a fh_getattr. It's only the pre-op i_version that comes out of the cache. Only btrfs, xfs, and ext4 have a real i_version counter today, and they just scrape that info out of the in-core inode. So while not completely atomic, you should see a difference in the change_info4 during any of those operations. FWIW, userland cephfs now supports a cluster-coherent change attribute, though the kernel client still needs some work before we can implement it there. Eventually we'll add that, and at that point we might need to have nfsd do a getattr in the pre part as well. -- Jeff Layton <jlay...@redhat.com> -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: [PATCH v8 17/18] xfs: minimal conversion to errseq_t writeback error reporting

2017-06-30 Thread Jeff Layton
ecomes a pretty straight conversion to file_write_and_wait_range -- one liner patches for the most part. I've started rolling patches to do that, but now I'm wondering... Should I aim to do that with an individual patch for each fs, or is it better to do a swath of them all at once in a single patch

Re: [PATCH v8 12/18] Documentation: flesh out the section in vfs.txt on storing and reporting writeback errors

2017-06-29 Thread Jeff Layton
On Thu, 2017-06-29 at 18:21 +, Matthew Wilcox wrote: > From: Jeff Layton [mailto:jlay...@poochiereds.net] > > On Thu, 2017-06-29 at 10:11 -0700, Darrick J. Wong wrote: > > > On Thu, Jun 29, 2017 at 09:19:48AM -0400, jlay...@kernel.org wrote: > > > > +Ha

Re: [PATCH v8 18/18] btrfs: minimal conversion to errseq_t writeback error reporting on fsync

2017-06-29 Thread Jeff Layton
On Thu, 2017-06-29 at 07:17 -0700, Christoph Hellwig wrote: > On Thu, Jun 29, 2017 at 09:19:54AM -0400, jlay...@kernel.org wrote: > > From: Jeff Layton <jlay...@redhat.com> > > > > Just check and advance the errseq_t in the file before returning. > > Intern

Re: [PATCH v8 16/18] ext4: use errseq_t based error handling for reporting data writeback errors

2017-06-29 Thread Jeff Layton
t also doesn't > seem to belong into this patch. I revised the patch description earlier to say: While we're at it, ensure we always "goto out" instead of just returning directly, so that we always hit the exit tracepoint. ...but I'm fine with taking that out if you pref

Re: [PATCH v8 03/18] fs: check for writeback errors after syncing out buffers in generic_file_fsync

2017-06-29 Thread Jeff Layton
On Thu, 2017-06-29 at 07:19 -0700, Christoph Hellwig wrote: > On Thu, Jun 29, 2017 at 09:19:39AM -0400, jlay...@kernel.org wrote: > > From: Jeff Layton <jlay...@redhat.com> > > > > ext2 currently does a test+clear of the AS_EIO flag, which is > > is pr

Re: [PATCH v8 12/18] Documentation: flesh out the section in vfs.txt on storing and reporting writeback errors

2017-06-29 Thread Jeff Layton
On Thu, 2017-06-29 at 10:11 -0700, Darrick J. Wong wrote: > On Thu, Jun 29, 2017 at 09:19:48AM -0400, jlay...@kernel.org wrote: > > From: Jeff Layton <jlay...@redhat.com> > > > > Let's try to make this extra clear for fs authors. > > > > Cc: Jan Kara <j.

Re: [PATCH v8 10/18] fs: new infrastructure for writeback error handling and reporting

2017-06-29 Thread Jeff Layton
On Thu, 2017-06-29 at 09:19 -0400, jlay...@kernel.org wrote: > From: Jeff Layton <jlay...@redhat.com> > > Most filesystems currently use mapping_set_error and > filemap_check_errors for setting and reporting/clearing writeback errors > at the mapping level. filemap_check

Re: [PATCH v8 10/18] fs: new infrastructure for writeback error handling and reporting

2017-06-29 Thread Jeff Layton
On Thu, 2017-06-29 at 09:19 -0400, jlay...@kernel.org wrote: > From: Jeff Layton <jlay...@redhat.com> > > Most filesystems currently use mapping_set_error and > filemap_check_errors for setting and reporting/clearing writeback errors > at the mapping level. filemap_check

Re: [PATCH v7 16/22] block: convert to errseq_t based writeback error tracking

2017-06-26 Thread Jeff Layton
On Sat, 2017-06-24 at 09:16 -0400, Jeff Layton wrote: > On Sat, 2017-06-24 at 04:59 -0700, Christoph Hellwig wrote: > > On Tue, Jun 20, 2017 at 01:44:44PM -0400, Jeff Layton wrote: > > > In order to query for errors with errseq_t, you need a previously- > > > sample

Re: [PATCH v7 16/22] block: convert to errseq_t based writeback error tracking

2017-06-24 Thread Jeff Layton
On Sat, 2017-06-24 at 04:59 -0700, Christoph Hellwig wrote: > On Tue, Jun 20, 2017 at 01:44:44PM -0400, Jeff Layton wrote: > > In order to query for errors with errseq_t, you need a previously- > > sampled point from which to check. When you call > > filemap_write_and_wait_r

Re: [PATCH v7 16/22] block: convert to errseq_t based writeback error tracking

2017-06-20 Thread Jeff Layton
ed point from which to check. When you call filemap_write_and_wait_range though you don't have a struct file and so no previously-sampled value. -- Jeff Layton <jlay...@redhat.com> -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: [PATCH v7 11/22] fs: new infrastructure for writeback error handling and reporting

2017-06-20 Thread Jeff Layton
t address_space *mapping, > > errseq_t since) > > Overly long line here (the patch has a few more) > Ok, I'll fix those up. -- Jeff Layton <jlay...@redhat.com> -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: [PATCH v7 00/22] fs: enhanced writeback error reporting with errseq_t (pile #1)

2017-06-20 Thread Jeff Layton
On Tue, 2017-06-20 at 09:25 +1000, Stephen Rothwell wrote: > Hi Jeff, > > On Mon, 19 Jun 2017 12:23:46 -0400 Jeff Layton <jlay...@redhat.com> wrote: > > > > If there are no major objections to this set, I'd like to have > > linux-next start picking it up

Re: [PATCH v7 00/22] fs: enhanced writeback error reporting with errseq_t (pile #1)

2017-06-19 Thread Jeff Layton
On Fri, 2017-06-16 at 15:34 -0400, Jeff Layton wrote: > v7: > === > This is the seventh posting of the patchset to revamp the way writeback > errors are tracked and reported. > > The main difference from the v6 posting is the removal of the > FS_WB_ERRSEQ flag. That

Re: [PATCH v7 15/22] dax: set errors in mapping when writeback fails

2017-06-17 Thread Jeff Layton
On Fri, 2017-06-16 at 15:34 -0400, Jeff Layton wrote: > Jan Kara's description for this patch is much better than mine, so I'm > quoting it verbatim here: > > DAX currently doesn't set errors in the mapping when cache flushing > fails in dax_writeback_mapping_range(). Since this f

[PATCH v7 07/22] mm: don't TestClearPageError in __filemap_fdatawait_range

2017-06-16 Thread Jeff Layton
across subsystems, let's just rely on marking the address space when there are writeback errors. Change the TestClearPageError call to ClearPageError, and make __filemap_fdatawait_range a void return function. Signed-off-by: Jeff Layton <jlay...@redhat.com> --- mm/filemap.

[PATCH v7 06/22] mm: clear AS_EIO/AS_ENOSPC when writeback initiation fails

2017-06-16 Thread Jeff Layton
. Signed-off-by: Jeff Layton <jlay...@redhat.com> --- mm/filemap.c | 6 ++ 1 file changed, 6 insertions(+) diff --git a/mm/filemap.c b/mm/filemap.c index 37f286df7c95..c349a5d3a34b 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -517,6 +517,9 @@ int filemap_write_and_wait(struct address

[PATCH v7 11/22] fs: new infrastructure for writeback error handling and reporting

2017-06-16 Thread Jeff Layton
code to use this new infrastructure for reporting errors at fsync time. Signed-off-by: Jeff Layton <jlay...@redhat.com> Reviewed-by: Jan Kara <j...@suse.cz> --- drivers/dax/device.c | 1 + fs/block_dev.c | 1 + fs/file_table.c | 1 + fs/open.c|

[PATCH v7 13/22] mm: set both AS_EIO/AS_ENOSPC and errseq_t in mapping_set_error

2017-06-16 Thread Jeff Layton
In later patches, we're going to want to convert many of these callers to check for errors since a well-defined point in time. For now, ensure that we can handle both sorts of checks by both setting errors in both places when there is a writeback failure. Signed-off-by: Jeff Layton <jlay.

[xfstests PATCH v5 5/5] btrfs: make a btrfs version of writeback error reporting test

2017-06-16 Thread Jeff Layton
Mason <c...@fb.com> Signed-off-by: Jeff Layton <jlay...@redhat.com> --- tests/btrfs/999 | 94 + tests/btrfs/999.out | 3 ++ tests/btrfs/group | 1 + 3 files changed, 98 insertions(+) create mode 100755 tests/btrfs/999 create

[PATCH v7 22/22] btrfs: minimal conversion to errseq_t writeback error reporting on fsync

2017-06-16 Thread Jeff Layton
Just check and advance the errseq_t in the file before returning. Internal callers of filemap_* functions are left as-is. Signed-off-by: Jeff Layton <jlay...@redhat.com> --- fs/btrfs/file.c | 7 +-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/fs/btrfs/file.c b/fs

[xfstests PATCH v5 4/5] generic: test writeback error handling on dmerror devices

2017-06-16 Thread Jeff Layton
Ensure that we get an error back on all fds when a block device is open by multiple writers and writeback fails. Signed-off-by: Jeff Layton <jlay...@redhat.com> --- tests/generic/998 | 63 +++ tests/generic/998.out | 2 ++ tests/generic

[PATCH v7 14/22] Documentation: flesh out the section in vfs.txt on storing and reporting writeback errors

2017-06-16 Thread Jeff Layton
Let's try to make this extra clear for fs authors. Cc: Jan Kara <j...@suse.cz> Signed-off-by: Jeff Layton <jlay...@redhat.com> --- Documentation/filesystems/vfs.txt | 43 --- 1 file changed, 40 insertions(+), 3 deletions(-) diff --git a/D

[PATCH v7 16/22] block: convert to errseq_t based writeback error tracking

2017-06-16 Thread Jeff Layton
in the mapping. Note that there are internal callers that call sync_blockdev and the like that are not affected by this. They'll continue to use the AS_EIO/AS_ENOSPC flags for error reporting like they always have for now. Signed-off-by: Jeff Layton <jlay...@redhat.com> --- fs/block_dev

[PATCH v7 19/22] ext4: add more robust reporting of metadata writeback errors

2017-06-16 Thread Jeff Layton
metadata writeback errors are only tracked on a per-device level, this does mean that we'll end up reporting an error on all open file descriptors when there is a metadata writeback failure. That's not ideal, but we can possibly improve upon it in the future. Signed-off-by: Jeff Layton <j

[PATCH v7 12/22] mm: tracepoints for writeback error events

2017-06-16 Thread Jeff Layton
To enable that, make __errseq_set return the value that it was set to when we exit the loop. Take heed that that value is not suitable as a later "since" value, as it will not have been marked seen. Signed-off-by: Jeff Layton <jlay...@redhat.com> --- include/linux/errse

[xfstests PATCH v5 2/5] ext3: allow it to put journal on a separate device when doing scratch_mkfs

2017-06-16 Thread Jeff Layton
Signed-off-by: Jeff Layton <jlay...@redhat.com> --- common/rc | 11 ++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/common/rc b/common/rc index 57001b47a8b7..43e160e91360 100644 --- a/common/rc +++ b/common/rc @@ -840,7 +840,16 @@ _scratch_mkfs() mk

[xfstests PATCH v5 0/5] new tests for writeback error reporting behavior

2017-06-16 Thread Jeff Layton
devices and one test for btrfs. The tests work with dmerror to make data writeback from the pagecache fail, and then tests how the kernel reports errors afterward. xfs, ext2/3/4 and btrfs all pass on a kernel with the patchset above. "Bare" block devices also work correctly. Jeff

[xfstests PATCH v5 3/5] generic: add a writeback error handling test

2017-06-16 Thread Jeff Layton
, and eventually make it a more general test. Signed-off-by: Jeff Layton <jlay...@redhat.com> --- .gitignore | 1 + common/dmerror | 13 ++- doc/auxiliary-programs.txt | 16 src/Makefile | 4 +- src/dmerror| 44 + src/fsync

[PATCH v7 20/22] ext2: convert to errseq_t based writeback error tracking

2017-06-16 Thread Jeff Layton
Set the flag to indicate that we want new-style data writeback error handling. This means that we need to override the open routines for files and directories so that we can sample the bdev wb_err at open. Signed-off-by: Jeff Layton <jlay...@redhat.com> --- fs/ext2/dir.c | 8 f

[PATCH v7 15/22] dax: set errors in mapping when writeback fails

2017-06-16 Thread Jeff Layton
the file or if sync(2) gets called before fsync(2). So convert DAX to using standard error reporting through the mapping. Signed-off-by: Jeff Layton <jlay...@redhat.com> Reviewed-by: Jan Kara <j...@suse.cz> Reviewed-by: Christoph Hellwig <h...@lst.de> Reviewed-and-Tested-by: Ross

[PATCH v7 21/22] xfs: minimal conversion to errseq_t writeback error reporting

2017-06-16 Thread Jeff Layton
Just check and advance the data errseq_t in struct file before before returning from fsync on normal files. Internal filemap_* callers are left as-is. Signed-off-by: Jeff Layton <jlay...@redhat.com> --- fs/xfs/xfs_file.c | 15 +++ 1 file changed, 11 insertions(+), 4 del

[PATCH v7 18/22] fs: add f_md_wb_err field to struct file for tracking metadata errors

2017-06-16 Thread Jeff Layton
Some filesystems keep a different mapping for metadata writeback. Add a second errseq_t to struct file for tracking metadata writeback errors. Also add a new function for checking a mapping of the caller's choosing vs. the f_md_wb_err value. Signed-off-by: Jeff Layton <jlay...@redhat.

[xfstests PATCH v5 1/5] ext4: allow ext4 to use $SCRATCH_LOGDEV

2017-06-16 Thread Jeff Layton
the same thing when _scratch_mkfs is called. Signed-off-by: Jeff Layton <jlay...@redhat.com> Reviewed-by: Darrick J. Wong <darrick.w...@oracle.com> --- common/rc | 3 +++ 1 file changed, 3 insertions(+) diff --git a/common/rc b/common/rc index ff1b75c9cd25..57001b47a8b7 100644 --- a/com

[PATCH v7 17/22] ext4: use errseq_t based error handling for reporting data writeback errors

2017-06-16 Thread Jeff Layton
. That will be added in a later patch. Signed-off-by: Jeff Layton <jlay...@redhat.com> --- fs/ext4/fsync.c | 15 +++ 1 file changed, 11 insertions(+), 4 deletions(-) diff --git a/fs/ext4/fsync.c b/fs/ext4/fsync.c index 9d549608fd30..03d6259d8662 100644 --- a/fs/ext4/fsync.c +++ b/f

[PATCH v7 10/22] lib: add errseq_t type and infrastructure for handling it

2017-06-16 Thread Jeff Layton
t was last bumped. Later patches will build on this infrastructure to change how writeback errors are tracked in the kernel. Signed-off-by: Jeff Layton <jlay...@redhat.com> Reviewed-by: NeilBrown <ne...@suse.com> Reviewed-by: Jan Kara <j...@suse.cz> --- include/linux/errseq

[PATCH v7 08/22] mm: clean up error handling in write_one_page

2017-06-16 Thread Jeff Layton
Don't try to check PageError since that's potentially racy and not necessarily going to be set after writepage errors out. Instead, check the mapping for an error after writepage returns. Signed-off-by: Jeff Layton <jlay...@redhat.com> Reviewed-by: Jan Kara <j...@suse.cz>

[PATCH v7 09/22] fs: always sync metadata in __generic_file_fsync

2017-06-16 Thread Jeff Layton
attempting to write out the metadata, even when a flush of the data reports an error. Signed-off-by: Jeff Layton <jlay...@redhat.com> --- fs/libfs.c | 9 - 1 file changed, 4 insertions(+), 5 deletions(-) diff --git a/fs/libfs.c b/fs/libfs.c index 1dec90819366..c93e77ecb49c

[PATCH v7 05/22] jbd2: don't clear and reset errors after waiting on writeback

2017-06-16 Thread Jeff Layton
Resetting this flag is almost certainly racy, and will be problematic with some coming changes. Make filemap_fdatawait_keep_errors return int, but not clear the flag(s). Have jbd2 call it instead of filemap_fdatawait and don't attempt to re-set the error flag if it fails. Signed-off-by: Jeff

[PATCH v7 04/22] buffer: set errors in mapping at the time that the error occurs

2017-06-16 Thread Jeff Layton
that it's first detected. Signed-off-by: Jeff Layton <jlay...@redhat.com> Reviewed-by: Jan Kara <j...@suse.cz> --- fs/buffer.c | 20 +--- fs/gfs2/lops.c | 2 +- include/linux/buffer_head.h | 1 + 3 files changed, 15 insertions(+), 8 deletions(-)

[PATCH v7 03/22] fs: check for writeback errors after syncing out buffers in generic_file_fsync

2017-06-16 Thread Jeff Layton
these errors properly as well. With that, we don't need to twiddle it in ext2. Suggested-by: Jan Kara <j...@suse.cz> Signed-off-by: Jeff Layton <jlay...@redhat.com> Reviewed-by: Christoph Hellwig <h...@lst.de> Reviewed-by: Jan Kara <j...@suse.cz> Reviewed-by: Matthew Wilcox

[PATCH v7 00/22] fs: enhanced writeback error reporting with errseq_t (pile #1)

2017-06-16 Thread Jeff Layton
the filesystem layer reports errors after that. Jeff Layton (22): fs: remove call_fsync helper function buffer: use mapping_set_error instead of setting the flag fs: check for writeback errors after syncing out buffers in generic_file_fsync buffer: set errors in mapping at the time

[PATCH v7 02/22] buffer: use mapping_set_error instead of setting the flag

2017-06-16 Thread Jeff Layton
Signed-off-by: Jeff Layton <jlay...@redhat.com> Reviewed-by: Jan Kara <j...@suse.cz> Reviewed-by: Matthew Wilcox <mawil...@microsoft.com> Reviewed-by: Christoph Hellwig <h...@lst.de> --- fs/buffer.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/f

[PATCH v7 01/22] fs: remove call_fsync helper function

2017-06-16 Thread Jeff Layton
Requested-by: Christoph Hellwig <h...@infradead.org> Signed-off-by: Jeff Layton <jlay...@redhat.com> --- fs/sync.c | 2 +- include/linux/fs.h | 6 -- ipc/shm.c | 2 +- 3 files changed, 2 insertions(+), 8 deletions(-) diff --git a/fs/sync.c b/fs/sync.c index

Re: [PATCH v6 12/20] fs: add a new fstype flag to indicate how writeback errors are tracked

2017-06-15 Thread Jeff Layton
On Thu, 2017-06-15 at 07:57 -0700, Christoph Hellwig wrote: > On Thu, Jun 15, 2017 at 06:42:12AM -0400, Jeff Layton wrote: > > Correct. > > > > But if there is a data writeback error, should we report an error on all > > open fds at that time (like we will for fsync

Re: [PATCH v6 12/20] fs: add a new fstype flag to indicate how writeback errors are tracked

2017-06-15 Thread Jeff Layton
On Thu, 2017-06-15 at 01:22 -0700, Christoph Hellwig wrote: > On Wed, Jun 14, 2017 at 01:24:43PM -0400, Jeff Layton wrote: > > In this smaller set, it's only really used for DAX. > > DAX only is implemented by three filesystems, please just fix them > up in one go. > Ok.

Re: [PATCH v6 12/20] fs: add a new fstype flag to indicate how writeback errors are tracked

2017-06-14 Thread Jeff Layton
On Tue, 2017-06-13 at 23:47 -0700, Christoph Hellwig wrote: > On Tue, Jun 13, 2017 at 06:24:32AM -0400, Jeff Layton wrote: > > That's definitely what I want for the endgame here. My plan was to add > > this flag for now, and then eventually reverse it (or drop it) once all > >

Re: [xfstests PATCH v4 5/5] btrfs: make a btrfs version of writeback error reporting test

2017-06-14 Thread Jeff Layton
On Tue, 2017-06-13 at 16:40 +0800, Eryu Guan wrote: > On Mon, Jun 12, 2017 at 08:42:13AM -0400, Jeff Layton wrote: > > Make a new btrfs/999 test that works the way Chris Mason suggested: > > > > Build a filesystem with 2 devices that stripes the data across > > both de

Re: [xfstests PATCH v4 0/5] new tests for writeback error reporting behavior

2017-06-13 Thread Jeff Layton
On Tue, 2017-06-13 at 16:32 +0800, Eryu Guan wrote: > On Mon, Jun 12, 2017 at 08:42:08AM -0400, Jeff Layton wrote: > > v4: respin set based on Eryu's comments > > > > These tests are companion tests to the patchset I recently posted with > > the cover letter: >

Re: [PATCH v6 19/20] xfs: minimal conversion to errseq_t writeback error reporting

2017-06-13 Thread Jeff Layton
On Mon, 2017-06-12 at 21:30 -0700, Darrick J. Wong wrote: > On Mon, Jun 12, 2017 at 08:23:15AM -0400, Jeff Layton wrote: > > Just set the FS_WB_ERRSEQ flag to indicate that we want to use errseq_t > > based error reporting. Internal filemap_* calls are left as-is for now. >

Re: [PATCH v6 12/20] fs: add a new fstype flag to indicate how writeback errors are tracked

2017-06-13 Thread Jeff Layton
On Mon, 2017-06-12 at 05:45 -0700, Christoph Hellwig wrote: > On Mon, Jun 12, 2017 at 08:23:06AM -0400, Jeff Layton wrote: > > Add a new FS_WB_ERRSEQ flag to the fstype. Later patches will set and > > key off of that to decide what behavior should be used. > > Please invert t

[xfstests PATCH v4 3/5] generic: test writeback error handling on dmerror devices

2017-06-12 Thread Jeff Layton
Ensure that we get an error back on all fds when a block device is open by multiple writers and writeback fails. Signed-off-by: Jeff Layton <jlay...@redhat.com> --- tests/generic/998 | 64 +++ tests/generic/998.out | 2 ++ tests/generic

[xfstests PATCH v4 5/5] btrfs: make a btrfs version of writeback error reporting test

2017-06-12 Thread Jeff Layton
Make a new btrfs/999 test that works the way Chris Mason suggested: Build a filesystem with 2 devices that stripes the data across both devices, but mirrors metadata across both. Then, make one of the devices fail and see how fsync is handled. Signed-off-by: Jeff Layton <jlay...@redhat.

[xfstests PATCH v4 1/5] generic: add a writeback error handling test

2017-06-12 Thread Jeff Layton
, but we can fill it out with other commands as necessary. Signed-off-by: Jeff Layton <jlay...@redhat.com> --- .gitignore | 1 + common/dmerror | 13 ++- doc/auxiliary-programs.txt | 16 src/Makefile | 2 +- src/dmerror

[xfstests PATCH v4 2/5] ext4: allow ext4 to use $SCRATCH_LOGDEV

2017-06-12 Thread Jeff Layton
the same thing when _scratch_mkfs is called. Signed-off-by: Jeff Layton <jlay...@redhat.com> Reviewed-by: Darrick J. Wong <darrick.w...@oracle.com> --- common/rc | 3 +++ 1 file changed, 3 insertions(+) diff --git a/common/rc b/common/rc index 87e6ff08b18d..08807ac7c22a 100644 --- a/com

[xfstests PATCH v4 4/5] ext3: allow it to put journal on a separate device when doing scratch_mkfs

2017-06-12 Thread Jeff Layton
Signed-off-by: Jeff Layton <jlay...@redhat.com> --- common/rc | 11 ++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/common/rc b/common/rc index 08807ac7c22a..46b890cbff6a 100644 --- a/common/rc +++ b/common/rc @@ -832,7 +832,16 @@ _scratch_mkfs() mk

[xfstests PATCH v4 0/5] new tests for writeback error reporting behavior

2017-06-12 Thread Jeff Layton
filesystems fail this test? Jeff Layton (5): generic: add a writeback error handling test ext4: allow ext4 to use $SCRATCH_LOGDEV generic: test writeback error handling on dmerror devices ext3: allow it to put journal on a separate device when doing scratch_mkfs btrfs: make a btrfs version

[PATCH v6 00/20] fs: enhanced writeback error reporting with errseq_t (pile #1)

2017-06-12 Thread Jeff Layton
on top of that. Jeff Layton (20): mm: fix mapping_set_error call in me_pagecache_dirty buffer: use mapping_set_error instead of setting the flag fs: check for writeback errors after syncing out buffers in generic_file_fsync buffer: set errors in mapping at the time that the error occurs

[PATCH v6 06/20] mm: drop "wait" parameter from write_one_page

2017-06-12 Thread Jeff Layton
an error here in some fashion. Signed-off-by: Jeff Layton <jlay...@redhat.com> Reviewed-by: Ross Zwisler <ross.zwis...@linux.intel.com> Reviewed-by: Jan Kara <j...@suse.cz> Reviewed-by: Matthew Wilcox <mawil...@microsoft.com> Reviewed-by: Christoph Hellwig <h...@

[PATCH v6 05/20] mm: don't TestClearPageError in __filemap_fdatawait_range

2017-06-12 Thread Jeff Layton
across subsystems, let's just rely on marking the address space when there are writeback errors. Change the TestClearPageError call to ClearPageError, and make __filemap_fdatawait_range a void return function. Signed-off-by: Jeff Layton <jlay...@redhat.com> --- mm/filemap.

[PATCH v6 02/20] buffer: use mapping_set_error instead of setting the flag

2017-06-12 Thread Jeff Layton
Signed-off-by: Jeff Layton <jlay...@redhat.com> Reviewed-by: Jan Kara <j...@suse.cz> Reviewed-by: Matthew Wilcox <mawil...@microsoft.com> Reviewed-by: Christoph Hellwig <h...@lst.de> --- fs/buffer.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/f

[PATCH v6 10/20] mm: tracepoints for writeback error events

2017-06-12 Thread Jeff Layton
To enable that, make __errseq_set return the value that it was set to when we exit the loop. Take heed that that value is not suitable as a later "since" value, as it will not have been marked seen. Signed-off-by: Jeff Layton <jlay...@redhat.com> --- include/linux/errse

[PATCH v6 04/20] buffer: set errors in mapping at the time that the error occurs

2017-06-12 Thread Jeff Layton
that it's first detected. Signed-off-by: Jeff Layton <jlay...@redhat.com> Reviewed-by: Jan Kara <j...@suse.cz> --- fs/buffer.c | 20 +--- fs/gfs2/lops.c | 2 +- include/linux/buffer_head.h | 1 + 3 files changed, 15 insertions(+), 8 deletions(-)

[PATCH v6 03/20] fs: check for writeback errors after syncing out buffers in generic_file_fsync

2017-06-12 Thread Jeff Layton
these errors properly as well. With that, we don't need to twiddle it in ext2. Suggested-by: Jan Kara <j...@suse.cz> Signed-off-by: Jeff Layton <jlay...@redhat.com> Reviewed-by: Christoph Hellwig <h...@lst.de> Reviewed-by: Jan Kara <j...@suse.cz> Reviewed-by: Matthew Wilcox

[PATCH v6 10/13] ext4: add more robust reporting of metadata writeback errors

2017-06-12 Thread Jeff Layton
metadata writeback errors are only tracked on a per-device level, this does mean that we'll end up reporting an error on all open file descriptors when there is a metadata writeback failure. That's not ideal, but we can possibly improve upon it in the future. Signed-off-by: Jeff Layton <j

[PATCH v6 12/20] fs: add a new fstype flag to indicate how writeback errors are tracked

2017-06-12 Thread Jeff Layton
.some code is shared between filesystems and needs to be able to handle both error tracking schemes. Add a new FS_WB_ERRSEQ flag to the fstype. Later patches will set and key off of that to decide what behavior should be used. Signed-off-by: Jeff Layton <jlay...@redhat.com> --- includ

[PATCH v6 17/20] fs: add f_md_wb_err field to struct file for tracking metadata errors

2017-06-12 Thread Jeff Layton
Some filesystems keep a different mapping for metadata writeback. Add a second errseq_t to struct file for tracking metadata writeback errors. Also add a new function for checking a mapping of the caller's choosing vs. the f_md_wb_err value. Signed-off-by: Jeff Layton <jlay...@redhat.

[PATCH v6 14/20] dax: set errors in mapping when writeback fails

2017-06-12 Thread Jeff Layton
rently cleared in the older code when writeback initiation fails, only when we discover an error after waiting on writeback to complete, so we only want to do this with errseq_t based error handling to prevent seeing duplicate errors on fsync. Signed-off-by: Jeff Layton <jlay...@redhat.com> Revi

[PATCH v6 12/13] xfs: minimal conversion to errseq_t writeback error reporting

2017-06-12 Thread Jeff Layton
Just check and advance the data errseq_t in struct file before before returning from fsync on normal files. Internal filemap_* callers are left as-is. We also set the FS_WB_ERRSEQ flag just for completeness sake. Not much is really using it at this point. Signed-off-by: Jeff Layton <j

[PATCH v6 16/20] block: convert to errseq_t based writeback error tracking

2017-06-12 Thread Jeff Layton
-by: Jeff Layton <jlay...@redhat.com> --- fs/block_dev.c | 1 + 1 file changed, 1 insertion(+) diff --git a/fs/block_dev.c b/fs/block_dev.c index 4d62fe771587..4bf865cc99de 100644 --- a/fs/block_dev.c +++ b/fs/block_dev.c @@ -801,6 +801,7 @@ static struct file_system_type bd_type = {

[PATCH v6 11/20] mm: set both AS_EIO/AS_ENOSPC and errseq_t in mapping_set_error

2017-06-12 Thread Jeff Layton
In later patches, we're going to want to convert many of these callers to check for errors since a well-defined point in time. For now, ensure that we can handle both sorts of checks by both setting errors in both places when there is a writeback failure. Signed-off-by: Jeff Layton <jlay.

[PATCH v6 08/20] lib: add errseq_t type and infrastructure for handling it

2017-06-12 Thread Jeff Layton
t was last bumped. Later patches will build on this infrastructure to change how writeback errors are tracked in the kernel. Signed-off-by: Jeff Layton <jlay...@redhat.com> Reviewed-by: NeilBrown <ne...@suse.com> Reviewed-by: Jan Kara <j...@suse.cz> --- include/linux/errseq

[PATCH v6 18/20] ext4: use errseq_t based error handling for reporting data writeback errors

2017-06-12 Thread Jeff Layton
there is a metadata writeback failure. That's not ideal, but we can possibly improve upon it in the future. ext4 also has several calls to filemap_fdatawait and filemap_write_and_wait. Those will also be changed in a later patch to versions that use errseq_t based reporting. Signed-off-by: Jeff

[PATCH v6 20/20] btrfs: minimal conversion to errseq_t writeback error reporting on fsync

2017-06-12 Thread Jeff Layton
Set the FS_WB_ERRSEQ flag to opt-in to errseq_t based reporting. Internal call to filemap_* functions are left as-is. Signed-off-by: Jeff Layton <jlay...@redhat.com> --- fs/btrfs/super.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/fs/btrfs/super.c b/fs/btrfs/super.c

[PATCH v6 15/20] fs: have call_fsync call filemap_report_wb_err if FS_WB_ERRSEQ is set

2017-06-12 Thread Jeff Layton
Allow filesystems to opt-in to a final check of wb_err if FS_WB_ERRSEQ is set. Technically, we could just plumb these calls into all of the fsync operations, but I think this means less code, changes and churn. Signed-off-by: Jeff Layton <jlay...@redhat.com> --- include/linux/fs.

[PATCH v6 13/13] btrfs: minimal conversion to errseq_t writeback error reporting on fsync

2017-06-12 Thread Jeff Layton
Internal callers of filemap_* functions are left as-is. Signed-off-by: Jeff Layton <jlay...@redhat.com> --- fs/btrfs/file.c | 7 +-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/fs/btrfs/file.c b/fs/btrfs/file.c index da1096eb1a40..4632f16bc49c 100644 --- a/fs/btrfs/

[PATCH v6 11/13] Documentation: flesh out the section in vfs.txt on storing and reporting writeback errors

2017-06-12 Thread Jeff Layton
.cz> Signed-off-by: Jeff Layton <jlay...@redhat.com> --- Documentation/filesystems/vfs.txt | 50 --- 1 file changed, 47 insertions(+), 3 deletions(-) diff --git a/Documentation/filesystems/vfs.txt b/Documentation/filesystems/vfs.txt index

[PATCH v6 13/20] Documentation: flesh out the section in vfs.txt on storing and reporting writeback errors

2017-06-12 Thread Jeff Layton
Let's try to make this extra clear for fs authors. Also, although I think we'll eventually remove it once the transition is complete, I've gone ahead and documented the FS_WB_ERRSEQ flag as well. Cc: Jan Kara <j...@suse.cz> Signed-off-by: Jeff Layton <jlay...@redhat.com> --- D

[PATCH v6 19/20] xfs: minimal conversion to errseq_t writeback error reporting

2017-06-12 Thread Jeff Layton
Just set the FS_WB_ERRSEQ flag to indicate that we want to use errseq_t based error reporting. Internal filemap_* calls are left as-is for now. Signed-off-by: Jeff Layton <jlay...@redhat.com> --- fs/xfs/xfs_super.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/

[PATCH v6 09/20] fs: new infrastructure for writeback error handling and reporting

2017-06-12 Thread Jeff Layton
code to use this new infrastructure. Signed-off-by: Jeff Layton <jlay...@redhat.com> Reviewed-by: Jan Kara <j...@suse.cz> --- drivers/dax/device.c | 1 + fs/block_dev.c | 1 + fs/file_table.c | 1 + fs/open.c|

[PATCH v6 01/20] mm: fix mapping_set_error call in me_pagecache_dirty

2017-06-12 Thread Jeff Layton
The error code should be negative. Since this ends up in the default case anyway, this is harmless, but it's less confusing to negate it. Also, later patches will require a negative error code here. Signed-off-by: Jeff Layton <jlay...@redhat.com> Reviewed-by: Ross Zwisler <

[PATCH v6 07/20] mm: clean up error handling in write_one_page

2017-06-12 Thread Jeff Layton
Don't try to check PageError since that's potentially racy and not necessarily going to be set after writepage errors out. Instead, check the mapping for an error after writepage returns. Signed-off-by: Jeff Layton <jlay...@redhat.com> Reviewed-by: Jan Kara <j...@suse.cz>

[PATCH] btrfs: btrfs_wait_tree_block_writeback can be void return

2017-05-25 Thread Jeff Layton
Nothing checks its return value. Signed-off-by: Jeff Layton <jlay...@redhat.com> Reviewed-by: Jan Kara <j...@suse.cz> Reviewed-by: Liu Bo <bo.li@oracle.com> --- fs/btrfs/disk-io.c | 6 +++--- fs/btrfs/disk-io.h | 2 +- 2 files changed, 4 insertions(+), 4 deletions(-) dif

Re: btrfs list corruption and soft lockups while testing writeback error handling

2017-05-19 Thread Jeff Layton
On Thu, 2017-05-18 at 21:57 -0700, Liu Bo wrote: > On Fri, May 12, 2017 at 10:22:47AM -0400, Jeff Layton wrote: > > On Fri, 2017-05-12 at 08:12 -0400, Jeff Layton wrote: > > > On Thu, 2017-05-11 at 15:56 -0400, Chris Mason wrote: > > > > On 05/11/2017 03:52 PM, Jeff

Re: [PATCH v4 15/27] fs: retrofit old error reporting API onto new infrastructure

2017-05-15 Thread Jeff Layton
On Mon, 2017-05-15 at 12:42 +0200, Jan Kara wrote: > On Tue 09-05-17 11:49:18, Jeff Layton wrote: > > Now that we have a better way to store and report errors that occur > > during writeback, we need to convert the existing codebase to use it. We > > could just adapt all o

Re: btrfs list corruption and soft lockups while testing writeback error handling

2017-05-12 Thread Jeff Layton
On Fri, 2017-05-12 at 08:12 -0400, Jeff Layton wrote: > On Thu, 2017-05-11 at 15:56 -0400, Chris Mason wrote: > > On 05/11/2017 03:52 PM, Jeff Layton wrote: > > > On Thu, 2017-05-11 at 07:13 -0400, Jeff Layton wrote: > > > > I finally got my writeback error handling

Re: btrfs list corruption and soft lockups while testing writeback error handling

2017-05-12 Thread Jeff Layton
On Thu, 2017-05-11 at 15:56 -0400, Chris Mason wrote: > On 05/11/2017 03:52 PM, Jeff Layton wrote: > > On Thu, 2017-05-11 at 07:13 -0400, Jeff Layton wrote: > > > I finally got my writeback error handling test to work on btrfs (thanks, > > > Chris!), by making the

Re: [RFC PATCH v1 00/30] fs: inode->i_version rework and optimization

2017-05-12 Thread Jeff Layton
se if (IS_I_VERSION(inode)) { > - p = xdr_encode_hyper(p, inode->i_version); > + p = xdr_encode_hyper(p, nfsd4_change_attribute(inode)); > } else { > *p++ = cpu_to_be32(stat->ctime.tv_sec); > *p++ = cpu_to_be32(stat->cti

Re: btrfs list corruption and soft lockups while testing writeback error handling

2017-05-11 Thread Jeff Layton
On Thu, 2017-05-11 at 07:13 -0400, Jeff Layton wrote: > I finally got my writeback error handling test to work on btrfs (thanks, > Chris!), by making the filesystem stripe the data and mirror the > metadata across two devices. The test passes now, but on one run, I got > the fo

btrfs list corruption and soft lockups while testing writeback error handling

2017-05-11 Thread Jeff Layton
I finally got my writeback error handling test to work on btrfs (thanks, Chris!), by making the filesystem stripe the data and mirror the metadata across two devices. The test passes now, but on one run, I got the following list corruption warning and then a soft lockup (which is probably fallout

<    1   2   3   4   >