Re: [patch] file dedupe (and maybe clone) data corruption (was Re: [PATCH] generic: test for deduplication between different files)

2018-09-20 Thread Dave Chinner
On Wed, Sep 19, 2018 at 12:12:03AM -0400, Zygo Blaxell wrote: > On Mon, Sep 10, 2018 at 07:06:46PM +1000, Dave Chinner wrote: > > On Thu, Sep 06, 2018 at 11:53:06PM -0400, Zygo Blaxell wrote: > > > On Thu, Sep 06, 2018 at 06:38:09PM +1000, Dave Chinner wrote: > > > >

Re: [patch] file dedupe (and maybe clone) data corruption (was Re: [PATCH] generic: test for deduplication between different files)

2018-09-10 Thread Dave Chinner
On Thu, Sep 06, 2018 at 11:53:06PM -0400, Zygo Blaxell wrote: > On Thu, Sep 06, 2018 at 06:38:09PM +1000, Dave Chinner wrote: > > On Fri, Aug 31, 2018 at 01:10:45AM -0400, Zygo Blaxell wrote: > > > On Thu, Aug 30, 2018 at 04:27:43PM +1000, Dave Chinner wrote: > > > >

Re: [patch] file dedupe (and maybe clone) data corruption (was Re: [PATCH] generic: test for deduplication between different files)

2018-09-06 Thread Dave Chinner
On Fri, Aug 31, 2018 at 01:10:45AM -0400, Zygo Blaxell wrote: > On Thu, Aug 30, 2018 at 04:27:43PM +1000, Dave Chinner wrote: > > On Thu, Aug 23, 2018 at 08:58:49AM -0400, Zygo Blaxell wrote: > > > On Mon, Aug 20, 2018 at 08:33:49AM -0700, Darrick J. Wong wrote: > > > &

Re: [patch] file dedupe (and maybe clone) data corruption (was Re: [PATCH] generic: test for deduplication between different files)

2018-08-30 Thread Dave Chinner
On Thu, Aug 23, 2018 at 08:58:49AM -0400, Zygo Blaxell wrote: > On Mon, Aug 20, 2018 at 08:33:49AM -0700, Darrick J. Wong wrote: > > On Mon, Aug 20, 2018 at 11:09:32AM +1000, Dave Chinner wrote: > > > - is documenting rejection on request alignment grounds > > > (i

Re: [patch] file dedupe (and maybe clone) data corruption (was Re: [PATCH] generic: test for deduplication between different files)

2018-08-20 Thread Dave Chinner
On Mon, Aug 20, 2018 at 08:17:18PM -0500, Eric Sandeen wrote: > > > On 8/20/18 7:49 PM, Dave Chinner wrote: > > Upon successful completion of this ioctl, the number of > > bytes successfully deduplicated is returned in bytes_deduped > > and a status

Re: [patch] file dedupe (and maybe clone) data corruption (was Re: [PATCH] generic: test for deduplication between different files)

2018-08-20 Thread Dave Chinner
On Mon, Aug 20, 2018 at 08:33:49AM -0700, Darrick J. Wong wrote: > On Mon, Aug 20, 2018 at 11:09:32AM +1000, Dave Chinner wrote: > > So why was this dedupe request even accepted by the kernel? Well, > > I think there's a bug in the check just above this: > > > >

[patch] file dedupe (and maybe clone) data corruption (was Re: [PATCH] generic: test for deduplication between different files)

2018-08-19 Thread Dave Chinner
[cc linux-fsdevel now, too] On Mon, Aug 20, 2018 at 09:11:26AM +1000, Dave Chinner wrote: > [cc linux-...@vger.kernel.org] > > On Fri, Aug 17, 2018 at 09:39:24AM +0100, fdman...@kernel.org wrote: > > From: Filipe Manana > > > > Test that deduplication of an

Re: [PATCH] generic: test for deduplication between different files

2018-08-19 Thread Dave Chinner
t; +* > +11777540 ae ae ae ae ae ae ae ae > +11777550 > +File content after unmounting: > +000 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b > +* > +11467540 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b ae ae ae ae ae ae > +11467560 ae ae ae ae ae ae ae ae ae ae ae ae ae ae ae ae > +* > +11777540 ae ae ae ae ae ae ae ae > +11777550 > diff --git a/tests/generic/group b/tests/generic/group > index 55155de8..2ff1bd7e 100644 > --- a/tests/generic/group > +++ b/tests/generic/group > @@ -507,3 +507,4 @@ > 502 auto quick log > 503 auto quick dax punch collapse zero > 504 auto quick locks > +505 auto quick clone dedupe > -- > 2.11.0 > > -- Dave Chinner da...@fromorbit.com

Re: [PATCH V4] test online label ioctl

2018-05-17 Thread Dave Chinner
A slight change here to _require_xfs_io_command as well, so that tests > which simply fail with "Inappropriate ioctl" can be caught in the > common case. > > Signed-off-by: Eric Sandeen <sand...@redhat.com> > --- > > Now with new and improved sequential V4 vers

Re: [PATCH V3] test online label ioctl

2018-05-15 Thread Dave Chinner
put > > V3: lowercase local vars, simplify max label len function Looks good now, but I wondered about one thing the test doesn't cover: can you clear the label by setting it to a null string? i.e you check max length bounds, but don't check empty string behaviour... Cheers, Dave. -- Dave

Re: [PATCH V2] test online label ioctl

2018-05-14 Thread Dave Chinner
On Mon, May 14, 2018 at 06:26:07PM -0500, Eric Sandeen wrote: > On 5/14/18 6:11 PM, Dave Chinner wrote: > > On Mon, May 14, 2018 at 12:09:16PM -0500, Eric Sandeen wrote: > >> This tests the online label ioctl that btrfs has, which has been > >> recently propose

Re: [PATCH V2] test online label ioctl

2018-05-14 Thread Dave Chinner
+ > +# And that the it is still there when it's unmounted > +_scratch_unmount > +blkid -s LABEL $SCRATCH_DEV | _filter_scratch | sed -e "s/ $//g" Ok, so "LABEL" here is a special blkid match token > +# And that it persists after a remount > +_scratch_mount &

Re: [RFC][PATCH 0/76] vfs: 'views' for filesystems with more than one root

2018-05-09 Thread Dave Chinner
On Tue, May 08, 2018 at 10:06:44PM -0400, Jeff Mahoney wrote: > On 5/8/18 7:38 PM, Dave Chinner wrote: > > On Tue, May 08, 2018 at 11:03:20AM -0700, Mark Fasheh wrote: > >> Hi, > >> > >> The VFS's super_block covers a variety of filesystem functionality.

Re: [RFC][PATCH 0/76] vfs: 'views' for filesystems with more than one root

2018-05-08 Thread Dave Chinner
ields we need for a > subvolume namespace into their own structure. I'm not convinced yet - it still feels like it's the wrong layer to be solving the multiple namespace per superblock problem Cheers, Dave. -- Dave Chinner da...@fromorbit.com -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Symlink not persisted even after fsync

2018-04-16 Thread Dave Chinner
asn't actually guaranteed inode changes made prior to the fsync to be persistent on disk. i.e. that's a violation of ordered metadata semantics and probably a bug. Cheers, Dave. -- Dave Chinner da...@fromorbit.com -- To unsubscribe from this list: send the line "unsubscribe linux-btrf

Re: Symlink not persisted even after fsync

2018-04-14 Thread Dave Chinner
ilesystem or subsystem. There is no way this could possible trigger a directory fsync of the symlink parent, because the object being fsync()d may not even know what a filesystem is... If you want a symlink to have ordering behaviour like a dirent pointing to a regular file, then use hard links Cheers, Dave. -- Dave Chinner da...@fromorbit.com -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Symlink not persisted even after fsync

2018-04-13 Thread Dave Chinner
at. So when we look at symlinks, the parent directory has a ordering dependency with the symlink inode, not whatever is found by resolving the path in the symlink data. IOWs, there is no ordering relationship between the symlink's parent directory and whatever the symlink points at. i.e. it's a one-way r

Re: Symlink not persisted even after fsync

2018-04-13 Thread Dave Chinner
g strictly ordered metadata recovery semantics, so it should behave the same way as ext4 and XFS in tests like these. If it doesn't, then there's filesystem bugs that need fixing... Cheers, Dave. -- Dave Chinner da...@fromorbit.com -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: [PATCH v2 2/2] common/rc: raise mixed mode threshold to 1GB

2018-04-11 Thread Dave Chinner
PROG $MKFS_OPTIONS $mixed_opt -b $fssize $SCRATCH_DEV > ;; > jfs) Makes sense. Reviewed-by: Dave Chinner <dchin...@redhat.com> Cheers, Dave. -- Dave Chinner da...@fromorbit.com -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: [PATCH 2/2] generic/427: used mixed mode for Btrfs

2018-04-11 Thread Dave Chinner
> _scratch_mkfs_sized $((256 * 1024 * 1024)) >>$seqres.full 2>&1 But this uses a filesystem larger than the mixed mode threshold in _scratch_mkfs_sized(). Please update the generic threshold rather than special c

Re: [PATCH] fstests: generic test for fsync after fallocate

2018-04-09 Thread Dave Chinner
On Mon, Apr 09, 2018 at 11:00:52AM +0100, Filipe Manana wrote: > On Mon, Apr 9, 2018 at 10:51 AM, Dave Chinner <da...@fromorbit.com> wrote: > > On Sun, Apr 08, 2018 at 10:07:54AM +0800, Eryu Guan wrote: > >> On Thu, Apr 05, 2018 at 10:56:14PM +0100, fdman...@kernel.org wr

Re: [PATCH] fstests: generic test for fsync after fallocate

2018-04-09 Thread Dave Chinner
t;fsync" \ > > +$SCRATCH_MNT/baz You also cannot assume that two separate preallocations beyond EOF are going to be contiguous (i.e. it could be two separate extents. What you should just be checking is that there are extents allocated covering EOF to 3MB, not the exactl

Re: [PATCH v2 3/3] fstests: generic: Check the fs after each FUA writes

2018-03-28 Thread Dave Chinner
it was *completed*. If we've only replayed up to the FUA write with 1:63 in it, then no metadata writes should have been *issued* with 1:396 in it as the LSN that is stamped into metadata is only updated on log IO completion On first glance, this implies a bug in the underlying device write rep

Re: [PATCH v5 19/19] fs: handle inode->i_version more efficiently

2018-01-09 Thread Dave Chinner
lay...@redhat.com> > Reviewed-by: Jan Kara <j...@suse.cz> Documentation helps a lot in understanding all this. Thanks for adding it into the patch! Acked-by: Dave Chinner <dchin...@redhat.com> -- Dave Chinner da...@fromorbit.com -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: [PATCH v5 17/19] xfs: avoid setting XFS_ILOG_CORE if i_version doesn't need incrementing

2018-01-09 Thread Dave Chinner
.w...@oracle.com> Acked-by: Dave Chinner <dchin...@redhat.com> -- Dave Chinner da...@fromorbit.com -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: [PATCH v5 14/19] xfs: convert to new i_version API

2018-01-09 Thread Dave Chinner
On Tue, Jan 09, 2018 at 09:10:54AM -0500, Jeff Layton wrote: > From: Jeff Layton <jlay...@redhat.com> > > Signed-off-by: Jeff Layton <jlay...@redhat.com> > Acked-by: Darrick J. Wong <darrick.w...@oracle.com> Looks ok, but I haven't tested it at all. Acked-by: Dav

Re: [PATCH v3 06/10] writeback: introduce super_operations->write_metadata

2018-01-03 Thread Dave Chinner
On Wed, Jan 03, 2018 at 02:59:21PM +0100, Jan Kara wrote: > On Wed 03-01-18 13:32:19, Dave Chinner wrote: > > I think we could probably block ->write_metadata if necessary via a > > completion/wakeup style notification when a specific LSN is reached > > by the log

Re: [PATCH v3 06/10] writeback: introduce super_operations->write_metadata

2018-01-02 Thread Dave Chinner
On Tue, Jan 02, 2018 at 11:13:06AM -0500, Josef Bacik wrote: > On Wed, Dec 20, 2017 at 03:30:55PM +0100, Jan Kara wrote: > > On Wed 20-12-17 08:35:05, Dave Chinner wrote: > > > On Tue, Dec 19, 2017 at 01:07:09PM +0100, Jan Kara wrote: > > > > On Wed 13-12-1

Re: [PATCH v3 06/10] writeback: introduce super_operations->write_metadata

2017-12-19 Thread Dave Chinner
On Tue, Dec 19, 2017 at 01:07:09PM +0100, Jan Kara wrote: > On Wed 13-12-17 09:20:04, Dave Chinner wrote: > > On Tue, Dec 12, 2017 at 01:05:35PM -0500, Josef Bacik wrote: > > > On Tue, Dec 12, 2017 at 10:36:19AM +1100, Dave Chinner wrote: > > > > On Mon, Dec 11, 2

Re: [PATCH v3 19/19] fs: handle inode->i_version more efficiently

2017-12-18 Thread Dave Chinner
inode: inode to check > @@ -248,7 +250,7 @@ inode_query_iversion(struct inode *inode) > { > u64 cur, old, new; > > - cur = atomic64_read(>i_version); > + cur = inode_peek_iversion_raw(inode); > for (;;) { > /* If flag is already set, then

Re: [PATCH v2 01/19] fs: new API for handling inode->i_version

2017-12-16 Thread Dave Chinner
, so maybe it would be better to split relatively isolated functionality like this out while it's being reworked and you're already touching every file that uses it? Cheers, Dave. -- Dave Chinner da...@fromorbit.com -- To unsubscribe from this list: send the line "unsubscribe linux-btr

Re: [PATCH v3 06/10] writeback: introduce super_operations->write_metadata

2017-12-12 Thread Dave Chinner
On Tue, Dec 12, 2017 at 01:05:35PM -0500, Josef Bacik wrote: > On Tue, Dec 12, 2017 at 10:36:19AM +1100, Dave Chinner wrote: > > On Mon, Dec 11, 2017 at 04:55:31PM -0500, Josef Bacik wrote: > > > From: Josef Bacik <jba...@fb.com> > > > > > > Now that we

Re: [PATCH v4 72/73] xfs: Convert mru cache to XArray

2017-12-11 Thread Dave Chinner
On Mon, Dec 11, 2017 at 02:12:28PM -0800, Joe Perches wrote: > On Tue, 2017-12-12 at 08:43 +1100, Dave Chinner wrote: > > On Sat, Dec 09, 2017 at 09:00:18AM -0800, Joe Perches wrote: > > > On Sat, 2017-12-09 at 09:36 +1100, Dave Chinner wrote: > > > > 1. Usin

Re: [PATCH v3 06/10] writeback: introduce super_operations->write_metadata

2017-12-11 Thread Dave Chinner
their dirty metadata before sync() returns, even though it is not necessary to provide correct sync() semantics Mind you, writeback invocation is so convoluted now I could easily be mis-interpretting this code, but it does seem to me like this code is going to have some unintended behaviou

Re: [PATCH v4 72/73] xfs: Convert mru cache to XArray

2017-12-11 Thread Dave Chinner
On Sun, Dec 10, 2017 at 08:23:15PM -0800, Matthew Wilcox wrote: > On Mon, Dec 11, 2017 at 10:57:45AM +1100, Dave Chinner wrote: > > i.e. the fact the cmpxchg failed may not have anything to do with a > > race condtion - it failed because the slot wasn't empty like

Re: [PATCH v4 72/73] xfs: Convert mru cache to XArray

2017-12-11 Thread Dave Chinner
On Sat, Dec 09, 2017 at 09:00:18AM -0800, Joe Perches wrote: > On Sat, 2017-12-09 at 09:36 +1100, Dave Chinner wrote: > > 1. Using lockdep_set_novalidate_class() for anything other > > than device->mutex will throw checkpatch warnings. Nice. (*) > [] > > (*)

Re: [PATCH v4 72/73] xfs: Convert mru cache to XArray

2017-12-10 Thread Dave Chinner
On Fri, Dec 08, 2017 at 03:01:31PM -0800, Matthew Wilcox wrote: > On Thu, Dec 07, 2017 at 11:38:43AM +1100, Dave Chinner wrote: > > > > cmpxchg is for replacing a known object in a store - it's not really > > > > intended for doing initial inserts after a lookup tells

Re: Lockdep is less useful than it was

2017-12-08 Thread Dave Chinner
ructure just to add lockdep validation to a tree that doesn't actually need any extra locking validation... Cheers, Dave. -- Dave Chinner da...@fromorbit.com -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.ke

Re: [PATCH v4 72/73] xfs: Convert mru cache to XArray

2017-12-08 Thread Dave Chinner
Who-ever adds semaphore checking to lockdep can add those annotations. The externalisation of the development cost of new lockdep functionality is one of the problems here. -Dave. (*) checkpatch.pl is considered mostly harmful round here, too, but that's another rant (**) the frequent occurren

Re: [PATCH v4 72/73] xfs: Convert mru cache to XArray

2017-12-07 Thread Dave Chinner
On Fri, Dec 08, 2017 at 01:45:52PM +0900, Byungchul Park wrote: > On Fri, Dec 08, 2017 at 09:22:16AM +1100, Dave Chinner wrote: > > On Thu, Dec 07, 2017 at 11:06:34AM -0500, Theodore Ts'o wrote: > > > On Wed, Dec 06, 2017 at 06:06:48AM -0800, Matthew Wilcox wrote: > > >

Re: Lockdep is less useful than it was

2017-12-07 Thread Dave Chinner
problem, you'd be happier, right? I'd be much happier if it wasn't turned on by default in the first place. We gave plenty of warnings that there were still unsolved false positive problems with the new checks in the storage stack. Cheers, Dave. -- Dave Chinner da...@fromorbit.

Re: [PATCH v4 72/73] xfs: Convert mru cache to XArray

2017-12-07 Thread Dave Chinner
hich means it has zero coverage of the entire XFS metadata buffer subsystem and the complex locking orders we have for metadata updates. Put simply: lockdep doesn't provide me with any benefit, so I don't use it... Cheers, Dave. -- Dave Chinner da...@fromorbit.com -- To unsubscribe fro

Re: [PATCH v4 72/73] xfs: Convert mru cache to XArray

2017-12-06 Thread Dave Chinner
On Wed, Dec 06, 2017 at 06:06:48AM -0800, Matthew Wilcox wrote: > On Wed, Dec 06, 2017 at 07:44:04PM +1100, Dave Chinner wrote: > > On Tue, Dec 05, 2017 at 08:45:49PM -0800, Matthew Wilcox wrote: > > > That said, using xa_cmpxchg() in the dquot code looked like the right > &g

Re: [PATCH v4 72/73] xfs: Convert mru cache to XArray

2017-12-06 Thread Dave Chinner
On Tue, Dec 05, 2017 at 08:45:49PM -0800, Matthew Wilcox wrote: > On Wed, Dec 06, 2017 at 02:14:56PM +1100, Dave Chinner wrote: > > > The other conversions use the normal API instead of the advanced API, so > > > all of this gets hidden away. For example, the inode cache d

Re: [PATCH v4 72/73] xfs: Convert mru cache to XArray

2017-12-05 Thread Dave Chinner
On Tue, Dec 05, 2017 at 06:02:08PM -0800, Matthew Wilcox wrote: > On Wed, Dec 06, 2017 at 12:36:48PM +1100, Dave Chinner wrote: > > > - if (radix_tree_preload(GFP_NOFS)) > > > - return -ENOMEM; > > > - > > > INIT_LIST_HEAD(>list_node); > >

Re: [PATCH v4 00/73] XArray version 4

2017-12-05 Thread Dave Chinner
On Tue, Dec 05, 2017 at 06:05:15PM -0800, Matthew Wilcox wrote: > On Wed, Dec 06, 2017 at 12:45:49PM +1100, Dave Chinner wrote: > > On Tue, Dec 05, 2017 at 04:40:46PM -0800, Matthew Wilcox wrote: > > > From: Matthew Wilcox <mawil...@microsoft.com> > > >

Re: [PATCH v4 00/73] XArray version 4

2017-12-05 Thread Dave Chinner
his time. -Dave. > > > -Original Message----- > > From: Dave Chinner [mailto:da...@fromorbit.com] > > Sent: Tuesday, December 5, 2017 8:51 PM > > To: Matthew Wilcox <wi...@infradead.org> > > Cc: Matthew Wilcox <mawil...@microsoft.com>; Ross Zwisler > >

Re: [PATCH v4 00/73] XArray version 4

2017-12-05 Thread Dave Chinner
On Wed, Dec 06, 2017 at 12:45:49PM +1100, Dave Chinner wrote: > On Tue, Dec 05, 2017 at 04:40:46PM -0800, Matthew Wilcox wrote: > > From: Matthew Wilcox <mawil...@microsoft.com> > > > > I looked through some notes and decided this was version 4 of the XArray. &g

Re: [PATCH v4 00/73] XArray version 4

2017-12-05 Thread Dave Chinner
On Tue, Dec 05, 2017 at 04:40:46PM -0800, Matthew Wilcox wrote: > From: Matthew Wilcox <mawil...@microsoft.com> > > I looked through some notes and decided this was version 4 of the XArray. > Last posted two weeks ago, this version includes a *lot* of changes. > I'd like

Re: [PATCH v4 72/73] xfs: Convert mru cache to XArray

2017-12-05 Thread Dave Chinner
tions. Turning that around so that a larger XFS structure and algorithm is now protected by an opaque internal lock from generic storage structure the forms part of the larger structure seems like a bad design pattern to me... Cheers, Dave. -- Dave Chinner da...@fromorbit.com -- To unsu

Re: [PATCH v2] iomap: report collisions between directio and buffered writes to userspace

2017-11-21 Thread Dave Chinner
On Tue, Nov 21, 2017 at 04:52:53AM -0800, Matthew Wilcox wrote: > On Tue, Nov 21, 2017 at 05:48:15PM +1100, Dave Chinner wrote: > > On Mon, Nov 20, 2017 at 08:32:40PM -0800, Matthew Wilcox wrote: > > > On Mon, Nov 20, 2017 at 05:37:53PM -0800, Darrick J. Wong wrote: > >

Re: [PATCH v2] iomap: report collisions between directio and buffered writes to userspace

2017-11-20 Thread Dave Chinner
On Mon, Nov 20, 2017 at 08:32:40PM -0800, Matthew Wilcox wrote: > On Mon, Nov 20, 2017 at 05:37:53PM -0800, Darrick J. Wong wrote: > > On Tue, Nov 21, 2017 at 09:27:49AM +1100, Dave Chinner wrote: > > > First thing I noticed was that "xa" as a prefix is already qu

Re: [PATCH v2] iomap: report collisions between directio and buffered writes to userspace

2017-11-20 Thread Dave Chinner
On Mon, Nov 20, 2017 at 01:51:00PM -0800, Matthew Wilcox wrote: > On Tue, Nov 21, 2017 at 07:26:06AM +1100, Dave Chinner wrote: > > On Mon, Nov 20, 2017 at 08:18:29AM -0800, Matthew Wilcox wrote: > > > On Fri, Nov 17, 2017 at 11:39:25AM -0800, Darrick J. Wong wrote: >

Re: [PATCH v2] iomap: report collisions between directio and buffered writes to userspace

2017-11-20 Thread Dave Chinner
solving this problem. The XArray is going to introduce a set > of entries which can be stored to locations in the page cache that I'm > calling 'wait entries'. What's this XArray thing you speak of? Cheers, Dave. -- Dave Chinner da...@fromorbit.com -- To unsubscribe from this list: send the line &q

Re: [PATCH 2/6] writeback: allow for dirty metadata accounting

2017-11-09 Thread Dave Chinner
ny thoughts about how we could efficiently support accounting for variable sized, non-page based metadata with this generic infrastructure? Cheers, Dave. -- Dave Chinner da...@fromorbit.com -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to ma

Re: [ANNOUNCE] fsperf: a simple fs/block performance testing framework

2017-10-09 Thread Dave Chinner
On Mon, Oct 09, 2017 at 09:00:51AM -0400, Josef Bacik wrote: > On Mon, Oct 09, 2017 at 04:17:31PM +1100, Dave Chinner wrote: > > On Sun, Oct 08, 2017 at 10:25:10PM -0400, Josef Bacik wrote: > > > > Integrating into fstests means it will be immediately available to > > &

Re: [ANNOUNCE] fsperf: a simple fs/block performance testing framework

2017-10-08 Thread Dave Chinner
On Sun, Oct 08, 2017 at 10:25:10PM -0400, Josef Bacik wrote: > On Mon, Oct 09, 2017 at 11:51:37AM +1100, Dave Chinner wrote: > > On Fri, Oct 06, 2017 at 05:09:57PM -0400, Josef Bacik wrote: > > > Hello, > > > > > > One thing that comes up a lot every LSF is the

Re: [ANNOUNCE] fsperf: a simple fs/block performance testing framework

2017-10-08 Thread Dave Chinner
omparison options so you can compare against averages of all > previous runs and such. Yup, that fits exactly into what fstests is for... :P Integrating into fstests means it will be immediately available to all fs developers, it'll run on everything that everyone already has setup for filesystem te

Re: Something like ZFS Channel Programs for BTRFS & probably XFS or even VFS?

2017-10-03 Thread Dave Chinner
On Tue, Oct 03, 2017 at 01:40:51PM -0700, Matthew Wilcox wrote: > On Wed, Oct 04, 2017 at 07:10:35AM +1100, Dave Chinner wrote: > > On Tue, Oct 03, 2017 at 03:19:18PM +0200, Martin Steigerwald wrote: > > > [repost. I didn´t notice autocompletion gave me wrong address

Re: Something like ZFS Channel Programs for BTRFS & probably XFS or even VFS?

2017-10-03 Thread Dave Chinner
On Tue, Oct 03, 2017 at 03:19:18PM +0200, Martin Steigerwald wrote: > [repost. I didn´t notice autocompletion gave me wrong address for fsdevel, > blacklisted now] > > Hello. > > What do you think of > > http://open-zfs.org/wiki/Projects/ZFS_Channel_Programs Domain not

Re: [PATCH] fstests: btrfs/150 regression test for reading compressed data

2017-09-20 Thread Dave Chinner
ure > + # invalidate the page cache > + $XFS_IO_PROG -f -c "fadvise -d 0 128K" $SCRATCH_MNT/foobar | > _filter_xfs_io > + > + enable_io_failure > + od -x $SCRATCH_MNT/foobar > /dev/null & why are you using od to read the data when the output is piped

Re: [RFC PATCH v1 00/30] fs: inode->i_version rework and optimization

2017-04-04 Thread Dave Chinner
On Mon, Apr 03, 2017 at 04:00:55PM +0200, Jan Kara wrote: > On Sun 02-04-17 09:05:26, Dave Chinner wrote: > > On Thu, Mar 30, 2017 at 12:12:31PM -0400, J. Bruce Fields wrote: > > > On Thu, Mar 30, 2017 at 07:11:48AM -0400, Jeff Layton wrote: > > > > On Thu, 2017-

Re: [RFC PATCH v1 00/30] fs: inode->i_version rework and optimization

2017-04-01 Thread Dave Chinner
was a crash at mount time because their architecture always leaves a consistent filesystem on disk (e.g. COW filesystems) > I wonder if repeated crashes can lead to any odd corner cases. WIthout defined, locked down behavour of the superblock counter, the almost certainly corner cases

Re: [RFC PATCH v1 00/30] fs: inode->i_version rework and optimization

2017-03-29 Thread Dave Chinner
that entails), by the time the statx syscall returns to userspace the i_version value may not match the data/metadata in the inode(*). IOWs, by the time i_version gets to userspace, it is out of date and any use of it for data versioning from userspace is going to be prone to race conditions. Cheers, Dave. (*) fiemap has exactly the same "stale the moment internal fs locks are released" race conditions, which is why it cannot safely be used for mapping holes when copying file data -- Dave Chinner da...@fromorbit.com -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: [RFC PATCH v1 00/30] fs: inode->i_version rework and optimization

2017-03-21 Thread Dave Chinner
ccessing and requiring synchronisation. > Not sure how big a problem that really is. This coherency problem has always existed on the server side... Cheers, Dave. -- Dave Chinner da...@fromorbit.com -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: [PATCH 5/8] nowait aio: return on congested block device

2017-03-16 Thread Dave Chinner
g/ on EAGAIN. Why isn't the congestion check at a higher layer like we do for page cache readahead? i.e. using the bdi*congested() API at the time we are doing all the other filesystem blocking checks. -Dave. -- Dave Chinner da...@fromorbit.com -- To unsubscribe from this list: send the line

Re: [PATCH 4/6] xfs: use memalloc_nofs_{save,restore} instead of memalloc_noio*

2017-02-06 Thread Dave Chinner
caller paths is done and we're 100% certain that there are no lurking deadlocks. For example, I'm pretty sure we can call into _xfs_buf_map_pages() outside of a transaction context but with an inode ILOCK held exclusively. If we then recurse into memory reclaim and try to run a transaction during

Re: [PATCH 2/9] xfs: introduce and use KM_NOLOCKDEP to silence reclaim lockdep false positives

2016-12-20 Thread Dave Chinner
On Mon, Dec 19, 2016 at 02:06:19PM -0800, Darrick J. Wong wrote: > On Tue, Dec 20, 2016 at 08:24:13AM +1100, Dave Chinner wrote: > > On Thu, Dec 15, 2016 at 03:07:08PM +0100, Michal Hocko wrote: > > > From: Michal Hocko <mho...@suse.com> > > > > &g

Re: [PATCH 2/9] xfs: introduce and use KM_NOLOCKDEP to silence reclaim lockdep false positives

2016-12-19 Thread Dave Chinner
es which audits and changes all the unnecessary KM_NOFS allocations in one go. I've never liked whack-a-mole style changes like this - do it once, do it properly Cheers, Dave. -- Dave Chinner da...@fromorbit.com -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs

Re: [PATCH 1/6] Btrfs: add mount option for dax

2016-12-09 Thread Dave Chinner
On Fri, Dec 09, 2016 at 10:41:34AM -0800, Liu Bo wrote: > On Fri, Dec 09, 2016 at 03:47:20PM +1100, Dave Chinner wrote: > > On Wed, Dec 07, 2016 at 01:45:05PM -0800, Liu Bo wrote: > > > Signed-off-by: Liu Bo <bo.li@oracle.com> > > > --- > > > fs/bt

Re: [PATCH 4/6] Btrfs: add DAX support for nocow btrfs

2016-12-08 Thread Dave Chinner
hitecture is correctly structured. i.e DAX should be able to work even with most of btrfs's special magic still enabled. Cheers, Dave. -- Dave Chinner da...@fromorbit.com -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vge

Re: [PATCH 1/6] Btrfs: add mount option for dax

2016-12-08 Thread Dave Chinner
meant to be a permanent thing. The permanent functionality for DAX is supposed to be per-inode inheritable DAX flags - not mount options - so that applications can choose on a per file basis to enable/disable DAX access as they see fit. This also enables the filesystem to reject the atte

Re: [PATCH 01/10] fstests: common: Introduce function to check qgroup correctness

2016-11-29 Thread Dave Chinner
On Wed, Nov 30, 2016 at 08:56:03AM +0800, Qu Wenruo wrote: > > > At 11/30/2016 05:01 AM, Dave Chinner wrote: > >On Tue, Nov 29, 2016 at 03:32:54PM +0800, Qu Wenruo wrote: > >>Old btrfs qgroup test cases uses fix golden output numbers, which limits > >>the cover

Re: [PATCH 01/10] fstests: common: Introduce function to check qgroup correctness

2016-11-29 Thread Dave Chinner
rnel/progs combinations... > + > +_btrfs_check_scratch_qgroup() > +{ > + _require_btrfs_check_qgroup This needs to go in the test itself before the test is run, not get hidden in a function call at the end of the test. Cheers, Dave. -- Dave Chinner da...@fromorbit.com -- To u

Re: [RFC PATCH 0/2] Btrfs: make a source length of 0 imply EOF for dedupe

2016-11-23 Thread Dave Chinner
On Wed, Nov 23, 2016 at 06:14:47PM -0500, Zygo Blaxell wrote: > On Thu, Nov 24, 2016 at 09:13:28AM +1100, Dave Chinner wrote: > > On Wed, Nov 23, 2016 at 08:55:59AM -0500, Zygo Blaxell wrote: > > > On Wed, Nov 23, 2016 at 03:26:32PM +1100, Dave Chinner wrote: > > > >

Re: [RFC PATCH 0/2] Btrfs: make a source length of 0 imply EOF for dedupe

2016-11-23 Thread Dave Chinner
On Wed, Nov 23, 2016 at 08:55:59AM -0500, Zygo Blaxell wrote: > On Wed, Nov 23, 2016 at 03:26:32PM +1100, Dave Chinner wrote: > > On Tue, Nov 22, 2016 at 09:02:10PM -0500, Zygo Blaxell wrote: > > > On Thu, Nov 17, 2016 at 04:07:48PM -0800, Omar Sandoval wrote: > > >

Re: [RFC PATCH 0/2] Btrfs: make a source length of 0 imply EOF for dedupe

2016-11-22 Thread Dave Chinner
t the limit is, other than by trial > and error. It's not even in a header file, userspace just has to *know*. So add a define to the API to make it visible to applications and document it in the man page. Cheers, Dave. -- Dave Chinner da...@fromorbit.com -- To unsubscribe from this list: send the

Re: [PATCH] fstests: Block btrfs from test case generic/372

2016-11-16 Thread Dave Chinner
thin_tolerance range would mean the test would validate file1 on all reflink supporting filesystems and we don't need to exclude btrfs at all... Cheers, Dave. -- Dave Chinner da...@fromorbit.com -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: [PATCH] fstests: btrfs: Check false ENOSPC bug caused by incorrect metadata reserve

2016-11-09 Thread Dave Chinner
On Thu, Nov 10, 2016 at 11:24:34AM +0800, Eryu Guan wrote: > On Thu, Nov 10, 2016 at 10:42:36AM +0800, Qu Wenruo wrote: > > > > > > At 11/10/2016 10:19 AM, Dave Chinner wrote: > > > On Thu, Nov 10, 2016 at 08:34:20AM +0800, Qu Wenruo wrote: > > > &g

Re: [PATCH] fstests: btrfs: Check false ENOSPC bug caused by incorrect metadata reserve

2016-11-09 Thread Dave Chinner
tirity every day on every change I make or review, so I'm not suggesting that you should do anything I don't already do. Cheers, Dave. -- Dave Chinner da...@fromorbit.com -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: [PATCH] generic: create and delete files repeatedly to exercise ENOSPC behaviour

2016-11-01 Thread Dave Chinner
rm -f $testfile1 $testfile2 > + done > +} What are you trying to test here that other ENOSPC tests don't exercise? Why cant you just use preallocation to trigger ENOSPC repeatedly instead of writing data? That would allow multiple iterations per second, not one every few minutes... Che

Re: [PATCH v4] generic: make 17[1-4] work well when btrfs compression is enabled

2016-11-01 Thread Dave Chinner
t's included from common/populate. So there is no reason to put it in common/rc. Cheers, Dave. -- Dave Chinner da...@fromorbit.com -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at

Re: [PATCH 5/5] fs: don't set *REFERENCED unless we are on the lru list

2016-10-27 Thread Dave Chinner
On Thu, Oct 27, 2016 at 09:13:44AM -0400, Josef Bacik wrote: > On 10/26/2016 08:30 PM, Dave Chinner wrote: > >On Wed, Oct 26, 2016 at 11:11:35AM -0400, Josef Bacik wrote: > >>On 10/25/2016 06:01 PM, Dave Chinner wrote: > >>>On Tue, Oct 25, 2016 at 02:41

Re: bio linked list corruption.

2016-10-26 Thread Dave Chinner
and to all slip through the ENOSPC detection without the correct metadata reservations and all require multiple metadata blocks to be allocated durign writeback... If you've got a way to trigger it quickly and reliably, that would be helpful... Cheers, Dave. -- Dave Chinner da...@fromorbit.

Re: [PATCH 5/5] fs: don't set *REFERENCED unless we are on the lru list

2016-10-26 Thread Dave Chinner
On Wed, Oct 26, 2016 at 11:11:35AM -0400, Josef Bacik wrote: > On 10/25/2016 06:01 PM, Dave Chinner wrote: > >On Tue, Oct 25, 2016 at 02:41:44PM -0400, Josef Bacik wrote: > >>With anything that populates the inode/dentry cache with a lot of one time > >>use > &g

Re: [PATCH 5/5] fs: don't set *REFERENCED unless we are on the lru list

2016-10-26 Thread Dave Chinner
On Wed, Oct 26, 2016 at 04:03:54PM -0400, Josef Bacik wrote: > On 10/25/2016 07:36 PM, Dave Chinner wrote: > >So, 2-way has not improved. If changing referenced behaviour was an > >obvious win for btrfs, we'd expect to see that here as well. > >however, because 4-way improve

Re: [PATCH 5/5] fs: don't set *REFERENCED unless we are on the lru list

2016-10-25 Thread Dave Chinner
On Wed, Oct 26, 2016 at 09:01:13AM +1100, Dave Chinner wrote: > On Tue, Oct 25, 2016 at 02:41:44PM -0400, Josef Bacik wrote: > > With anything that populates the inode/dentry cache with a lot of one time > > use > > inodes we can really put a lot of pressure on the syste

Re: [PATCH 5/5] fs: don't set *REFERENCED unless we are on the lru list

2016-10-25 Thread Dave Chinner
or XFS and ~1.5% for ext4 is well within the run-to-run variation of fsmark. It looks like it might be slightly faster, but it's not a cut-and-dried win for anything other than btrfs. Cheers, Dave. -- Dave Chinner da...@fromorbit.com -- To unsubscribe from this list: send the line "uns

Re: [PATCH v2] generic: make 17[1-4] work well when btrfs compression is enabled

2016-10-10 Thread Dave Chinner
of any significant size, especially as the data will compress down to nearly nothing. Trying to hack around compression artifacts by inflating the size of the file just doesn't work reliably. The way to fix this is to either use one of the "fill filesystem" functions that isn't aff

Re: [PATCH] generic/175: disable inline data feature for btrfs

2016-10-10 Thread Dave Chinner
re are specific mount options that needs to be avoided, then add an option to filter them out. e.g. something like this: _scratch_options_filter btrfs compress so that it removes any compression option from the btrfs mount/mkfs that is run for that test. Cheers, Dave. -- Dave Chinner da...@fromorbit.

Re: Is is possible to submit binary image as fstest test case?

2016-10-09 Thread Dave Chinner
On Fri, Oct 07, 2016 at 06:05:51PM +0200, David Sterba wrote: > On Fri, Oct 07, 2016 at 08:18:38PM +1100, Dave Chinner wrote: > > On Thu, Oct 06, 2016 at 04:12:56PM +0800, Qu Wenruo wrote: > > > So I'm wondering if I can just upload a zipped raw image as part of &

Re: Making statfs return qgroup info (for container use case etc.)

2016-10-09 Thread Dave Chinner
On Fri, Oct 07, 2016 at 06:58:47PM +0200, David Sterba wrote: > On Fri, Oct 07, 2016 at 09:40:11AM +1100, Dave Chinner wrote: > > XFS does this with directory tree quotas. It was implmented 10 years > > ago or so, IIRC... > > Sometimes, the line between a historical remark

Re: [PATCH] generic: make 17[1-4] work well when btrfs compression is enabled

2016-10-07 Thread Dave Chinner
$seqres.full 2>&1 > +dd if=/dev/zero of=$testdir/eat_my_space >> $seqres.full 2>&1 Please don't replace xfs_io writes using a specific data pattern with dd calls that write zeros. Indeed, we don't use dd for new tests anymore - xfs_io should be used. Write a function that fi

Re: Is is possible to submit binary image as fstest test case?

2016-10-07 Thread Dave Chinner
On Fri, Oct 07, 2016 at 05:26:27PM +0800, Qu Wenruo wrote: > > > At 10/07/2016 05:18 PM, Dave Chinner wrote: > >On Thu, Oct 06, 2016 at 04:12:56PM +0800, Qu Wenruo wrote: > >>Hi, > >> > >>Just as the title says, for some case(OK, btrfs again) we need to

Re: Is is possible to submit binary image as fstest test case?

2016-10-07 Thread Dave Chinner
hooks are for. The XFS kernel code has both, xfstests use both, and they pretty much do away with the need for custom binary filesystem images for such testing... Cheers, Dave. -- Dave Chinner da...@fromorbit.com -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs&

Re: Making statfs return qgroup info (for container use case etc.)

2016-10-06 Thread Dave Chinner
ing in an option at mount time - such as qgroup level > maybe?) , instead of the global filesystem data in f_bfree f_blocks etc. XFS does this with directory tree quotas. It was implmented 10 years ago or so, IIRC... Cheers, Dave. -- Dave Chinner da...@fromorbit.com -- To unsubscribe from this list:

Re: [RFC] Preliminary BTRFS Encryption

2016-09-15 Thread Dave Chinner
odels should you then implement something that is unique to btrfs Cheers, Dave. -- Dave Chinner da...@fromorbit.com -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: [PATCH v6 1/6] fstests: common: Introduce _post_mount_hook for btrfs

2016-09-14 Thread Dave Chinner
E_BTRFS_FEATURE; do What's this magic, undefined, undocumented variable? Cheers, Dave. -- Dave Chinner da...@fromorbit.com -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: [PATCH 2/3] writeback: allow for dirty metadata accounting

2016-09-12 Thread Dave Chinner
e writeback implementations similar to the way memcg is completely hidden from the shrinker reclaim implementations... Cheers, Dave. -- Dave Chinner da...@fromorbit.com -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: [PATCH 2/3] writeback: allow for dirty metadata accounting

2016-09-12 Thread Dave Chinner
Mon 12-09-16 10:46:56, Dave Chinner wrote: > >>On Fri, Sep 09, 2016 at 10:17:43AM +0200, Jan Kara wrote: > >>>On Mon 22-08-16 13:35:01, Josef Bacik wrote: > >>>>Provide a mechanism for file systems to indicate how much dirty metadata > >>>>they >

  1   2   3   4   5   >