[PATCH 57/76] fs/orangefs: Use inode_sb() helper instead of inode->i_sb

Signed-off-by: Mark Fasheh <mfas...@suse.de> --- fs/orangefs/file.c| 2 +- fs/orangefs/namei.c | 12 fs/orangefs/orangefs-kernel.h | 8 3 files changed, 13 insertions(+), 9 deletions(-) diff --git a/fs/orangefs/file.c b/fs/orangefs/file.c

[PATCH 59/76] fs/proc: Use inode_sb() helper instead of inode->i_sb

Signed-off-by: Mark Fasheh <mfas...@suse.de> --- fs/proc/array.c | 2 +- fs/proc/base.c| 22 -- fs/proc/fd.c | 4 ++-- fs/proc/generic.c | 2 +- fs/proc/namespaces.c | 2 +- fs/proc/nommu.c | 2 +- fs/proc/proc_sysctl.c | 4 ++-- f

[PATCH 60/76] fs/qnx4: Use inode_sb() helper instead of inode->i_sb

Signed-off-by: Mark Fasheh <mfas...@suse.de> --- fs/qnx4/dir.c | 2 +- fs/qnx4/inode.c | 4 ++-- fs/qnx4/namei.c | 4 ++-- 3 files changed, 5 insertions(+), 5 deletions(-) diff --git a/fs/qnx4/dir.c b/fs/qnx4/dir.c index a6ee23aadd28..c0e764ce79dd 100644 --- a/fs/qnx4/dir.c +++ b/fs/qnx4

[PATCH 61/76] fs/qnx6: Use inode_sb() helper instead of inode->i_sb

Signed-off-by: Mark Fasheh <mfas...@suse.de> --- fs/qnx6/dir.c | 8 fs/qnx6/inode.c | 4 ++-- fs/qnx6/namei.c | 2 +- 3 files changed, 7 insertions(+), 7 deletions(-) diff --git a/fs/qnx6/dir.c b/fs/qnx6/dir.c index c1cfb8a19e9d..655d0eb9d82a 100644 --- a/fs/qnx6/dir.c +++ b/f

[PATCH 62/76] fs/quota: Use inode_sb() helper instead of inode->i_sb

Signed-off-by: Mark Fasheh <mfas...@suse.de> --- fs/quota/dquot.c | 30 +++--- 1 file changed, 15 insertions(+), 15 deletions(-) diff --git a/fs/quota/dquot.c b/fs/quota/dquot.c index 020c597ef9b6..ba6d549323cb 100644 --- a/fs/quota/dquot.c +++ b/fs/quota/d

[PATCH 64/76] fs/read: Use inode_sb() helper instead of inode->i_sb

Signed-off-by: Mark Fasheh <mfas...@suse.de> --- fs/read_write.c | 11 ++- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/fs/read_write.c b/fs/read_write.c index f8547b82dfb3..cf9900707558 100644 --- a/fs/read_write.c +++ b/fs/read_write.c @@ -146,7 +146,7 @@

[PATCH 67/76] fs/squashfs: Use inode_sb() helper instead of inode->i_sb

Signed-off-by: Mark Fasheh <mfas...@suse.de> --- fs/squashfs/dir.c | 26 ++ fs/squashfs/export.c | 2 +- fs/squashfs/file.c| 34 ++ fs/squashfs/file_cache.c | 5 +++-- fs/squashfs/file_direct.c | 9 +--

[PATCH 63/76] fs/ramfs: Use inode_sb() helper instead of inode->i_sb

Signed-off-by: Mark Fasheh <mfas...@suse.de> --- fs/ramfs/inode.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/fs/ramfs/inode.c b/fs/ramfs/inode.c index 11201b2d06b9..57b78ae51ed1 100644 --- a/fs/ramfs/inode.c +++ b/fs/ramfs/inode.c @@ -101,7 +101,7 @@ struct

[PATCH 68/76] fs/sysv: Use inode_sb() helper instead of inode->i_sb

Signed-off-by: Mark Fasheh <mfas...@suse.de> --- fs/sysv/dir.c| 12 ++-- fs/sysv/ialloc.c | 8 fs/sysv/inode.c | 6 +++--- fs/sysv/itree.c | 29 +++-- fs/sysv/namei.c | 4 ++-- 5 files changed, 30 insertions(+), 29 deletions(-) diff --gi

[PATCH 66/76] fs/romfs: Use inode_sb() helper instead of inode->i_sb

Signed-off-by: Mark Fasheh <mfas...@suse.de> --- fs/romfs/mmap-nommu.c | 4 ++-- fs/romfs/super.c | 24 +--- 2 files changed, 15 insertions(+), 13 deletions(-) diff --git a/fs/romfs/mmap-nommu.c b/fs/romfs/mmap-nommu.c index 1118a0dc6b45..0dbf9be30283 100644 --

[PATCH 69/76] fs/ubifs: Use inode_sb() helper instead of inode->i_sb

Signed-off-by: Mark Fasheh <mfas...@suse.de> --- fs/ubifs/crypto.c | 4 ++-- fs/ubifs/dir.c| 30 +++--- fs/ubifs/file.c | 42 +- fs/ubifs/ioctl.c | 4 ++-- fs/ubifs/super.c | 4 ++-- fs/ubifs/xattr.c | 10 +--

[PATCH 70/76] fs/udf: Use inode_sb() helper instead of inode->i_sb

Signed-off-by: Mark Fasheh <mfas...@suse.de> --- fs/udf/dir.c | 2 +- fs/udf/directory.c | 30 fs/udf/file.c | 6 +- fs/udf/ialloc.c| 24 +++--- fs/udf/inode.c | 209 +++-- fs/udf/misc.c | 4 +-

[PATCH 71/76] fs/ufs: Use inode_sb() helper instead of inode->i_sb

Signed-off-by: Mark Fasheh <mfas...@suse.de> --- fs/ufs/balloc.c | 23 --- fs/ufs/dir.c| 34 +- fs/ufs/ialloc.c | 4 ++-- fs/ufs/inode.c | 37 +++-- fs/ufs/namei.c | 6 +++--- 5 files chang

[PATCH 73/76] vfs: Move s_dev to to struct fs_view

dev and removes s_dev from struct super_block. I also include a helper, inode_view() to make referencing inode->i_view fields less clunky. Signed-off-by: Mark Fasheh <mfas...@suse.de> --- include/linux/fs.h | 7 ++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/include/linux/fs

[PATCH 72/76] fs/xfs: Use inode_sb() helper instead of inode->i_sb

Signed-off-by: Mark Fasheh <mfas...@suse.de> --- fs/xfs/xfs_acl.c| 2 +- fs/xfs/xfs_aops.c | 4 ++-- fs/xfs/xfs_export.c | 4 ++-- fs/xfs/xfs_file.c | 10 - fs/xfs/xfs_ioctl.c | 8 +++ fs/xfs/xfs_iops.c | 6 ++--- fs/xfs/xfs_pnfs.c | 2 +- fs/xfs/xfs_trace.h

[PATCH 74/76] fs: Use fs_view device from struct inode.

Replace calls of inode_sb(inode)->s_dev with inode_view(inode)->v_dev. Signed-off-by: Mark Fasheh <mfas...@suse.de> --- arch/arc/kernel/troubleshoot.c | 2 +- drivers/staging/lustre/lustre/llite/dir.c | 2 +- drivers/staging/lustre/lustre/llite/file.c | 2 +- fs

[PATCH 76/76] btrfs: Use fs_view in roots, point inodes to it

ded in their owning root. Signed-off-by: Mark Fasheh <mfas...@suse.de> --- fs/btrfs/ctree.h | 7 ++- fs/btrfs/disk-io.c | 14 -- fs/btrfs/disk-io.h | 2 +- fs/btrfs/inode.c | 5 +++-- fs/btrfs/root-tree.c | 2 +- 5 files changed, 15 insertions(+), 15 deletions(-) diff -

[PATCH 75/76] fs: Use fs view device from struct super_block

We have some places which access s_dev directly from struct super_block. Convert those to get v_dev from the default super block view. Signed-off-by: Mark Fasheh <mfas...@suse.de> --- drivers/mtd/mtdsuper.c | 2 +- drivers/staging/lustre/lustre/llite/llite_lib.

Re: Announcing btrfs-dedupe

On Mon, Nov 7, 2016 at 6:17 PM, Darrick J. Wong wrote: > On Mon, Nov 07, 2016 at 09:54:09PM +0100, Adam Borowski wrote: >> Mark has already included XFS in documentation of duperemove, all that looks >> amiss is btrfs-extent-same having an obsolete name. But then, I

Re: Announcing btrfs-dedupe

On Mon, Nov 7, 2016 at 6:40 PM, Christoph Anton Mitterer wrote: > On Mon, 2016-11-07 at 15:02 +0100, David Sterba wrote: >> I think adding a whole-file dedup mode to duperemove would be better >> (from user's POV) than writing a whole new tool > > What would IMO be really

Re: Announcing btrfs-dedupe

Hi James, Re the following text on your project page: "IMPORTANT CAVEAT — I have read that there are race and/or error conditions which can cause filesystem corruption in the kernel implementation of the deduplication ioctl." Can you expound on that? I'm not aware of any bugs right now but if

Re: Announcing btrfs-dedupe

Hi David and James, On Mon, Nov 7, 2016 at 6:02 AM, David Sterba wrote: > On Sun, Nov 06, 2016 at 02:30:52PM +0100, James Pharaoh wrote: >> I'm pleased to announce my btrfs deduplication utility, written in Rust. >> This operates on whole files, is fast, and I believe

Re: [PATCH v13 00/15] Btrfs In-band De-duplication

ed to recent infrastructure change, > like io_tree and quota flags change. > > We ran xfstests with dedupe enabled. Is there an xfstests patch for this I can look at? We want to be able to run and reproduce the same tests as you. Also where are the disk portion patches or did I miss them someho

Re: [PATCH] vfs: allow FILE_EXTENT_SAME (dedupe_file_range) on a file opened ro

tly being > deduped gets ETXTBSY. > > Issuing this ioctl on a ro file was already allowed for root/cap. > > Tested on btrfs and not-yet-merged xfs, as only them implement this ioctl. > > Signed-off-by: Adam Borowski <kilob...@angband.pl> Reviewed-by: Mark Fasheh <mfas...@

Re: [PATCH v5 0/4] Btrfs in-band de-duplication test cases

s/203.out > > -- > 2.9.0 > > > > -- > To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in > the body of a message to majord...@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html -- Mark Fasheh -- To unsubscribe

[PATCH 2/2] btrfs-progs: btrfsck: verify qgroups above level 0

. Most other sections of this code went unchanged, in particular the root counting works independently of the accounting. Signed-off-by: Mark Fasheh <mfas...@suse.de> --- qgroup-verify.c | 305 ++-- 1 file changed, 251 insertions(+), 54 del

[PATCH 0/2] btrfs-progs: qgroup verification update

Hi David, The following two patches update the qgroup verification code in btrfsck to understand entire qgroup hierarchies. The first patch is a simple bugfix for some leaked objects and can be taken separately if you like. The 2nd patch implements the actual verification update. If you prefer

Re: [PATCH v3] btrfs: fix check_shared for fiemap ioctl

ry then you deadlock. Fiemap is locking down extents which may also get locked down when you allocate within those locks. See my e-mail here for details, http://www.spinics.net/lists/linux-btrfs/msg55789.html --Mark -- Mark Fasheh -- To unsubscribe from this list: send the line "unsubs

Re: [PATCH v10 09/21] btrfs: dedupe: Inband in-memory only de-duplication implement

On Tue, Jun 07, 2016 at 08:42:46AM +0800, Qu Wenruo wrote: > > > At 06/07/2016 03:54 AM, Mark Fasheh wrote: > >On Sat, Jun 04, 2016 at 06:26:39PM +0800, Qu Wenruo wrote: > >> > >> > >>On 06/03/2016 10:27 PM, Josef Bacik wrote: &g

Re: [PATCH v10 09/21] btrfs: dedupe: Inband in-memory only de-duplication implement

On Sat, Jun 04, 2016 at 06:26:39PM +0800, Qu Wenruo wrote: > > > On 06/03/2016 10:27 PM, Josef Bacik wrote: > >On 06/01/2016 09:12 PM, Qu Wenruo wrote: > >> > >> > >>At 06/02/2016 06:08 AM, Mark Fasheh wrote: > >>>On Fri, Apr 01, 2016 at 0

Re: [PATCH v2] btrfs: fix check_shared for fiemap ioctl

On Thu, Jun 02, 2016 at 02:17:40PM -0700, Mark Fasheh wrote: > On Thu, Jun 02, 2016 at 04:56:06PM -0400, Jeff Mahoney wrote: > > On 6/2/16 3:08 PM, Mark Fasheh wrote: > > > On Thu, Jun 02, 2016 at 07:07:32PM +0200, David Sterba wrote: > > >> On Wed, Jun 01, 2016 at

Re: [PATCH v2] btrfs: fix check_shared for fiemap ioctl

On Thu, Jun 02, 2016 at 04:56:06PM -0400, Jeff Mahoney wrote: > On 6/2/16 3:08 PM, Mark Fasheh wrote: > > On Thu, Jun 02, 2016 at 07:07:32PM +0200, David Sterba wrote: > >> On Wed, Jun 01, 2016 at 02:15:22PM -0700, Mark Fasheh wrote: > >>>> +/* dynamically a

Re: [PATCH v2] btrfs: fix check_shared for fiemap ioctl

On Thu, Jun 02, 2016 at 01:46:27PM +0800, Lu Fengqi wrote: > > At 06/02/2016 05:15 AM, Mark Fasheh wrote: > >Thanks for trying to fix this problem, comments below. > > > >On Wed, Jun 01, 2016 at 01:48:05PM +0800, Lu Fengqi wrote: > >>Only in the case of differe

Re: [PATCH v2] btrfs: fix check_shared for fiemap ioctl

On Thu, Jun 02, 2016 at 07:07:32PM +0200, David Sterba wrote: > On Wed, Jun 01, 2016 at 02:15:22PM -0700, Mark Fasheh wrote: > > > +/* dynamically allocate and initialize a ref_root */ > > > +static struct ref_root *ref_root_alloc(void) > > > +{ >

Re: [PATCH v10 10/21] btrfs: try more times to alloc metadata reserve space

s sort of issue is a reason why I strongly feel we don't want to merge this series piecemeal until we know that after everything is complete, we can end up with a fully baked in-band dedupe implementation. Luckily Qu says he's on it so if he posts a workable fix here my whole point can become moot. Unt

Re: [PATCH v10 09/21] btrfs: dedupe: Inband in-memory only de-duplication implement

ret = btrfs_dedupe_del(trans, info, bytenr); > + if (ret < 0) { > + btrfs_abort_transaction(trans, extent_root, > + ret); I don't see why an error here should lead to a readonly fs. --Mark -- Mark Fasheh -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: [PATCH v10 08/21] btrfs: ordered-extent: Add support for dedupe

upe > + * is disabled. As its delayed ref is already increased. > + */ Initially, I had a hard time understanding this comment but I'm pretty sure I know what you mean. /* * A hash hit means we have already incremented the extents delayed ref. * We must handle this even if anothe

Re: [PATCH v7 0/8] Inband dedupe for btrfs-progs

' is not confusing the two forms we have. I don't personally care what other name is used and of course it could have 'dedupe' in the name just not solely 'dedupe'. As a poor example, we could call it 'btrfs inband-dedupe ...'. Thanks, --Mark -- Mark Fasheh -- To unsubscribe

Re: [PATCH v2] btrfs: fix check_shared for fiemap ioctl

On Wed, Jun 01, 2016 at 02:15:22PM -0700, Mark Fasheh wrote: > > +static int ref_tree_add(struct ref_root *ref_tree, u64 root_id, u64 > > object_id, > > + u64 offset, u64 parent, int count) > > +{ > > + struct ref_node *node = NULL; >

Re: [PATCH v2] btrfs: fix check_shared for fiemap ioctl

> + if (ret) { > + kfree(node); > + return ret; > + } If you put the open coded comparisons into their own function, then it should be trivial to call that here and we can have a 'standard' looking rbtree insert instead of

Re: [PATCH v10 04/21] btrfs: dedupe: Introduce function to remove hash from in-memory tree

ntry_safe(entry, tmp, _info->lru_list, lru_list) > + __inmem_del(dedupe_info, entry); > + mutex_unlock(_info->lock); > +} > + > +int btrfs_dedupe_disable(struct btrfs_fs_info *fs_info) > +{ > + struct btrfs_dedupe_info *dedupe_info; > + int ret; >

Re: [PATCH v10 03/21] btrfs: dedupe: Introduce function to add hash into in-memory tree

gt; + int ret = 0; > + u16 type = dedupe_info->hash_type; > + struct inmem_hash *ihash; > + > + ihash = inmem_alloc_hash(type); > + > + if (!ihash) > + return -ENOMEM; > + > + /* Copy the data out */ > + ihash->bytenr = hash

Re: [PATCH] Improve balance performance when qgroups are turned on

any patches. --Mark -- Mark Fasheh -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: [PATCH] Improve balance performance when qgroups are turned on

On Mon, May 30, 2016 at 03:48:14PM +0800, Qu Wenruo wrote: > > > Mark Fasheh wrote on 2016/05/26 17:18 -0700: > >The btrfs balance operation is significantly slower when qgroups are > >enabled. To the best of my knowledge, a balance shouldn't have an effect on >

Re: [PATCH] Btrfs: don't BUG_ON() in btrfs_orphan_add

: Josef Bacik <jba...@fb.com> Reviewed-by: Mark Fasheh <mfas...@suse.de> -- Mark Fasheh -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: [PATCH] fstests: btrfs: add test for qgroup handle de-refer

status=0 +fi That way we're not keying on some specific value showing up but instead that qgroup validation passes (which is really what we want to test). Thanks, --Mark -- Mark Fasheh -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

[PATCH] Improve balance performance when qgroups are turned on

sys 2m0.852s Balance with qgroups enabeld, after patch: # time btrfs balance start --full-balance /btrfs Done, had to relocate 26 out of 26 chunks real2m2.806s user0m0.000s sys 0m54.174s Signed-off-by: Mark Fasheh <mfas...@suse

Re: [PATCH] btrfs,vfs: allow FILE_EXTENT_SAME on a file opened ro

being cloned too, correct? If that's the case then I wonder how this issue gets solved for other ioctls. --Mark -- Mark Fasheh -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: About in-band dedupe for v4.7

On Wed, May 11, 2016 at 07:36:59PM +0200, David Sterba wrote: > On Tue, May 10, 2016 at 07:52:11PM -0700, Mark Fasheh wrote: > > Taking your history with qgroups out of this btw, my opinion does not > > change. > > > > With respect to in-memory only dedu

Re: [PATCH v4] btrfs: qgroup: Fix qgroup accounting when creating snapshot

On Wed, May 11, 2016 at 09:59:52AM -0700, Josef Bacik wrote: > On 05/11/2016 09:57 AM, Mark Fasheh wrote: > >Hi Josef, > > > >On Fri, Apr 22, 2016 at 02:12:11PM -0400, Josef Bacik wrote: > >>On 04/15/2016 05:08 AM, Qu Wenruo wrote: > >>>Current btrfs qgro

Re: [PATCH v4] btrfs: qgroup: Fix qgroup accounting when creating snapshot

gt; >+ > >+out: > >+mutex_unlock(_info->tree_log_mutex); > >+ > >+/* > >+ * Force parent root to be updated, as we recorded it before so its > >+ * last_trans == cur_transid. > >+ * Or it won't be committed again onto disk after l

Re: About in-band dedupe for v4.7

, > to ensure dedupe won't corrupt any existing test case. As Satoru mentioned, this is something that everybody needs to be able to run. I would also like to see some basic analysis done on write-heavy workloads. I think it's fair to understand what sort of impact this will have on the write path. --Mark -- Mark Fasheh -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: About in-band dedupe for v4.7

On Wed, May 11, 2016 at 09:03:24AM +0800, Qu Wenruo wrote: > > > Mark Fasheh wrote on 2016/05/10 15:11 -0700: > >On Tue, May 10, 2016 at 03:19:52PM +0800, Qu Wenruo wrote: > >>Hi, Chris, Josef and David, > >> > >>As merge window for v4.7 is coming, it

Re: [PATCH v10 02/21] btrfs: dedupe: Introduce function to initialize dedupe info

t this, we need to be able to bisect a kernel without random patches breaking the build. --Mark -- Mark Fasheh -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: About in-band dedupe for v4.7

mbers! Sorry to be so harsh. --Mark -- Mark Fasheh -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Idea on compatibility for old distributions

n of btrfs then fi du won't report any shared extents. This is unfortunate but so is running btrfs from 2.6.32. --Mark -- Mark Fasheh -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info

Re: [PATCH] Btrfs: fix qgroup accounting when snapshotting

for obvious reasons. As soon as I think I have something that works though, it falls over once I poke it with a larger test. In particular, I've been trying to move around the points at which we are taking our values for qgroup_inherit(). I can get rfer values _almost_ always correct by recording

Re: [PATCH] Btrfs: fix qgroup accounting when snapshotting

sive compressed 32768 disk: exclusive 16384 exclusive compressed 16384 diff: exclusive 16384 exclusive compressed 16384 --Mark -- Mark Fasheh From: Mark Fasheh <mfas...@suse.de> [PATCH] btrfs: Test that qgroup counts are valid after snapshot creation This has b

Re: [PATCH] btrfs: Test that qgroup counts are valid after snapshot creation

On Fri, Apr 22, 2016 at 08:26:33AM +0800, Qu Wenruo wrote: > > > Mark Fasheh wrote on 2016/04/21 16:53 -0700: > >Thank you for the review, comments are below. > > > >On Wed, Apr 20, 2016 at 09:48:54AM +0900, Satoru Takeuchi wrote: > >>On 2016/04/20 7:25, Mark

Re: [PATCH v4] btrfs: qgroup: Fix qgroup accounting when creating snapshot

On Fri, Apr 22, 2016 at 02:23:59PM -0400, Josef Bacik wrote: > On 04/22/2016 02:21 PM, Mark Fasheh wrote: > >On Fri, Apr 22, 2016 at 02:12:11PM -0400, Josef Bacik wrote: > >>On 04/15/2016 05:08 AM, Qu Wenruo wrote: > >>>+ /* > >>>+ * Force parent root

Re: [PATCH v4] btrfs: qgroup: Fix qgroup accounting when creating snapshot

a while now and as slow as this patch might be, it actually works where nothing else has so far. --Mark -- Mark Fasheh -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: About fi du and reflink/dedupe

fiemap heavily. You will have very little luck in asking them to modify their applications. If btrfs fiemap is broken, we fix that full stop. More specifically, If in-band dedupe is causing fiemap to go out to lunch 'for a year', we need to address the core problem in in-band dedupe. If it's a gen

[PATCH v2] btrfs: Test that qgroup counts are valid after snapshot creation

This has been broken since Linux v4.1. We may have worked out a solution on the btrfs list but in the meantime sending a test to expose the issue seems like a good idea. Changes from v1-v2: - cleanups - added 122.out Signed-off-by: Mark Fasheh <mfas...@suse.de> --- tests/btrfs/122

Re: [PATCH] btrfs: Test that qgroup counts are valid after snapshot creation

Thank you for the review, comments are below. On Wed, Apr 20, 2016 at 09:48:54AM +0900, Satoru Takeuchi wrote: > On 2016/04/20 7:25, Mark Fasheh wrote: > >+# Force a small leaf size to make it easier to blow out our root > >+# subvolume tree > >+_scratch_mkfs "--nod

[PATCH] btrfs: Test that qgroup counts are valid after snapshot creation

This has been broken since Linux v4.1. We may have worked out a solution on the btrfs list but in the meantime sending a test to expose the issue seems like a good idea. Signed-off-by: Mark Fasheh <mfas...@suse.de> --- tests/btrfs/122

Re: [PATCH v4] btrfs: qgroup: Fix qgroup accounting when creating snapshot

o switch roots. > This leads to the same nr_old_roots, and this extent just got ignored by > qgroup, which means this extent is wrongly accounted. > > Fix it by call commit_cowonly_roots() after qgroup_account_extent() in > create_pending_snapshot(), with needed preparation. &g

Re: [PATCH v3] btrfs: qgroup: Fix qgroup accounting when creating snapshot

On Fri, Apr 15, 2016 at 09:00:06AM +0800, Qu Wenruo wrote: > > > Mark Fasheh wrote on 2016/04/14 14:42 -0700: > >Hi Qu, > > > >On Thu, Apr 14, 2016 at 01:38:40PM +0800, Qu Wenruo wrote: > >>Current btrfs qgroup design implies a requirement that after call

[PATCH V2] btrfs: test snapshot create with invalid parent qgroup

://thread.gmane.org/gmane.comp.file-systems.btrfs/54755 Thanks, --Mark Signed-off-by: Mark Fasheh <mfas...@suse

Re: [PATCH v3] btrfs: qgroup: Fix qgroup accounting when creating snapshot

ere is no switch roots. > This leads to the same nr_old_roots, and this extent just got ignored by > qgroup, which means this extent is wrongly accounted. > > Fix it by call commit_cowonly_roots() after qgroup_account_extent() in > create_pending_snapshot(), with needed preparation. &g

Re: WARN_ON in record_root_in_trans() when deleting freshly renamed subvolume

On Mon, Apr 11, 2016 at 09:05:47AM +0800, Qu Wenruo wrote: > > > Mark Fasheh wrote on 2016/04/08 12:18 -0700: > >On Fri, Apr 08, 2016 at 03:10:35PM +0200, Holger Hoffstätte wrote: > >>[cc: Mark and Qu] > >> > >>On 04/08/16 13:51, Holger Hoffstätte wr

Re: WARN_ON in record_root_in_trans() when deleting freshly renamed subvolume

napshot" > from last Wednesday immediately fixes the problem. Not surprising, I had some issues testing it out too. I'm pretty sure this patch is corrupting memory, I just haven't found where yet though my educated guess is that the transaction is being reused improperly. --Mark --

Re: [PATCH] btrfs: qgroup: Fix qgroup accounting when creating snapshot

pens every time for me. Just wait about 30 seconds or so (my guess is to let a transaction commit kick in). Also I can force the issue to show up if I unmount. --Mark -- Mark Fasheh -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to

Re: [PATCH] btrfs: qgroup: Fix qgroup accounting when creating snapshot

i" echo "remove snapshot $S" btrfs su de $S done; This is on Linux 4.5 with my inherit fix and your patch applied. The script I pasted above ran with no problems until I added your patch to my kernel so my guess is it's not related to the btrfs_qgroup_inherit() patc

Re: [PATCH] btrfs: test snapshot create with invalid parent qgroup

On Wed, Apr 06, 2016 at 10:40:02PM +0100, Filipe Manana wrote: > On Wed, Apr 6, 2016 at 10:30 PM, Mark Fasheh <mfas...@suse.de> wrote: > > Test that an invalid parent qgroup does not cause snapshot create to > > force the FS readonly. > > > > In btrfs, create_pe

Re: [PATCH] btrfs: test snapshot create with invalid parent qgroup

On Wed, Apr 06, 2016 at 02:30:34PM -0700, Mark Fasheh wrote: > Test that an invalid parent qgroup does not cause snapshot create to > force the FS readonly. > > In btrfs, create_pending_snapshot() will go readonly on _any_ error return > from > btrfs_qgroup_inherit(). If q

[PATCH] btrfs: test snapshot create with invalid parent qgroup

, --Mark Signed-off-by: Mark Fasheh <mfas...@suse.de> --- tests/btrfs/119 | 70 + tests/btrfs/119.out | 2 ++ tests/btrfs/group | 1 + 3 files changed, 73 insertions(+) create mode 100755 tests/btrfs/119 create mode 100644

Re: [PATCH] btrfs: handle non-fatal errors in btrfs_qgroup_inherit()

today. --Mark -- Mark Fasheh -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Qgroups wrong after snapshot create

On Tue, Apr 05, 2016 at 03:16:54PM -0700, Mark Fasheh wrote: > On Tue, Apr 05, 2016 at 09:27:01AM +0800, Qu Wenruo wrote: > > Mark Fasheh wrote on 2016/04/04 16:06 -0700: > > >Hi, > > > > > >Making a snapshot gets us the wrong qgroup numbers. This is very easy t

Re: Qgroups wrong after snapshot create

On Tue, Apr 05, 2016 at 09:27:01AM +0800, Qu Wenruo wrote: > Mark Fasheh wrote on 2016/04/04 16:06 -0700: > >Hi, > > > >Making a snapshot gets us the wrong qgroup numbers. This is very easy to > >reproduce. From a fresh btrfs filesystem, simply enable qgrou

Qgroups wrong after snapshot create

-debug-tree for the FS used in this example. -- Mark Fasheh root tree leaf 29884416 items 17 free space 11820 generation 11 owner 1 fs uuid f7e55c97-b0b3-44e5-bab1-1fd55d54409b chunk uuid b78fe016-e35f-4f57-8211-796cbc9be3a4 item 0 key (EXTENT_TREE ROOT_ITEM 0) itemoff 15844 itemsize 439

[PATCH] btrfs: handle non-fatal errors in btrfs_qgroup_inherit()

ume snapshot -i 1/10 $SCRATCH_MNT $SCRATCH_MNT/snap1 _scratch_unmount echo "Silence is golden" status=0 exit Signed-off-by: Mark Fasheh <mfas...@suse.de> --- fs/btrfs/qgroup.c | 54 -- 1 file changed, 32 insertions(+), 22

[RESEND][PATCH] btrfs: Add qgroup tracing

This patch adds tracepoints to the qgroup code on both the reporting side (insert_dirty_extents) and the accounting side. Taken together it allows us to see what qgroup operations have happened, and what their result was. Signed-off-by: Mark Fasheh <mfas...@suse.de> --- fs/btrfs/qg

Re: [PATCH] btrfs-progs: add 'du' command

On Tue, Feb 02, 2016 at 12:43:45AM +0100, David Sterba wrote: > Hi, > > On Wed, Jan 20, 2016 at 01:49:24PM -0800, Mark Fasheh wrote: > > A git tree of the patches can be found here: > > > > https://github.com/markfasheh/btrfs-progs-patches/tree/du > > what c

Re: Regression in: [PATCH 4/4] btrfs: qgroup: account shared subtree during snapshot delete

On Wed, Nov 04, 2015 at 09:01:36AM +0800, Qu Wenruo wrote: > > > Mark Fasheh wrote on 2015/11/03 11:26 -0800: > >On Mon, Nov 02, 2015 at 09:34:24AM +0800, Qu Wenruo wrote: > >> > >> > >>Stefan Priebe wrote on 2015/11/01 21:49 +0100: > >>>

[PATCH 3/3] btrfs: qgroup: account shared subtree during snapshot delete

/inc functions so we don't have to add actions beyond what we had originally. Signed-off-by: Mark Fasheh <mfas...@suse.de> --- fs/btrfs/extent-tree.c | 47 --- fs/btrfs/qgroup.c | 2 ++ 2 files changed, 42 insertions(+), 7 deletions(-) diff

[PATCH 2/3] btrfs: Add qgroup tracing

This patch adds tracepoints to the qgroup code on both the reporting side (insert_dirty_extents) and the accounting side. Taken together it allows us to see what qgroup operations have happened, and what their result was. Signed-off-by: Mark Fasheh <mfas...@suse.de> --- fs/btrfs/qg

[PATCH 0/3] btrfs: update qgroups in drop snapshot, V2

Hi, The following 3 patches fix a regression introduced in Linux 4.2 where btrfs_drop_snapshot() wasn't updating qgroups, resulting in them going bad. The original e-mail pointing this out is below: http://www.spinics.net/lists/linux-btrfs/msg46093.html The first patch is from Josef and fix

[PATCH 1/3] Btrfs: use btrfs_get_fs_root in resolve_indirect_ref

e able to search down the snapshot we are deleting, which will cause us to miss roots. So use btrfs_get_fs_root and send false for check_ref so we can always get the root we're looking for. Thanks, Signed-off-by: Josef Bacik <jba...@fb.com> Signed-off-by: Mark Fasheh <mfas...@suse.de

Re: Regression in: [PATCH 4/4] btrfs: qgroup: account shared subtree during snapshot delete

test on one of my machines and see what I get. I'm not surprised that the overhead is noticable, and I agree it's easy enough to try things like replacing the allocation once we have a test going. Thanks, --Mark -- Mark Fasheh -- To unsubscribe from this list: send the line

Re: Regression in: [PATCH 4/4] btrfs: qgroup: account shared subtree during snapshot delete

e are opinion and not based in fact. Yes btw, we might have to do more work for the uncommon case of a qgroup being referenced by higher level groups but that is clearly not happening here (and honestly it's not a common case at all). --Mark -- Mark Fasheh -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Regression in: [PATCH 4/4] btrfs: qgroup: account shared subtree during snapshot delete

is very unlikely to be your actual problem as it won't be doing anything (ok some kmalloc/free of a very tiny object) since qgroups are disabled. Also, btrfs had working subtree accounting in that code for the last N releases (doing the same exact thing) and it only changed for the one release that Q

Re: Regression in: [PATCH 4/4] btrfs: qgroup: account shared subtree during snapshot delete

just rebooting to the other kernel. That's fine, disregard my previous e-mail - I just saw the mail Qu sent me. There's a problem in the code that the patch calls which is causing your performance issues. I'll CC you when I put out a fix. Thanks, --Mark -- Mark Fasheh -- To unsubscribe from

Re: [PATCH 4/4] btrfs: qgroup: account shared subtree during snapshot delete

On Mon, Nov 02, 2015 at 09:59:01AM +0800, Qu Wenruo wrote: > > > Mark Fasheh wrote on 2015/09/22 13:15 -0700: > >Commit 0ed4792 ('btrfs: qgroup: Switch to new extent-oriented qgroup > >mechanism.') removed our qgroup accounting during > >btrfs_drop_snapshot(). Predicta

Re: [PATCH] btrfs: fix use after free iterating extrefs

gt; > cc: sta...@vger.kernel.org # v3.7+ > cc: Mark Fasheh <mfas...@suse.de> Reviewed-by: Mark Fasheh <mfas...@suse.de> Thanks for the CC Chris. --Mark -- Mark Fasheh -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the bo

Re: [RFC][PATCH] btrfs: add ioctl to monitor subvolume dropping

On Tue, Oct 06, 2015 at 10:25:52AM +0200, David Sterba wrote: > On Thu, Oct 01, 2015 at 02:30:47PM -0700, Mark Fasheh wrote: > > At the moment, userspace has no way of knowing when a snapshot is finally > > removed. This has become a problem when writing tests for btrfs

Re: [PATCH] fstests: btrfs: add test for quota groups and drop snapshot

On Tue, Sep 29, 2015 at 09:28:58AM +1000, Dave Chinner wrote: > On Wed, Sep 23, 2015 at 02:05:16PM -0700, Mark Fasheh wrote: > > Since the last time I sent this test, drop snapshot was broken again with > > respect to qgroups. What practical step could I take to get a test for t

[RFC][PATCH] btrfs: add ioctl to monitor subvolume dropping

strerror(ret)); return ret; } close(fd); if (ret == ENOENT) printf("Subvolume not found or already dropped\n"); else printf("Subvolume %"PRIu64" drop is at object: %"PRIu64"\n"

Re: [PATCH 0/4] btrfs: update qgroups in drop snapshot

p the patch titled: Btrfs: keep dropped roots in cache until transaction commit since it is already in integration-4.3. Everything else seems to apply on my end. --Mark -- Mark Fasheh -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the

Re: [PATCH 0/4] btrfs: update qgroups in drop snapshot

l orphaning transaction to finish. > >+ > >+_scratch_unmount > >+ > >+# generate a qgroup report and look for inconsistent groups > >+# - don't use _run_btrfs_util_prog here as it captures the output and > >+#we need to grep it. > >+$BTRFS_UTIL_PRO

Re: [PATCH] fstests: btrfs: add test for quota groups and drop snapshot

Hey Dave, thanks for the review. A refreshed patch for you to look at is attached. On Wed, Sep 23, 2015 at 12:47:08PM +1000, Dave Chinner wrote: > On Tue, Sep 22, 2015 at 03:16:49PM -0700, Mark Fasheh wrote: > > +tmp=/tmp/$$ > > +status=1 # failure is the default! > >

<    1   2   3   4   5   >