Re: kernel BUG at linux-4.2.0/fs/btrfs/extent-tree.c:1833 on rebalance

On 09/15/2015 10:47 AM, Stéphane Lesimple wrote: I've been experiencing repetitive "kernel BUG" occurences in the past few days trying to balance a raid5 filesystem after adding a new drive. It occurs on both 4.2.0 and 4.1.7, using 4.2 userspace tools. I've ran a scrub on this filesystem after

Re: Horrible dbench performance

On 09/14/2015 11:28 AM, Liu Bo wrote: Hi, Both [1] and [2] had run dbench on btrfs with fast storage, and showed bad numbers, I got an impression that after refractoring btree lock to smart rwlock, we have mitigated this issue.. Not got a fast-enough ssd handy, does anyone confirm the result

Re: [PATCH] Btrfs: keep dropped roots in cache until transaciton commit

On 09/10/2015 09:25 PM, Qu Wenruo wrote: Josef Bacik wrote on 2015/09/10 16:27 -0400: When dropping a snapshot we need to account for the qgroup changes. If we drop the snapshot in all one go then the backref code will fail to find blocks from the snapshot we dropped since it won't be able

[PATCH] Btrfs: use btrfs_get_fs_root in resolve_indirect_ref

are deleting, which will cause us to miss roots. So use btrfs_get_fs_root and send false for check_ref so we can always get the root we're looking for. Thanks, Signed-off-by: Josef Bacik <jba...@fb.com> --- fs/btrfs/backref.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/fs

[PATCH] Btrfs: keep dropped roots in cache until transaciton commit

properly, and fixes a problem Mark was seeing with snapshot delete and qgroups. Thanks, Signed-off-by: Josef Bacik <jba...@fb.com> --- fs/btrfs/extent-tree.c | 2 +- fs/btrfs/transaction.c | 12 fs/btrfs/transaction.h | 10 ++ 3 files changed, 23 insertions(+), 1 de

Re: [PATCH v1 3/8] btrfs: add .copy_file_range file operation

ts own function. It retains the core btrfs error checks that should be shared. Signed-off-by: Zach Brown <z...@redhat.com> Signed-off-by: Anna Schumaker <anna.schuma...@netapp.com> This bit looks fine to me, Reviewed-by: Josef Bacik <jba...@fb.com> Thanks, Josef -- To un

Re: [PATCH 5/6] Btrfs: wire up the free space tree to the extent tree

On 09/02/2015 12:42 AM, Omar Sandoval wrote: On Tue, Sep 01, 2015 at 03:48:57PM -0400, Josef Bacik wrote: On 09/01/2015 03:05 PM, Omar Sandoval wrote: From: Omar Sandoval <osan...@fb.com> The free space tree is updated in tandem with the extent tree. There are only a handful of places

Re: [PATCH 6/6] Btrfs: add free space tree mount option

space tree the default and stop setting the cache generation in mkfs. Signed-off-by: Omar Sandoval <osan...@fb.com> Reviewed-by: Josef Bacik <jba...@fb.com> Thanks, Josef -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message

Re: [PATCH 3/6] Btrfs: introduce the free space B-tree on-disk format

P type and have a bitmap item attached, which is just an array of bytes. Signed-off-by: Omar Sandoval <osan...@fb.com> Reviewed-by: Josef Bacik <jba...@fb.com> Thanks, Josef -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to ma

Re: [PATCH 2/6] Btrfs: add helpers for read-only compat bits

On 09/01/2015 03:01 PM, Omar Sandoval wrote: From: Omar Sandoval <osan...@fb.com> We're finally going to add one of these for the free space tree, so let's add the same nice helpers that we have for the incompat bits. Signed-off-by: Omar Sandoval <osan...@fb.com> Reviewed-by:

Re: [PATCH 4/6] Btrfs: implement the free space B-tree

On 09/01/2015 03:13 PM, Omar Sandoval wrote: From: Omar Sandoval The free space cache has turned out to be a scalability bottleneck on large, busy filesystems. When the cache for a lot of block groups needs to be written out, we can get extremely long commit times; if this

Re: [PATCH 1/6] Btrfs: add extent buffer bitmap operations

On 09/01/2015 03:01 PM, Omar Sandoval wrote: From: Omar Sandoval These are going to be used for the free space tree bitmap items. Signed-off-by: Omar Sandoval Can we get sanity tests for these operations so we know they are properly unit tested? ---

Re: [PATCH 5/6] Btrfs: wire up the free space tree to the extent tree

On 09/01/2015 03:05 PM, Omar Sandoval wrote: From: Omar Sandoval The free space tree is updated in tandem with the extent tree. There are only a handful of places where we need to hook in: 1. Block group creation 2. Block group deletion 3. Delayed refs (extent creation and

Re: [PATCH 4/6] Btrfs: implement the free space B-tree

On 09/01/2015 04:06 PM, Omar Sandoval wrote: On Tue, Sep 01, 2015 at 03:44:27PM -0400, Josef Bacik wrote: On 09/01/2015 03:13 PM, Omar Sandoval wrote: From: Omar Sandoval <osan...@fb.com> The free space cache has turned out to be a scalability bottleneck on large, busy filesystems

Re: [PATCH] Btrfs: deal with error on secondary log properly

On 08/28/2015 09:23 AM, Filipe David Manana wrote: On Wed, Aug 26, 2015 at 8:06 PM, Josef Bacik jba...@fb.com wrote: On 08/25/2015 10:06 PM, Liu Bo wrote: On Tue, Aug 25, 2015 at 01:09:43PM -0400, Josef Bacik wrote: If we have an fsync at the same time in two seperate subvolumes we could

Re: [PATCH] Btrfs: deal with error on secondary log properly

On 08/25/2015 10:06 PM, Liu Bo wrote: On Tue, Aug 25, 2015 at 01:09:43PM -0400, Josef Bacik wrote: If we have an fsync at the same time in two seperate subvolumes we could end up with the tree log pointing at invalid blocks. We need to notice if our writeout Mind to share more details

[PATCH] Btrfs: deal with error on secondary log properly

. Thanks, Signed-off-by: Josef Bacik jba...@fb.com --- fs/btrfs/file.c | 4 fs/btrfs/tree-log.c | 4 2 files changed, 8 insertions(+) diff --git a/fs/btrfs/file.c b/fs/btrfs/file.c index b823fac..c8f49f5 100644 --- a/fs/btrfs/file.c +++ b/fs/btrfs/file.c @@ -2054,6 +2054,10 @@ int

Re: kernel BUG at fs/btrfs/extent-tree.c:8113! (4.1.3 kernel)

On 08/12/2015 10:47 AM, Marc MERLIN wrote: On Tue, Aug 11, 2015 at 11:40:45AM -0400, Josef Bacik wrote: From a48cf7a9ae44a17d927df5542c8b0be287aee9ed Mon Sep 17 00:00:00 2001 From: Josef Bacik jba...@fb.com Date: Tue, 11 Aug 2015 11:39:37 -0400 Subject: [PATCH] Btrfs: kill BUG_ON

Re: [PATCH] Btrfs: check if previous transaction aborted to avoid fs corruption

waiting for the previous transaction we verify that it was aborted. Signed-off-by: Filipe Manana fdman...@suse.com Eesh good catch Filpe, Reviewed-by: Josef Bacik jba...@fb.com Thanks, Josef -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message

Re: kernel BUG at fs/btrfs/extent-tree.c:8113! (4.1.3 kernel)

On 08/12/2015 12:09 PM, Marc MERLIN wrote: On Wed, Aug 12, 2015 at 11:15:39AM -0400, Josef Bacik wrote: On 08/12/2015 10:47 AM, Marc MERLIN wrote: On Tue, Aug 11, 2015 at 11:40:45AM -0400, Josef Bacik wrote: From a48cf7a9ae44a17d927df5542c8b0be287aee9ed Mon Sep 17 00:00:00 2001 From: Josef

Re: kernel BUG at fs/btrfs/extent-tree.c:8113! (4.1.3 kernel)

the kernel) From a48cf7a9ae44a17d927df5542c8b0be287aee9ed Mon Sep 17 00:00:00 2001 From: Josef Bacik jba...@fb.com Date: Tue, 11 Aug 2015 11:39:37 -0400 Subject: [PATCH] Btrfs: kill BUG_ON() in btrfs_lookup_extent_info() Replace it with an ASSERT(0) for the developers and an error

Re: [PATCH V3 02/11] Btrfs: Compute and look up csums based on sectorsized blocks

++; - total_bytes += bvec-bv_len; - this_sum_bytes += bvec-bv_len; - offset += bvec-bv_len; bvec++; } this_sum_bytes = 0; This is fine, Reviewed-by: Josef Bacik jba...@fb.com Thanks, Josef -- To unsubscribe from this list: send the line

Re: [PATCH V2 03/11] Btrfs: Direct I/O read: Work on sectorsized blocks

On 08/07/2015 03:05 AM, Chandan Rajendra wrote: The direct I/O read's endio and corresponding repair functions work on page sized blocks. This commit adds the ability for direct I/O read to work on subpagesized blocks. Signed-off-by: Chandan Rajendra chan...@linux.vnet.ibm.com ---

Re: [PATCH V2 02/11] Btrfs: Compute and look up csums based on sectorsized blocks

On 08/07/2015 03:05 AM, Chandan Rajendra wrote: Checksums are applicable to sectorsize units. The current code uses bio-bv_len units to compute and look up checksums. This works on machines where sectorsize == PAGE_SIZE. This patch makes the checksum computation and look up code to work with

Re: [PATCH] generic/311: Disable dmesg check

On 07/22/2015 01:27 AM, Dave Chinner wrote: On Tue, Jul 21, 2015 at 08:37:39PM -0400, Josef Bacik wrote: On 07/21/2015 08:01 PM, Dave Chinner wrote: On Tue, Jul 21, 2015 at 04:03:17PM +0530, Chandan Rajendra wrote: On Tuesday 21 Jul 2015 08:12:20 Dave Chinner wrote: On Mon, Jul 20, 2015

Re: [PATCH] generic/311: Disable dmesg check

On 07/22/2015 01:27 AM, Dave Chinner wrote: On Tue, Jul 21, 2015 at 08:37:39PM -0400, Josef Bacik wrote: On 07/21/2015 08:01 PM, Dave Chinner wrote: On Tue, Jul 21, 2015 at 04:03:17PM +0530, Chandan Rajendra wrote: On Tuesday 21 Jul 2015 08:12:20 Dave Chinner wrote: On Mon, Jul 20, 2015

Re: [PATCH] generic/311: Disable dmesg check

On 07/21/2015 08:01 PM, Dave Chinner wrote: On Tue, Jul 21, 2015 at 04:03:17PM +0530, Chandan Rajendra wrote: On Tuesday 21 Jul 2015 08:12:20 Dave Chinner wrote: On Mon, Jul 20, 2015 at 08:55:32AM -0400, Josef Bacik wrote: On 07/19/2015 07:54 PM, Dave Chinner wrote: On Fri, Jul 17, 2015

Re: [PATCH] generic/311: Disable dmesg check

On 07/19/2015 07:54 PM, Dave Chinner wrote: On Fri, Jul 17, 2015 at 05:10:50PM +0530, Chandan Rajendra wrote: On Friday 17 Jul 2015 06:16:02 Brian Foster wrote: On Fri, Jul 17, 2015 at 12:56:43AM -0400, Chandan Rajendra wrote: When running generic/311 on Btrfs' subpagesize-blocksize patchset

Re: [PATCH] check: check so offset is not bigger then the leaf

On 06/25/2015 09:06 AM, David Sterba wrote: On Thu, Jun 18, 2015 at 10:16:54AM -0700, Josef Bacik wrote: On 06/18/2015 09:44 AM, David Sterba wrote: On Thu, Jun 18, 2015 at 01:59:13AM +0200, Robert Marklund wrote: This could crash before because of dangerous dangling offset of pointer

Re: [PATCH] check: check so offset is not bigger then the leaf

On 06/18/2015 09:44 AM, David Sterba wrote: On Thu, Jun 18, 2015 at 01:59:13AM +0200, Robert Marklund wrote: This could crash before because of dangerous dangling offset of pointer. That's right, this can happen. There are more btrfs_item_ptr that would be good to validate that way, namely in

Re: [PATCH] Btrfs: use received_uuid of parent during send

On 06/11/2015 01:09 PM, Hugo Mills wrote: On Thu, Jun 04, 2015 at 05:17:25PM -0400, Josef Bacik wrote: Neil Horman pointed out a problem where if he did something like this receive A snap A B change B send -p A B and then on another box do recieve A receive B the receive B would fail

[PATCH] btrfs-progs: make receive work inside of subvolumes

chroot'ed and are using -m. Then strip this extra path off of the subvol we find so we can look up our parent properly. Thanks Reported-by: Neil Horman nhor...@redhat.com Signed-off-by: Josef Bacik jba...@fb.com --- cmds-receive.c | 89

Re: [GIT PULL] Qgroup rework with other Fujitsu fix.

On 06/09/2015 10:23 AM, adam900710 wrote: Sorry, forgot to reply to all. To Josef, I am afraid that your conclusion is not right, as in the patch, it will only merge delayed refs when they are in same sequence number. In fact, in the patch, conditions for merge is even more restrict than

Re: [GIT PULL] Qgroup rework with other Fujitsu fix.

On 06/07/2015 11:06 PM, Qu Wenruo wrote: Hi Chris, Please pull the 19 patchset from my branch for_chris_4.2. We have tested it in a week. Although it is originally based on 4.1-rc5, not the integration branch. Quick tests shows no new bugs, although we will rerun the full test, I'll send the

[PATCH] Btrfs: use received_uuid of parent during send

constant is the received uuid. So instead check to see if we have received_uuid set on the root, and if so use that as the clone source, as btrfs receive looks for matches either in received_uuid or uuid. Thanks, Reported-by: Neil Horman nhor...@redhat.com Signed-off-by: Josef Bacik jba...@fb.com

Re: [PATCH] btrfs-progs: specify mountpoint for recieve

On 05/28/2015 08:37 AM, David Sterba wrote: On Wed, May 27, 2015 at 01:51:29PM -0400, Josef Bacik wrote: In a chroot environment we may not have /proc mounted, which makes btrfs receive freak out since it wants to know the base directory where are are mounted for things like clone

[PATCH] btrfs-progs: specify mountpoint for recieve

in a chroot. Thanks, Signed-off-by: Josef Bacik jba...@fb.com --- cmds-receive.c | 47 +++ 1 file changed, 31 insertions(+), 16 deletions(-) diff --git a/cmds-receive.c b/cmds-receive.c index b7cf3f9..28ae8e9 100644 --- a/cmds-receive.c +++ b/cmds-receive.c

[PATCH 2/2] fstests: btrfs balance with dm log writes test

This test runs fsstress+balance+defrag and then replays every FUA in the log and mounts, scrubs and then fscks the fs to make sure it does the balance recovery properly. Thanks, Signed-off-by: Josef Bacik jba...@fb.com --- tests/btrfs/093 | 135

Re: [PATCH] fstests: test btrfs send after swapping directory names differently

btrfs patch: Btrfs: incremental send, check if orphanized dir inode needs delayed rename Signed-off-by: Filipe Manana fdman...@suse.com Verified it passed with the patches applied and failed without them. Test looks straightforward and good to me, you can add Reviewed-by: Josef Bacik jba

[PATCH V2] Btrfs-progs: check for matchin free space in cache

can be sure our tests only catch real problems. Thanks, Signed-off-by: Josef Bacik jba...@fb.com --- V1-V2: - only check matched if __load_free_space_cache is successful ctree.h| 1 + extent-tree.c | 51 +++ free-space-cache.c | 11

Re: [PATCH v2] Btrfs: fix data loss after concurrent fsyncs for files in the same subvol

On 04/17/2015 02:20 PM, Filipe Manana wrote: If we have concurrent fsync calls against files living in the same subvolume, we have some time window where we don't add the collected ordered extents to the running transaction's list of ordered extents and return success to userspace. This can

[PATCH] Btrfs-progs: check for matchin free space in cache

can be sure our tests only catch real problems. Thanks, Signed-off-by: Josef Bacik jba...@fb.com --- ctree.h| 1 + extent-tree.c | 51 +++ free-space-cache.c | 11 +++ 3 files changed, 63 insertions(+) diff --git a/ctree.h b

[PATCH] Btrfs: log entire range for FULL_SYNC

and then only log up to the range we've inlcluded then fsck complains because i_size is the last extent in the flie. Fix this by making sure we log the entire file and not just the range requested. With this patch we now pass the log writes test. Thanks, Signed-off-by: Josef Bacik jba...@fb.com

Re: [PATCH v2] fstests: regression test for btrfs file range cloning

by the following linux kernel patch: Btrfs: fix range cloning when same inode used as source and destination Signed-off-by: Filipe Manana fdman...@suse.com Reviewed-by: Josef Bacik jba...@fb.com Thanks, Josef -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body

Re: [PATCH 1/3] dm: log writes target

On 03/21/2015 05:50 PM, Dave Chinner wrote: On Thu, Mar 19, 2015 at 04:31:08PM -0400, Josef Bacik wrote: This creates a new target that is meant for file system developers to test file system integrity at particular points in the life of a file system. We capture all write requests

Re: [dm-devel] [PATCH 1/3] dm: log writes target

On 03/23/2015 02:02 PM, Vivek Goyal wrote: On Thu, Mar 19, 2015 at 04:31:08PM -0400, Josef Bacik wrote: [..] + * We log writes only after they have been flushed, this makes the log describe + * close to the order in which the data hits the actual disk, not its cache. So + * for example

Re: [PATCH v3] fstests: test for btrfs transaction abortion on device with discard support

Couldn't read tree root Couldn't open file system (...) Signed-off-by: Filipe Manana fdman...@suse.com Reviewed-by: Josef Bacik jba...@fb.com Thanks, Josef -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More

[PATCH 1/3] dm: log writes target V2

to do this replay. The idea behind this is to give file system developers to verify that the file system is always consistent. Thanks, Signed-off-by: Josef Bacik jba...@fb.com --- V1-V2: fixed up stuff based on Zachs review. Documentation/device-mapper/dm-log-writes.txt | 136 + drivers/md

[PATCH 3/3] fstests: btrfs balance with dm log writes test

This test runs fsstress+balance+defrag and then replays every FUA in the log and mounts, scrubs and then fscks the fs to make sure it does the balance recovery properly. Thanks, Signed-off-by: Josef Bacik jba...@fb.com --- tests/btrfs/083 | 135

[PATCH 0/3] Device mapper log writes patches

Here are my patches for adding the dm-log-writes target to the kernel and the supporting xfstests to go along with it. The dm patch has a pretty detailed documentation file to describe the methodology behind the target and how to use it. The xfstest that is generic has been tested on btrfs, xfs

[PATCH 2/3] fstests: add dm-log-writes test and supporting code

. This test relies on the supporting userspace code I've written for dm-logs-writes. It can be found here https://github.com/josefbacik/log-writes.git Thanks, Signed-off-by: Josef Bacik jba...@fb.com --- README| 2 + common/config | 1 + common/dmlogwrites| 80

[PATCH 1/3] dm: log writes target

to do this replay. The idea behind this is to give file system developers to verify that the file system is always consistent. Thanks, Signed-off-by: Josef Bacik jba...@fb.com --- Documentation/device-mapper/dm-log-writes.txt | 136 + drivers/md/Kconfig| 16

Re: [PATCH 2/4] Btrfs: just free dummy extent buffers

On 03/18/2015 09:56 AM, David Sterba wrote: On Tue, Mar 17, 2015 at 04:38:01PM -0400, Josef Bacik wrote: If we fail during our sanity tests we could get NULL deref's because we unload the module before the dummy extent buffers are free'd via RCU. So check for this case and just free the things

[PATCH] Btrfs: fix outstanding_extents accounting in DIO

less than MAX_EXTENT_SIZE when we request an amount larger than MAX_EXTENT_SIZE. This fixes the problem Filipe reported with generic/300. Thanks, Reported-by: Filipe Manana fdman...@suse.com Signed-off-by: Josef Bacik jba...@fb.com --- fs/btrfs/inode.c | 19 ++- 1 file changed, 18

[PATCH 0/4] Accounting fixes

I've changed all of these a whole bunch recently, I think I've gotten all the issues worked out, please review/test to make sure there's not some other fucking corner I've missed. Thanks, Josef -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to

[PATCH 4/4] Btrfs: fix outstanding_extents accounting in DIO

Signed-off-by: Josef Bacik jba...@fb.com --- fs/btrfs/inode.c | 37 +++-- 1 file changed, 35 insertions(+), 2 deletions(-) diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c index 3717b3d..aa1fb53 100644 --- a/fs/btrfs/inode.c +++ b/fs/btrfs/inode.c @@ -7239,7 +7239,7

[PATCH 1/4] Btrfs: account merges/splits properly

reserved, but we only need 3, so we need to drop one of the reservations. The same problem existed for splits, we'd think we only need 3 extents when creating the hole but in reality we need 4. Thanks, Signed-off-by: Josef Bacik jba...@fb.com --- fs/btrfs/inode.c | 57

[PATCH 2/4] Btrfs: just free dummy extent buffers

If we fail during our sanity tests we could get NULL deref's because we unload the module before the dummy extent buffers are free'd via RCU. So check for this case and just free the things directly. Thanks, Signed-off-by: Josef Bacik jba...@fb.com --- fs/btrfs/extent_io.c | 6 ++ 1 file

[PATCH 3/4] Btrfs: add sanity test for outstanding_extents accounting

in the future. Thanks, Signed-off-by: Josef Bacik jba...@fb.com --- fs/btrfs/ctree.h | 3 + fs/btrfs/extent-tree.c | 3 + fs/btrfs/inode.c | 15 fs/btrfs/tests/inode-tests.c | 197 ++- 4 files changed, 217 insertions

[PATCH] Btrfs: account merges properly

reserved, but we only need 3, so we need to drop one of the reservations. This fixes the last accounting problem. Thanks, Signed-off-by: Josef Bacik jba...@fb.com --- -Same as the v2 of Btrfs: fix merge delalloc logic except it was rebased onto Chris's integration branch as an incremental to keep

[PATCH] Btrfs: just free dummy extent buffers

If we fail during our sanity tests we could get NULL deref's because we unload the module before the dummy extent buffers are free'd via RCU. So check for this case and just free the things directly. Thanks, Signed-off-by: Josef Bacik jba...@fb.com --- fs/btrfs/extent_io.c | 6 ++ 1 file

[PATCH] Btrfs: fix merge delalloc logic V2

two slightly larger than MAX_EXTENT size extents the accounting would incorrectly think we needed 4 outstanding extents when in fact we'd only need 3. Thanks, Reported-by: Markus Trippelsdorf mar...@trippelsdorf.de Signed-off-by: Josef Bacik jba...@fb.com --- V1-V2: fixed another corner case where

[PATCH] Btrfs: add sanity test for outstanding_extents accounting

in the future. Thanks, Signed-off-by: Josef Bacik jba...@fb.com --- fs/btrfs/ctree.h | 3 + fs/btrfs/inode.c | 15 fs/btrfs/tests/inode-tests.c | 160 ++- 3 files changed, 177 insertions(+), 1 deletion(-) diff --git a/fs/btrfs

[PATCH] Btrfs: account for the correct number of extents for delalloc reservations

Direct IO can easily pass in an buffer that is greater than BTRFS_MAX_EXTENT_SIZE, so take this into account when reserving extents in the delalloc reservation code. Thanks, Signed-off-by: Josef Bacik jba...@fb.com --- fs/btrfs/extent-tree.c | 6 +- 1 file changed, 5 insertions(+), 1

[PATCH] Btrfs: fix merge delalloc logic

a tiny extent into a huge extent, when in reality we already had a huge extent and needed to use the other side in our logic. This fixes the regression that was reported by a user on list. Thanks, Reported-by: Markus Trippelsdorf mar...@trippelsdorf.de Signed-off-by: Josef Bacik jba...@fb.com --- fs

Re: [PATCH] fstest: btrfs/083: Test for incorrect exclusive refernce number after file clone.

On 03/12/2015 08:38 AM, Filipe David Manana wrote: On Tue, Mar 10, 2015 at 7:46 AM, Qu Wenruo quwen...@cn.fujitsu.com wrote: [Problem] Since commit fcebe4562dec83b3f8d308 (Btrfs: rework qgroup accounting), quota data update is delayed after delayed_ref calculation, and lacks correct protection

[PATCH] Btrfs: fix ASSERT(list_empty(cur_trans-dirty_bgs_list) V3

but not the extent root. Thanks, Reported-by: David Sterba dste...@suse.cz Signed-off-by: Josef Bacik jba...@fb.com --- V2-V3: Actually fix the bug. Move the write dirty bgs stuff out of the update cowonly roots, then just check if we dirtied roots again and do the whole thing over again. fs/btrfs

[PATCH] Btrfs: fix ASSERT(list_empty(cur_trans-dirty_bgs_list) V2

but not the extent root. Thanks, Reported-by: David Sterba dste...@suse.cz Signed-off-by: Josef Bacik jba...@fb.com --- V1-V2: moved the logic to the loop in commit_cowonly_roots so we can be sure that we're always running delayed refs and always writing the drity block groups. fs/btrfs/transaction.c

Re: [PATCH] Btrfs: fix ASSERT(list_empty(cur_trans-dirty_bgs_list)

On 03/03/2015 10:12 PM, Liu Bo wrote: On Tue, Mar 03, 2015 at 05:35:33PM +0100, David Sterba wrote: On Tue, Mar 03, 2015 at 07:02:58PM +0800, Liu Bo wrote: On Mon, Mar 02, 2015 at 12:46:55PM -0500, Josef Bacik wrote: Dave could hit this assert consistently running btrfs/078. This is because

Re: [PATCH] Btrfs: fix ASSERT(list_empty(cur_trans-dirty_bgs_list)

On 03/03/2015 11:35 AM, David Sterba wrote: On Tue, Mar 03, 2015 at 07:02:58PM +0800, Liu Bo wrote: On Mon, Mar 02, 2015 at 12:46:55PM -0500, Josef Bacik wrote: Dave could hit this assert consistently running btrfs/078. This is because when we update the block groups we could truncate

Re: [PATCH 3/3] btrfs: qgroup: fix a wrong parameter of no_quota.

On 03/03/2015 06:18 AM, Dongsheng Yang wrote: On 02/26/2015 02:05 PM, Dongsheng Yang wrote: Wait a minute, this patch seems not working well in accounting quota number when deleting data shared by different subvolumes. I will investigate more about it and send a V2 out. I have sent a fstest

Re: [PATCH 1/7] btrfs: Check qgroup level in kernel qgroup assign.

create things) btrfs library with enough check, and only rely on kernel ioctl. So Add level checks in kernel too. Signed-off-by: Qu Wenruo quwen...@cn.fujitsu.com Reviewed-by: Josef Bacik jba...@fb.com Thanks, Josef -- To unsubscribe from this list: send the line unsubscribe linux-btrfs

Re: [PATCH 4/7] btrfs: Update btrfs qgroup status item when rescan is done.

On 02/27/2015 03:24 AM, Qu Wenruo wrote: Update qgroup status when rescan is done. Before this patch, status item is not updated on rescan finish, which causing the RESCAN and INCONSISTENT flags never cleared. Signed-off-by: Qu Wenruo quwen...@cn.fujitsu.com Reviewed-by: Josef Bacik jba

Re: [PATCH 1/9] btrfs: qgroup: move WARN_ON() to the correct location.

it normally. This patch move this WARN_ON() before freeing qg-excl. Signed-off-by: Dongsheng Yang yangds.f...@cn.fujitsu.com Reviewed-by: Satoru Takeuchi takeuchi_sat...@jp.fujitsu.com Reviewed-by: Josef Bacik jba...@fb.com Thanks, Josef -- To unsubscribe from this list: send the line unsubscribe linux

Re: [PATCH 3/7] btrfs: qgroup: Fix dead judgement on qgroup_rescan_leaf() return value.

(). This caused the bug that INCONSISTENT flag is never cleared. So change the comment and fix the dead judgment. Signed-off-by: Qu Wenruo quwen...@cn.fujitsu.com Reviewed-by: Josef Bacik jba...@fb.com Thanks, Josef -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body

Re: [PATCH 2/7] btrfs: Don't allow subvolid = (1 BTRFS_QGROUP_LEVEL_SHIFT) to be created

) - 256(last free objectid)) to (u48 max -256(first free objectid)). But we still have near u48(that's 15 digits in dec), so that should not be a huge problem. Signed-off-by: Qu Wenruo quwen...@cn.fujitsu.com Reviewed-by: Josef Bacik jba...@fb.com Thanks, Josef -- To unsubscribe from this list: send

Re: [PATCH 6/7] btrfs: quota: Automatically update related qgroups or mark INCONSISTENT flags when assigning/deleting a qgroup relations.

, and return 1 to info user-land. BTW since the quick path is much the same of qgroup_excl_accounting(), so move the core of it to __qgroup_excl_accounting() and reuse it. Reviewed-by: Josef Bacik jba...@fb.com Thanks, Josef -- To unsubscribe from this list: send the line unsubscribe linux-btrfs

Re: [PATCH 7/7] btrfs: quota: Update quota tree after qgroup relationship change.

to in memory quota trees and user will get up-to-date results. Reviewed-by: Josef Bacik jba...@fb.com Thanks, Josef -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org

Re: [PATCH 2/9] btrfs: qgroup: inherit limit info from srcgroup in creating snapshot.

-by: Josef Bacik jba...@fb.com Thanks, Josef -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: [PATCH 5/7] btrfs: qgroup: clear STATUS_FLAG_ON in disabling quota.

# btrfs quota rescan /mnt quota rescan started --- expecting it fail here. # echo $? 0 Signed-off-by: Dongsheng Yang yangds.f...@cn.fujitsu.com Reviewed-by: Josef Bacik jba...@fb.com Thanks, Josef -- To unsubscribe from this list: send the line unsubscribe linux-btrfs

[PATCH] Btrfs: fix ASSERT(list_empty(cur_trans-dirty_bgs_list)

but not the extent root. Thanks, Reported-by: David Sterba dste...@suse.cz Signed-off-by: Josef Bacik jba...@fb.com --- fs/btrfs/transaction.c | 12 +--- 1 file changed, 9 insertions(+), 3 deletions(-) diff --git a/fs/btrfs/transaction.c b/fs/btrfs/transaction.c index 038fcf6..a7a413f 100644

[PATCH] Btrfs: remove extra run_delayed_refs in update_cowonly_root

This got added with my dirty_bgs patch, it's not needed. Thanks, Signed-off-by: Josef Bacik jba...@fb.com --- fs/btrfs/transaction.c | 3 --- 1 file changed, 3 deletions(-) diff --git a/fs/btrfs/transaction.c b/fs/btrfs/transaction.c index a7a413f..3d017d6 100644 --- a/fs/btrfs/transaction.c

[PATCH] Btrfs: make async space flushing suck less when we're full

. If we have 98% of our space actually used then doing things async really isn't going to be helpful, so skip it. This patch made the rm's on our extremely full gluster boxes not take all eternity. Thanks, Signed-off-by: Josef Bacik jba...@fb.com --- fs/btrfs/extent-tree.c | 14

[PATCH] Btrfs: throttle transactions based on dirty bgs

groups and we abort the transaction. So keep track of the number of dirty block groups and use that space in seeing if we have enough global reserve. Thanks, Signed-off-by: Josef Bacik jba...@fb.com --- fs/btrfs/extent-tree.c | 3 +++ fs/btrfs/transaction.c | 1 + fs/btrfs/transaction.h | 1

Re: [PATCH 0/3] btrfs: ENOMEM bugfixes

On 02/20/2015 04:20 PM, Omar Sandoval wrote: On Tue, Feb 17, 2015 at 02:51:06AM -0800, Omar Sandoval wrote: Hi, As it turns out, running with low memory is a really easy way to shake out undesirable behavior in Btrfs. This can be especially bad when considering that a memory limit is really

[PATCH] Btrfs: abort the transaction if we fail to update the free space cache inode

if this fails we have to abort the transaction so we make sure we don't end up with stale free space cache. Thanks, Signed-off-by: Josef Bacik jba...@fb.com --- fs/btrfs/extent-tree.c | 16 1 file changed, 16 insertions(+) diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c

[PATCH] Btrfs: add metadump_v2 support to the kernel

, Signed-off-by: Josef Bacik jba...@fb.com --- fs/btrfs/ctree.h | 1 + fs/btrfs/inode.c | 4 2 files changed, 5 insertions(+) diff --git a/fs/btrfs/ctree.h b/fs/btrfs/ctree.h index 1675602..bc16147 100644 --- a/fs/btrfs/ctree.h +++ b/fs/btrfs/ctree.h @@ -356,6 +356,7 @@ static inline unsigned

Re: [PATCH 3/3] Btrfs: account for large extents with enospc

On 02/11/2015 11:36 PM, Liu Bo wrote: On Wed, Feb 11, 2015 at 03:08:59PM -0500, Josef Bacik wrote: On our gluster boxes we stream large tar balls of backups onto our fses. With 160gb of ram this means we get really large contiguous ranges of dirty data, but the way our ENOSPC stuff works

[PATCH 3/3 v2] Btrfs: account for large extents with enospc

. Thanks, Signed-off-by: Josef Bacik jba...@fb.com --- V1-V2: -use bit shift instead of div64_u64. -skip math if num_bytes max_size in drop_outstanding_extents fs/btrfs/ctree.h | 8 fs/btrfs/extent-tree.c | 16 ++- fs/btrfs/extent_io.c | 2 +- fs/btrfs/inode.c

[PATCH 3/3] Btrfs: account for large extents with enospc

. Thanks, Signed-off-by: Josef Bacik jba...@fb.com --- fs/btrfs/ctree.h | 2 ++ fs/btrfs/extent-tree.c | 16 + fs/btrfs/extent_io.c | 2 +- fs/btrfs/inode.c | 63 +- 4 files changed, 76 insertions(+), 7 deletions(-) diff

[PATCH 2/3] Btrfs: don't set and clear delalloc for O_DIRECT writes

We do this to get the space accounting, but this is just needless churn on the io_tree, so just drop setting/clearing delalloc and just drop the reserved data space when we have a successfull allocation. Thanks, Signed-off-by: Josef Bacik jba...@fb.com --- fs/btrfs/inode.c | 8 ++-- 1 file

[PATCH 1/3] Btrfs: only adjust outstanding_extents when we do a short write

satisfy the entire range request in get_blocks_direct, otherwise we are good using our original reservation. So drop the unconditional inc and the drop of the metadata space that we have for the unconditional inc. Thanks, Signed-off-by: Josef Bacik jba...@fb.com --- fs/btrfs/inode.c | 11

Re: [PATCH v3] btrfs: Fix out-of-space bug

On 02/11/2015 05:01 AM, Zhaolei wrote: From: Zhao Lei zhao...@cn.fujitsu.com Btrfs will report NO_SPACE when we create and remove files for several times, and we can't write to filesystem until mount it again. Steps to reproduce: 1: Create a single-dev btrfs fs with default option 2: Write

Re: [PATCH 09/16] Btrfs-progs: Introduce metadump_v2

On 02/10/2015 02:14 PM, David Sterba wrote: On Mon, Feb 09, 2015 at 03:03:12PM -0500, Josef Bacik wrote: The METADUMP super flag makes us skip doing the chunk tree reading which isn't helpful for the new restore since we have a valid chunk tree. But we still want to have a way for the kernel

[PATCH 08/16] Btrfs-progs: multi-thread btrfs-image restore

For some reason we only allow btrfs-image restore to have one thread, which is incredibly slow with large images. So allow us to do work with more than just one thread. This made my restore go from 16 minutes to 3 minutes. Thanks, Signed-off-by: Josef Bacik jba...@fb.com --- btrfs-image.c

[PATCH 11/16] Btrfs-progs: remove global transaction from fsck

from ending up with ENOSPC because we pinned everything and allows the code to be a bit simpler. Thanks, Signed-off-by: Josef Bacik jba...@fb.com --- cmds-check.c | 230 +- ctree.h | 1 + disk-io.c | 2 + extent-tree.c | 7

[PATCH 12/16] Btrfs-progs: unpin excluded extents as we fix things

We don't want to keep extent records pinned down if we fix stuff as we may need the space and we can be pretty sure that these records are correct. Thanks, Signed-off-by: Josef Bacik jba...@fb.com --- cmds-check.c | 12 +++- 1 file changed, 11 insertions(+), 1 deletion(-) diff --git

[PATCH 14/16] Btrfs-progs: make debug-tree spit out full_backref flag

Currently btrfs-debug-tree ignores the FULL_BACKREF flag which makes it hard to figure out problems related to FULL_BACKREF. Thanks, Signed-off-by: Josef Bacik jba...@fb.com --- print-tree.c | 4 1 file changed, 4 insertions(+) diff --git a/print-tree.c b/print-tree.c index 3a7c13c

[PATCH 09/16] Btrfs-progs: Introduce metadump_v2

to skip some of the device extent checks in fsck since those will obviously not match. Thanks, Signed-off-by: Josef Bacik jba...@fb.com --- btrfs-image.c | 3 +++ cmds-check.c | 9 +++-- ctree.h | 1 + 3 files changed, 11 insertions(+), 2 deletions(-) diff --git a/btrfs-image.c b/btrfs

[PATCH 15/16] Btrfs-progs: skip opening all devices with restore

is failing. This will make it so that we use the fd we've already opened for opening our ctree. Thanks, Signed-off-by: Josef Bacik jba...@fb.com --- btrfs-find-root.c | 2 +- btrfs-image.c | 9 ++--- chunk-recover.c | 2 +- disk-io.c | 8 +--- disk-io.h | 3 ++- super

<    4   5   6   7   8   9   10   11   12   13   >