Re: [PATCH v3 0/9] Btrfs: free space B-tree

2015-09-14 Thread Omar Sandoval
On Mon, Sep 14, 2015 at 04:51:48PM +0200, Holger Hoffstätte wrote: > On 09/14/15 08:04, Omar Sandoval wrote: > > I went back and fixed the issues that came up since v2. Changes below. I > > removed Josef's Reviewed-by on patch 9 because it was completely > > rewritten to cha

[PATCH v3 5/9] Btrfs: introduce the free space B-tree on-disk format

2015-09-14 Thread Omar Sandoval
From: Omar Sandoval <osan...@fb.com> The on-disk format for the free space tree is straightforward. Each block group is represented in the free space tree by a free space info item that stores accounting information: whether the free space for this block group is stored as bitmaps or e

[PATCH v3 2/9] Btrfs: add extent buffer bitmap sanity tests

2015-09-14 Thread Omar Sandoval
From: Omar Sandoval <osan...@fb.com> Sanity test the extent buffer bitmap operations (test, set, and clear) against the equivalent standard kernel operations. Signed-off-by: Omar Sandoval <osan...@fb.com> --- fs/btrfs/extent_io.c | 34 ++ fs/btrfs

[PATCH v3 8/9] Btrfs: wire up the free space tree to the extent tree

2015-09-14 Thread Omar Sandoval
From: Omar Sandoval <osan...@fb.com> The free space tree is updated in tandem with the extent tree. There are only a handful of places where we need to hook in: 1. Block group creation 2. Block group deletion 3. Delayed refs (extent creation and deletion) 4. Block group caching Sign

[PATCH v3 6/9] Btrfs: implement the free space B-tree

2015-09-14 Thread Omar Sandoval
From: Omar Sandoval <osan...@fb.com> The free space cache has turned out to be a scalability bottleneck on large, busy filesystems. When the cache for a lot of block groups needs to be written out, we can get extremely long commit times; if this happens in the critical section,

[PATCH v3 1/9] Btrfs: add extent buffer bitmap operations

2015-09-14 Thread Omar Sandoval
From: Omar Sandoval <osan...@fb.com> These are going to be used for the free space tree bitmap items. Signed-off-by: Omar Sandoval <osan...@fb.com> --- fs/btrfs/extent_io.c | 149 +++ fs/btrfs/extent_io.h | 6 +++ 2 files changed, 15

[PATCH v3 4/9] Btrfs: refactor caching_thread()

2015-09-14 Thread Omar Sandoval
From: Omar Sandoval <osan...@fb.com> We're also going to load the free space tree from caching_thread(), so we should refactor some of the common code. Signed-off-by: Omar Sandoval <osan...@fb.com> --- fs/btrfs/ctree.h | 3 +++ fs/btrfs/extent

[PATCH v3 3/9] Btrfs: add helpers for read-only compat bits

2015-09-14 Thread Omar Sandoval
From: Omar Sandoval <osan...@fb.com> We're finally going to add one of these for the free space tree, so let's add the same nice helpers that we have for the incompat bits. While we're add it, also add helpers to clear the bits. Reviewed-by: Josef Bacik <jba...@fb.com> Signed-

[PATCH v3 9/9] Btrfs: add free space tree mount option

2015-09-14 Thread Omar Sandoval
From: Omar Sandoval <osan...@fb.com> Now we can finally hook up everything so we can actually use free space tree. The free space tree is enabled by passing the space_cache=v2 mount option. On the first mount with the this option set, the free space tree will be created and the FREE_SPAC

[PATCH v3 7/9] Btrfs: add free space tree sanity tests

2015-09-14 Thread Omar Sandoval
From: Omar Sandoval <osan...@fb.com> This tests the operations on the free space tree trying to excercise all of the main cases for both formats. Between this and xfstests, the free space tree should have pretty good coverage. Signed-off-by: Omar Sandoval <osan...@fb.com> --- fs/bt

[PATCH v3 0/9] Btrfs: free space B-tree

2015-09-14 Thread Omar Sandoval
ommented a bunch of stuff in the extent buffer bitmap operations and refactored some of the complicated logic v1: http://www.spinics.net/lists/linux-btrfs/msg46713.html Omar Sandoval (9): Btrfs: add extent buffer bitmap operations Btrfs: add extent buffer bitmap sanity tests Btrfs: add helpe

[PATCH v3 3/3] btrfs-progs: check the free space tree in btrfsck

2015-09-14 Thread Omar Sandoval
From: Omar Sandoval <osan...@fb.com> This reuses the existing code for checking the free space cache, we just need to load the free space tree. While we do that, we check a couple of invariants on the free space tree itself. This requires pulling in some code from the kernel to exclude the

[PATCH v3 2/3] btrfs-progs: add basic awareness of the free space tree

2015-09-14 Thread Omar Sandoval
From: Omar Sandoval <osan...@fb.com> To start, let's tell btrfs-progs to read the free space root and how to print the on-disk format of the free space tree. However, we're not adding the FREE_SPACE_TREE read-only compat bit to the set of supported bits because progs doesn't know how t

[PATCH v3 1/3] btrfs-progs: use calloc instead of malloc+memset for tree roots

2015-09-14 Thread Omar Sandoval
From: Omar Sandoval <osan...@fb.com> Signed-off-by: Omar Sandoval <osan...@fb.com> --- disk-io.c | 22 +++--- 1 file changed, 7 insertions(+), 15 deletions(-) diff --git a/disk-io.c b/disk-io.c index 1d4889322411..8496aded31c4 100644 --- a/disk-io.c +++ b/disk-io.

Re: Crash when trying to start a replace on missing device

2015-09-12 Thread Omar Sandoval
On Sat, Sep 12, 2015 at 04:51:18PM -0600, Martin Bakiev wrote: > Hi guys, > > I'm just doing testing with btrfs and I ran into a crash when > simulating a failed drive. I yanked out one (/dev/sdc) of 4 drives and > tried to replace it with another (/dev/sdf) with this command: > > btrfs replace

Re: [PATCH v2 9/9] Btrfs: add free space tree mount option

2015-09-10 Thread Omar Sandoval
On Wed, Sep 09, 2015 at 02:00:23PM +0200, David Sterba wrote: > On Thu, Sep 03, 2015 at 12:44:27PM -0700, Omar Sandoval wrote: > > Now we can finally hook up everything so we can actually use free space > > tree. On the first mount with the free_space_tree mount option, the free

Re: [PATCH v2 0/9] free space B-tree

2015-09-10 Thread Omar Sandoval
free space cache and not too much more than not using the cache. Now that I think about it, I only profiled it under heavy load, though, it'd probably be a good idea to get some numbers for more typical workloads, but I don't currently have access to any reasonable hardware. Thanks, Omar > Omar Sa

Re: [PATCH v2 0/9] free space B-tree

2015-09-10 Thread Omar Sandoval
On Fri, Sep 11, 2015 at 11:58:13AM +0800, Qu Wenruo wrote: > > > Omar Sandoval wrote on 2015/09/10 20:48 -0700: > >On Fri, Sep 11, 2015 at 09:21:13AM +0800, Qu Wenruo wrote: > >>Hi Omar, > >> > >>Thanks for your patchset. > >>Quite a nice one, a

[PATCH v2 4/9] Btrfs: refactor caching_thread()

2015-09-03 Thread Omar Sandoval
We're also going to load the free space tree from caching_thread(), so we should refactor some of the common code. Signed-off-by: Omar Sandoval <osan...@fb.com> --- fs/btrfs/ctree.h | 3 +++ fs/btrfs/extent-tree.c | 59 -- 2 files c

[PATCH v2 6/9] Btrfs: implement the free space B-tree

2015-09-03 Thread Omar Sandoval
the extent tree. Signed-off-by: Omar Sandoval <osan...@fb.com> --- fs/btrfs/Makefile |2 +- fs/btrfs/ctree.h | 25 +- fs/btrfs/extent-tree.c | 15 +- fs/btrfs/free-space-tree.c | 1501 fs/btrfs/free-space-tree.h

[PATCH v2 7/9] Btrfs: add free space tree sanity tests

2015-09-03 Thread Omar Sandoval
This tests the operations on the free space tree trying to excercise all of the main cases for both formats. Between this and xfstests, the free space tree should have pretty good coverage. Signed-off-by: Omar Sandoval <osan...@fb.com> --- fs/btrfs/Makefile | 3

[PATCH v2 2/9] Btrfs: add extent buffer bitmap sanity tests

2015-09-03 Thread Omar Sandoval
Sanity test the extent buffer bitmap operations (test, set, and clear) against the equivalent standard kernel operations. Signed-off-by: Omar Sandoval <osan...@fb.com> --- fs/btrfs/extent_io.c | 34 ++ fs/btrfs/extent_io.h | 4 +- fs/btrfs/tests/ext

[PATCH v2 8/9] Btrfs: wire up the free space tree to the extent tree

2015-09-03 Thread Omar Sandoval
The free space tree is updated in tandem with the extent tree. There are only a handful of places where we need to hook in: 1. Block group creation 2. Block group deletion 3. Delayed refs (extent creation and deletion) 4. Block group caching Signed-off-by: Omar Sandoval <osan...@fb.com> -

[PATCH v2 0/9] free space B-tree

2015-09-03 Thread Omar Sandoval
t - Commented a bunch of stuff in the extent buffer bitmap operations and refactored some of the complicated logic - Added sanity tests for the extent buffer bitmap operations and free space tree (patches 2 and 6) - Added Josef's Reviewed-by tags Omar Sandoval (9): Btrfs: add extent buffer bitm

[PATCH v2 1/9] Btrfs: add extent buffer bitmap operations

2015-09-03 Thread Omar Sandoval
These are going to be used for the free space tree bitmap items. Signed-off-by: Omar Sandoval <osan...@fb.com> --- fs/btrfs/extent_io.c | 149 +++ fs/btrfs/extent_io.h | 6 +++ 2 files changed, 155 insertions(+) diff --git a/fs/btrfs/exten

[PATCH v2 5/9] Btrfs: introduce the free space B-tree on-disk format

2015-09-03 Thread Omar Sandoval
<jba...@fb.com> Signed-off-by: Omar Sandoval <osan...@fb.com> --- fs/btrfs/ctree.h | 38 ++ include/trace/events/btrfs.h | 3 ++- 2 files changed, 40 insertions(+), 1 deletion(-) diff --git a/fs/btrfs/ctree.h b/fs/btrfs/ctree.h index

[PATCH v2 9/9] Btrfs: add free space tree mount option

2015-09-03 Thread Omar Sandoval
Bacik <jba...@fb.com> Signed-off-by: Omar Sandoval <osan...@fb.com> --- fs/btrfs/ctree.h | 7 ++- fs/btrfs/disk-io.c | 26 ++ fs/btrfs/super.c | 21 +++-- 3 files changed, 51 insertions(+), 3 deletions(-) diff --git a/fs/btrfs/ctree.

Re: [PATCH v2 0/9] free space B-tree

2015-09-03 Thread Omar Sandoval
On Fri, Sep 04, 2015 at 09:29:45AM +0800, Zhao Lei wrote: > Hi, Omar Sandoval > > [PATCH 7/9] have following compiler warning: > fs/btrfs/tests/free-space-tree-tests.c: In function > '__check_free_space_extents': > fs/btrfs/tests/free-space-tree-tests.c:45: warning: 'o

Re: [PATCH v2 8/9] Btrfs: wire up the free space tree to the extent tree

2015-09-03 Thread Omar Sandoval
On Thu, Sep 03, 2015 at 12:44:26PM -0700, Omar Sandoval wrote: > The free space tree is updated in tandem with the extent tree. There are > only a handful of places where we need to hook in: > > 1. Block group creation > 2. Block group deletion > 3. Delayed refs (extent creation

[PATCH 1/6] Btrfs: add extent buffer bitmap operations

2015-09-01 Thread Omar Sandoval
From: Omar Sandoval <osan...@fb.com> These are going to be used for the free space tree bitmap items. Signed-off-by: Omar Sandoval <osan...@fb.com> --- fs/btrfs/extent_io.c | 101 +++ fs/btrfs/extent_io.h | 6 +++ 2 files changed, 10

[PATCH 0/6] free space B-tree

2015-09-01 Thread Omar Sandoval
From: Omar Sandoval <osan...@fb.com> Hi, At Facebook, we are still running into the issue of long commit stalls on large filesystems (say, 10s of TBs). 1bbc621ef284 ("Btrfs: allow block group cache writeout outside critical section in commit") was a stopgap, but it wasn't enoug

[PATCH 3/6] Btrfs: introduce the free space B-tree on-disk format

2015-09-01 Thread Omar Sandoval
From: Omar Sandoval <osan...@fb.com> The on-disk format for the free space tree is straightforward. Each block group is represented in the free space tree by a free space info item that stores accounting information: whether the free space for this block group is stored as bitmaps or e

[PATCH 2/6] Btrfs: add helpers for read-only compat bits

2015-09-01 Thread Omar Sandoval
From: Omar Sandoval <osan...@fb.com> We're finally going to add one of these for the free space tree, so let's add the same nice helpers that we have for the incompat bits. Signed-off-by: Omar Sandoval <osan...@fb.com> --- fs/btrfs/ctree.h | 34 ++ 1

[PATCH 4/6] Btrfs: implement the free space B-tree

2015-09-01 Thread Omar Sandoval
From: Omar Sandoval <osan...@fb.com> The free space cache has turned out to be a scalability bottleneck on large, busy filesystems. When the cache for a lot of block groups needs to be written out, we can get extremely long commit times; if this happens in the critical section,

Re: [PATCH 1/6] Btrfs: add extent buffer bitmap operations

2015-09-01 Thread Omar Sandoval
On Tue, Sep 01, 2015 at 03:25:54PM -0400, Josef Bacik wrote: > On 09/01/2015 03:01 PM, Omar Sandoval wrote: > >From: Omar Sandoval <osan...@fb.com> > > > >These are going to be used for the free space tree bitmap items. > > > >Signed-off-by: Omar Sandoval

Re: [PATCH 0/6] free space B-tree

2015-09-01 Thread Omar Sandoval
Urgh, sorry about the duplicates, either git send-email or Gmail is being weird... -- Omar -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

[PATCH 5/6] Btrfs: wire up the free space tree to the extent tree

2015-09-01 Thread Omar Sandoval
From: Omar Sandoval <osan...@fb.com> The free space tree is updated in tandem with the extent tree. There are only a handful of places where we need to hook in: 1. Block group creation 2. Block group deletion 3. Delayed refs (extent creation and deletion) 4. Block group caching Sign

[PATCH 6/6] Btrfs: add free space tree mount option

2015-09-01 Thread Omar Sandoval
From: Omar Sandoval <osan...@fb.com> Now we can finally hook up everything so we can actually use free space tree. On the first mount with the free_space_tree mount option, the free space tree will be created and the FREE_SPACE_TREE read-only compat bit will be set. Any time the file

[PATCH 3/3] btrfs-progs: check the free space tree in btrfsck

2015-09-01 Thread Omar Sandoval
From: Omar Sandoval <osan...@fb.com> This reuses the existing code for checking the free space cache, we just need to load the free space tree. While we do that, we check a couple of invariants on the free space tree itself. Signed-off-by: Omar Sandoval <osan...@fb.com> ---

[PATCH 2/3] btrfs-progs: add basic awareness of the free space tree

2015-09-01 Thread Omar Sandoval
From: Omar Sandoval <osan...@fb.com> To start, let's tell btrfs-progs to read the free space root and how to print the on-disk format of the free space tree. However, we're not adding the FREE_SPACE_TREE read-only compat bit to the set of supported bits because progs doesn't know how t

[PATCH 1/3] btrfs-progs: use calloc instead of malloc+memset for tree roots

2015-09-01 Thread Omar Sandoval
From: Omar Sandoval <osan...@fb.com> Signed-off-by: Omar Sandoval <osan...@fb.com> --- disk-io.c | 22 +++--- 1 file changed, 7 insertions(+), 15 deletions(-) diff --git a/disk-io.c b/disk-io.c index 1d4889322411..8496aded31c4 100644 --- a/disk-io.c +++ b/disk-io.

Re: [PATCH 4/6] Btrfs: implement the free space B-tree

2015-09-01 Thread Omar Sandoval
On Tue, Sep 01, 2015 at 03:44:27PM -0400, Josef Bacik wrote: > On 09/01/2015 03:13 PM, Omar Sandoval wrote: > >From: Omar Sandoval <osan...@fb.com> > > > >The free space cache has turned out to be a scalability bottleneck on > >large, busy filesystems. When th

Re: [PATCH 5/6] Btrfs: wire up the free space tree to the extent tree

2015-09-01 Thread Omar Sandoval
On Tue, Sep 01, 2015 at 03:48:57PM -0400, Josef Bacik wrote: > On 09/01/2015 03:05 PM, Omar Sandoval wrote: > >From: Omar Sandoval <osan...@fb.com> > > > >The free space tree is updated in tandem with the extent tree. There are > >only a handful of places where we

Re: mount command now shows subvol and subvolid

2015-08-31 Thread Omar Sandoval
On Sun, Aug 30, 2015 at 11:17:44AM -0600, Chris Murphy wrote: > Does anyone know when this changed? Maybe it's a 4.2 thing... anyway > it's very much welcome! > > /dev/sda2 on /home type btrfs > (rw,relatime,seclabel,space_cache,subvolid=258,subvol=/home) > > /dev/sdc1 on /brick0 type btrfs >

Re: [BUG][v4.2-rc7] Problem replacing a missing disk for a raid5-degraded filesystem

2015-08-19 Thread Omar Sandoval
On Wed, Aug 19, 2015 at 11:41:55AM -0700, Omar Sandoval wrote: On Wed, Aug 19, 2015 at 07:11:20PM +0200, Goffredo Baroncelli wrote: Hi all, playing with raid5 and btrfs replace I found a BUG. Basically it seems that if I try to replace a missing disk of a degraded filesystem I got

Re: [BUG][v4.2-rc7] Problem replacing a missing disk for a raid5-degraded filesystem

2015-08-19 Thread Omar Sandoval
On Wed, Aug 19, 2015 at 07:11:20PM +0200, Goffredo Baroncelli wrote: Hi all, playing with raid5 and btrfs replace I found a BUG. Basically it seems that if I try to replace a missing disk of a degraded filesystem I got a kernel BUG. This is reproducible at 100% for me. Hi, Goffredo, this

Re: [PATCH 2/3] btrfs: add replace missing and replace RAID 5/6 to profile configs

2015-07-28 Thread Omar Sandoval
On Tue, Jul 28, 2015 at 06:52:07PM +0200, David Sterba wrote: On Tue, Jul 28, 2015 at 12:22:56AM +0200, Brendan Hide wrote: It does not, I apparently forgot that you could use single to concatenate multiple devices. I'll fix that in v2. Thanks for reviewing! Late to the party.

Re: [PATCH] fstests: regression test for the btrfs clone ioctl

2015-07-24 Thread Omar Sandoval
trace e3d3376b23a57041 ]--- This issue is addressed by the following linux kernel patch for btrfs: Btrfs: fix file corruption after cloning inline extents. Signed-off-by: Filipe Manana fdman...@suse.com Reviewed-by: Omar Sandoval osan...@fb.com Tested that it passes on 4.2-rc3 and fails

Re: [PATCH] fstests: btrfs test to exercise shared extent reference accounting

2015-07-24 Thread Omar Sandoval
$seqres.full + +_scratch_mkfs $seqres.full 21 +_init_flakey +_mount_flakey + +# Create prealloc extent covering range [160K, 620K[ +$XFS_IO_PROG -f -c falloc 160K 460K $SCRATCH_MNT/foo You're using falloc, so I think you need _require_xfs_io_command falloc. Otherwise, Reviewed-by: Omar Sandoval osan

Re: [PATCH 2/3] btrfs: add replace missing and replace RAID 5/6 to profile configs

2015-07-24 Thread Omar Sandoval
On Fri, Jul 24, 2015 at 02:09:46PM +0200, David Sterba wrote: On Thu, Jul 23, 2015 at 01:51:50PM -0700, Omar Sandoval wrote: + # We can't do replace with these profiles because they + # imply only one device ($SCRATCH_DEV), and we need

Re: [PATCH 3/3] btrfs: add a test of replace missing dev in diff raid

2015-07-24 Thread Omar Sandoval
On Fri, Jul 24, 2015 at 11:28:32AM +0800, Eryu Guan wrote: On Thu, Jul 23, 2015 at 01:51:51PM -0700, Omar Sandoval wrote: From: Wang Yanfeng wangyf-f...@cn.fujitsu.com Test of missing device replace in different raid modes. This test requires SCRATCH_DEV_POOL contain 5 same size devices

[PATCH 0/3] xfstests: test btrfs replace on RAID 5/6

2015-07-23 Thread Omar Sandoval
added by patch 2. Thanks! Omar Sandoval (2): btrfs/011: test replace on RAID 5/6 now that it's supported btrfs: add replace missing and replace RAID 5/6 to profile configs Wang Yanfeng (1): btrfs: add a test of replace missing dev in diff raid common/rc | 96

[PATCH 1/3] btrfs/011: test replace on RAID 5/6 now that it's supported

2015-07-23 Thread Omar Sandoval
btrfs replace has been supported on RAID 5/6 since Linux 3.19. Signed-off-by: Omar Sandoval osan...@fb.com --- tests/btrfs/011 | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/tests/btrfs/011 b/tests/btrfs/011 index f4f2fbed68d8..c7d35fa46062 100755 --- a/tests/btrfs/011

[PATCH 2/3] btrfs: add replace missing and replace RAID 5/6 to profile configs

2015-07-23 Thread Omar Sandoval
Replacing and scrubbing RAID 5/6 is now supported on Btrfs. Enable it in _btrfs_get_profile_configs while making it more generic to also support replace missing. Signed-off-by: Omar Sandoval osan...@fb.com --- common/rc | 96 --- 1 file

[PATCH 3/3] btrfs: add a test of replace missing dev in diff raid

2015-07-23 Thread Omar Sandoval
wangyf-f...@cn.fujitsu.com Signed-off-by: Omar Sandoval osan...@fb.com --- tests/btrfs/095 | 93 + tests/btrfs/095.out | 2 ++ tests/btrfs/group | 1 + 3 files changed, 96 insertions(+) create mode 100755 tests/btrfs/095 create mode 100644

Re: [PATCH] Btrfs: fix quick exhaustion of the system array in the superblock

2015-07-21 Thread Omar Sandoval
caches for all dirty block groups (btrfs_start_dirty_block_groups()). Reported-by: Omar Sandoval osan...@fb.com Signed-off-by: Filipe Manana fdman...@suse.com Thanks a lot for taking a look. Tested-by: Omar Sandoval osan...@fb.com --- fs/btrfs/extent-tree.c | 18 ++ 1

btrfs_create_pending_block_groups:9460: errno=-27 unknown

2015-07-17 Thread Omar Sandoval
Hey, Filipe, I've been seeing errors of this sort: [ 658.221300] [ cut here ] [ 658.221948] WARNING: CPU: 0 PID: 1636 at fs/btrfs/extent-tree.c:9460 btrfs_create_pending_block_groups+0x16b/0x210() [ 658.223274] CPU: 0 PID: 1636 Comm: btrfs-transacti Not tainted

Re: size 2.73TiB used 240.97GiB after balance

2015-07-06 Thread Omar Sandoval
On 07/06/2015 01:01 PM, Donald Pearson wrote: Based on my experience Hugo's advice is critical, get the bad drive out of the pool when in raid56 and do not try to replace or delete it while it's still attached and recognized. If you add a new device, mount degraded and rebalance. If you

[PATCH RESEND] Btrfs: add autodefrag inode flag

2015-06-30 Thread Omar Sandoval
In some cases, we may not want to enable automatic defragmentation for the whole filesystem with the autodefrag mount option but we still want to defragment specific files or directories. Add an inode flag which allows us to do specify that. Signed-off-by: Omar Sandoval osan...@fb.com

Re: [PATCH] Btrfs: add autodefrag inode flag

2015-06-30 Thread Omar Sandoval
On 06/30/2015 09:18 AM, David Sterba wrote: On Thu, Jun 18, 2015 at 07:10:47PM -0700, Omar Sandoval wrote: In some cases, we may not want to enable automatic defragmentation for the whole filesystem with the autodefrag mount option but we still want to defragment specific files or directories

Re: [PATCH v2 0/5] Btrfs: RAID 5/6 missing device scrub+replace

2015-06-25 Thread Omar Sandoval
On Thu, Jun 25, 2015 at 01:03:57PM +0800, wangyf wrote: I confirmed this bug report, and found the reason is that I compiled the patched module with a dirty kernel. This morning I tested this patch again, and didn't see above error, this patch is OK. Sorry for this bug report. : ( It's

[PATCH 0/2] btrfs device remove alias

2015-06-24 Thread Omar Sandoval
if needed Thanks! Omar Sandoval (2): btrfs-progs: replace struct cmd_group-hidden with flags btrfs-progs: alias btrfs device delete to btrfs device remove Documentation/btrfs-device.asciidoc | 5 - cmds-device.c | 35 ++- cmds

[PATCH 1/2] btrfs-progs: replace struct cmd_group-hidden with flags

2015-06-24 Thread Omar Sandoval
We're also going to want to support aliases, so rather than adding another member, replace hidden with a flags member. Signed-off-by: Omar Sandoval osan...@fb.com --- cmds-filesystem.c | 2 +- commands.h| 8 ++-- help.c| 2 +- 3 files changed, 8 insertions(+), 4 deletions

[PATCH 2/2] btrfs-progs: alias btrfs device delete to btrfs device remove

2015-06-24 Thread Omar Sandoval
There's an awkward asymmetry between btrfs device add and btrfs device delete. Resolve this by aliasing delete to remove. Signed-off-by: Omar Sandoval osan...@fb.com --- Documentation/btrfs-device.asciidoc | 5 - cmds-device.c | 35

Re: [PATCH] btrfs-progs: alias btrfs device delete to btrfs device remove

2015-06-23 Thread Omar Sandoval
On Tue, Jun 23, 2015 at 05:40:39PM +0200, David Sterba wrote: On Thu, Jun 18, 2015 at 03:07:09PM -0700, Omar Sandoval wrote: @@ -586,12 +606,13 @@ out: const struct cmd_group device_cmd_group = { device_cmd_group_usage, NULL, { { add, cmd_add_dev, cmd_add_dev_usage, NULL

Re: [PATCH v2 0/5] Btrfs: RAID 5/6 missing device scrub+replace

2015-06-23 Thread Omar Sandoval
On Tue, Jun 23, 2015 at 11:07:00AM +0800, wangyf wrote: Hi, I have tested your PATCH v2 , but something wrong happened. kernel: 4.1.0-rc7+ with your five patches vitrualBox ubuntu14.10-server + LVM I make a new btrfs.ko with your patches, rmmod original module and insmod the new. When

[PATCH v2 1/5] Btrfs: remove misleading handling of missing device scrub

2015-06-19 Thread Omar Sandoval
scrub_submit() claims that it can handle a bio with a NULL block device, but this is misleading, as calling bio_add_page() on a bio with a NULL -bi_bdev would've already crashed. Delete this, as we're about to properly handle a missing block device. Signed-off-by: Omar Sandoval osan...@fb.com

[PATCH v2 4/5] Btrfs: fix device replace of a missing RAID 5/6 device

2015-06-19 Thread Omar Sandoval
-off-by: Omar Sandoval osan...@fb.com --- fs/btrfs/scrub.c | 157 +++ 1 file changed, 147 insertions(+), 10 deletions(-) diff --git a/fs/btrfs/scrub.c b/fs/btrfs/scrub.c index b94694d59de5..b75f1e9c6adc 100644 --- a/fs/btrfs/scrub.c +++ b/fs

[PATCH v2 0/5] Btrfs: RAID 5/6 missing device scrub+replace

2015-06-19 Thread Omar Sandoval
missing scrub_wr_submit() in scrub_missing_raid56_worker() - Add clarifying comment in dev-missing case of scrub_stripe() (Zhaolei) - Add fix for scrub with missing device (patch 5) Omar Sandoval (5): Btrfs: remove misleading handling of missing device scrub Btrfs: count devices correctly

[PATCH v2 5/5] Btrfs: fix parity scrub of RAID 5/6 with missing device

2015-06-19 Thread Omar Sandoval
already mapped. If scrub_remap_extent() finds a missing block device, it doesn't initialize extent_dev, so we're left with a NULL struct btrfs_device. The solution is to use btrfs_map_block() directly. Reported-by: Zhao Lei zhao...@cn.fujitsu.com Signed-off-by: Omar Sandoval osan...@fb.com --- fs

[PATCH v2 3/5] Btrfs: add RAID 5/6 BTRFS_RBIO_REBUILD_MISSING operation

2015-06-19 Thread Omar Sandoval
with a NULL blkdev. We could do manual manipulation of bio-bi_io_vec, but that's pretty gross. So instead, add a separate path that allows us to manually add pages to the rbio. Signed-off-by: Omar Sandoval osan...@fb.com --- fs/btrfs/raid56.c | 87

[PATCH v2 2/5] Btrfs: count devices correctly in readahead during RAID 5/6 replace

2015-06-19 Thread Omar Sandoval
-by: Omar Sandoval osan...@fb.com --- fs/btrfs/reada.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/fs/btrfs/reada.c b/fs/btrfs/reada.c index 0e7beea92b4c..4645cd16d5ba 100644 --- a/fs/btrfs/reada.c +++ b/fs/btrfs/reada.c @@ -328,6 +328,7 @@ static struct reada_extent

[PATCH] Btrfs: add autodefrag inode flag

2015-06-18 Thread Omar Sandoval
In some cases, we may not want to enable automatic defragmentation for the whole filesystem with the autodefrag mount option but we still want to defragment specific files or directories. Add an inode flag which allows us to do specify that. Signed-off-by: Omar Sandoval osan...@fb.com --- fs

[PATCH] btrfs-progs: alias btrfs device delete to btrfs device remove

2015-06-18 Thread Omar Sandoval
There's an awkward asymmetry between btrfs device add and btrfs device delete. Resolve this by aliasing delete to remove. Signed-off-by: Omar Sandoval osan...@fb.com --- Documentation/btrfs-device.asciidoc | 5 - cmds-device.c | 33 +++-- 2

Re: [PATCH 4/4] Btrfs: fix device replace of a missing RAID 5/6 device

2015-06-12 Thread Omar Sandoval
Hi, Zhaolei, On Thu, Jun 11, 2015 at 06:29:15PM +0800, Zhao Lei wrote: Hi, Omar Sandoval -Original Message- From: linux-btrfs-ow...@vger.kernel.org [mailto:linux-btrfs-ow...@vger.kernel.org] On Behalf Of Omar Sandoval Sent: Monday, May 11, 2015 3:58 PM To: linux-btrfs

Re: [PATCH 0/4] Btrfs: RAID 5/6 missing device replace

2015-06-11 Thread Omar Sandoval
On Thu, Jun 11, 2015 at 11:52:30AM +0800, Zhao Lei wrote: Hi, Omar Sandoval I tested this patchset with my script, but see general protection fault again. NODE: kvm with virtio disk ROOTFS: RHEL6 with btrfs-progs v4.0 KERNEL: v4.1-rc6 with 4 patchs in this patchset Maybe my test have

[PATCH v3] btrfs: test premature submount unmounting when deleting default subvolume

2015-06-05 Thread Omar Sandoval
Add a regression test for a problem where attempting to delete the default subvolume would fail (as expected), but not until after all submounts under the subvolume were unmounted. Reviewed-by: Eryu Guan eg...@redhat.com Signed-off-by: Omar Sandoval osan...@osandov.com --- v2-v3: - Update

Re: [PATCH] Btrfs: don't invalidate root dentry when subvolume deletion fails

2015-06-02 Thread Omar Sandoval
On Mon, Jun 01, 2015 at 05:56:43PM +0100, Filipe David Manana wrote: On Sat, May 30, 2015 at 9:59 AM, Omar Sandoval osan...@osandov.com wrote: Since commit bafc9b754f75 (vfs: More precise tests in d_invalidate), mounted subvolumes can be deleted because d_invalidate() won't fail. However

[PATCH v2] Btrfs: don't invalidate root dentry when subvolume deletion fails

2015-06-02 Thread Omar Sandoval
, the shrink_dcache_sb() isn't necessary; d_invalidate() will prune the dcache for the deleted subvolume. Cc: sta...@vger.kernel.org Fixes: bafc9b754f75 (vfs: More precise tests in d_invalidate) Reported-by: Markus Schauler mschau...@gmail.com Signed-off-by: Omar Sandoval osan...@osandov.com --- v1

[PATCH] btrfs: default subvolume deletion test

2015-06-02 Thread Omar Sandoval
Add a regression test for a problem where attempting to delete the default subvolume would fail (as expected), but not until after all submounts under the subvolume were unmounted. Signed-off-by: Omar Sandoval osan...@osandov.com --- tests/btrfs/089 | 82

Re: [PATCH] btrfs: default subvolume deletion test

2015-06-02 Thread Omar Sandoval
On Wed, Jun 03, 2015 at 11:53:37AM +0800, Eryu Guan wrote: On Tue, Jun 02, 2015 at 07:35:03PM -0700, Omar Sandoval wrote: Add a regression test for a problem where attempting to delete the default subvolume would fail (as expected), but not until after all submounts under the subvolume were

Re: [PATCH] btrfs: default subvolume deletion test

2015-06-02 Thread Omar Sandoval
On Tue, Jun 02, 2015 at 09:40:13PM -0700, Omar Sandoval wrote: On Wed, Jun 03, 2015 at 11:53:37AM +0800, Eryu Guan wrote: On Tue, Jun 02, 2015 at 07:35:03PM -0700, Omar Sandoval wrote: Add a regression test for a problem where attempting to delete the default subvolume would fail

[PATCH v2] btrfs: test premature submount unmounting when deleting default subvolume

2015-06-02 Thread Omar Sandoval
Add a regression test for a problem where attempting to delete the default subvolume would fail (as expected), but not until after all submounts under the subvolume were unmounted. Signed-off-by: Omar Sandoval osan...@osandov.com --- v1-v2: - Simpler test: just depends on umount of the bind mount

Re: BTRFS issue: deleting default snapshot causes kernel oops

2015-05-30 Thread Omar Sandoval
On Fri, May 29, 2015 at 11:14:13PM +0200, Markus Schauler wrote: Hi, I'm having an issue with Btrfs on an opensuse Tumbleweed system using kernel 4.0.4-1 Here's what happened: I use the snapper tool to create periodic snapshots of the root FS After having problems with an update, I

[PATCH] Btrfs: don't invalidate root dentry when subvolume deletion fails

2015-05-30 Thread Omar Sandoval
with the deletion. Cc: sta...@vger.kernel.org Fixes: bafc9b754f75 (vfs: More precise tests in d_invalidate) Reported-by: Markus Schauler mschau...@gmail.com Signed-off-by: Omar Sandoval osan...@osandov.com --- The other fix for preventing all mounted subvolumes from being deleted would preclude

Re: [PATCH 3/4] btrfs: use scrub_pause_on/off() to reduce code in scrub_enumerate_chunks()

2015-05-29 Thread Omar Sandoval
On Fri, May 29, 2015 at 07:55:34PM +0800, Zhaolei wrote: From: Zhao Lei zhao...@cn.fujitsu.com Use new intruduced scrub_pause_on/off() can make this code block clean and more readable. Signed-off-by: Zhao Lei zhao...@cn.fujitsu.com Reviewed-by: Omar Sandoval osan...@osandov.com -- Omar

Re: [PATCH 2/4] btrfs: Separate scrub_blocked_if_needed() to scrub_pause_on/off()

2015-05-29 Thread Omar Sandoval
-by: Zhao Lei zhao...@cn.fujitsu.com Reviewed-by: Omar Sandoval osan...@osandov.com -- Omar -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Help needed to recover a RAID5 btrfs

2015-05-26 Thread Omar Sandoval
On Mon, May 25, 2015 at 06:06:20PM +0200, Felix Koop wrote: Hello, I have a RAID5 filesystem where one disk has crashed. Now the filesystem is not recognized any more. Any help available? Here is some info: root@server:~# uname -a Linux server 4.0.0-1-amd64 #1 SMP Debian 4.0.2-1

Re: [PATCH 0/4] Btrfs: RAID 5/6 missing device replace

2015-05-26 Thread Omar Sandoval
On Mon, May 11, 2015 at 12:58:11AM -0700, Omar Sandoval wrote: A user reported on Bugzilla that they were seeing kernel BUGs when attempting to replace a missing device on a RAID 6 array. After identifying the apparent cause of the BUG, I reached the conclusion that there wasn't a quick fix

Re: [PATCH v2 0/6] Btrfs: show subvolume name and ID in /proc/mounts

2015-05-11 Thread Omar Sandoval
On Thu, Apr 09, 2015 at 02:34:50PM -0700, Omar Sandoval wrote: Here's version 2 of providing the subvolume name and ID in /proc/mounts. It turns out that getting the name of a subvolume reliably is a bit trickier than it would seem because of how mounting subvolumes by ID is implemented

Re: [PATCH v2 0/6] Btrfs: show subvolume name and ID in /proc/mounts

2015-05-11 Thread Omar Sandoval
On Mon, May 11, 2015 at 04:51:37PM +0200, David Sterba wrote: On Mon, May 11, 2015 at 02:42:58AM -0700, Omar Sandoval wrote: Sorry for the long-winded email! Thoughts, David, Qu? I'm ok with the approach and the patchset as a whole. There was one minor issue with the aliasing

Re: [PATCH] Revert btrfs: delete chunk allocation attemp when setting block group ro

2015-04-29 Thread Omar Sandoval
On Sun, Apr 19, 2015 at 01:17:05AM -0700, Omar Sandoval wrote: This reverts commit 2f0810880f082fa8ba66ab2c33b02e4ff9770a5e. This tried to fix the following test case: mkfs.ext4 -F /dev/sda btrfs-convert /dev/sda mount /dev/sda /mnt btrfs device add -f /dev/sdb

Re: btrfs balance start -dconvert=raid5 on raid1 not converting to raid5

2015-04-25 Thread Omar Sandoval
On Sat, Apr 25, 2015 at 08:18:30AM +, Duncan wrote: None None posted on Sat, 25 Apr 2015 05:56:50 +0200 as excerpted: Omar Sandoval osan...@osandov.com írta: On Sat, Apr 25, 2015 at 04:47:31AM +0200, None None wrote: I tried to convert my btrfs from raid1 to raid5 but after the balance

Re: btrfs balance start -dconvert=raid5 on raid1 not converting to raid5

2015-04-24 Thread Omar Sandoval
On Sat, Apr 25, 2015 at 04:47:31AM +0200, None None wrote: I tried to convert my btrfs from raid1 to raid5 but after the balance command it's still raid1. Also for raid56 the wiki says Parity may be inconsistent after a crash (the write hole) does that mean if I convert metadata to raid5/6

[PATCH] Revert btrfs: delete chunk allocation attemp when setting block group ro

2015-04-19 Thread Omar Sandoval
-v /mnt btrfs device add -f /dev/sdb /mnt btrfs balance start -v -dconvert=raid1 -mconvert=raid1 /mnt Reported-by: Holger Hoffstätte holger.hoffstae...@googlemail.com Cc: Shaohua Li s...@fb.com Signed-off-by: Omar Sandoval osan...@osandov.com --- Holger reported this awhile ago

Re: [PATCH RESEND] btrfs: unlock i_mutex after attempting to delete subvolume during send

2015-04-13 Thread Omar Sandoval
On Fri, Apr 10, 2015 at 02:20:40PM -0700, Omar Sandoval wrote: Whenever the check for a send in progress introduced in commit 521e0546c970 (btrfs: protect snapshots from deleting during send) is hit, we return without unlocking inode-i_mutex. This is easy to see with lockdep enabled

Re: [PATCH v2 0/3] btrfs: ENOMEM bugfixes

2015-04-13 Thread Omar Sandoval
On Fri, Mar 27, 2015 at 02:06:49PM -0700, Omar Sandoval wrote: On Fri, Mar 13, 2015 at 12:43:42PM -0700, Omar Sandoval wrote: On Fri, Mar 13, 2015 at 12:04:30PM +0100, David Sterba wrote: On Wed, Mar 11, 2015 at 09:40:17PM -0700, Omar Sandoval wrote: Ping. For anyone following along

[PATCH RESEND] btrfs: unlock i_mutex after attempting to delete subvolume during send

2015-04-10 Thread Omar Sandoval
: (type-i_mutex_dir_key){+.+.+.}, at: [8135b8df] btrfs_ioctl_snap_destroy+0x2df/0x7a0 Make sure we unlock it in the error path. Reviewed-by: Filipe Manana fdman...@suse.com Reviewed-by: David Sterba dste...@suse.cz Cc: sta...@vger.kernel.org Signed-off-by: Omar Sandoval osan...@osandov.com

[PATCH v2 6/6] Btrfs: show subvol= and subvolid= in /proc/mounts

2015-04-09 Thread Omar Sandoval
Now that we're guaranteed to have a meaningful root dentry, we can just export seq_dentry() and use it in btrfs_show_options(). The subvolume ID is easy to get and can also be useful, so put that in there, too. Signed-off-by: Omar Sandoval osan...@osandov.com --- fs/btrfs/super.c | 4 fs

[PATCH v2 4/6] Btrfs: fail on mismatched subvol and subvolid mount options

2015-04-09 Thread Omar Sandoval
/mounts). Signed-off-by: Omar Sandoval osan...@osandov.com --- fs/btrfs/super.c | 32 1 file changed, 24 insertions(+), 8 deletions(-) diff --git a/fs/btrfs/super.c b/fs/btrfs/super.c index ab100e5..20b470d 100644 --- a/fs/btrfs/super.c +++ b/fs/btrfs/super.c

<    2   3   4   5   6   7   8   9   >