Re: [PATCH] btrfs: Don't call btrfs_start_transaction() on frozen fs to avoid deadlock.

2015-01-20 Thread Qu Wenruo
Original Message Subject: Re: [PATCH] btrfs: Don't call btrfs_start_transaction() on frozen fs to avoid deadlock. From: Miao Xie To: Qu Wenruo , Chris Mason Date: 2015年01月21日 15:04 On Wed, 21 Jan 2015 11:53:34 +0800, Qu Wenruo wrote: [snipped] This will cause another probl

Re: [PATCH] btrfs: Don't call btrfs_start_transaction() on frozen fs to avoid deadlock.

2015-01-20 Thread Miao Xie
On Wed, 21 Jan 2015 11:53:34 +0800, Qu Wenruo wrote: >> +/* >> + * Test if the fs is frozen, or start_trasaction >> + * will deadlock on itself. >> + */ >> +if (__sb_start_write(sb, SB_FREEZE_FS, fal

[PATCH] fstests: btrfs/082: Check return value of "btrfs filesystem show" command executed on umounted device.

2015-01-20 Thread Qu Wenruo
The return value should always be 0 if no problem happens, but "btrfs filesystem show" executed on umounted device will always return 1 even there is no problem. This testcase just checks it. Signed-off-by: Qu Wenruo --- tests/btrfs/082 | 71 +

[PATCH v2] fstests: btrfs/079: Fix wrong value passed to available space check.

2015-01-20 Thread Qu Wenruo
Wrong value is passed to _require_fs_space, which should be in unit of kilobyte(1024), but passed in unit of gigabyte(1024^3). Fix it. Signed-off-by: Qu Wenruo --- tests/btrfs/079 | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/btrfs/079 b/tests/btrfs/079 index 202d3e6

Re: [PATCH v2] xfstests: btrfs: fix up 001.out

2015-01-20 Thread Anand Jain
Dave, On 01/21/2015 12:26 PM, Dave Chinner wrote: On Fri, Jan 02, 2015 at 09:04:29PM +0800, Anand Jain wrote: The subvol delete output has changed with btrfs-progs -Delete subvolume 'SCRATCH_MNT/snap' +Delete subvolume (no-commit): 'SCRATCH_MNT/snap' so fix 001 failing. Signed-off-

Re: [PATCH] fstest: btrfs/006: Add extra check on return value and 'fi show' by device

2015-01-20 Thread Dave Chinner
On Wed, Jan 21, 2015 at 12:19:32PM +0800, Qu Wenruo wrote: > From: Dave Chinner > >On Fri, Jan 16, 2015 at 09:57:10AM +0800, Qu Wenruo wrote: > >>Reported in Red Hat BZ#1181627, 'btrfs fi show' on unmounted device will > >>return 1 even no error happens. > >Please describe the bug in the commit me

Re: [PATCH v2] xfstests: btrfs: fix up 001.out

2015-01-20 Thread Dave Chinner
On Fri, Jan 02, 2015 at 09:04:29PM +0800, Anand Jain wrote: > The subvol delete output has changed with btrfs-progs > -Delete subvolume 'SCRATCH_MNT/snap' > +Delete subvolume (no-commit): 'SCRATCH_MNT/snap' > > so fix 001 failing. > > Signed-off-by: Anand Jain > > v2: Thanks Filipe for

Re: [PATCH] fstests: btrfs/079: Fix wrong value passed to available space check.

2015-01-20 Thread Qu Wenruo
Original Message Subject: Re: [PATCH] fstests: btrfs/079: Fix wrong value passed to available space check. From: Dave Chinner To: Qu Wenruo Date: 2015年01月21日 12:19 On Mon, Dec 29, 2014 at 03:17:18PM +0800, Qu Wenruo wrote: Before the patch, we passed wrong value to _requir

Re: [PATCH] fstest: btrfs/006: Add extra check on return value and 'fi show' by device

2015-01-20 Thread Qu Wenruo
Original Message Subject: Re: [PATCH] fstest: btrfs/006: Add extra check on return value and 'fi show' by device From: Dave Chinner To: Qu Wenruo Date: 2015年01月21日 12:03 On Fri, Jan 16, 2015 at 09:57:10AM +0800, Qu Wenruo wrote: Reported in Red Hat BZ#1181627, 'btrfs fi sh

Re: [PATCH] fstests: btrfs/079: Fix wrong value passed to available space check.

2015-01-20 Thread Dave Chinner
On Mon, Dec 29, 2014 at 03:17:18PM +0800, Qu Wenruo wrote: > Before the patch, we passed wrong value to _require_fs_space, which > should be in unit of 1024, but passed in unit of GB. Yes, that needs fixing. > Fix it and add better prompt for falloc failure. That doesn't > > Signed-off-by:

Re: btrfs convert running out of space

2015-01-20 Thread Chris Murphy
On Tue, Jan 20, 2015 at 4:04 PM, Gareth Pye wrote: > Yeah, we don't have that much space spare :( > > File system has been going strong from when it was created with early > RAID5 code, then converted to RAID10 with kernel 3.12. > > There aren't any nocow files to my knowledge but there are plenty

Re: [PATCH] fstest: btrfs/006: Add extra check on return value and 'fi show' by device

2015-01-20 Thread Dave Chinner
On Fri, Jan 16, 2015 at 09:57:10AM +0800, Qu Wenruo wrote: > Reported in Red Hat BZ#1181627, 'btrfs fi show' on unmounted device will > return 1 even no error happens. Please describe the bug in the commit message, don't make me have to go look at some web page to work out what you are trying to f

Re: [PATCH] btrfs: Don't call btrfs_start_transaction() on frozen fs to avoid deadlock.

2015-01-20 Thread Qu Wenruo
Original Message Subject: Re: [PATCH] btrfs: Don't call btrfs_start_transaction() on frozen fs to avoid deadlock. From: Miao Xie To: Qu Wenruo , Chris Mason Date: 2015年01月21日 11:26 On Wed, 21 Jan 2015 11:15:41 +0800, Qu Wenruo wrote: Original Message Subj

Re: [PATCH] btrfs: Don't call btrfs_start_transaction() on frozen fs to avoid deadlock.

2015-01-20 Thread Miao Xie
On Wed, 21 Jan 2015 11:15:41 +0800, Qu Wenruo wrote: > > Original Message > Subject: Re: [PATCH] btrfs: Don't call btrfs_start_transaction() on frozen fs > to > avoid deadlock. > From: Miao Xie > To: Chris Mason , Qu Wenruo > Date: 2015年01月21日 11:10 >> On Tue, 20 Jan 2015 20:1

Re: [PATCH] btrfs: Don't call btrfs_start_transaction() on frozen fs to avoid deadlock.

2015-01-20 Thread Qu Wenruo
Original Message Subject: Re: [PATCH] btrfs: Don't call btrfs_start_transaction() on frozen fs to avoid deadlock. From: Miao Xie To: Chris Mason , Qu Wenruo Date: 2015年01月21日 11:10 On Tue, 20 Jan 2015 20:10:56 -0500, Chris Mason wrote: On Tue, Jan 20, 2015 at 8:09 PM, Qu W

Re: [PATCH] btrfs: Don't call btrfs_start_transaction() on frozen fs to avoid deadlock.

2015-01-20 Thread Miao Xie
On Tue, 20 Jan 2015 20:10:56 -0500, Chris Mason wrote: > On Tue, Jan 20, 2015 at 8:09 PM, Qu Wenruo wrote: >> >> Original Message >> Subject: Re: [PATCH] btrfs: Don't call btrfs_start_transaction() on frozen fs >> to avoid deadlock. >> From: Chris Mason >> To: Qu Wenruo >> Date

Re: [PATCH] btrfs: Don't call btrfs_start_transaction() on frozen fs to avoid deadlock.

2015-01-20 Thread Chris Mason
On Tue, Jan 20, 2015 at 8:09 PM, Qu Wenruo wrote: Original Message Subject: Re: [PATCH] btrfs: Don't call btrfs_start_transaction() on frozen fs to avoid deadlock. From: Chris Mason To: Qu Wenruo Date: 2015年01月21日 09:05 On Tue, Jan 20, 2015 at 7:58 PM, Qu Wenruo wrote

Re: [PATCH] btrfs: Don't call btrfs_start_transaction() on frozen fs to avoid deadlock.

2015-01-20 Thread Qu Wenruo
Original Message Subject: Re: [PATCH] btrfs: Don't call btrfs_start_transaction() on frozen fs to avoid deadlock. From: Chris Mason To: Qu Wenruo Date: 2015年01月21日 09:05 On Tue, Jan 20, 2015 at 7:58 PM, Qu Wenruo wrote: Original Message Subject: Re:

Re: [PATCH] btrfs: Don't call btrfs_start_transaction() on frozen fs to avoid deadlock.

2015-01-20 Thread Chris Mason
On Tue, Jan 20, 2015 at 7:58 PM, Qu Wenruo wrote: Original Message Subject: Re: [PATCH] btrfs: Don't call btrfs_start_transaction() on frozen fs to avoid deadlock. From: David Sterba To: Qu Wenruo Date: 2015年01月21日 01:13 On Mon, Jan 19, 2015 at 03:42:41PM +0800, Qu Wen

Re: [PATCH] btrfs: Don't call btrfs_start_transaction() on frozen fs to avoid deadlock.

2015-01-20 Thread Qu Wenruo
Original Message Subject: Re: [PATCH] btrfs: Don't call btrfs_start_transaction() on frozen fs to avoid deadlock. From: David Sterba To: Qu Wenruo Date: 2015年01月21日 01:13 On Mon, Jan 19, 2015 at 03:42:41PM +0800, Qu Wenruo wrote: --- a/fs/btrfs/super.c +++ b/fs/btrfs/super

Re: btrfs convert running out of space

2015-01-20 Thread Hugo Mills
On Tue, Jan 20, 2015 at 02:41:13PM -0700, Chris Murphy wrote: > On Tue, Jan 20, 2015 at 2:25 PM, Gareth Pye wrote: > > Yeah, I have updated btrfs-progs to 3.18. While it is plausible that > > the bug was created by using 3.12, none of the behavior has changed > > now I'm using 3.18. > > > > I was

Re: btrfs convert running out of space

2015-01-20 Thread Gareth Pye
Yeah, we don't have that much space spare :( File system has been going strong from when it was created with early RAID5 code, then converted to RAID10 with kernel 3.12. There aren't any nocow files to my knowledge but there are plenty of files larger than a gig on the file system. The first few

Re: btrfs convert running out of space

2015-01-20 Thread Chris Murphy
On Tue, Jan 20, 2015 at 2:49 PM, Gareth Pye wrote: > The conversion is going the other way (raid10->raid1), but I expect > the analysis is going to be the same. I'll wait on 3.19 kernel then > (or probably 3.19.1) as this system is slightly more production than > use of btrfs would suggest. > > Th

RE: Recovery options for FS forced readonly due to 3.17 snapshot bug

2015-01-20 Thread Brett King
From: Filipe David Manana Sent: Tue 01-20-2015 11:40 pm Subject:Re: Recovery options for FS forced readonly due to 3.17 snapshot bug To: brett.k...@commandict.com.au; CC: linux-btrfs@vger.kernel.org; > On Tue, Jan 20, 2015 at 12:15 PM, wrote: > > Hi, > > My FS has been for

Re: btrfs convert running out of space

2015-01-20 Thread Gareth Pye
The conversion is going the other way (raid10->raid1), but I expect the analysis is going to be the same. I'll wait on 3.19 kernel then (or probably 3.19.1) as this system is slightly more production than use of btrfs would suggest. The flags 17 messages are from the non-converting balance to clea

Re: btrfs convert running out of space

2015-01-20 Thread Chris Murphy
On Tue, Jan 20, 2015 at 2:25 PM, Gareth Pye wrote: > Yeah, I have updated btrfs-progs to 3.18. While it is plausible that > the bug was created by using 3.12, none of the behavior has changed > now I'm using 3.18. > > I was experimenting with -dusage values to try and process the blocks > in a dif

Re: btrfs convert running out of space

2015-01-20 Thread Gareth Pye
Yeah, I have updated btrfs-progs to 3.18. While it is plausible that the bug was created by using 3.12, none of the behavior has changed now I'm using 3.18. I was experimenting with -dusage values to try and process the blocks in a different order to see if that made any difference. It did let me

[PATCH 3/3] btrfs: remove a no-op unfreeze superbock callback

2015-01-20 Thread David Sterba
Signed-off-by: David Sterba --- fs/btrfs/super.c | 6 -- 1 file changed, 6 deletions(-) diff --git a/fs/btrfs/super.c b/fs/btrfs/super.c index 6f49b2872a64..05fef198ff94 100644 --- a/fs/btrfs/super.c +++ b/fs/btrfs/super.c @@ -1958,11 +1958,6 @@ static int btrfs_freeze(struct super_block *sb

[PATCH 2/3] btrfs: Don't call btrfs_start_transaction() on frozen fs to avoid deadlock.

2015-01-20 Thread David Sterba
From: Qu Wenruo Commit 6b5fe46dfa52 (btrfs: do commit in sync_fs if there are pending changes) will call btrfs_start_transaction() in sync_fs(), to handle some operations needed to be done in next transaction. However this can cause deadlock if the filesystem is frozen, with the following sys_r+

[PATCH 1/3] btrfs: Fix the bug that fs_info->pending_changes is never cleared.

2015-01-20 Thread David Sterba
From: Qu Wenruo Fs_info->pending_changes is never cleared since the original code uses cmpxchg(&fs_info->pending_changes, 0, 0), which will only clear it if pending_changes is already 0. This will cause a lot of problem when mount it with inode_cache mount option. If the btrfs is mounted as inod

[PULL] [PATCH 0/3] Btrfs, fixes for freezing vs pending changes

2015-01-20 Thread David Sterba
There was some churn regarding the patches to $SUBJ, here's what I think is enough to fix it for 3.19. It's based on top of my other patch "btrfs: sync ioctl, handle errors after transaction start" that might land in Chris' for-linus already so it's not included here. The patches are for review b

Re: Snapshot cannot be deleted

2015-01-20 Thread Theodore Ts'o
+linux-btrfs,-linux-ext4 On Tue, Jan 20, 2015 at 01:28:04PM +0100, Andreas Philipp wrote: > > Due to the known (and fixed) bug in kernel 3.17.0 one of my btrfs > volume suffers from unreadable and - even worse - uneraseable > snapshots. Whenever such a snapshot is accessed there is "parent > tran

Re: [PATCH] btrfs: Don't call btrfs_start_transaction() on frozen fs to avoid deadlock.

2015-01-20 Thread David Sterba
On Mon, Jan 19, 2015 at 03:42:41PM +0800, Qu Wenruo wrote: > --- a/fs/btrfs/super.c > +++ b/fs/btrfs/super.c > @@ -1000,6 +1000,14 @@ int btrfs_sync_fs(struct super_block *sb, int wait) >*/ > if (fs_info->pending_changes == 0) >

Re: [PATCH RFC v2 1/5] btrfs: Fix the bug that fs_info->pending_changes is never cleared.

2015-01-20 Thread David Sterba
On Tue, Jan 20, 2015 at 05:05:33PM +0800, Qu Wenruo wrote: > Fs_info->pending_changes is never cleared since the original code uses > cmpxchg(&fs_info->pending_changes, 0, 0), which will only clear it if > pending_changes is already 0. > > This will cause a lot of problem when mount it with inode_

Re: [PATCH RFC v2 0/5] Fix bugs introduced by the new pending changes

2015-01-20 Thread David Sterba
On Tue, Jan 20, 2015 at 05:05:32PM +0800, Qu Wenruo wrote: > Commit 572d9ab7845ea0e0 ("btrfs: add support for processing pending > changes") introduced several bugs which will eventually cause a > deadlock. > > The deadlock can be triggered by fstests/generic/068 with inode_cache > mount option. >

[PATCH] Btrfs: fix setup_leaf_for_split() to avoid leaf corruption

2015-01-20 Thread Filipe Manana
We were incorrectly detecting when the target key didn't exist anymore after releasing the path and re-searching the tree. This could make us split or duplicate (btrfs_split_item() and btrfs_duplicate_item() are its only callers at the moment) an item when we should not. For the case of duplicatin

Re: Recovery options for FS forced readonly due to 3.17 snapshot bug

2015-01-20 Thread Filipe David Manana
On Tue, Jan 20, 2015 at 12:15 PM, wrote: > Hi, > My FS has been forced readonly by the early 3.17 snapshot bug. After much > reading, I'm looking for validation of some recovery scenarios: > > 1) btrfsck --repair under a later kernel. > > 2) replacing the devices one by one under a later kernel,

Re: Recovery options for FS forced readonly due to 3.17 snapshot bug

2015-01-20 Thread Hugo Mills
On Tue, Jan 20, 2015 at 11:15:22PM +1100, brett.k...@commandict.com.au wrote: > Hi, > My FS has been forced readonly by the early 3.17 snapshot bug. After much > reading, I'm looking for validation of some recovery scenarios: > > 1) btrfsck --repair under a later kernel. Kernel won't make a d

Recovery options for FS forced readonly due to 3.17 snapshot bug

2015-01-20 Thread brett . king
Hi, My FS has been forced readonly by the early 3.17 snapshot bug. After much reading, I'm looking for validation of some recovery scenarios: 1) btrfsck --repair under a later kernel. 2) replacing the devices one by one under a later kernel, effectively removing the corruption. 3) Just copying

[PATCH RFC v2 5/5] Revert "btrfs: move commit out of sysfs when changing features"

2015-01-20 Thread Qu Wenruo
This reverts commit 0eae2747ec1dd ("btrfs: move commit out of sysfs when changing features"). Unlike most btrfs file operations, sysfs don't go through the normal open/read/write routine which will initiate a sb_start_*write/read. This is very dangerous on frozen btrfs, if using the pending chang

[PATCH RFC v2 3/5] btrfs: Handle pending changes in btrfs_freeze().

2015-01-20 Thread Qu Wenruo
Before the patch, btrfs_freeze() does not start a transaction if there is pending changes but no transaction is running. This will lead to deadlock if the fs is frozen but there are pending changes, since sync_fs will start a transaction with s_umount mutex, blocked by frozen fs. And later unfreez

[PATCH RFC v2 2/5] btrfs: Add btrfs_start_transaction_freeze() to start transaction in btrfs_freeze()

2015-01-20 Thread Qu Wenruo
Add btrfs_start_transaction_freeze() function. This is needed for btrfs_freeze() to handle the pending changes. If there is no running transaction when btrfs_freeze() is called, btrfs_freeze() needs to start a transaction to handle the pending changes, and it can't initial a sb_start_intwrite() ca

[PATCH RFC v2 4/5] Revert "btrfs: move commit out of sysfs when changing label"

2015-01-20 Thread Qu Wenruo
This reverts commit a6f69dc8018d ("btrfs: move commit out of sysfs when changing label"). Unlike most btrfs file operations, sysfs don't go through the normal open/read/write routine which will initiate a sb_start_*write/read. This is very dangerous on frozen btrfs, if using the pending changes me

[PATCH RFC v2 0/5] Fix bugs introduced by the new pending changes

2015-01-20 Thread Qu Wenruo
Commit 572d9ab7845ea0e0 ("btrfs: add support for processing pending changes") introduced several bugs which will eventually cause a deadlock. The deadlock can be triggered by fstests/generic/068 with inode_cache mount option. The deadlock happens in the following flow: moutn with inode_cache -> f

[PATCH RFC v2 1/5] btrfs: Fix the bug that fs_info->pending_changes is never cleared.

2015-01-20 Thread Qu Wenruo
Fs_info->pending_changes is never cleared since the original code uses cmpxchg(&fs_info->pending_changes, 0, 0), which will only clear it if pending_changes is already 0. This will cause a lot of problem when mount it with inode_cache mount option. If the btrfs is mounted as inode_cache, pending_c

Re: [PATCH] btrfs: Don't call btrfs_start_transaction() on frozen fs to avoid deadlock.

2015-01-20 Thread Miao Xie
On Tue, 20 Jan 2015 11:17:07 +0800, Qu Wenruo wrote: >> --- a/fs/btrfs/super.c >> +++ b/fs/btrfs/super.c >> @@ -1000,6 +1000,14 @@ int btrfs_sync_fs(struct super_block *sb, int >> wait) >> */ >>if (fs_info->pending_changes == 0) >>