Why subvolume and not just volume?

2015-08-05 Thread Martin
Hi, Does anyone know the reason subvolumes are not called just volumes? I mean, the top subvolume is not called a volume, so there is nothing to be sub of. Also, what is the penalty of a subvolume compared to a directory? From a design perspective, couldn't all directories just be subvolumes?

Re: Why subvolume and not just volume?

2015-08-05 Thread Qu Wenruo
Martin wrote on 2015/08/05 09:06 +0200: Hi, Does anyone know the reason subvolumes are not called just volumes? I mean, the top subvolume is not called a volume, so there is nothing to be sub of. Because normally a volume is referred as a complete filesystem. So from this respect, subvolume

Re: [PATCH v2] fstests: btrfs: Add regression test for reserved space leak.

2015-08-05 Thread Filipe David Manana
On Wed, Aug 5, 2015 at 2:08 AM, Qu Wenruo quwen...@cn.fujitsu.com wrote: The regression is introduced in v4.2-rc1, with the big btrfs qgroup change. The problem is, qgroup reserved space is never freed, causing even we increase the limit, we can still hit the EDQUOT much faster than it

[PATCH 1/3] btrfs-progs: fsck: Print correct file hole

2015-08-05 Thread Qu Wenruo
If a file lost all its file extents, fsck will unable to print out the hole. Add an extra judgment to print out the hole range. Signed-off-by: Qu Wenruo quwen...@cn.fujitsu.com --- cmds-check.c | 7 ++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/cmds-check.c b/cmds-check.c

[PATCH 0/3] Fix for infinite loop on non-empty inode but with no file extent

2015-08-05 Thread Qu Wenruo
A bug reported by Robert Munteanu, which btrfsck infinite loops on an inode with discount file extent. This patchset includes a fix in printing file extent hole, fix the infinite loop, and corresponding test case. BTW, thanks Robert Munteanu a lot for its detailed debug report, makes it super

[PATCH 2/3] btrfs-progs: fsck: Fix a infinite loop on discount file extent repair

2015-08-05 Thread Qu Wenruo
For a special case, discount file extent repair function will cause infinite loop. The case is, if the file loses all its file extent, we won't have a hole to fill, causing repair function doing nothing, and since the I_ERR_DISCOUNT doesn't disappear, the fsck will do infinite loop. For such

[PATCH 3/3] btrfs-progs: fsck-tests: Add test case for inode lost all its file extent

2015-08-05 Thread Qu Wenruo
Add test case with no file extents, but still non-zero inode size. To test whether fsck will infinite loop. Signed-off-by: Qu Wenruo quwen...@cn.fujitsu.com --- .../017-missing-all-file-extent/default_case.img.xz | Bin 0 - 1104 bytes 1 file changed, 0 insertions(+), 0 deletions(-) create

Lockup in BTRFS_IOC_CLONE/Kernel 4.2.0-rc5

2015-08-05 Thread Elias Probst
I can reproduce a hard btrfs lockup (process issuing the ioctl() is in D-state, same goes for btrfs-transacti process) on Kernel 4.2.0-rc5. I had the same issue on 4.1, so it's unlikely a regression introduced in 4.2. ## With the following steps, I can reproduce the problem: 1. Create a new

[PATCH] btrfs-progs: Modify confuse error message in scrub

2015-08-05 Thread Zhao Lei
Scrub output following error message in my test: ERROR: scrubbing /var/ltf/tester/scratch_mnt failed for device id 5 (Success) It is caused by a broken kernel and fs, but the we need to avoid outputting both error and success in oneline message as above. This patch modified above message to:

[PATCH v4 2/4] btrfs: Separate scrub_blocked_if_needed() to scrub_pause_on/off()

2015-08-05 Thread Zhao Lei
It can reduce current duplicated code which is similar to scrub_blocked_if_needed() but can not call it because little different. It also used by my next patch which is in same case. Signed-off-by: Zhao Lei zhao...@cn.fujitsu.com --- fs/btrfs/scrub.c | 11 ++- 1 file changed, 10

[PATCH v4 3/4] btrfs: use scrub_pause_on/off() to reduce code in scrub_enumerate_chunks()

2015-08-05 Thread Zhao Lei
Use new intruduced scrub_pause_on/off() can make this code block clean and more readable. Signed-off-by: Zhao Lei zhao...@cn.fujitsu.com --- fs/btrfs/scrub.c | 10 +++--- 1 file changed, 3 insertions(+), 7 deletions(-) diff --git a/fs/btrfs/scrub.c b/fs/btrfs/scrub.c index cbfb8c7..a882a34

[PATCH v4 4/4] btrfs: Fix data checksum error cause by replace with io-load.

2015-08-05 Thread Zhao Lei
xfstests btrfs/070 sometimes failed. In my test machine, its fail rate is about 30%. In another vm(vmware), its fail rate is about 50%. Reason: btrfs/070 do replace and defrag with fsstress simultaneously, after above operation, checksum error is found by scrub. Actually, it have no

[PATCH v4 0/4] btrfs: Fix data checksum error cause by replace with io-load

2015-08-05 Thread Zhao Lei
This patchset is used to fix data checksum error cause by replace with io-load. It cause xfstests btrfs/070(071) failed randomly. See description in [PATCH 4/4] for detail. Changelog v3-v4: 1: Fix regression of xfstests/061 Patch v3 cause xfstests/061 failed in some case, because

[PATCH v4 1/4] btrfs: Use ref_cnt for set_block_group_ro()

2015-08-05 Thread Zhao Lei
More than one code call set_block_group_ro() and restore rw in fail. Old code use bool bit to save blockgroup's ro state, it can not support parallel case(it is confirmd exist in my debug log). This patch use ref count to store ro state, and rename set_block_group_ro/set_block_group_rw to

[RFC 1/8] mm, oom: Give __GFP_NOFAIL allocations access to memory reserves

2015-08-05 Thread mhocko
From: Michal Hocko mho...@suse.com __GFP_NOFAIL is a big hammer used to ensure that the allocation request can never fail. This is a strong requirement and as such it also deserves a special treatment when the system is OOM. The primary problem here is that the allocation request might have come

[RFC 8/8] btrfs: use __GFP_NOFAIL in alloc_btrfs_bio

2015-08-05 Thread mhocko
From: Michal Hocko mho...@suse.com alloc_btrfs_bio is relying on GFP_NOFS to allocate a bio but since mm: page_alloc: do not lock up GFP_NOFS allocations upon OOM this is allowed to fail which can lead to [ 37.928625] kernel BUG at fs/btrfs/extent_io.c:4045 This is clearly undesirable and the

[RFC 7/8] btrfs: Prevent from early transaction abort

2015-08-05 Thread mhocko
From: Michal Hocko mho...@suse.com Btrfs relies on GFP_NOFS allocation when commiting the transaction but since mm: page_alloc: do not lock up GFP_NOFS allocations upon OOM those allocations are allowed to fail which can lead to a pre-mature transaction abort: [ 55.328093] Call Trace: [

[RFC 4/8] jbd, jbd2: Do not fail journal because of frozen_buffer allocation failure

2015-08-05 Thread mhocko
From: Michal Hocko mho...@suse.com Journal transaction might fail prematurely because the frozen_buffer is allocated by GFP_NOFS request: [ 72.440013] do_get_write_access: OOM for frozen_buffer [ 72.440014] EXT4-fs: ext4_reserve_inode_write:4729: aborting transaction: Out of memory in

[RFC 6/8] ext3: Do not abort journal prematurely

2015-08-05 Thread mhocko
From: Michal Hocko mho...@suse.com journal_get_undo_access is relying on GFP_NOFS allocation yet it is essential for the journal transaction: [ 83.256914] journal_get_undo_access: No memory for committed data [ 83.258022] EXT3-fs: ext3_free_blocks_sb: aborting transaction: Out of memory in

[RFC 5/8] ext4: Do not fail journal due to block allocator

2015-08-05 Thread mhocko
From: Michal Hocko mho...@suse.com Since mm: page_alloc: do not lock up GFP_NOFS allocations upon OOM memory allocator doesn't endlessly loop to satisfy low-order allocations and instead fails them to allow callers to handle them gracefully. Some of the callers are not yet prepared for this

[RFC 3/8] mm: page_alloc: do not lock up GFP_NOFS allocations upon OOM

2015-08-05 Thread mhocko
From: Johannes Weiner han...@cmpxchg.org GFP_NOFS allocations are not allowed to invoke the OOM killer since their reclaim abilities are severely diminished. However, without the OOM killer available there is no hope of progress once the reclaimable pages have been exhausted. Don't risk hanging

[RFC 2/8] mm: Allow GFP_IOFS for page_cache_read page cache allocation

2015-08-05 Thread mhocko
From: Michal Hocko mho...@suse.com page_cache_read has been historically using page_cache_alloc_cold to allocate a new page. This means that mapping_gfp_mask is used as the base for the gfp_mask. Many filesystems are setting this mask to GFP_NOFS to prevent from fs recursion issues.

[RFC 0/8] Allow GFP_NOFS allocation to fail

2015-08-05 Thread mhocko
Hi, small GFP_NOFS, like GFP_KERNEL, allocations have not been not failing traditionally even though their reclaim capabilities are restricted because the VM code cannot recurse into filesystems to clean dirty pages. At the same time these allocation requests do not allow to trigger the OOM killer

[PATCH 3/3] btrfs: Cleanup: Remove chunk_objectid argument from btrfs_relocate_chunk()

2015-08-05 Thread Zhao Lei
Remove chunk_objectid argument from btrfs_relocate_chunk() because it is not necessary, it can also cleanup some code in caller for prepare its value. Signed-off-by: Zhao Lei zhao...@cn.fujitsu.com --- fs/btrfs/volumes.c | 10 ++ 1 file changed, 2 insertions(+), 8 deletions(-) diff

[PATCH 2/3] btrfs: Cleanup: Remove objectid's init-value in create_reloc_inode()

2015-08-05 Thread Zhao Lei
objectid's init-value is not used in any case, remove it. Signed-off-by: Zhao Lei zhao...@cn.fujitsu.com --- fs/btrfs/relocation.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/fs/btrfs/relocation.c b/fs/btrfs/relocation.c index 1659c94..4698928 100644 ---

[PATCH 1/3] btrfs: Error handle for get_ref_objectid_v0() in relocate_block_group()

2015-08-05 Thread Zhao Lei
We need error checking code for get_ref_objectid_v0() in relocate_block_group(), to avoid unpredictable result, especially for accessing uninitialized value(when function failed) after this line. Signed-off-by: Zhao Lei zhao...@cn.fujitsu.com --- fs/btrfs/relocation.c | 4 1 file changed, 4

Re: mount btrfs takes 30 minutes, btrfs check runs out of memory

2015-08-05 Thread Austin S Hemmelgarn
On 2015-08-04 13:36, John Ettedgui wrote: On Tue, Aug 4, 2015 at 4:28 AM, Austin S Hemmelgarn ahferro...@gmail.com wrote: On 2015-08-04 00:58, John Ettedgui wrote: On Mon, Aug 3, 2015 at 8:01 PM, Qu Wenruo quwen...@cn.fujitsu.com wrote: Although the best practice is staying away from such

Re: [RFC 4/8] jbd, jbd2: Do not fail journal because of frozen_buffer allocation failure

2015-08-05 Thread Jan Kara
On Wed 05-08-15 11:51:20, mho...@kernel.org wrote: From: Michal Hocko mho...@suse.com Journal transaction might fail prematurely because the frozen_buffer is allocated by GFP_NOFS request: [ 72.440013] do_get_write_access: OOM for frozen_buffer [ 72.440014] EXT4-fs:

Re: [RFC 5/8] ext4: Do not fail journal due to block allocator

2015-08-05 Thread Jan Kara
On Wed 05-08-15 11:51:21, mho...@kernel.org wrote: From: Michal Hocko mho...@suse.com Since mm: page_alloc: do not lock up GFP_NOFS allocations upon OOM memory allocator doesn't endlessly loop to satisfy low-order allocations and instead fails them to allow callers to handle them gracefully.

Re: BTRFS disaster (of my own making). Is this recoverable?

2015-08-05 Thread Sonic
On Tue, Aug 4, 2015 at 4:23 PM, Sonic sonicsm...@gmail.com wrote: Seems that if there was someway to edit something in those first overwritten 32MB of disc 2 to say hey, I'm really here, just a bit screwed up maybe some of the recovery tools could actually work. Just want to reiterate this

Re: BTRFS disaster (of my own making). Is this recoverable?

2015-08-05 Thread Sonic
On Wed, Aug 5, 2015 at 8:31 AM, Sonic sonicsm...@gmail.com wrote: The basic error in most cases with the tools at hand is that Disc 2 is missing so there's little the tools can do. Somewhere in those first 32MB should be something to properly identify the disc as part of the array. Somehow

Re: [RFC 8/8] btrfs: use __GFP_NOFAIL in alloc_btrfs_bio

2015-08-05 Thread David Sterba
On Wed, Aug 05, 2015 at 11:51:24AM +0200, mho...@kernel.org wrote: From: Michal Hocko mho...@suse.com alloc_btrfs_bio is relying on GFP_NOFS to allocate a bio but since mm: page_alloc: do not lock up GFP_NOFS allocations upon OOM this is allowed to fail which can lead to [ 37.928625]

Re: [RFC 7/8] btrfs: Prevent from early transaction abort

2015-08-05 Thread David Sterba
On Wed, Aug 05, 2015 at 11:51:23AM +0200, mho...@kernel.org wrote: From: Michal Hocko mho...@suse.com ... Fix this by reintroducing the no-fail behavior of this allocation path with the explicit __GFP_NOFAIL. Signed-off-by: Michal Hocko mho...@suse.com Reviewed-by: David Sterba

Re: [PATCH 0/3] Fix for infinite loop on non-empty inode but with no file extent

2015-08-05 Thread David Sterba
On Wed, Aug 05, 2015 at 04:03:11PM +0800, Qu Wenruo wrote: A bug reported by Robert Munteanu, which btrfsck infinite loops on an inode with discount file extent. This patchset includes a fix in printing file extent hole, fix the infinite loop, and corresponding test case. BTW, thanks

Re: [RFC 4/8] jbd, jbd2: Do not fail journal because of frozen_buffer allocation failure

2015-08-05 Thread Greg Thelen
mho...@kernel.org wrote: From: Michal Hocko mho...@suse.com Journal transaction might fail prematurely because the frozen_buffer is allocated by GFP_NOFS request: [ 72.440013] do_get_write_access: OOM for frozen_buffer [ 72.440014] EXT4-fs: ext4_reserve_inode_write:4729: aborting

Re: [PATCH] btrfs-progs: Modify confuse error message in scrub

2015-08-05 Thread David Sterba
On Wed, Aug 05, 2015 at 04:32:26PM +0800, Zhao Lei wrote: Scrub output following error message in my test: ERROR: scrubbing /var/ltf/tester/scratch_mnt failed for device id 5 (Success) It is caused by a broken kernel and fs, In what way is it broken? Can we turn it into tests? but the

Re: [PATCH 2/3] btrfs: Cleanup: Remove objectid's init-value in create_reloc_inode()

2015-08-05 Thread David Sterba
On Wed, Aug 05, 2015 at 06:00:03PM +0800, Zhao Lei wrote: objectid's init-value is not used in any case, remove it. Signed-off-by: Zhao Lei zhao...@cn.fujitsu.com Reviewed-by: David Sterba dste...@suse.com -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of

Re: BTRFS disaster (of my own making). Is this recoverable?

2015-08-05 Thread Chris Murphy
On Wed, Aug 5, 2015 at 6:31 AM, Sonic sonicsm...@gmail.com wrote: On Tue, Aug 4, 2015 at 4:23 PM, Sonic sonicsm...@gmail.com wrote: Seems that if there was someway to edit something in those first overwritten 32MB of disc 2 to say hey, I'm really here, just a bit screwed up maybe some of the

Re: [PATCH 3/3] btrfs: Cleanup: Remove chunk_objectid argument from btrfs_relocate_chunk()

2015-08-05 Thread David Sterba
On Wed, Aug 05, 2015 at 06:00:04PM +0800, Zhao Lei wrote: Remove chunk_objectid argument from btrfs_relocate_chunk() because it is not necessary, it can also cleanup some code in caller for prepare its value. Signed-off-by: Zhao Lei zhao...@cn.fujitsu.com Reviewed-by: David Sterba

Re: [PATCH 1/3] btrfs: Error handle for get_ref_objectid_v0() in relocate_block_group()

2015-08-05 Thread David Sterba
On Wed, Aug 05, 2015 at 06:00:02PM +0800, Zhao Lei wrote: We need error checking code for get_ref_objectid_v0() in relocate_block_group(), to avoid unpredictable result, especially for accessing uninitialized value(when function failed) after this line. Signed-off-by: Zhao Lei

Re: Why subvolume and not just volume?

2015-08-05 Thread David Sterba
On Wed, Aug 05, 2015 at 09:06:40AM +0200, Martin wrote: Also, what is the penalty of a subvolume compared to a directory? From a design perspective, couldn't all directories just be subvolumes? They could, but this would bring severe performance drop. * creating a subvolume implies a

Re: [PATCH 0/6] sysfs-part2 Add seed device representation on the sysfs

2015-08-05 Thread David Sterba
On Wed, Jul 08, 2015 at 03:32:48PM +0800, Anand Jain wrote: This patch adds the support to show seed device on the btrfs sysfs. This is a revamped version of the previously single patch 6/6, and plus incorporates David suggestion to add seed fsid under the 'seed' kobject. Since this adds new

Re: RAID1: system stability

2015-08-05 Thread Austin S Hemmelgarn
On 2015-07-22 07:00, Russell Coker wrote: On Tue, 23 Jun 2015 02:52:43 AM Chris Murphy wrote: OK I actually don't know what the intended block layer behavior is when unplugging a device, if it is supposed to vanish, or change state somehow so that thing that depend on it can know it's missing

Re: RAID1: system stability

2015-08-05 Thread Martin Steigerwald
Am Mittwoch, 5. August 2015, 13:32:41 schrieb Austin S Hemmelgarn: On 2015-07-22 07:00, Russell Coker wrote: On Tue, 23 Jun 2015 02:52:43 AM Chris Murphy wrote: OK I actually don't know what the intended block layer behavior is when unplugging a device, if it is supposed to vanish, or

Re: [RFC 0/8] Allow GFP_NOFS allocation to fail

2015-08-05 Thread Andreas Dilger
On Aug 5, 2015, at 3:51 AM, mho...@kernel.org wrote: Hi, small GFP_NOFS, like GFP_KERNEL, allocations have not been not failing traditionally even though their reclaim capabilities are restricted because the VM code cannot recurse into filesystems to clean dirty pages. At the same time these

Re: [PATCH 0/6] sysfs-part2 Add seed device representation on the sysfs

2015-08-05 Thread Anand Jain
Hi David, Thanks. more below. On 08/06/2015 01:29 AM, David Sterba wrote: On Wed, Jul 08, 2015 at 03:32:48PM +0800, Anand Jain wrote: This patch adds the support to show seed device on the btrfs sysfs. This is a revamped version of the previously single patch 6/6, and plus incorporates

bedup --defrag freezing

2015-08-05 Thread Konstantin Svist
Hi, I've been running btrfs on Fedora for a while now, with bedup --defrag running in a night-time cronjob. Last few runs seem to have gotten stuck, without possibility of even killing the process (kill -9 doesn't work) -- all I could do is hard power cycle. Did something change recently? Is

RE: [PATCH 1/3] btrfs: Error handle for get_ref_objectid_v0() in relocate_block_group()

2015-08-05 Thread Zhao Lei
Hi, David Sterba -Original Message- From: David Sterba [mailto:dste...@suse.com] Sent: Thursday, August 06, 2015 1:03 AM To: Zhao Lei Cc: linux-btrfs@vger.kernel.org Subject: Re: [PATCH 1/3] btrfs: Error handle for get_ref_objectid_v0() in relocate_block_group() On Wed, Aug 05,

RE: [PATCH] btrfs-progs: add newline to some error messages

2015-08-05 Thread Zhao Lei
Hi, Itoh-san -Original Message- From: linux-btrfs-ow...@vger.kernel.org [mailto:linux-btrfs-ow...@vger.kernel.org] On Behalf Of Zhao Lei Sent: Thursday, August 06, 2015 11:51 AM To: 'Tsutomu Itoh'; linux-btrfs@vger.kernel.org Subject: RE: [PATCH] btrfs-progs: add newline to some

Re: [PATCH] btrfs-progs: add newline to some error messages

2015-08-05 Thread Tsutomu Itoh
On 2015/08/06 12:51, Zhao Lei wrote: Hi, Itoh -Original Message- From: linux-btrfs-ow...@vger.kernel.org [mailto:linux-btrfs-ow...@vger.kernel.org] On Behalf Of Tsutomu Itoh Sent: Thursday, August 06, 2015 11:06 AM To: linux-btrfs@vger.kernel.org Subject: [PATCH] btrfs-progs: add

RE: [PATCH] btrfs-progs: Modify confuse error message in scrub

2015-08-05 Thread Zhao Lei
Hi, David Sterba Thanks for review this patch. -Original Message- From: David Sterba [mailto:dste...@suse.com] Sent: Thursday, August 06, 2015 12:51 AM To: Zhao Lei Cc: linux-btrfs@vger.kernel.org Subject: Re: [PATCH] btrfs-progs: Modify confuse error message in scrub On Wed,

Re: BTRFS disaster (of my own making). Is this recoverable?

2015-08-05 Thread Chris Murphy
On Wed, Aug 5, 2015 at 6:45 PM, Paul Jones p...@pauljones.id.au wrote: Would it be possible to store this type of critical information twice on each disk, at the beginning and end? I thought BTRFS already did that, but I might be thinking of some other filesystem. I've had my share of these

[PATCH] btrfs-progs: add newline to some error messages

2015-08-05 Thread Tsutomu Itoh
Added a missing newline to some error messages. Signed-off-by: Tsutomu Itoh t-i...@jp.fujitsu.com --- btrfs-corrupt-block.c | 2 +- cmds-check.c | 4 ++-- cmds-send.c | 4 ++-- dir-item.c| 6 +++--- mkfs.c| 2 +- 5 files changed, 9 insertions(+), 9

[PATCH v2 1/2] btrfs-progs: use switch instead of a series of ifs for output errormsg

2015-08-05 Thread Zhao Lei
switch statement is more suitable for outputing currsponding message for errno. Suggested-by: David Sterba dste...@suse.com Signed-off-by: Zhao Lei zhao...@cn.fujitsu.com --- cmds-scrub.c | 33 ++--- 1 file changed, 18 insertions(+), 15 deletions(-) diff --git

[PATCH v2 2/2] btrfs-progs: Modify confuse error message in scrub

2015-08-05 Thread Zhao Lei
Scrub output following error message in my test: ERROR: scrubbing /var/ltf/tester/scratch_mnt failed for device id 5 (Success) It is caused by a broken kernel and fs, but the we need to avoid outputting both error and success in oneline message as above. This patch modified above message to:

RE: BTRFS disaster (of my own making). Is this recoverable?

2015-08-05 Thread Paul Jones
-Original Message- From: linux-btrfs-ow...@vger.kernel.org [mailto:linux-btrfs- ow...@vger.kernel.org] On Behalf Of Chris Murphy Sent: Thursday, 6 August 2015 2:54 AM To: Sonic sonicsm...@gmail.com Cc: Btrfs BTRFS linux-btrfs@vger.kernel.org; Hugo Mills h...@carfax.org.uk Subject:

RE: [PATCH] btrfs-progs: add newline to some error messages

2015-08-05 Thread Zhao Lei
Hi, Itoh -Original Message- From: linux-btrfs-ow...@vger.kernel.org [mailto:linux-btrfs-ow...@vger.kernel.org] On Behalf Of Tsutomu Itoh Sent: Thursday, August 06, 2015 11:06 AM To: linux-btrfs@vger.kernel.org Subject: [PATCH] btrfs-progs: add newline to some error messages Added