Re: [PATCH 00/16] btrfs-progs: Split lowmem mode check to its own

On 01/19/2018 01:37 PM, Qu Wenruo wrote: The long planned cmds-check re-construction is finally here. As the original cmds-check.c is getting larger and larger (already over 15K lines), it's always a good idea to split it into its own check/ directory. This patchset do the following work: 1)

Re: [PATCH v2 2/3] btrfs-progs: dir-item: Don't do extra filetype validaction check for btrfs_match_dir_item_name

On 2018年01月19日 15:39, Su Yue wrote: > > > On 01/19/2018 03:25 PM, Qu Wenruo wrote: >> btrfs_match_dir_item_name() will check if its filetype is valid before >> doing search, this makes btrfs-progs unable to locate and remove invalid >> dir_index for btrfs_unlink(). >> >> This function only

Re: [PATCH v2 3/3] btrfs-progs: dir-item: Make btrfs_delete_one_dir_name more robust to handle corrupted name len

On 01/19/2018 03:25 PM, Qu Wenruo wrote: Function btrfs_delete_one_dir_name() will check if the dir_item is the last content of the item, and delete the whole item if needed. However if @name_len of one dir_item/dir_index is corrupted and larger than the item size, the function will still try

Re: [PATCH v2 2/3] btrfs-progs: dir-item: Don't do extra filetype validaction check for btrfs_match_dir_item_name

On 01/19/2018 03:25 PM, Qu Wenruo wrote: btrfs_match_dir_item_name() will check if its filetype is valid before doing search, this makes btrfs-progs unable to locate and remove invalid dir_index for btrfs_unlink(). This function only affects btrfs_link() and btrfs_unlink() in upper layer, and

Re: [PATCH v2 1/3] btrfs-progs: lowmem fsck: Remove corupted link before re-add correct link

On 01/19/2018 03:25 PM, Qu Wenruo wrote: For repair_ternary_lowmem() used in lowmem mode, if it found 1 of DIR_INDEX/DIR_ITEM/INODE_REF missing, it will try to insert correct link. However for case like invalid type in DIR_INDEX, we should delete the corrupted DIR_INDEX first before inserting

Re: [PATCH 05/16] btrfs-progs: check: Export check global variables to check/common.h

On 2018年01月19日 14:55, Su Yue wrote: > > > On 01/19/2018 01:37 PM, Qu Wenruo wrote: >> There are a dozen of variables which are used as "check global" >> variables, like @total_csum_bytes or @no_holes. >> >> These variables are used freely across the check code, however since >> we're splitting

[PATCH v2 0/3] Lowmem fsck repair to fix filetype mismatch

Sebastian reported a filesystem corruption where DIR_INDEX has wrong filetype against INODE_ITEM. Lowmem mode normally handles such problem by checking DIR_INDEX, DIR_ITEM and INODE_REF/INODE_ITEM to determine the correct file type. In such case, lowmem mode fsck can get the correct filetype.

[PATCH v2 1/3] btrfs-progs: lowmem fsck: Remove corupted link before re-add correct link

For repair_ternary_lowmem() used in lowmem mode, if it found 1 of DIR_INDEX/DIR_ITEM/INODE_REF missing, it will try to insert correct link. However for case like invalid type in DIR_INDEX, we should delete the corrupted DIR_INDEX first before inserting the correct link. This patch will remove

[PATCH v2 3/3] btrfs-progs: dir-item: Make btrfs_delete_one_dir_name more robust to handle corrupted name len

Function btrfs_delete_one_dir_name() will check if the dir_item is the last content of the item, and delete the whole item if needed. However if @name_len of one dir_item/dir_index is corrupted and larger than the item size, the function will still try to treat it as partly remove, which will

Re: [PATCH 08/16] btrfs-progs: check: Move count_csum_range function to check/common.c

On 01/19/2018 01:37 PM, Qu Wenruo wrote: Despite of moving it to check/common.c, also: 1) Add extra comment of the function 2) Change @root paramter to @fs_info 'paramter' may be misspelled - perhaps 'parameter'? Thanks, Su Since @root is never used, csum_root is picked from fs_info

Re: [PATCH 04/16] btrfs-progs: check: Move node_ptr structure to check/common.h

On 2018年01月19日 13:52, Su Yue wrote: > The structure name is 'node_refs' not 'node_ptr'. > Misspelt the patch name? > You got me! I'll just update the title in github. Thanks, Qu > Thanks, > Su > > On 01/19/2018 01:37 PM, Qu Wenruo wrote: >> Signed-off-by: Qu Wenruo >> ---

Re: [PATCH 04/16] btrfs-progs: check: Move node_ptr structure to check/common.h

The structure name is 'node_refs' not 'node_ptr'. Misspelt the patch name? Thanks, Su On 01/19/2018 01:37 PM, Qu Wenruo wrote: Signed-off-by: Qu Wenruo --- check/common.h | 39 +++ check/main.c | 11 +-- 2 files changed, 40

[PATCH 14/16] btrfs-progs: check: Move reset_cached_block_groups to check/common.c

Signed-off-by: Qu Wenruo --- check/common.c | 25 + check/common.h | 1 + check/main.c | 27 --- 3 files changed, 26 insertions(+), 27 deletions(-) diff --git a/check/common.c b/check/common.c index 4cdc46b0ba7c..d6abf6d6733c

[PATCH 13/16] btrfs-progs: check: Move check_child_node to check/common.c

Signed-off-by: Qu Wenruo --- check/common.c | 49 + check/common.h | 2 ++ check/main.c | 49 - 3 files changed, 51 insertions(+), 49 deletions(-) diff --git a/check/common.c

[PATCH 10/16] btrfs-progs: check: Move link_inode_to_lostfound function to common.c

Signed-off-by: Qu Wenruo --- check/common.c | 93 ++ check/common.h | 5 check/main.c | 92 - 3 files changed, 98 insertions(+), 92 deletions(-) diff --git

[PATCH 16/16] btrfs-progs: check/lowmem: Cleanup unnecessary _v2 suffix

There used to be some functions with _v2 suffix to distinguish them from original mode similar functions. However now moved lowmem code to their own check/lowmem.[ch], cleanup such _v2 suffixes, and for functions really needs to be distinguished from original mode (exported functions), change the

[PATCH 09/16] btrfs-progs: check: Move __create_inode_item function to check/common.c

Move __create_inode_item() function to check/common.c and rename it to insert_inode_item(), with comment added. Signed-off-by: Qu Wenruo --- check/common.c | 45 + check/common.h | 3 +++ check/main.c | 36

[PATCH 12/16] btrfs-progs: check: move reada_walk_down to check/common.c

Both original and lowmem mode shares this function to do readahead. Signed-off-by: Qu Wenruo --- check/common.c | 23 +++ check/common.h | 2 ++ check/main.c | 22 -- 3 files changed, 25 insertions(+), 22 deletions(-) diff --git

[PATCH 11/16] btrfs-progs: check: Move check_dev_size_alignment to check/common.c

Signed-off-by: Qu Wenruo --- check/common.c | 15 +++ check/common.h | 1 + check/main.c | 16 3 files changed, 16 insertions(+), 16 deletions(-) diff --git a/check/common.c b/check/common.c index 9051936a61cb..7392ed6b472f 100644 ---

[PATCH 08/16] btrfs-progs: check: Move count_csum_range function to check/common.c

Despite of moving it to check/common.c, also: 1) Add extra comment of the function 2) Change @root paramter to @fs_info Since @root is never used, csum_root is picked from fs_info anyway. Signed-off-by: Qu Wenruo --- Makefile | 2 +- check/common.c | 101

[PATCH 01/16] btrfs-progs: Moves cmds-check.c to check/main.c

Signed-off-by: Qu Wenruo --- Makefile | 4 ++-- cmds-check.c => check/main.c | 0 2 files changed, 2 insertions(+), 2 deletions(-) rename cmds-check.c => check/main.c (100%) diff --git a/Makefile b/Makefile index 6369e8f4209c..c4e2dc5b68a9 100644 ---

[PATCH 00/16] btrfs-progs: Split lowmem mode check to its own

The long planned cmds-check re-construction is finally here. As the original cmds-check.c is getting larger and larger (already over 15K lines), it's always a good idea to split it into its own check/ directory. This patchset do the following work: 1) Move cmds-check.c to check/main.c 2) Put

[PATCH 03/16] btrfs-progs: check: Move definitions of lowmem mode to check/lowmem.h

Unlike original mode, lowmem mode mostly uses normal tree operations, so no structure definitions, only a lot of random error bits. Signed-off-by: Qu Wenruo --- check/lowmem.h | 62 ++ check/main.c | 39

[PATCH 05/16] btrfs-progs: check: Export check global variables to check/common.h

There are a dozen of variables which are used as "check global" variables, like @total_csum_bytes or @no_holes. These variables are used freely across the check code, however since we're splitting check code, they need to be exported so they can be used in other files. This patch just export

[PATCH 07/16] btrfs-progs: check: Move fs_root_objectid function to check/common.h

Just another small wrapper shared between original and lowmem mode. Signed-off-by: Qu Wenruo --- check/common.h | 8 check/main.c | 10 -- 2 files changed, 8 insertions(+), 10 deletions(-) diff --git a/check/common.h b/check/common.h index

[PATCH 06/16] btrfs-progs: check: Move imode_to_type function to check/common.h

This function is shared between original and lowmem mode, and it's small enough, so move it to check/common.h. Signed-off-by: Qu Wenruo --- check/common.h | 19 +++ check/main.c | 17 - 2 files changed, 19 insertions(+), 17 deletions(-) diff

[PATCH 04/16] btrfs-progs: check: Move node_ptr structure to check/common.h

Signed-off-by: Qu Wenruo --- check/common.h | 39 +++ check/main.c | 11 +-- 2 files changed, 40 insertions(+), 10 deletions(-) create mode 100644 check/common.h diff --git a/check/common.h b/check/common.h new file mode 100644 index

[PATCH 02/16] btrfs-progs: check: Move original mode definitions to check/original.h

Signed-off-by: Qu Wenruo --- check/main.c | 269 +- check/original.h | 293 +++ 2 files changed, 294 insertions(+), 268 deletions(-) create mode 100644 check/original.h diff

Re: [PATCH] btrfs: Use IS_ALIGNED in btrfs_truncate_block instead of opencoding it

On 2018年01月18日 20:47, Nikolay Borisov wrote: > No functional changes, just makes the code more readable > > Signed-off-by: Nikolay Borisov Reviewed-by: Qu Wenruo Thanks, Qu > --- > fs/btrfs/inode.c | 4 ++-- > 1 file changed, 2 insertions(+), 2

Re: [PATCH v5 01/19] fs: new API for handling inode->i_version

On Thu, 2018-01-18 at 16:38 -0500, J. Bruce Fields wrote: > On Tue, Jan 09, 2018 at 09:10:41AM -0500, Jeff Layton wrote: > > --- /dev/null > > +++ b/include/linux/iversion.h > > @@ -0,0 +1,236 @@ > > +/* SPDX-License-Identifier: GPL-2.0 */ > > +#ifndef _LINUX_IVERSION_H > > +#define

Re: [PATCH v5 02/19] fs: don't take the i_lock in inode_inc_iversion

On Tue, Jan 09, 2018 at 09:10:42AM -0500, Jeff Layton wrote: > From: Jeff Layton > > The rationale for taking the i_lock when incrementing this value is > lost in antiquity. The readers of the field don't take it (at least > not universally), so my assumption is that it was

Re: [PATCH v5 01/19] fs: new API for handling inode->i_version

On Tue, Jan 09, 2018 at 09:10:41AM -0500, Jeff Layton wrote: > --- /dev/null > +++ b/include/linux/iversion.h > @@ -0,0 +1,236 @@ > +/* SPDX-License-Identifier: GPL-2.0 */ > +#ifndef _LINUX_IVERSION_H > +#define _LINUX_IVERSION_H > + > +#include > + > +/* > + * The change attribute (i_version) is

Re: [PATCH RESEND v4 0/4] device_list_add() peparation to add reappearing missing device

On Thu, Jan 18, 2018 at 10:02:32PM +0800, Anand Jain wrote: > (Apply on top of my patchset >[PATCH v4 0/6] preparatory work to add device forget > for conflict free apply. They don't actually depend on > each other though). > Cleanup of device_list_add(), mainly in preparation to handle >

Re: [PATCH v6 00/99] XArray version 6

On Thu, Jan 18, 2018 at 05:56:12PM +0100, David Sterba wrote: > On Thu, Jan 18, 2018 at 08:48:43AM -0800, Matthew Wilcox wrote: > > Thank you! I shall attempt to debug. Was this with a btrfs root > > filesystem? I'm most suspicious of those patches right now, since they've > > received next to

Re: [PATCH v6 00/99] XArray version 6

On Thu, Jan 18, 2018 at 08:48:43AM -0800, Matthew Wilcox wrote: > Thank you! I shall attempt to debug. Was this with a btrfs root > filesystem? I'm most suspicious of those patches right now, since they've > received next to no testing. I'm going to put together a smaller patchset > which just

Re: [PATCH v2 00/10] bugfixes and regression tests of btrfs_get_extent

On Mon, Jan 08, 2018 at 08:57:34PM +0100, David Sterba wrote: > On Fri, Jan 05, 2018 at 12:51:07PM -0700, Liu Bo wrote: > > Although > > commit e6c4efd87ab0 ("btrfs: Fix and enhance merge_extent_mapping() to > > insert best fitted extent map") > > fixed up the negetive em->len, it has introduced

Re: [PATCH v6 00/99] XArray version 6

On Thu, Jan 18, 2018 at 05:07:50PM +0100, David Sterba wrote: > On Wed, Jan 17, 2018 at 12:20:24PM -0800, Matthew Wilcox wrote: > > From: Matthew Wilcox > > > > This version of the XArray has no known bugs. > > I've booted this patchset on 2 boxes, both had random

Re: [PATCH v4 2/4] btrfs: cleanup btrfs_mount() using btrfs_mount_root()

On Thu, Jan 18, 2018 at 12:48:37PM +0800, Anand Jain wrote: > > On 2018/01/16 20:45, Anand Jain wrote: > >> On 01/16/2018 03:26 AM, David Sterba wrote: > >>> On Fri, Jan 12, 2018 at 06:14:40PM +0800, Anand Jain wrote: > > Misono, > > This change is causing subsequent

Re: [PATCH v6 00/99] XArray version 6

On Wed, Jan 17, 2018 at 12:20:24PM -0800, Matthew Wilcox wrote: > From: Matthew Wilcox > > This version of the XArray has no known bugs. I've booted this patchset on 2 boxes, both had random problems during boot. On one I was not able to diagnose what went wrong. On the

Re: [PATCH] Btrfs: do not cache rbio pages if using raid6 recover

On Fri, Jan 12, 2018 at 06:07:02PM -0700, Liu Bo wrote: > Since raid6 recover tries all possible combinations of failed stripes, > > - when raid6 rebuild algorithm is used, i.e. raid6_datap_recov() and > raid6_2data_recov(), it may change the in-memory content of failed > stripes, if such a

Re: [PATCH] Btrfs: raid56: iterate raid56 internal bio with bio_for_each_segment_all

On Fri, Jan 12, 2018 at 06:07:01PM -0700, Liu Bo wrote: > Bio iterated by set_bio_pages_uptodate() is raid56 internal one, so it > will never be a BIO_CLONED bio, and since this is called by end_io > functions, bio->bi_iter.bi_size is zero, we mustn't use > bio_for_each_segment() as that is a

Re: [PATCH 2/2] btrfs: remove unused arg from parse_subvol_options()

On Wed, Jan 17, 2018 at 05:38:31PM +0900, Misono, Tomohiro wrote: > Remove unused arg 'holder' from parse_subvol_options(), which has been > forgotten to be cleaned in the commit b99beb110e2d ("btrfs: split > parse_early_options() in two"). > > Signed-off-by: Tomohiro Misono

Re: [PATCH 1/2] btrfs: fix the bug of device scan/ready for mounted, filesystem

On Wed, Jan 17, 2018 at 05:37:39PM +0900, Misono, Tomohiro wrote: > commit ae3acc5fc0bf ("btrfs: cleanup btrfs_mount() using > btrfs_mount_root()") introduces a bug that "btrfs device scan/ready" for > mounted filesystem fails. > > This is because fs_info->bdev_holder has been changed to hold >

Re: [PATCH v6 85/99] btrfs: Remove unused spinlock

On Wed, Jan 17, 2018 at 12:21:49PM -0800, Matthew Wilcox wrote: > From: Matthew Wilcox > > The reada_lock in struct btrfs_device was only initialised, and not > actually used. That's good because there's another lock also called > reada_lock in the btrfs_fs_info that was

Re: [PATCH 1/2] Btrfs: fix missing inode i_size update after zero range operation

On Thu, Jan 18, 2018 at 11:34:20AM +, fdman...@kernel.org wrote: > From: Filipe Manana > > For a fallocate's zero range operation that targets a range with an end > that is not aligned to the sector size, we can end up not updating the > inode's i_size. This happens when

Re: [PATCH 2/2] Btrfs: fix space leak after fallocate and zero range operations

On Thu, Jan 18, 2018 at 01:45:20PM +0200, Nikolay Borisov wrote: > > [95795.015577] R10: 06b4 R11: 0246 R12: > > 7fa6791bae64 > > [95795.016569] R13: R14: 563386706210 R15: > > 7ffccf0ab160 > > [95795.017662] Code: 00 00 00 4c 8b a3 98 25

Re: [PATCH] Fstests: btrfs/011 fix device mounted when tests aborts

On 01/17/2018 04:10 AM, Liu Bo wrote: One of btrfs tests, btrfs/011, uses SCRATCH_DEV_POOL and puts a non-SCRATCH_DEV device as the first one when doing mkfs, and this makes _require_scratch{_nocheck} confused since it checks mount point with SCRATCH_DEV only. This adds _scratch_umount to

[PATCH 2/4] btrfs: set the total_devices in device_list_add()

There is no other parent for device_list_add() except for btrfs_scan_one_device(), which would set btrfs_fs_devices::total_devices if device_list_add is successful and this can be done with in device_list_add() itself. Signed-off-by: Anand Jain Reviewed-by: Josef Bacik

[PATCH 3/4] btrfs: get device pointer from device_list_add()

Instead of pointer to btrfs_fs_devices as an arg in device_list_add() better to get pointer to btrfs_device as return value, then we have both, pointer to btrfs_device and btrfs_fs_devices. btrfs_device is needed to handle reappearing missing device. Signed-off-by: Anand Jain

[PATCH 4/4] btrfs: drop devid as device_list_add() arg

As struct btrfs_disk_super is being passed, so it can get devid the same way its parent does. Signed-off-by: Anand Jain Reviewed-by: Josef Bacik --- fs/btrfs/volumes.c | 8 +++- 1 file changed, 3 insertions(+), 5 deletions(-) diff --git

[PATCH 1/4] btrfs: move pr_info into device_list_add

Commit 60999ca4b403 ("btrfs: make device scan less noisy") adds return value 1 to device_list_add(), so that parent function can call pr_info only when new device is added. Move the pr_info() part into device_list_add() so that this function can be kept simple. Signed-off-by: Anand Jain

[PATCH RESEND v4 0/4] device_list_add() peparation to add reappearing missing device

(Apply on top of my patchset [PATCH v4 0/6] preparatory work to add device forget for conflict free apply. They don't actually depend on each other though). v3->v4: @3/4: Just return device instead of PTR_ERR(ERR_PTR(device)); v2->v3: Fix device_list_add() fn description which was still

[PATCH v5 0/6] preparatory work to add device forget

v4->v5: Fix fn name btrfs_free_stale_device() to btrfs_free_stale_devices() in the comments and commit title. No code change. Add received reviewed-by. v3->v4: Mainly fix as per comments from Josef. @3/6: rename btrfs_free_stale_device() to btrfs_free_stale_devices() @4/6: reorg logic, init

[PATCH 2/6] btrfs: no need to check for btrfs_fs_devices::seeding

There is no need to check for btrfs_fs_devices::seeding when we have checked for btrfs_fs_devices::opened, because we can't sprout without its seed FS being opened. Signed-off-by: Anand Jain Reviewed-by: Josef Bacik --- fs/btrfs/volumes.c | 2 -- 1 file

[PATCH 6/6] btrfs: make btrfs_free_stale_devices() to match the path

From: Anand Jain The btrfs_free_stale_devices() is updated to match for the given device path and delete it. (It searches for only unmounted list of devices.) Also drop the comment about different path being used for the same device, since now we will have cli to clean any

[PATCH 4/6] btrfs: make btrfs_free_stale_devices() argument optional

From: Anand Jain This updates btrfs_free_stale_devices() helper function to delete all unmouted devices, when arg is NULL. Signed-off-by: Anand Jain Reviewed-by: Josef Bacik --- fs/btrfs/volumes.c | 14 +- 1 file

[PATCH 3/6] btrfs: make btrfs_free_stale_device() to iterate all stales

From: Anand Jain Let the list iterator iterate further and find other stale devices and delete it. This is in preparation to add support for user land request-able stale devices cleanup. Also rename btrfs_free_stale_device() to btrfs_free_stale_devices(). Signed-off-by:

[PATCH 1/6] btrfs: cleanup btrfs_free_stale_device() usage

We call btrfs_free_stale_device() only when we alloc a new struct btrfs_device (ret=1), so move it closer to where we alloc the new device. Also drop the comments. Signed-off-by: Anand Jain Reviewed-by: Josef Bacik --- fs/btrfs/volumes.c | 8 +--- 1

Re: [PATCH 1/2] btrfs: fix device order consistency

On 01/18/2018 04:32 PM, Nikolay Borisov wrote: On 18.01.2018 04:32, Anand Jain wrote: By maintaining the device order consistency it makes reproducing the problem more consistent. So fix this by having the devices Which problem is that ? I noticed when trying to reproduce raid1 missed

[PATCH] btrfs: Use IS_ALIGNED in btrfs_truncate_block instead of opencoding it

No functional changes, just makes the code more readable Signed-off-by: Nikolay Borisov --- fs/btrfs/inode.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c index 269d129ffb1f..e9690e2aba09 100644 ---

Re: [PATCH 2/2] Btrfs: fix space leak after fallocate and zero range operations

On 18.01.2018 13:34, fdman...@kernel.org wrote: > From: Filipe Manana > > If we do a buffered write after a zero range operation that has an > unaligned (with the filesystem's sector size) end which also falls within > an unwritten (prealloc) extent that is currently beyond

[PATCH 1/2] Btrfs: fix missing inode i_size update after zero range operation

From: Filipe Manana For a fallocate's zero range operation that targets a range with an end that is not aligned to the sector size, we can end up not updating the inode's i_size. This happens when the last page of the range maps to an unwritten (prealloc) extent and before

[PATCH 2/2] Btrfs: fix space leak after fallocate and zero range operations

From: Filipe Manana If we do a buffered write after a zero range operation that has an unaligned (with the filesystem's sector size) end which also falls within an unwritten (prealloc) extent that is currently beyond the inode's i_size, and the zero range operation has the

Re: [PATCH 1/2] btrfs: fix device order consistency

On 18.01.2018 04:32, Anand Jain wrote: > By maintaining the device order consistency it makes reproducing > the problem more consistent. So fix this by having the devices Which problem is that ? > sorted by some order within the kernel, lets say by devid. > > Signed-off-by: Anand Jain