[PATCH 12/20] btrfs-progs: cmds-check.c: introduce __create_inode_item

2017-02-28 Thread Su Yue
Introduce '__create_inode' to create and insert inode item. Modify origin 'create_inode_item' call it. Create 'create_inode_item_lowmem' call it. The patch is for further lowmem repair. Signed-off-by: Su Yue --- cmds-check.c | 83 ++-- 1 fi

Re: [PATCH 1/2] Btrfs: fix unexpected file hole after disk errors

2017-02-28 Thread Qu Wenruo
At 03/01/2017 09:04 AM, Liu Bo wrote: Btrfs creates hole extents to cover any unwritten section right before doing buffer writes after commit 3ac0d7b96a26 ("btrfs: Change the expanding write sequence to fix snapshot related bug."). However, that takes the start position of the buffered write t

Re: [PATCH 1/2] Btrfs: fix unexpected file hole after disk errors

2017-02-28 Thread Qu Wenruo
It seems that my previous mail doesn't reach mail list. So send again. At 03/01/2017 09:04 AM, Liu Bo wrote: Btrfs creates hole extents to cover any unwritten section right before doing buffer writes after commit 3ac0d7b96a26 ("btrfs: Change the expanding write sequence to fix snapshot related b

[PATCH 03/20] btrfs-progs: cmds-check.c: inode orphan item repair

2017-02-28 Thread Su Yue
Add a function named 'repair_inode_orphan_item_lowmem'. Signed-off-by: Su Yue --- cmds-check.c | 50 +- 1 file changed, 49 insertions(+), 1 deletion(-) diff --git a/cmds-check.c b/cmds-check.c index f13ce317..fb239968 100644 --- a/cmds-check.c +++

[PATCH 11/20] btrfs-progs: inode.c: alter btrfs_add_link

2017-02-28 Thread Su Yue
Add an arg 'ignore_existed' to btrfs_add_link. If ignore_existed=1, continue to add while relative dir index/item or inode ref is already existed. This patch is for further repair. Signed-off-by: Su Yue --- cmds-check.c | 6 +++--- convert/main.c | 2 +- ctree.h| 2 +- inode.c

[PATCH 05/20] btrfs-progs: cmds-check.c: modify check_fs_first_inode

2017-02-28 Thread Su Yue
Modify check_fs_first_inode to check firt_inode inode item and inode ref. Signed-off-by: Su Yue --- cmds-check.c | 57 ++--- 1 file changed, 42 insertions(+), 15 deletions(-) diff --git a/cmds-check.c b/cmds-check.c index 246f4735..892a22ba 10

Re: [PATCH 3/8] nowait aio: return if direct write will trigger writeback

2017-02-28 Thread Matthew Wilcox
On Tue, Feb 28, 2017 at 05:36:05PM -0600, Goldwyn Rodrigues wrote: > Find out if the write will trigger a wait due to writeback. If yes, > return -EAGAIN. > > This introduces a new function filemap_range_has_page() which > returns true if the file's mapping has a page within the range > mentioned.

[PATCH 10/20] btrfs-progs: dir-item.c: modify btrfs_insert_dir_item

2017-02-28 Thread Su Yue
In the function 'btrfs_insert_dir_item', let it continue to insert dir index if the dir item is existed and set ret value to 0. This further is for further repair. Signed-off-by: Su Yue --- dir-item.c | 13 - 1 file changed, 12 insertions(+), 1 deletion(-) diff --git a/dir-item.c b

Re: [PATCH 3/4] reflink: test adjacency of reflinked blocks

2017-02-28 Thread Eryu Guan
On Tue, Feb 28, 2017 at 08:31:50AM -0800, Darrick J. Wong wrote: > On Tue, Feb 28, 2017 at 04:15:02PM +0800, Eryu Guan wrote: > > On Fri, Feb 24, 2017 at 05:12:57PM -0800, Darrick J. Wong wrote: > > > From: Darrick J. Wong > > > > > > If we reflink a file with N blocks to another file one block a

[PATCH 07/20] btrfs-progs: cmds-check.c: introduce print_inode_ref

2017-02-28 Thread Su Yue
Introduce 'print_inode_ref' to print error msg while checking inode ref. Add args 'name_ret' and 'namelen_ret' to 'check_inode_ref' because they are essential while doing nlinks repair. Signed-off-by: Su Yue --- cmds-check.c | 93 +++- 1 f

[PATCH 04/20] btrfs-progs: cmds-check.c: change find_inode_ref's arg

2017-02-28 Thread Su Yue
For further lowmem repairs, change the index type u64 to u64 *. So we could get the index of ref. Signed-off-by: Su Yue --- cmds-check.c | 24 +++- 1 file changed, 15 insertions(+), 9 deletions(-) diff --git a/cmds-check.c b/cmds-check.c index fb239968..246f4735 100644 --- a

[PATCH 13/20] btrfs-progs: cmds-check.c: repair_inode_item_missing

2017-02-28 Thread Su Yue
Introduce 'repair_inode_item_missing' to fix INODE_ITEM_MISSING This patch is for further repair. Signed-off-by: Su Yue --- cmds-check.c | 43 +++ 1 file changed, 43 insertions(+) diff --git a/cmds-check.c b/cmds-check.c index 256bfbc9..6dafbd7d 100644 --

[PATCH 08/20] btrfs-progs: cmds-check.c: print_dir_item_err

2017-02-28 Thread Su Yue
Introduce 'print_dir_item_err' to print error msg while checking dir_item/dir_index. 'check_dir_item' now checks relative dir item and calls 'print_dir_itm_err" to print error msg. Signed-off-by: Su Yue --- cmds-check.c | 127 --- 1 file ch

[PATCH 00/20] Enable lowmem repair for fs/subvolume tree

2017-02-28 Thread Su Yue
It can be feched from my github: https://github.com/Damenly/btrfs-progs.git lowmem_repair This patchset can repair errors found in fs tree in lowmem mode. This patchset request includes: 1) Repair inode nbytes error. 2) Repair dir isize error. 3) Repair orpahan inode item. 4) Repair dir item/inde

[PATCH 09/20] btrfs-progs: cmds-check.c: introduce count_dir_isize

2017-02-28 Thread Su Yue
Introduce 'count_dir_isize' to get dir isize. This function is called only under lowmme repair mode. Signed-off-by: Su Yue --- cmds-check.c | 95 ++-- 1 file changed, 92 insertions(+), 3 deletions(-) diff --git a/cmds-check.c b/cmds-check.

[PATCH 14/20] btrfs-progs: cmds-check.c: repair_fs_first_inode

2017-02-28 Thread Su Yue
Introduce 'repair_fs_first_inode' to repair first inode errors. Signed-off-by: Su Yue --- cmds-check.c | 69 1 file changed, 69 insertions(+) diff --git a/cmds-check.c b/cmds-check.c index 6dafbd7d..85e87884 100644 --- a/cmds-check.c

[PATCH 02/20] btrfs-progs: cmds-check.c: supports dir isize fix in lowmem

2017-02-28 Thread Su Yue
Add a function 'repair_dir_isize_lowmem' to support dir isize repair in lowmem mode. Signed-off-by: Su Yue --- cmds-check.c | 69 +--- 1 file changed, 66 insertions(+), 3 deletions(-) diff --git a/cmds-check.c b/cmds-check.c index 40f9d21e

[PATCH 19/20] btrfs-progs: cmds-check.c: add punch_extent_hole

2017-02-28 Thread Su Yue
Introduce 'punch_extent_hole' to punch holes while repair file extent. Signed-off-by: Su Yue --- cmds-check.c | 51 +-- 1 file changed, 45 insertions(+), 6 deletions(-) diff --git a/cmds-check.c b/cmds-check.c index ae80d5f0..d228fc62 100644 --- a

[PATCH 01/20] btrfs-progs: cmds-check.c: supports inode nbytes fix in lowmem

2017-02-28 Thread Su Yue
Add a function 'repair_inode_nbytes_lowmem' to correct inode item nbytes error in lowmem mode. Signed-off-by: Su Yue --- cmds-check.c | 82 +++- 1 file changed, 76 insertions(+), 6 deletions(-) diff --git a/cmds-check.c b/cmds-check.c inde

[PATCH 18/20] btrfs-progs: cmds-check.c: repair nlinks lowmem

2017-02-28 Thread Su Yue
Introduce 'repair_inode_nlinks_lowmem'. If ref is 0, move the inode to "lost + found". Set inode item's nlink to ref_count. Signed-off-by: Su Yue --- cmds-check.c | 233 +-- 1 file changed, 179 insertions(+), 54 deletions(-) diff --git a/c

[PATCH 15/20] btrfs-progs: cmds-check.c: introduce repair_ternary_lowmem

2017-02-28 Thread Su Yue
Introduce 'repair_ternary_lowmem' to repair error while checking dir_item/index, inode_ref by the rule: 1. If two of three is missing or mismatched, delete the existed one. 2. If one of three is missing or mismatched, add the missing one. Signed-off-by: Su Yue --- cmds-check.c | 57 +

[PATCH 17/20] btrfs-progs: cmds-check.c: repair inode ref

2017-02-28 Thread Su Yue
Call repair_ternary_lowmem while checking inode ref. Introduce 'repair_dir_item' calls above function to repair dir_item. Signed-off-by: Su Yue --- cmds-check.c | 29 +++-- 1 file changed, 27 insertions(+), 2 deletions(-) diff --git a/cmds-check.c b/cmds-check.c index 1

[PATCH 16/20] btrfs-progs: cmds-check.c: Introduce repair_dir_item

2017-02-28 Thread Su Yue
Introduce 'repair_dir_item' to repair dir item/index missing/mismatch and relative inode item missing while checking. Signed-off-by: Su Yue --- cmds-check.c | 78 1 file changed, 78 insertions(+) diff --git a/cmds-check.c b/cmds-check

[PATCH 20/20] btrfs-progs: fsck-check: Allow fsck check test to repair in lowmem mode for certain test cases

2017-02-28 Thread Su Yue
From: Qu Wenruo Since lowmem mode can repair certain corruptions (mostly in fs tree), insert a beacon into each fsck test cases to allow some of them be tested for lowmem mode. With this patch, fsck option override will check the beacon file ".lowmem_repairbale" in the same directory of the test

[PATCH 06/20] btrfs-progs: cmds-check.c: change find_dir_index/item

2017-02-28 Thread Su Yue
For further lowmem_repair, introduce 'find_dir_index' to get the index by other inode item information. Remove 'check_dir_item' error msg print. Adjust 'find_dir_item' args and remove err msg print. Signed-off-by: Su Yue --- cmds-check.c | 231 +++

[PATCH 1/2] Btrfs: fix unexpected file hole after disk errors

2017-02-28 Thread Liu Bo
Btrfs creates hole extents to cover any unwritten section right before doing buffer writes after commit 3ac0d7b96a26 ("btrfs: Change the expanding write sequence to fix snapshot related bug."). However, that takes the start position of the buffered write to compare against the current EOF, hole ex

Re: [PATCH 1/7] Btrfs: create a helper for getting chunk map

2017-02-28 Thread Liu Bo
On Mon, Feb 20, 2017 at 11:20:33AM +0800, Qu Wenruo wrote: > > > At 02/18/2017 09:28 AM, Liu Bo wrote: > > We have similar code here and there, this merges them into a helper. > > > > Signed-off-by: Liu Bo > > Looks good overall. > > Although small nitpick inlined below. Thank you for going

[PATCH 2/2] Btrfs: remove start_pos

2017-02-28 Thread Liu Bo
@pos, not aligned @start_pos, should be used to check whether the eof page needs to be marked as readonly, thus @start_pos can be removed. Signed-off-by: Liu Bo --- fs/btrfs/file.c | 7 +-- 1 file changed, 1 insertion(+), 6 deletions(-) diff --git a/fs/btrfs/file.c b/fs/btrfs/file.c index 0

Re: assertion failed: last_size == new_size, file: fs/btrfs/inode.c

2017-02-28 Thread Liu Bo
On Mon, Feb 27, 2017 at 11:23:42AM -0500, Dave Jones wrote: > On Mon, Feb 27, 2017 at 07:53:48AM -0800, Liu Bo wrote: > > On Sun, Feb 26, 2017 at 07:18:42PM -0500, Dave Jones wrote: > > > Hitting this fairly frequently.. I'm not sure if this is the same bug > I've > > > been hitting occasionall

[PATCH 2/2] Btrfs: remove start_pos

2017-02-28 Thread Liu Bo
@pos, not aligned @start_pos, should be used to check whether the eof page needs to be marked as readonly, thus @start_pos can be removed. Signed-off-by: Liu Bo --- fs/btrfs/file.c | 7 +-- 1 file changed, 1 insertion(+), 6 deletions(-) diff --git a/fs/btrfs/file.c b/fs/btrfs/file.c index 0

[PATCH 1/2] Btrfs: fix unexpected file hole after disk errors

2017-02-28 Thread Liu Bo
Btrfs creates hole extents to cover any unwritten section right before doing buffer writes after commit 3ac0d7b96a26 ("btrfs: Change the expanding write sequence to fix snapshot related bug."). However, that takes the start position of the buffered write to compare against the current EOF, hole ex

[PATCH 8/8] nowait aio: btrfs

2017-02-28 Thread Goldwyn Rodrigues
From: Goldwyn Rodrigues Return EAGAIN if any of the following checks fail + i_rwsem is not lockable + NODATACOW or PREALLOC is not set + Cannot nocow at the desired location + Writing beyond end of file which is not allocated Signed-off-by: Goldwyn Rodrigues --- fs/btrfs/file.c | 25 +

[PATCH 4/8] nowait aio: Introduce IOMAP_NOWAIT

2017-02-28 Thread Goldwyn Rodrigues
From: Goldwyn Rodrigues IOCB_NOWAIT translates to IOMAP_NOWAIT for iomaps. This is used by XFS in the XFS patch. Signed-off-by: Goldwyn Rodrigues --- fs/iomap.c| 2 ++ include/linux/iomap.h | 1 + 2 files changed, 3 insertions(+) diff --git a/fs/iomap.c b/fs/iomap.c index a51cb4c.

[PATCH 3/8] nowait aio: return if direct write will trigger writeback

2017-02-28 Thread Goldwyn Rodrigues
From: Goldwyn Rodrigues Find out if the write will trigger a wait due to writeback. If yes, return -EAGAIN. This introduces a new function filemap_range_has_page() which returns true if the file's mapping has a page within the range mentioned. Return -EINVAL for buffered AIO: there are multiple

[PATCH 7/8] nowait aio: xfs

2017-02-28 Thread Goldwyn Rodrigues
From: Goldwyn Rodrigues If IOCB_NOWAIT is set, bail if the i_rwsem is not lockable immediately. IF IOMAP_NOWAIT is set, return EAGAIN in xfs_file_iomap_begin if it needs allocation either due to file extending, writing to a hole, or COW. Signed-off-by: Goldwyn Rodrigues --- fs/xfs/xfs_file.c

[PATCH 0/8 v2] Non-blocking AIO

2017-02-28 Thread Goldwyn Rodrigues
This series adds nonblocking feature to asynchronous I/O writes. io_submit() can be delayed because of a number of reason: - Block allocation for files - Data writebacks for direct I/O - Sleeping because of waiting to acquire i_rwsem - Congested block device The goal of the patch series is to

[PATCH 2/8] nowait aio: Return if cannot get hold of i_rwsem

2017-02-28 Thread Goldwyn Rodrigues
From: Goldwyn Rodrigues A failure to lock i_rwsem would mean there is I/O being performed by another thread. So, let's bail. Signed-off-by: Goldwyn Rodrigues --- mm/filemap.c | 7 ++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/mm/filemap.c b/mm/filemap.c index 3f9afde..78

[PATCH 5/8] nowait aio: return on congested block device

2017-02-28 Thread Goldwyn Rodrigues
From: Goldwyn Rodrigues A new flag BIO_NOWAIT is introduced to identify bio's orignating from iocb with IOCB_NOWAIT. This flag indicates to return immediately if a request cannot be made instead of retrying. Signed-off-by: Goldwyn Rodrigues --- block/blk-core.c | 13 +++-- fs/

[PATCH 6/8] nowait aio: ext4

2017-02-28 Thread Goldwyn Rodrigues
From: Goldwyn Rodrigues Return EAGAIN if any of the following checks fail for direct I/O: + i_rwsem is lockable + Writing beyond end of file (will trigger allocation) + Blocks are not allocated at the write location Signed-off-by: Goldwyn Rodrigues --- fs/ext4/file.c | 53 ++

[PATCH 1/8] nowait aio: Introduce IOCB_FLAG_NOWAIT

2017-02-28 Thread Goldwyn Rodrigues
From: Goldwyn Rodrigues This flag informs kernel to bail out if an AIO request will block for reasons such as file allocations, or a writeback triggered, or would block while allocating requests while performing direct I/O. IOCB_FLAG_NOWAIT is translated to IOCB_NOWAIT for iocb->ki_flags. Signe

Re: [PULL] Btrfs cleanups for 4.11, part 2

2017-02-28 Thread Chris Mason
On 02/28/2017 10:09 AM, David Sterba wrote: Hi, this is the second half of the 4.11 batch, the rest of the cleanups. Please pull, thanks. The following changes since commit 6288d6eabc7505f42dda34a2c2962f91914be3a4: Btrfs: use the correct type when creating cow dio extent (2017-02-22 15:55:

[PATCH v3] btrfs: remove btrfs_err_str function from uapi/linux/btrfs.h

2017-02-28 Thread Dmitry V. Levin
btrfs_err_str function is not called from anywhere and is replicated in the userspace headers for btrfs-progs. It's removal also fixes the following linux/btrfs.h userspace compilation error: /usr/include/linux/btrfs.h: In function 'btrfs_err_str': /usr/include/linux/btrfs.h:740:11: error: 'NULL'

Re: [PATCH 3/4] reflink: test adjacency of reflinked blocks

2017-02-28 Thread Darrick J. Wong
On Tue, Feb 28, 2017 at 04:15:02PM +0800, Eryu Guan wrote: > On Fri, Feb 24, 2017 at 05:12:57PM -0800, Darrick J. Wong wrote: > > From: Darrick J. Wong > > > > If we reflink a file with N blocks to another file one block at a time, > > does the destination file end up with the same number of exte

[PULL] Btrfs cleanups for 4.11, part 2

2017-02-28 Thread David Sterba
Hi, this is the second half of the 4.11 batch, the rest of the cleanups. Please pull, thanks. The following changes since commit 6288d6eabc7505f42dda34a2c2962f91914be3a4: Btrfs: use the correct type when creating cow dio extent (2017-02-22 15:55:03 -0800) are available in the git repository

Re: [PATCH 3/3] btrfs: do proper error handling in btrfs_insert_xattr_item

2017-02-28 Thread David Sterba
On Tue, Feb 21, 2017 at 09:39:05PM -0800, Liu Bo wrote: > On Mon, Feb 20, 2017 at 07:25:06PM +0100, David Sterba wrote: > > The space check in btrfs_insert_xattr_item is duplicated in it's caller > > (do_setxattr) so we won't hit the BUG_ON. Continuing without any check > > could be disasterous so

Re: [PATCH 3/4] reflink: test adjacency of reflinked blocks

2017-02-28 Thread Eryu Guan
On Fri, Feb 24, 2017 at 05:12:57PM -0800, Darrick J. Wong wrote: > From: Darrick J. Wong > > If we reflink a file with N blocks to another file one block at a time, > does the destination file end up with the same number of extents as the > source file? In other words, does the filesystem succee