Re: [PATCH 9/9] btrfs: add explicit check for replace result no error

2018-11-07 Thread Anand Jain
On 11/07/2018 08:15 PM, Nikolay Borisov wrote: On 7.11.18 г. 13:43 ч., Anand Jain wrote: We recast the replace return status BTRFS_IOCTL_DEV_REPLACE_RESULT_SCRUB_INPROGRESS to 0, to indicate no error. And since BTRFS_IOCTL_DEV_REPLACE_RESULT_NO_ERROR should also return 0, which is also

Re: [PATCH -next] btrfs: compression: remove set but not used variable 'tree'

2018-11-07 Thread Nikolay Borisov
On 8.11.18 г. 4:15 ч., YueHaibing wrote: > Fixes gcc '-Wunused-but-set-variable' warning: > > fs/btrfs/compression.c: In function 'end_compressed_bio_write': > fs/btrfs/compression.c:232:25: warning: > variable 'tree' set but not used [-Wunused-but-set-variable] > > It not used any more

Re: [PATCH -next] btrfs: remove set but not used variable 'tree'

2018-11-07 Thread Nikolay Borisov
On 8.11.18 г. 4:14 ч., YueHaibing wrote: > Fixes gcc '-Wunused-but-set-variable' warning: > > fs/btrfs/extent_io.c: In function 'end_extent_writepage': > fs/btrfs/extent_io.c:2406:25: warning: > variable 'tree' set but not used [-Wunused-but-set-variable] > > It not used any more after >

[PATCH v2 5/6] btrfs: qgroup: Use delayed subtree rescan for balance

2018-11-07 Thread Qu Wenruo
Before this patch, qgroup code trace the whole subtree of file and reloc trees unconditionally. This makes qgroup numbers consistent, but it could cause tons of unnecessary extent trace, which cause a lot of overhead. However for subtree swap of balance, since both subtree contains the same

[PATCH v2 2/6] btrfs: relocation: Delay reloc tree deletion after merge_reloc_roots()

2018-11-07 Thread Qu Wenruo
Relocation code will drop btrfs_root::reloc_root as soon as merge_reloc_root() finishes. However later qgroup code will need to access btrfs_root::reloc_root after merge_reloc_root() for delayed subtree rescan. So alter the timming of resetting btrfs_root:::reloc_root, make it happens after

[PATCH v2 0/6] btrfs: qgroup: Delay subtree scan to reduce overhead

2018-11-07 Thread Qu Wenruo
This patchset can be fetched from github: https://github.com/adam900710/linux/tree/qgroup_delayed_subtree_rebased Which is based on v4.20-rc1. This patch address the heavy load subtree scan, but delaying it until we're going to modify the swapped tree block. The overall workflow is: 1) Record

[PATCH v2 6/6] btrfs: qgroup: Cleanup old subtree swap code

2018-11-07 Thread Qu Wenruo
Since it's replaced by new delayed subtree swap code, remove the original code. The cleanup is small since most of its core function is still used by delayed subtree swap trace. Signed-off-by: Qu Wenruo --- fs/btrfs/qgroup.c | 94 ---

[PATCH v2 3/6] btrfs: qgroup: Refactor btrfs_qgroup_trace_subtree_swap()

2018-11-07 Thread Qu Wenruo
Refactor btrfs_qgroup_trace_subtree_swap() into qgroup_trace_subtree_swap(), which only needs two extent buffer and some other bool to control the behavior. Also, allow depending functions to accept parameter @exec_post to determine whether we need to trigger backref walk. This provides the

[PATCH v2 4/6] btrfs: qgroup: Introduce per-root swapped blocks infrastructure

2018-11-07 Thread Qu Wenruo
To allow delayed subtree swap rescan, btrfs needs to record per-root info about which tree blocks get swapped. So this patch introduces per-root btrfs_qgroup_swapped_blocks structure, which records which tree blocks get swapped. The designed workflow will be: 1) Record the subtree root block

[PATCH v2 1/6] btrfs: qgroup: Allow btrfs_qgroup_extent_record::old_roots unpopulated at insert time

2018-11-07 Thread Qu Wenruo
Commit fb235dc06fac ("btrfs: qgroup: Move half of the qgroup accounting time out of commit trans") makes btrfs_qgroup_extent_record::old_roots populated at insert time. It's OK for most cases as btrfs_qgroup_extent_record is inserted at delayed ref head insert time, which has a less restrict lock

[PATCH -next] btrfs: compression: remove set but not used variable 'tree'

2018-11-07 Thread YueHaibing
Fixes gcc '-Wunused-but-set-variable' warning: fs/btrfs/compression.c: In function 'end_compressed_bio_write': fs/btrfs/compression.c:232:25: warning: variable 'tree' set but not used [-Wunused-but-set-variable] It not used any more after commit 2922040236f9 ("btrfs: Remove

[PATCH -next] btrfs: remove set but not used variable 'tree'

2018-11-07 Thread YueHaibing
Fixes gcc '-Wunused-but-set-variable' warning: fs/btrfs/extent_io.c: In function 'end_extent_writepage': fs/btrfs/extent_io.c:2406:25: warning: variable 'tree' set but not used [-Wunused-but-set-variable] It not used any more after commit 2922040236f9 ("btrfs: Remove

Re: [PATCH] Btrfs: fix data corruption due to cloning of eof block

2018-11-07 Thread Josef Bacik
On Mon, Nov 05, 2018 at 11:14:17AM +, fdman...@kernel.org wrote: > From: Filipe Manana > > We currently allow cloning a range from a file which includes the last > block of the file even if the file's size is not aligned to the block > size. This is fine and useful when the destination file

Re: [PATCH v2] Btrfs: fix missing delayed iputs on unmount

2018-11-07 Thread David Sterba
On Wed, Oct 31, 2018 at 10:06:08AM -0700, Omar Sandoval wrote: > From: Omar Sandoval > > There's a race between close_ctree() and cleaner_kthread(). > close_ctree() sets btrfs_fs_closing(), and the cleaner stops when it > sees it set, but this is racy; the cleaner might have already checked >

Re: [PATCH v9 0/6] Btrfs: implement swap file support

2018-11-07 Thread David Sterba
On Wed, Nov 07, 2018 at 05:07:00PM +0200, Nikolay Borisov wrote: > > > On 7.11.18 г. 16:49 ч., David Sterba wrote: > > On Tue, Nov 06, 2018 at 10:54:51AM +0100, David Sterba wrote: > >> On Thu, Sep 27, 2018 at 11:17:32AM -0700, Omar Sandoval wrote: > >>> From: Omar Sandoval > >>> This series

Re: [PATCH v9 0/6] Btrfs: implement swap file support

2018-11-07 Thread Nikolay Borisov
On 7.11.18 г. 16:49 ч., David Sterba wrote: > On Tue, Nov 06, 2018 at 10:54:51AM +0100, David Sterba wrote: >> On Thu, Sep 27, 2018 at 11:17:32AM -0700, Omar Sandoval wrote: >>> From: Omar Sandoval >>> This series implements swap file support for Btrfs. >>> >>> Changes from v8 [1]: >>> >>> -

Re: [PATCH v9 0/6] Btrfs: implement swap file support

2018-11-07 Thread David Sterba
On Tue, Nov 06, 2018 at 10:54:51AM +0100, David Sterba wrote: > On Thu, Sep 27, 2018 at 11:17:32AM -0700, Omar Sandoval wrote: > > From: Omar Sandoval > > This series implements swap file support for Btrfs. > > > > Changes from v8 [1]: > > > > - Fixed a bug in btrfs_swap_activate() which would

Re: [PATCH 2/9] btrfs: replace go back to suspended if target missing

2018-11-07 Thread Nikolay Borisov
On 7.11.18 г. 13:43 ч., Anand Jain wrote: > At the time of forced unmount we place the running replace to > BTRFS_IOCTL_DEV_REPLACE_STATE_SUSPENDED state, so when the system comes > back and suppose the target device is missing, then let the replace > state continue to be in

Re: [PATCH 5/9] btrfs: replace cancel is successful if scrub cancel is successful

2018-11-07 Thread Nikolay Borisov
On 7.11.18 г. 14:25 ч., Nikolay Borisov wrote: > > > On 7.11.18 г. 13:43 ч., Anand Jain wrote: >> In btrfs_dev_replace_cancel() we should check if the >> btrfs_scrub_cancel() is successful. If the btrfs_scrub_cancel() fails, return >> BTRFS_IOCTL_DEV_REPLACE_RESULT_NOT_STARTED so that user

Re: [PATCH 5/9] btrfs: replace cancel is successful if scrub cancel is successful

2018-11-07 Thread Nikolay Borisov
On 7.11.18 г. 13:43 ч., Anand Jain wrote: > In btrfs_dev_replace_cancel() we should check if the > btrfs_scrub_cancel() is successful. If the btrfs_scrub_cancel() fails, return > BTRFS_IOCTL_DEV_REPLACE_RESULT_NOT_STARTED so that user can try to > cancel the replace again. > > Signed-off-by:

Re: [PATCH 6/9] btrfs: replace's scrub must not be running in replace suspended state

2018-11-07 Thread Nikolay Borisov
On 7.11.18 г. 13:43 ч., Anand Jain wrote: > + /* scrub for replace must not be running in suspended state */ > + if (btrfs_scrub_cancel(fs_info) != -ENOTCONN) > + ASSERT(0); ASSERT(btrfs_scrub_cancel(fs_info) == -ENOTCONN)

Re: [PATCH 7/9] btrfs: quiten warn if the replace is canceled at finish

2018-11-07 Thread Nikolay Borisov
On 7.11.18 г. 13:43 ч., Anand Jain wrote: > - WARN_ON(ret); > + if (ret != -ECANCELED) > + WARN_ON(ret); WARN_ON(ret && ret != -ECANCELED)

Re: [PATCH 9/9] btrfs: add explicit check for replace result no error

2018-11-07 Thread Nikolay Borisov
On 7.11.18 г. 13:43 ч., Anand Jain wrote: > We recast the replace return status > BTRFS_IOCTL_DEV_REPLACE_RESULT_SCRUB_INPROGRESS to 0, to indicate no > error. > And since BTRFS_IOCTL_DEV_REPLACE_RESULT_NO_ERROR should also return 0, > which is also declared as 0, so we just return. Instead add

Re: [PATCH 1/9] btrfs: mark btrfs_dev_replace_start() as static

2018-11-07 Thread Nikolay Borisov
On 7.11.18 г. 13:43 ч., Anand Jain wrote: > There isn't any other consumer other than in its own file dev-replace.c. > > Signed-off-by: Anand Jain Reviewed-by: Nikolay Borisov > --- > fs/btrfs/dev-replace.c | 2 +- > fs/btrfs/dev-replace.h | 3 --- > 2 files changed, 1 insertion(+), 4

[PATCH 7/9] btrfs: quiten warn if the replace is canceled at finish

2018-11-07 Thread Anand Jain
When we successfully cancel the replace its scrub returns -ECANCELED, which then passed to btrfs_dev_replace_finishing(), it cleans up based on the scrub returned status and propagates the same -ECANCELED back the parent function. So skip the -ECANCELED error to log the WARN. Signed-off-by: Anand

[PATCH 8/9] btrfs: user requsted replace cancel is not an error

2018-11-07 Thread Anand Jain
As of now only user requested replace cancel can cancel the replace-scrub so no need to log error for it. Signed-off-by: Anand Jain --- fs/btrfs/dev-replace.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/fs/btrfs/dev-replace.c b/fs/btrfs/dev-replace.c index

[PATCH 6/9] btrfs: replace's scrub must not be running in replace suspended state

2018-11-07 Thread Anand Jain
When the replace state is placed in the suspended state, btrfs_scrub_cancel() should fail with -ENOTCONN as there is no scrub running, as a safety catch check if btrfs_scrub_cancel() returns -ENOTCONN and assert if it doesn't. Signed-off-by: Anand Jain --- fs/btrfs/dev-replace.c | 4 +++- 1

[PATCH 9/9] btrfs: add explicit check for replace result no error

2018-11-07 Thread Anand Jain
We recast the replace return status BTRFS_IOCTL_DEV_REPLACE_RESULT_SCRUB_INPROGRESS to 0, to indicate no error. And since BTRFS_IOCTL_DEV_REPLACE_RESULT_NO_ERROR should also return 0, which is also declared as 0, so we just return. Instead add it to the if statement so that there is enough clarity

[PATCH 0/9] fix replace-start and replace-cancel racing

2018-11-07 Thread Anand Jain
Replace-start and replace-cancel threads can race to create a messy situation leading to UAF. We use the scrub code to write the blocks on the replace target. So if we haven't have set the replace-scrub-running yet, without this patch we just ignore the error and free the target device. When this

[PATCH 5/9] btrfs: replace cancel is successful if scrub cancel is successful

2018-11-07 Thread Anand Jain
In btrfs_dev_replace_cancel() we should check if the btrfs_scrub_cancel() is successful. If the btrfs_scrub_cancel() fails, return BTRFS_IOCTL_DEV_REPLACE_RESULT_NOT_STARTED so that user can try to cancel the replace again. Signed-off-by: Anand Jain --- fs/btrfs/dev-replace.c | 22

[PATCH 2/9] btrfs: replace go back to suspended if target missing

2018-11-07 Thread Anand Jain
At the time of forced unmount we place the running replace to BTRFS_IOCTL_DEV_REPLACE_STATE_SUSPENDED state, so when the system comes back and suppose the target device is missing, then let the replace state continue to be in BTRFS_IOCTL_DEV_REPLACE_STATE_SUSPENDED state instead of

[PATCH 1/9] btrfs: mark btrfs_dev_replace_start() as static

2018-11-07 Thread Anand Jain
There isn't any other consumer other than in its own file dev-replace.c. Signed-off-by: Anand Jain --- fs/btrfs/dev-replace.c | 2 +- fs/btrfs/dev-replace.h | 3 --- 2 files changed, 1 insertion(+), 4 deletions(-) diff --git a/fs/btrfs/dev-replace.c b/fs/btrfs/dev-replace.c index

[PATCH 4/9] btrfs: fix UAF due to race between replace start and cancel

2018-11-07 Thread Anand Jain
replace cancel thread can race with the replace start thread and if fs_info::scrubs_running is not yet set the btrfs_scrub_cancel() will fail to stop the scrub thread, so the scrub thread continue with the scrub for replace which then shall try to write to the target device and which is already

[PATCH 3/9] btrfs: replace back to suspend state if EXCL OP is running

2018-11-07 Thread Anand Jain
In a secnario where balance and replace co-exists as below, start balance; pause balance; start replace; reboot and when system restarts balance restarts first and the replace restart will fail as EXCL OP lock is already held by the balance. If so place the replace state back to

Re: [PATCH 09/13] btrfs-progs: Revert "btrfs-progs: Record orphan data extent ref to corresponding root."

2018-11-07 Thread Qu Wenruo
On 2018/11/7 下午5:09, Su Yanjun wrote: > > > On 10/24/2018 8:29 AM, Qu Wenruo wrote: >> >> On 2018/10/23 下午5:41, Su Yue wrote: >>> From: Su Yanjun >>> >>> The reason for revert is that according to the existing situation, the >>> probability of problem in the extent tree is higher than in

Re: [PATCH 09/13] btrfs-progs: Revert "btrfs-progs: Record orphan data extent ref to corresponding root."

2018-11-07 Thread Su Yanjun
On 10/24/2018 8:29 AM, Qu Wenruo wrote: On 2018/10/23 下午5:41, Su Yue wrote: From: Su Yanjun The reason for revert is that according to the existing situation, the probability of problem in the extent tree is higher than in the fs Tree. So this feature should be removed. The same problem