[PATCH RESEND v12] btrfs-progs: add cli to forget one or all scanned devices

2018-11-30 Thread Anand Jain
This patch adds cli btrfs device forget [dev] to remove the given device structure in the kernel if the device is unmounted. If no argument is given it shall remove all stale (device which are not mounted) from the kernel. Signed-off-by: Anand Jain Reviewed-by: Nikolay Borisov --- v11->

[PATCH RESEND] fstests: btrfs use forget if not reload

2018-11-30 Thread Anand Jain
to forget one or all scanned devices btrfs: introduce feature to forget a btrfs device Signed-off-by: Anand Jain --- common/btrfs| 20 tests/btrfs/124 | 6 +++--- tests/btrfs/125 | 6 +++--- tests/btrfs/154 | 6 +++--- tests/btrfs/164 | 4 ++-- 5 files changed, 31

[PATCH RESEND v12] Add cli and ioctl to forget scanned device(s)

2018-11-30 Thread Anand Jain
: refactor btrfs_free_stale_device() to get device list delete to [PATCH 1/2] btrfs: add function to device list delete Adds cli and ioctl to forget a scanned device or forget all stale devices in the kernel. Anand Jain (1): btrfs: introduce feature to forget a btrfs device fs/btr

[PATCH RESEND] fstests: btrfs use forget if not reload

2018-11-30 Thread Anand Jain
to forget one or all scanned devices btrfs: introduce feature to forget a btrfs device Signed-off-by: Anand Jain --- common/btrfs| 20 tests/btrfs/124 | 6 +++--- tests/btrfs/125 | 6 +++--- tests/btrfs/154 | 6 +++--- tests/btrfs/164 | 4 ++-- 5 files changed, 31

[PATCH RESEND v12] btrfs: introduce feature to forget a btrfs device

2018-11-30 Thread Anand Jain
like in split brain raid1. . Running test cases which requires btrfs.ko-reload if the rootfs is btrfs. Signed-off-by: Anand Jain Reviewed-by: Nikolay Borisov --- v11->v12: fix coding style add spacing before after ":". v1->v11: Pls ref to the cover-letter. (sorry about t

Re: [PATCH 2/2] btrfs: scrub: fix circular locking dependency warning

2018-11-29 Thread Anand Jain
On 11/30/2018 09:05 AM, Anand Jain wrote: On 11/29/2018 10:31 PM, David Sterba wrote: On Wed, Nov 28, 2018 at 04:47:27PM +0800, Anand Jain wrote: 2. scrub_workers_refcnt must eventually be converted to refcount_t type    ok. Added in v2 patch set. No such thing is in v2 and this would

[PATCH v3 0/2] btrfs: scrub: fix scrub_lock

2018-11-29 Thread Anand Jain
v3: Drops the patch [1]from this set. [1] btrfs: scrub: maintain the unlock order in scrub thread Fixes the circular locking dependency warning as in patch 1/2, and patch 2/2 adds lockdep_assert_held() to scrub_workers_get(). Anand Jain (2): btrfs: scrub: fix circular locking

[PATCH v3 1/2] btrfs: scrub: fix circular locking dependency warning

2018-11-29 Thread Anand Jain
0000801000 R14: R15: 7f61d3937700 [ 76.175217] btrfs (4065) used greatest stack depth: 11424 bytes left Signed-off-by: Anand Jain --- v2->v3: none v1->v2: none fs/btrfs/scrub.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/fs/btrfs/scrub.c b/fs/btrfs

[PATCH v3 2/2] btrfs: scrub: add scrub_lock lockdep check in scrub_workers_get

2018-11-29 Thread Anand Jain
scrub_workers_refcnt is protected by scrub_lock, add lockdep_assert_held() function in scrub_workers_get(). Signed-off-by: Anand Jain Suggested-by: Nikolay Borisov --- v3: none v2: born fs/btrfs/scrub.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/fs/btrfs/scrub.c b/fs/btrfs/scrub.c

Re: [PATCH 2/2] btrfs: scrub: fix circular locking dependency warning

2018-11-29 Thread Anand Jain
On 11/29/2018 10:31 PM, David Sterba wrote: On Wed, Nov 28, 2018 at 04:47:27PM +0800, Anand Jain wrote: 2. scrub_workers_refcnt must eventually be converted to refcount_t type ok. Added in v2 patch set. No such thing is in v2 and this would actually get rid of the need to hold

Re: [PATCH v2 1/3] btrfs: scrub: maintain the unlock order in scrub thread

2018-11-29 Thread Anand Jain
On 11/29/2018 06:36 PM, Filipe Manana wrote: On Thu, Nov 29, 2018 at 9:27 AM Anand Jain wrote: The device_list_mutex and scrub_lock creates a nested locks in btrfs_scrub_dev(). During lock the order is device_list_mutex and then scrub_lock, and during unlock, the order

[PATCH v2 2/3] btrfs: scrub: fix circular locking dependency warning

2018-11-29 Thread Anand Jain
0000801000 R14: R15: 7f61d3937700 [ 76.175217] btrfs (4065) used greatest stack depth: 11424 bytes left Signed-off-by: Anand Jain --- v1->v2: none fs/btrfs/scrub.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/fs/btrfs/scrub.c b/fs/btrfs/scrub.c index b5a19b

[PATCH v2 1/3] btrfs: scrub: maintain the unlock order in scrub thread

2018-11-29 Thread Anand Jain
-by: Anand Jain --- v1->v2: change the order of lock acquire first scrub_lock and then device_list_mutex, which matches with the order of unlock. The extra line which are now in the scrub_lock are ok to be under the scrub_lock. fs/btrfs/scrub.c | 13 +++-- 1 f

[PATCH v2 0/3] btrfs: scrub: fix scrub_lock

2018-11-29 Thread Anand Jain
Idea was to fix the circular locking dependency warning as in patch 2/3, and in the process also fixes the other identified cleanups patches 1/3,3/3 and they aren't dependent on 2ttch /3. Anand Jain (3): btrfs: scrub: maintain the unlock order in scrub thread btrfs: scrub: fix circular

[PATCH v2 3/3] btrfs: scrub: add scrub_lock lockdep check in scrub_workers_get

2018-11-29 Thread Anand Jain
scrub_workers_refcnt is protected by scrub_lock, add lockdep_assert_held() function in scrub_workers_get(). Signed-off-by: Anand Jain Suggested-by: Nikolay Borisov --- v2: born fs/btrfs/scrub.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/fs/btrfs/scrub.c b/fs/btrfs/scrub.c index

Re: [PATCH 2/2] btrfs: scrub: fix circular locking dependency warning

2018-11-28 Thread Anand Jain
On 11/26/2018 05:59 PM, Nikolay Borisov wrote: On 26.11.18 г. 11:07 ч., Anand Jain wrote: Circular locking dependency check reports warning[1], that's because the btrfs_scrub_dev() calls the stack #0 below with, the fs_info::scrub_lock held. The test case leading to this warning

Re: [PATCH 1/2] btrfs: scrub: maintain the unlock order in scrub thread

2018-11-27 Thread Anand Jain
On 11/26/2018 05:47 PM, Nikolay Borisov wrote: On 26.11.18 г. 11:07 ч., Anand Jain wrote: The fs_info::device_list_mutex and fs_info::scrub_lock creates a nested locks in btrfs_scrub_dev(). During the lock acquire the hierarchy is fs_info::device_list_mutex and then fs_info::scrub_lock, so

[PATCH 2/2] btrfs: scrub: fix circular locking dependency warning

2018-11-26 Thread Anand Jain
0000801000 R14: R15: 7f61d3937700 [ 76.175217] btrfs (4065) used greatest stack depth: 11424 bytes left Signed-off-by: Anand Jain --- fs/btrfs/scrub.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/fs/btrfs/scrub.c b/fs/btrfs/scrub.c index b1c2d1cdbd4b..3f

[PATCH 1/2] btrfs: scrub: maintain the unlock order in scrub thread

2018-11-26 Thread Anand Jain
::device_list_mutex. Signed-off-by: Anand Jain --- fs/btrfs/scrub.c | 16 +++- 1 file changed, 7 insertions(+), 9 deletions(-) diff --git a/fs/btrfs/scrub.c b/fs/btrfs/scrub.c index 902819d3cf41..b1c2d1cdbd4b 100644 --- a/fs/btrfs/scrub.c +++ b/fs/btrfs/scrub.c @@ -3865,7 +3865,6 @@ int

[PATCH 0/2] btrfs: scrub: fix scrub_lock

2018-11-26 Thread Anand Jain
Anand Jain (2): btrfs: scrub: maintain the unlock order in scrub thread btrfs: scrub: fix circular locking dependency warning fs/btrfs/scrub.c | 19 ++- 1 file changed, 10 insertions(+), 9 deletions(-) -- 1.8.3.1

[PATCH v12] Add cli and ioctl to forget scanned device(s)

2018-11-21 Thread Anand Jain
: refactor btrfs_free_stale_device() to get device list delete to [PATCH 1/2] btrfs: add function to device list delete Adds cli and ioctl to forget a scanned device or forget all stale devices in the kernel. Anand Jain (1): btrfs: introduce feature to forget a btrfs device fs/btr

[PATCH] fstests: btrfs use forget if not reload

2018-11-21 Thread Anand Jain
to forget one or all scanned devices btrfs: introduce feature to forget a btrfs device Signed-off-by: Anand Jain --- common/btrfs| 20 tests/btrfs/124 | 6 +++--- tests/btrfs/125 | 6 +++--- tests/btrfs/154 | 6 +++--- tests/btrfs/164 | 4 ++-- 5 files changed, 31

[PATCH v12] btrfs: introduce feature to forget a btrfs device

2018-11-21 Thread Anand Jain
like in split brain raid1. . Running test cases which requires btrfs.ko-reload if the rootfs is btrfs. Signed-off-by: Anand Jain Reviewed-by: Nikolay Borisov --- v11->v12: fix coding style add spacing before after ":". v1->v11: Pls ref to the cover-letter. (sorry about t

[PATCH v12] btrfs-progs: add cli to forget one or all scanned devices

2018-11-21 Thread Anand Jain
This patch adds cli btrfs device forget [dev] to remove the given device structure in the kernel if the device is unmounted. If no argument is given it shall remove all stale (device which are not mounted) from the kernel. Signed-off-by: Anand Jain Reviewed-by: Nikolay Borisov --- v11->

Re: [RFC] BTRFS_DEV_REPLACE_ITEM_STATE_* doesn't match with on disk

2018-11-20 Thread Anand Jain
David, any comments on this please. Thanks, Anand On 11/13/2018 06:32 PM, Anand Jain wrote: David, Gentle ping. Thanks, Anand On 11/12/2018 03:50 PM, Nikolay Borisov wrote: On 12.11.18 г. 6:58 ч., Anand Jain wrote: The dev_replace_state defines are miss matched between

[PATCH RESEND v2 1/2] btrfs: quieten warn if replace is canceled at finish

2018-11-20 Thread Anand Jain
to quieten the warn here. Signed-off-by: Anand Jain --- [fix: quieten spelling] v1->v2: Use the condition within the WARN_ON() fs/btrfs/dev-replace.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/fs/btrfs/dev-replace.c b/fs/btrfs/dev-replace.c index 1dc8e86546db..9031a3629

[PATCH RESEND v2 2/2] btrfs: user requsted replace cancel is not an error

2018-11-20 Thread Anand Jain
As of now only user requested replace cancel can cancel the replace-scrub so no need to log error for it. Signed-off-by: Anand Jain --- v1->v2: none. fs/btrfs/dev-replace.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/fs/btrfs/dev-replace.c b/fs/btrfs/dev-replac

[PATCH RESEND 0/9 v2] fix warn_on for replace cancel

2018-11-20 Thread Anand Jain
between replace start and cancel Anand Jain (2): btrfs: quieten warn if the replace is canceled at finish btrfs: user requsted replace cancel is not an error

[PATCH v6 1/3] btrfs: add helper function describe_block_group()

2018-11-20 Thread Anand Jain
Improve on describe_relocation() add a common helper function to describe the block groups. Signed-off-by: Anand Jain Reviewed-by: David Sterba --- v5->v6: Use () in the body for the args sent in defines Use right indent to align '\' Use goto to out_overflow instead of ret

[PATCH v6 0/3] btrfs: balance: improve kernel logs

2018-11-20 Thread Anand Jain
patchs make balance logs easy to review. Anand Jain (3): btrfs: add helper function describe_block_group() btrfs: balance: add args info during start and resume btrfs: balance: add kernel log for end or paused fs/btrfs/relocation.c | 30 +-- fs/btrfs/volume

[PATCH v6 3/3] btrfs: balance: add kernel log for end or paused

2018-11-20 Thread Anand Jain
Add a kernel log when the balance ends, either for cancel or completed or if it is paused. Signed-off-by: Anand Jain --- v5->v6: Quite soul. nothing. v4->v5: nothing. v3->v4: nothing. v2->v3: nothing. v1->v2: Moved from 2/3 to 3/3 fs/btrfs/volumes.c | 7 +++ 1 file change

[PATCH v6 2/3] btrfs: balance: add args info during start and resume

2018-11-20 Thread Anand Jain
=10..20 -msoft,profiles=raid1,convert=single -ssoft,profiles=raid1,convert=single Signed-off-by: Anand Jain --- v5->v6: Use () in the body for the args sent in defines Use right indent to align '\' Use goto to out_overflow instead of return (also fixes a mem leak in v5)

Re: [PATCH v5 2/3] btrfs: balance: add args info during start and resume

2018-11-20 Thread Anand Jain
On 11/20/2018 01:07 AM, David Sterba wrote: On Wed, Nov 14, 2018 at 09:17:11PM +0800, Anand Jain wrote: Balance arg info is an important information to be reviewed for the system audit. So this patch adds them to the kernel log. Example: ->btrfs bal start -f -mprofiles=raid1,convert=sin

Re: [PATCH v5 1/3] btrfs: add helper function describe_block_group()

2018-11-20 Thread Anand Jain
Thanks for the review.. more below. On 11/20/2018 01:02 AM, David Sterba wrote: On Wed, Nov 14, 2018 at 09:17:10PM +0800, Anand Jain wrote: Improve on describe_relocation() add a common helper function to describe the block groups. Signed-off-by: Anand Jain Reviewed-by: David Sterba

Re: bad tree block start, want 705757184 have 82362368

2018-11-18 Thread Anand Jain
On 11/18/2018 03:56 PM, Stephan Olbrich wrote: Am Sonntag, 18. November 2018, 01:30:14 CET schrieb Qu Wenruo: Late on I got the same errors for my /home partition (on the same drive) as well. I have snapshots of all partitions on another drive made by btrbk. To get a working system, I made

Re: [PATCH 2/5] btrfs: Refactor btrfs_can_relocate

2018-11-16 Thread Anand Jain
On 10/26/2018 07:43 PM, Nikolay Borisov wrote: btrfs_can_relocate returns 0 when it concludes the given chunk can be relocated and -1 otherwise. Since this function is used as a predicated and it return a binary value it makes no sense to have it's return value as an int so change it to bool.

Re: [PATCH 5/5] btrfs: Replace BUG_ON with ASSERT in find_lock_delalloc_range

2018-11-16 Thread Anand Jain
is where lock_delalloc_pages gets its return value. The latter always returns 0 or -EAGAIN so the invariant holds. No functional changes. Signed-off-by: Nikolay Borisov Reviewed-by: Anand Jain Thanks, Anand --- fs/btrfs/extent_io.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff

Re: [PATCH 4/5] btrfs: Sink find_lock_delalloc_range's 'max_bytes' argument

2018-11-16 Thread Anand Jain
On 10/26/2018 07:43 PM, Nikolay Borisov wrote: All callers of this function pass BTRFS_MAX_EXTENT_SIZE (128M) so let's reduce the argument count and make that a local variable. No functional changes. Signed-off-by: Nikolay Borisov Reviewed-by: Anand Jain Thanks, Anand --- fs/btrfs

Re: [PATCH 3/5] btrfs: Remove superfluous check form btrfs_remove_chunk

2018-11-16 Thread Anand Jain
On 10/26/2018 07:43 PM, Nikolay Borisov wrote: It's unnecessary to check map->stripes[i].dev for NULL given its value is already set and dereferenced above the the check. No functional changes. Signed-off-by: Nikolay Borisov Reviewed-by: Anand Jain Thanks, Anand --- fs/bt

Re: BTRFS on production: NVR 16+ IP Cameras

2018-11-16 Thread Anand Jain
On 11/16/2018 03:51 AM, Nikolay Borisov wrote: On 15.11.18 г. 20:39 ч., Juan Alberto Cirez wrote: Is BTRFS mature enough to be deployed on a production system to underpin the storage layer of a 16+ ipcameras-based NVR (or VMS if you prefer)? Based on our limited experience with BTRFS (1+

Re: [PATCH 7/9] btrfs: quiten warn if the replace is canceled at finish

2018-11-16 Thread Anand Jain
On 11/15/2018 11:35 PM, David Sterba wrote: On Sun, Nov 11, 2018 at 10:22:22PM +0800, Anand Jain wrote: When we successfully cancel the replace its scrub returns -ECANCELED, which then passed to btrfs_dev_replace_finishing(), it cleans up based on the scrub returned status and propagates

Re: [PATCH 8/9] btrfs: user requsted replace cancel is not an error

2018-11-16 Thread Anand Jain
On 11/16/2018 06:29 PM, Anand Jain wrote: On 11/15/2018 11:31 PM, David Sterba wrote: On Sun, Nov 11, 2018 at 10:22:23PM +0800, Anand Jain wrote: As of now only user requested replace cancel can cancel the replace-scrub so no need to log error for it. This has probably some user

Re: [PATCH 8/9] btrfs: user requsted replace cancel is not an error

2018-11-16 Thread Anand Jain
On 11/15/2018 11:31 PM, David Sterba wrote: On Sun, Nov 11, 2018 at 10:22:23PM +0800, Anand Jain wrote: As of now only user requested replace cancel can cancel the replace-scrub so no need to log error for it. This has probably some user visible effect or threre are steps to reproduce

Re: [PATCH 0/9 v2] fix replace-start and replace-cancel racing

2018-11-16 Thread Anand Jain
On 11/15/2018 11:41 PM, David Sterba wrote: On Sun, Nov 11, 2018 at 10:22:15PM +0800, Anand Jain wrote: v1->v2: 2/9: Drop writeback required 3/9: Drop writeback required 7/9: Use the condition within the WARN_ON() 6/9: Use the condition within the ASSERT() Replace-st

Re: [PATCH 5/9] btrfs: replace cancel is successful if scrub cancel is successful

2018-11-15 Thread Anand Jain
On 11/15/2018 11:27 PM, David Sterba wrote: On Sun, Nov 11, 2018 at 10:22:20PM +0800, Anand Jain wrote: In btrfs_dev_replace_cancel() we should check if the btrfs_scrub_cancel() is successful. If the btrfs_scrub_cancel() fails, return BTRFS_IOCTL_DEV_REPLACE_RESULT_NOT_STARTED so that user

Re: [PATCH 4/9] btrfs: fix UAF due to race between replace start and cancel

2018-11-15 Thread Anand Jain
On 11/15/2018 11:25 PM, David Sterba wrote: On Thu, Nov 15, 2018 at 03:00:21PM +0100, David Sterba wrote: On Wed, Nov 14, 2018 at 09:28:34AM +0800, Anand Jain wrote: mutex_unlock(_replace->lock_finishing_cancel_unmount); return result; There's a compiler warning: fs/bt

Re: [PATCH RFC] btrfs: harden agaist duplicate fsid

2018-11-14 Thread Anand Jain
On 11/13/2018 11:47 PM, Anand Jain wrote: On 11/13/2018 11:31 PM, David Sterba wrote: On Mon, Oct 01, 2018 at 09:31:04PM +0800, Anand Jain wrote: +    /* + * we are going to replace the device path, make sure its the + * same device if the device mounted

Re: [PATCH] btrfs: introduce feature to forget a btrfs device

2018-11-14 Thread Anand Jain
On 11/14/2018 07:28 PM, Filipe Manana wrote: On Wed, Nov 14, 2018 at 11:15 AM Filipe Manana wrote: On Wed, Nov 14, 2018 at 9:14 AM Anand Jain wrote: Support for a new command 'btrfs dev forget [dev]' is proposed here to undo the effects of 'btrfs dev scan [dev]'. For this purpose

Re: [PATCH] btrfs: introduce feature to forget a btrfs device

2018-11-14 Thread Anand Jain
On 11/14/2018 07:15 PM, Filipe Manana wrote: On Wed, Nov 14, 2018 at 9:14 AM Anand Jain wrote: Support for a new command 'btrfs dev forget [dev]' is proposed here to undo the effects of 'btrfs dev scan [dev]'. For this purpose this patch proposes to use ioctl #5 as it was empty

Re: [PATCH] Btrfs: ensure path name is null terminated at btrfs_control_ioctl

2018-11-14 Thread Anand Jain
a non-null terminated patch, leading to buffer overrun problems in the kernel. So just set the last byte of the path to a null character, similar to what we do in other ioctls (add/remove/resize device, snapshot creation, etc). Signed-off-by: Filipe Manana Reviewed-by: Anand Jain Thanks

Re: [PATCH] btrfs: Fix suspicious RCU usage warning in device_list_add

2018-11-14 Thread Anand Jain
lock/unlock of RCU string. Fixes: 1f265fc6f58b ("btrfs: harden agaist duplicate fsid on scanned devices") Signed-off-by: Lu Fengqi Thanks Lu. I missed it. Reviewed-by: Anand Jain --- fs/btrfs/volumes.c | 8 1 file changed, 4 insertions(+), 4 deletions(-) diff --git

[PATCH v5 3/3] btrfs: balance: add kernel log for end or paused

2018-11-14 Thread Anand Jain
Add a kernel log when the balance ends, either for cancel or completed or if it is paused. Signed-off-by: Anand Jain --- v4->v5: nothing. v3->v4: nothing. v2->v3: nothing. v1->v2: Moved from 2/3 to 3/3 fs/btrfs/volumes.c | 7 +++ 1 file changed, 7 insertions(+) diff --gi

[PATCH v5 2/3] btrfs: balance: add args info during start and resume

2018-11-14 Thread Anand Jain
=10..20 -msoft,profiles=raid1,convert=single -ssoft,profiles=raid1,convert=single Signed-off-by: Anand Jain --- v4.1->v5: Per David review comment the code.. bp += snprintf(bp, buf - bp + size_buf, "soft,"); is not safe if 'buf - bp + size_buf' beco

[PATCH v5 1/3] btrfs: add helper function describe_block_group()

2018-11-14 Thread Anand Jain
Improve on describe_relocation() add a common helper function to describe the block groups. Signed-off-by: Anand Jain Reviewed-by: David Sterba --- v4.1->v5: Initialize buf[128] to null. v4->v4.1: Use strcpy(buf, "|NONE"); as in the original v3->v4: Just pass full flag

[PATCH v5 0/3] btrfs: balance: improve kernel logs

2018-11-14 Thread Anand Jain
function and use it to log the balance operations. Kernel logs are very important for the forensic investigations of the issues, these patchs make balance logs easy to review. Anand Jain (3): btrfs: add helper function describe_block_group() btrfs: balance: add args info during start and resum

Re: [PATCH v4.1b 2/3] btrfs: balance: add args info during start and resume

2018-11-14 Thread Anand Jain
On 05/31/2018 05:47 PM, David Sterba wrote: On Fri, May 25, 2018 at 11:05:47AM +0800, Anand Jain wrote: Balance arg info is an important information to be reviewed for the system audit. So this patch adds them to the kernel log. Example: ->btrfs bal start -f -mprofiles=raid1,convert=sin

[PATCH] btrfs-progs: add cli to forget one or all scanned devices

2018-11-14 Thread Anand Jain
This patch adds cli btrfs device forget [dev] to remove the given device structure in the kernel if the device is unmounted. If no argument is given it shall remove all stale (device which are not mounted) from the kernel. Signed-off-by: Anand Jain Reviewed-by: Nikolay Borisov --- cmds

[PATCH RESEND v11] Add cli and ioctl to forget scanned device(s)

2018-11-14 Thread Anand Jain
to [PATCH 1/2] btrfs: add function to device list delete Adds cli and ioctl to forget a scanned device or forget all stale devices in the kernel. Anand Jain (1): btrfs: introduce feature to forget a btrfs device fs/btrfs/super.c | 3 +++ fs/btrfs/volumes.c | 9 + fs

[PATCH] btrfs: introduce feature to forget a btrfs device

2018-11-14 Thread Anand Jain
like in split brain raid1. . Running test cases which requires btrfs.ko-reload if the rootfs is btrfs. Signed-off-by: Anand Jain Reviewed-by: Nikolay Borisov --- fs/btrfs/super.c | 3 +++ fs/btrfs/volumes.c | 9 + fs/btrfs/volumes.h | 1 + include/uapi

[PATCH 4/9 v2.1] btrfs: fix UAF due to race between replace start and cancel

2018-11-13 Thread Anand Jain
btrfs_dev_replace_finishing() is called when btrfs_scub_dev() returns along with the scrub return status. Signed-off-by: Anand Jain --- v2->v2.1: Fix compiler warning. (I couldn't reproduce on gcc 4.8.5) fs/btrfs/dev-replace.c: In function ‘btrfs_dev_replace_cancel’: fs/btrfs/dev-replace.c:865:9:

Re: [PATCH 4/9] btrfs: fix UAF due to race between replace start and cancel

2018-11-13 Thread Anand Jain
On 11/14/2018 01:24 AM, David Sterba wrote: On Sun, Nov 11, 2018 at 10:22:19PM +0800, Anand Jain wrote: replace cancel thread can race with the replace start thread and if fs_info::scrubs_running is not yet set the btrfs_scrub_cancel() will fail to stop the scrub thread, so the scrub thread

Re: [PATCH V7] Btrfs: enhance raid1/10 balance heuristic

2018-11-13 Thread Anand Jain
I am ok with the least used path approach here for the IO routing that's probably most reasonable in generic configurations. It can be default read mirror policy as well. But as I mentioned. Not all configurations would agree to the heuristic approach here. For example: To make use of the SAN

Re: [PATCH RFC] btrfs: harden agaist duplicate fsid

2018-11-13 Thread Anand Jain
On 11/13/2018 11:31 PM, David Sterba wrote: On Mon, Oct 01, 2018 at 09:31:04PM +0800, Anand Jain wrote: +    /* + * we are going to replace the device path, make sure its the + * same device if the device mounted + */ +    if (device->b

Re: [PATCH RFC RESEND] btrfs: harden agaist duplicate fsid

2018-11-13 Thread Anand Jain
On 11/13/2018 11:21 PM, David Sterba wrote: On Mon, Oct 15, 2018 at 10:45:17AM +0800, Anand Jain wrote: (Thanks for the comments on requiring to warn_on if we fail the device change.) (This fixes an ugly bug, I appreciate if you have any further comments). Its not that impossible to imagine

Re: [RFC] BTRFS_DEV_REPLACE_ITEM_STATE_* doesn't match with on disk

2018-11-13 Thread Anand Jain
David, Gentle ping. Thanks, Anand On 11/12/2018 03:50 PM, Nikolay Borisov wrote: On 12.11.18 г. 6:58 ч., Anand Jain wrote: The dev_replace_state defines are miss matched between the BTRFS_IOCTL_DEV_REPLACE_STATE_* and BTRFS_DEV_REPLACE_ITEM_STATE_* [1]. [1

Re: [PATCH] btrfs: fix computation of max fs size for multiple device fs tests

2018-11-13 Thread Anand Jain
9\n20" | sort 11 20 9 $ echo -e "11\n9\n20" | sort -n 9 11 20 Signed-off-by: Filipe Manana Thanks for the fix. Reviewed-by: Anand Jain --- tests/btrfs/124 | 2 +- tests/btrfs/125 | 2 +- tests/btrfs/154 | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git

Re: Full filesystem btrfs rebalance kernel panic to read-only lock

2018-11-11 Thread Anand Jain
On 11/12/2018 10:12 AM, Qu Wenruo wrote: On 2018/11/12 上午9:35, Anand Jain wrote: On 11/09/2018 09:21 AM, Qu Wenruo wrote: On 2018/11/9 上午6:40, Pieter Maes wrote: Hello, So, I've had the full disk issue, so when I tried re-balancing, I got a panic, that pushed filesystem read-only

[PATCH] btrfs: remove redundant replace_state init

2018-11-11 Thread Anand Jain
dev_replace::replace_state has been set to BTRFS_DEV_REPLACE_ITEM_STATE_NEVER_STARTED (0) in the same function, So delete the line which sets replace_state = 0; Signed-off-by: Anand Jain --- fs/btrfs/dev-replace.c | 1 - 1 file changed, 1 deletion(-) diff --git a/fs/btrfs/dev-replace.c b/fs

[RFC] BTRFS_DEV_REPLACE_ITEM_STATE_* doesn't match with on disk

2018-11-11 Thread Anand Jain
The dev_replace_state defines are miss matched between the BTRFS_IOCTL_DEV_REPLACE_STATE_* and BTRFS_DEV_REPLACE_ITEM_STATE_* [1]. [1] - btrfs.h:#define BTRFS_IOCTL_DEV_REPLACE_STATE_FINISHED 2 btrfs.h:#define BTRFS_IOCTL_DEV_REPLACE_STATE_CANCELED

Re: Full filesystem btrfs rebalance kernel panic to read-only lock

2018-11-11 Thread Anand Jain
On 11/09/2018 09:21 AM, Qu Wenruo wrote: On 2018/11/9 上午6:40, Pieter Maes wrote: Hello, So, I've had the full disk issue, so when I tried re-balancing, I got a panic, that pushed filesystem read-only and I'm unable to balance or grow the filesystem now. fs info: btrfs fi show / Label:

[PATCH 9/9] btrfs: add explicit check for replace result no error

2018-11-11 Thread Anand Jain
while reading the code. Signed-off-by: Anand Jain --- fs/btrfs/dev-replace.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/fs/btrfs/dev-replace.c b/fs/btrfs/dev-replace.c index 40a0942b4659..cc25a34f87b0 100644 --- a/fs/btrfs/dev-replace.c +++ b/fs/btrfs/dev-replace.c

[PATCH 5/9] btrfs: replace cancel is successful if scrub cancel is successful

2018-11-11 Thread Anand Jain
In btrfs_dev_replace_cancel() we should check if the btrfs_scrub_cancel() is successful. If the btrfs_scrub_cancel() fails, return BTRFS_IOCTL_DEV_REPLACE_RESULT_NOT_STARTED so that user can try to cancel the replace again. Signed-off-by: Anand Jain --- fs/btrfs/dev-replace.c | 22

[PATCH 6/9] btrfs: replace's scrub must not be running in replace suspended state

2018-11-11 Thread Anand Jain
When the replace state is placed in the suspended state, btrfs_scrub_cancel() should fail with -ENOTCONN as there is no scrub running, as a safety catch check if btrfs_scrub_cancel() returns -ENOTCONN and assert if it doesn't. Signed-off-by: Anand Jain --- fs/btrfs/dev-replace.c | 4 +++- 1

[PATCH 7/9] btrfs: quiten warn if the replace is canceled at finish

2018-11-11 Thread Anand Jain
to quieten the warn here. Signed-off-by: Anand Jain --- fs/btrfs/dev-replace.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/fs/btrfs/dev-replace.c b/fs/btrfs/dev-replace.c index 1dc8e86546db..9031a362921a 100644 --- a/fs/btrfs/dev-replace.c +++ b/fs/btrfs/dev-replace.c

[PATCH 8/9] btrfs: user requsted replace cancel is not an error

2018-11-11 Thread Anand Jain
As of now only user requested replace cancel can cancel the replace-scrub so no need to log error for it. Signed-off-by: Anand Jain --- fs/btrfs/dev-replace.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/fs/btrfs/dev-replace.c b/fs/btrfs/dev-replace.c index 9031a362921a

[PATCH 3/9] btrfs: replace back to suspend state if EXCL OP is running

2018-11-11 Thread Anand Jain
to BTRFS_IOCTL_DEV_REPLACE_STATE_SUSPENDED state. Signed-off-by: Anand Jain --- fs/btrfs/dev-replace.c | 4 1 file changed, 4 insertions(+) diff --git a/fs/btrfs/dev-replace.c b/fs/btrfs/dev-replace.c index 3c29b0976087..35ce10f18607 100644 --- a/fs/btrfs/dev-replace.c +++ b/fs/btrfs/dev-replace.c @@ -897,6

[PATCH 2/9] btrfs: replace go back to suspended if target missing

2018-11-11 Thread Anand Jain
of BTRFS_IOCTL_DEV_REPLACE_STATE_STARTED as there isn't any matching scrub running as part of replace. Signed-off-by: Anand Jain --- fs/btrfs/dev-replace.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/fs/btrfs/dev-replace.c b/fs/btrfs/dev-replace.c index 59991165e126..3c29b0976087 100644 --- a/fs/btrfs

[PATCH 1/9] btrfs: mark btrfs_dev_replace_start() as static

2018-11-11 Thread Anand Jain
There isn't any other consumer other than in its own file dev-replace.c. Signed-off-by: Anand Jain Reviewed-by: Nikolay Borisov --- fs/btrfs/dev-replace.c | 2 +- fs/btrfs/dev-replace.h | 3 --- 2 files changed, 1 insertion(+), 4 deletions(-) diff --git a/fs/btrfs/dev-replace.c b/fs/btrfs/dev

[PATCH 0/9 v2] fix replace-start and replace-cancel racing

2018-11-11 Thread Anand Jain
/dev/sdb /btrfs && fillfs /btrfs 1 btrfs replace start /dev/sdb /dev/sdc /btrfs wait_for_user("scrub running is set..waiting"); AND OR wait_for_user("scrub running is NOT set..waiting"); reboot mount -o degraded /dev/sdb /btrfs btrfs replace s

[PATCH 4/9] btrfs: fix UAF due to race between replace start and cancel

2018-11-11 Thread Anand Jain
btrfs_dev_replace_finishing() is called when btrfs_scub_dev() returns along with the scrub return status. Signed-off-by: Anand Jain --- fs/btrfs/dev-replace.c | 61 -- 1 file changed, 39 insertions(+), 22 deletions(-) diff --git a/fs/btrfs/dev-replace

Re: [PATCH 2/9] btrfs: replace go back to suspended if target missing

2018-11-11 Thread Anand Jain
On 11/07/2018 08:35 PM, Nikolay Borisov wrote: On 7.11.18 г. 13:43 ч., Anand Jain wrote: At the time of forced unmount we place the running replace to BTRFS_IOCTL_DEV_REPLACE_STATE_SUSPENDED state, so when the system comes back and suppose the target device is missing, then let the replace

Re: [PATCH] btrfs: Check for missing device before bio submission in btrfs_map_bio

2018-11-10 Thread Anand Jain
t;bdev check. Doing so ensures that no bio cloning/submission happens for both async/sync requests in the face of missing device. This makes the async io submission path slightly shorter in terms of instruction count. No functional changes. Signed-off-by: Nikolay Borisov Reviewed-by: Anan

Re: [PATCH v15.1 00/13] Btrfs In-band De-duplication

2018-11-09 Thread Anand Jain
De-duplication must also let use cases to enable de-duplication on per subvolume level, using the subvolume properties. Similar to compression and future-encryption. Thanks, Anand

Re: [PATCH 6/9] btrfs: replace's scrub must not be running in replace suspended state

2018-11-08 Thread Anand Jain
On 11/08/2018 04:52 PM, Nikolay Borisov wrote: On 8.11.18 г. 10:33 ч., Anand Jain wrote: On 11/07/2018 08:19 PM, Nikolay Borisov wrote: On 7.11.18 г. 13:43 ч., Anand Jain wrote: +    /* scrub for replace must not be running in suspended state */ +    if (btrfs_scrub_cancel

Re: [PATCH 6/9] btrfs: replace's scrub must not be running in replace suspended state

2018-11-08 Thread Anand Jain
On 11/07/2018 08:19 PM, Nikolay Borisov wrote: On 7.11.18 г. 13:43 ч., Anand Jain wrote: + /* scrub for replace must not be running in suspended state */ + if (btrfs_scrub_cancel(fs_info) != -ENOTCONN) + ASSERT(0); ASSERT

Re: [PATCH 7/9] btrfs: quiten warn if the replace is canceled at finish

2018-11-08 Thread Anand Jain
On 11/07/2018 08:17 PM, Nikolay Borisov wrote: On 7.11.18 г. 13:43 ч., Anand Jain wrote: - WARN_ON(ret); + if (ret != -ECANCELED) + WARN_ON(ret); WARN_ON(ret && ret != -ECANCELED) Will fix. Thanks, Anand

Re: [PATCH 9/9] btrfs: add explicit check for replace result no error

2018-11-07 Thread Anand Jain
On 11/07/2018 08:15 PM, Nikolay Borisov wrote: On 7.11.18 г. 13:43 ч., Anand Jain wrote: We recast the replace return status BTRFS_IOCTL_DEV_REPLACE_RESULT_SCRUB_INPROGRESS to 0, to indicate no error. And since BTRFS_IOCTL_DEV_REPLACE_RESULT_NO_ERROR should also return 0, which is also

[PATCH 7/9] btrfs: quiten warn if the replace is canceled at finish

2018-11-07 Thread Anand Jain
When we successfully cancel the replace its scrub returns -ECANCELED, which then passed to btrfs_dev_replace_finishing(), it cleans up based on the scrub returned status and propagates the same -ECANCELED back the parent function. So skip the -ECANCELED error to log the WARN. Signed-off-by: Anand

[PATCH 8/9] btrfs: user requsted replace cancel is not an error

2018-11-07 Thread Anand Jain
As of now only user requested replace cancel can cancel the replace-scrub so no need to log error for it. Signed-off-by: Anand Jain --- fs/btrfs/dev-replace.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/fs/btrfs/dev-replace.c b/fs/btrfs/dev-replace.c index c14c41b70287

[PATCH 6/9] btrfs: replace's scrub must not be running in replace suspended state

2018-11-07 Thread Anand Jain
When the replace state is placed in the suspended state, btrfs_scrub_cancel() should fail with -ENOTCONN as there is no scrub running, as a safety catch check if btrfs_scrub_cancel() returns -ENOTCONN and assert if it doesn't. Signed-off-by: Anand Jain --- fs/btrfs/dev-replace.c | 4 +++- 1

[PATCH 9/9] btrfs: add explicit check for replace result no error

2018-11-07 Thread Anand Jain
while reading the code. Signed-off-by: Anand Jain --- fs/btrfs/dev-replace.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/fs/btrfs/dev-replace.c b/fs/btrfs/dev-replace.c index cf3554554616..ca44998189c7 100644 --- a/fs/btrfs/dev-replace.c +++ b/fs/btrfs/dev-replace.c

[PATCH 0/9] fix replace-start and replace-cancel racing

2018-11-07 Thread Anand Jain
wait_for_user("scrub running is NOT set..waiting"); reboot mount -o degraded /dev/sdb /btrfs btrfs replace status /btrfs btrfs replace cancel /btrfs btrfs replace status /btrfs umount /btrfs mount /dev/sdb /btrfs Anand Jain (9): btrfs: mark btrfs_dev_replace_start() as static btr

[PATCH 5/9] btrfs: replace cancel is successful if scrub cancel is successful

2018-11-07 Thread Anand Jain
In btrfs_dev_replace_cancel() we should check if the btrfs_scrub_cancel() is successful. If the btrfs_scrub_cancel() fails, return BTRFS_IOCTL_DEV_REPLACE_RESULT_NOT_STARTED so that user can try to cancel the replace again. Signed-off-by: Anand Jain --- fs/btrfs/dev-replace.c | 22

[PATCH 2/9] btrfs: replace go back to suspended if target missing

2018-11-07 Thread Anand Jain
of BTRFS_IOCTL_DEV_REPLACE_STATE_STARTED as there isn't any matching scrub running as part of replace. Signed-off-by: Anand Jain --- fs/btrfs/dev-replace.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/fs/btrfs/dev-replace.c b/fs/btrfs/dev-replace.c index 59991165e126..47d6768a9cde 100644 --- a/fs/btrfs

[PATCH 1/9] btrfs: mark btrfs_dev_replace_start() as static

2018-11-07 Thread Anand Jain
There isn't any other consumer other than in its own file dev-replace.c. Signed-off-by: Anand Jain --- fs/btrfs/dev-replace.c | 2 +- fs/btrfs/dev-replace.h | 3 --- 2 files changed, 1 insertion(+), 4 deletions(-) diff --git a/fs/btrfs/dev-replace.c b/fs/btrfs/dev-replace.c index 2aa48aecc52b

[PATCH 4/9] btrfs: fix UAF due to race between replace start and cancel

2018-11-07 Thread Anand Jain
btrfs_dev_replace_finishing() is called when btrfs_scub_dev() returns along with the scrub return status. Signed-off-by: Anand Jain --- fs/btrfs/dev-replace.c | 61 -- 1 file changed, 39 insertions(+), 22 deletions(-) diff --git a/fs/btrfs/dev-replace

[PATCH 3/9] btrfs: replace back to suspend state if EXCL OP is running

2018-11-07 Thread Anand Jain
to BTRFS_IOCTL_DEV_REPLACE_STATE_SUSPENDED state. Signed-off-by: Anand Jain --- fs/btrfs/dev-replace.c | 5 + 1 file changed, 5 insertions(+) diff --git a/fs/btrfs/dev-replace.c b/fs/btrfs/dev-replace.c index 47d6768a9cde..e001c2418940 100644 --- a/fs/btrfs/dev-replace.c +++ b/fs/btrfs/dev-replace.c @@ -898,6

[PATCH v3] fstests: btrfs verify hardening agaist duplicate fsid

2018-10-26 Thread Anand Jain
system to change its fsid using the 'btrfstune -u' command. Signed-off-by: Anand Jain --- v2->v3: Check the return code and use _fail to verify and accordingly fix golden output. Rename dev_foo(bar) to device_1(2) Don't log dd retun to $seqres.full v1->v2: dont play around wi

Re: [PATCH v2 rev log added] fstests: btrfs verify hardening agaist duplicate fsid

2018-10-26 Thread Anand Jain
On 10/26/2018 11:52 PM, Nikolay Borisov wrote: On 26.10.18 г. 18:34 ч., Anand Jain wrote: On 10/26/2018 11:02 PM, Nikolay Borisov wrote: On 8.10.18 г. 21:28 ч., Anand Jain wrote: We have a known bug in btrfs, that we let the device path be changed after the device has been mounted

Re: [PATCH v2 rev log added] fstests: btrfs verify hardening agaist duplicate fsid

2018-10-26 Thread Anand Jain
On 10/26/2018 11:02 PM, Nikolay Borisov wrote: On 8.10.18 г. 21:28 ч., Anand Jain wrote: We have a known bug in btrfs, that we let the device path be changed after the device has been mounted. So using this loop hole the new copied device would appears as if its mounted immediately after

  1   2   3   4   5   6   7   8   9   10   >