Re:

2018-11-22 Thread Chris Murphy
On Thu, Nov 22, 2018 at 11:41 PM Andy Leadbetter wrote: > > I have a failing 2TB disk that is part of a 4 disk RAID 6 system. I > have added a new 2TB disk to the computer, and started a BTRFS replace > for the old and new disk. The process starts correctly however some > hours into the job,

Re: [PATCH 1/6] btrfs: add btrfs_delete_ref_head helper

2018-11-22 Thread Nikolay Borisov
On 22.11.18 г. 11:12 ч., Nikolay Borisov wrote: > > > On 21.11.18 г. 20:59 ч., Josef Bacik wrote: >> From: Josef Bacik >> >> We do this dance in cleanup_ref_head and check_ref_cleanup, unify it >> into a helper and cleanup the calling functions. >> >> Signed-off-by: Josef Bacik >>

Re: [PATCH 3/6] btrfs: cleanup extent_op handling

2018-11-22 Thread Nikolay Borisov
On 21.11.18 г. 20:59 ч., Josef Bacik wrote: > From: Josef Bacik > > The cleanup_extent_op function actually would run the extent_op if it > needed running, which made the name sort of a misnomer. Change it to > run_and_cleanup_extent_op, and move the actual cleanup work to >

Re: [PATCH 4/6] btrfs: only track ref_heads in delayed_ref_updates

2018-11-22 Thread Nikolay Borisov
On 21.11.18 г. 20:59 ч., Josef Bacik wrote: > From: Josef Bacik > > We use this number to figure out how many delayed refs to run, but > __btrfs_run_delayed_refs really only checks every time we need a new > delayed ref head, so we always run at least one ref head completely no > matter what

Re: [PATCH 3/6] btrfs: cleanup extent_op handling

2018-11-22 Thread Lu Fengqi
On Wed, Nov 21, 2018 at 01:59:09PM -0500, Josef Bacik wrote: >From: Josef Bacik > >The cleanup_extent_op function actually would run the extent_op if it >needed running, which made the name sort of a misnomer. Change it to >run_and_cleanup_extent_op, and move the actual cleanup work to

[PATCH] btrfs: Remove unnecessary code from __btrfs_rebalance

2018-11-22 Thread Nikolay Borisov
The first step fo the rebalance process, ensuring there is 1mb free on each device. This number seems rather small. And in fact when talking to the original authors their opinions were: "man that's a little bonkers" "i don't think we even need that code anymore" "I think it was there to make sure

[PATCH] btrfs: Remove extent_io_ops::readpage_io_failed_hook

2018-11-22 Thread Nikolay Borisov
For data inodes this hook does nothing but to return -EAGAIN which is used to signal to the endio routines that this bio belongs to a data inode. If this is the case the actual retrying is handled by bio_readpage_error. Alternatively, if this bio belongs to the btree inode then

Re: [PATCH 1/6] btrfs: add btrfs_delete_ref_head helper

2018-11-22 Thread Nikolay Borisov
On 21.11.18 г. 20:59 ч., Josef Bacik wrote: > From: Josef Bacik > > We do this dance in cleanup_ref_head and check_ref_cleanup, unify it > into a helper and cleanup the calling functions. > > Signed-off-by: Josef Bacik > Reviewed-by: Omar Sandoval > --- > fs/btrfs/delayed-ref.c | 14

[PATCH] btrfs: improve error handling of btrfs_add_link()

2018-11-22 Thread Johannes Thumshirn
err holds the return value of either btrfs_del_root_ref() or btrfs_del_inode_ref() but it hasn't been checked since it's introduction with commit fe66a05a0679 (Btrfs: improve error handling for btrfs_insert_dir_item callers) in 2012. The first attempt in removing the variable was rejected, so

Re: [PATCH] btrfs: improve error handling of btrfs_add_link()

2018-11-22 Thread David Sterba
On Thu, Nov 22, 2018 at 02:15:28PM +0100, Johannes Thumshirn wrote: > err holds the return value of either btrfs_del_root_ref() or > btrfs_del_inode_ref() but it hasn't been checked since it's > introduction with commit fe66a05a0679 (Btrfs: improve error handling > for btrfs_insert_dir_item

btrfs-cleaner 100% busy on an idle filesystem with 4.19.3

2018-11-22 Thread Tomasz Chmielewski
Yet another system upgraded to 4.19 and showing strange issues. btrfs-cleaner is showing as ~90-100% busy in iotop: TID PRIO USER DISK READ DISK WRITE SWAPIN IO>COMMAND 1340 be/4 root0.00 B/s0.00 B/s 0.00 % 92.88 % [btrfs-cleaner] Note disk read, disk write

Re: btrfs-cleaner 100% busy on an idle filesystem with 4.19.3

2018-11-22 Thread Nikolay Borisov
On 22.11.18 г. 14:31 ч., Tomasz Chmielewski wrote: > Yet another system upgraded to 4.19 and showing strange issues. > > btrfs-cleaner is showing as ~90-100% busy in iotop: > >    TID  PRIO  USER DISK READ  DISK WRITE  SWAPIN IO>    COMMAND >   1340 be/4 root    0.00 B/s    0.00

Re: [PATCH] btrfs: improve error handling of btrfs_add_link()

2018-11-22 Thread Johannes Thumshirn
On 22/11/2018 14:35, David Sterba wrote: > On Thu, Nov 22, 2018 at 02:15:28PM +0100, Johannes Thumshirn wrote: >> err holds the return value of either btrfs_del_root_ref() or >> btrfs_del_inode_ref() but it hasn't been checked since it's >> introduction with commit fe66a05a0679 (Btrfs: improve

Re: btrfs-cleaner 100% busy on an idle filesystem with 4.19.3

2018-11-22 Thread Tomasz Chmielewski
On 2018-11-22 21:46, Nikolay Borisov wrote: # echo w > /proc/sysrq-trigger # dmesg -c [  931.585611] sysrq: SysRq : Show Blocked State [  931.585715]   task    PC stack   pid father [  931.590168] btrfs-cleaner   D    0  1340  2 0x8000 [  931.590175] Call Trace: [ 

Re: [PATCH v2] Btrfs: fix deadlock when enabling quotas due to concurrent snapshot creation

2018-11-22 Thread David Sterba
On Tue, Nov 20, 2018 at 08:32:42AM +0800, Qu Wenruo wrote: > >>> @@ -1013,16 +1013,22 @@ int btrfs_quota_enable(struct btrfs_fs_info > >>> *fs_info) > >>> btrfs_abort_transaction(trans, ret); > >>> goto out_free_path; > >>> } > >>> -

Re: [PATCH v2] Btrfs: fix deadlock when enabling quotas due to concurrent snapshot creation

2018-11-22 Thread Qu Wenruo
On 2018/11/22 下午9:12, David Sterba wrote: > On Tue, Nov 20, 2018 at 08:32:42AM +0800, Qu Wenruo wrote: > @@ -1013,16 +1013,22 @@ int btrfs_quota_enable(struct btrfs_fs_info > *fs_info) > btrfs_abort_transaction(trans, ret); > goto out_free_path; >

Re: [RFC][PATCH v4 09/09] btrfs: use common file type conversion

2018-11-22 Thread Jan Kara
On Wed 21-11-18 19:07:06, Phillip Potter wrote: > Deduplicate the btrfs file type conversion implementation - file systems > that use the same file types as defined by POSIX do not need to define > their own versions and can use the common helper functions decared in > fs_types.h and implemented

Re: btrfs-cleaner 100% busy on an idle filesystem with 4.19.3

2018-11-22 Thread Roman Mamedov
On Thu, 22 Nov 2018 22:07:25 +0900 Tomasz Chmielewski wrote: > Spot on! > > Removed "discard" from fstab and added "ssd", rebooted - no more > btrfs-cleaner running. Recently there has been a bugfix for TRIM in Btrfs: btrfs: Ensure btrfs_trim_fs can trim the whole fs

Re: [PATCH 3/4] btrfs: remove redundant csum buffer in btrfs_io_bio

2018-11-22 Thread Nikolay Borisov
On 22.11.18 г. 18:16 ч., David Sterba wrote: > The io_bio tracks checksums and has an inline buffer or an allocated > one. And there's a third member that points to the right one, but we > don't need to use an extra pointer for that. Let btrfs_io_bio::csum > point to the right buffer and check

Re: btrfs-cleaner 100% busy on an idle filesystem with 4.19.3

2018-11-22 Thread Qu Wenruo
On 2018/11/22 下午10:03, Roman Mamedov wrote: > On Thu, 22 Nov 2018 22:07:25 +0900 > Tomasz Chmielewski wrote: > >> Spot on! >> >> Removed "discard" from fstab and added "ssd", rebooted - no more >> btrfs-cleaner running. > > Recently there has been a bugfix for TRIM in Btrfs: > > btrfs:

Re: [PATCH v2] Btrfs: fix deadlock when enabling quotas due to concurrent snapshot creation

2018-11-22 Thread David Sterba
On Thu, Nov 22, 2018 at 09:46:44PM +0800, Qu Wenruo wrote: > >>> it and after > >>> we started (or joined) a transaction, a lot could of modifications may > >>> have happened. > >>> Nevertheless I don't think it's unexpected for anyone to have the > >>> accounting happening > >>> only after the

[PATCH 2/4] btrfs: replace async_cow::root with fs_info

2018-11-22 Thread David Sterba
The async_cow::root is used to propagate fs_info to async_cow_submit. We can't use inode to reach it because it could become NULL after write without compression in async_cow_start. Signed-off-by: David Sterba --- fs/btrfs/inode.c | 9 +++-- 1 file changed, 3 insertions(+), 6 deletions(-)

Re: [PATCH 1/4] btrfs: merge btrfs_submit_bio_done to its caller

2018-11-22 Thread Nikolay Borisov
On 22.11.18 г. 18:16 ч., David Sterba wrote: > There's one caller and its code is simple, we can open code it in > run_one_async_done. The errors are passed through bio. > > Signed-off-by: David Sterba Reviewed-by: Nikolay Borisov > --- > fs/btrfs/disk-io.c | 18 +- >

Re: [PATCH] btrfs: improve error handling of btrfs_add_link()

2018-11-22 Thread David Sterba
On Thu, Nov 22, 2018 at 02:44:35PM +0100, Johannes Thumshirn wrote: > On 22/11/2018 14:35, David Sterba wrote: > > On Thu, Nov 22, 2018 at 02:15:28PM +0100, Johannes Thumshirn wrote: > >> err holds the return value of either btrfs_del_root_ref() or > >> btrfs_del_inode_ref() but it hasn't been

Re: [PATCH 2/4] btrfs: replace async_cow::root with fs_info

2018-11-22 Thread Nikolay Borisov
On 22.11.18 г. 18:16 ч., David Sterba wrote: > The async_cow::root is used to propagate fs_info to async_cow_submit. > We can't use inode to reach it because it could become NULL after > write without compression in async_cow_start. > > Signed-off-by: David Sterba Reviewed-by: Nikolay

[PATCH 0/4] Minor helper and struct member cleanups

2018-11-22 Thread David Sterba
Assorted updates to code vaguely related to the bio callbacks and structures. Remove call indirection, remove redundant struct members, etc. David Sterba (4): btrfs: merge btrfs_submit_bio_done to its caller btrfs: replace async_cow::root with fs_info btrfs: remove redundant csum buffer in

[PATCH 1/4] btrfs: merge btrfs_submit_bio_done to its caller

2018-11-22 Thread David Sterba
There's one caller and its code is simple, we can open code it in run_one_async_done. The errors are passed through bio. Signed-off-by: David Sterba --- fs/btrfs/disk-io.c | 18 +- fs/btrfs/inode.c | 23 --- 2 files changed, 17 insertions(+), 24

[PATCH 4/4] btrfs: replace btrfs_io_bio::end_io with a simple helper

2018-11-22 Thread David Sterba
The end_io callback implemented as btrfs_io_bio_endio_readpage only calls kfree. Also the callback is set only in case the csum buffer is allocated and not pointing to the inline buffer. We can use that information to drop the indirection and call a helper that will free the csums only in the

[PATCH 3/4] btrfs: remove redundant csum buffer in btrfs_io_bio

2018-11-22 Thread David Sterba
The io_bio tracks checksums and has an inline buffer or an allocated one. And there's a third member that points to the right one, but we don't need to use an extra pointer for that. Let btrfs_io_bio::csum point to the right buffer and check that the inline buffer is not accidentally freed. This

Re: [PATCH 4/4] btrfs: replace btrfs_io_bio::end_io with a simple helper

2018-11-22 Thread Nikolay Borisov
On 22.11.18 г. 18:16 ч., David Sterba wrote: > The end_io callback implemented as btrfs_io_bio_endio_readpage only > calls kfree. Also the callback is set only in case the csum buffer is > allocated and not pointing to the inline buffer. We can use that > information to drop the indirection and

[PATCH] btrfs: tree-checker: Don't check max block group size as current max chunk size limit is unreliable

2018-11-22 Thread Qu Wenruo
[BUG] A completely valid btrfs will refuse to mount, with error message like: BTRFS critical (device sdb2): corrupt leaf: root=2 block=239681536 slot=172 \ bg_start=12018974720 bg_len=10888413184, invalid block group size, \ have 10888413184 expect (0, 10737418240] Btrfs check returns

[no subject]

2018-11-22 Thread Andy Leadbetter
I have a failing 2TB disk that is part of a 4 disk RAID 6 system. I have added a new 2TB disk to the computer, and started a BTRFS replace for the old and new disk. The process starts correctly however some hours into the job, there is an error and kernel oops. relevant log below. The disks are

Re: btrfs-cleaner 100% busy on an idle filesystem with 4.19.3

2018-11-22 Thread Chris Murphy
On Thu, Nov 22, 2018 at 6:07 AM Tomasz Chmielewski wrote: > > On 2018-11-22 21:46, Nikolay Borisov wrote: > > >> # echo w > /proc/sysrq-trigger > >> > >> # dmesg -c > >> [ 931.585611] sysrq: SysRq : Show Blocked State > >> [ 931.585715] taskPC stack pid father > >> [