[PATCH] fstests: introduce btrfs/volume group

The btrfs/volume group represent a set of btrfs test-cases, which shall intend to verify the relevant btrfs volume operations. Under this new group all the existing btrfs/replace group would come under, and also the device operations test cases which does not have any group as of now. This group

Re: [PATCH 2nd try v4] btrfs: drop uuid_mutex in btrfs_free_extra_devids()

On 05/29/2018 06:57 AM, Anand Jain wrote: On 05/28/2018 11:40 PM, David Sterba wrote: On Mon, May 28, 2018 at 10:43:29PM +0800, Anand Jain wrote: btrfs_free_extra_devids() is called only in the mount context which traverses through the fs_devices::devices and frees the orphan devices

Re: [PATCH 00/11] Summer argument cleanup

On Mon, May 28, 2018 at 09:36:39AM +0300, Nikolay Borisov wrote: >Hello, > >Here is yet another series removing a bunch of extraneous argument. The series >constitutes no functional changes. Some of the function actually have a kernel >counterpart (btrfs_lookup_extent_info/clean_tree_block).

Re: [PATCH 2/3] btrfs: Use btrfs_mark_bg_unused() to replace open code

On 28.05.2018 12:20, Qu Wenruo wrote: > Introduce a small helper, btrfs_mark_bg_unused(), to accquire needed > locks and add a block group to unused_bgs list. > > No functional modification, and only 3 callers are involved. > > Signed-off-by: Qu Wenruo Reviewed-by: Nikolay Borisov > --- >

Re: [PATCH 2nd try v4] btrfs: drop uuid_mutex in btrfs_free_extra_devids()

On 05/28/2018 11:40 PM, David Sterba wrote: On Mon, May 28, 2018 at 10:43:29PM +0800, Anand Jain wrote: btrfs_free_extra_devids() is called only in the mount context which traverses through the fs_devices::devices and frees the orphan devices devices in the given %fs_devices if any. As the

Re: csum failed root raveled during balance

On 05/23/2018 01:03 PM, Austin S. Hemmelgarn wrote: > On 2018-05-23 06:09, ein wrote: >> On 05/23/2018 11:09 AM, Duncan wrote: >>> ein posted on Wed, 23 May 2018 10:03:52 +0200 as excerpted: >>> > IMHO the best course of action would be to disable checksumming for > you > vm files.

Re: [PATCH 1/2] btrfs: kill btrfs_write_inode

On Tue, May 22, 2018 at 01:47:22PM -0400, Josef Bacik wrote: > From: Josef Bacik > > We don't actually need this. It used to be in place for O_SYNC writes, > but we've used the normal fsync() path for that for years now. The > other case we hit this is through sync(), which will commit the >

Re: csum failed root raveled during balance

On 05/27/2018 11:41 AM, Nikolay Borisov wrote: > > > On 27.05.2018 08:50, Andrei Borzenkov wrote: >> 23.05.2018 09:32, Nikolay Borisov пишет: >>> >>> >>> On 22.05.2018 23:05, ein wrote: Hello devs, I tested BTRFS in production for about a month: 21:08:17 up 34 days,

Re: [PATCH v3 1/2] btrfs: qgroup: Search commit root for rescan to avoid missing extent

On Wed, May 23, 2018 at 10:32:13AM +0300, Nikolay Borisov wrote: > > > On 23.05.2018 10:29, Qu Wenruo wrote: > > When doing qgroup rescan using the following script (modified from > > btrfs/017 test case), we can sometimes hit qgroup corruption. > > > > -- > > umount $dev &> /dev/null > >

Re: [PATCH v2 0/6] btrfs_search_slot cleanups

On Mon, May 28, 2018 at 04:40:28PM +0200, David Sterba wrote: > On Fri, May 18, 2018 at 11:00:18AM +0800, Liu Bo wrote: > > Here are a collection of patches I did for btrfs_search_slot(). > > > > v2: more explicit commit log for each patch. > > > > Liu Bo (6): > > Btrfs: remove superfluous

[PATCH v4] btrfs: drop uuid_mutex in btrfs_free_extra_devids()

btrfs_free_extra_devids() is called only in the mount context which traverses through the fs_devices::devices and frees the orphan devices devices in the given %fs_devices if any. As the search for the orphan device is limited to fs_devices::devices so we don't need the global uuid_mutex. There

[PATCH 00/11] Summer argument cleanup

Hello, Here is yet another series removing a bunch of extraneous argument. The series constitutes no functional changes. Some of the function actually have a kernel counterpart (btrfs_lookup_extent_info/clean_tree_block). However the former's userspace signature differs and the patch in this

[PATCH 11/11] btrfs-progs: Remove fs_info argument from write_ctree_super

This function already takes a transaction handle which has a reference to the fs_info, so use that to obtain it. Signed-off-by: Nikolay Borisov --- disk-io.c | 6 +++--- disk-io.h | 3 +-- transaction.c | 2 +- 3 files changed, 5 insertions(+), 6 deletions(-) diff

Re: [PATCH 2/2] btrfs: Refactor running of delayed inode items during transaction commit

On Mon, May 28, 2018 at 03:26:58PM +0300, Nikolay Borisov wrote: > dmesg looks like: > [6.649213] WARNING: CPU: 0 PID: 2838 at fs/btrfs/transaction.c:303 > record_root_in_trans+0x38/0xd0 Found in the logs. I reported it to the patch that added the assertion but I did not

[PATCH 05/11] btrfs-progs: check: Make update_pinned_extents take btrfs_fs_info

This function needs btrfs_fs_info and not a root. So make it directly take btrfs_fs_info, Signed-off-by: Nikolay Borisov --- extent-tree.c | 12 ++-- 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/extent-tree.c b/extent-tree.c index

Re: [External] Re: [RFC PATCH v2 00/12] get rid of GFP_ZONE_TABLE/BAD

On Fri 25-05-18 09:43:09, Huaisheng HS1 Ye wrote: > From: Michal Hocko [mailto:mho...@kernel.org] > Sent: Thursday, May 24, 2018 8:19 PM> > > > Let me try to reply your questions. > > > Exactly, GFP_ZONE_TABLE is too complicated. I think there are two > > > advantages > > > from the series of

[PATCH 01/11] btrfs-progs: check: Remove root argument from delete_extent_records

This function is always passed the extent_root as "root" parameter. In turn it uses the root parameter to mostly access fs_info and performs only a single call to btrfs_update_block_group where it passses the passed root. This is all redundant since fs_info can be referenced from the transaction

[PATCH 07/11] btrfs-progs: check: Remove unused root argument from btrfs_extent_post_op

This is no longer used by the callees of that function so remove it. Signed-off-by: Nikolay Borisov --- check/main.c | 2 +- ctree.h | 3 +-- extent-tree.c | 5 ++--- 3 files changed, 4 insertions(+), 6 deletions(-) diff --git a/check/main.c b/check/main.c index

[PATCH 09/11] btrfs-progs: Remove root argument from btrfs_set_block_flags

It's used only to get a reference to fs_info, which can be obtained from the transaction handle. Signed-off-by: Nikolay Borisov --- ctree.c | 2 +- ctree.h | 5 ++--- extent-tree.c | 22 ++ 3 files changed, 13 insertions(+), 16 deletions(-)

[PATCH 03/11] btrfs-progs: check: Remove root parameter from del_pending_extents

This function always operates on the extent root which can be referenced from trans->fs_info. Do that to simplify function's signature. Signed-off-by: Nikolay Borisov --- extent-tree.c | 26 +- 1 file changed, 13 insertions(+), 13 deletions(-) diff

Re: [PATCH] btrfs: drop unused space_info parameter from create_space_info

On Mon, May 28, 2018 at 02:30:27PM +0800, Lu Fengqi wrote: > Since commit dc2d3005d27d ("btrfs: remove dead create_space_info > calls"), there is only one caller btrfs_init_space_info. However, it > doesn't need create_space_info to return space_info at all. > > Signed-off-by: Lu Fengqi

Re: [RFC PATCH v4 3/6] Btrfs: push EXCL_OP set into btrfs_rm_device()

On Thu, May 24, 2018 at 02:41:27PM -0700, Omar Sandoval wrote: > From: Omar Sandoval > > btrfs_ioctl_rm_dev() and btrfs_ioctl_rm_dev_v2() both manipulate this > bit. Let's move it into the common btrfs_rm_device(), which also makes > the following change to deal with swap files

Re: [PATCH 2/2] btrfs: Refactor running of delayed inode items during transaction commit

On 28.05.2018 14:51, Qu Wenruo wrote: > > > On 2018年05月28日 16:51, Nikolay Borisov wrote: >> >> >> On 28.05.2018 11:35, Nikolay Borisov wrote: >>> >>> >>> On 28.05.2018 11:21, Misono Tomohiro wrote: Hello, I found current misc-next sometimes fails btrfs/152 when the number

[PATCH 3/3] btrfs: Delayed empty block group auto removal to next transaction

Under certain KVM load and LTP tests, we are possible to hit the following calltrace if quota is enabled: -- BTRFS critical (device vda2): unable to find logical 8820195328 length 4096 BTRFS critical (device vda2): unable to find logical 8820195328 length 4096 [ cut here

Re: [RFC PATCH v4 6/6] Btrfs: support swap files

On Thu, May 24, 2018 at 02:41:30PM -0700, Omar Sandoval wrote: > From: Omar Sandoval > > Implement the swap file a_ops on Btrfs. Activation needs to make sure > that the file can be used as a swap file, which currently means it must > be fully allocated as nocow with no

[PATCH 06/11] btrfs-progs: Remove unused argument from clean_tree_block

This function actually uses only the extent_buffer arg but takes 3 arguments. Furthermore, it's current interface doesn't even mirror the kernel counterpart. Just remove the extra arguments. Signed-off-by: Nikolay Borisov --- ctree.c | 12 ++-- disk-io.c

[PATCH 3/3] btrfs: Delayed empty block group auto removal to next transaction

Under certain KVM load and LTP tests, we are possible to hit the following calltrace if quota is enabled: -- BTRFS critical (device vda2): unable to find logical 8820195328 length 4096 BTRFS critical (device vda2): unable to find logical 8820195328 length 4096 [ cut here

Re: [RFC PATCH v2 00/12] get rid of GFP_ZONE_TABLE/BAD

On Fri 25-05-18 05:00:44, Matthew Wilcox wrote: > On Thu, May 24, 2018 at 05:29:43PM +0200, Michal Hocko wrote: > > > ie if we had more, > > > could we solve our pain by making them more generic? > > > > Well, if you have more you will consume more bits in the struct pages, > > right? > > Not

[PATCH 2/3] btrfs: Use btrfs_mark_bg_unused() to replace open code

Introduce a small helper, btrfs_mark_bg_unused(), to accquire needed locks and add a block group to unused_bgs list. No functional modification, and only 3 callers are involved. Signed-off-by: Qu Wenruo --- fs/btrfs/ctree.h | 1 + fs/btrfs/extent-tree.c | 36

[PATCH 10/11] btrfs-progs: Remove root argument from write_one_cache_group

It's not needed since we can acquire a reference to the fs_info from the transaction handle already passed. Signed-off-by: Nikolay Borisov --- extent-tree.c | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/extent-tree.c b/extent-tree.c index

Re: [PATCH 2/2] btrfs: Refactor running of delayed inode items during transaction commit

On 28.05.2018 11:21, Misono Tomohiro wrote: > Hello, > > I found current misc-next sometimes fails btrfs/152 when the number > of cpu is >= 4 in my vm and git bisect points this commit. > (btrfs/152 performs parallel send/receive.) > > The failure is caused by _check_dmesg and sometimes also

[PATCH 0/3] btrfs: Delay block group auto removal to avoid interfering qgroups

The patchset can be fetched from github: https://github.com/adam900710/linux/tree/delayed_bg_removal It's based on v4.17-rc5 branch. This bug is reported from SUSE openQA, although it's pretty hard to hit in real world (even real world VM), it's believed block group auto removal (anyway, there

[PATCH 1/3] btrfs: trace: Add trace points for unused block groups

This patch will add the following trace events: 1) btrfs_remove_block_group For btrfs_remove_block_group() function. Triggered when a block group is really removed. 2) btrfs_add_unused_block_group Triggered which block group is added to unused_bgs list. 3) btrfs_skip_unused_block_group

[PATCH 02/11] btrfs-progs: check: Remove root parameter from btrfs_fix_block_accounting

It's always set to extent_root and the function already takes a transaction handle where fs_info could be referenced and in turn the extent_tree. Signed-off-by: Nikolay Borisov --- check/main.c| 2 +- check/mode-lowmem.c | 2 +- ctree.h | 3 +--

[PATCH 04/11] btrfs-progs: check: Remove root argument from finish_current_insert

Just reference it directly from trans->fs_info. Signed-off-by: Nikolay Borisov --- extent-tree.c | 27 +-- 1 file changed, 13 insertions(+), 14 deletions(-) diff --git a/extent-tree.c b/extent-tree.c index e950ba6de3cc..89fed5b73b1f 100644 ---

[PATCH 2/3] btrfs: Use btrfs_mark_bg_unused() to replace open code

Introduce a small helper, btrfs_mark_bg_unused(), to accquire needed locks and add a block group to unused_bgs list. No functional modification, and only 3 callers are involved. Signed-off-by: Qu Wenruo --- fs/btrfs/ctree.h | 1 + fs/btrfs/extent-tree.c | 36

[PATCH 0/3] btrfs: Delay block group auto removal to avoid interfering qgroups

The patchset can be fetched from github: https://github.com/adam900710/linux/tree/delayed_bg_removal It's based on v4.17-rc5 branch. This bug is reported from SUSE openQA, although it's pretty hard to hit in real world (even real world VM), it's believed block group auto removal (anyway, there

Re: [RFC PATCH v2 0/6] btrfs send stream version 2

On Sun, May 27, 2018 at 10:14:29PM -0400, Howard McLauchlan wrote: > This is v2 of send stream version 2. The goal is to provide proper > versioning/compatibility as new features are implemented. v1 can be found here > [1]. We need to decide the overall approach to the versioning updates. The

[PATCH 08/11] btrfs-progs: Change btrfs_root to btrfs_fs_info argument in btrfs_lookup_extent_info

That function really wants an fs_info and not a root. Accidentally, this also makes the kernel/user space signatures to be coherent. Signed-off-by: Nikolay Borisov --- check/main.c| 10 +- check/mode-lowmem.c | 4 ++-- ctree.c | 3 ++- ctree.h

Re: [PATCH v2 3/4] btrfs: lzo: Add header length check to avoid slab out of bounds access

On Fri, May 25, 2018 at 09:31:30AM +0800, Qu Wenruo wrote: > > > On 2018年05月25日 00:43, David Sterba wrote: > > On Wed, May 23, 2018 at 07:38:28AM +0800, Qu Wenruo wrote: > --- a/fs/btrfs/lzo.c > +++ b/fs/btrfs/lzo.c > @@ -281,6 +281,7 @@ static int lzo_decompress_bio(struct

[PATCH 1/3] btrfs: trace: Add trace points for unused block groups

This patch will add the following trace events: 1) btrfs_remove_block_group For btrfs_remove_block_group() function. Triggered when a block group is really removed. 2) btrfs_add_unused_block_group Triggered which block group is added to unused_bgs list. 3) btrfs_skip_unused_block_group

Re: csum failed root raveled during balance

On 27.05.2018 08:50, Andrei Borzenkov wrote: > 23.05.2018 09:32, Nikolay Borisov пишет: >> >> >> On 22.05.2018 23:05, ein wrote: >>> Hello devs, >>> >>> I tested BTRFS in production for about a month: >>> >>> 21:08:17 up 34 days, 2:21, 3 users, load average: 0.06, 0.02, 0.00 >>> >>> Without

Re: [PATCH 2nd try v4] btrfs: drop uuid_mutex in btrfs_free_extra_devids()

On Mon, May 28, 2018 at 10:43:29PM +0800, Anand Jain wrote: > btrfs_free_extra_devids() is called only in the mount context which > traverses through the fs_devices::devices and frees the orphan devices > devices in the given %fs_devices if any. As the search for the orphan > device is limited to

Re: [PATCH] btrfs: drop unused space_info parameter from create_space_info

On 28.05.2018 09:30, Lu Fengqi wrote: > Since commit dc2d3005d27d ("btrfs: remove dead create_space_info > calls"), there is only one caller btrfs_init_space_info. However, it > doesn't need create_space_info to return space_info at all. > > Signed-off-by: Lu Fengqi

Re: [PATCH 2/2] btrfs: Refactor running of delayed inode items during transaction commit

On 28.05.2018 11:35, Nikolay Borisov wrote: > > > On 28.05.2018 11:21, Misono Tomohiro wrote: >> Hello, >> >> I found current misc-next sometimes fails btrfs/152 when the number >> of cpu is >= 4 in my vm and git bisect points this commit. >> (btrfs/152 performs parallel send/receive.) >> >>

Re: [PATCH 0/4] btrfs check --check-data-csum enhancement for

On Mon, May 28, 2018 at 08:20:35PM +0800, Qu Wenruo wrote: > Or do I need to rebase the patchset and resend? The patches are in devel for a long time. If you have any updates, please send as incremental patches, thanks. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in

Re: [PATCH v2 4/6] Btrfs: remove unused check of skip_locking

On 28.05.2018 17:21, David Sterba wrote: > On Fri, May 18, 2018 at 01:27:50PM +0800, Qu Wenruo wrote: >> >> >> On 2018年05月18日 11:00, Liu Bo wrote: >>> The check is superfluous since all of callers who set search_for_commit >>> also have skip_locking set. >>> >>> Signed-off-by: Liu Bo

Re: [PATCH v2 5/6] Btrfs: grab write lock directly if write_lock_level is the max level

On Fri, May 18, 2018 at 11:00:23AM +0800, Liu Bo wrote: > Typically, when acquiring root node's lock, btrfs tries its best to get > read lock and trade for write lock if @write_lock_level implies to do so. > > In case of (cow && (p->keep_locks || p->lowest_level)), write_lock_level > is set to

Re: [PATCH v2 4/6] Btrfs: remove unused check of skip_locking

On Fri, May 18, 2018 at 01:27:50PM +0800, Qu Wenruo wrote: > > > On 2018年05月18日 11:00, Liu Bo wrote: > > The check is superfluous since all of callers who set search_for_commit > > also have skip_locking set. > > > > Signed-off-by: Liu Bo > > Reviewed-by: Qu Wenruo

Re: [PATCH 2/2] btrfs: Refactor running of delayed inode items during transaction commit

On 2018年05月28日 16:51, Nikolay Borisov wrote: > > > On 28.05.2018 11:35, Nikolay Borisov wrote: >> >> >> On 28.05.2018 11:21, Misono Tomohiro wrote: >>> Hello, >>> >>> I found current misc-next sometimes fails btrfs/152 when the number >>> of cpu is >= 4 in my vm and git bisect points this

Re: [PATCH v2 0/6] btrfs_search_slot cleanups

On Fri, May 18, 2018 at 11:00:18AM +0800, Liu Bo wrote: > Here are a collection of patches I did for btrfs_search_slot(). > > v2: more explicit commit log for each patch. > > Liu Bo (6): > Btrfs: remove superfluous free_extent_buffer > Btrfs: use more straightforward extent_buffer_uptodate >

[PATCH 2nd try v4] btrfs: drop uuid_mutex in btrfs_free_extra_devids()

btrfs_free_extra_devids() is called only in the mount context which traverses through the fs_devices::devices and frees the orphan devices devices in the given %fs_devices if any. As the search for the orphan device is limited to fs_devices::devices so we don't need the global uuid_mutex. There

Re: Questions from aspiring btrfs mini-debugger/mini-developer

On 2018年05月28日 17:21, james harvey wrote: > I'm tracking down some more bugs. Yeah, more bugs for us to fix. > > Useful information for you to track down these bugs isn't in this > email. This is more about an aspiring btrfs > mini-debugger/mini-developer asking for some guidance, to be able

Re: [PATCH 0/4] btrfs check --check-data-csum enhancement for

Gentle ping. Or do I need to rebase the patchset and resend? Thanks, Qu On 2018年02月27日 17:12, Qu Wenruo wrote: > Log-writes tool from Josef exposed btrfs data corruption. > > Although still digging, at least enhance --check-data-csum option so > that log-replay --check can stop exactly at the

[PATCH v2 2/4] fstests: btrfs: nested seed device test

Test case to verify that a sprout device can be a seed device Signed-off-by: Anand Jain --- tests/btrfs/162 | 103 tests/btrfs/162.out | 9 + tests/btrfs/group | 1 + 3 files changed, 113 insertions(+)

[PATCH v2 4/4] fstests: btrfs: seed device delete test

Test case to verify that a seed device can be deleted Signed-off-by: Anand Jain --- tests/btrfs/164 | 114 tests/btrfs/164.out | 9 + tests/btrfs/group | 1 + 3 files changed, 124 insertions(+) create

[PATCH v2 3/4] fstests: btrfs: seed device replace test

Test case to verify that a seed device can be replaced Signed-off-by: Anand Jain --- tests/btrfs/163 | 114 tests/btrfs/163.out | 9 + tests/btrfs/group | 1 + 3 files changed, 124 insertions(+) create

[PATCH v2 1/4] fstests: btrfs: add seed sprout functionality test

Create a seed device and add the sprout device to it. Signed-off-by: Anand Jain --- v1->v2: Use functions to do the respective operations. Add data verification. common/config | 1 + tests/btrfs/161 | 90 +

[PATCH v3 0/4] fstests: btrfs seed test cases

1/4 was sent separately which is now integrated into this set, with some changes as indicated in the patch. Though rest of the patches 2/4..4/4 are new, they carry the set version v2. These test cases verify the seed sprout opertions. Anand Jain (4): fstests: btrfs: add seed sprout

Questions from aspiring btrfs mini-debugger/mini-developer

I'm tracking down some more bugs. Useful information for you to track down these bugs isn't in this email. This is more about an aspiring btrfs mini-debugger/mini-developer asking for some guidance, to be able to get the more useful information. I ran across some mirrored files that are

Re: RAID-1 refuses to balance large drive

Brad Templeton posted on Sun, 27 May 2018 11:22:07 -0700 as excerpted: > BTW, I decided to follow the original double replace strategy suggested -- > replace 6TB with 8TB and replace 4TB with 6TB. That should be sure to > leave the 2 large drives each with 2TB free once expanded, and thus able

Re: [PATCH 2/2] btrfs: Refactor running of delayed inode items during transaction commit

Hello, I found current misc-next sometimes fails btrfs/152 when the number of cpu is >= 4 in my vm and git bisect points this commit. (btrfs/152 performs parallel send/receive.) The failure is caused by _check_dmesg and sometimes also leads to inconsistent fs. dmesg looks like: [6.649213]

[PATCH] btrfs: drop unused space_info parameter from create_space_info

Since commit dc2d3005d27d ("btrfs: remove dead create_space_info calls"), there is only one caller btrfs_init_space_info. However, it doesn't need create_space_info to return space_info at all. Signed-off-by: Lu Fengqi --- fs/btrfs/extent-tree.c | 13 + 1