Re: Status of RAID5/6

On Wed, Mar 21, 2018 at 9:50 AM, Menion wrote: > Hi all > I am trying to understand the status of RAID5/6 in BTRFS > I know that there are some discussion ongoing on the RFC patch > proposed by Liu bo > But it seems that everything stopped last summary. Also it mentioned > about

Re: [PATCH] btrfs-progs: mkfs: add uuid and otime to ROOT_ITEM of FS_TREE

On Mon, Mar 19, 2018 at 05:16:42PM +0900, Misono, Tomohiro wrote: > Currently, the top-level subvolume lacks the UUID. As a result, both > non-snapshot subvolume and snapshot of top-level subvolume do not have > Parent UUID and cannot be distinguisued. Therefore "fi show" of > top-level lists all

Re: Status of RAID5/6

Hey. Some things would IMO be nice to get done/clarified (i.e. documented in the Wiki and manpages) from users'/admin's POV: Some basic questions: - Starting with which kernels (including stable kernel versions) does it contain the fixes for the bigger issues from some time ago? - Exactly what

Out of space and incorrect size reported

> uname -a Linux rockstor 4.4.5-1.el7.elrepo.x86_64 #1 SMP Thu Mar 10 11:45:51 EST 2016 x86_64 x86_64 x86_64 GNU/Linux > btrfs —version btrfs-progs v4.4.1 > btrfs fi df /mnt2/pool_homes Data, RAID1: total=240.00GiB, used=239.78GiB System, RAID1: total=8.00MiB, used=64.00KiB Metadata, RAID1:

Re: Status of RAID5/6

I am on 4.15.5 :) Yes I agree that Journaling is better on the same array, still should be unit failure tolerant, so maybe it should go in a RAID1 scheme. Will a raid56 array built with older kernel be compatible with the new forecoming code? Bye 2018-03-21 18:24 GMT+01:00 Liu Bo

Re: [PATCH] btrfs: Allow non-privileged user to delete empty subvolume by default

On 03/21/2018 12:47 PM, Austin S. Hemmelgarn wrote: > I agree as well, with the addendum that I'd love to see a new ioctl that does > proper permissions checks.  While letting rmdir(2) work for an empty > subvolume with the appropriate permissions would be great (it will let rm -r > work

Re: Out of space and incorrect size reported

On Wed, Mar 21, 2018 at 09:53:39PM +, Shane Walton wrote: > > uname -a > Linux rockstor 4.4.5-1.el7.elrepo.x86_64 #1 SMP Thu Mar 10 11:45:51 EST 2016 > x86_64 x86_64 x86_64 GNU/Linux > > > btrfs —version > btrfs-progs v4.4.1 > > > btrfs fi df /mnt2/pool_homes > Data, RAID1:

Re: [PATCH] btrfs-progs: Beautify owner when printing leaf/nodes

On Tue, Mar 20, 2018 at 08:52:22PM +0800, Qu Wenruo wrote: > > > On 2018年03月20日 16:45, Nikolay Borisov wrote: > > Currently we print the raw values of the owner field of leaf/nodes. > > This can result in output like the following: > > > > leaf 30490624 items 2 free space 16061 generation 4

Re: Crashes running btrfs scrub

On Tue, Mar 20, 2018 at 7:01 PM, Qu Wenruo wrote: > > > On 2018年03月21日 01:44, Mike Stevens wrote: >> 30 devices is really not that much, heck you get 90 disks top load JBOD storage chassis these days and BTRFS does sound like an attractive choice for things

Re: [PATCH] btrfs: Remove obsolete function declaration

On Wed, Mar 21, 2018 at 09:03:41AM +0200, Nikolay Borisov wrote: > This function was removed in 24bc2843edd5 ("btrfs: > Refactor __get_raid_index() to btrfs_bg_flags_to_raid_index()") what > remains is a defunct declaration. Remove it. > > Signed-off-by: Nikolay Borisov > ---

Re: [PATCH] btrfs: Remove obsolete function declaration

On Wed, Mar 21, 2018 at 03:16:25PM +0800, Qu Wenruo wrote: > > > On 2018年03月21日 15:03, Nikolay Borisov wrote: > > This function was removed in 24bc2843edd5 ("btrfs: > > Refactor __get_raid_index() to btrfs_bg_flags_to_raid_index()") what > > remains is a defunct declaration. Remove it. > > Nice

[PATCH] btrfs-progs: wipe all copies of the stale superblock

Recovering from the other copies of the superblock is fundamental to BTRFS, which provides resilient against single LBA failure in the DUP group profile. Further, in the test case [1] it shows a good but stale superblock at copy#2. This will lead to confusion during auto/manual recovery. So

Re: [PATCH] btrfs: Remove obsolete function declaration

On 21.03.2018 09:16, Qu Wenruo wrote: > > > On 2018年03月21日 15:03, Nikolay Borisov wrote: >> This function was removed in 24bc2843edd5 ("btrfs: >> Refactor __get_raid_index() to btrfs_bg_flags_to_raid_index()") what >> remains is a defunct declaration. Remove it. > > Nice find, thanks for get

Re: [PATCH] btrfs: Allow non-privileged user to delete empty subvolume by default

On 20.03.2018 22:06, Goffredo Baroncelli wrote: > On 03/20/2018 07:45 AM, Misono, Tomohiro wrote: >> Deletion of subvolume by non-privileged user is completely restricted >> by default because we can delete a subvolume even if it is not empty >> and may cause data loss. In other words, when

Re: [PATCH] btrfs: Remove obsolete function declaration

On 2018年03月21日 15:38, Nikolay Borisov wrote: > > > On 21.03.2018 09:16, Qu Wenruo wrote: >> >> >> On 2018年03月21日 15:03, Nikolay Borisov wrote: >>> This function was removed in 24bc2843edd5 ("btrfs: >>> Refactor __get_raid_index() to btrfs_bg_flags_to_raid_index()") what >>> remains is a

[PATCH 0/8] Delayed refs cleanups/streamlining

This patchkit aims to simplify and streamline the logic involved in initialising and adding delayed refs/delayed head structures which deal with modification of the extent tree. Currently the logic for init and add was contained in one function for each type of structure. This resulted in very

[PATCH 2/8] btrfs: Use init_delayed_ref_common in add_delayed_tree_ref

Use the newly introduced common helper. No functional changes Signed-off-by: Nikolay Borisov --- fs/btrfs/delayed-ref.c | 35 +++ 1 file changed, 11 insertions(+), 24 deletions(-) diff --git a/fs/btrfs/delayed-ref.c b/fs/btrfs/delayed-ref.c

[PATCH 4/8] btrfs: Open-code add_delayed_tree_ref

Now that the initialization part and the critical section code have been split it's a lot easier to open code add_delayed_tree_ref. Do so in the following manner: 1. The commin init code is put immediately after memory-to-be-init is allocate, followed by the ref-specific member initialization.

[PATCH 7/8] btrfs: Use init_delayed_ref_head in add_delayed_ref_head

Use the newly introduced function when initialising the head_ref in add_delayed_ref_head. No functional changes. Signed-off-by: Nikolay Borisov --- fs/btrfs/delayed-ref.c | 62 -- 1 file changed, 4 insertions(+), 58 deletions(-)

[PATCH 8/8] btrfs: split delayed ref head initialization and addition

add_delayed_ref_head really performed 2 independent operations - initialisting the ref head and adding it to a list. Now that the init part is in a separate function let's complete the separation between both operations. This results in a lot simpler interface for add_delayed_ref_head since the

[PATCH 1/8] btrfs: Factor out common delayed refs init code

THe majority of the init code for struct btrfs_delayed_ref_node is duplicated in add_delayed_data_ref and add_delayed_tree_ref. Factor out the common bits in init_delayed_ref_common. This function is going to be used in future patches to clean that up. No functional changes Signed-off-by: Nikolay

[PATCH 3/8] btrfs: Use init_delayed_ref_common in add_delayed_data_ref

Use the newly introduced helper and remove the duplicate code. No functional changes Signed-off-by: Nikolay Borisov --- fs/btrfs/delayed-ref.c | 34 ++ 1 file changed, 10 insertions(+), 24 deletions(-) diff --git a/fs/btrfs/delayed-ref.c

[PATCH 5/8] btrfs: Open-code add_delayed_data_ref

Now that the initialization part and the critical section code have been split it's a lot easier to open code add_delayed_data_ref. Do so in the following manner: 1. The common init function is put immediately after memory-to-be-init is allocated, followed by the specific data ref initialization.

[PATCH 6/8] btrfs: Introduce init_delayed_ref_head

add_delayed_ref_head implements the logic to both initialize a head_ref structure as well as perform the necessary operations to add it to the delayed ref machinery. This has resulted in a very cumebrsome interface with loads of parameters and code, which at first glance, looks very unwieldy.

Re: [PATCH 1/2] btrfs: remove dead create_space_info calls

On 20.03.2018 21:25, je...@suse.com wrote: > From: Jeff Mahoney > > Since commit 2be12ef79 (btrfs: Separate space_info create/update), we've > separated out the creation and updating of the space info structures. > That commit was a straightforward refactoring of the two parts

[PATCH] btrfs: Remove obsolete function declaration

This function was removed in 24bc2843edd5 ("btrfs: Refactor __get_raid_index() to btrfs_bg_flags_to_raid_index()") what remains is a defunct declaration. Remove it. Signed-off-by: Nikolay Borisov --- David, This is a fixlet to the aforementioned patch which is only in your

Re: [PATCH v2 1/2] btrfs: cleanup btrfs_check_super_csum() to check the csum_type to the type

On 21.03.2018 01:33, Anand Jain wrote: > %csum_type is being checked to check the number of csum types we support, > which is 1 as of now. Instead just check if the type matches with the > only type we support, that is BTRFS_CSUM_TYPE_CRC32. And further adds > cleanup. > > Signed-off-by: Anand

Re: [PATCH v2 2/2] btrfs: return required error from btrfs_check_super_csum

On 21.03.2018 01:33, Anand Jain wrote: > Return the required -EINVAL and -EUCLEAN from the function > btrfs_check_super_csum(). And more the error log into the > parent function. > > Signed-off-by: Anand Jain > --- > fs/btrfs/disk-io.c | 28 +---

Re: [PATCH 2/2] btrfs: defer adding raid type kobject until after chunk relocation

On 20.03.2018 21:25, je...@suse.com wrote: > From: Jeff Mahoney > > Any time the first block group of a new type is created, we add a new > kobject to sysfs to hold the attributes for that type. Kobject-internal > allocations always use GFP_KERNEL, making them prone to

Re: [PATCH] btrfs: Remove obsolete function declaration

On 2018年03月21日 15:03, Nikolay Borisov wrote: > This function was removed in 24bc2843edd5 ("btrfs: > Refactor __get_raid_index() to btrfs_bg_flags_to_raid_index()") what > remains is a defunct declaration. Remove it. Nice find, thanks for get this. But strangely, I just fetched David's repo,

[PATCH 2/3] fstests: log-writes: Add support for METADATA flag

Signed-off-by: Qu Wenruo --- src/log-writes/log-writes.c | 3 ++- src/log-writes/log-writes.h | 9 + 2 files changed, 7 insertions(+), 5 deletions(-) diff --git a/src/log-writes/log-writes.c b/src/log-writes/log-writes.c index a872429d..5dc22c24 100644 ---

[PATCH 3/3] fstests: generic: Check the fs after each FUA writes

Basic test case which triggers fsstress with dm-log-writes, and then check the fs after each FUA writes. With needed infrastructure and special handlers for journal based fs. Signed-off-by: Qu Wenruo --- Unfortunately, neither xfs nor ext4 survies this test for even single

[PATCH 1/3] fstests: log-writes: Add support to output human readable flags

Also change the flag numeric output to hex. Signed-off-by: Qu Wenruo --- src/log-writes/log-writes.c | 70 - 1 file changed, 63 insertions(+), 7 deletions(-) diff --git a/src/log-writes/log-writes.c b/src/log-writes/log-writes.c index

[PATCH v3 1/2] btrfs: cleanup btrfs_check_super_csum() to check the csum_type to the type

%csum_type is being checked to check the number of csum types we support, which is 1 as of now. Instead just check if the type matches with the only type we support, that is BTRFS_CSUM_TYPE_CRC32. And further adds cleanup. Signed-off-by: Anand Jain --- fs/btrfs/disk-io.c

[PATCH v3 2/2] btrfs: return required error from btrfs_check_super_csum

Return the required -EINVAL and -EUCLEAN from the function btrfs_check_super_csum(). And more the error log into the parent function. Signed-off-by: Anand Jain --- fs/btrfs/disk-io.c | 25 ++--- 1 file changed, 14 insertions(+), 11 deletions(-) diff

[PATCH v3 0/2] Preparatory to add the csum check in the scan context

v2->v3: 1/2: Keep the check %(csum_type >= ARRAY_SIZE(btrfs_csum_sizes)) because it is inline to support future expansion, further btrfs-progs is already copied it. 2/2: Drop pr_err for default error which won't be possible. v1->v2: Merge 2/3 and 3/3 Use -EUCLEAN for csum

Re: [PATCH 3/3] fstests: generic: Check the fs after each FUA writes

On Wed, Mar 21, 2018 at 10:01 AM, Qu Wenruo wrote: > Basic test case which triggers fsstress with dm-log-writes, and then > check the fs after each FUA writes. > With needed infrastructure and special handlers for journal based fs. > > Signed-off-by: Qu Wenruo > ---

Re: [PATCH] btrfs: Allow non-privileged user to delete empty subvolume by default

On 2018-03-21 03:46, Nikolay Borisov wrote: On 20.03.2018 22:06, Goffredo Baroncelli wrote: On 03/20/2018 07:45 AM, Misono, Tomohiro wrote: Deletion of subvolume by non-privileged user is completely restricted by default because we can delete a subvolume even if it is not empty and may cause

Re: [PATCH] btrfs-progs: wipe all copies of the stale superblock

On 21.03.2018 12:19, Anand Jain wrote: > Recovering from the other copies of the superblock is fundamental > to BTRFS, which provides resilient against single LBA failure in nit: s/resilient/resilience. > the DUP group profile. Furthermore, the number of superblock copies doesn't really depend

Re: Out of space and incorrect size reported

Unfortunately this didn’t seem to correct the problem. Please see below: > uname -a Linux rockstor 4.12.4-1.el7.elrepo.x86_64 #1 SMP Thu Jul 27 20:03:28 EDT 2017 x86_64 x86_64 x86_64 GNU/Linux > btrfs —version btrfs-progs v4.12 > btrfs fi df -H /mnt2/pool_homes Data, RAID1: total=257.70GB,

Re: spurious full btrfs corruption

Just some addition on this: On Fri, 2018-03-16 at 01:03 +0100, Christoph Anton Mitterer wrote: > The issue that newer btrfs-progs/kernel don't restore anything at all > from my corrupted fs: 4.13.3 seems to be already buggy... 4.7.3 works, but interestingly btrfs-find-super seems to hang on it

Re: Crashes running btrfs scrub

On 2018年03月22日 01:13, Liu Bo wrote: > On Tue, Mar 20, 2018 at 7:01 PM, Qu Wenruo wrote: >> >> >> On 2018年03月21日 01:44, Mike Stevens wrote: >>> > 30 devices is really not that much, heck you get 90 disks top load JBOD > storage chassis these days and BTRFS does

Re: [PATCH v3 0/6] Fix long standing -EOPNOTSUPP problem caused by large inline extent

On 2018年03月21日 23:51, David Sterba wrote: > On Tue, Mar 20, 2018 at 02:42:23PM +0800, Qu Wenruo wrote: >> The patch is based on v4.15.1, and is designed to replace the old patch >> in devel branch. >> >> Kernel doesn't support dropping range inside inline extent, and prevents >> such thing

grub_probe/grub-mkimage does not find all drives in BTRFS RAID1

This is almost definitely a bug in GRUB, but I wanted to get the btrfs mailing list opinion first. Symptoms: I have a btrfs raid1 /boot and root filesystem. Ever since I replaced a drive, when I run the grub utilities to create my grub.cfg and install to boot sector, it only recognizes one of

Re: [PATCH 3/3] fstests: generic: Check the fs after each FUA writes

On Wed, Mar 21, 2018 at 02:22:29PM +0200, Amir Goldstein wrote: > > + > > +_log_writes_mount > > +$FSSTRESS_PROG $fsstress_args > /dev/null 2>&1 > > You should run fsstress with run_check() so output will go to $seqres.full > this way if you are able to catch a bug, you can take the random seed >

[PATCH V2] Btrfs: replace: cache rbio when rebuild data on missing device

Rebuild on missing device is as same as recover, after it's done, rbio has data which is consistent with on-disk data, so it can be cached to avoid further reads. Signed-off-by: Liu Bo Signed-off-by: Liu Bo --- v2: Add comments to explain why

Re: [PATCH] btrfs-progs: inspect-dump-tree: add '-f|--follow' options to print all children tree blocks for '-b'

On Tue, Mar 20, 2018 at 01:19:21PM +0800, Qu Wenruo wrote: > > > On 2018年03月20日 02:44, David Sterba wrote: > > On Wed, Mar 14, 2018 at 09:05:38AM +0800, Qu Wenruo wrote: > >> When debuging with "btrfs inspect dump-tree", it's not that handy if we > >> want to iterate all child tree blocks

Re: [PATCH v3 0/6] Fix long standing -EOPNOTSUPP problem caused by large inline extent

On Tue, Mar 20, 2018 at 02:42:23PM +0800, Qu Wenruo wrote: > The patch is based on v4.15.1, and is designed to replace the old patch > in devel branch. > > Kernel doesn't support dropping range inside inline extent, and prevents > such thing happening by limiting max inline extent size to >

Status of RAID5/6

Hi all I am trying to understand the status of RAID5/6 in BTRFS I know that there are some discussion ongoing on the RFC patch proposed by Liu bo But it seems that everything stopped last summary. Also it mentioned about a "separate disk for journal", does it mean that the final implementation of