[PATCH v5] btrfs: handle dynamically reappearing missing device

If the device is not present at the time of (-o degrade) mount, the mount context will create a dummy missing struct btrfs_device. Later this device may reappear after the FS is mounted and then device is included in the device list but it missed the open_device part. So this patch handles that

Re: [PATCH v2] btrfs-progs: qgroup: Remove support for specifying inherit type

On 20.12.2017 07:19, Qu Wenruo wrote: > Since kernel is deprecating the support for specifying inherit type, > remove the support in btrfs-progs too. > > Thankfully, the options for qgroup inheritance is hidden and not > documented, so user shouldn't be affected at all. > > Signed-off-by: Qu

[PATCH v4] btrfs: handle dynamically reappearing missing device

If the device is not present at the time of (-o degrade) mount, the mount context will create a dummy missing struct btrfs_device. Later this device may reappear after the FS is mounted and then device is included in the device list but it missed the open_device part. So this patch handles that

Re: [PATCH v10 1/5] add infrastructure for tagging functions as error injectable

On Tue, 19 Dec 2017 18:14:17 -0800 Alexei Starovoitov wrote: > On 12/18/17 10:29 PM, Masami Hiramatsu wrote: > >> > >> +#if defined(__KERNEL__) && !defined(__ASSEMBLY__) > >> +#ifdef CONFIG_BPF_KPROBE_OVERRIDE > > > > BTW, CONFIG_BPF_KPROBE_OVERRIDE is also confusable name. > >

Re: [RFC PATCH] btrfs: qgroup: Deprecate the ability to manually inherit rfer/excl numbers

On 19.12.2017 21:36, Omar Sandoval wrote: > On Tue, Dec 19, 2017 at 09:19:10PM +0200, Nikolay Borisov wrote: >> >> >> On 19.12.2017 21:01, Omar Sandoval wrote: >>> My only objection is that we shouldn't rename the field names in the >>> UAPI header. Let's just add a comment that the two counters

[PATCH] btrfs: optimize code converge mutex unlock

No functional change rearrange the mutex_unlock. Signed-off-by: Anand Jain --- fs/btrfs/transaction.c | 7 ++- 1 file changed, 2 insertions(+), 5 deletions(-) diff --git a/fs/btrfs/transaction.c b/fs/btrfs/transaction.c index 6348573e26a7..04f07144b45c 100644 ---

Re: Cannot balance, ENOSPC errors 4.14.2 vanilla kernel

Also, if you're OK to compile kernel, would you please try the following diff to help us to further enhance btrfs chunk allocator? The chunk allocator itself is designed to handle your case, so it should pick up the remaining devices and allocate new RAID10 chunk with the unallocated space. But

Re: Cannot balance, ENOSPC errors 4.14.2 vanilla kernel

On 2017年12月20日 13:00, Adam Bahe wrote: > I'm using raid10. Pretty much the same. Raid10 is RAID1 first then RAID0. For your 20 disks (well, quite amazing) layout, btrfs will try to allocate using all disks for RAID10. Any unfortunately, devid 7, 9, 12, 13, 14, 15 are already full. Normally

Re: [PATCH v2 00/17] btrfs-progs: lowmem check: avoid extents overwrite

On 2017年12月20日 12:57, Su Yue wrote: > This patchset can be fetched from my github: > https://github.com/Damenly/btrfs-progs/tree/lowmem_repair > based on kdave/devel. > > Thanks to Qu Wenruo's ideas and suggestions first. > Any suggestions about names of functions and variables are welcome. >

Re: [PATCH v2 09/17] btrfs-progs: lowmem check: start to remove parameters @trans in lowmem

On 2017年12月20日 12:57, Su Yue wrote: > Since extents can be avoid overwrite by excluding or new chunk > allocation. It's unnessesary to do all repairs in one transaction. > > This patch removes parameter @trans of repair_extent_data_item(). > repair_extent_data_item() calls

Re: [PATCH v2 07/17] btrfs-progs: lowmem check: introduce try_avoid_extents_overwrite()

On 2017年12月20日 12:57, Su Yue wrote: > Define a global enum extents_operation to record extents are pinned, > excluded or new chunk is allocated for extents. > Although global variable is not so graceful, it simplifies codes much. > > New function try_avoid_extents_overwrite() will try to mark

Re: [PATCH v2 06/17] btrfs-progs: lowmem check: introduce force_cow_in_new_chunk()

On 2017年12月20日 12:57, Su Yue wrote: > Introduce create_chunk_and_block_block_group() to allocate new chunk > and corresponding block group. > > The new function force_cow_in_new_chunk() first allocates new chunk > and records its start. > Then it modifies all metadata block groups cached and

Re: [PATCH v2 05/17] btrfs-progs: lowmem check: introduce modify_block_groups_cache()

On 2017年12月20日 12:57, Su Yue wrote: > Excluding or pining all metadata blocks is not time-efficient for large > storage filesystems. > Here is another way to mark all metadata block groups full and allocate > a new chunk for CoW. So new reservered extents never overwrite > extents. > >

[PATCH v2] btrfs-progs: qgroup: Remove support for specifying inherit type

Since kernel is deprecating the support for specifying inherit type, remove the support in btrfs-progs too. Thankfully, the options for qgroup inheritance is hidden and not documented, so user shouldn't be affected at all. Signed-off-by: Qu Wenruo --- changelog: v2: Remove the

[PATCH v2] btrfs: qgroup: Deprecate the ability to manually inherit rfer/excl numbers

btrfs_qgroup_inherit structure has two members, num_ref_copies and num_excl_copies, to info btrfs kernel modules to inherit (copy) rfer/excl numbers at snapshot/subvolume creation time. Since qgroup number is already hard to maintain for multi-level qgroup scenario, allowing user to manually

Re: Cannot balance, ENOSPC errors 4.14.2 vanilla kernel

I'm using raid10. On Tue, Dec 19, 2017 at 10:51 PM, Qu Wenruo wrote: > > > On 2017年12月20日 10:51, Adam Bahe wrote: >> Forgot to add, I should have plenty of space: >> >> Label: 'nas' uuid: 4fcd5725-b6c6-4d8a-9860-f2fc5474cbcb >> Total devices 20 FS bytes used

[PATCH v2 10/17] btrfs-progs: lowmem check: remove parameter @trans of delete_extent_item()

This patch removes the parameter @trans of delete_extent_item(). It calls try_avoid_extents_overwrite() and starts a transaction by itself. Note: This patch and next patches cause error in lowmem repair like: "Error: Commit_root already set when starting transaction". This error will disappear

[PATCH v2 11/17] btrfs-progs: lowmem check: remove parameter @trans of repair_chunk_item()

This patch removes parameter @trans of repair_chunk_item(). It calls try_avoid_extents_overwrite() and starts a transaction by itself. Note: This patch and next patches cause error in lowmem repair like: "Error: Commit_root already set when starting transaction". This error will disappear after

[PATCH v2 15/17] btrfs-progs: lowmem check: remove parameter @trans of check_btrfs_root()

Remove parameters @trans of delete_extent_item() and walk_down_tree_v2(). Signed-off-by: Su Yue --- cmds-check.c | 18 -- 1 file changed, 8 insertions(+), 10 deletions(-) diff --git a/cmds-check.c b/cmds-check.c index 90270f1f8f72..81e125b9315f 100644

[PATCH v2 17/17] btrfs-progs: lowmem check: end of removing parameters @trans in lowmem

Remove @trans in check_chunks_and_extents. After this patch, Lowmem repair should work again. Signed-off-by: Su Yue --- cmds-check.c | 12 1 file changed, 12 deletions(-) diff --git a/cmds-check.c b/cmds-check.c index b51fcfd8ffdb..3f409c967f5e 100644 ---

[PATCH v2 09/17] btrfs-progs: lowmem check: start to remove parameters @trans in lowmem

Since extents can be avoid overwrite by excluding or new chunk allocation. It's unnessesary to do all repairs in one transaction. This patch removes parameter @trans of repair_extent_data_item(). repair_extent_data_item() calls try_avoid_extents_overwrite() and starts a transaction by itself.

[PATCH v2 13/17] btrfs-progs: lowmem check: remove parameter @trans of check_leaf_items()

This patch removes parameter @trans of check_leaf_items(). Note: This patch and next patches cause error in lowmem repair like: "Error: Commit_root already set when starting transaction". This error will disappear after removing @trans finished. Signed-off-by: Su Yue

[PATCH v2 12/17] btrfs-progs: lowmem check: remove parameter @trans of repair_extent_item()

This patch removes parameter @trans of repair_extent_item(). It calls try_avoid_extents_overwrite() and starts a transaction by itself. Note: This patch and next patches cause error in lowmem repair like: "Error: Commit_root already set when starting transaction". This error will disappear after

[PATCH v2 07/17] btrfs-progs: lowmem check: introduce try_avoid_extents_overwrite()

Define a global enum extents_operation to record extents are pinned, excluded or new chunk is allocated for extents. Although global variable is not so graceful, it simplifies codes much. New function try_avoid_extents_overwrite() will try to mark block groups full and allocate a new chunk. If it

[PATCH v2 14/17] btrfs-progs: lowmem check: remove parameter @trans of repair_tree_back_ref()

This patch removes parameter @trans of repair_tree_back_ref(). It calls try_avoid_extents_overwrite() and starts a transaction by itself. Note: This patch and next patches cause error in lowmem repair like: "Error: Commit_root already set when starting transaction". This error will disappear

[PATCH v2 16/17] btrfs-progs: lowmem check: introduce repair_block_accounting()

Introduce repair_block_accounting() which calls btrfs_fix_block_accounting() to repair block group accouting. Replace btrfs_fix_block_accounting() with the new function. Signed-off-by: Su Yue --- cmds-check.c | 26 +- 1 file changed, 25

[PATCH v2 05/17] btrfs-progs: lowmem check: introduce modify_block_groups_cache()

Excluding or pining all metadata blocks is not time-efficient for large storage filesystems. Here is another way to mark all metadata block groups full and allocate a new chunk for CoW. So new reservered extents never overwrite extents. Introduce modify_block_groups_cache() to modify all blocks

[PATCH v2 04/17] btrfs-progs: lowmem check: exclude extents of metadata blocks

Commit d17d6663c99c ("btrfs-progs: lowmem check: Fix regression which screws up extent allocator") removes pin_metadata_blocks() from lowmem repair. So we have to find another way to exclude extents which should be occupied by tree blocks. Modify pin_down_tree_blocks() only for code reuse. So

[PATCH v2 08/17] btrfs-progs: lowmem check: exclude extents if init-extent-tree in lowmem

If options '--init-extent-tree' and '--mode=lowmem' are both input, all metadata blocks will be traversed twice. First one is done by pin_metadata_blocks() in reinit_extent_tree(). Second one is in check_chunks_and_extents_v2(). Excluding instead of pining metadata blocks before reinit extent

[PATCH v2 00/17] btrfs-progs: lowmem check: avoid extents overwrite

This patchset can be fetched from my github: https://github.com/Damenly/btrfs-progs/tree/lowmem_repair based on kdave/devel. Thanks to Qu Wenruo's ideas and suggestions first. Any suggestions about names of functions and variables are welcome. Patch[1-3] fix minor problems of lowmem repair.

[PATCH v2 06/17] btrfs-progs: lowmem check: introduce force_cow_in_new_chunk()

Introduce create_chunk_and_block_block_group() to allocate new chunk and corresponding block group. The new function force_cow_in_new_chunk() first allocates new chunk and records its start. Then it modifies all metadata block groups cached and full. Finally it marks the new block group uncached

[PATCH v2 03/17] btrfs-progs: lowmem check: assign @parent early in repair_extent_data_item()

The variable @eb is assigned to leaf in fs_tree before insertion of backref. It will causes wrong parent of new inserted backref. Set @parent in the begin solves the problem. Signed-off-by: Su Yue --- cmds-check.c | 10 +- 1 file changed, 5 insertions(+), 5

[PATCH v2 02/17] btrfs-progs: lowmem check: record returned errors after walk_down_tree_v2()

In lowmem mode with '--repair', check_chunks_and_extents_v2() will fix accounting in block groups and clear the error bit BG_ACCOUNTING_ERROR. However, return value of check_btrfs_root() is 0 either 1 instead of error bits. If extent tree is on error, lowmem repair always prints error and returns

[PATCH v2 01/17] btrfs-progs: lowmem check: release path in repair_extent_data_item()

In repair_extent_data_item(), path is not be released if some errors occurs which causes extent buffer leak. So release path in end of the function. Signed-off-by: Su Yue --- cmds-check.c | 1 + 1 file changed, 1 insertion(+) diff --git a/cmds-check.c b/cmds-check.c

Re: Cannot balance, ENOSPC errors 4.14.2 vanilla kernel

On 2017年12月20日 10:51, Adam Bahe wrote: > Forgot to add, I should have plenty of space: > > Label: 'nas' uuid: 4fcd5725-b6c6-4d8a-9860-f2fc5474cbcb > Total devices 20 FS bytes used 26.69TiB > devid1 size 3.64TiB used 3.28TiB path /dev/sdm > devid2 size 3.64TiB

Re: [PATCH RESEND 1/2] btrfs: handle volume split brain scenario

Suppose you have 3 devices in a raid1 and one goes missing.  The user can now, in trying to fix this, mount each of the two remaining devices separately (by accident of course), in which case you can still have a split brain situation. Austin, that's not possible a N disk raid1 can still

Re: Cannot balance, ENOSPC errors 4.14.2 vanilla kernel

Forgot to add, I should have plenty of space: Label: 'nas' uuid: 4fcd5725-b6c6-4d8a-9860-f2fc5474cbcb Total devices 20 FS bytes used 26.69TiB devid1 size 3.64TiB used 3.28TiB path /dev/sdm devid2 size 3.64TiB used 3.28TiB path /dev/sde devid3 size

Cannot balance, ENOSPC errors 4.14.2 vanilla kernel

I have been having ENOSPC errors on any btrfs device delete, btrfs balance, btrfs device add actions for awhile now. How do I fix this? I need to be able to remove devices and balance my filesystem again. [Tue Dec 19 15:25:26 2017] BTRFS info (device sdc): relocating block group 190774812082176

Re: [PATCH v10 1/5] add infrastructure for tagging functions as error injectable

On 12/18/17 10:29 PM, Masami Hiramatsu wrote: +#if defined(__KERNEL__) && !defined(__ASSEMBLY__) +#ifdef CONFIG_BPF_KPROBE_OVERRIDE BTW, CONFIG_BPF_KPROBE_OVERRIDE is also confusable name. Since this feature override a function to just return with some return value (as far as I understand, or

Re: [PATCH] btrfs-progs: qgroup: Remove support for specifying inherit type

On 2017年12月20日 08:34, Misono, Tomohiro wrote: > On 2017/12/19 19:53, Qu Wenruo wrote: >> Since kernel is deprecating the support for specifying inherit type, >> remove the support in btrfs-progs too. >> >> Thankfully, the options for qgroup inheritance is hidden and not >> documented, so user

Re: [PATCH] btrfs-progs: qgroup: Remove support for specifying inherit type

On 2017/12/19 19:53, Qu Wenruo wrote: > Since kernel is deprecating the support for specifying inherit type, > remove the support in btrfs-progs too. > > Thankfully, the options for qgroup inheritance is hidden and not > documented, so user shouldn't be affected at all. > > Signed-off-by: Qu

Re: Unexpected raid1 behaviour

On Tue, Dec 19, 2017 at 2:17 PM, Tomasz Pala wrote: > On Tue, Dec 19, 2017 at 12:47:33 -0700, Chris Murphy wrote: > >> The more verbose man pages are, the more likely it is that information >> gets stale. We already see this with the Btrfs Wiki. So are you > > True. The same

Re: Unexpected raid1 behaviour

On Tue, Dec 19, 2017 at 1:41 PM, Tomasz Pala wrote: > On Tue, Dec 19, 2017 at 12:35:20 -0700, Chris Murphy wrote: > >> with a read only file system. Another reason is the kernel code and >> udev rule for device "readiness" means the volume is not "ready" until >> all member

Re: Unexpected raid1 behaviour

On Tue, Dec 19, 2017 at 1:11 PM, Austin S. Hemmelgarn wrote: > On 2017-12-19 12:56, Tomasz Pala wrote: >> BTRFS lacks all of these - there are major functional changes in current >> kernels and it reaches far beyond LTS. All the knowledge YOU have here, >> on this maillist,

Re: Unexpected raid1 behaviour

On Tue, Dec 19, 2017 at 15:47:03 -0500, Austin S. Hemmelgarn wrote: >> Sth like this? I got such problem a few months ago, my solution was >> accepted upstream: >> https://github.com/systemd/systemd/commit/0e8856d25ab71764a279c2377ae593c0f2460d8f >> >> Rationale is in referred ticket, udev would

Re: Unexpected raid1 behaviour

On Tue, Dec 19, 2017 at 15:11:22 -0500, Austin S. Hemmelgarn wrote: > Except the systems running on those ancient kernel versions are not > necessarily using a recent version of btrfs-progs. Still much easier to update a userspace tools than kernel (consider binary drivers for various

Re: [PATCH v3 06/10] writeback: introduce super_operations->write_metadata

On Tue, Dec 19, 2017 at 01:07:09PM +0100, Jan Kara wrote: > On Wed 13-12-17 09:20:04, Dave Chinner wrote: > > On Tue, Dec 12, 2017 at 01:05:35PM -0500, Josef Bacik wrote: > > > On Tue, Dec 12, 2017 at 10:36:19AM +1100, Dave Chinner wrote: > > > > On Mon, Dec 11, 2017 at 04:55:31PM -0500, Josef

Re: [PATCH 1/4] Btrfs: btrfs_dedupe_file_range() ioctl, remove 16MiB restriction

On Tue, Dec 19, 2017 at 01:02:44PM +0300, Timofey Titovets wrote: > At now btrfs_dedupe_file_range() restricted to 16MiB range for > limit locking time and memory requirement for dedup ioctl() > > For too big input range code silently set range to 16MiB > > Let's remove that restriction by do

Re: Unexpected raid1 behaviour

On Tue, Dec 19, 2017 at 12:47:33 -0700, Chris Murphy wrote: > The more verbose man pages are, the more likely it is that information > gets stale. We already see this with the Btrfs Wiki. So are you True. The same applies to git documentation (3rd paragraph):

Re: Unexpected raid1 behaviour

On 2017-12-19 15:41, Tomasz Pala wrote: On Tue, Dec 19, 2017 at 12:35:20 -0700, Chris Murphy wrote: with a read only file system. Another reason is the kernel code and udev rule for device "readiness" means the volume is not "ready" until all member devices are present. And while the volume is

Re: Unexpected raid1 behaviour

On Tue, Dec 19, 2017 at 12:35:20 -0700, Chris Murphy wrote: > with a read only file system. Another reason is the kernel code and > udev rule for device "readiness" means the volume is not "ready" until > all member devices are present. And while the volume is not "ready" > systemd will not even

Re: Unexpected raid1 behaviour

On Tue, Dec 19, 2017 at 10:31:40 -0800, George Mitchell wrote: > I have significant experience as a user of raid1. I spent years using > software raid1 and then more years using hardware (3ware) raid1 and now > around 3 years using btrfs raid1. I have not found btrfs raid1 to be > less

Re: Unexpected raid1 behaviour

On 2017-12-19 12:56, Tomasz Pala wrote: On Tue, Dec 19, 2017 at 11:35:02 -0500, Austin S. Hemmelgarn wrote: 2. printed on screen when creating/converting "RAID1" profile (by btrfs tools), I don't agree on this one. It is in no way unreasonable to expect that someone has read the

Re: Unexpected raid1 behaviour

On Tue, Dec 19, 2017 at 10:56 AM, Tomasz Pala wrote: > On Tue, Dec 19, 2017 at 11:35:02 -0500, Austin S. Hemmelgarn wrote: > >>> 2. printed on screen when creating/converting "RAID1" profile (by btrfs >>> tools), >> I don't agree on this one. It is in no way unreasonable to

Re: [RFC PATCH] btrfs: qgroup: Deprecate the ability to manually inherit rfer/excl numbers

On Tue, Dec 19, 2017 at 11:36:58AM -0800, Omar Sandoval wrote: > On Tue, Dec 19, 2017 at 09:19:10PM +0200, Nikolay Borisov wrote: > > > > > > On 19.12.2017 21:01, Omar Sandoval wrote: > > > My only objection is that we shouldn't rename the field names in the > > > UAPI header. Let's just add a

Re: [RFC PATCH] btrfs: qgroup: Deprecate the ability to manually inherit rfer/excl numbers

On Tue, Dec 19, 2017 at 09:19:10PM +0200, Nikolay Borisov wrote: > > > On 19.12.2017 21:01, Omar Sandoval wrote: > > My only objection is that we shouldn't rename the field names in the > > UAPI header. Let's just add a comment that the two counters are ignored. > > Besides that, > > Why is

Re: Unexpected raid1 behaviour

On Tue, Dec 19, 2017 at 7:46 AM, Tomasz Pala wrote: >Secondly - permanent failures are not handled "just > fine", as there is (1) no automatic mount as degraded, so the machine > won't reboot properly and (2) the r/w degraded mount is[*] one-timer. > Again, this should be: One

Re: [RFC PATCH] btrfs: qgroup: Deprecate the ability to manually inherit rfer/excl numbers

On 19.12.2017 21:01, Omar Sandoval wrote: > My only objection is that we shouldn't rename the field names in the > UAPI header. Let's just add a comment that the two counters are ignored. > Besides that, Why is that? -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in

Re: [RFC PATCH] btrfs: qgroup: Deprecate the ability to manually inherit rfer/excl numbers

On Tue, Dec 19, 2017 at 06:45:11PM +0800, Qu Wenruo wrote: > btrfs_qgroup_inherit structure has two members, num_ref_copies and > num_excl_copies, to info btrfs kernel modules to inherit (copy) > rfer/excl numbers at snapshot/subvolume creation time. > > Since qgroup number is already hard to

Re: Unexpected raid1 behaviour

On 12/19/2017 06:46 AM, Tomasz Pala wrote: On Tue, Dec 19, 2017 at 07:25:49 -0500, Austin S. Hemmelgarn wrote: Well, the RAID1+ is all about the failing hardware. About catastrophically failing hardware, not intermittent failure. It shouldn't matter - as long as disk failing once is kicked

Re: Unexpected raid1 behaviour

On Tue, Dec 19, 2017 at 11:35:02 -0500, Austin S. Hemmelgarn wrote: >> 2. printed on screen when creating/converting "RAID1" profile (by btrfs >> tools), > I don't agree on this one. It is in no way unreasonable to expect that > someone has read the documentation _before_ trying to use

Re: kernel hangs during balance

On Tue, Dec 19, 2017 at 9:14 AM, Hans van Kranenburg wrote: > Just wild first guess... are you also using btrfs send/receive > functionality where the system having problems is the sending part? No. >>> Every night I'm getting a kernel hang, sometimes caught by

Re: kernel hangs during balance

On 12/19/2017 06:08 PM, Rich Rauenzahn wrote: > What's also confusing is I just ran a manual balance on the fs using > defaults (which are aggressive) and it completed with no problems. > It smells more like a race condition than a particular corruption. Just wild first guess... are you also

Re: [PATCH v3 19/19] fs: handle inode->i_version more efficiently

On Tue, 2017-12-19 at 10:29 +0100, Jan Kara wrote: > On Mon 18-12-17 12:22:20, Jeff Layton wrote: > > On Mon, 2017-12-18 at 17:34 +0100, Jan Kara wrote: > > > On Mon 18-12-17 10:11:56, Jeff Layton wrote: > > > > static inline bool > > > > inode_maybe_inc_iversion(struct inode *inode, bool force)

Re: kernel hangs during balance

What's also confusing is I just ran a manual balance on the fs using defaults (which are aggressive) and it completed with no problems. It smells more like a race condition than a particular corruption. On Tue, Dec 19, 2017 at 8:09 AM, Rich Rauenzahn wrote: > I'm running

Re: Unexpected raid1 behaviour

On 2017-12-19 09:46, Tomasz Pala wrote: On Tue, Dec 19, 2017 at 07:25:49 -0500, Austin S. Hemmelgarn wrote: Well, the RAID1+ is all about the failing hardware. About catastrophically failing hardware, not intermittent failure. It shouldn't matter - as long as disk failing once is kicked out

kernel hangs during balance

I'm running 4.4.106-1.el7.elrepo.x86_64 and I do a btrfs balance everynight. Every night I'm getting a kernel hang, sometimes caught by my watchdog, sometimes not. Last night's hang was on the balance of DATA on / at 70. I'm not sure how to further trace this down to help you -- the console by

Netdata support for monitoring of BTRFS volumes.

For those who are interested in better monitoring, netdata [1] [2] has just merged support for low-level usage monitoring of BTRFS volumes. It provides graphs for both physical on-disk usage by chunk type (both allocated and used) and per-chunk type graphs of usage as reported by `btrfs

Re: Unexpected raid1 behaviour

On Tue, Dec 19, 2017 at 07:25:49 -0500, Austin S. Hemmelgarn wrote: >> Well, the RAID1+ is all about the failing hardware. > About catastrophically failing hardware, not intermittent failure. It shouldn't matter - as long as disk failing once is kicked out of the array *if possible*. Or

Re: Unexpected raid1 behaviour

[ ... ] > The advantage of writing single chunks when degraded, is in > the case where a missing device returns (is readded, > intact). Catching up that device with the first drive, is a > manual but simple invocation of 'btrfs balance start > -dconvert=raid1,soft -mconvert=raid1,soft' The

Re: xfstests/btrfs/100 and 151 failure

> > I've attached my master config before compiling a kernel with it I just > do make olddefconfig to sync it with latest kbuild options. > >> Thanks for the config file, will check with this and update the status. Cheers, Lakshmipathi.G http://www.giis.co.in http://www.webminal.org -- To

Re: Unexpected raid1 behaviour

On Tue, Dec 19, 2017 at 1:28 AM, Chris Murphy wrote: > On Mon, Dec 18, 2017 at 1:49 AM, Anand Jain wrote: > >> Agreed. IMO degraded-raid1-single-chunk is an accidental feature >> caused by [1], which we should revert back, since.. >>- balance

Re: Unexpected raid1 behaviour

On 2017-12-18 17:01, Peter Grandi wrote: The fact is, the only cases where this is really an issue is if you've either got intermittently bad hardware, or are dealing with external Well, the RAID1+ is all about the failing hardware. storage devices. For the majority of people who are using

Re: Unexpected raid1 behaviour

On Mon, Dec 18, 2017 at 03:28:14PM -0700, Chris Murphy wrote: > On Mon, Dec 18, 2017 at 1:49 AM, Anand Jain wrote: > > Agreed. IMO degraded-raid1-single-chunk is an accidental feature > > caused by [1], which we should revert back, since.. > >- balance (to raid1

Re: Unexpected raid1 behaviour

On 2017-12-18 14:43, Tomasz Pala wrote: On Mon, Dec 18, 2017 at 08:06:57 -0500, Austin S. Hemmelgarn wrote: The fact is, the only cases where this is really an issue is if you've either got intermittently bad hardware, or are dealing with external Well, the RAID1+ is all about the failing

Re: [PATCH v3 06/10] writeback: introduce super_operations->write_metadata

On Mon 11-12-17 16:55:31, Josef Bacik wrote: > @@ -1621,12 +1647,18 @@ static long writeback_sb_inodes(struct super_block > *sb, >* background threshold and other termination conditions. >*/ > if (wrote) { > - if

Re: [PATCH v3 06/10] writeback: introduce super_operations->write_metadata

On Wed 13-12-17 09:20:04, Dave Chinner wrote: > On Tue, Dec 12, 2017 at 01:05:35PM -0500, Josef Bacik wrote: > > On Tue, Dec 12, 2017 at 10:36:19AM +1100, Dave Chinner wrote: > > > On Mon, Dec 11, 2017 at 04:55:31PM -0500, Josef Bacik wrote: > > > > From: Josef Bacik > > > > > > >

Re: [RFC PATCH] btrfs: qgroup: Deprecate the ability to manually inherit rfer/excl numbers

On 2017年12月19日 19:24, Nikolay Borisov wrote: > > > On 19.12.2017 13:20, Qu Wenruo wrote: >> Naming like pad1/2 will make the check in btrfs_qgroup_inherit() look >> quite weird. >> >> Although I don't have any better idea, so I'm mostly fine with such rename. > > Right, I saw the check now,

Re: [RFC PATCH] btrfs: qgroup: Deprecate the ability to manually inherit rfer/excl numbers

On 19.12.2017 13:20, Qu Wenruo wrote: > Naming like pad1/2 will make the check in btrfs_qgroup_inherit() look > quite weird. > > Although I don't have any better idea, so I'm mostly fine with such rename. Right, I saw the check now, missed it the first time. I think we should remove it and the

Re: [RFC PATCH] btrfs: qgroup: Deprecate the ability to manually inherit rfer/excl numbers

On 2017年12月19日 19:12, Nikolay Borisov wrote: > > > On 19.12.2017 12:45, Qu Wenruo wrote: >> btrfs_qgroup_inherit structure has two members, num_ref_copies and >> num_excl_copies, to info btrfs kernel modules to inherit (copy) >> rfer/excl numbers at snapshot/subvolume creation time. >> >>

Re: [RFC PATCH] btrfs: qgroup: Deprecate the ability to manually inherit rfer/excl numbers

On 19.12.2017 12:45, Qu Wenruo wrote: > btrfs_qgroup_inherit structure has two members, num_ref_copies and > num_excl_copies, to info btrfs kernel modules to inherit (copy) > rfer/excl numbers at snapshot/subvolume creation time. > > Since qgroup number is already hard to maintain for

[PATCH] btrfs-progs: qgroup: Remove support for specifying inherit type

Since kernel is deprecating the support for specifying inherit type, remove the support in btrfs-progs too. Thankfully, the options for qgroup inheritance is hidden and not documented, so user shouldn't be affected at all. Signed-off-by: Qu Wenruo --- cmds-subvolume.c | 21

[RFC PATCH] btrfs: qgroup: Deprecate the ability to manually inherit rfer/excl numbers

btrfs_qgroup_inherit structure has two members, num_ref_copies and num_excl_copies, to info btrfs kernel modules to inherit (copy) rfer/excl numbers at snapshot/subvolume creation time. Since qgroup number is already hard to maintain for multi-level qgroup scenario, allowing user to manually

[PATCH 1/4] Btrfs: btrfs_dedupe_file_range() ioctl, remove 16MiB restriction

At now btrfs_dedupe_file_range() restricted to 16MiB range for limit locking time and memory requirement for dedup ioctl() For too big input range code silently set range to 16MiB Let's remove that restriction by do iterating over dedup range. That's backward compatible and will not change

[PATCH 3/4] Btrfs: allow btrfs_defrag_file() uncompress files on defragmentation

Currently defrag ioctl only support recompress files with specified compression type. Allow set compression type to none, while call defrag, and use BTRFS_DEFRAG_RANGE_COMPRESS as flag, that user request change of compression type. Signed-off-by: Timofey Titovets ---

[PATCH 2/4] Btrfs: make should_defrag_range() understood compressed extents

Both, defrag ioctl and autodefrag - call btrfs_defrag_file() for file defragmentation. Kernel default target extent size - 256KiB. Btrfs progs default - 32MiB. Both bigger then maximum size of compressed extent - 128KiB. That lead to rewrite all compressed data on disk. Fix that by check

[PATCH 0/4] Btrfs: just bunch of patches to ioctl.c

1st patch, remove 16MiB restriction from extent_same ioctl(), by doing iterations over passed range. I did not see much difference in performance, so it's just remove logic restriction. 2-3 pathes, update defrag ioctl(): - Fix bad behaviour with full rewriting all compressed extents in

[PATCH 4/4] Btrfs: reduce size of struct btrfs_inode

Currently btrfs_inode have size equal 1136 bytes. (On x86_64). struct btrfs_inode store several vars releated to compression code, all states use 1 or 2 bits. Lets declare bitfields for compression releated vars, to reduce sizeof btrfs_inode to 1128 bytes. Signed-off-by: Timofey Titovets

Re: [PATCH v3 19/19] fs: handle inode->i_version more efficiently

On Mon 18-12-17 12:22:20, Jeff Layton wrote: > On Mon, 2017-12-18 at 17:34 +0100, Jan Kara wrote: > > On Mon 18-12-17 10:11:56, Jeff Layton wrote: > > > static inline bool > > > inode_maybe_inc_iversion(struct inode *inode, bool force) > > > { > > > - atomic64_t *ivp = (atomic64_t *)>i_version;