[PATCH 3/3] btrfs: keep trim from interfering with transaction commits

2018-09-06 Thread jeffm
From: Jeff Mahoney Commit 499f377f49f08 (btrfs: iterate over unused chunk space in FITRIM) fixed free space trimming, but introduced latency when it was running. This is due to it pinning the transaction using both a incremented refcount and holding the commit root sem for the duration of a

[PATCH 1/3] btrfs: use ->devices list instead of ->alloc_list in btrfs_trim_fs

2018-09-06 Thread jeffm
From: Jeff Mahoney btrfs_trim_fs iterates over the fs_devices->alloc_list while holding the device_list_mutex. The problem is that ->alloc_list is protected by the chunk mutex. We don't want to hold the chunk mutex over the trim of the entire file system. Fortunately, the ->dev_list list is

[PATCH 0/3] btrfs: trim latency improvements

2018-09-06 Thread jeffm
From: Jeff Mahoney This patch set fixes a few issues with trim. 1) Fix device list iteration. We're iterating the ->alloc_list while holding the device_list_mutex. The ->alloc_list is protected by the chunk mutex and we don't want to hold it across the entire trim execution.

[PATCH 2/3] btrfs: don't attempt to trim devices that don't support it

2018-09-06 Thread jeffm
From: Jeff Mahoney We check whether any device the file system is using supports discard in the ioctl call, but then we attempt to trim free extents on every device regardless of whether discard is supported. Due to the way we mask off EOPNOTSUPP, we can end up issuing the trim operations on

[PATCH RESEND] btrfs: fix error handling in free_log_tree

2018-09-06 Thread jeffm
From: Jeff Mahoney When we hit an I/O error in free_log_tree->walk_log_tree during file system shutdown we can crash due to there not being a valid transaction handle. Use btrfs_handle_fs_error when there's no transaction handle to use. BUG: unable to handle kernel NULL pointer dereference at

[PATCH] btrfs: fix error handling in btrfs_dev_replace_start

2018-09-06 Thread jeffm
From: Jeff Mahoney When we fail to start a transaction in btrfs_dev_replace_start, we leave dev_replace->replace_start set to STARTED but clear ->srcdev and ->tgtdev. Later, that can result in an Oops in btrfs_dev_replace_progress when having state set to STARTED or SUSPENDED implies that

[PATCH 03/18] btrfs-progs: constify pathnames passed as arguments

2018-05-16 Thread jeffm
From: Jeff Mahoney It's unlikely we're going to modify a pathname argument, so codify that and use const. Reviewed-by: Qu Wenruo Signed-off-by: Jeff Mahoney --- chunk-recover.c | 4 ++-- cmds-device.c | 2 +- cmds-fi-usage.c | 6 +++---

[PATCH v3 00/18] btrfs-progs: qgroups-usability

2018-05-16 Thread jeffm
From: Jeff Mahoney Changes since v2: - Updated Reviewed-by tags were provided. - Fixed the typoed commands that I commented on in the previous posting. - Dropped the btrfs_cleanup_root_info since it's unnecessary with the switch to libbtrfsutil. - Updated the qgroups pathname

[PATCH 05/18] btrfs-progs: qgroups: add pathname to show output

2018-05-16 Thread jeffm
From: Jeff Mahoney The btrfs qgroup show command currently only exports qgroup IDs, forcing the user to resolve which subvolume each corresponds to. This patch adds pathname resolution to qgroup show so that when the -P option is used, the last column contains the pathname of

[PATCH 14/18] btrfs-progs: pass cmd_struct to clean_args_no_options{,_relaxed}

2018-05-16 Thread jeffm
From: Jeff Mahoney Now that we have a cmd_struct everywhere, we can pass it to clean_args_no_options and have it resolve the usage string from it there. This is necessary for it to pass the cmd_struct to usage() in the next patch. Signed-off-by: Jeff Mahoney

[PATCH 01/18] btrfs-progs: quota: Add -W option to rescan to wait without starting rescan

2018-05-16 Thread jeffm
From: Jeff Mahoney This patch adds a new -W option to wait for a rescan without starting a new operation. This is useful for things like xfstests where we want do to do a "btrfs quota enable" and not continue until the subsequent rescan has finished. In addition to documenting

[PATCH 11/18] btrfs-progs: filesystem balance: split out special handling

2018-05-16 Thread jeffm
From: Jeff Mahoney In preparation to use cmd_struct as the command entry point, we need to split out the 'filesystem balance' handling to not call cmd_balance directly. The reason is that the flags that indicate a command is hidden are a part of cmd_struct and so we can use a

[PATCH 18/18] btrfs-progs: qgroups: don't print dead qgroups

2018-05-16 Thread jeffm
From: Jeff Mahoney When qgroup items get left behind, we still print them in 'btrfs qgroup show' even though there is nothing to show. Since we now look up the pathname and that means we look up the subvolume, we can filter out first-level qgroups that correspond to roots that

[PATCH 15/18] btrfs-progs: pass cmd_struct to usage()

2018-05-16 Thread jeffm
From: Jeff Mahoney Now that every call site has a cmd_struct, we can just pass the cmd_struct to usage to print the usager information. This allows us to interpret the format flags we'll add later in this series to inform the user of which output formats any given command

[PATCH 13/18] btrfs-progs: pass cmd_struct to command callback function

2018-05-16 Thread jeffm
From: Jeff Mahoney This patch passes the cmd_struct to the command callback function. This has several purposes: It allows the command callback to identify which command was used to call it. It also gives us direct access to the usage associated with that command.

[PATCH 04/18] btrfs-progs: btrfs-list: add rb_entry helpers for root_info

2018-05-16 Thread jeffm
From: Jeff Mahoney We use rb_entry all over the place for the root_info pointers. Add a helper to make the code more readable. Signed-off-by: Jeff Mahoney --- btrfs-list.c | 30 -- 1 file changed, 20 insertions(+), 10 deletions(-)

[PATCH 08/18] btrfs-progs: subvolume: add quota info to btrfs sub show

2018-05-16 Thread jeffm
From: Jeff Mahoney This patch reports on the first-level qgroup, if any, associated with a particular subvolume. It displays the usage and limit, subject to the usual unit parameters. Signed-off-by: Jeff Mahoney --- cmds-subvolume.c | 51

[PATCH 10/18] btrfs-progs: reorder placement of help declarations for send/receive

2018-05-16 Thread jeffm
From: Jeff Mahoney The usage definitions for send and receive follow the command definitions, which use them. This works because we declare them in commands.h. When we move to using cmd_struct as the entry point, these declarations will be removed, breaking the commands. Since

[PATCH 12/18] btrfs-progs: use cmd_struct as command entry point

2018-05-16 Thread jeffm
From: Jeff Mahoney Rather than having global command usage and callbacks used to create cmd_structs in the command array, establish the cmd_struct structures separately and use those. The next commit in the series passes the cmd_struct to the command callbacks such that we can

[PATCH 17/18] btrfs-progs: handle command groups directly for common case

2018-05-16 Thread jeffm
From: Jeff Mahoney Most command groups just pass their own command group to handle_command_group. We can remove the explicit definitions of command group callbacks by passing the cmd_struct to handle_command_group and allowing it to resolve the group from it. Signed-off-by:

[PATCH 09/18] btrfs-progs: help: convert ints used as bools to bool

2018-05-16 Thread jeffm
From: Jeff Mahoney We use an int for 'full', 'all', and 'err' when we really mean a boolean. Reviewed-by: Qu Wenruo Signed-off-by: Jeff Mahoney --- btrfs.c | 14 +++--- help.c | 25 + help.h | 4 ++-- 3 files

[PATCH 07/18] btrfs-progs: qgroups: introduce btrfs_qgroup_query

2018-05-16 Thread jeffm
From: Jeff Mahoney The only mechanism we have in the progs for searching qgroups is to load all of them and filter the results. This works for qgroup show but to add quota information to 'btrfs subvoluem show' it's pretty wasteful. This patch splits out setting up the search

[PATCH 06/18] btrfs-progs: qgroups: introduce and use info and limit structures

2018-05-16 Thread jeffm
From: Jeff Mahoney We use structures to pass the info and limit from the kernel as items but store the individual values separately in btrfs_qgroup. We already have a btrfs_qgroup_limit structure that's used for setting the limit. This patch introduces a btrfs_qgroup_info

[PATCH 16/18] btrfs-progs: add support for output formats

2018-05-16 Thread jeffm
From: Jeff Mahoney This adds a global --format option to request extended output formats from each command. Most of it is plumbing a new cmd_context structure that's established at the beginning of argument parsing into the command callbacks. That structure currently only

[PATCH 02/18] btrfs-progs: qgroups: fix misleading index check

2018-05-16 Thread jeffm
From: Jeff Mahoney In print_single_qgroup_table we check the loop index against BTRFS_QGROUP_CHILD, but what we really mean is "last column." Since we have an enum value to indicate the last value, use that instead of assuming that BTRFS_QGROUP_CHILD is always last.

[PATCH 2/3] btrfs: qgroups, remove unnecessary memset before btrfs_init_work

2018-05-02 Thread jeffm
From: Jeff Mahoney btrfs_init_work clears the work struct except for ->wq, so the memset before calling btrfs_init_work in qgroup_rescan_init is unnecessary. We'll also initialize ->wq in btrfs_init_work so that it's obvious. Signed-off-by: Jeff Mahoney ---

[PATCH 3/3] btrfs: qgroup, don't try to insert status item after ENOMEM in rescan worker

2018-05-02 Thread jeffm
From: Jeff Mahoney If we fail to allocate memory for a path, don't bother trying to insert the qgroup status item. We haven't done anything yet and it'll fail also. Just print an error and be done with it. Signed-off-by: Jeff Mahoney --- fs/btrfs/qgroup.c | 9

[PATCH v3 0/3] btrfs: qgroup rescan races (part 1)

2018-05-02 Thread jeffm
From: Jeff Mahoney Hi Dave - Here's the updated patchset for the rescan races. This fixes the issue where we'd try to start multiple workers. It introduces a new "ready" bool that we set during initialization and clear while queuing the worker. The queuer is also now

[PATCH 1/3] btrfs: qgroups, fix rescan worker running races

2018-05-02 Thread jeffm
From: Jeff Mahoney Commit 8d9eddad194 (Btrfs: fix qgroup rescan worker initialization) fixed the issue with BTRFS_IOC_QUOTA_RESCAN_WAIT being racy, but ended up reintroducing the hang-on-unmount bug that the commit it intended to fix addressed. The race this time is between

[PATCH 2/3] btrfs-progs: build: autoconf 2.63 compatibility

2018-04-30 Thread jeffm
From: Jeff Mahoney Commit 2e1932e6a38 (btrfs-progs: build: simplify version tracking) started m4_chomp to strip the newlines from the version file. m4_chomp was introduced in autoconf 2.64 but SLE11 ships with autoconf 2.63. For purposes of just stripping the newline, m4_flatten

[PATCH 1/3] btrfs-progs: convert: fix support for e2fsprogs < 1.42

2018-04-30 Thread jeffm
From: Jeff Mahoney Commit 324d4c1857a (btrfs-progs: convert: Add larger device support) introduced new dependencies on the 64-bit API provided by e2fsprogs. That API was introduced in v1.42 (along with bigalloc). This patch maps the following to their equivalents in e2fsprogs <

[PATCH 3/3 v2] btrfs-progs: build: detect whether -std=gnu90 is supported

2018-04-30 Thread jeffm
From: Jeff Mahoney GCC releases prior to 4.5.0 don't support -std=gnu90 so btrfs-progs won't build at all on older distros. We can detect whether the compiler supports -std=gnu90 and fall back to -std=gnu89 if it doesn't. AX_CHECK_COMPILE_FLAG is the right way to do this, but

[PATCH 3/3] btrfs-progs: build: use m4_flatten instead of m4_chomp

2018-04-27 Thread jeffm
From: Jeff Mahoney Commit 2e1932e6a38 (btrfs-progs: build: simplify version tracking) started m4_chomp to strip the newlines from the version file. m4_chomp was introduced in autoconf 2.64 but SLE11 ships with autoconf 2.63. For purposes of just stripping the newline, m4_flatten

[PATCH 2/3] btrfs-progs: build: detect whether -std=gnu90 is supported

2018-04-27 Thread jeffm
From: Jeff Mahoney Older versions of gcc don't support -std=gnu90 so btrfs-progs won't build at all on older distros. We can detect whether the compiler supports -std=gnu90 and fall back to -std=gnu89 if it doesn't. Signed-off-by: Jeff Mahoney --- Makefile

[PATCH 1/3] btrfs-progs: convert: fix support for e2fsprogs < 1.42

2018-04-27 Thread jeffm
From: Jeff Mahoney Commit 324d4c1857a (btrfs-progs: convert: Add larger device support) introduced new dependencies on the 64-bit API provided by e2fsprogs. That API was introduced in v1.42 (along with bigalloc). This patch maps the following to their equivalents in e2fsprogs <

[PATCH 0/3] btrfs-progs: build on SLE11

2018-04-27 Thread jeffm
From: Jeff Mahoney This patch set allows btrfs-progs to build without further patching on a stock SLE11 SP4 install. The major issues were: - new dependencies on e2fsprogs 1.42 - use of -std=gnu90, which gcc 4.3.4 doesn't support - use of m4_chomp, introduced in autoconf 2.64.

[PATCH 2/3] btrfs: qgroups, remove unnecessary memset before btrfs_init_work

2018-04-26 Thread jeffm
From: Jeff Mahoney btrfs_init_work clears the work struct except for ->wq, so the memset before calling btrfs_init_work in qgroup_rescan_init is unnecessary. We'll also initialize ->wq in btrfs_init_work so that it's obvious. Signed-off-by: Jeff Mahoney ---

[PATCH 1/3] btrfs: qgroups, fix rescan worker running races

2018-04-26 Thread jeffm
From: Jeff Mahoney Commit d2c609b834d6 (Btrfs: fix qgroup rescan worker initialization) fixed the issue with BTRFS_IOC_QUOTA_RESCAN_WAIT being racy, but ended up reintroducing the hang-on-unmount bug that the commit it intended to fix addressed. The race this time is between

[PATCH 3/3] btrfs: qgroup, don't try to insert status item after ENOMEM in rescan worker

2018-04-26 Thread jeffm
From: Jeff Mahoney If we fail to allocate memory for a path, don't bother trying to insert the qgroup status item. We haven't done anything yet and it'll fail also. Just print an error and be done with it. Signed-off-by: Jeff Mahoney --- fs/btrfs/qgroup.c | 2

[PATCH 2/2] btrfs: defer adding raid type kobject until after chunk relocation

2018-03-20 Thread jeffm
From: Jeff Mahoney Any time the first block group of a new type is created, we add a new kobject to sysfs to hold the attributes for that type. Kobject-internal allocations always use GFP_KERNEL, making them prone to fs-reclaim races. While it appears as if this can occur any

[PATCH 1/2] btrfs: remove dead create_space_info calls

2018-03-20 Thread jeffm
From: Jeff Mahoney Since commit 2be12ef79 (btrfs: Separate space_info create/update), we've separated out the creation and updating of the space info structures. That commit was a straightforward refactoring of the two parts of update_space_info, but we can go a step further.

[PATCH] btrfs: fix lockdep splat in btrfs_alloc_subvolume_writers

2018-03-16 Thread jeffm
From: Jeff Mahoney While running btrfs/011, I hit the following lockdep splat. This is the important bit: pcpu_alloc+0x1ac/0x5e0 __percpu_counter_init+0x4e/0xb0 btrfs_init_fs_root+0x99/0x1c0 [btrfs] btrfs_get_fs_root.part.54+0x5b/0x150 [btrfs]

[PATCH 06/20] btrfs-progs: qgroups: add pathname to show output

2018-03-07 Thread jeffm
From: Jeff Mahoney The btrfs qgroup show command currently only exports qgroup IDs, forcing the user to resolve which subvolume each corresponds to. This patch adds pathname resolution to qgroup show so that when the -P option is used, the last column contains the pathname of

[PATCH 02/20] btrfs-progs: qgroups: fix misleading index check

2018-03-07 Thread jeffm
From: Jeff Mahoney In print_single_qgroup_table we check the loop index against BTRFS_QGROUP_CHILD, but what we really mean is "last column." Since we have an enum value to indicate the last value, use that instead of assuming that BTRFS_QGROUP_CHILD is always last.

[PATCH 05/20] btrfs-progs: btrfs-list: add btrfs_cleanup_root_info

2018-03-07 Thread jeffm
From: Jeff Mahoney Currently we can pass back root_info structures to callers but have to free the strings manually. This adds a helper to do it and uses it in cmd_subvol_show. Signed-off-by: Jeff Mahoney --- btrfs-list.c | 18 +++---

[PATCH 12/20] btrfs-progs: filesystem balance: split out special handling

2018-03-07 Thread jeffm
From: Jeff Mahoney In preparation to use cmd_struct as the command entry point, we need to split out the 'filesystem balance' handling to not call cmd_balance directly. The reason is that the flags that indicate a command is hidden are a part of cmd_struct and so we can use a

[PATCH 11/20] btrfs-progs: reorder placement of help declarations for send/receive

2018-03-07 Thread jeffm
From: Jeff Mahoney The usage definitions for send and receive follow the command definitions, which use them. This works because we declare them in commands.h. When we move to using cmd_struct as the entry point, these declarations will be removed, breaking the commands. Since

[PATCH 03/20] btrfs-progs: constify pathnames passed as arguments

2018-03-07 Thread jeffm
From: Jeff Mahoney It's unlikely we're going to modify a pathname argument, so codify that and use const. Reviewed-by: Qu Wenruo Signed-off-by: Jeff Mahoney --- chunk-recover.c | 4 ++-- cmds-device.c | 2 +- cmds-fi-usage.c | 6 +++---

[PATCH 07/20] btrfs-progs: qgroups: introduce and use info and limit structures

2018-03-07 Thread jeffm
From: Jeff Mahoney We use structures to pass the info and limit from the kernel as items but store the individual values separately in btrfs_qgroup. We already have a btrfs_qgroup_limit structure that's used for setting the limit. This patch introduces a btrfs_qgroup_info

[PATCH 01/20] btrfs-progs: quota: Add -W option to rescan to wait without starting rescan

2018-03-07 Thread jeffm
From: Jeff Mahoney This patch adds a new -W option to wait for a rescan without starting a new operation. This is useful for things like xfstests where we want do to do a "btrfs quota enable" and not continue until the subsequent rescan has finished. In addition to documenting

[PATCH 09/20] btrfs-progs: subvolume: add quota info to btrfs sub show

2018-03-07 Thread jeffm
From: Jeff Mahoney This patch reports on the first-level qgroup, if any, associated with a particular subvolume. It displays the usage and limit, subject to the usual unit parameters. Signed-off-by: Jeff Mahoney --- cmds-subvolume.c | 50

[PATCH 04/20] btrfs-progs: btrfs-list: add rb_entry helpers for root_info

2018-03-07 Thread jeffm
From: Jeff Mahoney We use rb_entry all over the place for the root_info pointers. Add a helper to make the code more readable. Signed-off-by: Jeff Mahoney --- btrfs-list.c | 30 -- 1 file changed, 20 insertions(+), 10 deletions(-)

[PATCH 08/20] btrfs-progs: qgroups: introduce btrfs_qgroup_query

2018-03-07 Thread jeffm
From: Jeff Mahoney The only mechanism we have in the progs for searching qgroups is to load all of them and filter the results. This works for qgroup show but to add quota information to 'btrfs subvoluem show' it's pretty wasteful. This patch splits out setting up the search

[PATCH 17/20] btrfs-progs: add support for output formats

2018-03-07 Thread jeffm
From: Jeff Mahoney This adds a global --format option to request extended output formats from each command. Most of it is plumbing a new cmd_context structure that's established at the beginning of argument parsing into the command callbacks. That structure currently only

[PATCH 19/20] btrfs-progs: qgroups: add json output for usage command

2018-03-07 Thread jeffm
From: Jeff Mahoney One of the common requests I receive is for 'df' like facilities for subvolume usage. Really, the request is for monitoring tools to be able to understand when subvolumes may be approaching quota in the same manner traditional file systems approach ENOSPC.

[PATCH 20/20] btrfs-progs: handle command groups directly for common case

2018-03-07 Thread jeffm
From: Jeff Mahoney Most command groups just pass their own command group to handle_command_group. We can remove the explicit definitions of command group callbacks by passing the cmd_struct to handle_command_group and allowing it to resolve the group from it. Signed-off-by:

[PATCH 13/20] btrfs-progs: use cmd_struct as command entry point

2018-03-07 Thread jeffm
From: Jeff Mahoney Rather than having global command usage and callbacks used to create cmd_structs in the command array, establish the cmd_struct structures separately and use those. The next commit in the series passes the cmd_struct to the command callbacks such that we can

[PATCH 14/20] btrfs-progs: pass cmd_struct to command callback function

2018-03-07 Thread jeffm
From: Jeff Mahoney This patch passes the cmd_struct to the command callback function. This has several purposes: It allows the command callback to identify which command was used to call it. It also gives us direct access to the usage associated with that command.

[PATCH 16/20] btrfs-progs: pass cmd_struct to usage()

2018-03-07 Thread jeffm
From: Jeff Mahoney Now that every call site has a cmd_struct, we can just pass the cmd_struct to usage to print the usager information. This allows us to interpret the format flags we'll add later in this series to inform the user of which output formats any given command

[PATCH 18/20] btrfs-progs: add generic support for json output

2018-03-07 Thread jeffm
From: Jeff Mahoney This patch adds support for JSON and JSON-compat output. The latter is intended to be compatible with Javascript's integers being represented as 64-bit floats, with only 53 bits usable for the integer component. Compat mode output will post 64-bit integers as

[PATCH 15/20] btrfs-progs: pass cmd_struct to clean_args_no_options{,_relaxed}

2018-03-07 Thread jeffm
From: Jeff Mahoney Now that we have a cmd_struct everywhere, we can pass it to clean_args_no_options and have it resolve the usage string from it there. This is necessary for it to pass the cmd_struct to usage() in the next patch. Signed-off-by: Jeff Mahoney

[PATCH v2 00/20] btrfs-progs: qgroups usability

2018-03-07 Thread jeffm
From: Jeff Mahoney Thanks to Qu Wenruo, Nikolay Borisov, and Tomohiro Misono for taking the time to review my previous patchset. I've incorporated your suggestions into this version. Obviously this one is quite a bit longer than the first version. After I posted it, Dave and I

[PATCH 10/20] btrfs-progs: help: convert ints used as bools to bool

2018-03-07 Thread jeffm
From: Jeff Mahoney We use an int for 'full', 'all', and 'err' when we really mean a boolean. Signed-off-by: Jeff Mahoney --- btrfs.c | 14 +++--- help.c | 25 + help.h | 4 ++-- 3 files changed, 22 insertions(+), 21

[PATCH 3/8] btrfs-progs: constify pathnames passed as arguments

2018-03-02 Thread jeffm
From: Jeff Mahoney It's unlikely we're going to modify a pathname argument, so codify that and use const. Signed-off-by: Jeff Mahoney --- chunk-recover.c | 4 ++-- cmds-device.c | 2 +- cmds-fi-usage.c | 6 +++--- cmds-rescue.c | 4 ++-- send-utils.c| 4

[PATCH 6/8] btrfs-progs: qgroups: introduce btrfs_qgroup_query

2018-03-02 Thread jeffm
From: Jeff Mahoney The only mechanism we have in the progs for searching qgroups is to load all of them and filter the results. This works for qgroup show but to add quota information to 'btrfs subvoluem show' it's pretty wasteful. This patch splits out setting up the search

[PATCH 8/8] btrfs-progs: qgroups: export qgroups usage information as JSON

2018-03-02 Thread jeffm
From: Jeff Mahoney One of the common requests I receive is for 'df' like facilities for subvolume usage. Really, the request is for monitoring tools to be able to understand when subvolumes may be approaching quota in the same manner traditional file systems approach ENOSPC.

[PATCH 5/8] btrfs-progs: qgroups: introduce and use info and limit structures

2018-03-02 Thread jeffm
From: Jeff Mahoney We use structures to pass the info and limit from the kernel as items but store the individual values separately in btrfs_qgroup. We already have a btrfs_qgroup_limit structure that's used for setting the limit. This patch introduces a btrfs_qgroup_info

[PATCH 7/8] btrfs-progs: subvolume: add quota info to btrfs sub show

2018-03-02 Thread jeffm
From: Jeff Mahoney This patch reports on the first-level qgroup, if any, associated with a particular subvolume. It displays the usage and limit, subject to the usual unit parameters. Signed-off-by: Jeff Mahoney --- cmds-subvolume.c | 46

[PATCH 4/8] btrfs-progs: qgroups: add pathname to show output

2018-03-02 Thread jeffm
From: Jeff Mahoney The btrfs qgroup show command currently only exports qgroup IDs, forcing the user to resolve which subvolume each corresponds to. This patch adds pathname resolution to qgroup show so that when the -P option is used, the last column contains the pathname of

[PATCH 2/8] btrfs-progs: qgroups: fix misleading index check

2018-03-02 Thread jeffm
From: Jeff Mahoney In print_single_qgroup_table we check the loop index against BTRFS_QGROUP_CHILD, but what we really mean is "last column." Since we have an enum value to indicate the last value, use that instead of assuming that BTRFS_QGROUP_CHILD is always last.

[PATCH 1/8] btrfs-progs: quota: Add -W option to rescan to wait without starting rescan

2018-03-02 Thread jeffm
From: Jeff Mahoney This patch adds a new -W option to wait for a rescan without starting a new operation. This is useful for things like xfstests where we want do to do a "btrfs quota enable" and not continue until the subsequent rescan has finished. In addition to documenting

[PATCH 0/8] btrfs-progs: qgroups usability [corrected]

2018-03-02 Thread jeffm
From: Jeff Mahoney Hi all - The following series addresses some usability issues with the qgroups UI. 1) Adds -W option so we can wait on a rescan completing without starting one. 2) Adds qgroup information to 'btrfs subvolume show' 3) Adds a -P option to show pathnames for

[PATCH 4/8] btrfs-progs: qgroups: add pathname to show output

2018-03-02 Thread jeffm
From: Jeff Mahoney The btrfs qgroup show command currently only exports qgroup IDs, forcing the user to resolve which subvolume each corresponds to. This patch adds pathname resolution to qgroup show so that when the -P option is used, the last column contains the pathname of

[PATCH 8/8] btrfs-progs: qgroups: export qgroups usage information as JSON

2018-03-02 Thread jeffm
From: Jeff Mahoney One of the common requests I receive is for 'df' like facilities for subvolume usage. Really, the request is for monitoring tools to be able to understand when subvolumes may be approaching quota in the same manner traditional file systems approach ENOSPC.

[PATCH 1/8] btrfs-progs: quota: Add -W option to rescan to wait without starting rescan

2018-03-02 Thread jeffm
From: Jeff Mahoney This patch adds a new -W option to wait for a rescan without starting a new operation. This is useful for things like xfstests where we want do to do a "btrfs quota enable" and not continue until the subsequent rescan has finished. In addition to documenting

[PATCH 8/8] btrfs-progs: add quota info to btrfs sub show

2018-03-02 Thread jeffm
From: Jeff Mahoney This patch reports on the first-level qgroup, if any, associated with a particular subvolume. It displays the usage and limit, subject to the usual unit parameters. Signed-off-by: Jeff Mahoney --- cmds-subvolume.c | 46

[PATCH 5/8] btrfs-progs: qgroups: export qgroups usage information as JSON

2018-03-02 Thread jeffm
From: Jeff Mahoney One of the common requests I receive is for 'df' like facilities for subvolume usage. Really, the request is for monitoring tools to be able to understand when subvolumes may be approaching quota in the same manner traditional file systems approach ENOSPC.

[PATCH 6/8] btrfs-progs: qgroups: introduce btrfs_qgroup_query

2018-03-02 Thread jeffm
From: Jeff Mahoney The only mechanism we have in the progs for searching qgroups is to load all of them and filter the results. This works for qgroup show but to add quota information to 'btrfs subvoluem show' it's pretty wasteful. This patch splits out setting up the search

[PATCH 7/8] btrfs-progs: subvolume: add quota info to btrfs sub show

2018-03-02 Thread jeffm
From: Jeff Mahoney This patch reports on the first-level qgroup, if any, associated with a particular subvolume. It displays the usage and limit, subject to the usual unit parameters. Signed-off-by: Jeff Mahoney --- cmds-subvolume.c | 46

[PATCH 6/8] btrfs-progs: qgroups: introduce and use info and limit structures

2018-03-02 Thread jeffm
From: Jeff Mahoney We use structures to pass the info and limit from the kernel as items but store the individual values separately in btrfs_qgroup. We already have a btrfs_qgroup_limit structure that's used for setting the limit. This patch introduces a btrfs_qgroup_info

[PATCH 2/8] btrfs-progs: qgroups: fix misleading index check

2018-03-02 Thread jeffm
From: Jeff Mahoney In print_single_qgroup_table we check the loop index against BTRFS_QGROUP_CHILD, but what we really mean is "last column." Since we have an enum value to indicate the last value, use that instead of assuming that BTRFS_QGROUP_CHILD is always last.

[PATCH 5/8] btrfs-progs: qgroups: introduce and use info and limit structures

2018-03-02 Thread jeffm
From: Jeff Mahoney We use structures to pass the info and limit from the kernel as items but store the individual values separately in btrfs_qgroup. We already have a btrfs_qgroup_limit structure that's used for setting the limit. This patch introduces a btrfs_qgroup_info

[PATCH 3/8] btrfs-progs: constify pathnames passed as arguments

2018-03-02 Thread jeffm
From: Jeff Mahoney It's unlikely we're going to modify a pathname argument, so codify that and use const. Signed-off-by: Jeff Mahoney --- chunk-recover.c | 4 ++-- cmds-device.c | 2 +- cmds-fi-usage.c | 6 +++--- cmds-rescue.c | 4 ++-- send-utils.c| 4

[PATCH 7/8] btrfs-progs: qgroups: introduce btrfs_qgroup_query

2018-03-02 Thread jeffm
From: Jeff Mahoney The only mechanism we have in the progs for searching qgroups is to load all of them and filter the results. This works for qgroup show but to add quota information to 'btrfs subvoluem show' it's pretty wasteful. This patch splits out setting up the search

[PATCH 0/8] btrfs-progs: qgroups usability

2018-03-02 Thread jeffm
From: Jeff Mahoney Hi all - The following series addresses some usability issues with the qgroups UI. 1) Adds -W option so we can wait on a rescan completing without starting one. 2) Adds qgroup information to 'btrfs subvolume show' 3) Adds a -P option to show pathnames for

[PATCH] btrfs: qgroups, properly handle no reservations

2018-02-21 Thread jeffm
From: Jeff Mahoney There are several places where we call btrfs_qgroup_reserve_meta and assume that a return value of 0 means that the reservation was successful. Later, we use the original bytes value passed to that call to free bytes during error handling or to pass the number

[PATCH] btrfs: use kvzalloc to allocate btrfs_fs_info

2018-02-15 Thread jeffm
From: Jeff Mahoney The srcu_struct in btrfs_fs_infoa scales in size with NR_CPUS. On kernels built with NR_CPUS=8192, this can result in kmalloc failures that prevent mounting. There is work in progress to try to resolve this for every user of srcu_struct but using kvzalloc

[PATCH] btrfs: fix missing error return in btrfs_drop_snapshot

2017-12-04 Thread jeffm
From: Jeff Mahoney If btrfs_del_root fails in btrfs_drop_snapshot, we'll pick up the error but then return 0 anyway due to mixing err and ret. Fixes: 79787eaab4612 ("btrfs: replace many BUG_ONs with proper error handling") Cc: # v3.4+ Signed-off-by: Jeff

[PATCH] btrfs: handle errors while updating refcounts in update_ref_for_cow

2017-11-21 Thread jeffm
From: Jeff Mahoney Since commit fb235dc06fa (btrfs: qgroup: Move half of the qgroup accounting time out of commit trans) the assumption that btrfs_add_delayed_{data,tree}_ref can only return 0 or -ENOMEM has been false. The qgroup operations call into btrfs_search_slot and

[PATCH] btrfs: pass fs_info to routines that always take tree_root

2017-08-02 Thread jeffm
From: Jeff Mahoney btrfs_find_root and btrfs_del_root always use the tree_root. Let's pass fs_info instead. Signed-off-by: Jeff Mahoney --- fs/btrfs/ctree.h | 7 --- fs/btrfs/disk-io.c | 2 +- fs/btrfs/extent-tree.c | 4 ++--

[PATCH 0/5 v2] btrfs-progs: convert fixes + reiserfs support

2017-07-27 Thread jeffm
From: Jeff Mahoney Changes since v1: - reiserfs conversion: - use bool instead of int - catch 'impossible' condition of multiple discontiguous tails - properly handle hole followed by tail - add testing for combinations of real blocks, tails, and holes - print error

[PATCH 4/5] btrfs-progs: tests: fix typo in convert-tests/008-readonly-image

2017-07-27 Thread jeffm
From: Jeff Mahoney The dd in convert-tests/008-readonly-image is expected to fail, so there being a typo in the file name has gone unnoticed. Signed-off-by: Jeff Mahoney --- tests/convert-tests/008-readonly-image/test.sh | 4 +++- 1 file changed, 3

[PATCH 5/5] btrfs-progs: convert: add support for converting reiserfs

2017-07-27 Thread jeffm
From: Jeff Mahoney This patch adds support to convert reiserfs file systems in-place to btrfs. It will convert extended attribute files to btrfs extended attributes, translate ACLs, coalesce tails that consist of multiple items into one item, and convert tails that are too big

[PATCH 3/5] btrfs-progs: convert: use search_cache_extent in migrate_one_reserved_range

2017-07-27 Thread jeffm
From: Jeff Mahoney When we are looking for extents in migrate_one_reserved_range, it's likely that there will be multiple extents that fall into the 0-1MB range. If lookup_cache_extent is called with a range that covers multiple cache entries, it will return the first entry it

[PATCH 1/5] btrfs-progs: convert: properly handle reserved ranges while iterating files

2017-07-27 Thread jeffm
From: Jeff Mahoney Commit 522ef705e38 (btrfs-progs: convert: Introduce function to calculate the available space) changed how we handle migrating file data so that we never have btrfs space associated with the reserved ranges. This works pretty well and when we iterate over the

[PATCH 2/5] btrfs-progs: convert: add missing newlines for printfs

2017-07-27 Thread jeffm
From: Jeff Mahoney There are two printfs with missing newlines that end up making the output wonky. Signed-off-by: Jeff Mahoney --- convert/main.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/convert/main.c b/convert/main.c index

[PATCH 1/3] btrfs-progs: convert: properly handle reserved ranges while iterating files

2017-07-25 Thread jeffm
From: Jeff Mahoney Commit 522ef705e38 (btrfs-progs: convert: Introduce function to calculate the available space) changed how we handle migrating file data so that we never have btrfs space associated with the reserved ranges. This works pretty well and when we iterate over the

[PATCH 3/3] btrfs-progs: convert: add support for converting reiserfs

2017-07-25 Thread jeffm
From: Jeff Mahoney This patch adds support to convert reiserfs file systems in-place to btrfs. It will convert extended attribute files to btrfs extended attributes, translate ACLs, coalesce tails that consist of multiple items into one item, and convert tails that are too big

[PATCH 2/3] btrfs-progs: convert: add missing newlines for printfs

2017-07-25 Thread jeffm
From: Jeff Mahoney There are two printfs with missing newlines that end up making the output wonky. Signed-off-by: Jeff Mahoney --- convert/main.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/convert/main.c b/convert/main.c index

[PATCH 1/7] btrfs-progs: check: supplement extent backref list with rbtree

2017-07-25 Thread jeffm
From: Jeff Mahoney For the pathlogical case, like xfstests generic/297 that creates a large file consisting of one, repeating reflinked extent, fsck can take hours. The root cause is that calling find_data_backref while iterating the extent records is an O(n^2) algorithm. For

  1   2   3   >