[PATCH 3/3] btrfs: keep trim from interfering with transaction commits

[jeffm]

From: Jeff Mahoney 

Commit 499f377f49f08 (btrfs: iterate over unused chunk space in FITRIM)
fixed free space trimming, but introduced latency when it was running.
This is due to it pinning the transaction using both a incremented
refcount and holding the commit root sem for the duration of a single
trim operation.

This was to ensure safety but it's unnecessary.  We already hold the the
chunk mutex so we know that the chunk we're using can't be allocated
while we're trimming it.

In order to check against chunks allocated already in this transaction,
we need to check the pending chunks list.  To to that safely without
joining the transaction (or attaching than then having to commit it) we
need to ensure that the dev root's commit root doesn't change underneath
us and the pending chunk lists stays around until we're done with it.

We can ensure the former by holding the commit root sem and the latter
by pinning the transaction.  We do this now, but the critical section
covers the trim operation itself and we don't need to do that.

This patch moves the pinning and unpinning logic into helpers and
unpins the transaction after performing the search and check for
pending chunks.

Limiting the critical section of the transaction pinning improves
the latency substantially on slower storage (e.g. image files over NFS).

Fixes: 499f377f49f08 (btrfs: iterate over unused chunk space in FITRIM
Signed-off-by: Jeff Mahoney 
---
 fs/btrfs/extent-tree.c | 25 +
 1 file changed, 17 insertions(+), 8 deletions(-)

diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c
index 92e5e9fd9bdd..8dc8e090667c 100644
--- a/fs/btrfs/extent-tree.c
+++ b/fs/btrfs/extent-tree.c
@@ -10870,14 +10870,16 @@ int btrfs_error_unpin_extent_range(struct 
btrfs_fs_info *fs_info,
  * We don't want a transaction for this since the discard may take a
  * substantial amount of time.  We don't require that a transaction be
  * running, but we do need to take a running transaction into account
- * to ensure that we're not discarding chunks that were released in
- * the current transaction.
+ * to ensure that we're not discarding chunks that were released or
+ * allocated in the current transaction.
  *
  * Holding the chunks lock will prevent other threads from allocating
  * or releasing chunks, but it won't prevent a running transaction
  * from committing and releasing the memory that the pending chunks
  * list head uses.  For that, we need to take a reference to the
- * transaction.
+ * transaction and hold the commit root sem.  We only need to hold
+ * it while performing the free space search since we have already
+ * held back allocations.
  */
 static int btrfs_trim_free_extents(struct btrfs_device *device,
   u64 minlen, u64 *trimmed)
@@ -10908,9 +10910,13 @@ static int btrfs_trim_free_extents(struct btrfs_device 
*device,
 
ret = mutex_lock_interruptible(_info->chunk_mutex);
if (ret)
-   return ret;
+   break;
 
-   down_read(_info->commit_root_sem);
+   ret = down_read_killable(_info->commit_root_sem);
+   if (ret) {
+   mutex_unlock(_info->chunk_mutex);
+   break;
+   }
 
spin_lock(_info->trans_lock);
trans = fs_info->running_transaction;
@@ -10918,13 +10924,17 @@ static int btrfs_trim_free_extents(struct 
btrfs_device *device,
refcount_inc(>use_count);
spin_unlock(_info->trans_lock);
 
+   if (!trans)
+   up_read(_info->commit_root_sem);
+
ret = find_free_dev_extent_start(trans, device, minlen, start,
 , );
-   if (trans)
+   if (trans) {
+   up_read(_info->commit_root_sem);
btrfs_put_transaction(trans);
+   }
 
if (ret) {
-   up_read(_info->commit_root_sem);
mutex_unlock(_info->chunk_mutex);
if (ret == -ENOSPC)
ret = 0;
@@ -10932,7 +10942,6 @@ static int btrfs_trim_free_extents(struct btrfs_device 
*device,
}
 
ret = btrfs_issue_discard(device->bdev, start, len, );
-   up_read(_info->commit_root_sem);
mutex_unlock(_info->chunk_mutex);
 
if (ret)
-- 
2.12.3

[PATCH 1/3] btrfs: use ->devices list instead of ->alloc_list in btrfs_trim_fs

[jeffm]

From: Jeff Mahoney 

btrfs_trim_fs iterates over the fs_devices->alloc_list while holding
the device_list_mutex.  The problem is that ->alloc_list is protected
by the chunk mutex.  We don't want to hold the chunk mutex over
the trim of the entire file system.  Fortunately, the ->dev_list
list is protected by the dev_list mutex and while it will give us
all devices, including read-only devices, we already just skip the
read-only devices.  Then we can continue to take and release the chunk
mutex while scanning each device.

Fixes: 499f377f49f (btrfs: iterate over unused chunk space in FITRIM)
Signed-off-by: Jeff Mahoney 
---
 fs/btrfs/extent-tree.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c
index 3d9fe58c0080..a0e82589c3e8 100644
--- a/fs/btrfs/extent-tree.c
+++ b/fs/btrfs/extent-tree.c
@@ -11008,8 +11008,8 @@ int btrfs_trim_fs(struct btrfs_fs_info *fs_info, struct 
fstrim_range *range)
}
 
mutex_lock(_info->fs_devices->device_list_mutex);
-   devices = _info->fs_devices->alloc_list;
-   list_for_each_entry(device, devices, dev_alloc_list) {
+   devices = _info->fs_devices->devices;
+   list_for_each_entry(device, devices, dev_list) {
ret = btrfs_trim_free_extents(device, range->minlen,
  _trimmed);
if (ret)
-- 
2.12.3

[PATCH 0/3] btrfs: trim latency improvements

[jeffm]

From: Jeff Mahoney 

This patch set fixes a few issues with trim.

1) Fix device list iteration.  We're iterating the ->alloc_list while
   holding the device_list_mutex.  The ->alloc_list is protected by
   the chunk mutex and we don't want to hold it across the entire
   trim execution.  Instead, use the ->devices list, which is protected
   by the device_list_mutex.

2) Skip trim on devices that don't support it.  Rather than letting
   the block layer reject it, bounce out early.

3) Don't keep the commit_root_sem locked and the transaction pinned
   across the block layer component of trim.  We only need these to
   ensure the pending chunks list doesn't go away underneath us, so
   it's safe to drop across the trim itself.  Historically, this
   caused issues when fstrim and balance would run at the same time
   since balance would produce lots of transactions and would
   have to wait constantly, causing problems for everything else that
   wanted to start a transaction.

-Jeff
---

Jeff Mahoney (3):
  btrfs: use ->devices list instead of ->alloc_list in btrfs_trim_fs
  btrfs: don't attempt to trim devices that don't support it
  btrfs: keep trim from interfering with transaction commits

 fs/btrfs/extent-tree.c | 33 +++--
 1 file changed, 23 insertions(+), 10 deletions(-)

-- 
2.12.3

[PATCH 2/3] btrfs: don't attempt to trim devices that don't support it

[jeffm]

From: Jeff Mahoney 

We check whether any device the file system is using supports discard
in the ioctl call, but then we attempt to trim free extents on every
device regardless of whether discard is supported.  Due to the way
we mask off EOPNOTSUPP, we can end up issuing the trim operations
on each free range on devices that don't support it, just wasting time.

Fixes: 499f377f49f08 (btrfs: iterate over unused chunk space in FITRIM)
Signed-off-by: Jeff Mahoney 
---
 fs/btrfs/extent-tree.c | 4 
 1 file changed, 4 insertions(+)

diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c
index a0e82589c3e8..92e5e9fd9bdd 100644
--- a/fs/btrfs/extent-tree.c
+++ b/fs/btrfs/extent-tree.c
@@ -10887,6 +10887,10 @@ static int btrfs_trim_free_extents(struct btrfs_device 
*device,
 
*trimmed = 0;
 
+   /* Discard not supported = nothing to do. */
+   if (!blk_queue_discard(bdev_get_queue(device->bdev)))
+   return 0;
+
/* Not writeable = nothing to do. */
if (!test_bit(BTRFS_DEV_STATE_WRITEABLE, >dev_state))
return 0;
-- 
2.12.3

[PATCH RESEND] btrfs: fix error handling in free_log_tree

[jeffm]

From: Jeff Mahoney 

When we hit an I/O error in free_log_tree->walk_log_tree during file system
shutdown we can crash due to there not being a valid transaction handle.

Use btrfs_handle_fs_error when there's no transaction handle to use.

BUG: unable to handle kernel NULL pointer dereference at 0060
IP: free_log_tree+0xd2/0x140 [btrfs]
PGD 0 P4D 0
Oops:  [#1] SMP DEBUG_PAGEALLOC PTI
Modules linked in: 
CPU: 2 PID: 23544 Comm: umount Tainted: GW4.12.14-kvmsmall #9 
SLE15 (unreleased)
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 
1.0.0-prebuilt.qemu-project.org 04/01/2014
task: 96bfd3478880 task.stack: a7cf40d78000
RIP: 0010:free_log_tree+0xd2/0x140 [btrfs]
RSP: 0018:a7cf40d7bd10 EFLAGS: 00010282
RAX: fffb RBX: fffb RCX: 0002
RDX:  RSI: 96c02f07d4c8 RDI: 0282
RBP: 96c013cf1000 R08: 96c02f07d4c8 R09: 96c02f07d4d0
R10:  R11: 0002 R12: 
R13: 96c005e800c0 R14: a7cf40d7bdb8 R15: 
FS:  7f17856bcfc0() GS:96c03f60() knlGS:
CS:  0010 DS:  ES:  CR0: 80050033
CR2: 0060 CR3: 45ed6002 CR4: 003606e0
DR0:  DR1:  DR2: 
DR3:  DR6: fffe0ff0 DR7: 0400
Call Trace:
 ? wait_for_writer+0xb0/0xb0 [btrfs]
 btrfs_free_log+0x17/0x30 [btrfs]
 btrfs_drop_and_free_fs_root+0x9a/0xe0 [btrfs]
 btrfs_free_fs_roots+0xc0/0x130 [btrfs]
 ? wait_for_completion+0xf2/0x100
 close_ctree+0xea/0x2e0 [btrfs]
 ? kthread_stop+0x161/0x260
 generic_shutdown_super+0x6c/0x120
 kill_anon_super+0xe/0x20
 btrfs_kill_super+0x13/0x100 [btrfs]
 deactivate_locked_super+0x3f/0x70
 cleanup_mnt+0x3b/0x70
 task_work_run+0x78/0x90
 exit_to_usermode_loop+0x77/0xa6
 do_syscall_64+0x1c5/0x1e0
 entry_SYSCALL_64_after_hwframe+0x42/0xb7
RIP: 0033:0x7f1784f90827
RSP: 002b:7ffdeeb03118 EFLAGS: 0246 ORIG_RAX: 00a6
RAX:  RBX: 556a60c62970 RCX: 7f1784f90827
RDX: 0001 RSI:  RDI: 556a60c62b50
RBP:  R08: 0005 R09: 
R10: 556a60c63900 R11: 0246 R12: 556a60c62b50
R13: 7f17854a81c4 R14:  R15: 
Code: 65 a1 fd ff be 01 00 00 00 48 89 ef e8 58 a1 fd ff 48 8b 7d 00 e8 9f 33 
fe ff 48 89 ef e8 17 6c d3 ed 48 83 c4 50 5b 5d 41 5c c3 <49> 8b 44 24 60 f0 0f 
ba a8 80 65 01 00 02 72 23 83 fb fb 75 39
RIP: free_log_tree+0xd2/0x140 [btrfs] RSP: a7cf40d7bd10
CR2: 0060
---[ end trace 3bc199fbf8fb4977 ]---

Cc:  # v3.13
Fixes: 681ae50917df9 (Btrfs: cleanup reserved space when freeing tree log on 
error)
Signed-off-by: Jeff Mahoney 
---
 fs/btrfs/tree-log.c | 9 ++---
 1 file changed, 6 insertions(+), 3 deletions(-)

diff --git a/fs/btrfs/tree-log.c b/fs/btrfs/tree-log.c
index f8220ec02036..a5f6971a125f 100644
--- a/fs/btrfs/tree-log.c
+++ b/fs/btrfs/tree-log.c
@@ -3143,9 +3143,12 @@ static void free_log_tree(struct btrfs_trans_handle 
*trans,
};
 
ret = walk_log_tree(trans, log, );
-   /* I don't think this can happen but just in case */
-   if (ret)
-   btrfs_abort_transaction(trans, ret);
+   if (ret) {
+   if (trans)
+   btrfs_abort_transaction(trans, ret);
+   else
+   btrfs_handle_fs_error(log->fs_info, ret, NULL);
+   }
 
while (1) {
ret = find_first_extent_bit(>dirty_log_pages,
-- 
2.12.3

[PATCH] btrfs: fix error handling in btrfs_dev_replace_start

[jeffm]

From: Jeff Mahoney 

When we fail to start a transaction in btrfs_dev_replace_start,
we leave dev_replace->replace_start set to STARTED but clear
->srcdev and ->tgtdev.  Later, that can result in an Oops in
btrfs_dev_replace_progress when having state set to STARTED or
SUSPENDED implies that ->srcdev is valid.

Also fix error handling when the state is already STARTED or
SUSPENDED while starting.  That, too, will clear ->srcdev and ->tgtdev
even though it doesn't own them.  This should be an impossible case to
hit since we should be protected by the BTRFS_FS_EXCL_OP bit being set.
Let's add an ASSERT there while we're at it.

Fixes: e93c89c1a (Btrfs: add new sources for device replace code)
Signed-off-by: Jeff Mahoney 
---
 fs/btrfs/dev-replace.c | 7 +--
 1 file changed, 5 insertions(+), 2 deletions(-)

diff --git a/fs/btrfs/dev-replace.c b/fs/btrfs/dev-replace.c
index e2ba0419297a..0581c8570a05 100644
--- a/fs/btrfs/dev-replace.c
+++ b/fs/btrfs/dev-replace.c
@@ -445,6 +445,7 @@ int btrfs_dev_replace_start(struct btrfs_fs_info *fs_info,
break;
case BTRFS_IOCTL_DEV_REPLACE_STATE_STARTED:
case BTRFS_IOCTL_DEV_REPLACE_STATE_SUSPENDED:
+   ASSERT(0);
ret = BTRFS_IOCTL_DEV_REPLACE_RESULT_ALREADY_STARTED;
goto leave;
}
@@ -487,6 +488,10 @@ int btrfs_dev_replace_start(struct btrfs_fs_info *fs_info,
if (IS_ERR(trans)) {
ret = PTR_ERR(trans);
btrfs_dev_replace_write_lock(dev_replace);
+   dev_replace->replace_state =
+   BTRFS_IOCTL_DEV_REPLACE_STATE_NEVER_STARTED;
+   dev_replace->srcdev = NULL;
+   dev_replace->tgtdev = NULL;
goto leave;
}
 
@@ -508,8 +513,6 @@ int btrfs_dev_replace_start(struct btrfs_fs_info *fs_info,
return ret;
 
 leave:
-   dev_replace->srcdev = NULL;
-   dev_replace->tgtdev = NULL;
btrfs_dev_replace_write_unlock(dev_replace);
btrfs_destroy_dev_replace_tgtdev(fs_info, tgt_device);
return ret;
-- 
2.12.3

[PATCH 03/18] btrfs-progs: constify pathnames passed as arguments

[jeffm]

From: Jeff Mahoney 

It's unlikely we're going to modify a pathname argument, so codify that
and use const.

Reviewed-by: Qu Wenruo 
Signed-off-by: Jeff Mahoney 
---
 chunk-recover.c | 4 ++--
 cmds-device.c   | 2 +-
 cmds-fi-usage.c | 6 +++---
 cmds-rescue.c   | 4 ++--
 4 files changed, 8 insertions(+), 8 deletions(-)

diff --git a/chunk-recover.c b/chunk-recover.c
index 705bcf52..1d30db51 100644
--- a/chunk-recover.c
+++ b/chunk-recover.c
@@ -1492,7 +1492,7 @@ out:
return ERR_PTR(ret);
 }
 
-static int recover_prepare(struct recover_control *rc, char *path)
+static int recover_prepare(struct recover_control *rc, const char *path)
 {
int ret;
int fd;
@@ -2296,7 +2296,7 @@ static void validate_rebuild_chunks(struct 
recover_control *rc)
 /*
  * Return 0 when successful, < 0 on error and > 0 if aborted by user
  */
-int btrfs_recover_chunk_tree(char *path, int verbose, int yes)
+int btrfs_recover_chunk_tree(const char *path, int verbose, int yes)
 {
int ret = 0;
struct btrfs_root *root = NULL;
diff --git a/cmds-device.c b/cmds-device.c
index 86459d1b..a49c9d9d 100644
--- a/cmds-device.c
+++ b/cmds-device.c
@@ -526,7 +526,7 @@ static const char * const cmd_device_usage_usage[] = {
NULL
 };
 
-static int _cmd_device_usage(int fd, char *path, unsigned unit_mode)
+static int _cmd_device_usage(int fd, const char *path, unsigned unit_mode)
 {
int i;
int ret = 0;
diff --git a/cmds-fi-usage.c b/cmds-fi-usage.c
index b9a2b1c8..d08ec4d3 100644
--- a/cmds-fi-usage.c
+++ b/cmds-fi-usage.c
@@ -227,7 +227,7 @@ static int cmp_btrfs_ioctl_space_info(const void *a, const 
void *b)
 /*
  * This function load all the information about the space usage
  */
-static struct btrfs_ioctl_space_args *load_space_info(int fd, char *path)
+static struct btrfs_ioctl_space_args *load_space_info(int fd, const char *path)
 {
struct btrfs_ioctl_space_args *sargs = NULL, *sargs_orig = NULL;
int ret, count;
@@ -305,7 +305,7 @@ static void get_raid56_used(struct chunk_info *chunks, int 
chunkcount,
 #defineMIN_UNALOCATED_THRESH   SZ_16M
 static int print_filesystem_usage_overall(int fd, struct chunk_info *chunkinfo,
int chunkcount, struct device_info *devinfo, int devcount,
-   char *path, unsigned unit_mode)
+   const char *path, unsigned unit_mode)
 {
struct btrfs_ioctl_space_args *sargs = NULL;
int i;
@@ -933,7 +933,7 @@ static void _cmd_filesystem_usage_linear(unsigned unit_mode,
 static int print_filesystem_usage_by_chunk(int fd,
struct chunk_info *chunkinfo, int chunkcount,
struct device_info *devinfo, int devcount,
-   char *path, unsigned unit_mode, int tabular)
+   const char *path, unsigned unit_mode, int tabular)
 {
struct btrfs_ioctl_space_args *sargs;
int ret = 0;
diff --git a/cmds-rescue.c b/cmds-rescue.c
index c40088ad..c61145bc 100644
--- a/cmds-rescue.c
+++ b/cmds-rescue.c
@@ -32,8 +32,8 @@ static const char * const rescue_cmd_group_usage[] = {
NULL
 };
 
-int btrfs_recover_chunk_tree(char *path, int verbose, int yes);
-int btrfs_recover_superblocks(char *path, int verbose, int yes);
+int btrfs_recover_chunk_tree(const char *path, int verbose, int yes);
+int btrfs_recover_superblocks(const char *path, int verbose, int yes);
 
 static const char * const cmd_rescue_chunk_recover_usage[] = {
"btrfs rescue chunk-recover [options] ",
-- 
2.15.1

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v3 00/18] btrfs-progs: qgroups-usability

[jeffm]

From: Jeff Mahoney 

Changes since v2:
- Updated Reviewed-by tags were provided.
- Fixed the typoed commands that I commented on in the previous posting.
- Dropped the btrfs_cleanup_root_info since it's unnecessary with the
  switch to libbtrfsutil.
- Updated the qgroups pathname patch to use libbtrfsutil
- Added a mini-filter to skip dead qgroups in 'qgroup show' output
  unless -v is specified.

The most notable change is that this posting doesn't include the JSON
format patches since there was still some open discussion there.  The
plumbing for passing global options around to commands is there, so we
really just need to decide on how we want to handle alternative formats.
For some commands, the formatting library that coreutils uses will
probably work but for qgroups show to represent nested groups, it's
unsuitable.

I've also posted this as a pull request to the devel branch:

https://github.com/kdave/btrfs-progs/pull/139

Jeff Mahoney (18):
  btrfs-progs: quota: Add -W option to rescan to wait without starting
rescan
  btrfs-progs: qgroups: fix misleading index check
  btrfs-progs: constify pathnames passed as arguments
  btrfs-progs: btrfs-list: add rb_entry helpers for root_info
  btrfs-progs: qgroups: add pathname to show output
  btrfs-progs: qgroups: introduce and use info and limit structures
  btrfs-progs: qgroups: introduce btrfs_qgroup_query
  btrfs-progs: subvolume: add quota info to btrfs sub show
  btrfs-progs: help: convert ints used as bools to bool
  btrfs-progs: reorder placement of help declarations for send/receive
  btrfs-progs: filesystem balance: split out special handling
  btrfs-progs: use cmd_struct as command entry point
  btrfs-progs: pass cmd_struct to command callback function
  btrfs-progs: pass cmd_struct to clean_args_no_options{,_relaxed}
  btrfs-progs: pass cmd_struct to usage()
  btrfs-progs: add support for output formats
  btrfs-progs: handle command groups directly for common case
  btrfs-progs: qgroups: don't print dead qgroups

 Documentation/btrfs-qgroup.asciidoc |   4 +
 Documentation/btrfs-quota.asciidoc  |  10 +-
 btrfs-list.c|  30 ++-
 btrfs.c | 174 +++-
 check/main.c|  10 +-
 chunk-recover.c |   4 +-
 cmds-balance.c  |  74 ---
 cmds-device.c   |  96 +
 cmds-fi-du.c|  11 +-
 cmds-fi-usage.c |  17 +-
 cmds-filesystem.c   | 113 +++
 cmds-inspect-dump-super.c   |  11 +-
 cmds-inspect-dump-tree.c|  11 +-
 cmds-inspect-tree-stats.c   |  11 +-
 cmds-inspect.c  |  78 
 cmds-property.c |  55 +++---
 cmds-qgroup.c   | 107 ++
 cmds-quota.c|  63 +++---
 cmds-receive.c  |  70 +++
 cmds-replace.c  |  45 +++--
 cmds-rescue.c   |  60 +++---
 cmds-restore.c  |  12 +-
 cmds-scrub.c|  64 +++---
 cmds-send.c |  74 +++
 cmds-subvolume.c| 163 ++-
 commands.h  | 152 --
 help.c  |  98 ++---
 help.h  |  14 +-
 kerncompat.h|   1 +
 qgroup.c| 384 +++-
 qgroup.h|  19 +-
 31 files changed, 1325 insertions(+), 710 deletions(-)

-- 
2.15.1

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 05/18] btrfs-progs: qgroups: add pathname to show output

[jeffm]

From: Jeff Mahoney 

The btrfs qgroup show command currently only exports qgroup IDs,
forcing the user to resolve which subvolume each corresponds to.

This patch adds pathname resolution to qgroup show so that when
the -P option is used, the last column contains the pathname of
the root of the subvolume it describes.  In the case of nested
qgroups, it will show the number of member qgroups or the paths
of the members if the -v option is used.

Pathname can also be used as a sort parameter.

Reviewed-by: Qu Wenruo 
Signed-off-by: Jeff Mahoney 
---
 Documentation/btrfs-qgroup.asciidoc |   4 +
 cmds-qgroup.c   |  18 -
 kerncompat.h|   1 +
 qgroup.c| 149 
 qgroup.h|   4 +-
 5 files changed, 157 insertions(+), 19 deletions(-)

diff --git a/Documentation/btrfs-qgroup.asciidoc 
b/Documentation/btrfs-qgroup.asciidoc
index 3108457c..360b3269 100644
--- a/Documentation/btrfs-qgroup.asciidoc
+++ b/Documentation/btrfs-qgroup.asciidoc
@@ -97,10 +97,14 @@ print child qgroup id.
 print limit of referenced size of qgroup.
 -e
 print limit of exclusive size of qgroup.
+-P
+print pathname to the root of the subvolume managed by qgroup.  For nested 
qgroups, the number of members will be printed unless -v is specified.
 -F
 list all qgroups which impact the given path(include ancestral qgroups)
 -f
 list all qgroups which impact the given path(exclude ancestral qgroups)
+-v
+Be more verbose.  Print pathnames of member qgroups when nested.
 --raw
 raw numbers in bytes, without the 'B' suffix.
 --human-readable
diff --git a/cmds-qgroup.c b/cmds-qgroup.c
index 93206900..33053725 100644
--- a/cmds-qgroup.c
+++ b/cmds-qgroup.c
@@ -282,8 +282,11 @@ static const char * const cmd_qgroup_show_usage[] = {
"   (including ancestral qgroups)",
"-f list all qgroups which impact the given path",
"   (excluding ancestral qgroups)",
+   "-P print first-level qgroups using pathname",
+   "   - nested qgroups will be reported as a count",
+   "-v verbose, prints pathnames for all nested qgroups",
HELPINFO_UNITS_LONG,
-   "--sort=qgroupid,rfer,excl,max_rfer,max_excl",
+   "--sort=qgroupid,rfer,excl,max_rfer,max_excl,pathname",
"   list qgroups sorted by specified items",
"   you can use '+' or '-' in front of each item.",
"   (+:ascending, -:descending, ascending default)",
@@ -302,6 +305,7 @@ static int cmd_qgroup_show(int argc, char **argv)
unsigned unit_mode;
int sync = 0;
enum btrfs_util_error err;
+   bool verbose = false;
 
struct btrfs_qgroup_comparer_set *comparer_set;
struct btrfs_qgroup_filter_set *filter_set;
@@ -319,10 +323,11 @@ static int cmd_qgroup_show(int argc, char **argv)
static const struct option long_options[] = {
{"sort", required_argument, NULL, GETOPT_VAL_SORT},
{"sync", no_argument, NULL, GETOPT_VAL_SYNC},
+   {"verbose", no_argument, NULL, 'v'},
{ NULL, 0, NULL, 0 }
};
 
-   c = getopt_long(argc, argv, "pcreFf", long_options, NULL);
+   c = getopt_long(argc, argv, "pPcreFfv", long_options, NULL);
if (c < 0)
break;
switch (c) {
@@ -330,6 +335,10 @@ static int cmd_qgroup_show(int argc, char **argv)
btrfs_qgroup_setup_print_column(
BTRFS_QGROUP_PARENT);
break;
+   case 'P':
+   btrfs_qgroup_setup_print_column(
+   BTRFS_QGROUP_PATHNAME);
+   break;
case 'c':
btrfs_qgroup_setup_print_column(
BTRFS_QGROUP_CHILD);
@@ -357,6 +366,9 @@ static int cmd_qgroup_show(int argc, char **argv)
case GETOPT_VAL_SYNC:
sync = 1;
break;
+   case 'v':
+   verbose = true;
+   break;
default:
usage(cmd_qgroup_show_usage);
}
@@ -398,7 +410,7 @@ static int cmd_qgroup_show(int argc, char **argv)
BTRFS_QGROUP_FILTER_PARENT,
qgroupid);
}
-   ret = btrfs_show_qgroups(fd, filter_set, comparer_set);
+   ret = btrfs_show_qgroups(fd, filter_set, comparer_set, verbose);
close_file_or_dir(fd, dirstream);
free(filter_set);
free(comparer_set);
diff --git a/kerncompat.h b/kerncompat.h
index 

[PATCH 14/18] btrfs-progs: pass cmd_struct to clean_args_no_options{,_relaxed}

[jeffm]

From: Jeff Mahoney 

Now that we have a cmd_struct everywhere, we can pass it to
clean_args_no_options and have it resolve the usage string from
it there.  This is necessary for it to pass the cmd_struct to
usage() in the next patch.

Signed-off-by: Jeff Mahoney 
---
 cmds-balance.c|  6 +++---
 cmds-device.c |  9 +
 cmds-filesystem.c |  8 
 cmds-inspect.c|  4 ++--
 cmds-qgroup.c | 16 
 cmds-quota.c  |  4 ++--
 cmds-rescue.c |  4 ++--
 cmds-scrub.c  | 15 ---
 cmds-subvolume.c  |  6 +++---
 help.c|  9 +
 help.h|  6 --
 11 files changed, 46 insertions(+), 41 deletions(-)

diff --git a/cmds-balance.c b/cmds-balance.c
index 1bd7b3ce..488fffcc 100644
--- a/cmds-balance.c
+++ b/cmds-balance.c
@@ -689,7 +689,7 @@ static int cmd_balance_pause(const struct cmd_struct *cmd,
int ret;
DIR *dirstream = NULL;
 
-   clean_args_no_options(argc, argv, cmd_balance_pause_usage);
+   clean_args_no_options(cmd, argc, argv);
 
if (check_argc_exact(argc - optind, 1))
usage(cmd_balance_pause_usage);
@@ -729,7 +729,7 @@ static int cmd_balance_cancel(const struct cmd_struct *cmd,
int ret;
DIR *dirstream = NULL;
 
-   clean_args_no_options(argc, argv, cmd_balance_cancel_usage);
+   clean_args_no_options(cmd, argc, argv);
 
if (check_argc_exact(argc - optind, 1))
usage(cmd_balance_cancel_usage);
@@ -770,7 +770,7 @@ static int cmd_balance_resume(const struct cmd_struct *cmd,
int fd;
int ret;
 
-   clean_args_no_options(argc, argv, cmd_balance_resume_usage);
+   clean_args_no_options(cmd, argc, argv);
 
if (check_argc_exact(argc - optind, 1))
usage(cmd_balance_resume_usage);
diff --git a/cmds-device.c b/cmds-device.c
index feb53f68..5be748f7 100644
--- a/cmds-device.c
+++ b/cmds-device.c
@@ -149,11 +149,12 @@ static int _cmd_device_remove(const struct cmd_struct 
*cmd,
char*mntpnt;
int i, fdmnt, ret = 0;
DIR *dirstream = NULL;
+   const char * const *usagestr = cmd->usagestr;
 
-   clean_args_no_options(argc, argv, cmd->usagestr);
+   clean_args_no_options(cmd, argc, argv);
 
if (check_argc_min(argc - optind, 2))
-   usage(cmd->usagestr);
+   usage(usagestr);
 
mntpnt = argv[argc - 1];
 
@@ -347,7 +348,7 @@ static int cmd_device_ready(const struct cmd_struct *cmd, 
int argc, char **argv)
int ret;
char*path;
 
-   clean_args_no_options(argc, argv, cmd->usagestr);
+   clean_args_no_options(cmd, argc, argv);
 
if (check_argc_exact(argc - optind, 1))
usage(cmd_device_ready_usage);
@@ -573,7 +574,7 @@ static int cmd_device_usage(const struct cmd_struct *cmd, 
int argc, char **argv)
 
unit_mode = get_unit_mode_from_arg(, argv, 1);
 
-   clean_args_no_options(argc, argv, cmd->usagestr);
+   clean_args_no_options(cmd, argc, argv);
 
if (check_argc_min(argc - optind, 1))
usage(cmd_device_usage_usage);
diff --git a/cmds-filesystem.c b/cmds-filesystem.c
index 649d97a9..2a9f530d 100644
--- a/cmds-filesystem.c
+++ b/cmds-filesystem.c
@@ -129,7 +129,7 @@ static int cmd_filesystem_df(const struct cmd_struct *cmd,
 
unit_mode = get_unit_mode_from_arg(, argv, 1);
 
-   clean_args_no_options(argc, argv, cmd_filesystem_df_usage);
+   clean_args_no_options(cmd, argc, argv);
 
if (check_argc_exact(argc - optind, 1))
usage(cmd_filesystem_df_usage);
@@ -822,7 +822,7 @@ static int cmd_filesystem_sync(const struct cmd_struct *cmd,
 {
enum btrfs_util_error err;
 
-   clean_args_no_options(argc, argv, cmd_filesystem_sync_usage);
+   clean_args_no_options(cmd, argc, argv);
 
if (check_argc_exact(argc - optind, 1))
usage(cmd_filesystem_sync_usage);
@@ -1095,7 +1095,7 @@ static int cmd_filesystem_resize(const struct cmd_struct 
*cmd,
DIR *dirstream = NULL;
struct stat st;
 
-   clean_args_no_options_relaxed(argc, argv);
+   clean_args_no_options_relaxed(cmd, argc, argv);
 
if (check_argc_exact(argc - optind, 2))
usage(cmd_filesystem_resize_usage);
@@ -1168,7 +1168,7 @@ static const char * const cmd_filesystem_label_usage[] = {
 static int cmd_filesystem_label(const struct cmd_struct *cmd,
int argc, char **argv)
 {
-   clean_args_no_options(argc, argv, cmd_filesystem_label_usage);
+   clean_args_no_options(cmd, argc, argv);
 
if (check_argc_min(argc - optind, 1) ||
check_argc_max(argc - optind, 2))
diff --git a/cmds-inspect.c b/cmds-inspect.c
index 46ea5551..b6d045a3 100644
--- a/cmds-inspect.c
+++ b/cmds-inspect.c
@@ -277,7 +277,7 @@ static int cmd_inspect_subvolid_resolve(const struct 
cmd_struct *cmd,
   

[PATCH 01/18] btrfs-progs: quota: Add -W option to rescan to wait without starting rescan

[jeffm]

From: Jeff Mahoney 

This patch adds a new -W option to wait for a rescan without starting a
new operation.  This is useful for things like xfstests where we want
do to do a "btrfs quota enable" and not continue until the subsequent
rescan has finished.

In addition to documenting the new option in the man page, I've cleaned
up the rescan entry to document the -w option a bit better.

Reviewed-by: Qu Wenruo 
Signed-off-by: Jeff Mahoney 
---
 Documentation/btrfs-quota.asciidoc | 10 +++---
 cmds-quota.c   | 20 ++--
 2 files changed, 21 insertions(+), 9 deletions(-)

diff --git a/Documentation/btrfs-quota.asciidoc 
b/Documentation/btrfs-quota.asciidoc
index 85ebf729..0b64a69b 100644
--- a/Documentation/btrfs-quota.asciidoc
+++ b/Documentation/btrfs-quota.asciidoc
@@ -238,15 +238,19 @@ Disable subvolume quota support for a filesystem.
 *enable* ::
 Enable subvolume quota support for a filesystem.
 
-*rescan* [-s] ::
+*rescan* [-s|-w|-W] ::
 Trash all qgroup numbers and scan the metadata again with the current config.
 +
 `Options`
 +
 -s
-show status of a running rescan operation.
+Show status of a running rescan operation.
+
 -w
-wait for rescan operation to finish(can be already in progress).
+Start rescan operation and wait until it has finished before exiting.  If a 
rescan is already running, wait until it finishes and then exit without 
starting a new one.
+
+-W
+Wait for rescan operation to finish and then exit.  If a rescan is not already 
running, exit silently.
 
 EXIT STATUS
 ---
diff --git a/cmds-quota.c b/cmds-quota.c
index 745889d1..7f933495 100644
--- a/cmds-quota.c
+++ b/cmds-quota.c
@@ -120,14 +120,20 @@ static int cmd_quota_rescan(int argc, char **argv)
int wait_for_completion = 0;
 
while (1) {
-   int c = getopt(argc, argv, "sw");
+   int c = getopt(argc, argv, "swW");
if (c < 0)
break;
switch (c) {
case 's':
ioctlnum = BTRFS_IOC_QUOTA_RESCAN_STATUS;
break;
+   case 'W':
+   ioctlnum = 0;
+   wait_for_completion = 1;
+   break;
case 'w':
+   /* Reset it in case the user did both -W and -w */
+   ioctlnum = BTRFS_IOC_QUOTA_RESCAN;
wait_for_completion = 1;
break;
default:
@@ -135,8 +141,8 @@ static int cmd_quota_rescan(int argc, char **argv)
}
}
 
-   if (ioctlnum != BTRFS_IOC_QUOTA_RESCAN && wait_for_completion) {
-   error("switch -w cannot be used with -s");
+   if (ioctlnum == BTRFS_IOC_QUOTA_RESCAN_STATUS && wait_for_completion) {
+   error("switches -w/-W cannot be used with -s");
return 1;
}
 
@@ -150,8 +156,10 @@ static int cmd_quota_rescan(int argc, char **argv)
if (fd < 0)
return 1;
 
-   ret = ioctl(fd, ioctlnum, );
-   e = errno;
+   if (ioctlnum) {
+   ret = ioctl(fd, ioctlnum, );
+   e = errno;
+   }
 
if (ioctlnum == BTRFS_IOC_QUOTA_RESCAN_STATUS) {
close_file_or_dir(fd, dirstream);
@@ -167,7 +175,7 @@ static int cmd_quota_rescan(int argc, char **argv)
return 0;
}
 
-   if (ret == 0) {
+   if (ioctlnum == BTRFS_IOC_QUOTA_RESCAN && ret == 0) {
printf("quota rescan started\n");
fflush(stdout);
} else if (ret < 0 && (!wait_for_completion || e != EINPROGRESS)) {
-- 
2.15.1

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 11/18] btrfs-progs: filesystem balance: split out special handling

[jeffm]

From: Jeff Mahoney 

In preparation to use cmd_struct as the command entry point, we need
to split out the 'filesystem balance' handling to not call cmd_balance
directly.  The reason is that the flags that indicate a command is
hidden are a part of cmd_struct and so we can use a cmd_struct as a
direct alias in another command group and ALSO have it be hidden
without declaring another cmd_struct.

This change has no immediate impact since cmd_balance will still
use its usage information directly from cmds-balance.c.  It will
take effect once we start passing cmd_structs around for usage
information.

Signed-off-by: Jeff Mahoney 
---
 cmds-filesystem.c | 17 +++--
 1 file changed, 15 insertions(+), 2 deletions(-)

diff --git a/cmds-filesystem.c b/cmds-filesystem.c
index 30a50bf5..01d639e3 100644
--- a/cmds-filesystem.c
+++ b/cmds-filesystem.c
@@ -1177,6 +1177,18 @@ static int cmd_filesystem_label(int argc, char **argv)
}
 }
 
+static const char * const cmd_filesystem_balance_usage[] = {
+   "btrfs filesystem balance [args...] (alias of \"btrfs balance\")",
+   "Please see \"btrfs balance --help\" for more information.",
+   NULL
+};
+
+/* Compatible old "btrfs filesystem balance" command */
+static int cmd_filesystem_balance(int argc, char **argv)
+{
+   return cmd_balance(argc, argv);
+}
+
 static const char filesystem_cmd_group_info[] =
 "overall filesystem tasks and information";
 
@@ -1190,8 +1202,9 @@ const struct cmd_group filesystem_cmd_group = {
0 },
{ "defragment", cmd_filesystem_defrag,
cmd_filesystem_defrag_usage, NULL, 0 },
-   { "balance", cmd_balance, NULL, _cmd_group,
-   CMD_HIDDEN },
+   { "balance", cmd_filesystem_balance,
+  cmd_filesystem_balance_usage, _cmd_group,
+  CMD_HIDDEN },
{ "resize", cmd_filesystem_resize, cmd_filesystem_resize_usage,
NULL, 0 },
{ "label", cmd_filesystem_label, cmd_filesystem_label_usage,
-- 
2.15.1

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 18/18] btrfs-progs: qgroups: don't print dead qgroups

[jeffm]

From: Jeff Mahoney 

When qgroup items get left behind, we still print them in
'btrfs qgroup show' even though there is nothing to show.  Since we
now look up the pathname and that means we look up the subvolume,
we can filter out first-level qgroups that correspond to roots
that have been removed.  Specifying -v will still show them.

Signed-off-by: Jeff Mahoney 
---
 qgroup.c | 10 +-
 1 file changed, 9 insertions(+), 1 deletion(-)

diff --git a/qgroup.c b/qgroup.c
index 647bc2f3..08e78887 100644
--- a/qgroup.c
+++ b/qgroup.c
@@ -313,6 +313,13 @@ static void print_qgroup_column(struct btrfs_qgroup 
*qgroup,
}
 }
 
+static bool qgroup_target_exists(const struct btrfs_qgroup *qgroup)
+{
+   if (btrfs_qgroup_level(qgroup->qgroupid) > 0)
+   return true;
+   return qgroup->pathname != NULL;
+}
+
 static void print_single_qgroup_table(struct btrfs_qgroup *qgroup, bool 
verbose)
 {
int i;
@@ -1369,7 +1376,8 @@ static void print_all_qgroups(struct qgroup_lookup 
*qgroup_lookup, bool verbose)
n = rb_first(_lookup->root);
while (n) {
entry = rb_entry(n, struct btrfs_qgroup, sort_node);
-   print_single_qgroup_table(entry, verbose);
+   if (qgroup_target_exists(entry) || verbose)
+   print_single_qgroup_table(entry, verbose);
n = rb_next(n);
}
 }
-- 
2.15.1

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 15/18] btrfs-progs: pass cmd_struct to usage()

[jeffm]

From: Jeff Mahoney 

Now that every call site has a cmd_struct, we can just pass the cmd_struct
to usage to print the usager information.  This allows us to interpret
the format flags we'll add later in this series to inform the user of
which output formats any given command supports.

Signed-off-by: Jeff Mahoney 
---
 check/main.c  |  4 ++--
 cmds-balance.c| 14 +++---
 cmds-device.c | 19 +--
 cmds-fi-du.c  |  4 ++--
 cmds-fi-usage.c   |  4 ++--
 cmds-filesystem.c | 18 +-
 cmds-inspect-dump-super.c |  4 ++--
 cmds-inspect-dump-tree.c  |  4 ++--
 cmds-inspect-tree-stats.c |  4 ++--
 cmds-inspect.c| 16 
 cmds-property.c   | 22 +-
 cmds-qgroup.c | 18 +-
 cmds-quota.c  |  8 
 cmds-receive.c|  4 ++--
 cmds-replace.c| 12 ++--
 cmds-rescue.c | 12 ++--
 cmds-restore.c|  8 
 cmds-scrub.c  | 25 ++---
 cmds-send.c   |  2 +-
 cmds-subvolume.c  | 30 +++---
 help.c|  6 +++---
 help.h|  2 +-
 22 files changed, 115 insertions(+), 125 deletions(-)

diff --git a/check/main.c b/check/main.c
index 0375eec3..49ccdf2f 100644
--- a/check/main.c
+++ b/check/main.c
@@ -9483,7 +9483,7 @@ static int cmd_check(const struct cmd_struct *cmd, int 
argc, char **argv)
break;
case '?':
case 'h':
-   usage(cmd_check_usage);
+   usage(cmd);
case GETOPT_VAL_REPAIR:
printf("enabling repair mode\n");
repair = 1;
@@ -9534,7 +9534,7 @@ static int cmd_check(const struct cmd_struct *cmd, int 
argc, char **argv)
}
 
if (check_argc_exact(argc - optind, 1))
-   usage(cmd_check_usage);
+   usage(cmd);
 
if (ctx.progress_enabled) {
ctx.tp = TASK_NOTHING;
diff --git a/cmds-balance.c b/cmds-balance.c
index 488fffcc..c639459f 100644
--- a/cmds-balance.c
+++ b/cmds-balance.c
@@ -585,12 +585,12 @@ static int cmd_balance_start(const struct cmd_struct *cmd,
background = 1;
break;
default:
-   usage(cmd_balance_start_usage);
+   usage(cmd);
}
}
 
if (check_argc_exact(argc - optind, 1))
-   usage(cmd_balance_start_usage);
+   usage(cmd);
 
/*
 * allow -s only under --force, otherwise do with system chunks
@@ -692,7 +692,7 @@ static int cmd_balance_pause(const struct cmd_struct *cmd,
clean_args_no_options(cmd, argc, argv);
 
if (check_argc_exact(argc - optind, 1))
-   usage(cmd_balance_pause_usage);
+   usage(cmd);
 
path = argv[optind];
 
@@ -732,7 +732,7 @@ static int cmd_balance_cancel(const struct cmd_struct *cmd,
clean_args_no_options(cmd, argc, argv);
 
if (check_argc_exact(argc - optind, 1))
-   usage(cmd_balance_cancel_usage);
+   usage(cmd);
 
path = argv[optind];
 
@@ -773,7 +773,7 @@ static int cmd_balance_resume(const struct cmd_struct *cmd,
clean_args_no_options(cmd, argc, argv);
 
if (check_argc_exact(argc - optind, 1))
-   usage(cmd_balance_resume_usage);
+   usage(cmd);
 
path = argv[optind];
 
@@ -856,12 +856,12 @@ static int cmd_balance_status(const struct cmd_struct 
*cmd,
verbose = 1;
break;
default:
-   usage(cmd_balance_status_usage);
+   usage(cmd);
}
}
 
if (check_argc_exact(argc - optind, 1))
-   usage(cmd_balance_status_usage);
+   usage(cmd);
 
path = argv[optind];
 
diff --git a/cmds-device.c b/cmds-device.c
index 5be748f7..6c74ca8e 100644
--- a/cmds-device.c
+++ b/cmds-device.c
@@ -77,12 +77,12 @@ static int cmd_device_add(const struct cmd_struct *cmd,
force = 1;
break;
default:
-   usage(cmd_device_add_usage);
+   usage(cmd);
}
}
 
if (check_argc_min(argc - optind, 2))
-   usage(cmd_device_add_usage);
+   usage(cmd);
 
last_dev = argc - 1;
mntpnt = argv[last_dev];
@@ -149,12 +149,11 @@ static int _cmd_device_remove(const struct cmd_struct 
*cmd,
char*mntpnt;
int i, fdmnt, ret = 0;
DIR *dirstream = NULL;
-   const char * const *usagestr = 

[PATCH 13/18] btrfs-progs: pass cmd_struct to command callback function

[jeffm]

From: Jeff Mahoney 

This patch passes the cmd_struct to the command callback function.  This
has several purposes: It allows the command callback to identify which
command was used to call it.  It also gives us direct access to the
usage associated with that command.

Signed-off-by: Jeff Mahoney 
---
 btrfs.c   |  8 
 check/main.c  |  2 +-
 cmds-balance.c| 19 ---
 cmds-device.c | 35 +++
 cmds-fi-du.c  |  3 ++-
 cmds-fi-usage.c   |  3 ++-
 cmds-filesystem.c | 24 
 cmds-inspect-dump-super.c |  3 ++-
 cmds-inspect-dump-tree.c  |  3 ++-
 cmds-inspect-tree-stats.c |  3 ++-
 cmds-inspect.c| 17 +++--
 cmds-property.c   | 12 
 cmds-qgroup.c | 18 +++---
 cmds-quota.c  |  9 +
 cmds-receive.c|  2 +-
 cmds-replace.c| 11 +++
 cmds-rescue.c | 14 +-
 cmds-restore.c|  2 +-
 cmds-scrub.c  | 23 +++
 cmds-send.c   |  2 +-
 cmds-subvolume.c  | 27 +--
 commands.h|  4 ++--
 22 files changed, 146 insertions(+), 98 deletions(-)

diff --git a/btrfs.c b/btrfs.c
index 1e68b0c0..49128182 100644
--- a/btrfs.c
+++ b/btrfs.c
@@ -148,7 +148,7 @@ static const char * const cmd_help_usage[] = {
NULL
 };
 
-static int cmd_help(int argc, char **argv)
+static int cmd_help(const struct cmd_struct *unused, int argc, char **argv)
 {
help_command_group(_cmd_group, argc, argv);
return 0;
@@ -162,7 +162,7 @@ static const char * const cmd_version_usage[] = {
NULL
 };
 
-static int cmd_version(int argc, char **argv)
+static int cmd_version(const struct cmd_struct *unused, int argc, char **argv)
 {
printf("%s\n", PACKAGE_STRING);
return 0;
@@ -231,13 +231,13 @@ void handle_special_globals(int shift, int argc, char 
**argv)
if (has_full)
usage_command_group(_cmd_group, true, false);
else
-   cmd_help(argc, argv);
+   cmd_execute(_struct_help, argc, argv);
exit(0);
}
 
for (i = 0; i < shift; i++)
if (strcmp(argv[i], "--version") == 0) {
-   cmd_version(argc, argv);
+   cmd_execute(_struct_version, argc, argv);
exit(0);
}
 }
diff --git a/check/main.c b/check/main.c
index a1b685e7..0375eec3 100644
--- a/check/main.c
+++ b/check/main.c
@@ -9392,7 +9392,7 @@ static const char * const cmd_check_usage[] = {
NULL
 };
 
-static int cmd_check(int argc, char **argv)
+static int cmd_check(const struct cmd_struct *cmd, int argc, char **argv)
 {
struct cache_tree root_cache;
struct btrfs_root *root;
diff --git a/cmds-balance.c b/cmds-balance.c
index 7a60be61..1bd7b3ce 100644
--- a/cmds-balance.c
+++ b/cmds-balance.c
@@ -515,7 +515,8 @@ static const char * const cmd_balance_start_usage[] = {
NULL
 };
 
-static int cmd_balance_start(int argc, char **argv)
+static int cmd_balance_start(const struct cmd_struct *cmd,
+int argc, char **argv)
 {
struct btrfs_ioctl_balance_args args;
struct btrfs_balance_args *ptrs[] = { , ,
@@ -680,7 +681,8 @@ static const char * const cmd_balance_pause_usage[] = {
NULL
 };
 
-static int cmd_balance_pause(int argc, char **argv)
+static int cmd_balance_pause(const struct cmd_struct *cmd,
+int argc, char **argv)
 {
const char *path;
int fd;
@@ -719,7 +721,8 @@ static const char * const cmd_balance_cancel_usage[] = {
NULL
 };
 
-static int cmd_balance_cancel(int argc, char **argv)
+static int cmd_balance_cancel(const struct cmd_struct *cmd,
+ int argc, char **argv)
 {
const char *path;
int fd;
@@ -758,7 +761,8 @@ static const char * const cmd_balance_resume_usage[] = {
NULL
 };
 
-static int cmd_balance_resume(int argc, char **argv)
+static int cmd_balance_resume(const struct cmd_struct *cmd,
+ int argc, char **argv)
 {
struct btrfs_ioctl_balance_args args;
const char *path;
@@ -826,7 +830,8 @@ static const char * const cmd_balance_status_usage[] = {
  *   1 : Successful to know status of a pending balance
  *   0 : When there is no pending balance or completed
  */
-static int cmd_balance_status(int argc, char **argv)
+static int cmd_balance_status(const struct cmd_struct *cmd,
+ int argc, char **argv)
 {
struct btrfs_ioctl_balance_args args;
const char *path;
@@ -904,7 +909,7 @@ out:
 }
 static DEFINE_SIMPLE_COMMAND(balance_status, "status");
 
-static int 

[PATCH 04/18] btrfs-progs: btrfs-list: add rb_entry helpers for root_info

[jeffm]

From: Jeff Mahoney 

We use rb_entry all over the place for the root_info pointers.  Add
a helper to make the code more readable.

Signed-off-by: Jeff Mahoney 
---
 btrfs-list.c | 30 --
 1 file changed, 20 insertions(+), 10 deletions(-)

diff --git a/btrfs-list.c b/btrfs-list.c
index e01c5899..90c98be1 100644
--- a/btrfs-list.c
+++ b/btrfs-list.c
@@ -44,6 +44,16 @@ struct root_lookup {
struct rb_root root;
 };
 
+static inline struct root_info *to_root_info(struct rb_node *node)
+{
+   return rb_entry(node, struct root_info, rb_node);
+}
+
+static inline struct root_info *to_root_info_sorted(struct rb_node *node)
+{
+   return rb_entry(node, struct root_info, sort_node);
+}
+
 static struct {
char*name;
char*column_name;
@@ -309,7 +319,7 @@ static int sort_tree_insert(struct root_lookup *sort_tree,
 
while (*p) {
parent = *p;
-   curr = rb_entry(parent, struct root_info, sort_node);
+   curr = to_root_info_sorted(parent);
 
ret = sort_comp(ins, curr, comp_set);
if (ret < 0)
@@ -340,7 +350,7 @@ static int root_tree_insert(struct root_lookup *root_tree,
 
while(*p) {
parent = *p;
-   curr = rb_entry(parent, struct root_info, rb_node);
+   curr = to_root_info(parent);
 
ret = comp_entry_with_rootid(ins, curr, 0);
if (ret < 0)
@@ -371,7 +381,7 @@ static struct root_info *root_tree_search(struct 
root_lookup *root_tree,
tmp.root_id = root_id;
 
while(n) {
-   entry = rb_entry(n, struct root_info, rb_node);
+   entry = to_root_info(n);
 
ret = comp_entry_with_rootid(, entry, 0);
if (ret < 0)
@@ -528,7 +538,7 @@ static void free_root_info(struct rb_node *node)
 {
struct root_info *ri;
 
-   ri = rb_entry(node, struct root_info, rb_node);
+   ri = to_root_info(node);
free(ri->name);
free(ri->path);
free(ri->full_path);
@@ -1268,7 +1278,7 @@ static void filter_and_sort_subvol(struct root_lookup 
*all_subvols,
 
n = rb_last(_subvols->root);
while (n) {
-   entry = rb_entry(n, struct root_info, rb_node);
+   entry = to_root_info(n);
 
ret = resolve_root(all_subvols, entry, top_id);
if (ret == -ENOENT) {
@@ -1300,7 +1310,7 @@ static int list_subvol_fill_paths(int fd, struct 
root_lookup *root_lookup)
while (n) {
struct root_info *entry;
int ret;
-   entry = rb_entry(n, struct root_info, rb_node);
+   entry = to_root_info(n);
ret = lookup_ino_path(fd, entry);
if (ret && ret != -ENOENT)
return ret;
@@ -1467,7 +1477,7 @@ static void print_all_subvol_info(struct root_lookup 
*sorted_tree,
 
n = rb_first(_tree->root);
while (n) {
-   entry = rb_entry(n, struct root_info, sort_node);
+   entry = to_root_info_sorted(n);
 
/* The toplevel subvolume is not listed by default */
if (entry->root_id == BTRFS_FS_TREE_OBJECTID)
@@ -1558,7 +1568,7 @@ int btrfs_get_toplevel_subvol(int fd, struct root_info 
*the_ri)
return ret;
 
rbn = rb_first();
-   ri = rb_entry(rbn, struct root_info, rb_node);
+   ri = to_root_info(rbn);
 
if (ri->root_id != BTRFS_FS_TREE_OBJECTID)
return -ENOENT;
@@ -1590,7 +1600,7 @@ int btrfs_get_subvol(int fd, struct root_info *the_ri)
 
rbn = rb_first();
while(rbn) {
-   ri = rb_entry(rbn, struct root_info, rb_node);
+   ri = to_root_info(rbn);
rr = resolve_root(, ri, root_id);
if (rr == -ENOENT) {
ret = -ENOENT;
@@ -1814,7 +1824,7 @@ char *btrfs_list_path_for_root(int fd, u64 root)
while (n) {
struct root_info *entry;
 
-   entry = rb_entry(n, struct root_info, rb_node);
+   entry = to_root_info(n);
ret = resolve_root(_lookup, entry, top_id);
if (ret == -ENOENT && entry->root_id == root) {
ret_path = NULL;
-- 
2.15.1

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 08/18] btrfs-progs: subvolume: add quota info to btrfs sub show

[jeffm]

From: Jeff Mahoney 

This patch reports on the first-level qgroup, if any, associated with
a particular subvolume.  It displays the usage and limit, subject
to the usual unit parameters.

Signed-off-by: Jeff Mahoney 
---
 cmds-subvolume.c | 51 ++-
 1 file changed, 50 insertions(+), 1 deletion(-)

diff --git a/cmds-subvolume.c b/cmds-subvolume.c
index 45363a5a..d3c8e37c 100644
--- a/cmds-subvolume.c
+++ b/cmds-subvolume.c
@@ -910,6 +910,7 @@ static const char * const cmd_subvol_show_usage[] = {
"Show more information about the subvolume",
"-r|--rootid   rootid of the subvolume",
"-u|--uuid uuid of the subvolume",
+   HELPINFO_UNITS_SHORT_LONG,
"",
"If no option is specified,  will be shown, otherwise",
"the rootid or uuid are resolved relative to the  path.",
@@ -932,6 +933,13 @@ static int cmd_subvol_show(int argc, char **argv)
struct btrfs_util_subvolume_info subvol;
char *subvol_path = NULL;
enum btrfs_util_error err;
+   struct btrfs_qgroup_stats stats;
+   unsigned int unit_mode;
+   const char *referenced_size;
+   const char *referenced_limit_size = "-";
+   unsigned int field_width = 0;
+
+   unit_mode = get_unit_mode_from_arg(, argv, 1);
 
while (1) {
int c;
@@ -1101,7 +1109,48 @@ static int cmd_subvol_show(int argc, char **argv)
}
btrfs_util_destroy_subvolume_iterator(iter);
 
-   ret = 0;
+   ret = btrfs_qgroup_query(fd, subvol.id, );
+   if (ret && ret != -ENOTTY && ret != -ENODATA) {
+   fprintf(stderr,
+   "\nERROR: BTRFS_IOC_QUOTA_QUERY failed: %s\n",
+   strerror(-ret));
+   goto out;
+   }
+
+   printf("\tQuota Usage:\t\t");
+   fflush(stdout);
+   if (ret) {
+   if (ret == -ENOTTY)
+   printf("quotas not enabled\n");
+   else
+   printf("quotas not available\n");
+   goto out;
+   }
+
+   referenced_size = pretty_size_mode(stats.info.referenced, unit_mode);
+   if (stats.limit.max_referenced)
+   referenced_limit_size = pretty_size_mode(
+   stats.limit.max_referenced,
+   unit_mode);
+   field_width = max(strlen(referenced_size),
+ strlen(referenced_limit_size));
+
+   printf("%-*s referenced, %s exclusive\n ", field_width,
+  referenced_size,
+  pretty_size_mode(stats.info.exclusive, unit_mode));
+
+   printf("\tQuota Limits:\t\t");
+   if (stats.limit.max_referenced || stats.limit.max_exclusive) {
+   const char *excl = "-";
+
+   if (stats.limit.max_exclusive)
+   excl = pretty_size_mode(stats.limit.max_exclusive,
+   unit_mode);
+   printf("%-*s referenced, %s exclusive\n", field_width,
+  referenced_limit_size, excl);
+   } else
+   printf("None\n");
+
 out:
free(subvol_path);
close_file_or_dir(fd, dirstream1);
-- 
2.15.1

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 10/18] btrfs-progs: reorder placement of help declarations for send/receive

[jeffm]

From: Jeff Mahoney 

The usage definitions for send and receive follow the command
definitions, which use them.  This works because we declare them
in commands.h.  When we move to using cmd_struct as the entry point,
these declarations will be removed, breaking the commands.  Since
that would be an otherwise unrelated change, this patch reorders
them separately.

Signed-off-by: Jeff Mahoney 
---
 cmds-receive.c | 62 ++--
 cmds-send.c| 69 +-
 2 files changed, 66 insertions(+), 65 deletions(-)

diff --git a/cmds-receive.c b/cmds-receive.c
index 68123a31..b3709f36 100644
--- a/cmds-receive.c
+++ b/cmds-receive.c
@@ -1248,6 +1248,37 @@ out:
return ret;
 }
 
+const char * const cmd_receive_usage[] = {
+   "btrfs receive [options] \n"
+   "btrfs receive --dump [options]",
+   "Receive subvolumes from a stream",
+   "Receives one or more subvolumes that were previously",
+   "sent with btrfs send. The received subvolumes are stored",
+   "into MOUNT.",
+   "The receive will fail in case the receiving subvolume",
+   "already exists. It will also fail in case a previously",
+   "received subvolume has been changed after it was received.",
+   "After receiving a subvolume, it is immediately set to",
+   "read-only.",
+   "",
+   "-v   increase verbosity about performed actions",
+   "-f FILE  read the stream from FILE instead of stdin",
+   "-e   terminate after receiving an  marker in the 
stream.",
+   " Without this option the receiver side terminates only 
in case",
+   " of an error on end of file.",
+   "-C|--chroot  confine the process to  using chroot",
+   "-E|--max-errors NERR",
+   " terminate as soon as NERR errors occur while",
+   " stream processing commands from the stream.",
+   " Default value is 1. A value of 0 means no limit.",
+   "-m ROOTMOUNT the root mount point of the destination filesystem.",
+   " If /proc is not accessible, use this to tell us 
where",
+   " this file system is mounted.",
+   "--dump   dump stream metadata, one line per operation,",
+   " does not require the MOUNT parameter",
+   NULL
+};
+
 int cmd_receive(int argc, char **argv)
 {
char *tomnt = NULL;
@@ -1357,34 +1388,3 @@ out:
 
return !!ret;
 }
-
-const char * const cmd_receive_usage[] = {
-   "btrfs receive [options] \n"
-   "btrfs receive --dump [options]",
-   "Receive subvolumes from a stream",
-   "Receives one or more subvolumes that were previously",
-   "sent with btrfs send. The received subvolumes are stored",
-   "into MOUNT.",
-   "The receive will fail in case the receiving subvolume",
-   "already exists. It will also fail in case a previously",
-   "received subvolume has been changed after it was received.",
-   "After receiving a subvolume, it is immediately set to",
-   "read-only.",
-   "",
-   "-v   increase verbosity about performed actions",
-   "-f FILE  read the stream from FILE instead of stdin",
-   "-e   terminate after receiving an  marker in the 
stream.",
-   " Without this option the receiver side terminates only 
in case",
-   " of an error on end of file.",
-   "-C|--chroot  confine the process to  using chroot",
-   "-E|--max-errors NERR",
-   " terminate as soon as NERR errors occur while",
-   " stream processing commands from the stream.",
-   " Default value is 1. A value of 0 means no limit.",
-   "-m ROOTMOUNT the root mount point of the destination filesystem.",
-   " If /proc is not accessible, use this to tell us 
where",
-   " this file system is mounted.",
-   "--dump   dump stream metadata, one line per operation,",
-   " does not require the MOUNT parameter",
-   NULL
-};
diff --git a/cmds-send.c b/cmds-send.c
index c5ecdaa1..8365e9c9 100644
--- a/cmds-send.c
+++ b/cmds-send.c
@@ -489,6 +489,41 @@ static void free_send_info(struct btrfs_send *sctx)
subvol_uuid_search_finit(>sus);
 }
 
+
+const char * const cmd_send_usage[] = {
+   "btrfs send [-ve] [-p ] [-c ] [-f ] 
 [...]",
+   "Send the subvolume(s) to stdout.",
+   "Sends the subvolume(s) specified by  to stdout.",
+   " should be read-only here.",
+   "By default, this will send the whole subvolume. To do an incremental",
+   "send, use '-p '. If you want to allow btrfs to clone from",
+   "any additional local snapshots, use '-c ' (multiple times",
+   

[PATCH 12/18] btrfs-progs: use cmd_struct as command entry point

[jeffm]

From: Jeff Mahoney 

Rather than having global command usage and callbacks used to create
cmd_structs in the command array, establish the cmd_struct structures
separately and use those.  The next commit in the series passes the
cmd_struct to the command callbacks such that we can access flags
and determine which of several potential command we were called as.

This establishes several macros to more easily define the commands
within each command's source.

Signed-off-by: Jeff Mahoney 
---
 btrfs.c   |  48 ++
 check/main.c  |   5 +-
 cmds-balance.c|  27 ++
 cmds-device.c |  31 ++-
 cmds-fi-du.c  |   5 +-
 cmds-fi-usage.c   |   5 +-
 cmds-filesystem.c |  52 ++-
 cmds-inspect-dump-super.c |   5 +-
 cmds-inspect-dump-tree.c  |   5 +-
 cmds-inspect-tree-stats.c |   5 +-
 cmds-inspect.c|  36 +++--
 cmds-property.c   |  19 +++
 cmds-qgroup.c |  31 +--
 cmds-quota.c  |  17 ---
 cmds-receive.c|   5 +-
 cmds-replace.c|  19 +++
 cmds-rescue.c |  22 
 cmds-restore.c|   5 +-
 cmds-scrub.c  |  20 +---
 cmds-send.c   |   6 +--
 cmds-subvolume.c  |  38 --
 commands.h| 127 +++---
 help.c|  21 +---
 23 files changed, 317 insertions(+), 237 deletions(-)

diff --git a/btrfs.c b/btrfs.c
index fec1a135..1e68b0c0 100644
--- a/btrfs.c
+++ b/btrfs.c
@@ -42,10 +42,11 @@ static inline const char *skip_prefix(const char *str, 
const char *prefix)
 static int parse_one_token(const char *arg, const struct cmd_group *grp,
   const struct cmd_struct **cmd_ret)
 {
-   const struct cmd_struct *cmd = grp->commands;
const struct cmd_struct *abbrev_cmd = NULL, *ambiguous_cmd = NULL;
+   int i = 0;
 
-   for (; cmd->token; cmd++) {
+   for (i = 0; grp->commands[i]; i++) {
+   const struct cmd_struct *cmd = grp->commands[i];
const char *rest;
 
rest = skip_prefix(arg, cmd->token);
@@ -134,7 +135,7 @@ int handle_command_group(const struct cmd_group *grp, int 
argc,
handle_help_options_next_level(cmd, argc, argv);
 
fixup_argv0(argv, cmd->token);
-   return cmd->fn(argc, argv);
+   return cmd_execute(cmd, argc, argv);
 }
 
 static const struct cmd_group btrfs_cmd_group;
@@ -153,6 +154,8 @@ static int cmd_help(int argc, char **argv)
return 0;
 }
 
+static DEFINE_SIMPLE_COMMAND(help, "help");
+
 static const char * const cmd_version_usage[] = {
"btrfs version",
"Display btrfs-progs version",
@@ -164,6 +167,7 @@ static int cmd_version(int argc, char **argv)
printf("%s\n", PACKAGE_STRING);
return 0;
 }
+static DEFINE_SIMPLE_COMMAND(version, "version");
 
 /*
  * Parse global options, between binary name and first non-option argument
@@ -240,24 +244,24 @@ void handle_special_globals(int shift, int argc, char 
**argv)
 
 static const struct cmd_group btrfs_cmd_group = {
btrfs_cmd_group_usage, btrfs_cmd_group_info, {
-   { "subvolume", cmd_subvolume, NULL, _cmd_group, 0 },
-   { "filesystem", cmd_filesystem, NULL, _cmd_group, 0 
},
-   { "balance", cmd_balance, NULL, _cmd_group, 0 },
-   { "device", cmd_device, NULL, _cmd_group, 0 },
-   { "scrub", cmd_scrub, NULL, _cmd_group, 0 },
-   { "check", cmd_check, cmd_check_usage, NULL, 0 },
-   { "rescue", cmd_rescue, NULL, _cmd_group, 0 },
-   { "restore", cmd_restore, cmd_restore_usage, NULL, 0 },
-   { "inspect-internal", cmd_inspect, NULL, _cmd_group, 0 
},
-   { "property", cmd_property, NULL, _cmd_group, 0 },
-   { "send", cmd_send, cmd_send_usage, NULL, 0 },
-   { "receive", cmd_receive, cmd_receive_usage, NULL, 0 },
-   { "quota", cmd_quota, NULL, _cmd_group, 0 },
-   { "qgroup", cmd_qgroup, NULL, _cmd_group, 0 },
-   { "replace", cmd_replace, NULL, _cmd_group, 0 },
-   { "help", cmd_help, cmd_help_usage, NULL, 0 },
-   { "version", cmd_version, cmd_version_usage, NULL, 0 },
-   NULL_CMD_STRUCT
+   _struct_subvolume,
+   _struct_filesystem,
+   _struct_balance,
+   _struct_device,
+   _struct_scrub,
+   _struct_check,
+   _struct_rescue,
+   _struct_restore,
+   _struct_inspect,
+   _struct_property,
+   _struct_send,
+   _struct_receive,
+   _struct_quota,
+   _struct_qgroup,
+   _struct_replace,
+   _struct_help,
+   

[PATCH 17/18] btrfs-progs: handle command groups directly for common case

[jeffm]

From: Jeff Mahoney 

Most command groups just pass their own command group to
handle_command_group.  We can remove the explicit definitions
of command group callbacks by passing the cmd_struct to
handle_command_group and allowing it to resolve the group from it.

Signed-off-by: Jeff Mahoney 
---
 btrfs.c   | 14 +++---
 cmds-balance.c|  6 +++---
 cmds-device.c |  5 -
 cmds-filesystem.c |  6 --
 cmds-inspect.c|  5 -
 cmds-property.c   |  6 --
 cmds-qgroup.c |  5 -
 cmds-quota.c  |  5 -
 cmds-replace.c|  5 -
 cmds-rescue.c |  5 -
 cmds-scrub.c  |  6 --
 cmds-subvolume.c  |  5 -
 commands.h|  7 ---
 13 files changed, 14 insertions(+), 66 deletions(-)

diff --git a/btrfs.c b/btrfs.c
index 32b8e090..427e14c8 100644
--- a/btrfs.c
+++ b/btrfs.c
@@ -140,26 +140,26 @@ static void handle_help_options_next_level(const struct 
cmd_struct *cmd,
}
 }
 
-int handle_command_group(const struct cmd_group *grp,
+int handle_command_group(const struct cmd_struct *cmd,
 const struct cmd_context *cmdcxt,
 int argc, char **argv)
 
 {
-   const struct cmd_struct *cmd;
+   const struct cmd_struct *subcmd;
 
argc--;
argv++;
if (argc < 1) {
-   usage_command_group(grp, false, false);
+   usage_command_group(cmd->next, false, false);
exit(1);
}
 
-   cmd = parse_command_token(argv[0], grp);
+   subcmd = parse_command_token(argv[0], cmd->next);
 
-   handle_help_options_next_level(cmd, cmdcxt, argc, argv);
+   handle_help_options_next_level(subcmd, cmdcxt, argc, argv);
 
-   fixup_argv0(argv, cmd->token);
-   return cmd_execute(cmd, cmdcxt, argc, argv);
+   fixup_argv0(argv, subcmd->token);
+   return cmd_execute(subcmd, cmdcxt, argc, argv);
 }
 
 static const struct cmd_group btrfs_cmd_group;
diff --git a/cmds-balance.c b/cmds-balance.c
index c17b9ee3..e414ca27 100644
--- a/cmds-balance.c
+++ b/cmds-balance.c
@@ -943,7 +943,7 @@ static const struct cmd_group balance_cmd_group = {
}
 };
 
-static int cmd_balance(const struct cmd_struct *unused,
+static int cmd_balance(const struct cmd_struct *cmd,
   const struct cmd_context *cmdcxt, int argc, char **argv)
 {
if (argc == 2 && strcmp("start", argv[1]) != 0) {
@@ -956,7 +956,7 @@ static int cmd_balance(const struct cmd_struct *unused,
return do_balance(argv[1], , 0);
}
 
-   return handle_command_group(_cmd_group, cmdcxt, argc, argv);
+   return handle_command_group(cmd, cmdcxt, argc, argv);
 }
 
-DEFINE_GROUP_COMMAND(balance, "balance");
+DEFINE_COMMAND(balance, "balance", cmd_balance, NULL, _cmd_group, 0, 
0);
diff --git a/cmds-device.c b/cmds-device.c
index ac9e82b1..f8c0ff20 100644
--- a/cmds-device.c
+++ b/cmds-device.c
@@ -630,9 +630,4 @@ static const struct cmd_group device_cmd_group = {
}
 };
 
-static int cmd_device(const struct cmd_struct *unused,
- const struct cmd_context *cmdcxt, int argc, char **argv)
-{
-   return handle_command_group(_cmd_group, cmdcxt, argc, argv);
-}
 DEFINE_GROUP_COMMAND_TOKEN(device);
diff --git a/cmds-filesystem.c b/cmds-filesystem.c
index 6701b16f..24852ec6 100644
--- a/cmds-filesystem.c
+++ b/cmds-filesystem.c
@@ -1235,10 +1235,4 @@ static const struct cmd_group filesystem_cmd_group = {
}
 };
 
-static int cmd_filesystem(const struct cmd_struct *unused,
- const struct cmd_context *cmdcxt,
- int argc, char **argv)
-{
-   return handle_command_group(_cmd_group, cmdcxt, argc, argv);
-}
 DEFINE_GROUP_COMMAND_TOKEN(filesystem);
diff --git a/cmds-inspect.c b/cmds-inspect.c
index eecfd7f9..22c5a5d6 100644
--- a/cmds-inspect.c
+++ b/cmds-inspect.c
@@ -658,9 +658,4 @@ static const struct cmd_group inspect_cmd_group = {
}
 };
 
-static int cmd_inspect(const struct cmd_struct *unused,
-  const struct cmd_context *cmdcxt, int argc, char **argv)
-{
-   return handle_command_group(_cmd_group, cmdcxt, argc, argv);
-}
 DEFINE_GROUP_COMMAND(inspect, "inspect-internal");
diff --git a/cmds-property.c b/cmds-property.c
index 498fa456..58f6c48a 100644
--- a/cmds-property.c
+++ b/cmds-property.c
@@ -422,10 +422,4 @@ static const struct cmd_group property_cmd_group = {
}
 };
 
-static int cmd_property(const struct cmd_struct *unused,
-   const struct cmd_context *cmdcxt,
-   int argc, char **argv)
-{
-   return handle_command_group(_cmd_group, cmdcxt, argc, argv);
-}
 DEFINE_GROUP_COMMAND_TOKEN(property);
diff --git a/cmds-qgroup.c b/cmds-qgroup.c
index 9325b568..ce9aa1c6 100644
--- a/cmds-qgroup.c
+++ b/cmds-qgroup.c
@@ -541,9 +541,4 @@ static const struct cmd_group qgroup_cmd_group = {
}
 };
 
-static int cmd_qgroup(const 

[PATCH 09/18] btrfs-progs: help: convert ints used as bools to bool

[jeffm]

From: Jeff Mahoney 

We use an int for 'full', 'all', and 'err' when we really mean a boolean.

Reviewed-by: Qu Wenruo 
Signed-off-by: Jeff Mahoney 
---
 btrfs.c | 14 +++---
 help.c  | 25 +
 help.h  |  4 ++--
 3 files changed, 22 insertions(+), 21 deletions(-)

diff --git a/btrfs.c b/btrfs.c
index 2d39f2ce..fec1a135 100644
--- a/btrfs.c
+++ b/btrfs.c
@@ -109,7 +109,7 @@ static void handle_help_options_next_level(const struct 
cmd_struct *cmd,
argv++;
help_command_group(cmd->next, argc, argv);
} else {
-   usage_command(cmd, 1, 0);
+   usage_command(cmd, true, false);
}
 
exit(0);
@@ -125,7 +125,7 @@ int handle_command_group(const struct cmd_group *grp, int 
argc,
argc--;
argv++;
if (argc < 1) {
-   usage_command_group(grp, 0, 0);
+   usage_command_group(grp, false, false);
exit(1);
}
 
@@ -212,20 +212,20 @@ static int handle_global_options(int argc, char **argv)
 
 void handle_special_globals(int shift, int argc, char **argv)
 {
-   int has_help = 0;
-   int has_full = 0;
+   bool has_help = false;
+   bool has_full = false;
int i;
 
for (i = 0; i < shift; i++) {
if (strcmp(argv[i], "--help") == 0)
-   has_help = 1;
+   has_help = true;
else if (strcmp(argv[i], "--full") == 0)
-   has_full = 1;
+   has_full = true;
}
 
if (has_help) {
if (has_full)
-   usage_command_group(_cmd_group, 1, 0);
+   usage_command_group(_cmd_group, true, false);
else
cmd_help(argc, argv);
exit(0);
diff --git a/help.c b/help.c
index f1dd3946..99fd325b 100644
--- a/help.c
+++ b/help.c
@@ -196,8 +196,8 @@ static int do_usage_one_command(const char * const 
*usagestr,
 }
 
 static int usage_command_internal(const char * const *usagestr,
- const char *token, int full, int lst,
- int alias, FILE *outf)
+ const char *token, bool full, bool lst,
+ bool alias, FILE *outf)
 {
unsigned int flags = 0;
int ret;
@@ -223,17 +223,17 @@ static int usage_command_internal(const char * const 
*usagestr,
 }
 
 static void usage_command_usagestr(const char * const *usagestr,
-  const char *token, int full, int err)
+  const char *token, bool full, bool err)
 {
FILE *outf = err ? stderr : stdout;
int ret;
 
-   ret = usage_command_internal(usagestr, token, full, 0, 0, outf);
+   ret = usage_command_internal(usagestr, token, full, false, false, outf);
if (!ret)
fputc('\n', outf);
 }
 
-void usage_command(const struct cmd_struct *cmd, int full, int err)
+void usage_command(const struct cmd_struct *cmd, bool full, bool err)
 {
usage_command_usagestr(cmd->usagestr, cmd->token, full, err);
 }
@@ -241,11 +241,11 @@ void usage_command(const struct cmd_struct *cmd, int 
full, int err)
 __attribute__((noreturn))
 void usage(const char * const *usagestr)
 {
-   usage_command_usagestr(usagestr, NULL, 1, 1);
+   usage_command_usagestr(usagestr, NULL, true, true);
exit(1);
 }
 
-static void usage_command_group_internal(const struct cmd_group *grp, int full,
+static void usage_command_group_internal(const struct cmd_group *grp, bool 
full,
 FILE *outf)
 {
const struct cmd_struct *cmd = grp->commands;
@@ -265,7 +265,8 @@ static void usage_command_group_internal(const struct 
cmd_group *grp, int full,
}
 
usage_command_internal(cmd->usagestr, cmd->token, full,
-  1, cmd->flags & CMD_ALIAS, outf);
+  true, cmd->flags & CMD_ALIAS,
+  outf);
if (cmd->flags & CMD_ALIAS)
putchar('\n');
continue;
@@ -327,7 +328,7 @@ void usage_command_group_short(const struct cmd_group *grp)
fprintf(stderr, "All command groups have their manual page named 
'btrfs-'.\n");
 }
 
-void usage_command_group(const struct cmd_group *grp, int full, int err)
+void usage_command_group(const struct cmd_group *grp, bool full, bool err)
 {
const char * const *usagestr = grp->usagestr;
FILE *outf = err ? stderr : stdout;
@@ -350,7 +351,7 @@ __attribute__((noreturn))
 void help_unknown_token(const char *arg, const struct cmd_group *grp)
 {

[PATCH 07/18] btrfs-progs: qgroups: introduce btrfs_qgroup_query

[jeffm]

From: Jeff Mahoney 

The only mechanism we have in the progs for searching qgroups is to load
all of them and filter the results.  This works for qgroup show but
to add quota information to 'btrfs subvoluem show' it's pretty wasteful.

This patch splits out setting up the search and performing the search so
we can search for a single qgroupid more easily.  Since TREE_SEARCH
will give results that don't strictly match the search terms, we add
a filter to match only the results we care about.

Reviewed-by: Qu Wenruo 
Signed-off-by: Jeff Mahoney 
---
 qgroup.c | 143 ---
 qgroup.h |   7 
 2 files changed, 116 insertions(+), 34 deletions(-)

diff --git a/qgroup.c b/qgroup.c
index 247f1bfe..647bc2f3 100644
--- a/qgroup.c
+++ b/qgroup.c
@@ -1162,11 +1162,30 @@ static inline void print_status_flag_warning(u64 flags)
warning("qgroup data inconsistent, rescan recommended");
 }
 
-static int __qgroups_search(int fd, struct qgroup_lookup *qgroup_lookup)
+static bool key_in_range(const struct btrfs_key *key,
+const struct btrfs_ioctl_search_key *sk)
+{
+   if (key->objectid < sk->min_objectid ||
+   key->objectid > sk->max_objectid)
+   return false;
+
+   if (key->type < sk->min_type ||
+   key->type > sk->max_type)
+   return false;
+
+   if (key->offset < sk->min_offset ||
+   key->offset > sk->max_offset)
+   return false;
+
+   return true;
+}
+
+static int __qgroups_search(int fd, struct btrfs_ioctl_search_args *args,
+   struct qgroup_lookup *qgroup_lookup)
 {
int ret;
-   struct btrfs_ioctl_search_args args;
-   struct btrfs_ioctl_search_key *sk = 
+   struct btrfs_ioctl_search_key *sk = >key;
+   struct btrfs_ioctl_search_key filter_key = args->key;
struct btrfs_ioctl_search_header *sh;
unsigned long off = 0;
unsigned int i;
@@ -1177,30 +1196,15 @@ static int __qgroups_search(int fd, struct 
qgroup_lookup *qgroup_lookup)
u64 qgroupid;
u64 child, parent;
 
-   memset(, 0, sizeof(args));
-
-   sk->tree_id = BTRFS_QUOTA_TREE_OBJECTID;
-   sk->max_type = BTRFS_QGROUP_RELATION_KEY;
-   sk->min_type = BTRFS_QGROUP_STATUS_KEY;
-   sk->max_objectid = (u64)-1;
-   sk->max_offset = (u64)-1;
-   sk->max_transid = (u64)-1;
-   sk->nr_items = 4096;
-
qgroup_lookup_init(qgroup_lookup);
 
while (1) {
-   ret = ioctl(fd, BTRFS_IOC_TREE_SEARCH, );
+   ret = ioctl(fd, BTRFS_IOC_TREE_SEARCH, args);
if (ret < 0) {
-   if (errno == ENOENT) {
-   error("can't list qgroups: quotas not enabled");
+   if (errno == ENOENT)
ret = -ENOTTY;
-   } else {
-   error("can't list qgroups: %s",
-  strerror(errno));
+   else
ret = -errno;
-   }
-
break;
}
 
@@ -1214,37 +1218,46 @@ static int __qgroups_search(int fd, struct 
qgroup_lookup *qgroup_lookup)
 * read the root_ref item it contains
 */
for (i = 0; i < sk->nr_items; i++) {
-   sh = (struct btrfs_ioctl_search_header *)(args.buf +
+   struct btrfs_key key;
+
+   sh = (struct btrfs_ioctl_search_header *)(args->buf +
  off);
off += sizeof(*sh);
 
-   switch (btrfs_search_header_type(sh)) {
+   key.objectid = btrfs_search_header_objectid(sh);
+   key.type = btrfs_search_header_type(sh);
+   key.offset = btrfs_search_header_offset(sh);
+
+   if (!key_in_range(, _key))
+   goto next;
+
+   switch (key.type) {
case BTRFS_QGROUP_STATUS_KEY:
si = (struct btrfs_qgroup_status_item *)
-(args.buf + off);
+(args->buf + off);
flags = btrfs_stack_qgroup_status_flags(si);
 
print_status_flag_warning(flags);
break;
case BTRFS_QGROUP_INFO_KEY:
-   qgroupid = btrfs_search_header_offset(sh);
+   qgroupid = key.offset;
info = (struct btrfs_qgroup_info_item *)
-  (args.buf + off);
+  

[PATCH 06/18] btrfs-progs: qgroups: introduce and use info and limit structures

[jeffm]

From: Jeff Mahoney 

We use structures to pass the info and limit from the kernel as items
but store the individual values separately in btrfs_qgroup.  We already
have a btrfs_qgroup_limit structure that's used for setting the limit.

This patch introduces a btrfs_qgroup_info structure and uses that and
btrfs_qgroup_limit in btrfs_qgroup.

Reviewed-by: Qu Wenruo 
Signed-off-by: Jeff Mahoney 
---
 qgroup.c | 82 ++--
 qgroup.h |  8 +++
 2 files changed, 52 insertions(+), 38 deletions(-)

diff --git a/qgroup.c b/qgroup.c
index d8baca33..247f1bfe 100644
--- a/qgroup.c
+++ b/qgroup.c
@@ -47,20 +47,12 @@ struct btrfs_qgroup {
/*
 * info_item
 */
-   u64 generation;
-   u64 rfer;   /*referenced*/
-   u64 rfer_cmpr;  /*referenced compressed*/
-   u64 excl;   /*exclusive*/
-   u64 excl_cmpr;  /*exclusive compressed*/
+   struct btrfs_qgroup_info info;
 
/*
 *limit_item
 */
-   u64 flags;  /*which limits are set*/
-   u64 max_rfer;
-   u64 max_excl;
-   u64 rsv_rfer;
-   u64 rsv_excl;
+   struct btrfs_qgroup_limit limit;
 
/*qgroups this group is member of*/
struct list_head qgroups;
@@ -262,6 +254,11 @@ void print_pathname_column(struct btrfs_qgroup *qgroup, 
bool verbose)
fputs("", stdout);
 }
 
+static int print_u64(u64 value, int unit_mode, int max_len)
+{
+   return printf("%*s", max_len, pretty_size_mode(value, unit_mode));
+}
+
 static void print_qgroup_column(struct btrfs_qgroup *qgroup,
enum btrfs_qgroup_column_enum column,
bool verbose)
@@ -281,24 +278,26 @@ static void print_qgroup_column(struct btrfs_qgroup 
*qgroup,
print_qgroup_column_add_blank(BTRFS_QGROUP_QGROUPID, len);
break;
case BTRFS_QGROUP_RFER:
-   len = printf("%*s", max_len, pretty_size_mode(qgroup->rfer, 
unit_mode));
+   len = print_u64(qgroup->info.referenced, unit_mode, max_len);
break;
case BTRFS_QGROUP_EXCL:
-   len = printf("%*s", max_len, pretty_size_mode(qgroup->excl, 
unit_mode));
+   len = print_u64(qgroup->info.exclusive, unit_mode, max_len);
break;
case BTRFS_QGROUP_PARENT:
len = print_parent_column(qgroup);
print_qgroup_column_add_blank(BTRFS_QGROUP_PARENT, len);
break;
case BTRFS_QGROUP_MAX_RFER:
-   if (qgroup->flags & BTRFS_QGROUP_LIMIT_MAX_RFER)
-   len = printf("%*s", max_len, 
pretty_size_mode(qgroup->max_rfer, unit_mode));
+   if (qgroup->limit.flags & BTRFS_QGROUP_LIMIT_MAX_RFER)
+   len = print_u64(qgroup->limit.max_referenced,
+   unit_mode, max_len);
else
len = printf("%*s", max_len, "none");
break;
case BTRFS_QGROUP_MAX_EXCL:
-   if (qgroup->flags & BTRFS_QGROUP_LIMIT_MAX_EXCL)
-   len = printf("%*s", max_len, 
pretty_size_mode(qgroup->max_excl, unit_mode));
+   if (qgroup->limit.flags & BTRFS_QGROUP_LIMIT_MAX_EXCL)
+   len = print_u64(qgroup->limit.max_exclusive,
+   unit_mode, max_len);
else
len = printf("%*s", max_len, "none");
break;
@@ -441,9 +440,9 @@ static int comp_entry_with_rfer(struct btrfs_qgroup *entry1,
 {
int ret;
 
-   if (entry1->rfer > entry2->rfer)
+   if (entry1->info.referenced > entry2->info.referenced)
ret = 1;
-   else if (entry1->rfer < entry2->rfer)
+   else if (entry1->info.referenced < entry2->info.referenced)
ret = -1;
else
ret = 0;
@@ -457,9 +456,9 @@ static int comp_entry_with_excl(struct btrfs_qgroup *entry1,
 {
int ret;
 
-   if (entry1->excl > entry2->excl)
+   if (entry1->info.exclusive > entry2->info.exclusive)
ret = 1;
-   else if (entry1->excl < entry2->excl)
+   else if (entry1->info.exclusive < entry2->info.exclusive)
ret = -1;
else
ret = 0;
@@ -473,9 +472,9 @@ static int comp_entry_with_max_rfer(struct btrfs_qgroup 
*entry1,
 {
int ret;
 
-   if (entry1->max_rfer > entry2->max_rfer)
+   if (entry1->limit.max_referenced > entry2->limit.max_referenced)
ret = 1;
-   else if (entry1->max_rfer < entry2->max_rfer)
+   else if (entry1->limit.max_referenced < entry2->limit.max_referenced)
ret = -1;
else
ret = 0;
@@ -489,9 +488,9 @@ static int comp_entry_with_max_excl(struct btrfs_qgroup 
*entry1,
 {
int ret;
 
-   if 

[PATCH 16/18] btrfs-progs: add support for output formats

[jeffm]

From: Jeff Mahoney 

This adds a global --format option to request extended output formats
from each command.  Most of it is plumbing a new cmd_context structure
that's established at the beginning of argument parsing into the command
callbacks.  That structure currently only contains the output mode enum.

We currently only support text mode.  Command help reports what
output formats are available for each command.  Global help reports
what valid formats are.

If an invalid format is requested, an error is reported and we global usage
is dumped that lists the valid formats.

Each command sets a bitmask that describes which formats it is capable
of outputting.  If a globally valid format is requested of a command
that doesn't support it, an error is reported and command usage dumped.

Commands don't need to specify that they support text output.  All
commands are required to output text.

Signed-off-by: Jeff Mahoney 
---
 btrfs.c   | 110 ++
 check/main.c  |   3 +-
 cmds-balance.c|  16 +--
 cmds-device.c |  31 +
 cmds-fi-du.c  |   1 +
 cmds-fi-usage.c   |   1 +
 cmds-filesystem.c |  14 --
 cmds-inspect-dump-super.c |   1 +
 cmds-inspect-dump-tree.c  |   1 +
 cmds-inspect-tree-stats.c |   1 +
 cmds-inspect.c|  10 -
 cmds-property.c   |   6 ++-
 cmds-qgroup.c |  17 +--
 cmds-quota.c  |  14 --
 cmds-receive.c|   3 +-
 cmds-replace.c|   8 +++-
 cmds-rescue.c |   9 +++-
 cmds-restore.c|   3 +-
 cmds-scrub.c  |  21 ++---
 cmds-send.c   |   3 +-
 cmds-subvolume.c  |  24 +++---
 commands.h|  32 +++---
 help.c|  51 -
 help.h|   2 +
 24 files changed, 299 insertions(+), 83 deletions(-)

diff --git a/btrfs.c b/btrfs.c
index 49128182..32b8e090 100644
--- a/btrfs.c
+++ b/btrfs.c
@@ -26,7 +26,7 @@
 #include "help.h"
 
 static const char * const btrfs_cmd_group_usage[] = {
-   "btrfs [--help] [--version]  [...]  []",
+   "btrfs [--help] [--version] [--format ]  [...] 
 []",
NULL
 };
 
@@ -98,13 +98,36 @@ parse_command_token(const char *arg, const struct cmd_group 
*grp)
return cmd;
 }
 
+static bool cmd_provides_output_format(const struct cmd_struct *cmd,
+  const struct cmd_context *cmdcxt)
+{
+   if (!cmdcxt->output_mode)
+   return true;
+
+   return (1 << cmdcxt->output_mode) & cmd->cmd_format_flags;
+}
+
 static void handle_help_options_next_level(const struct cmd_struct *cmd,
-   int argc, char **argv)
+  const struct cmd_context *cmdcxt,
+  int argc, char **argv)
 {
+   int err = 0;
+
if (argc < 2)
return;
 
-   if (!strcmp(argv[1], "--help")) {
+   /* Check if the command can provide the requested output format */
+   if (!cmd->next && !cmd_provides_output_format(cmd, cmdcxt)) {
+   ASSERT(cmdcxt->output_mode >= 0);
+   ASSERT(cmdcxt->output_mode < CMD_OUTPUT_MAX);
+   fprintf(stderr,
+   "error: %s output is unsupported for this command.\n\n",
+   cmd_outputs[cmdcxt->output_mode]);
+
+   err = 1;
+   }
+
+   if (!strcmp(argv[1], "--help") || err) {
if (cmd->next) {
argc--;
argv++;
@@ -113,12 +136,13 @@ static void handle_help_options_next_level(const struct 
cmd_struct *cmd,
usage_command(cmd, true, false);
}
 
-   exit(0);
+   exit(err);
}
 }
 
-int handle_command_group(const struct cmd_group *grp, int argc,
-char **argv)
+int handle_command_group(const struct cmd_group *grp,
+const struct cmd_context *cmdcxt,
+int argc, char **argv)
 
 {
const struct cmd_struct *cmd;
@@ -132,10 +156,10 @@ int handle_command_group(const struct cmd_group *grp, int 
argc,
 
cmd = parse_command_token(argv[0], grp);
 
-   handle_help_options_next_level(cmd, argc, argv);
+   handle_help_options_next_level(cmd, cmdcxt, argc, argv);
 
fixup_argv0(argv, cmd->token);
-   return cmd_execute(cmd, argc, argv);
+   return cmd_execute(cmd, cmdcxt, argc, argv);
 }
 
 static const struct cmd_group btrfs_cmd_group;
@@ -148,7 +172,8 @@ static const char * const cmd_help_usage[] = {
NULL
 };
 
-static int cmd_help(const struct cmd_struct *unused, int argc, char **argv)
+static int cmd_help(const struct cmd_struct *unused,
+   const struct cmd_context *cmdcxt, int argc, char **argv)

[PATCH 02/18] btrfs-progs: qgroups: fix misleading index check

[jeffm]

From: Jeff Mahoney 

In print_single_qgroup_table we check the loop index against
BTRFS_QGROUP_CHILD, but what we really mean is "last column."  Since
we have an enum value to indicate the last value, use that instead
of assuming that BTRFS_QGROUP_CHILD is always last.

Reviewed-by: Qu Wenruo 
Reviewed-by: Nikolay Borisov 
Signed-off-by: Jeff Mahoney 
---
 qgroup.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/qgroup.c b/qgroup.c
index 267cd7f1..3269feb2 100644
--- a/qgroup.c
+++ b/qgroup.c
@@ -267,7 +267,7 @@ static void print_single_qgroup_table(struct btrfs_qgroup 
*qgroup)
continue;
print_qgroup_column(qgroup, i);
 
-   if (i != BTRFS_QGROUP_CHILD)
+   if (i != BTRFS_QGROUP_ALL - 1)
printf(" ");
}
printf("\n");
-- 
2.15.1

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 2/3] btrfs: qgroups, remove unnecessary memset before btrfs_init_work

[jeffm]

From: Jeff Mahoney 

btrfs_init_work clears the work struct except for ->wq, so the memset
before calling btrfs_init_work in qgroup_rescan_init is unnecessary.

We'll also initialize ->wq in btrfs_init_work so that it's obvious.

Signed-off-by: Jeff Mahoney 
---
 fs/btrfs/async-thread.c | 1 +
 fs/btrfs/qgroup.c   | 3 ---
 2 files changed, 1 insertion(+), 3 deletions(-)

diff --git a/fs/btrfs/async-thread.c b/fs/btrfs/async-thread.c
index d5540749f0e5..c614fb7b9b9d 100644
--- a/fs/btrfs/async-thread.c
+++ b/fs/btrfs/async-thread.c
@@ -354,6 +354,7 @@ void btrfs_init_work(struct btrfs_work *work, 
btrfs_work_func_t uniq_func,
INIT_WORK(>normal_work, uniq_func);
INIT_LIST_HEAD(>ordered_list);
work->flags = 0;
+   work->wq = NULL;
 }
 
 static inline void __btrfs_queue_work(struct __btrfs_workqueue *wq,
diff --git a/fs/btrfs/qgroup.c b/fs/btrfs/qgroup.c
index 466744741873..3d47700c6a30 100644
--- a/fs/btrfs/qgroup.c
+++ b/fs/btrfs/qgroup.c
@@ -2102,9 +2102,6 @@ static void queue_rescan_worker(struct btrfs_fs_info 
*fs_info)
init_completion(_info->qgroup_rescan_completion);
mutex_unlock(_info->qgroup_rescan_lock);
 
-   memset(_info->qgroup_rescan_work, 0,
-  sizeof(fs_info->qgroup_rescan_work));
-
btrfs_init_work(_info->qgroup_rescan_work,
btrfs_qgroup_rescan_helper,
btrfs_qgroup_rescan_worker, NULL, NULL);
-- 
2.12.3

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 3/3] btrfs: qgroup, don't try to insert status item after ENOMEM in rescan worker

[jeffm]

From: Jeff Mahoney 

If we fail to allocate memory for a path, don't bother trying to
insert the qgroup status item.  We haven't done anything yet and it'll
fail also.  Just print an error and be done with it.

Signed-off-by: Jeff Mahoney 
---
 fs/btrfs/qgroup.c | 9 -
 1 file changed, 4 insertions(+), 5 deletions(-)

diff --git a/fs/btrfs/qgroup.c b/fs/btrfs/qgroup.c
index 3d47700c6a30..44d5e3da835a 100644
--- a/fs/btrfs/qgroup.c
+++ b/fs/btrfs/qgroup.c
@@ -2666,7 +2666,6 @@ static void btrfs_qgroup_rescan_worker(struct btrfs_work 
*work)
btrfs_end_transaction(trans);
}
 
-out:
btrfs_free_path(path);
 
mutex_lock(_info->qgroup_rescan_lock);
@@ -2702,13 +2701,13 @@ static void btrfs_qgroup_rescan_worker(struct 
btrfs_work *work)
 
if (btrfs_fs_closing(fs_info)) {
btrfs_info(fs_info, "qgroup scan paused");
-   } else if (err >= 0) {
+   err = 0;
+   } else if (err >= 0)
btrfs_info(fs_info, "qgroup scan completed%s",
err > 0 ? " (inconsistency flag cleared)" : "");
-   } else {
+out:
+   if (err < 0)
btrfs_err(fs_info, "qgroup scan failed with %d", err);
-   }
-
 done:
mutex_lock(_info->qgroup_rescan_lock);
fs_info->qgroup_rescan_running = false;
-- 
2.12.3

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v3 0/3] btrfs: qgroup rescan races (part 1)

[jeffm]

From: Jeff Mahoney 

Hi Dave -

Here's the updated patchset for the rescan races.  This fixes the issue
where we'd try to start multiple workers.  It introduces a new "ready"
bool that we set during initialization and clear while queuing the worker.
The queuer is also now responsible for most of the initialization.

I have a separate patch set start that gets rid of the racy mess surrounding
the rescan worker startup.  We can handle it in btrfs_run_qgroups and
just set a flag to start it everywhere else.

-Jeff

---

Jeff Mahoney (3):
  btrfs: qgroups, fix rescan worker running races
  btrfs: qgroups, remove unnecessary memset before btrfs_init_work
  btrfs: qgroup, don't try to insert status item after ENOMEM in rescan
worker

 fs/btrfs/async-thread.c |   1 +
 fs/btrfs/ctree.h|   2 +
 fs/btrfs/qgroup.c   | 100 +++-
 3 files changed, 60 insertions(+), 43 deletions(-)

-- 
2.12.3

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 1/3] btrfs: qgroups, fix rescan worker running races

[jeffm]

From: Jeff Mahoney 

Commit 8d9eddad194 (Btrfs: fix qgroup rescan worker initialization)
fixed the issue with BTRFS_IOC_QUOTA_RESCAN_WAIT being racy, but
ended up reintroducing the hang-on-unmount bug that the commit it
intended to fix addressed.

The race this time is between qgroup_rescan_init setting
->qgroup_rescan_running = true and the worker starting.  There are
many scenarios where we initialize the worker and never start it.  The
completion btrfs_ioctl_quota_rescan_wait waits for will never come.
This can happen even without involving error handling, since mounting
the file system read-only returns between initializing the worker and
queueing it.

The right place to do it is when we're queuing the worker.  The flag
really just means that btrfs_ioctl_quota_rescan_wait should wait for
a completion.

Since the BTRFS_QGROUP_STATUS_FLAG_RESCAN flag is overloaded to
refer to both runtime behavior and on-disk state, we introduce a new
fs_info->qgroup_rescan_ready to indicate that we're initialized and
waiting to start.

This patch introduces a new helper, queue_rescan_worker, that handles
most of the initialization, the two flags, and queuing the worker,
including races with unmount.

While we're at it, ->qgroup_rescan_running is protected only by the
->qgroup_rescan_mutex.  btrfs_ioctl_quota_rescan_wait doesn't need
to take the spinlock too.

Fixes: 8d9eddad194 (Btrfs: fix qgroup rescan worker initialization)
Signed-off-by: Jeff Mahoney 
---
 fs/btrfs/ctree.h  |  2 ++
 fs/btrfs/qgroup.c | 94 +--
 2 files changed, 58 insertions(+), 38 deletions(-)

diff --git a/fs/btrfs/ctree.h b/fs/btrfs/ctree.h
index da308774b8a4..4003498bb714 100644
--- a/fs/btrfs/ctree.h
+++ b/fs/btrfs/ctree.h
@@ -1045,6 +1045,8 @@ struct btrfs_fs_info {
struct btrfs_workqueue *qgroup_rescan_workers;
struct completion qgroup_rescan_completion;
struct btrfs_work qgroup_rescan_work;
+   /* qgroup rescan worker is running or queued to run */
+   bool qgroup_rescan_ready;
bool qgroup_rescan_running; /* protected by qgroup_rescan_lock */
 
/* filesystem state */
diff --git a/fs/btrfs/qgroup.c b/fs/btrfs/qgroup.c
index aa259d6986e1..466744741873 100644
--- a/fs/btrfs/qgroup.c
+++ b/fs/btrfs/qgroup.c
@@ -101,6 +101,7 @@ static int
 qgroup_rescan_init(struct btrfs_fs_info *fs_info, u64 progress_objectid,
   int init_flags);
 static void qgroup_rescan_zero_tracking(struct btrfs_fs_info *fs_info);
+static void btrfs_qgroup_rescan_worker(struct btrfs_work *work);
 
 /* must be called with qgroup_ioctl_lock held */
 static struct btrfs_qgroup *find_qgroup_rb(struct btrfs_fs_info *fs_info,
@@ -2072,6 +2073,46 @@ int btrfs_qgroup_account_extents(struct 
btrfs_trans_handle *trans,
return ret;
 }
 
+static void queue_rescan_worker(struct btrfs_fs_info *fs_info)
+{
+   mutex_lock(_info->qgroup_rescan_lock);
+   if (btrfs_fs_closing(fs_info)) {
+   mutex_unlock(_info->qgroup_rescan_lock);
+   return;
+   }
+
+   if (WARN_ON(!fs_info->qgroup_rescan_ready)) {
+   btrfs_warn(fs_info, "rescan worker not ready");
+   mutex_unlock(_info->qgroup_rescan_lock);
+   return;
+   }
+   fs_info->qgroup_rescan_ready = false;
+
+   if (WARN_ON(fs_info->qgroup_rescan_running)) {
+   btrfs_warn(fs_info, "rescan worker already queued");
+   mutex_unlock(_info->qgroup_rescan_lock);
+   return;
+   }
+
+   /*
+* Being queued is enough for btrfs_qgroup_wait_for_completion
+* to need to wait.
+*/
+   fs_info->qgroup_rescan_running = true;
+   init_completion(_info->qgroup_rescan_completion);
+   mutex_unlock(_info->qgroup_rescan_lock);
+
+   memset(_info->qgroup_rescan_work, 0,
+  sizeof(fs_info->qgroup_rescan_work));
+
+   btrfs_init_work(_info->qgroup_rescan_work,
+   btrfs_qgroup_rescan_helper,
+   btrfs_qgroup_rescan_worker, NULL, NULL);
+
+   btrfs_queue_work(fs_info->qgroup_rescan_workers,
+_info->qgroup_rescan_work);
+}
+
 /*
  * called from commit_transaction. Writes all changed qgroups to disk.
  */
@@ -2123,8 +2164,7 @@ int btrfs_run_qgroups(struct btrfs_trans_handle *trans,
ret = qgroup_rescan_init(fs_info, 0, 1);
if (!ret) {
qgroup_rescan_zero_tracking(fs_info);
-   btrfs_queue_work(fs_info->qgroup_rescan_workers,
-_info->qgroup_rescan_work);
+   queue_rescan_worker(fs_info);
}
ret = 0;
}
@@ -2607,6 +2647,10 @@ static void btrfs_qgroup_rescan_worker(struct btrfs_work 
*work)
if (!path)
goto out;
 
+   mutex_lock(_info->qgroup_rescan_lock);
+   

[PATCH 2/3] btrfs-progs: build: autoconf 2.63 compatibility

[jeffm]

From: Jeff Mahoney 

Commit 2e1932e6a38 (btrfs-progs: build: simplify version tracking)
started m4_chomp to strip the newlines from the version file.  m4_chomp
was introduced in autoconf 2.64 but SLE11 ships with autoconf 2.63.
For purposes of just stripping the newline, m4_flatten is sufficient.

Signed-off-by: Jeff Mahoney 
---
 configure.ac | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/configure.ac b/configure.ac
index 2dea1c6..9e5603b 100644
--- a/configure.ac
+++ b/configure.ac
@@ -1,5 +1,5 @@
 AC_INIT([btrfs-progs],
-   m4_chomp(m4_include([VERSION])),
+   m4_flatten(m4_include([VERSION])),
[linux-btrfs@vger.kernel.org],,
[http://btrfs.wiki.kernel.org])
 
-- 
1.7.12.4

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 1/3] btrfs-progs: convert: fix support for e2fsprogs < 1.42

[jeffm]

From: Jeff Mahoney 

Commit 324d4c1857a (btrfs-progs: convert: Add larger device support)
introduced new dependencies on the 64-bit API provided by e2fsprogs.
That API was introduced in v1.42 (along with bigalloc).

This patch maps the following to their equivalents in e2fsprogs < 1.42.
- ext2fs_get_block_bitmap_range2
- ext2fs_inode_data_blocks2
- ext2fs_read_ext_attr2

Since we need to detect and define EXT2_FLAG_64BITS for compatibilty
anyway, it makes sense to use that to detect the older e2fsprogs instead
of defining a new flag ourselves.

Signed-off-by: Jeff Mahoney 
---
 configure.ac  |  6 +-
 convert/source-ext2.h | 12 +++-
 2 files changed, 12 insertions(+), 6 deletions(-)

diff --git a/configure.ac b/configure.ac
index af13a95..2dea1c6 100644
--- a/configure.ac
+++ b/configure.ac
@@ -140,11 +140,7 @@ BTRFSCONVERT_EXT2=0
 BTRFSCONVERT_REISERFS=0
 if test "x$enable_convert" = xyes; then
if test "x$with_convert" = "xauto" || echo "$with_convert" | grep -q 
"ext2"; then
-   PKG_CHECK_MODULES(EXT2FS, [ext2fs >= 1.42],,
-   [PKG_CHECK_MODULES(EXT2FS, [ext2fs],
-   [AC_DEFINE([HAVE_OLD_E2FSPROGS], [1],
- [E2fsprogs does not support 
BIGALLOC])]
-   )])
+   PKG_CHECK_MODULES(EXT2FS, [ext2fs])
PKG_CHECK_MODULES(COM_ERR, [com_err])
convertfs="${convertfs:+$convertfs,}ext2"
BTRFSCONVERT_EXT2=1
diff --git a/convert/source-ext2.h b/convert/source-ext2.h
index 80833b2..c321412 100644
--- a/convert/source-ext2.h
+++ b/convert/source-ext2.h
@@ -33,8 +33,18 @@
  * BIGALLOC.
  * Unlike normal RO compat flag, BIGALLOC affects how e2fsprogs check used
  * space, and btrfs-convert heavily relies on it.
+ *
+ * e2fsprogs 1.42 also introduced the 64-bit API.  Any file system
+ * that requires it will have EXT4_FEATURE_INCOMPAT_64BIT set and
+ * will fail to open with earlier releases.  We can map it to the
+ * older API without risk of corruption.
  */
-#ifdef HAVE_OLD_E2FSPROGS
+#ifndef EXT2_FLAG_64BITS
+#define EXT2_FLAG_64BITS   (0)
+#define ext2fs_get_block_bitmap_range2 ext2fs_get_block_bitmap_range
+#define ext2fs_inode_data_blocks2 ext2fs_inode_data_blocks
+#define ext2fs_read_ext_attr2 ext2fs_read_ext_attr
+#define ext2fs_blocks_count(s) ((s)->s_blocks_count)
 #define EXT2FS_CLUSTER_RATIO(fs)   (1)
 #define EXT2_CLUSTERS_PER_GROUP(s) (EXT2_BLOCKS_PER_GROUP(s))
 #define EXT2FS_B2C(fs, blk)(blk)
-- 
1.7.12.4

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 3/3 v2] btrfs-progs: build: detect whether -std=gnu90 is supported

[jeffm]

From: Jeff Mahoney 

GCC releases prior to 4.5.0 don't support -std=gnu90 so btrfs-progs won't
build at all on older distros.  We can detect whether the compiler
supports -std=gnu90 and fall back to -std=gnu89 if it doesn't.

AX_CHECK_COMPILE_FLAG is the right way to do this, but it depends on
autoconf 2.64.  AX_GCC_VERSION has been deprecated, so we'll use that
only for earlier autoconf versions so we can drop it when we drop
support for older autoconf releases.

Signed-off-by: Jeff Mahoney 
---
 Makefile|  1 -
 Makefile.inc.in |  1 +
 configure.ac|  1 +
 m4/ax_check_compile_flag.m4 | 74 +
 m4/ax_gcc_version.m4| 37 +++
 m4/btrfs_detect_cstd.m4 | 20 
 6 files changed, 133 insertions(+), 1 deletion(-)
 create mode 100644 m4/ax_check_compile_flag.m4
 create mode 100644 m4/ax_gcc_version.m4
 create mode 100644 m4/btrfs_detect_cstd.m4

diff --git a/Makefile b/Makefile
index 5ba76d2..c0eb3d6 100644
--- a/Makefile
+++ b/Makefile
@@ -63,7 +63,6 @@ ABSTOPDIR = $(shell pwd)
 TOPDIR := .
 
 # Common build flags
-CSTD = -std=gnu90
 CFLAGS = $(SUBST_CFLAGS) \
 $(CSTD) \
 -include config.h \
diff --git a/Makefile.inc.in b/Makefile.inc.in
index 52410f6..fb32461 100644
--- a/Makefile.inc.in
+++ b/Makefile.inc.in
@@ -4,6 +4,7 @@
 export
 
 CC = @CC@
+CSTD = @BTRFS_CSTD_FLAGS@
 LN_S = @LN_S@
 AR = @AR@
 RM = @RM@
diff --git a/configure.ac b/configure.ac
index 9e5603b..4f63bb6 100644
--- a/configure.ac
+++ b/configure.ac
@@ -26,6 +26,7 @@ AC_CONFIG_SRCDIR([btrfs.c])
 AC_PREFIX_DEFAULT([/usr/local])
 
 AC_PROG_CC
+BTRFS_DETECT_CSTD
 AC_CANONICAL_HOST
 AC_C_CONST
 AC_C_VOLATILE
diff --git a/m4/ax_check_compile_flag.m4 b/m4/ax_check_compile_flag.m4
new file mode 100644
index 000..dcabb92
--- /dev/null
+++ b/m4/ax_check_compile_flag.m4
@@ -0,0 +1,74 @@
+# ===
+#  https://www.gnu.org/software/autoconf-archive/ax_check_compile_flag.html
+# ===
+#
+# SYNOPSIS
+#
+#   AX_CHECK_COMPILE_FLAG(FLAG, [ACTION-SUCCESS], [ACTION-FAILURE], 
[EXTRA-FLAGS], [INPUT])
+#
+# DESCRIPTION
+#
+#   Check whether the given FLAG works with the current language's compiler
+#   or gives an error.  (Warnings, however, are ignored)
+#
+#   ACTION-SUCCESS/ACTION-FAILURE are shell commands to execute on
+#   success/failure.
+#
+#   If EXTRA-FLAGS is defined, it is added to the current language's default
+#   flags (e.g. CFLAGS) when the check is done.  The check is thus made with
+#   the flags: "CFLAGS EXTRA-FLAGS FLAG".  This can for example be used to
+#   force the compiler to issue an error when a bad flag is given.
+#
+#   INPUT gives an alternative input source to AC_COMPILE_IFELSE.
+#
+#   NOTE: Implementation based on AX_CFLAGS_GCC_OPTION. Please keep this
+#   macro in sync with AX_CHECK_{PREPROC,LINK}_FLAG.
+#
+# LICENSE
+#
+#   Copyright (c) 2008 Guido U. Draheim 
+#   Copyright (c) 2011 Maarten Bosmans 
+#
+#   This program is free software: you can redistribute it and/or modify it
+#   under the terms of the GNU General Public License as published by the
+#   Free Software Foundation, either version 3 of the License, or (at your
+#   option) any later version.
+#
+#   This program is distributed in the hope that it will be useful, but
+#   WITHOUT ANY WARRANTY; without even the implied warranty of
+#   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General
+#   Public License for more details.
+#
+#   You should have received a copy of the GNU General Public License along
+#   with this program. If not, see .
+#
+#   As a special exception, the respective Autoconf Macro's copyright owner
+#   gives unlimited permission to copy, distribute and modify the configure
+#   scripts that are the output of Autoconf when processing the Macro. You
+#   need not follow the terms of the GNU General Public License when using
+#   or distributing such scripts, even though portions of the text of the
+#   Macro appear in them. The GNU General Public License (GPL) does govern
+#   all other use of the material that constitutes the Autoconf Macro.
+#
+#   This special exception to the GPL applies to versions of the Autoconf
+#   Macro released by the Autoconf Archive. When you make and distribute a
+#   modified version of the Autoconf Macro, you may extend this special
+#   exception to the GPL to apply to your modified version as well.
+
+#serial 5
+
+AC_DEFUN([AX_CHECK_COMPILE_FLAG],
+[AC_PREREQ(2.64)dnl for _AC_LANG_PREFIX and AS_VAR_IF
+AS_VAR_PUSHDEF([CACHEVAR],[ax_cv_check_[]_AC_LANG_ABBREV[]flags_$4_$1])dnl
+AC_CACHE_CHECK([whether _AC_LANG compiler accepts $1], CACHEVAR, [
+  ax_check_save_flags=$[]_AC_LANG_PREFIX[]FLAGS
+  

[PATCH 3/3] btrfs-progs: build: use m4_flatten instead of m4_chomp

[jeffm]

From: Jeff Mahoney 

Commit 2e1932e6a38 (btrfs-progs: build: simplify version tracking)
started m4_chomp to strip the newlines from the version file.  m4_chomp
was introduced in autoconf 2.64 but SLE11 ships with autoconf 2.63.
For purposes of just stripping the newline, m4_flatten is sufficient.

Signed-off-by: Jeff Mahoney 
---
 configure.ac | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/configure.ac b/configure.ac
index 17880206..a0cebf15 100644
--- a/configure.ac
+++ b/configure.ac
@@ -1,5 +1,5 @@
 AC_INIT([btrfs-progs],
-   m4_chomp(m4_include([VERSION])),
+   m4_flatten(m4_include([VERSION])),
[linux-btrfs@vger.kernel.org],,
[http://btrfs.wiki.kernel.org])
 
-- 
2.12.3

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 2/3] btrfs-progs: build: detect whether -std=gnu90 is supported

[jeffm]

From: Jeff Mahoney 

Older versions of gcc don't support -std=gnu90 so btrfs-progs won't
build at all on older distros.  We can detect whether the compiler
supports -std=gnu90 and fall back to -std=gnu89 if it doesn't.

Signed-off-by: Jeff Mahoney 
---
 Makefile|  1 -
 Makefile.inc.in |  1 +
 configure.ac|  2 ++
 m4/ax_check_compile_flag.m4 | 74 +
 4 files changed, 77 insertions(+), 1 deletion(-)
 create mode 100644 m4/ax_check_compile_flag.m4

diff --git a/Makefile b/Makefile
index 5ba76d2e..c0eb3d68 100644
--- a/Makefile
+++ b/Makefile
@@ -63,7 +63,6 @@ ABSTOPDIR = $(shell pwd)
 TOPDIR := .
 
 # Common build flags
-CSTD = -std=gnu90
 CFLAGS = $(SUBST_CFLAGS) \
 $(CSTD) \
 -include config.h \
diff --git a/Makefile.inc.in b/Makefile.inc.in
index 52410f69..fc3dec69 100644
--- a/Makefile.inc.in
+++ b/Makefile.inc.in
@@ -4,6 +4,7 @@
 export
 
 CC = @CC@
+CSTD = @CSTD@
 LN_S = @LN_S@
 AR = @AR@
 RM = @RM@
diff --git a/configure.ac b/configure.ac
index 2dea1c64..17880206 100644
--- a/configure.ac
+++ b/configure.ac
@@ -30,6 +30,8 @@ AC_CANONICAL_HOST
 AC_C_CONST
 AC_C_VOLATILE
 AC_C_BIGENDIAN
+AX_CHECK_COMPILE_FLAG([-std=gnu90],[CSTD=-std=gnu90],[CSTD=-std=gnu89])
+AC_SUBST([CSTD])
 
 AC_SYS_LARGEFILE
 
diff --git a/m4/ax_check_compile_flag.m4 b/m4/ax_check_compile_flag.m4
new file mode 100644
index ..dcabb92a
--- /dev/null
+++ b/m4/ax_check_compile_flag.m4
@@ -0,0 +1,74 @@
+# ===
+#  https://www.gnu.org/software/autoconf-archive/ax_check_compile_flag.html
+# ===
+#
+# SYNOPSIS
+#
+#   AX_CHECK_COMPILE_FLAG(FLAG, [ACTION-SUCCESS], [ACTION-FAILURE], 
[EXTRA-FLAGS], [INPUT])
+#
+# DESCRIPTION
+#
+#   Check whether the given FLAG works with the current language's compiler
+#   or gives an error.  (Warnings, however, are ignored)
+#
+#   ACTION-SUCCESS/ACTION-FAILURE are shell commands to execute on
+#   success/failure.
+#
+#   If EXTRA-FLAGS is defined, it is added to the current language's default
+#   flags (e.g. CFLAGS) when the check is done.  The check is thus made with
+#   the flags: "CFLAGS EXTRA-FLAGS FLAG".  This can for example be used to
+#   force the compiler to issue an error when a bad flag is given.
+#
+#   INPUT gives an alternative input source to AC_COMPILE_IFELSE.
+#
+#   NOTE: Implementation based on AX_CFLAGS_GCC_OPTION. Please keep this
+#   macro in sync with AX_CHECK_{PREPROC,LINK}_FLAG.
+#
+# LICENSE
+#
+#   Copyright (c) 2008 Guido U. Draheim 
+#   Copyright (c) 2011 Maarten Bosmans 
+#
+#   This program is free software: you can redistribute it and/or modify it
+#   under the terms of the GNU General Public License as published by the
+#   Free Software Foundation, either version 3 of the License, or (at your
+#   option) any later version.
+#
+#   This program is distributed in the hope that it will be useful, but
+#   WITHOUT ANY WARRANTY; without even the implied warranty of
+#   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General
+#   Public License for more details.
+#
+#   You should have received a copy of the GNU General Public License along
+#   with this program. If not, see .
+#
+#   As a special exception, the respective Autoconf Macro's copyright owner
+#   gives unlimited permission to copy, distribute and modify the configure
+#   scripts that are the output of Autoconf when processing the Macro. You
+#   need not follow the terms of the GNU General Public License when using
+#   or distributing such scripts, even though portions of the text of the
+#   Macro appear in them. The GNU General Public License (GPL) does govern
+#   all other use of the material that constitutes the Autoconf Macro.
+#
+#   This special exception to the GPL applies to versions of the Autoconf
+#   Macro released by the Autoconf Archive. When you make and distribute a
+#   modified version of the Autoconf Macro, you may extend this special
+#   exception to the GPL to apply to your modified version as well.
+
+#serial 5
+
+AC_DEFUN([AX_CHECK_COMPILE_FLAG],
+[AC_PREREQ(2.64)dnl for _AC_LANG_PREFIX and AS_VAR_IF
+AS_VAR_PUSHDEF([CACHEVAR],[ax_cv_check_[]_AC_LANG_ABBREV[]flags_$4_$1])dnl
+AC_CACHE_CHECK([whether _AC_LANG compiler accepts $1], CACHEVAR, [
+  ax_check_save_flags=$[]_AC_LANG_PREFIX[]FLAGS
+  _AC_LANG_PREFIX[]FLAGS="$[]_AC_LANG_PREFIX[]FLAGS $4 $1"
+  AC_COMPILE_IFELSE([m4_default([$5],[AC_LANG_PROGRAM()])],
+[AS_VAR_SET(CACHEVAR,[yes])],
+[AS_VAR_SET(CACHEVAR,[no])])
+  _AC_LANG_PREFIX[]FLAGS=$ax_check_save_flags])
+AS_VAR_IF(CACHEVAR,yes,
+  [m4_default([$2], :)],
+  [m4_default([$3], :)])
+AS_VAR_POPDEF([CACHEVAR])dnl
+])dnl AX_CHECK_COMPILE_FLAGS
-- 
2.12.3

--
To unsubscribe from this list: send 

[PATCH 1/3] btrfs-progs: convert: fix support for e2fsprogs < 1.42

[jeffm]

From: Jeff Mahoney 

Commit 324d4c1857a (btrfs-progs: convert: Add larger device support)
introduced new dependencies on the 64-bit API provided by e2fsprogs.
That API was introduced in v1.42 (along with bigalloc).

This patch maps the following to their equivalents in e2fsprogs < 1.42.
- ext2fs_get_block_bitmap_range2
- ext2fs_inode_data_blocks2
- ext2fs_read_ext_attr2

Since we need to detect and define EXT2_FLAG_64BITS for compatibilty
anyway, it makes sense to use that to detect the older e2fsprogs instead
of defining a new flag ourselves.

Signed-off-by: Jeff Mahoney 
---
 configure.ac  |  6 +-
 convert/source-ext2.h | 12 +++-
 2 files changed, 12 insertions(+), 6 deletions(-)

diff --git a/configure.ac b/configure.ac
index af13a959..2dea1c64 100644
--- a/configure.ac
+++ b/configure.ac
@@ -140,11 +140,7 @@ BTRFSCONVERT_EXT2=0
 BTRFSCONVERT_REISERFS=0
 if test "x$enable_convert" = xyes; then
if test "x$with_convert" = "xauto" || echo "$with_convert" | grep -q 
"ext2"; then
-   PKG_CHECK_MODULES(EXT2FS, [ext2fs >= 1.42],,
-   [PKG_CHECK_MODULES(EXT2FS, [ext2fs],
-   [AC_DEFINE([HAVE_OLD_E2FSPROGS], [1],
- [E2fsprogs does not support 
BIGALLOC])]
-   )])
+   PKG_CHECK_MODULES(EXT2FS, [ext2fs])
PKG_CHECK_MODULES(COM_ERR, [com_err])
convertfs="${convertfs:+$convertfs,}ext2"
BTRFSCONVERT_EXT2=1
diff --git a/convert/source-ext2.h b/convert/source-ext2.h
index 80833b21..c3214125 100644
--- a/convert/source-ext2.h
+++ b/convert/source-ext2.h
@@ -33,8 +33,18 @@
  * BIGALLOC.
  * Unlike normal RO compat flag, BIGALLOC affects how e2fsprogs check used
  * space, and btrfs-convert heavily relies on it.
+ *
+ * e2fsprogs 1.42 also introduced the 64-bit API.  Any file system
+ * that requires it will have EXT4_FEATURE_INCOMPAT_64BIT set and
+ * will fail to open with earlier releases.  We can map it to the
+ * older API without risk of corruption.
  */
-#ifdef HAVE_OLD_E2FSPROGS
+#ifndef EXT2_FLAG_64BITS
+#define EXT2_FLAG_64BITS   (0)
+#define ext2fs_get_block_bitmap_range2 ext2fs_get_block_bitmap_range
+#define ext2fs_inode_data_blocks2 ext2fs_inode_data_blocks
+#define ext2fs_read_ext_attr2 ext2fs_read_ext_attr
+#define ext2fs_blocks_count(s) ((s)->s_blocks_count)
 #define EXT2FS_CLUSTER_RATIO(fs)   (1)
 #define EXT2_CLUSTERS_PER_GROUP(s) (EXT2_BLOCKS_PER_GROUP(s))
 #define EXT2FS_B2C(fs, blk)(blk)
-- 
2.12.3

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 0/3] btrfs-progs: build on SLE11

[jeffm]

From: Jeff Mahoney 

This patch set allows btrfs-progs to build without further patching
on a stock SLE11 SP4 install.  The major issues were:
- new dependencies on e2fsprogs 1.42
- use of -std=gnu90, which gcc 4.3.4 doesn't support
- use of m4_chomp, introduced in autoconf 2.64.

This patch set is also posted as a GitHub pull request:
https://github.com/kdave/btrfs-progs/pull/125

-Jeff

---

Jeff Mahoney (3):
  btrfs-progs: convert: fix support for e2fsprogs < 1.42
  btrfs-progs: build: detect whether -std=gnu90 is supported
  btrfs-progs: build: use m4_flatten instead of m4_chomp

 Makefile|  1 -
 Makefile.inc.in |  1 +
 configure.ac| 10 +++---
 convert/source-ext2.h   | 12 +++-
 m4/ax_check_compile_flag.m4 | 74 +
 5 files changed, 90 insertions(+), 8 deletions(-)
 create mode 100644 m4/ax_check_compile_flag.m4

-- 
2.12.3

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 2/3] btrfs: qgroups, remove unnecessary memset before btrfs_init_work

[jeffm]

From: Jeff Mahoney 

btrfs_init_work clears the work struct except for ->wq, so the memset
before calling btrfs_init_work in qgroup_rescan_init is unnecessary.

We'll also initialize ->wq in btrfs_init_work so that it's obvious.

Signed-off-by: Jeff Mahoney 
---
 fs/btrfs/async-thread.c | 1 +
 fs/btrfs/qgroup.c   | 2 --
 2 files changed, 1 insertion(+), 2 deletions(-)

diff --git a/fs/btrfs/async-thread.c b/fs/btrfs/async-thread.c
index d5540749f0e5..c614fb7b9b9d 100644
--- a/fs/btrfs/async-thread.c
+++ b/fs/btrfs/async-thread.c
@@ -354,6 +354,7 @@ void btrfs_init_work(struct btrfs_work *work, 
btrfs_work_func_t uniq_func,
INIT_WORK(>normal_work, uniq_func);
INIT_LIST_HEAD(>ordered_list);
work->flags = 0;
+   work->wq = NULL;
 }
 
 static inline void __btrfs_queue_work(struct __btrfs_workqueue *wq,
diff --git a/fs/btrfs/qgroup.c b/fs/btrfs/qgroup.c
index be491b6c020a..8de423a0c7e3 100644
--- a/fs/btrfs/qgroup.c
+++ b/fs/btrfs/qgroup.c
@@ -2740,8 +2740,6 @@ qgroup_rescan_init(struct btrfs_fs_info *fs_info, u64 
progress_objectid,
spin_unlock(_info->qgroup_lock);
mutex_unlock(_info->qgroup_rescan_lock);
 
-   memset(_info->qgroup_rescan_work, 0,
-  sizeof(fs_info->qgroup_rescan_work));
btrfs_init_work(_info->qgroup_rescan_work,
btrfs_qgroup_rescan_helper,
btrfs_qgroup_rescan_worker, NULL, NULL);
-- 
2.12.3

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 1/3] btrfs: qgroups, fix rescan worker running races

[jeffm]

From: Jeff Mahoney 

Commit d2c609b834d6 (Btrfs: fix qgroup rescan worker initialization)
fixed the issue with BTRFS_IOC_QUOTA_RESCAN_WAIT being racy, but
ended up reintroducing the hang-on-unmount bug that the commit it
intended to fix addressed.

The race this time is between qgroup_rescan_init setting
->qgroup_rescan_running = true and the worker starting.  There are
many scenarios where we initialize the worker and never start it.  The
completion btrfs_ioctl_quota_rescan_wait waits for will never come.
This can happen even without involving error handling, since mounting
the file system read-only returns between initializing the worker and
queueing it.

The right place to do it is when we're queuing the worker.  The flag
really just means that btrfs_ioctl_quota_rescan_wait should wait for
a completion.

This patch introduces a new helper, queue_rescan_worker, that handles
the ->qgroup_rescan_running flag, including any races with umount.

While we're at it, ->qgroup_rescan_running is protected only by the
->qgroup_rescan_mutex.  btrfs_ioctl_quota_rescan_wait doesn't need
to take the spinlock too.

Fixes: d2c609b834d6 (Btrfs: fix qgroup rescan worker initialization)
Signed-off-by: Jeff Mahoney 
---
 fs/btrfs/ctree.h  |  1 +
 fs/btrfs/qgroup.c | 40 
 2 files changed, 29 insertions(+), 12 deletions(-)

diff --git a/fs/btrfs/ctree.h b/fs/btrfs/ctree.h
index da308774b8a4..dbba615f4d0f 100644
--- a/fs/btrfs/ctree.h
+++ b/fs/btrfs/ctree.h
@@ -1045,6 +1045,7 @@ struct btrfs_fs_info {
struct btrfs_workqueue *qgroup_rescan_workers;
struct completion qgroup_rescan_completion;
struct btrfs_work qgroup_rescan_work;
+   /* qgroup rescan worker is running or queued to run */
bool qgroup_rescan_running; /* protected by qgroup_rescan_lock */
 
/* filesystem state */
diff --git a/fs/btrfs/qgroup.c b/fs/btrfs/qgroup.c
index aa259d6986e1..be491b6c020a 100644
--- a/fs/btrfs/qgroup.c
+++ b/fs/btrfs/qgroup.c
@@ -2072,6 +2072,30 @@ int btrfs_qgroup_account_extents(struct 
btrfs_trans_handle *trans,
return ret;
 }
 
+static void queue_rescan_worker(struct btrfs_fs_info *fs_info)
+{
+   mutex_lock(_info->qgroup_rescan_lock);
+   if (btrfs_fs_closing(fs_info)) {
+   mutex_unlock(_info->qgroup_rescan_lock);
+   return;
+   }
+   if (WARN_ON(fs_info->qgroup_rescan_running)) {
+   btrfs_warn(fs_info, "rescan worker already queued");
+   mutex_unlock(_info->qgroup_rescan_lock);
+   return;
+   }
+
+   /*
+* Being queued is enough for btrfs_qgroup_wait_for_completion
+* to need to wait.
+*/
+   fs_info->qgroup_rescan_running = true;
+   mutex_unlock(_info->qgroup_rescan_lock);
+
+   btrfs_queue_work(fs_info->qgroup_rescan_workers,
+_info->qgroup_rescan_work);
+}
+
 /*
  * called from commit_transaction. Writes all changed qgroups to disk.
  */
@@ -2123,8 +2147,7 @@ int btrfs_run_qgroups(struct btrfs_trans_handle *trans,
ret = qgroup_rescan_init(fs_info, 0, 1);
if (!ret) {
qgroup_rescan_zero_tracking(fs_info);
-   btrfs_queue_work(fs_info->qgroup_rescan_workers,
-_info->qgroup_rescan_work);
+   queue_rescan_worker(fs_info);
}
ret = 0;
}
@@ -2713,7 +2736,6 @@ qgroup_rescan_init(struct btrfs_fs_info *fs_info, u64 
progress_objectid,
sizeof(fs_info->qgroup_rescan_progress));
fs_info->qgroup_rescan_progress.objectid = progress_objectid;
init_completion(_info->qgroup_rescan_completion);
-   fs_info->qgroup_rescan_running = true;
 
spin_unlock(_info->qgroup_lock);
mutex_unlock(_info->qgroup_rescan_lock);
@@ -2785,9 +2807,7 @@ btrfs_qgroup_rescan(struct btrfs_fs_info *fs_info)
 
qgroup_rescan_zero_tracking(fs_info);
 
-   btrfs_queue_work(fs_info->qgroup_rescan_workers,
-_info->qgroup_rescan_work);
-
+   queue_rescan_worker(fs_info);
return 0;
 }
 
@@ -2798,9 +2818,7 @@ int btrfs_qgroup_wait_for_completion(struct btrfs_fs_info 
*fs_info,
int ret = 0;
 
mutex_lock(_info->qgroup_rescan_lock);
-   spin_lock(_info->qgroup_lock);
running = fs_info->qgroup_rescan_running;
-   spin_unlock(_info->qgroup_lock);
mutex_unlock(_info->qgroup_rescan_lock);
 
if (!running)
@@ -2819,12 +2837,10 @@ int btrfs_qgroup_wait_for_completion(struct 
btrfs_fs_info *fs_info,
  * this is only called from open_ctree where we're still single threaded, thus
  * locking is omitted here.
  */
-void
-btrfs_qgroup_rescan_resume(struct btrfs_fs_info *fs_info)
+void btrfs_qgroup_rescan_resume(struct btrfs_fs_info *fs_info)
 {
if (fs_info->qgroup_flags & 

[PATCH 3/3] btrfs: qgroup, don't try to insert status item after ENOMEM in rescan worker

[jeffm]

From: Jeff Mahoney 

If we fail to allocate memory for a path, don't bother trying to
insert the qgroup status item.  We haven't done anything yet and it'll
fail also.  Just print an error and be done with it.

Signed-off-by: Jeff Mahoney 
---
 fs/btrfs/qgroup.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/fs/btrfs/qgroup.c b/fs/btrfs/qgroup.c
index 8de423a0c7e3..4c0978bce5b9 100644
--- a/fs/btrfs/qgroup.c
+++ b/fs/btrfs/qgroup.c
@@ -2648,7 +2648,6 @@ static void btrfs_qgroup_rescan_worker(struct btrfs_work 
*work)
btrfs_end_transaction(trans);
}
 
-out:
btrfs_free_path(path);
 
mutex_lock(_info->qgroup_rescan_lock);
@@ -2688,6 +2687,7 @@ static void btrfs_qgroup_rescan_worker(struct btrfs_work 
*work)
btrfs_info(fs_info, "qgroup scan completed%s",
err > 0 ? " (inconsistency flag cleared)" : "");
} else {
+out:
btrfs_err(fs_info, "qgroup scan failed with %d", err);
}
 
-- 
2.12.3

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 2/2] btrfs: defer adding raid type kobject until after chunk relocation

[jeffm]

From: Jeff Mahoney 

Any time the first block group of a new type is created, we add a new
kobject to sysfs to hold the attributes for that type.  Kobject-internal
allocations always use GFP_KERNEL, making them prone to fs-reclaim races.
While it appears as if this can occur any time a block group is created,
the only times the first block group of a new type can be created in
memory is at mount and when we create the first new block group during
raid conversion.

This patch adds a new list to track pending kobject additions and then
handles them after we do chunk relocation.  Between relocating the
target chunk (or forcing allocation of a new chunk in the case of data)
and removing the old chunk, we're in a safe place for fs-reclaim to
occur.  We're holding the volume mutex, which is already held across
page faults, and the delete_unused_bgs_mutex, which will only stall
the cleaner thread.

Signed-off-by: Jeff Mahoney 
---
 fs/btrfs/ctree.h   |  6 -
 fs/btrfs/disk-io.c |  2 ++
 fs/btrfs/extent-tree.c | 59 +++---
 fs/btrfs/sysfs.c   |  2 +-
 fs/btrfs/volumes.c | 12 ++
 5 files changed, 61 insertions(+), 20 deletions(-)

diff --git a/fs/btrfs/ctree.h b/fs/btrfs/ctree.h
index ffbb05aa66fa..75dbdf1bbead 100644
--- a/fs/btrfs/ctree.h
+++ b/fs/btrfs/ctree.h
@@ -381,8 +381,9 @@ struct btrfs_dev_replace {
 
 /* For raid type sysfs entries */
 struct raid_kobject {
-   int raid_type;
+   u64 flags;
struct kobject kobj;
+   struct list_head list;
 };
 
 struct btrfs_space_info {
@@ -938,6 +939,8 @@ struct btrfs_fs_info {
int thread_pool_size;
 
struct kobject *space_info_kobj;
+   struct list_head pending_raid_kobjs;
+   spinlock_t pending_raid_kobjs_lock; /* uncontended */
 
u64 total_pinned;
 
@@ -2688,6 +2691,7 @@ int btrfs_can_relocate(struct btrfs_fs_info *fs_info, u64 
bytenr);
 int btrfs_make_block_group(struct btrfs_trans_handle *trans,
   struct btrfs_fs_info *fs_info, u64 bytes_used,
   u64 type, u64 chunk_offset, u64 size);
+void btrfs_add_raid_kobjects(struct btrfs_fs_info *fs_info);
 struct btrfs_trans_handle *btrfs_start_trans_remove_block_group(
struct btrfs_fs_info *fs_info,
const u64 chunk_offset);
diff --git a/fs/btrfs/disk-io.c b/fs/btrfs/disk-io.c
index eb6bb3169a9e..d5e1c2ff71ff 100644
--- a/fs/btrfs/disk-io.c
+++ b/fs/btrfs/disk-io.c
@@ -2447,6 +2447,8 @@ int open_ctree(struct super_block *sb,
INIT_LIST_HEAD(_info->delayed_iputs);
INIT_LIST_HEAD(_info->delalloc_roots);
INIT_LIST_HEAD(_info->caching_block_groups);
+   INIT_LIST_HEAD(_info->pending_raid_kobjs);
+   spin_lock_init(_info->pending_raid_kobjs_lock);
spin_lock_init(_info->delalloc_root_lock);
spin_lock_init(_info->trans_lock);
spin_lock_init(_info->fs_roots_radix_lock);
diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c
index 0e9a21230217..bb5368faa937 100644
--- a/fs/btrfs/extent-tree.c
+++ b/fs/btrfs/extent-tree.c
@@ -9908,9 +9908,39 @@ int btrfs_free_block_groups(struct btrfs_fs_info *info)
return 0;
 }
 
+/* link_block_group will queue up kobjects to add when we're reclaim-safe */
+void btrfs_add_raid_kobjects(struct btrfs_fs_info *fs_info)
+{
+   struct btrfs_space_info *space_info;
+   struct raid_kobject *rkobj;
+   LIST_HEAD(list);
+   int index;
+   int ret = 0;
+
+   spin_lock(_info->pending_raid_kobjs_lock);
+   list_splice_init(_info->pending_raid_kobjs, );
+   spin_unlock(_info->pending_raid_kobjs_lock);
+
+   list_for_each_entry(rkobj, , list) {
+   space_info = __find_space_info(fs_info, rkobj->flags);
+   index = __get_raid_index(rkobj->flags);
+
+   ret = kobject_add(>kobj, _info->kobj,
+ "%s", get_raid_name(index));
+   if (ret) {
+   kobject_put(>kobj);
+   break;
+   }
+   }
+   if (ret)
+   btrfs_warn(fs_info,
+  "failed to add kobject for block cache, ignoring");
+}
+
 static void link_block_group(struct btrfs_block_group_cache *cache)
 {
struct btrfs_space_info *space_info = cache->space_info;
+   struct btrfs_fs_info *fs_info = cache->fs_info;
int index = get_block_group_index(cache);
bool first = false;
 
@@ -9921,27 +9951,19 @@ static void link_block_group(struct 
btrfs_block_group_cache *cache)
up_write(_info->groups_sem);
 
if (first) {
-   struct raid_kobject *rkobj;
-   int ret;
-
-   rkobj = kzalloc(sizeof(*rkobj), GFP_NOFS);
-   if (!rkobj)
-   goto out_err;
-   rkobj->raid_type = index;
-   kobject_init(>kobj, _raid_ktype);

[PATCH 1/2] btrfs: remove dead create_space_info calls

[jeffm]

From: Jeff Mahoney 

Since commit 2be12ef79 (btrfs: Separate space_info create/update), we've
separated out the creation and updating of the space info structures.
That commit was a straightforward refactoring of the two parts of
update_space_info, but we can go a step further.  Since commits
c59021f84 (Btrfs: fix OOPS of empty filesystem after balance) and
b742bb82f (Btrfs: Link block groups of different raid types), we know
that the space_info structures will be created at mount and there will
only ever be, at most, three of them.

This patch cleans out the create_space_info calls after __find_space_info
returns NULL since __find_space_info *can't* return NULL.

The initial cause for reviewing this was the kobject_add calls from
create_space_info occuring in sites where fs-reclaim wasn't allowed.  Now
we are certain they occur only early in the mount process and are safe.

Signed-off-by: Jeff Mahoney 
---
 fs/btrfs/ctree.h   |  6 +++---
 fs/btrfs/extent-tree.c | 16 ++--
 2 files changed, 5 insertions(+), 17 deletions(-)

diff --git a/fs/btrfs/ctree.h b/fs/btrfs/ctree.h
index da308774b8a4..ffbb05aa66fa 100644
--- a/fs/btrfs/ctree.h
+++ b/fs/btrfs/ctree.h
@@ -952,9 +952,9 @@ struct btrfs_fs_info {
struct btrfs_fs_devices *fs_devices;
 
/*
-* the space_info list is almost entirely read only.  It only changes
-* when we add a new raid type to the FS, and that happens
-* very rarely.  RCU is used to protect it.
+* The space_info list is effectively read only after initial
+* setup.  It is populated at mount time and cleaned up after
+* all block groups are removed.  RCU is used to protect it.
 */
struct list_head space_info;
 
diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c
index 2d5e963fae88..0e9a21230217 100644
--- a/fs/btrfs/extent-tree.c
+++ b/fs/btrfs/extent-tree.c
@@ -4603,11 +4603,7 @@ static int do_chunk_alloc(struct btrfs_trans_handle 
*trans,
return -ENOSPC;
 
space_info = __find_space_info(fs_info, flags);
-   if (!space_info) {
-   ret = create_space_info(fs_info, flags, _info);
-   if (ret)
-   return ret;
-   }
+   ASSERT(space_info);
 
 again:
spin_lock(_info->lock);
@@ -10255,15 +10251,7 @@ int btrfs_make_block_group(struct btrfs_trans_handle 
*trans,
 * with its ->space_info set.
 */
cache->space_info = __find_space_info(fs_info, cache->flags);
-   if (!cache->space_info) {
-   ret = create_space_info(fs_info, cache->flags,
-  >space_info);
-   if (ret) {
-   btrfs_remove_free_space_cache(cache);
-   btrfs_put_block_group(cache);
-   return ret;
-   }
-   }
+   ASSERT(cache->space_info);
 
ret = btrfs_add_block_group_cache(fs_info, cache);
if (ret) {
-- 
2.15.1

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH] btrfs: fix lockdep splat in btrfs_alloc_subvolume_writers

[jeffm]

From: Jeff Mahoney 

While running btrfs/011, I hit the following lockdep splat.

This is the important bit:
   pcpu_alloc+0x1ac/0x5e0
   __percpu_counter_init+0x4e/0xb0
   btrfs_init_fs_root+0x99/0x1c0 [btrfs]
   btrfs_get_fs_root.part.54+0x5b/0x150 [btrfs]
   resolve_indirect_refs+0x130/0x830 [btrfs]
   find_parent_nodes+0x69e/0xff0 [btrfs]
   btrfs_find_all_roots_safe+0xa0/0x110 [btrfs]
   btrfs_find_all_roots+0x50/0x70 [btrfs]
   btrfs_qgroup_prepare_account_extents+0x53/0x90 [btrfs]
   btrfs_commit_transaction+0x3ce/0x9b0 [btrfs]

The percpu_counter_init call in btrfs_alloc_subvolume_writers
uses GFP_KERNEL, which we can't do during transaction commit.

This switches it to GFP_NOFS.


WARNING: possible irq lock inversion dependency detected
4.12.14-kvmsmall #8 Tainted: GW

kswapd0/50 just changed the state of lock:
 (_node->mutex){+.+.-.}, at: [] 
__btrfs_release_delayed_node+0x3a/0x1f0 [btrfs]
but this lock took another, RECLAIM_FS-unsafe lock in the past:
 (pcpu_alloc_mutex){+.+.+.}

and interrupts could create inverse lock ordering between them.

other info that might help us debug this:
Chain exists of:
  _node->mutex --> >groups_sem --> pcpu_alloc_mutex

 Possible interrupt unsafe locking scenario:

   CPU0CPU1
   
  lock(pcpu_alloc_mutex);
   local_irq_disable();
   lock(_node->mutex);
   lock(>groups_sem);
  
lock(_node->mutex);

 *** DEADLOCK ***

2 locks held by kswapd0/50:
 #0:  (shrinker_rwsem){..}, at: [] shrink_slab+0x7f/0x5b0
 #1:  (>s_umount_key#30){+.}, at: [] 
trylock_super+0x16/0x50

the shortest dependencies between 2nd lock and 1st lock:
   -> (pcpu_alloc_mutex){+.+.+.} ops: 4904 {
  HARDIRQ-ON-W at:
  __mutex_lock+0x4e/0x8c0
  pcpu_alloc+0x1ac/0x5e0
  alloc_kmem_cache_cpus.isra.70+0x25/0xa0
  __do_tune_cpucache+0x2c/0x220
  do_tune_cpucache+0x26/0xc0
  enable_cpucache+0x6d/0xf0
  kmem_cache_init_late+0x42/0x75
  start_kernel+0x343/0x4cb
  x86_64_start_kernel+0x127/0x134
  secondary_startup_64+0xa5/0xb0
  SOFTIRQ-ON-W at:
  __mutex_lock+0x4e/0x8c0
  pcpu_alloc+0x1ac/0x5e0
  alloc_kmem_cache_cpus.isra.70+0x25/0xa0
  __do_tune_cpucache+0x2c/0x220
  do_tune_cpucache+0x26/0xc0
  enable_cpucache+0x6d/0xf0
  kmem_cache_init_late+0x42/0x75
  start_kernel+0x343/0x4cb
  x86_64_start_kernel+0x127/0x134
  secondary_startup_64+0xa5/0xb0
  RECLAIM_FS-ON-W at:
 __kmalloc+0x47/0x310
 pcpu_extend_area_map+0x2b/0xc0
 pcpu_alloc+0x3ec/0x5e0
 alloc_kmem_cache_cpus.isra.70+0x25/0xa0
 __do_tune_cpucache+0x2c/0x220
 do_tune_cpucache+0x26/0xc0
 enable_cpucache+0x6d/0xf0
 __kmem_cache_create+0x1bf/0x390
 create_cache+0xba/0x1b0
 kmem_cache_create+0x1f8/0x2b0
 ksm_init+0x6f/0x19d
 do_one_initcall+0x50/0x1b0
 kernel_init_freeable+0x201/0x289
 kernel_init+0xa/0x100
 ret_from_fork+0x3a/0x50
  INITIAL USE at:
 __mutex_lock+0x4e/0x8c0
 pcpu_alloc+0x1ac/0x5e0
 alloc_kmem_cache_cpus.isra.70+0x25/0xa0
 setup_cpu_cache+0x2f/0x1f0
 __kmem_cache_create+0x1bf/0x390
 create_boot_cache+0x8b/0xb1
 kmem_cache_init+0xa1/0x19e
 start_kernel+0x270/0x4cb
 x86_64_start_kernel+0x127/0x134
 secondary_startup_64+0xa5/0xb0
}
... key  at: [] pcpu_alloc_mutex+0x70/0xa0
... acquired at:
   pcpu_alloc+0x1ac/0x5e0
   __percpu_counter_init+0x4e/0xb0
   btrfs_init_fs_root+0x99/0x1c0 [btrfs]
   btrfs_get_fs_root.part.54+0x5b/0x150 [btrfs]
   resolve_indirect_refs+0x130/0x830 [btrfs]
   find_parent_nodes+0x69e/0xff0 [btrfs]
   btrfs_find_all_roots_safe+0xa0/0x110 [btrfs]
   btrfs_find_all_roots+0x50/0x70 [btrfs]
   btrfs_qgroup_prepare_account_extents+0x53/0x90 [btrfs]
   

[PATCH 06/20] btrfs-progs: qgroups: add pathname to show output

[jeffm]

From: Jeff Mahoney 

The btrfs qgroup show command currently only exports qgroup IDs,
forcing the user to resolve which subvolume each corresponds to.

This patch adds pathname resolution to qgroup show so that when
the -P option is used, the last column contains the pathname of
the root of the subvolume it describes.  In the case of nested
qgroups, it will show the number of member qgroups or the paths
of the members if the -v option is used.

Pathname can also be used as a sort parameter.

Signed-off-by: Jeff Mahoney 
---
 Documentation/btrfs-qgroup.asciidoc |   4 +
 cmds-qgroup.c   |  18 -
 kerncompat.h|   1 +
 qgroup.c| 152 
 qgroup.h|   4 +-
 utils.c |  23 --
 utils.h |   2 +
 7 files changed, 178 insertions(+), 26 deletions(-)

diff --git a/Documentation/btrfs-qgroup.asciidoc 
b/Documentation/btrfs-qgroup.asciidoc
index 3108457c..360b3269 100644
--- a/Documentation/btrfs-qgroup.asciidoc
+++ b/Documentation/btrfs-qgroup.asciidoc
@@ -97,10 +97,14 @@ print child qgroup id.
 print limit of referenced size of qgroup.
 -e
 print limit of exclusive size of qgroup.
+-P
+print pathname to the root of the subvolume managed by qgroup.  For nested 
qgroups, the number of members will be printed unless -v is specified.
 -F
 list all qgroups which impact the given path(include ancestral qgroups)
 -f
 list all qgroups which impact the given path(exclude ancestral qgroups)
+-v
+Be more verbose.  Print pathnames of member qgroups when nested.
 --raw
 raw numbers in bytes, without the 'B' suffix.
 --human-readable
diff --git a/cmds-qgroup.c b/cmds-qgroup.c
index 48686436..d704aeaf 100644
--- a/cmds-qgroup.c
+++ b/cmds-qgroup.c
@@ -280,8 +280,11 @@ static const char * const cmd_qgroup_show_usage[] = {
"   (including ancestral qgroups)",
"-f list all qgroups which impact the given path",
"   (excluding ancestral qgroups)",
+   "-P print first-level qgroups using pathname",
+   "   - nested qgroups will be reported as a count",
+   "-v verbose, prints pathnames for all nested qgroups",
HELPINFO_UNITS_LONG,
-   "--sort=qgroupid,rfer,excl,max_rfer,max_excl",
+   "--sort=qgroupid,rfer,excl,max_rfer,max_excl,pathname",
"   list qgroups sorted by specified items",
"   you can use '+' or '-' in front of each item.",
"   (+:ascending, -:descending, ascending default)",
@@ -299,6 +302,7 @@ static int cmd_qgroup_show(int argc, char **argv)
int filter_flag = 0;
unsigned unit_mode;
int sync = 0;
+   bool verbose = false;
 
struct btrfs_qgroup_comparer_set *comparer_set;
struct btrfs_qgroup_filter_set *filter_set;
@@ -316,10 +320,11 @@ static int cmd_qgroup_show(int argc, char **argv)
static const struct option long_options[] = {
{"sort", required_argument, NULL, GETOPT_VAL_SORT},
{"sync", no_argument, NULL, GETOPT_VAL_SYNC},
+   {"verbose", no_argument, NULL, 'v'},
{ NULL, 0, NULL, 0 }
};
 
-   c = getopt_long(argc, argv, "pcreFf", long_options, NULL);
+   c = getopt_long(argc, argv, "pPcreFfv", long_options, NULL);
if (c < 0)
break;
switch (c) {
@@ -327,6 +332,10 @@ static int cmd_qgroup_show(int argc, char **argv)
btrfs_qgroup_setup_print_column(
BTRFS_QGROUP_PARENT);
break;
+   case 'P':
+   btrfs_qgroup_setup_print_column(
+   BTRFS_QGROUP_PATHNAME);
+   break;
case 'c':
btrfs_qgroup_setup_print_column(
BTRFS_QGROUP_CHILD);
@@ -354,6 +363,9 @@ static int cmd_qgroup_show(int argc, char **argv)
case GETOPT_VAL_SYNC:
sync = 1;
break;
+   case 'v':
+   verbose = true;
+   break;
default:
usage(cmd_qgroup_show_usage);
}
@@ -394,7 +406,7 @@ static int cmd_qgroup_show(int argc, char **argv)
BTRFS_QGROUP_FILTER_PARENT,
qgroupid);
}
-   ret = btrfs_show_qgroups(fd, filter_set, comparer_set);
+   ret = btrfs_show_qgroups(fd, filter_set, comparer_set, verbose);
close_file_or_dir(fd, dirstream);
free(filter_set);
free(comparer_set);
diff --git 

[PATCH 02/20] btrfs-progs: qgroups: fix misleading index check

[jeffm]

From: Jeff Mahoney 

In print_single_qgroup_table we check the loop index against
BTRFS_QGROUP_CHILD, but what we really mean is "last column."  Since
we have an enum value to indicate the last value, use that instead
of assuming that BTRFS_QGROUP_CHILD is always last.

Reviewed-by: Qu Wenruo 
Reviewed-by: Nikolay Borisov 
Signed-off-by: Jeff Mahoney 
---
 qgroup.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/qgroup.c b/qgroup.c
index 11659e83..67bc0738 100644
--- a/qgroup.c
+++ b/qgroup.c
@@ -267,7 +267,7 @@ static void print_single_qgroup_table(struct btrfs_qgroup 
*qgroup)
continue;
print_qgroup_column(qgroup, i);
 
-   if (i != BTRFS_QGROUP_CHILD)
+   if (i != BTRFS_QGROUP_ALL - 1)
printf(" ");
}
printf("\n");
-- 
2.12.3

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 05/20] btrfs-progs: btrfs-list: add btrfs_cleanup_root_info

[jeffm]

From: Jeff Mahoney 

Currently we can pass back root_info structures to callers but
have to free the strings manually.  This adds a helper to do it
and uses it in cmd_subvol_show.

Signed-off-by: Jeff Mahoney 
---
 btrfs-list.c | 18 +++---
 btrfs-list.h |  1 +
 cmds-subvolume.c |  5 +
 3 files changed, 17 insertions(+), 7 deletions(-)

diff --git a/btrfs-list.c b/btrfs-list.c
index 90c98be1..2fe31e9c 100644
--- a/btrfs-list.c
+++ b/btrfs-list.c
@@ -533,15 +533,27 @@ static int add_root_backref(struct root_lookup 
*root_lookup, u64 root_id,
name_len, 0, 0, 0, NULL, NULL, NULL);
 }
 
+static void __btrfs_free_root_info_strings(struct root_info *ri)
+{
+   free(ri->name);
+   free(ri->path);
+   free(ri->full_path);
+}
+
+void btrfs_cleanup_root_info(struct root_info *ri)
+{
+   __btrfs_free_root_info_strings(ri);
+   ri->name = NULL;
+   ri->path = NULL;
+   ri->full_path = NULL;
+}
 
 static void free_root_info(struct rb_node *node)
 {
struct root_info *ri;
 
ri = to_root_info(node);
-   free(ri->name);
-   free(ri->path);
-   free(ri->full_path);
+   __btrfs_free_root_info_strings(ri);
free(ri);
 }
 
diff --git a/btrfs-list.h b/btrfs-list.h
index 6e5fc778..9d0478b8 100644
--- a/btrfs-list.h
+++ b/btrfs-list.h
@@ -176,5 +176,6 @@ char *btrfs_list_path_for_root(int fd, u64 root);
 int btrfs_list_get_path_rootid(int fd, u64 *treeid);
 int btrfs_get_subvol(int fd, struct root_info *the_ri);
 int btrfs_get_toplevel_subvol(int fd, struct root_info *the_ri);
+void btrfs_cleanup_root_info(struct root_info *ri);
 
 #endif
diff --git a/cmds-subvolume.c b/cmds-subvolume.c
index 8a473f7a..769d2a76 100644
--- a/cmds-subvolume.c
+++ b/cmds-subvolume.c
@@ -1113,10 +1113,7 @@ static int cmd_subvol_show(int argc, char **argv)
1, raw_prefix);
 
 out:
-   /* clean up */
-   free(get_ri.path);
-   free(get_ri.name);
-   free(get_ri.full_path);
+   btrfs_cleanup_root_info(_ri);
free(filter_set);
 
close_file_or_dir(fd, dirstream1);
-- 
2.12.3

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 12/20] btrfs-progs: filesystem balance: split out special handling

[jeffm]

From: Jeff Mahoney 

In preparation to use cmd_struct as the command entry point, we need
to split out the 'filesystem balance' handling to not call cmd_balance
directly.  The reason is that the flags that indicate a command is
hidden are a part of cmd_struct and so we can use a cmd_struct as a
direct alias in another command group and ALSO have it be hidden
without declaring another cmd_struct.

This change has no immediate impact since cmd_balance will still
use its usage information directly from cmds-balance.c.  It will
take effect once we start passing cmd_structs around for usage
information.

Signed-off-by: Jeff Mahoney 
---
 cmds-filesystem.c | 17 +++--
 1 file changed, 15 insertions(+), 2 deletions(-)

diff --git a/cmds-filesystem.c b/cmds-filesystem.c
index 467aff11..62112705 100644
--- a/cmds-filesystem.c
+++ b/cmds-filesystem.c
@@ -1184,6 +1184,18 @@ static int cmd_filesystem_label(int argc, char **argv)
}
 }
 
+static const char * const cmd_filesystem_balance_usage[] = {
+   "btrfs filesystem balance [args...] (alias of \"btrfs balance\")",
+   "Please see \"btrfs balance --help\" for more information.",
+   NULL
+};
+
+/* Compatible old "btrfs filesystem balance" command */
+static int cmd_filesystem_balance(int argc, char **argv)
+{
+   return cmd_balance(argc, argv);
+}
+
 static const char filesystem_cmd_group_info[] =
 "overall filesystem tasks and information";
 
@@ -1197,8 +1209,9 @@ const struct cmd_group filesystem_cmd_group = {
0 },
{ "defragment", cmd_filesystem_defrag,
cmd_filesystem_defrag_usage, NULL, 0 },
-   { "balance", cmd_balance, NULL, _cmd_group,
-   CMD_HIDDEN },
+   { "balance", cmd_filesystem_balance,
+  cmd_filesystem_balance_usage, _cmd_group,
+  CMD_HIDDEN },
{ "resize", cmd_filesystem_resize, cmd_filesystem_resize_usage,
NULL, 0 },
{ "label", cmd_filesystem_label, cmd_filesystem_label_usage,
-- 
2.12.3

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 11/20] btrfs-progs: reorder placement of help declarations for send/receive

[jeffm]

From: Jeff Mahoney 

The usage definitions for send and receive follow the command
definitions, which use them.  This works because we declare them
in commands.h.  When we move to using cmd_struct as the entry point,
these declarations will be removed, breaking the commands.  Since
that would be an otherwise unrelated change, this patch reorders
them separately.

Signed-off-by: Jeff Mahoney 
---
 cmds-receive.c | 62 ++--
 cmds-send.c| 69 +-
 2 files changed, 66 insertions(+), 65 deletions(-)

diff --git a/cmds-receive.c b/cmds-receive.c
index 68123a31..b3709f36 100644
--- a/cmds-receive.c
+++ b/cmds-receive.c
@@ -1248,6 +1248,37 @@ out:
return ret;
 }
 
+const char * const cmd_receive_usage[] = {
+   "btrfs receive [options] \n"
+   "btrfs receive --dump [options]",
+   "Receive subvolumes from a stream",
+   "Receives one or more subvolumes that were previously",
+   "sent with btrfs send. The received subvolumes are stored",
+   "into MOUNT.",
+   "The receive will fail in case the receiving subvolume",
+   "already exists. It will also fail in case a previously",
+   "received subvolume has been changed after it was received.",
+   "After receiving a subvolume, it is immediately set to",
+   "read-only.",
+   "",
+   "-v   increase verbosity about performed actions",
+   "-f FILE  read the stream from FILE instead of stdin",
+   "-e   terminate after receiving an  marker in the 
stream.",
+   " Without this option the receiver side terminates only 
in case",
+   " of an error on end of file.",
+   "-C|--chroot  confine the process to  using chroot",
+   "-E|--max-errors NERR",
+   " terminate as soon as NERR errors occur while",
+   " stream processing commands from the stream.",
+   " Default value is 1. A value of 0 means no limit.",
+   "-m ROOTMOUNT the root mount point of the destination filesystem.",
+   " If /proc is not accessible, use this to tell us 
where",
+   " this file system is mounted.",
+   "--dump   dump stream metadata, one line per operation,",
+   " does not require the MOUNT parameter",
+   NULL
+};
+
 int cmd_receive(int argc, char **argv)
 {
char *tomnt = NULL;
@@ -1357,34 +1388,3 @@ out:
 
return !!ret;
 }
-
-const char * const cmd_receive_usage[] = {
-   "btrfs receive [options] \n"
-   "btrfs receive --dump [options]",
-   "Receive subvolumes from a stream",
-   "Receives one or more subvolumes that were previously",
-   "sent with btrfs send. The received subvolumes are stored",
-   "into MOUNT.",
-   "The receive will fail in case the receiving subvolume",
-   "already exists. It will also fail in case a previously",
-   "received subvolume has been changed after it was received.",
-   "After receiving a subvolume, it is immediately set to",
-   "read-only.",
-   "",
-   "-v   increase verbosity about performed actions",
-   "-f FILE  read the stream from FILE instead of stdin",
-   "-e   terminate after receiving an  marker in the 
stream.",
-   " Without this option the receiver side terminates only 
in case",
-   " of an error on end of file.",
-   "-C|--chroot  confine the process to  using chroot",
-   "-E|--max-errors NERR",
-   " terminate as soon as NERR errors occur while",
-   " stream processing commands from the stream.",
-   " Default value is 1. A value of 0 means no limit.",
-   "-m ROOTMOUNT the root mount point of the destination filesystem.",
-   " If /proc is not accessible, use this to tell us 
where",
-   " this file system is mounted.",
-   "--dump   dump stream metadata, one line per operation,",
-   " does not require the MOUNT parameter",
-   NULL
-};
diff --git a/cmds-send.c b/cmds-send.c
index c5ecdaa1..8365e9c9 100644
--- a/cmds-send.c
+++ b/cmds-send.c
@@ -489,6 +489,41 @@ static void free_send_info(struct btrfs_send *sctx)
subvol_uuid_search_finit(>sus);
 }
 
+
+const char * const cmd_send_usage[] = {
+   "btrfs send [-ve] [-p ] [-c ] [-f ] 
 [...]",
+   "Send the subvolume(s) to stdout.",
+   "Sends the subvolume(s) specified by  to stdout.",
+   " should be read-only here.",
+   "By default, this will send the whole subvolume. To do an incremental",
+   "send, use '-p '. If you want to allow btrfs to clone from",
+   "any additional local snapshots, use '-c ' (multiple times",
+   

[PATCH 03/20] btrfs-progs: constify pathnames passed as arguments

[jeffm]

From: Jeff Mahoney 

It's unlikely we're going to modify a pathname argument, so codify that
and use const.

Reviewed-by: Qu Wenruo 
Signed-off-by: Jeff Mahoney 
---
 chunk-recover.c | 4 ++--
 cmds-device.c   | 2 +-
 cmds-fi-usage.c | 6 +++---
 cmds-rescue.c   | 4 ++--
 4 files changed, 8 insertions(+), 8 deletions(-)

diff --git a/chunk-recover.c b/chunk-recover.c
index 705bcf52..1d30db51 100644
--- a/chunk-recover.c
+++ b/chunk-recover.c
@@ -1492,7 +1492,7 @@ out:
return ERR_PTR(ret);
 }
 
-static int recover_prepare(struct recover_control *rc, char *path)
+static int recover_prepare(struct recover_control *rc, const char *path)
 {
int ret;
int fd;
@@ -2296,7 +2296,7 @@ static void validate_rebuild_chunks(struct 
recover_control *rc)
 /*
  * Return 0 when successful, < 0 on error and > 0 if aborted by user
  */
-int btrfs_recover_chunk_tree(char *path, int verbose, int yes)
+int btrfs_recover_chunk_tree(const char *path, int verbose, int yes)
 {
int ret = 0;
struct btrfs_root *root = NULL;
diff --git a/cmds-device.c b/cmds-device.c
index 86459d1b..a49c9d9d 100644
--- a/cmds-device.c
+++ b/cmds-device.c
@@ -526,7 +526,7 @@ static const char * const cmd_device_usage_usage[] = {
NULL
 };
 
-static int _cmd_device_usage(int fd, char *path, unsigned unit_mode)
+static int _cmd_device_usage(int fd, const char *path, unsigned unit_mode)
 {
int i;
int ret = 0;
diff --git a/cmds-fi-usage.c b/cmds-fi-usage.c
index de7ad668..9a1c76ab 100644
--- a/cmds-fi-usage.c
+++ b/cmds-fi-usage.c
@@ -227,7 +227,7 @@ static int cmp_btrfs_ioctl_space_info(const void *a, const 
void *b)
 /*
  * This function load all the information about the space usage
  */
-static struct btrfs_ioctl_space_args *load_space_info(int fd, char *path)
+static struct btrfs_ioctl_space_args *load_space_info(int fd, const char *path)
 {
struct btrfs_ioctl_space_args *sargs = NULL, *sargs_orig = NULL;
int ret, count;
@@ -305,7 +305,7 @@ static void get_raid56_used(struct chunk_info *chunks, int 
chunkcount,
 #defineMIN_UNALOCATED_THRESH   SZ_16M
 static int print_filesystem_usage_overall(int fd, struct chunk_info *chunkinfo,
int chunkcount, struct device_info *devinfo, int devcount,
-   char *path, unsigned unit_mode)
+   const char *path, unsigned unit_mode)
 {
struct btrfs_ioctl_space_args *sargs = NULL;
int i;
@@ -931,7 +931,7 @@ static void _cmd_filesystem_usage_linear(unsigned unit_mode,
 static int print_filesystem_usage_by_chunk(int fd,
struct chunk_info *chunkinfo, int chunkcount,
struct device_info *devinfo, int devcount,
-   char *path, unsigned unit_mode, int tabular)
+   const char *path, unsigned unit_mode, int tabular)
 {
struct btrfs_ioctl_space_args *sargs;
int ret = 0;
diff --git a/cmds-rescue.c b/cmds-rescue.c
index c40088ad..c61145bc 100644
--- a/cmds-rescue.c
+++ b/cmds-rescue.c
@@ -32,8 +32,8 @@ static const char * const rescue_cmd_group_usage[] = {
NULL
 };
 
-int btrfs_recover_chunk_tree(char *path, int verbose, int yes);
-int btrfs_recover_superblocks(char *path, int verbose, int yes);
+int btrfs_recover_chunk_tree(const char *path, int verbose, int yes);
+int btrfs_recover_superblocks(const char *path, int verbose, int yes);
 
 static const char * const cmd_rescue_chunk_recover_usage[] = {
"btrfs rescue chunk-recover [options] ",
-- 
2.12.3

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 07/20] btrfs-progs: qgroups: introduce and use info and limit structures

[jeffm]

From: Jeff Mahoney 

We use structures to pass the info and limit from the kernel as items
but store the individual values separately in btrfs_qgroup.  We already
have a btrfs_qgroup_limit structure that's used for setting the limit.

This patch introduces a btrfs_qgroup_info structure and uses that and
btrfs_qgroup_limit in btrfs_qgroup.

Signed-off-by: Jeff Mahoney 
---
 qgroup.c | 82 ++--
 qgroup.h |  8 +++
 2 files changed, 52 insertions(+), 38 deletions(-)

diff --git a/qgroup.c b/qgroup.c
index 5600da99..57815718 100644
--- a/qgroup.c
+++ b/qgroup.c
@@ -46,20 +46,12 @@ struct btrfs_qgroup {
/*
 * info_item
 */
-   u64 generation;
-   u64 rfer;   /*referenced*/
-   u64 rfer_cmpr;  /*referenced compressed*/
-   u64 excl;   /*exclusive*/
-   u64 excl_cmpr;  /*exclusive compressed*/
+   struct btrfs_qgroup_info info;
 
/*
 *limit_item
 */
-   u64 flags;  /*which limits are set*/
-   u64 max_rfer;
-   u64 max_excl;
-   u64 rsv_rfer;
-   u64 rsv_excl;
+   struct btrfs_qgroup_limit limit;
 
/*qgroups this group is member of*/
struct list_head qgroups;
@@ -260,6 +252,11 @@ void print_pathname_column(struct btrfs_qgroup *qgroup, 
bool verbose)
fputs("", stdout);
 }
 
+static int print_u64(u64 value, int unit_mode, int max_len)
+{
+   return printf("%*s", max_len, pretty_size_mode(value, unit_mode));
+}
+
 static void print_qgroup_column(struct btrfs_qgroup *qgroup,
enum btrfs_qgroup_column_enum column,
bool verbose)
@@ -279,24 +276,26 @@ static void print_qgroup_column(struct btrfs_qgroup 
*qgroup,
print_qgroup_column_add_blank(BTRFS_QGROUP_QGROUPID, len);
break;
case BTRFS_QGROUP_RFER:
-   len = printf("%*s", max_len, pretty_size_mode(qgroup->rfer, 
unit_mode));
+   len = print_u64(qgroup->info.referenced, unit_mode, max_len);
break;
case BTRFS_QGROUP_EXCL:
-   len = printf("%*s", max_len, pretty_size_mode(qgroup->excl, 
unit_mode));
+   len = print_u64(qgroup->info.exclusive, unit_mode, max_len);
break;
case BTRFS_QGROUP_PARENT:
len = print_parent_column(qgroup);
print_qgroup_column_add_blank(BTRFS_QGROUP_PARENT, len);
break;
case BTRFS_QGROUP_MAX_RFER:
-   if (qgroup->flags & BTRFS_QGROUP_LIMIT_MAX_RFER)
-   len = printf("%*s", max_len, 
pretty_size_mode(qgroup->max_rfer, unit_mode));
+   if (qgroup->limit.flags & BTRFS_QGROUP_LIMIT_MAX_RFER)
+   len = print_u64(qgroup->limit.max_referenced,
+   unit_mode, max_len);
else
len = printf("%*s", max_len, "none");
break;
case BTRFS_QGROUP_MAX_EXCL:
-   if (qgroup->flags & BTRFS_QGROUP_LIMIT_MAX_EXCL)
-   len = printf("%*s", max_len, 
pretty_size_mode(qgroup->max_excl, unit_mode));
+   if (qgroup->limit.flags & BTRFS_QGROUP_LIMIT_MAX_EXCL)
+   len = print_u64(qgroup->limit.max_exclusive,
+   unit_mode, max_len);
else
len = printf("%*s", max_len, "none");
break;
@@ -439,9 +438,9 @@ static int comp_entry_with_rfer(struct btrfs_qgroup *entry1,
 {
int ret;
 
-   if (entry1->rfer > entry2->rfer)
+   if (entry1->info.referenced > entry2->info.referenced)
ret = 1;
-   else if (entry1->rfer < entry2->rfer)
+   else if (entry1->info.referenced < entry2->info.referenced)
ret = -1;
else
ret = 0;
@@ -455,9 +454,9 @@ static int comp_entry_with_excl(struct btrfs_qgroup *entry1,
 {
int ret;
 
-   if (entry1->excl > entry2->excl)
+   if (entry1->info.exclusive > entry2->info.exclusive)
ret = 1;
-   else if (entry1->excl < entry2->excl)
+   else if (entry1->info.exclusive < entry2->info.exclusive)
ret = -1;
else
ret = 0;
@@ -471,9 +470,9 @@ static int comp_entry_with_max_rfer(struct btrfs_qgroup 
*entry1,
 {
int ret;
 
-   if (entry1->max_rfer > entry2->max_rfer)
+   if (entry1->limit.max_referenced > entry2->limit.max_referenced)
ret = 1;
-   else if (entry1->max_rfer < entry2->max_rfer)
+   else if (entry1->limit.max_referenced < entry2->limit.max_referenced)
ret = -1;
else
ret = 0;
@@ -487,9 +486,9 @@ static int comp_entry_with_max_excl(struct btrfs_qgroup 
*entry1,
 {
int ret;
 
-   if (entry1->max_excl > entry2->max_excl)
+

[PATCH 01/20] btrfs-progs: quota: Add -W option to rescan to wait without starting rescan

[jeffm]

From: Jeff Mahoney 

This patch adds a new -W option to wait for a rescan without starting a
new operation.  This is useful for things like xfstests where we want
do to do a "btrfs quota enable" and not continue until the subsequent
rescan has finished.

In addition to documenting the new option in the man page, I've cleaned
up the rescan entry to document the -w option a bit better.

Reviewed-by: Qu Wenruo 
Signed-off-by: Jeff Mahoney 
---
 Documentation/btrfs-quota.asciidoc | 10 +++---
 cmds-quota.c   | 20 ++--
 2 files changed, 21 insertions(+), 9 deletions(-)

diff --git a/Documentation/btrfs-quota.asciidoc 
b/Documentation/btrfs-quota.asciidoc
index 85ebf729..0b64a69b 100644
--- a/Documentation/btrfs-quota.asciidoc
+++ b/Documentation/btrfs-quota.asciidoc
@@ -238,15 +238,19 @@ Disable subvolume quota support for a filesystem.
 *enable* ::
 Enable subvolume quota support for a filesystem.
 
-*rescan* [-s] ::
+*rescan* [-s|-w|-W] ::
 Trash all qgroup numbers and scan the metadata again with the current config.
 +
 `Options`
 +
 -s
-show status of a running rescan operation.
+Show status of a running rescan operation.
+
 -w
-wait for rescan operation to finish(can be already in progress).
+Start rescan operation and wait until it has finished before exiting.  If a 
rescan is already running, wait until it finishes and then exit without 
starting a new one.
+
+-W
+Wait for rescan operation to finish and then exit.  If a rescan is not already 
running, exit silently.
 
 EXIT STATUS
 ---
diff --git a/cmds-quota.c b/cmds-quota.c
index 745889d1..7f933495 100644
--- a/cmds-quota.c
+++ b/cmds-quota.c
@@ -120,14 +120,20 @@ static int cmd_quota_rescan(int argc, char **argv)
int wait_for_completion = 0;
 
while (1) {
-   int c = getopt(argc, argv, "sw");
+   int c = getopt(argc, argv, "swW");
if (c < 0)
break;
switch (c) {
case 's':
ioctlnum = BTRFS_IOC_QUOTA_RESCAN_STATUS;
break;
+   case 'W':
+   ioctlnum = 0;
+   wait_for_completion = 1;
+   break;
case 'w':
+   /* Reset it in case the user did both -W and -w */
+   ioctlnum = BTRFS_IOC_QUOTA_RESCAN;
wait_for_completion = 1;
break;
default:
@@ -135,8 +141,8 @@ static int cmd_quota_rescan(int argc, char **argv)
}
}
 
-   if (ioctlnum != BTRFS_IOC_QUOTA_RESCAN && wait_for_completion) {
-   error("switch -w cannot be used with -s");
+   if (ioctlnum == BTRFS_IOC_QUOTA_RESCAN_STATUS && wait_for_completion) {
+   error("switches -w/-W cannot be used with -s");
return 1;
}
 
@@ -150,8 +156,10 @@ static int cmd_quota_rescan(int argc, char **argv)
if (fd < 0)
return 1;
 
-   ret = ioctl(fd, ioctlnum, );
-   e = errno;
+   if (ioctlnum) {
+   ret = ioctl(fd, ioctlnum, );
+   e = errno;
+   }
 
if (ioctlnum == BTRFS_IOC_QUOTA_RESCAN_STATUS) {
close_file_or_dir(fd, dirstream);
@@ -167,7 +175,7 @@ static int cmd_quota_rescan(int argc, char **argv)
return 0;
}
 
-   if (ret == 0) {
+   if (ioctlnum == BTRFS_IOC_QUOTA_RESCAN && ret == 0) {
printf("quota rescan started\n");
fflush(stdout);
} else if (ret < 0 && (!wait_for_completion || e != EINPROGRESS)) {
-- 
2.12.3

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 09/20] btrfs-progs: subvolume: add quota info to btrfs sub show

[jeffm]

From: Jeff Mahoney 

This patch reports on the first-level qgroup, if any, associated with
a particular subvolume.  It displays the usage and limit, subject
to the usual unit parameters.

Signed-off-by: Jeff Mahoney 
---
 cmds-subvolume.c | 50 ++
 1 file changed, 50 insertions(+)

diff --git a/cmds-subvolume.c b/cmds-subvolume.c
index 769d2a76..96fb7b06 100644
--- a/cmds-subvolume.c
+++ b/cmds-subvolume.c
@@ -972,6 +972,7 @@ static const char * const cmd_subvol_show_usage[] = {
"Show more information about the subvolume",
"-r|--rootid   rootid of the subvolume",
"-u|--uuid uuid of the subvolume",
+   HELPINFO_UNITS_SHORT_LONG,
"",
"If no option is specified,  will be shown, otherwise",
"the rootid or uuid are resolved relative to the  path.",
@@ -993,6 +994,13 @@ static int cmd_subvol_show(int argc, char **argv)
int by_uuid = 0;
u64 rootid_arg;
u8 uuid_arg[BTRFS_UUID_SIZE];
+   struct btrfs_qgroup_stats stats;
+   unsigned int unit_mode;
+   const char *referenced_size;
+   const char *referenced_limit_size = "-";
+   unsigned int field_width = 0;
+
+   unit_mode = get_unit_mode_from_arg(, argv, 1);
 
while (1) {
int c;
@@ -1112,6 +1120,48 @@ static int cmd_subvol_show(int argc, char **argv)
btrfs_list_subvols_print(fd, filter_set, NULL, BTRFS_LIST_LAYOUT_RAW,
1, raw_prefix);
 
+   ret = btrfs_qgroup_query(fd, get_ri.root_id, );
+   if (ret && ret != -ENOTTY && ret != -ENODATA) {
+   fprintf(stderr,
+   "\nERROR: BTRFS_IOC_QUOTA_QUERY failed: %s\n",
+   strerror(-ret));
+   goto out;
+   }
+
+   printf("\tQuota Usage:\t\t");
+   fflush(stdout);
+   if (ret) {
+   if (ret == -ENOTTY)
+   printf("quotas not enabled\n");
+   else
+   printf("quotas not available\n");
+   goto out;
+   }
+
+   referenced_size = pretty_size_mode(stats.info.referenced, unit_mode);
+   if (stats.limit.max_referenced)
+   referenced_limit_size = pretty_size_mode(
+   stats.limit.max_referenced,
+   unit_mode);
+   field_width = max(strlen(referenced_size),
+ strlen(referenced_limit_size));
+
+   printf("%-*s referenced, %s exclusive\n ", field_width,
+  referenced_size,
+  pretty_size_mode(stats.info.exclusive, unit_mode));
+
+   printf("\tQuota Limits:\t\t");
+   if (stats.limit.max_referenced || stats.limit.max_exclusive) {
+   const char *excl = "-";
+
+   if (stats.limit.max_exclusive)
+   excl = pretty_size_mode(stats.limit.max_exclusive,
+   unit_mode);
+   printf("%-*s referenced, %s exclusive\n", field_width,
+  referenced_limit_size, excl);
+   } else
+   printf("None\n");
+
 out:
btrfs_cleanup_root_info(_ri);
free(filter_set);
-- 
2.12.3

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 04/20] btrfs-progs: btrfs-list: add rb_entry helpers for root_info

[jeffm]

From: Jeff Mahoney 

We use rb_entry all over the place for the root_info pointers.  Add
a helper to make the code more readable.

Signed-off-by: Jeff Mahoney 
---
 btrfs-list.c | 30 --
 1 file changed, 20 insertions(+), 10 deletions(-)

diff --git a/btrfs-list.c b/btrfs-list.c
index e01c5899..90c98be1 100644
--- a/btrfs-list.c
+++ b/btrfs-list.c
@@ -44,6 +44,16 @@ struct root_lookup {
struct rb_root root;
 };
 
+static inline struct root_info *to_root_info(struct rb_node *node)
+{
+   return rb_entry(node, struct root_info, rb_node);
+}
+
+static inline struct root_info *to_root_info_sorted(struct rb_node *node)
+{
+   return rb_entry(node, struct root_info, sort_node);
+}
+
 static struct {
char*name;
char*column_name;
@@ -309,7 +319,7 @@ static int sort_tree_insert(struct root_lookup *sort_tree,
 
while (*p) {
parent = *p;
-   curr = rb_entry(parent, struct root_info, sort_node);
+   curr = to_root_info_sorted(parent);
 
ret = sort_comp(ins, curr, comp_set);
if (ret < 0)
@@ -340,7 +350,7 @@ static int root_tree_insert(struct root_lookup *root_tree,
 
while(*p) {
parent = *p;
-   curr = rb_entry(parent, struct root_info, rb_node);
+   curr = to_root_info(parent);
 
ret = comp_entry_with_rootid(ins, curr, 0);
if (ret < 0)
@@ -371,7 +381,7 @@ static struct root_info *root_tree_search(struct 
root_lookup *root_tree,
tmp.root_id = root_id;
 
while(n) {
-   entry = rb_entry(n, struct root_info, rb_node);
+   entry = to_root_info(n);
 
ret = comp_entry_with_rootid(, entry, 0);
if (ret < 0)
@@ -528,7 +538,7 @@ static void free_root_info(struct rb_node *node)
 {
struct root_info *ri;
 
-   ri = rb_entry(node, struct root_info, rb_node);
+   ri = to_root_info(node);
free(ri->name);
free(ri->path);
free(ri->full_path);
@@ -1268,7 +1278,7 @@ static void filter_and_sort_subvol(struct root_lookup 
*all_subvols,
 
n = rb_last(_subvols->root);
while (n) {
-   entry = rb_entry(n, struct root_info, rb_node);
+   entry = to_root_info(n);
 
ret = resolve_root(all_subvols, entry, top_id);
if (ret == -ENOENT) {
@@ -1300,7 +1310,7 @@ static int list_subvol_fill_paths(int fd, struct 
root_lookup *root_lookup)
while (n) {
struct root_info *entry;
int ret;
-   entry = rb_entry(n, struct root_info, rb_node);
+   entry = to_root_info(n);
ret = lookup_ino_path(fd, entry);
if (ret && ret != -ENOENT)
return ret;
@@ -1467,7 +1477,7 @@ static void print_all_subvol_info(struct root_lookup 
*sorted_tree,
 
n = rb_first(_tree->root);
while (n) {
-   entry = rb_entry(n, struct root_info, sort_node);
+   entry = to_root_info_sorted(n);
 
/* The toplevel subvolume is not listed by default */
if (entry->root_id == BTRFS_FS_TREE_OBJECTID)
@@ -1558,7 +1568,7 @@ int btrfs_get_toplevel_subvol(int fd, struct root_info 
*the_ri)
return ret;
 
rbn = rb_first();
-   ri = rb_entry(rbn, struct root_info, rb_node);
+   ri = to_root_info(rbn);
 
if (ri->root_id != BTRFS_FS_TREE_OBJECTID)
return -ENOENT;
@@ -1590,7 +1600,7 @@ int btrfs_get_subvol(int fd, struct root_info *the_ri)
 
rbn = rb_first();
while(rbn) {
-   ri = rb_entry(rbn, struct root_info, rb_node);
+   ri = to_root_info(rbn);
rr = resolve_root(, ri, root_id);
if (rr == -ENOENT) {
ret = -ENOENT;
@@ -1814,7 +1824,7 @@ char *btrfs_list_path_for_root(int fd, u64 root)
while (n) {
struct root_info *entry;
 
-   entry = rb_entry(n, struct root_info, rb_node);
+   entry = to_root_info(n);
ret = resolve_root(_lookup, entry, top_id);
if (ret == -ENOENT && entry->root_id == root) {
ret_path = NULL;
-- 
2.12.3

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 08/20] btrfs-progs: qgroups: introduce btrfs_qgroup_query

[jeffm]

From: Jeff Mahoney 

The only mechanism we have in the progs for searching qgroups is to load
all of them and filter the results.  This works for qgroup show but
to add quota information to 'btrfs subvoluem show' it's pretty wasteful.

This patch splits out setting up the search and performing the search so
we can search for a single qgroupid more easily.  Since TREE_SEARCH
will give results that don't strictly match the search terms, we add
a filter to match only the results we care about.

Signed-off-by: Jeff Mahoney 
---
 qgroup.c | 143 ---
 qgroup.h |   7 
 2 files changed, 116 insertions(+), 34 deletions(-)

diff --git a/qgroup.c b/qgroup.c
index 57815718..d076b1de 100644
--- a/qgroup.c
+++ b/qgroup.c
@@ -1165,11 +1165,30 @@ static inline void print_status_flag_warning(u64 flags)
warning("qgroup data inconsistent, rescan recommended");
 }
 
-static int __qgroups_search(int fd, struct qgroup_lookup *qgroup_lookup)
+static bool key_in_range(const struct btrfs_key *key,
+const struct btrfs_ioctl_search_key *sk)
+{
+   if (key->objectid < sk->min_objectid ||
+   key->objectid > sk->max_objectid)
+   return false;
+
+   if (key->type < sk->min_type ||
+   key->type > sk->max_type)
+   return false;
+
+   if (key->offset < sk->min_offset ||
+   key->offset > sk->max_offset)
+   return false;
+
+   return true;
+}
+
+static int __qgroups_search(int fd, struct btrfs_ioctl_search_args *args,
+   struct qgroup_lookup *qgroup_lookup)
 {
int ret;
-   struct btrfs_ioctl_search_args args;
-   struct btrfs_ioctl_search_key *sk = 
+   struct btrfs_ioctl_search_key *sk = >key;
+   struct btrfs_ioctl_search_key filter_key = args->key;
struct btrfs_ioctl_search_header *sh;
unsigned long off = 0;
unsigned int i;
@@ -1180,30 +1199,15 @@ static int __qgroups_search(int fd, struct 
qgroup_lookup *qgroup_lookup)
u64 qgroupid;
u64 qgroupid1;
 
-   memset(, 0, sizeof(args));
-
-   sk->tree_id = BTRFS_QUOTA_TREE_OBJECTID;
-   sk->max_type = BTRFS_QGROUP_RELATION_KEY;
-   sk->min_type = BTRFS_QGROUP_STATUS_KEY;
-   sk->max_objectid = (u64)-1;
-   sk->max_offset = (u64)-1;
-   sk->max_transid = (u64)-1;
-   sk->nr_items = 4096;
-
qgroup_lookup_init(qgroup_lookup);
 
while (1) {
-   ret = ioctl(fd, BTRFS_IOC_TREE_SEARCH, );
+   ret = ioctl(fd, BTRFS_IOC_TREE_SEARCH, args);
if (ret < 0) {
-   if (errno == ENOENT) {
-   error("can't list qgroups: quotas not enabled");
+   if (errno == ENOENT)
ret = -ENOTTY;
-   } else {
-   error("can't list qgroups: %s",
-  strerror(errno));
+   else
ret = -errno;
-   }
-
break;
}
 
@@ -1217,37 +1221,46 @@ static int __qgroups_search(int fd, struct 
qgroup_lookup *qgroup_lookup)
 * read the root_ref item it contains
 */
for (i = 0; i < sk->nr_items; i++) {
-   sh = (struct btrfs_ioctl_search_header *)(args.buf +
+   struct btrfs_key key;
+
+   sh = (struct btrfs_ioctl_search_header *)(args->buf +
  off);
off += sizeof(*sh);
 
-   switch (btrfs_search_header_type(sh)) {
+   key.objectid = btrfs_search_header_objectid(sh);
+   key.type = btrfs_search_header_type(sh);
+   key.offset = btrfs_search_header_offset(sh);
+
+   if (!key_in_range(, _key))
+   goto next;
+
+   switch (key.type) {
case BTRFS_QGROUP_STATUS_KEY:
si = (struct btrfs_qgroup_status_item *)
-(args.buf + off);
+(args->buf + off);
flags = btrfs_stack_qgroup_status_flags(si);
 
print_status_flag_warning(flags);
break;
case BTRFS_QGROUP_INFO_KEY:
-   qgroupid = btrfs_search_header_offset(sh);
+   qgroupid = key.offset;
info = (struct btrfs_qgroup_info_item *)
-  (args.buf + off);
+  (args->buf + off);
 
  

[PATCH 17/20] btrfs-progs: add support for output formats

[jeffm]

From: Jeff Mahoney 

This adds a global --format option to request extended output formats
from each command.  Most of it is plumbing a new cmd_context structure
that's established at the beginning of argument parsing into the command
callbacks.  That structure currently only contains the output mode enum.

We currently only support text mode.  Command help reports what
output formats are available for each command.  Global help reports
what valid formats are.

If an invalid format is requested, an error is reported and we global usage
is dumped that lists the valid formats.

Each command sets a bitmask that describes which formats it is capable
of outputting.  If a globally valid format is requested of a command
that doesn't support it, an error is reported and command usage dumped.

Commands don't need to specify that they support text output.  All
commands are required to output text.

Signed-off-by: Jeff Mahoney 
---
 btrfs-debug-tree.c|   3 +-
 btrfs.c   | 110 ++
 check/main.c  |   3 +-
 cmds-balance.c|  16 +--
 cmds-device.c |  31 +
 cmds-fi-du.c  |   1 +
 cmds-fi-usage.c   |   1 +
 cmds-filesystem.c |  14 --
 cmds-inspect-dump-super.c |   1 +
 cmds-inspect-dump-tree.c  |   1 +
 cmds-inspect-tree-stats.c |   1 +
 cmds-inspect.c|  10 -
 cmds-property.c   |   6 ++-
 cmds-qgroup.c |  17 +--
 cmds-quota.c  |  14 --
 cmds-receive.c|   3 +-
 cmds-replace.c|   8 +++-
 cmds-rescue.c |   9 +++-
 cmds-restore.c|   3 +-
 cmds-scrub.c  |  21 ++---
 cmds-send.c   |   3 +-
 cmds-subvolume.c  |  24 +++---
 commands.h|  32 +++---
 help.c|  51 -
 help.h|   2 +
 25 files changed, 301 insertions(+), 84 deletions(-)

diff --git a/btrfs-debug-tree.c b/btrfs-debug-tree.c
index 49a2e949..8cd05d53 100644
--- a/btrfs-debug-tree.c
+++ b/btrfs-debug-tree.c
@@ -26,6 +26,7 @@ int main(int argc, char **argv)
 {
const struct cmd_struct *cmd = _struct_inspect_dump_tree;
int ret;
+   struct cmd_context cmdcxt = {};
 
set_argv0(argv);
 
@@ -34,7 +35,7 @@ int main(int argc, char **argv)
 
radix_tree_init();
 
-   ret = cmd_execute(cmd, argc, argv);
+   ret = cmd_execute(cmd, , argc, argv);
 
btrfs_close_all_devices();
 
diff --git a/btrfs.c b/btrfs.c
index 49128182..32b8e090 100644
--- a/btrfs.c
+++ b/btrfs.c
@@ -26,7 +26,7 @@
 #include "help.h"
 
 static const char * const btrfs_cmd_group_usage[] = {
-   "btrfs [--help] [--version]  [...]  []",
+   "btrfs [--help] [--version] [--format ]  [...] 
 []",
NULL
 };
 
@@ -98,13 +98,36 @@ parse_command_token(const char *arg, const struct cmd_group 
*grp)
return cmd;
 }
 
+static bool cmd_provides_output_format(const struct cmd_struct *cmd,
+  const struct cmd_context *cmdcxt)
+{
+   if (!cmdcxt->output_mode)
+   return true;
+
+   return (1 << cmdcxt->output_mode) & cmd->cmd_format_flags;
+}
+
 static void handle_help_options_next_level(const struct cmd_struct *cmd,
-   int argc, char **argv)
+  const struct cmd_context *cmdcxt,
+  int argc, char **argv)
 {
+   int err = 0;
+
if (argc < 2)
return;
 
-   if (!strcmp(argv[1], "--help")) {
+   /* Check if the command can provide the requested output format */
+   if (!cmd->next && !cmd_provides_output_format(cmd, cmdcxt)) {
+   ASSERT(cmdcxt->output_mode >= 0);
+   ASSERT(cmdcxt->output_mode < CMD_OUTPUT_MAX);
+   fprintf(stderr,
+   "error: %s output is unsupported for this command.\n\n",
+   cmd_outputs[cmdcxt->output_mode]);
+
+   err = 1;
+   }
+
+   if (!strcmp(argv[1], "--help") || err) {
if (cmd->next) {
argc--;
argv++;
@@ -113,12 +136,13 @@ static void handle_help_options_next_level(const struct 
cmd_struct *cmd,
usage_command(cmd, true, false);
}
 
-   exit(0);
+   exit(err);
}
 }
 
-int handle_command_group(const struct cmd_group *grp, int argc,
-char **argv)
+int handle_command_group(const struct cmd_group *grp,
+const struct cmd_context *cmdcxt,
+int argc, char **argv)
 
 {
const struct cmd_struct *cmd;
@@ -132,10 +156,10 @@ int handle_command_group(const struct cmd_group *grp, int 
argc,
 
cmd = parse_command_token(argv[0], grp);
 
-   

[PATCH 19/20] btrfs-progs: qgroups: add json output for usage command

[jeffm]

From: Jeff Mahoney 

One of the common requests I receive is for 'df' like facilities
for subvolume usage.  Really, the request is for monitoring tools to be
able to understand when subvolumes may be approaching quota in the same
manner traditional file systems approach ENOSPC.

This patch allows us to export the qgroups data in a machine-readable
format so that monitoring tools can parse it easily.

Signed-off-by: Jeff Mahoney 
---
 Documentation/btrfs-qgroup.asciidoc |   3 +
 cmds-qgroup.c   |  22 +++-
 qgroup.c| 222 
 qgroup.h|   8 +-
 4 files changed, 251 insertions(+), 4 deletions(-)

diff --git a/Documentation/btrfs-qgroup.asciidoc 
b/Documentation/btrfs-qgroup.asciidoc
index 360b3269..7863a4d9 100644
--- a/Documentation/btrfs-qgroup.asciidoc
+++ b/Documentation/btrfs-qgroup.asciidoc
@@ -87,6 +87,9 @@ the btrfs filesystem identified by .
 *show* [options] ::
 Show all qgroups in the btrfs filesystem identified by .
 +
+If enabled, this command supports extended output in json and json:compat 
modes.
+Use of either json mode implies -P and --raw.
++
 `Options`
 +
 -p
diff --git a/cmds-qgroup.c b/cmds-qgroup.c
index f9e81e30..fd637a45 100644
--- a/cmds-qgroup.c
+++ b/cmds-qgroup.c
@@ -301,6 +301,10 @@ static const char * const cmd_qgroup_show_usage[] = {
"   you can use '+' or '-' in front of each item.",
"   (+:ascending, -:descending, ascending default)",
"--sync force sync of the filesystem before getting info",
+   "",
+#ifdef HAVE_JSON
+   "json and json:compat output implies -P and --raw.",
+#endif
NULL
 };
 
@@ -386,6 +390,10 @@ static int cmd_qgroup_show(const struct cmd_struct *cmd,
}
btrfs_qgroup_setup_units(unit_mode);
 
+   if (cmdcxt->output_mode == CMD_OUTPUT_JSON ||
+   cmdcxt->output_mode == CMD_OUTPUT_JSON_COMPAT)
+   unit_mode = UNITS_RAW;
+
if (check_argc_exact(argc - optind, 1))
usage(cmd);
 
@@ -420,7 +428,15 @@ static int cmd_qgroup_show(const struct cmd_struct *cmd,
BTRFS_QGROUP_FILTER_PARENT,
qgroupid);
}
-   ret = btrfs_show_qgroups(fd, filter_set, comparer_set, verbose);
+
+   if (cmdcxt->output_mode == CMD_OUTPUT_JSON ||
+   cmdcxt->output_mode == CMD_OUTPUT_JSON_COMPAT) {
+   bool compat = (cmdcxt->output_mode == CMD_OUTPUT_JSON_COMPAT);
+
+   ret = btrfs_qgroup_output_json(fd, filter_set, comparer_set,
+  compat);
+   } else
+   ret = btrfs_show_qgroups(fd, filter_set, comparer_set, verbose);
close_file_or_dir(fd, dirstream);
free(filter_set);
free(comparer_set);
@@ -428,7 +444,9 @@ static int cmd_qgroup_show(const struct cmd_struct *cmd,
 out:
return !!ret;
 }
-static DEFINE_SIMPLE_COMMAND(qgroup_show, "show");
+static DEFINE_COMMAND(qgroup_show, "show", cmd_qgroup_show,
+ cmd_qgroup_show_usage, NULL, 0,
+ CMD_OUTPUT_FLAG(JSON)|CMD_OUTPUT_FLAG(JSON_COMPAT));
 
 static const char * const cmd_qgroup_limit_usage[] = {
"btrfs qgroup limit [options] |none [] ",
diff --git a/qgroup.c b/qgroup.c
index d076b1de..95d443db 100644
--- a/qgroup.c
+++ b/qgroup.c
@@ -16,12 +16,16 @@
  * Boston, MA 021110-1307, USA.
  */
 
+#include "version.h"
 #include "qgroup.h"
 #include 
 #include "ctree.h"
 #include "ioctl.h"
 #include "utils.h"
 #include 
+#ifdef HAVE_JSON
+#include 
+#endif
 
 #define BTRFS_QGROUP_NFILTERS_INCREASE (2 * BTRFS_QGROUP_FILTER_MAX)
 #define BTRFS_QGROUP_NCOMPS_INCREASE (2 * BTRFS_QGROUP_COMP_MAX)
@@ -1398,6 +1402,224 @@ int btrfs_show_qgroups(int fd,
return ret;
 }
 
+#ifdef HAVE_JSON
+#define QGROUPID_FORMAT_BUF_LEN (20 + 20 + 1 + 1)
+static void format_qgroupid(char *buf, size_t size, u64 qgroupid)
+{
+   int ret;
+
+   ret = snprintf(buf, size, "%llu/%llu",
+  btrfs_qgroup_level(qgroupid),
+  btrfs_qgroup_subvid(qgroupid));
+   ASSERT(ret < sizeof(buf));
+}
+
+static json_object *export_one_u64(u64 value, bool compat)
+{
+   json_object *array, *tmp;
+
+   if (!compat)
+   return json_object_new_int64(value);
+
+   array = json_object_new_array();
+   if (!array)
+   return NULL;
+
+   tmp = json_object_new_int(value >> 32);
+   if (!tmp)
+   goto failure;
+   json_object_array_add(array, tmp);
+
+   tmp = json_object_new_int(value & 0x);
+   if (!tmp)
+   goto failure;
+   json_object_array_add(array, tmp);
+
+   return array;
+failure:
+   json_object_put(array);
+   return NULL;
+}
+
+static bool export_one_qgroup(json_object *container,
+

[PATCH 20/20] btrfs-progs: handle command groups directly for common case

[jeffm]

From: Jeff Mahoney 

Most command groups just pass their own command group to
handle_command_group.  We can remove the explicit definitions
of command group callbacks by passing the cmd_struct to
handle_command_group and allowing it to resolve the group from it.

Signed-off-by: Jeff Mahoney 
---
 btrfs.c   | 14 +++---
 cmds-balance.c|  6 +++---
 cmds-device.c |  5 -
 cmds-filesystem.c |  6 --
 cmds-inspect.c|  5 -
 cmds-property.c   |  6 --
 cmds-qgroup.c |  5 -
 cmds-quota.c  |  5 -
 cmds-replace.c|  5 -
 cmds-rescue.c |  5 -
 cmds-scrub.c  |  6 --
 cmds-subvolume.c  |  5 -
 commands.h|  7 ---
 13 files changed, 14 insertions(+), 66 deletions(-)

diff --git a/btrfs.c b/btrfs.c
index 32b8e090..427e14c8 100644
--- a/btrfs.c
+++ b/btrfs.c
@@ -140,26 +140,26 @@ static void handle_help_options_next_level(const struct 
cmd_struct *cmd,
}
 }
 
-int handle_command_group(const struct cmd_group *grp,
+int handle_command_group(const struct cmd_struct *cmd,
 const struct cmd_context *cmdcxt,
 int argc, char **argv)
 
 {
-   const struct cmd_struct *cmd;
+   const struct cmd_struct *subcmd;
 
argc--;
argv++;
if (argc < 1) {
-   usage_command_group(grp, false, false);
+   usage_command_group(cmd->next, false, false);
exit(1);
}
 
-   cmd = parse_command_token(argv[0], grp);
+   subcmd = parse_command_token(argv[0], cmd->next);
 
-   handle_help_options_next_level(cmd, cmdcxt, argc, argv);
+   handle_help_options_next_level(subcmd, cmdcxt, argc, argv);
 
-   fixup_argv0(argv, cmd->token);
-   return cmd_execute(cmd, cmdcxt, argc, argv);
+   fixup_argv0(argv, subcmd->token);
+   return cmd_execute(subcmd, cmdcxt, argc, argv);
 }
 
 static const struct cmd_group btrfs_cmd_group;
diff --git a/cmds-balance.c b/cmds-balance.c
index c17b9ee3..e414ca27 100644
--- a/cmds-balance.c
+++ b/cmds-balance.c
@@ -943,7 +943,7 @@ static const struct cmd_group balance_cmd_group = {
}
 };
 
-static int cmd_balance(const struct cmd_struct *unused,
+static int cmd_balance(const struct cmd_struct *cmd,
   const struct cmd_context *cmdcxt, int argc, char **argv)
 {
if (argc == 2 && strcmp("start", argv[1]) != 0) {
@@ -956,7 +956,7 @@ static int cmd_balance(const struct cmd_struct *unused,
return do_balance(argv[1], , 0);
}
 
-   return handle_command_group(_cmd_group, cmdcxt, argc, argv);
+   return handle_command_group(cmd, cmdcxt, argc, argv);
 }
 
-DEFINE_GROUP_COMMAND(balance, "balance");
+DEFINE_COMMAND(balance, "balance", cmd_balance, NULL, _cmd_group, 0, 
0);
diff --git a/cmds-device.c b/cmds-device.c
index ac9e82b1..f8c0ff20 100644
--- a/cmds-device.c
+++ b/cmds-device.c
@@ -630,9 +630,4 @@ static const struct cmd_group device_cmd_group = {
}
 };
 
-static int cmd_device(const struct cmd_struct *unused,
- const struct cmd_context *cmdcxt, int argc, char **argv)
-{
-   return handle_command_group(_cmd_group, cmdcxt, argc, argv);
-}
 DEFINE_GROUP_COMMAND_TOKEN(device);
diff --git a/cmds-filesystem.c b/cmds-filesystem.c
index 27e0865c..e3e54864 100644
--- a/cmds-filesystem.c
+++ b/cmds-filesystem.c
@@ -1242,10 +1242,4 @@ static const struct cmd_group filesystem_cmd_group = {
}
 };
 
-static int cmd_filesystem(const struct cmd_struct *unused,
- const struct cmd_context *cmdcxt,
- int argc, char **argv)
-{
-   return handle_command_group(_cmd_group, cmdcxt, argc, argv);
-}
 DEFINE_GROUP_COMMAND_TOKEN(filesystem);
diff --git a/cmds-inspect.c b/cmds-inspect.c
index ade9db7e..561a0fbd 100644
--- a/cmds-inspect.c
+++ b/cmds-inspect.c
@@ -658,9 +658,4 @@ static const struct cmd_group inspect_cmd_group = {
}
 };
 
-static int cmd_inspect(const struct cmd_struct *unused,
-  const struct cmd_context *cmdcxt, int argc, char **argv)
-{
-   return handle_command_group(_cmd_group, cmdcxt, argc, argv);
-}
 DEFINE_GROUP_COMMAND(inspect, "inspect");
diff --git a/cmds-property.c b/cmds-property.c
index 498fa456..58f6c48a 100644
--- a/cmds-property.c
+++ b/cmds-property.c
@@ -422,10 +422,4 @@ static const struct cmd_group property_cmd_group = {
}
 };
 
-static int cmd_property(const struct cmd_struct *unused,
-   const struct cmd_context *cmdcxt,
-   int argc, char **argv)
-{
-   return handle_command_group(_cmd_group, cmdcxt, argc, argv);
-}
 DEFINE_GROUP_COMMAND_TOKEN(property);
diff --git a/cmds-qgroup.c b/cmds-qgroup.c
index fd637a45..e20c1159 100644
--- a/cmds-qgroup.c
+++ b/cmds-qgroup.c
@@ -558,9 +558,4 @@ static const struct cmd_group qgroup_cmd_group = {
}
 };
 
-static int cmd_qgroup(const struct 

[PATCH 13/20] btrfs-progs: use cmd_struct as command entry point

[jeffm]

From: Jeff Mahoney 

Rather than having global command usage and callbacks used to create
cmd_structs in the command array, establish the cmd_struct structures
separately and use those.  The next commit in the series passes the
cmd_struct to the command callbacks such that we can access flags
and determine which of several potential command we were called as.

This establishes several macros to more easily define the commands
within each command's source.

Signed-off-by: Jeff Mahoney 
---
 btrfs-calc-size.c |   5 +-
 btrfs-debug-tree.c|   5 +-
 btrfs-show-super.c|   6 +--
 btrfs.c   |  48 ++
 check/main.c  |   5 +-
 cmds-balance.c|  27 ++
 cmds-device.c |  31 ++-
 cmds-fi-du.c  |   5 +-
 cmds-fi-usage.c   |   5 +-
 cmds-filesystem.c |  52 ++-
 cmds-inspect-dump-super.c |   5 +-
 cmds-inspect-dump-tree.c  |   5 +-
 cmds-inspect-tree-stats.c |   5 +-
 cmds-inspect.c|  36 +++--
 cmds-property.c   |  19 +++
 cmds-qgroup.c |  31 +--
 cmds-quota.c  |  17 ---
 cmds-receive.c|   5 +-
 cmds-replace.c|  19 +++
 cmds-rescue.c |  22 
 cmds-restore.c|   5 +-
 cmds-scrub.c  |  20 +---
 cmds-send.c   |   6 +--
 cmds-subvolume.c  |  38 --
 commands.h| 127 +++---
 help.c|  21 +---
 26 files changed, 326 insertions(+), 244 deletions(-)

diff --git a/btrfs-calc-size.c b/btrfs-calc-size.c
index d2d68ab2..908e830f 100644
--- a/btrfs-calc-size.c
+++ b/btrfs-calc-size.c
@@ -23,15 +23,16 @@
 
 int main(int argc, char **argv)
 {
+   const struct cmd_struct *cmd = _struct_inspect_tree_stats;
int ret;
 
warning(
 "\nthe tool has been deprecated, please use 'btrfs inspect-internal 
tree-stats' instead\n");
 
if (argc > 1 && !strcmp(argv[1], "--help"))
-   usage(cmd_inspect_tree_stats_usage);
+   usage(cmd->usagestr);
 
-   ret = cmd_inspect_tree_stats(argc, argv);
+   ret = cmd_execute(cmd, argc, argv);
 
btrfs_close_all_devices();
 
diff --git a/btrfs-debug-tree.c b/btrfs-debug-tree.c
index 7bee018f..7f254519 100644
--- a/btrfs-debug-tree.c
+++ b/btrfs-debug-tree.c
@@ -24,16 +24,17 @@
 
 int main(int argc, char **argv)
 {
+   const struct cmd_struct *cmd = _struct_inspect_dump_tree;
int ret;
 
set_argv0(argv);
 
if (argc > 1 && !strcmp(argv[1], "--help"))
-   usage(cmd_inspect_dump_tree_usage);
+   usage(cmd->usagestr);
 
radix_tree_init();
 
-   ret = cmd_inspect_dump_tree(argc, argv);
+   ret = cmd_execute(cmd, argc, argv);
 
btrfs_close_all_devices();
 
diff --git a/btrfs-show-super.c b/btrfs-show-super.c
index 4273e42d..ee717c33 100644
--- a/btrfs-show-super.c
+++ b/btrfs-show-super.c
@@ -22,7 +22,7 @@
 
 int main(int argc, char **argv)
 {
-
+   const struct cmd_struct *cmd = _struct_inspect_dump_super;
int ret;
 
set_argv0(argv);
@@ -31,9 +31,9 @@ int main(int argc, char **argv)
 "\nthe tool has been deprecated, please use 'btrfs inspect-internal 
dump-super' instead\n");
 
if (argc > 1 && !strcmp(argv[1], "--help"))
-   usage(cmd_inspect_dump_super_usage);
+   usage(cmd->usagestr);
 
-   ret = cmd_inspect_dump_super(argc, argv);
+   ret = cmd_execute(cmd, argc, argv);
 
return ret;
 }
diff --git a/btrfs.c b/btrfs.c
index fec1a135..1e68b0c0 100644
--- a/btrfs.c
+++ b/btrfs.c
@@ -42,10 +42,11 @@ static inline const char *skip_prefix(const char *str, 
const char *prefix)
 static int parse_one_token(const char *arg, const struct cmd_group *grp,
   const struct cmd_struct **cmd_ret)
 {
-   const struct cmd_struct *cmd = grp->commands;
const struct cmd_struct *abbrev_cmd = NULL, *ambiguous_cmd = NULL;
+   int i = 0;
 
-   for (; cmd->token; cmd++) {
+   for (i = 0; grp->commands[i]; i++) {
+   const struct cmd_struct *cmd = grp->commands[i];
const char *rest;
 
rest = skip_prefix(arg, cmd->token);
@@ -134,7 +135,7 @@ int handle_command_group(const struct cmd_group *grp, int 
argc,
handle_help_options_next_level(cmd, argc, argv);
 
fixup_argv0(argv, cmd->token);
-   return cmd->fn(argc, argv);
+   return cmd_execute(cmd, argc, argv);
 }
 
 static const struct cmd_group btrfs_cmd_group;
@@ -153,6 +154,8 @@ static int cmd_help(int argc, char **argv)
return 0;
 }
 
+static DEFINE_SIMPLE_COMMAND(help, "help");
+
 static const char * const cmd_version_usage[] = {
"btrfs version",
"Display btrfs-progs version",
@@ -164,6 +167,7 @@ static int cmd_version(int 

[PATCH 14/20] btrfs-progs: pass cmd_struct to command callback function

[jeffm]

From: Jeff Mahoney 

This patch passes the cmd_struct to the command callback function.  This
has several purposes: It allows the command callback to identify which
command was used to call it.  It also gives us direct access to the
usage associated with that command.

Signed-off-by: Jeff Mahoney 
---
 btrfs.c   |  8 
 check/main.c  |  2 +-
 cmds-balance.c| 19 ---
 cmds-device.c | 35 +++
 cmds-fi-du.c  |  3 ++-
 cmds-fi-usage.c   |  3 ++-
 cmds-filesystem.c | 24 
 cmds-inspect-dump-super.c |  3 ++-
 cmds-inspect-dump-tree.c  |  3 ++-
 cmds-inspect-tree-stats.c |  3 ++-
 cmds-inspect.c| 17 +++--
 cmds-property.c   | 12 
 cmds-qgroup.c | 18 +++---
 cmds-quota.c  |  9 +
 cmds-receive.c|  2 +-
 cmds-replace.c| 11 +++
 cmds-rescue.c | 14 +-
 cmds-restore.c|  2 +-
 cmds-scrub.c  | 23 +++
 cmds-send.c   |  2 +-
 cmds-subvolume.c  | 27 +--
 commands.h|  4 ++--
 22 files changed, 146 insertions(+), 98 deletions(-)

diff --git a/btrfs.c b/btrfs.c
index 1e68b0c0..49128182 100644
--- a/btrfs.c
+++ b/btrfs.c
@@ -148,7 +148,7 @@ static const char * const cmd_help_usage[] = {
NULL
 };
 
-static int cmd_help(int argc, char **argv)
+static int cmd_help(const struct cmd_struct *unused, int argc, char **argv)
 {
help_command_group(_cmd_group, argc, argv);
return 0;
@@ -162,7 +162,7 @@ static const char * const cmd_version_usage[] = {
NULL
 };
 
-static int cmd_version(int argc, char **argv)
+static int cmd_version(const struct cmd_struct *unused, int argc, char **argv)
 {
printf("%s\n", PACKAGE_STRING);
return 0;
@@ -231,13 +231,13 @@ void handle_special_globals(int shift, int argc, char 
**argv)
if (has_full)
usage_command_group(_cmd_group, true, false);
else
-   cmd_help(argc, argv);
+   cmd_execute(_struct_help, argc, argv);
exit(0);
}
 
for (i = 0; i < shift; i++)
if (strcmp(argv[i], "--version") == 0) {
-   cmd_version(argc, argv);
+   cmd_execute(_struct_version, argc, argv);
exit(0);
}
 }
diff --git a/check/main.c b/check/main.c
index 4b8f7678..bd31fb9f 100644
--- a/check/main.c
+++ b/check/main.c
@@ -9440,7 +9440,7 @@ static const char * const cmd_check_usage[] = {
NULL
 };
 
-static int cmd_check(int argc, char **argv)
+static int cmd_check(const struct cmd_struct *cmd, int argc, char **argv)
 {
struct cache_tree root_cache;
struct btrfs_root *root;
diff --git a/cmds-balance.c b/cmds-balance.c
index 7a60be61..1bd7b3ce 100644
--- a/cmds-balance.c
+++ b/cmds-balance.c
@@ -515,7 +515,8 @@ static const char * const cmd_balance_start_usage[] = {
NULL
 };
 
-static int cmd_balance_start(int argc, char **argv)
+static int cmd_balance_start(const struct cmd_struct *cmd,
+int argc, char **argv)
 {
struct btrfs_ioctl_balance_args args;
struct btrfs_balance_args *ptrs[] = { , ,
@@ -680,7 +681,8 @@ static const char * const cmd_balance_pause_usage[] = {
NULL
 };
 
-static int cmd_balance_pause(int argc, char **argv)
+static int cmd_balance_pause(const struct cmd_struct *cmd,
+int argc, char **argv)
 {
const char *path;
int fd;
@@ -719,7 +721,8 @@ static const char * const cmd_balance_cancel_usage[] = {
NULL
 };
 
-static int cmd_balance_cancel(int argc, char **argv)
+static int cmd_balance_cancel(const struct cmd_struct *cmd,
+ int argc, char **argv)
 {
const char *path;
int fd;
@@ -758,7 +761,8 @@ static const char * const cmd_balance_resume_usage[] = {
NULL
 };
 
-static int cmd_balance_resume(int argc, char **argv)
+static int cmd_balance_resume(const struct cmd_struct *cmd,
+ int argc, char **argv)
 {
struct btrfs_ioctl_balance_args args;
const char *path;
@@ -826,7 +830,8 @@ static const char * const cmd_balance_status_usage[] = {
  *   1 : Successful to know status of a pending balance
  *   0 : When there is no pending balance or completed
  */
-static int cmd_balance_status(int argc, char **argv)
+static int cmd_balance_status(const struct cmd_struct *cmd,
+ int argc, char **argv)
 {
struct btrfs_ioctl_balance_args args;
const char *path;
@@ -904,7 +909,7 @@ out:
 }
 static DEFINE_SIMPLE_COMMAND(balance_status, "status");
 
-static int 

[PATCH 16/20] btrfs-progs: pass cmd_struct to usage()

[jeffm]

From: Jeff Mahoney 

Now that every call site has a cmd_struct, we can just pass the cmd_struct
to usage to print the usager information.  This allows us to interpret
the format flags we'll add later in this series to inform the user of
which output formats any given command supports.

Signed-off-by: Jeff Mahoney 
---
 btrfs-calc-size.c |  2 +-
 btrfs-debug-tree.c|  2 +-
 btrfs-show-super.c|  2 +-
 check/main.c  |  4 ++--
 cmds-balance.c| 14 +++---
 cmds-device.c | 19 +--
 cmds-fi-du.c  |  4 ++--
 cmds-fi-usage.c   |  4 ++--
 cmds-filesystem.c | 18 +-
 cmds-inspect-dump-super.c |  4 ++--
 cmds-inspect-dump-tree.c  |  4 ++--
 cmds-inspect-tree-stats.c |  4 ++--
 cmds-inspect.c| 16 
 cmds-property.c   | 22 +-
 cmds-qgroup.c | 18 +-
 cmds-quota.c  |  8 
 cmds-receive.c|  4 ++--
 cmds-replace.c| 12 ++--
 cmds-rescue.c | 12 ++--
 cmds-restore.c|  8 
 cmds-scrub.c  | 25 ++---
 cmds-send.c   |  2 +-
 cmds-subvolume.c  | 30 +++---
 help.c|  6 +++---
 help.h|  2 +-
 25 files changed, 118 insertions(+), 128 deletions(-)

diff --git a/btrfs-calc-size.c b/btrfs-calc-size.c
index 908e830f..4c8fcc19 100644
--- a/btrfs-calc-size.c
+++ b/btrfs-calc-size.c
@@ -30,7 +30,7 @@ int main(int argc, char **argv)
 "\nthe tool has been deprecated, please use 'btrfs inspect-internal 
tree-stats' instead\n");
 
if (argc > 1 && !strcmp(argv[1], "--help"))
-   usage(cmd->usagestr);
+   usage(cmd);
 
ret = cmd_execute(cmd, argc, argv);
 
diff --git a/btrfs-debug-tree.c b/btrfs-debug-tree.c
index 7f254519..49a2e949 100644
--- a/btrfs-debug-tree.c
+++ b/btrfs-debug-tree.c
@@ -30,7 +30,7 @@ int main(int argc, char **argv)
set_argv0(argv);
 
if (argc > 1 && !strcmp(argv[1], "--help"))
-   usage(cmd->usagestr);
+   usage(cmd);
 
radix_tree_init();
 
diff --git a/btrfs-show-super.c b/btrfs-show-super.c
index ee717c33..b4a5b693 100644
--- a/btrfs-show-super.c
+++ b/btrfs-show-super.c
@@ -31,7 +31,7 @@ int main(int argc, char **argv)
 "\nthe tool has been deprecated, please use 'btrfs inspect-internal 
dump-super' instead\n");
 
if (argc > 1 && !strcmp(argv[1], "--help"))
-   usage(cmd->usagestr);
+   usage(cmd);
 
ret = cmd_execute(cmd, argc, argv);
 
diff --git a/check/main.c b/check/main.c
index bd31fb9f..0bb8633a 100644
--- a/check/main.c
+++ b/check/main.c
@@ -9530,7 +9530,7 @@ static int cmd_check(const struct cmd_struct *cmd, int 
argc, char **argv)
break;
case '?':
case 'h':
-   usage(cmd_check_usage);
+   usage(cmd);
case GETOPT_VAL_REPAIR:
printf("enabling repair mode\n");
repair = 1;
@@ -9581,7 +9581,7 @@ static int cmd_check(const struct cmd_struct *cmd, int 
argc, char **argv)
}
 
if (check_argc_exact(argc - optind, 1))
-   usage(cmd_check_usage);
+   usage(cmd);
 
if (ctx.progress_enabled) {
ctx.tp = TASK_NOTHING;
diff --git a/cmds-balance.c b/cmds-balance.c
index 488fffcc..c639459f 100644
--- a/cmds-balance.c
+++ b/cmds-balance.c
@@ -585,12 +585,12 @@ static int cmd_balance_start(const struct cmd_struct *cmd,
background = 1;
break;
default:
-   usage(cmd_balance_start_usage);
+   usage(cmd);
}
}
 
if (check_argc_exact(argc - optind, 1))
-   usage(cmd_balance_start_usage);
+   usage(cmd);
 
/*
 * allow -s only under --force, otherwise do with system chunks
@@ -692,7 +692,7 @@ static int cmd_balance_pause(const struct cmd_struct *cmd,
clean_args_no_options(cmd, argc, argv);
 
if (check_argc_exact(argc - optind, 1))
-   usage(cmd_balance_pause_usage);
+   usage(cmd);
 
path = argv[optind];
 
@@ -732,7 +732,7 @@ static int cmd_balance_cancel(const struct cmd_struct *cmd,
clean_args_no_options(cmd, argc, argv);
 
if (check_argc_exact(argc - optind, 1))
-   usage(cmd_balance_cancel_usage);
+   usage(cmd);
 
path = argv[optind];
 
@@ -773,7 +773,7 @@ static int cmd_balance_resume(const struct cmd_struct *cmd,
clean_args_no_options(cmd, argc, argv);
 
if (check_argc_exact(argc - optind, 1))
-

[PATCH 18/20] btrfs-progs: add generic support for json output

[jeffm]

From: Jeff Mahoney 

This patch adds support for JSON and JSON-compat output.  The latter is
intended to be compatible with Javascript's integers being represented
as 64-bit floats, with only 53 bits usable for the integer component.
Compat mode output will post 64-bit integers as an array of two 32-bit
integers in [high, low] format.

This patch also adds support for reporting which output formats a
command supports as well as detection of the json-c library.

Signed-off-by: Jeff Mahoney 
---
 Makefile.inc.in |  4 ++--
 commands.h  | 13 +
 configure.ac|  6 ++
 help.c  | 25 ++---
 4 files changed, 35 insertions(+), 13 deletions(-)

diff --git a/Makefile.inc.in b/Makefile.inc.in
index 56271903..68bddbed 100644
--- a/Makefile.inc.in
+++ b/Makefile.inc.in
@@ -18,9 +18,9 @@ BTRFSRESTORE_ZSTD = @BTRFSRESTORE_ZSTD@
 SUBST_CFLAGS = @CFLAGS@
 SUBST_LDFLAGS = @LDFLAGS@
 
-LIBS_BASE = @UUID_LIBS@ @BLKID_LIBS@ -L. -pthread
+LIBS_BASE = @UUID_LIBS@ @BLKID_LIBS@ @JSON_LIBS@ -L. -pthread
 LIBS_COMP = @ZLIB_LIBS@ @LZO2_LIBS@ @ZSTD_LIBS@
-STATIC_LIBS_BASE = @UUID_LIBS_STATIC@ @BLKID_LIBS_STATIC@ -L. -pthread
+STATIC_LIBS_BASE = @UUID_LIBS_STATIC@ @BLKID_LIBS_STATIC@ @JSON_LIBS_STATIC@ 
-L. -pthread
 STATIC_LIBS_COMP = @ZLIB_LIBS_STATIC@ @LZO2_LIBS_STATIC@ @ZSTD_LIBS_STATIC@
 
 prefix ?= @prefix@
diff --git a/commands.h b/commands.h
index 83316c6d..bf74eaf8 100644
--- a/commands.h
+++ b/commands.h
@@ -19,9 +19,22 @@
 
 enum cmd_output {
CMD_OUTPUT_TEXT = 0,
+#ifdef HAVE_JSON
+   CMD_OUTPUT_JSON,
+   CMD_OUTPUT_JSON_COMPAT,
+#endif
CMD_OUTPUT_MAX,
 };
 
+/*
+ * If we don't have the JSON library, map the flags to text to avoid
+ * more ifdefs elsewhere.
+ */
+#ifndef HAVE_JSON
+#define CMD_OUTPUT_JSONCMD_OUTPUT_TEXT
+#define CMD_OUTPUT_JSON_COMPAT CMD_OUTPUT_TEXT
+#endif
+
 #define CMD_OUTPUT_FLAG(x) (1 << (CMD_OUTPUT_##x))
 
 struct cmd_context {
diff --git a/configure.ac b/configure.ac
index 56d17c3a..6aec672a 100644
--- a/configure.ac
+++ b/configure.ac
@@ -197,6 +197,12 @@ PKG_STATIC(UUID_LIBS_STATIC, [uuid])
 PKG_CHECK_MODULES(ZLIB, [zlib])
 PKG_STATIC(ZLIB_LIBS_STATIC, [zlib])
 
+PKG_CHECK_MODULES(JSON, [json-c], [
+   AC_DEFINE(HAVE_JSON, [1], [Have JSON]),
+   PKG_STATIC(JSON_LIBS_STATIC, [json-c], [
+   AC_DEFINE(HAVE_JSON_STATIC, [1], [Have JSON static])], [true])
+   ], [true])
+
 AC_ARG_ENABLE([zstd],
AS_HELP_STRING([--disable-zstd], [build without zstd support]),
[], [enable_zstd=yes]
diff --git a/help.c b/help.c
index 063e9740..f1710621 100644
--- a/help.c
+++ b/help.c
@@ -32,6 +32,10 @@
 
 const char *cmd_outputs[CMD_OUTPUT_MAX] = {
"text",
+#ifdef HAVE_JSON
+   [CMD_OUTPUT_JSON] = "json",
+   [CMD_OUTPUT_JSON_COMPAT] = "json:compat",
+#endif
 };
 
 static char argv0_buf[ARGV0_BUF_SIZE] = "btrfs";
@@ -186,18 +190,17 @@ static int do_usage_one_command(const char * const 
*usagestr,
fprintf(outf, "%*s%s\n", pad, "", *usagestr++);
 
/* options (optional) */
-   if (!*usagestr || ((flags & USAGE_OPTIONS) == 0))
-   return 0;
-
-   /*
-* options (if present) should always (even if there is no long
-* description) be prepended with an empty line, skip it
-*/
-   usagestr++;
+   if (*usagestr && (flags & USAGE_OPTIONS)) {
+   /*
+* options (if present) should always (even if there is no long
+* description) be prepended with an empty line, skip it
+*/
+   usagestr++;
 
-   fputc('\n', outf);
-   while (*usagestr)
-   fprintf(outf, "%*s%s\n", pad, "", *usagestr++);
+   fputc('\n', outf);
+   while (*usagestr)
+   fprintf(outf, "%*s%s\n", pad, "", *usagestr++);
+   }
 
if (flags & USAGE_FORMAT) {
/* We always support text */
-- 
2.12.3

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 15/20] btrfs-progs: pass cmd_struct to clean_args_no_options{,_relaxed}

[jeffm]

From: Jeff Mahoney 

Now that we have a cmd_struct everywhere, we can pass it to
clean_args_no_options and have it resolve the usage string from
it there.  This is necessary for it to pass the cmd_struct to
usage() in the next patch.

Signed-off-by: Jeff Mahoney 
---
 cmds-balance.c|  6 +++---
 cmds-device.c |  9 +
 cmds-filesystem.c |  8 
 cmds-inspect.c|  4 ++--
 cmds-qgroup.c | 16 
 cmds-quota.c  |  4 ++--
 cmds-rescue.c |  4 ++--
 cmds-scrub.c  | 15 ---
 cmds-subvolume.c  |  6 +++---
 help.c|  9 +
 help.h|  7 ---
 11 files changed, 46 insertions(+), 42 deletions(-)

diff --git a/cmds-balance.c b/cmds-balance.c
index 1bd7b3ce..488fffcc 100644
--- a/cmds-balance.c
+++ b/cmds-balance.c
@@ -689,7 +689,7 @@ static int cmd_balance_pause(const struct cmd_struct *cmd,
int ret;
DIR *dirstream = NULL;
 
-   clean_args_no_options(argc, argv, cmd_balance_pause_usage);
+   clean_args_no_options(cmd, argc, argv);
 
if (check_argc_exact(argc - optind, 1))
usage(cmd_balance_pause_usage);
@@ -729,7 +729,7 @@ static int cmd_balance_cancel(const struct cmd_struct *cmd,
int ret;
DIR *dirstream = NULL;
 
-   clean_args_no_options(argc, argv, cmd_balance_cancel_usage);
+   clean_args_no_options(cmd, argc, argv);
 
if (check_argc_exact(argc - optind, 1))
usage(cmd_balance_cancel_usage);
@@ -770,7 +770,7 @@ static int cmd_balance_resume(const struct cmd_struct *cmd,
int fd;
int ret;
 
-   clean_args_no_options(argc, argv, cmd_balance_resume_usage);
+   clean_args_no_options(cmd, argc, argv);
 
if (check_argc_exact(argc - optind, 1))
usage(cmd_balance_resume_usage);
diff --git a/cmds-device.c b/cmds-device.c
index feb53f68..5be748f7 100644
--- a/cmds-device.c
+++ b/cmds-device.c
@@ -149,11 +149,12 @@ static int _cmd_device_remove(const struct cmd_struct 
*cmd,
char*mntpnt;
int i, fdmnt, ret = 0;
DIR *dirstream = NULL;
+   const char * const *usagestr = cmd->usagestr;
 
-   clean_args_no_options(argc, argv, cmd->usagestr);
+   clean_args_no_options(cmd, argc, argv);
 
if (check_argc_min(argc - optind, 2))
-   usage(cmd->usagestr);
+   usage(usagestr);
 
mntpnt = argv[argc - 1];
 
@@ -347,7 +348,7 @@ static int cmd_device_ready(const struct cmd_struct *cmd, 
int argc, char **argv)
int ret;
char*path;
 
-   clean_args_no_options(argc, argv, cmd->usagestr);
+   clean_args_no_options(cmd, argc, argv);
 
if (check_argc_exact(argc - optind, 1))
usage(cmd_device_ready_usage);
@@ -573,7 +574,7 @@ static int cmd_device_usage(const struct cmd_struct *cmd, 
int argc, char **argv)
 
unit_mode = get_unit_mode_from_arg(, argv, 1);
 
-   clean_args_no_options(argc, argv, cmd->usagestr);
+   clean_args_no_options(cmd, argc, argv);
 
if (check_argc_min(argc - optind, 1))
usage(cmd_device_usage_usage);
diff --git a/cmds-filesystem.c b/cmds-filesystem.c
index c2ee8595..b793532b 100644
--- a/cmds-filesystem.c
+++ b/cmds-filesystem.c
@@ -127,7 +127,7 @@ static int cmd_filesystem_df(const struct cmd_struct *cmd,
 
unit_mode = get_unit_mode_from_arg(, argv, 1);
 
-   clean_args_no_options(argc, argv, cmd_filesystem_df_usage);
+   clean_args_no_options(cmd, argc, argv);
 
if (check_argc_exact(argc - optind, 1))
usage(cmd_filesystem_df_usage);
@@ -822,7 +822,7 @@ static int cmd_filesystem_sync(const struct cmd_struct *cmd,
char*path;
DIR *dirstream = NULL;
 
-   clean_args_no_options(argc, argv, cmd_filesystem_sync_usage);
+   clean_args_no_options(cmd, argc, argv);
 
if (check_argc_exact(argc - optind, 1))
usage(cmd_filesystem_sync_usage);
@@ -1102,7 +1102,7 @@ static int cmd_filesystem_resize(const struct cmd_struct 
*cmd,
DIR *dirstream = NULL;
struct stat st;
 
-   clean_args_no_options_relaxed(argc, argv, cmd_filesystem_resize_usage);
+   clean_args_no_options_relaxed(cmd, argc, argv);
 
if (check_argc_exact(argc - optind, 2))
usage(cmd_filesystem_resize_usage);
@@ -1175,7 +1175,7 @@ static const char * const cmd_filesystem_label_usage[] = {
 static int cmd_filesystem_label(const struct cmd_struct *cmd,
int argc, char **argv)
 {
-   clean_args_no_options(argc, argv, cmd_filesystem_label_usage);
+   clean_args_no_options(cmd, argc, argv);
 
if (check_argc_min(argc - optind, 1) ||
check_argc_max(argc - optind, 2))
diff --git a/cmds-inspect.c b/cmds-inspect.c
index 1bdc8bd9..ece8c8d4 100644
--- a/cmds-inspect.c
+++ b/cmds-inspect.c
@@ -277,7 +277,7 @@ static int 

[PATCH v2 00/20] btrfs-progs: qgroups usability

[jeffm]

From: Jeff Mahoney 

Thanks to Qu Wenruo, Nikolay Borisov, and Tomohiro Misono for taking
the time to review my previous patchset.  I've incorporated your
suggestions into this version.

Obviously this one is quite a bit longer than the first version.  After
I posted it, Dave and I talked offline about whether it would make sense
to add the ability to output in JSON to other commands.  If it does,
and we agreed that it did, it would make sense for the choice of format
to be a global option.  In order to do that, I've had to rework some of
how we handle command definition and execution.  Mostly that is around
how to pass format flags and how to inform the user about which output
formats are available for each command.

So, here's the updated series:

* btrfs-progs: quota: Add -W option to rescan to wait without starting
  rescan
  - unchanged

* btrfs-progs: qgroups: fix misleading index check
  - unchanged

* btrfs-progs: constify pathnames passed as arguments
  - removed stray formatting change in send-utils.c

* btrfs-progs: btrfs-list: add rb_entry helpers for root_info
  - new patch, accessors for root_info rb_nodes

* btrfs-progs: btrfs-list: add btrfs_cleanup_root_info
  - new patch, adds a helper to clean up strings attached to root_info

* btrfs-progs: qgroups: add pathname to show output
  - Fixed help text to be more accurate
  - Fixed coding style issues
  - Added checks for NULL pathname
  - Free root_info strings after looking up pathname
  - Free pathname during teardown

* btrfs-progs: qgroups: introduce and use info and limit structures
  - edited to 80 columns

* btrfs-progs: qgroups: introduce btrfs_qgroup_query
  - Fixed issue with ENOENT vs ENOTTY
  - Added filter for search results
  - Use temporary key for search header
  - Cache passed search key for comparison since the loop modifies it

* btrfs-progs: subvolume: add quota info to btrfs sub show
  - Fixed/improved error reporting

* btrfs-progs: help: convert ints used as bools to bool
  - new patch

* btrfs-progs: reorder placement of help declarations for send/receive
  - new patch, required to remove usage declarations from commands.h

* btrfs-progs: filesystem balance: split out special handling
  - new patch, stop directly aliasing 'filesystem balance' to
'balance' -- it still does the right thing for normal execution
but help says "go read the balance help instead"

* btrfs-progs: use cmd_struct as command entry point
  - new patch, removes most command callback and usage declarations
  - replaces with cmd_struct declarations that are used in command
group arrays directly, similar to how sysfs attributes are
defined

* btrfs-progs: pass cmd_struct to command callback function
  - new patch, required to pass flags and have access to usage array

* btrfs-progs: pass cmd_struct to clean_args_no_options{,_relaxed}
  - new patch, required to pass cmd_struct to usage()

* btrfs-progs: pass cmd_struct to usage()
  - new patch, required to dynamically print what output formats
are available based on flags defined in command

* btrfs-progs: add support for output formats
  - new patch, adds infrastructure for output formats, including passing
caller context to commands

* btrfs-progs: add generic support for json output
  - new patch, split out JSON library detection from qgroups patch

* btrfs-progs: handle command groups directly for common case
  - new patch, remove most simple command group callbacks

* btrfs-progs: qgroups: add json output for usage command
  - remove -j and --compat-json options in favor of global --format json
or --format json:compat
  - added macro for qgroupid format buffer length
  - handle NULL pathnames better

-Jeff

Jeff Mahoney (20):
  btrfs-progs: quota: Add -W option to rescan to wait without starting
rescan
  btrfs-progs: qgroups: fix misleading index check
  btrfs-progs: constify pathnames passed as arguments
  btrfs-progs: btrfs-list: add rb_entry helpers for root_info
  btrfs-progs: btrfs-list: add btrfs_cleanup_root_info
  btrfs-progs: qgroups: add pathname to show output
  btrfs-progs: qgroups: introduce and use info and limit structures
  btrfs-progs: qgroups: introduce btrfs_qgroup_query
  btrfs-progs: subvolume: add quota info to btrfs sub show
  btrfs-progs: help: convert ints used as bools to bool
  btrfs-progs: reorder placement of help declarations for send/receive
  btrfs-progs: filesystem balance: split out special handling
  btrfs-progs: use cmd_struct as command entry point
  btrfs-progs: pass cmd_struct to command callback function
  btrfs-progs: pass cmd_struct to clean_args_no_options{,_relaxed}
  btrfs-progs: pass cmd_struct to usage()
  btrfs-progs: add support for output formats
  btrfs-progs: add generic support for json output
  btrfs-progs: qgroups: add json output for usage command
  btrfs-progs: handle command groups directly for common case

 Documentation/btrfs-qgroup.asciidoc |   7 +
 Documentation/btrfs-quota.asciidoc  |  

[PATCH 10/20] btrfs-progs: help: convert ints used as bools to bool

[jeffm]

From: Jeff Mahoney 

We use an int for 'full', 'all', and 'err' when we really mean a boolean.

Signed-off-by: Jeff Mahoney 
---
 btrfs.c | 14 +++---
 help.c  | 25 +
 help.h  |  4 ++--
 3 files changed, 22 insertions(+), 21 deletions(-)

diff --git a/btrfs.c b/btrfs.c
index 2d39f2ce..fec1a135 100644
--- a/btrfs.c
+++ b/btrfs.c
@@ -109,7 +109,7 @@ static void handle_help_options_next_level(const struct 
cmd_struct *cmd,
argv++;
help_command_group(cmd->next, argc, argv);
} else {
-   usage_command(cmd, 1, 0);
+   usage_command(cmd, true, false);
}
 
exit(0);
@@ -125,7 +125,7 @@ int handle_command_group(const struct cmd_group *grp, int 
argc,
argc--;
argv++;
if (argc < 1) {
-   usage_command_group(grp, 0, 0);
+   usage_command_group(grp, false, false);
exit(1);
}
 
@@ -212,20 +212,20 @@ static int handle_global_options(int argc, char **argv)
 
 void handle_special_globals(int shift, int argc, char **argv)
 {
-   int has_help = 0;
-   int has_full = 0;
+   bool has_help = false;
+   bool has_full = false;
int i;
 
for (i = 0; i < shift; i++) {
if (strcmp(argv[i], "--help") == 0)
-   has_help = 1;
+   has_help = true;
else if (strcmp(argv[i], "--full") == 0)
-   has_full = 1;
+   has_full = true;
}
 
if (has_help) {
if (has_full)
-   usage_command_group(_cmd_group, 1, 0);
+   usage_command_group(_cmd_group, true, false);
else
cmd_help(argc, argv);
exit(0);
diff --git a/help.c b/help.c
index 311a4320..ef7986b4 100644
--- a/help.c
+++ b/help.c
@@ -196,8 +196,8 @@ static int do_usage_one_command(const char * const 
*usagestr,
 }
 
 static int usage_command_internal(const char * const *usagestr,
- const char *token, int full, int lst,
- int alias, FILE *outf)
+ const char *token, bool full, bool lst,
+ bool alias, FILE *outf)
 {
unsigned int flags = 0;
int ret;
@@ -223,17 +223,17 @@ static int usage_command_internal(const char * const 
*usagestr,
 }
 
 static void usage_command_usagestr(const char * const *usagestr,
-  const char *token, int full, int err)
+  const char *token, bool full, bool err)
 {
FILE *outf = err ? stderr : stdout;
int ret;
 
-   ret = usage_command_internal(usagestr, token, full, 0, 0, outf);
+   ret = usage_command_internal(usagestr, token, full, false, false, outf);
if (!ret)
fputc('\n', outf);
 }
 
-void usage_command(const struct cmd_struct *cmd, int full, int err)
+void usage_command(const struct cmd_struct *cmd, bool full, bool err)
 {
usage_command_usagestr(cmd->usagestr, cmd->token, full, err);
 }
@@ -241,11 +241,11 @@ void usage_command(const struct cmd_struct *cmd, int 
full, int err)
 __attribute__((noreturn))
 void usage(const char * const *usagestr)
 {
-   usage_command_usagestr(usagestr, NULL, 1, 1);
+   usage_command_usagestr(usagestr, NULL, true, true);
exit(1);
 }
 
-static void usage_command_group_internal(const struct cmd_group *grp, int full,
+static void usage_command_group_internal(const struct cmd_group *grp, bool 
full,
 FILE *outf)
 {
const struct cmd_struct *cmd = grp->commands;
@@ -265,7 +265,8 @@ static void usage_command_group_internal(const struct 
cmd_group *grp, int full,
}
 
usage_command_internal(cmd->usagestr, cmd->token, full,
-  1, cmd->flags & CMD_ALIAS, outf);
+  true, cmd->flags & CMD_ALIAS,
+  outf);
if (cmd->flags & CMD_ALIAS)
putchar('\n');
continue;
@@ -327,7 +328,7 @@ void usage_command_group_short(const struct cmd_group *grp)
fprintf(stderr, "All command groups have their manual page named 
'btrfs-'.\n");
 }
 
-void usage_command_group(const struct cmd_group *grp, int full, int err)
+void usage_command_group(const struct cmd_group *grp, bool full, bool err)
 {
const char * const *usagestr = grp->usagestr;
FILE *outf = err ? stderr : stdout;
@@ -350,7 +351,7 @@ __attribute__((noreturn))
 void help_unknown_token(const char *arg, const struct cmd_group *grp)
 {
fprintf(stderr, "%s: unknown token '%s'\n", 

[PATCH 3/8] btrfs-progs: constify pathnames passed as arguments

[jeffm]

From: Jeff Mahoney 

It's unlikely we're going to modify a pathname argument, so codify that
and use const.

Signed-off-by: Jeff Mahoney 
---
 chunk-recover.c | 4 ++--
 cmds-device.c   | 2 +-
 cmds-fi-usage.c | 6 +++---
 cmds-rescue.c   | 4 ++--
 send-utils.c| 4 ++--
 5 files changed, 10 insertions(+), 10 deletions(-)

diff --git a/chunk-recover.c b/chunk-recover.c
index 705bcf52..1d30db51 100644
--- a/chunk-recover.c
+++ b/chunk-recover.c
@@ -1492,7 +1492,7 @@ out:
return ERR_PTR(ret);
 }
 
-static int recover_prepare(struct recover_control *rc, char *path)
+static int recover_prepare(struct recover_control *rc, const char *path)
 {
int ret;
int fd;
@@ -2296,7 +2296,7 @@ static void validate_rebuild_chunks(struct 
recover_control *rc)
 /*
  * Return 0 when successful, < 0 on error and > 0 if aborted by user
  */
-int btrfs_recover_chunk_tree(char *path, int verbose, int yes)
+int btrfs_recover_chunk_tree(const char *path, int verbose, int yes)
 {
int ret = 0;
struct btrfs_root *root = NULL;
diff --git a/cmds-device.c b/cmds-device.c
index 86459d1b..a49c9d9d 100644
--- a/cmds-device.c
+++ b/cmds-device.c
@@ -526,7 +526,7 @@ static const char * const cmd_device_usage_usage[] = {
NULL
 };
 
-static int _cmd_device_usage(int fd, char *path, unsigned unit_mode)
+static int _cmd_device_usage(int fd, const char *path, unsigned unit_mode)
 {
int i;
int ret = 0;
diff --git a/cmds-fi-usage.c b/cmds-fi-usage.c
index de7ad668..9a1c76ab 100644
--- a/cmds-fi-usage.c
+++ b/cmds-fi-usage.c
@@ -227,7 +227,7 @@ static int cmp_btrfs_ioctl_space_info(const void *a, const 
void *b)
 /*
  * This function load all the information about the space usage
  */
-static struct btrfs_ioctl_space_args *load_space_info(int fd, char *path)
+static struct btrfs_ioctl_space_args *load_space_info(int fd, const char *path)
 {
struct btrfs_ioctl_space_args *sargs = NULL, *sargs_orig = NULL;
int ret, count;
@@ -305,7 +305,7 @@ static void get_raid56_used(struct chunk_info *chunks, int 
chunkcount,
 #defineMIN_UNALOCATED_THRESH   SZ_16M
 static int print_filesystem_usage_overall(int fd, struct chunk_info *chunkinfo,
int chunkcount, struct device_info *devinfo, int devcount,
-   char *path, unsigned unit_mode)
+   const char *path, unsigned unit_mode)
 {
struct btrfs_ioctl_space_args *sargs = NULL;
int i;
@@ -931,7 +931,7 @@ static void _cmd_filesystem_usage_linear(unsigned unit_mode,
 static int print_filesystem_usage_by_chunk(int fd,
struct chunk_info *chunkinfo, int chunkcount,
struct device_info *devinfo, int devcount,
-   char *path, unsigned unit_mode, int tabular)
+   const char *path, unsigned unit_mode, int tabular)
 {
struct btrfs_ioctl_space_args *sargs;
int ret = 0;
diff --git a/cmds-rescue.c b/cmds-rescue.c
index c40088ad..c61145bc 100644
--- a/cmds-rescue.c
+++ b/cmds-rescue.c
@@ -32,8 +32,8 @@ static const char * const rescue_cmd_group_usage[] = {
NULL
 };
 
-int btrfs_recover_chunk_tree(char *path, int verbose, int yes);
-int btrfs_recover_superblocks(char *path, int verbose, int yes);
+int btrfs_recover_chunk_tree(const char *path, int verbose, int yes);
+int btrfs_recover_superblocks(const char *path, int verbose, int yes);
 
 static const char * const cmd_rescue_chunk_recover_usage[] = {
"btrfs rescue chunk-recover [options] ",
diff --git a/send-utils.c b/send-utils.c
index b5289e76..8ce94de1 100644
--- a/send-utils.c
+++ b/send-utils.c
@@ -28,8 +28,8 @@
 #include "ioctl.h"
 #include "btrfs-list.h"
 
-static int btrfs_subvolid_resolve_sub(int fd, char *path, size_t *path_len,
- u64 subvol_id);
+static int btrfs_subvolid_resolve_sub(int fd, char *path,
+ size_t *path_len, u64 subvol_id);
 
 static int btrfs_get_root_id_by_sub_path(int mnt_fd, const char *sub_path,
 u64 *root_id)
-- 
2.15.1

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 6/8] btrfs-progs: qgroups: introduce btrfs_qgroup_query

[jeffm]

From: Jeff Mahoney 

The only mechanism we have in the progs for searching qgroups is to load
all of them and filter the results.  This works for qgroup show but
to add quota information to 'btrfs subvoluem show' it's pretty wasteful.

This patch splits out setting up the search and performing the search so
we can search for a single qgroupid more easily.

Signed-off-by: Jeff Mahoney 
---
 qgroup.c | 98 +---
 qgroup.h |  7 +
 2 files changed, 77 insertions(+), 28 deletions(-)

diff --git a/qgroup.c b/qgroup.c
index b1be3311..2d0a6947 100644
--- a/qgroup.c
+++ b/qgroup.c
@@ -1146,11 +1146,11 @@ static inline void print_status_flag_warning(u64 flags)
warning("qgroup data inconsistent, rescan recommended");
 }
 
-static int __qgroups_search(int fd, struct qgroup_lookup *qgroup_lookup)
+static int __qgroups_search(int fd, struct btrfs_ioctl_search_args *args,
+   struct qgroup_lookup *qgroup_lookup)
 {
int ret;
-   struct btrfs_ioctl_search_args args;
-   struct btrfs_ioctl_search_key *sk = 
+   struct btrfs_ioctl_search_key *sk = >key;
struct btrfs_ioctl_search_header *sh;
unsigned long off = 0;
unsigned int i;
@@ -1161,30 +1161,12 @@ static int __qgroups_search(int fd, struct 
qgroup_lookup *qgroup_lookup)
u64 qgroupid;
u64 qgroupid1;
 
-   memset(, 0, sizeof(args));
-
-   sk->tree_id = BTRFS_QUOTA_TREE_OBJECTID;
-   sk->max_type = BTRFS_QGROUP_RELATION_KEY;
-   sk->min_type = BTRFS_QGROUP_STATUS_KEY;
-   sk->max_objectid = (u64)-1;
-   sk->max_offset = (u64)-1;
-   sk->max_transid = (u64)-1;
-   sk->nr_items = 4096;
-
qgroup_lookup_init(qgroup_lookup);
 
while (1) {
-   ret = ioctl(fd, BTRFS_IOC_TREE_SEARCH, );
+   ret = ioctl(fd, BTRFS_IOC_TREE_SEARCH, args);
if (ret < 0) {
-   if (errno == ENOENT) {
-   error("can't list qgroups: quotas not enabled");
-   ret = -ENOTTY;
-   } else {
-   error("can't list qgroups: %s",
-  strerror(errno));
-   ret = -errno;
-   }
-
+   ret = -errno;
break;
}
 
@@ -1198,14 +1180,14 @@ static int __qgroups_search(int fd, struct 
qgroup_lookup *qgroup_lookup)
 * read the root_ref item it contains
 */
for (i = 0; i < sk->nr_items; i++) {
-   sh = (struct btrfs_ioctl_search_header *)(args.buf +
+   sh = (struct btrfs_ioctl_search_header *)(args->buf +
  off);
off += sizeof(*sh);
 
switch (btrfs_search_header_type(sh)) {
case BTRFS_QGROUP_STATUS_KEY:
si = (struct btrfs_qgroup_status_item *)
-(args.buf + off);
+(args->buf + off);
flags = btrfs_stack_qgroup_status_flags(si);
 
print_status_flag_warning(flags);
@@ -1213,7 +1195,7 @@ static int __qgroups_search(int fd, struct qgroup_lookup 
*qgroup_lookup)
case BTRFS_QGROUP_INFO_KEY:
qgroupid = btrfs_search_header_offset(sh);
info = (struct btrfs_qgroup_info_item *)
-  (args.buf + off);
+  (args->buf + off);
 
ret = update_qgroup_info(fd, qgroup_lookup,
 qgroupid, info);
@@ -1221,7 +1203,7 @@ static int __qgroups_search(int fd, struct qgroup_lookup 
*qgroup_lookup)
case BTRFS_QGROUP_LIMIT_KEY:
qgroupid = btrfs_search_header_offset(sh);
limit = (struct btrfs_qgroup_limit_item *)
-   (args.buf + off);
+   (args->buf + off);
 
ret = update_qgroup_limit(fd, qgroup_lookup,
  qgroupid, limit);
@@ -1267,6 +1249,66 @@ static int __qgroups_search(int fd, struct qgroup_lookup 
*qgroup_lookup)
return ret;
 }
 
+static int qgroups_search_all(int fd, struct qgroup_lookup *qgroup_lookup)
+{
+   struct btrfs_ioctl_search_args args = {
+   .key = {
+   .tree_id = BTRFS_QUOTA_TREE_OBJECTID,
+   .max_type = BTRFS_QGROUP_RELATION_KEY,
+   .min_type = 

[PATCH 8/8] btrfs-progs: qgroups: export qgroups usage information as JSON

[jeffm]

From: Jeff Mahoney 

One of the common requests I receive is for 'df' like facilities
for subvolume usage.  Really, the request is for monitoring tools to be
able to understand when subvolumes may be approaching quota in the same
manner traditional file systems approach ENOSPC.

This patch allows us to export the qgroups data in a machine-readable
format so that monitoring tools can parse it easily.

There are two modes since JSON can technically handle 64-bit numbers
but JavaScript proper cannot.  show -j enables JSON mode using 64-bit
integers directly.  --json-compat presents 64-bit numbers as an array
of two 32-bit numbers (high followed by low).

Signed-off-by: Jeff Mahoney 
---
 Documentation/btrfs-qgroup.asciidoc |   4 +
 Makefile.inc.in |   4 +-
 cmds-qgroup.c   |  36 +-
 configure.ac|   6 +
 qgroup.c| 211 
 qgroup.h|   3 +
 6 files changed, 258 insertions(+), 6 deletions(-)

diff --git a/Documentation/btrfs-qgroup.asciidoc 
b/Documentation/btrfs-qgroup.asciidoc
index 360b3269..22a9c2a7 100644
--- a/Documentation/btrfs-qgroup.asciidoc
+++ b/Documentation/btrfs-qgroup.asciidoc
@@ -105,6 +105,10 @@ list all qgroups which impact the given path(include 
ancestral qgroups)
 list all qgroups which impact the given path(exclude ancestral qgroups)
 -v
 Be more verbose.  Print pathnames of member qgroups when nested.
+-j
+If enabled, export qgroup usage information in JSON format.  This implies 
--raw.
+--json-compat
+By default, JSON output contains full 64-bit integers, which may be 
incompatible with some JSON parsers.  This option exports those values as an 
array of 32-bit numbers in [high, low] format.
 --raw
 raw numbers in bytes, without the 'B' suffix.
 --human-readable
diff --git a/Makefile.inc.in b/Makefile.inc.in
index 56271903..68bddbed 100644
--- a/Makefile.inc.in
+++ b/Makefile.inc.in
@@ -18,9 +18,9 @@ BTRFSRESTORE_ZSTD = @BTRFSRESTORE_ZSTD@
 SUBST_CFLAGS = @CFLAGS@
 SUBST_LDFLAGS = @LDFLAGS@
 
-LIBS_BASE = @UUID_LIBS@ @BLKID_LIBS@ -L. -pthread
+LIBS_BASE = @UUID_LIBS@ @BLKID_LIBS@ @JSON_LIBS@ -L. -pthread
 LIBS_COMP = @ZLIB_LIBS@ @LZO2_LIBS@ @ZSTD_LIBS@
-STATIC_LIBS_BASE = @UUID_LIBS_STATIC@ @BLKID_LIBS_STATIC@ -L. -pthread
+STATIC_LIBS_BASE = @UUID_LIBS_STATIC@ @BLKID_LIBS_STATIC@ @JSON_LIBS_STATIC@ 
-L. -pthread
 STATIC_LIBS_COMP = @ZLIB_LIBS_STATIC@ @LZO2_LIBS_STATIC@ @ZSTD_LIBS_STATIC@
 
 prefix ?= @prefix@
diff --git a/cmds-qgroup.c b/cmds-qgroup.c
index 94cd0fd3..eee15ef1 100644
--- a/cmds-qgroup.c
+++ b/cmds-qgroup.c
@@ -282,6 +282,10 @@ static const char * const cmd_qgroup_show_usage[] = {
"   (excluding ancestral qgroups)",
"-P print first-level qgroups using pathname",
"-v verbose, prints all nested subvolumes",
+#ifdef HAVE_JSON
+   "-j export in JSON format",
+   "--json-compat  export in JSON compatibility mode",
+#endif
HELPINFO_UNITS_LONG,
"--sort=qgroupid,rfer,excl,max_rfer,max_excl,pathname",
"   list qgroups sorted by specified items",
@@ -302,6 +306,8 @@ static int cmd_qgroup_show(int argc, char **argv)
unsigned unit_mode;
int sync = 0;
bool verbose = false;
+   bool export_json = false;
+   bool compat_json = false;
 
struct btrfs_qgroup_comparer_set *comparer_set;
struct btrfs_qgroup_filter_set *filter_set;
@@ -314,16 +320,26 @@ static int cmd_qgroup_show(int argc, char **argv)
int c;
enum {
GETOPT_VAL_SORT = 256,
-   GETOPT_VAL_SYNC
+   GETOPT_VAL_SYNC,
+   GETOPT_VAL_JSCOMPAT,
};
static const struct option long_options[] = {
{"sort", required_argument, NULL, GETOPT_VAL_SORT},
{"sync", no_argument, NULL, GETOPT_VAL_SYNC},
{"verbose", no_argument, NULL, 'v'},
+#ifdef HAVE_JSON
+   {"json-compat", no_argument, NULL, GETOPT_VAL_JSCOMPAT},
+#endif
{ NULL, 0, NULL, 0 }
};
-
-   c = getopt_long(argc, argv, "pPcreFfv", long_options, NULL);
+   const char getopt_chars[] = {
+   'p', 'P', 'c', 'r', 'e', 'F', 'f', 'v',
+#ifdef HAVE_JSON
+   'j',
+#endif
+   '\0' };
+
+   c = getopt_long(argc, argv, getopt_chars, long_options, NULL);
if (c < 0)
break;
switch (c) {
@@ -353,6 +369,14 @@ static int cmd_qgroup_show(int argc, char **argv)
case 'f':
filter_flag |= 0x2;
break;
+#ifdef HAVE_JSON
+   case 

[PATCH 5/8] btrfs-progs: qgroups: introduce and use info and limit structures

[jeffm]

From: Jeff Mahoney 

We use structures to pass the info and limit from the kernel as items
but store the individual values separately in btrfs_qgroup.  We already
have a btrfs_qgroup_limit structure that's used for setting the limit.

This patch introduces a btrfs_qgroup_info structure and uses that and
btrfs_qgroup_limit in btrfs_qgroup.

Signed-off-by: Jeff Mahoney 
---
 qgroup.c | 73 +++-
 qgroup.h |  8 +++
 2 files changed, 43 insertions(+), 38 deletions(-)

diff --git a/qgroup.c b/qgroup.c
index 83918134..b1be3311 100644
--- a/qgroup.c
+++ b/qgroup.c
@@ -46,20 +46,12 @@ struct btrfs_qgroup {
/*
 * info_item
 */
-   u64 generation;
-   u64 rfer;   /*referenced*/
-   u64 rfer_cmpr;  /*referenced compressed*/
-   u64 excl;   /*exclusive*/
-   u64 excl_cmpr;  /*exclusive compressed*/
+   struct btrfs_qgroup_info info;
 
/*
 *limit_item
 */
-   u64 flags;  /*which limits are set*/
-   u64 max_rfer;
-   u64 max_excl;
-   u64 rsv_rfer;
-   u64 rsv_excl;
+   struct btrfs_qgroup_limit limit;
 
/*qgroups this group is member of*/
struct list_head qgroups;
@@ -272,24 +264,24 @@ static void print_qgroup_column(struct btrfs_qgroup 
*qgroup,
print_qgroup_column_add_blank(BTRFS_QGROUP_QGROUPID, len);
break;
case BTRFS_QGROUP_RFER:
-   len = printf("%*s", max_len, pretty_size_mode(qgroup->rfer, 
unit_mode));
+   len = printf("%*s", max_len, 
pretty_size_mode(qgroup->info.referenced, unit_mode));
break;
case BTRFS_QGROUP_EXCL:
-   len = printf("%*s", max_len, pretty_size_mode(qgroup->excl, 
unit_mode));
+   len = printf("%*s", max_len, 
pretty_size_mode(qgroup->info.exclusive, unit_mode));
break;
case BTRFS_QGROUP_PARENT:
len = print_parent_column(qgroup);
print_qgroup_column_add_blank(BTRFS_QGROUP_PARENT, len);
break;
case BTRFS_QGROUP_MAX_RFER:
-   if (qgroup->flags & BTRFS_QGROUP_LIMIT_MAX_RFER)
-   len = printf("%*s", max_len, 
pretty_size_mode(qgroup->max_rfer, unit_mode));
+   if (qgroup->limit.flags & BTRFS_QGROUP_LIMIT_MAX_RFER)
+   len = printf("%*s", max_len, 
pretty_size_mode(qgroup->limit.max_referenced, unit_mode));
else
len = printf("%*s", max_len, "none");
break;
case BTRFS_QGROUP_MAX_EXCL:
-   if (qgroup->flags & BTRFS_QGROUP_LIMIT_MAX_EXCL)
-   len = printf("%*s", max_len, 
pretty_size_mode(qgroup->max_excl, unit_mode));
+   if (qgroup->limit.flags & BTRFS_QGROUP_LIMIT_MAX_EXCL)
+   len = printf("%*s", max_len, 
pretty_size_mode(qgroup->limit.max_exclusive, unit_mode));
else
len = printf("%*s", max_len, "none");
break;
@@ -432,9 +424,9 @@ static int comp_entry_with_rfer(struct btrfs_qgroup *entry1,
 {
int ret;
 
-   if (entry1->rfer > entry2->rfer)
+   if (entry1->info.referenced > entry2->info.referenced)
ret = 1;
-   else if (entry1->rfer < entry2->rfer)
+   else if (entry1->info.referenced < entry2->info.referenced)
ret = -1;
else
ret = 0;
@@ -448,9 +440,9 @@ static int comp_entry_with_excl(struct btrfs_qgroup *entry1,
 {
int ret;
 
-   if (entry1->excl > entry2->excl)
+   if (entry1->info.exclusive > entry2->info.exclusive)
ret = 1;
-   else if (entry1->excl < entry2->excl)
+   else if (entry1->info.exclusive < entry2->info.exclusive)
ret = -1;
else
ret = 0;
@@ -464,9 +456,9 @@ static int comp_entry_with_max_rfer(struct btrfs_qgroup 
*entry1,
 {
int ret;
 
-   if (entry1->max_rfer > entry2->max_rfer)
+   if (entry1->limit.max_referenced > entry2->limit.max_referenced)
ret = 1;
-   else if (entry1->max_rfer < entry2->max_rfer)
+   else if (entry1->limit.max_referenced < entry2->limit.max_referenced)
ret = -1;
else
ret = 0;
@@ -480,9 +472,9 @@ static int comp_entry_with_max_excl(struct btrfs_qgroup 
*entry1,
 {
int ret;
 
-   if (entry1->max_excl > entry2->max_excl)
+   if (entry1->limit.max_exclusive > entry2->limit.max_exclusive)
ret = 1;
-   else if (entry1->max_excl < entry2->max_excl)
+   else if (entry1->limit.max_exclusive < entry2->limit.max_exclusive)
ret = -1;
else
ret = 0;
@@ -739,11 +731,13 @@ static int update_qgroup_info(int fd, struct 
qgroup_lookup *qgroup_lookup,
if (IS_ERR_OR_NULL(bq))
return 

[PATCH 7/8] btrfs-progs: subvolume: add quota info to btrfs sub show

[jeffm]

From: Jeff Mahoney 

This patch reports on the first-level qgroup, if any, associated with
a particular subvolume.  It displays the usage and limit, subject
to the usual unit parameters.

Signed-off-by: Jeff Mahoney 
---
 cmds-subvolume.c | 46 ++
 1 file changed, 46 insertions(+)

diff --git a/cmds-subvolume.c b/cmds-subvolume.c
index 8a473f7a..29d0e0e5 100644
--- a/cmds-subvolume.c
+++ b/cmds-subvolume.c
@@ -972,6 +972,7 @@ static const char * const cmd_subvol_show_usage[] = {
"Show more information about the subvolume",
"-r|--rootid   rootid of the subvolume",
"-u|--uuid uuid of the subvolume",
+   HELPINFO_UNITS_SHORT_LONG,
"",
"If no option is specified,  will be shown, otherwise",
"the rootid or uuid are resolved relative to the  path.",
@@ -993,6 +994,13 @@ static int cmd_subvol_show(int argc, char **argv)
int by_uuid = 0;
u64 rootid_arg;
u8 uuid_arg[BTRFS_UUID_SIZE];
+   struct btrfs_qgroup_stats stats;
+   unsigned int unit_mode;
+   const char *referenced_size;
+   const char *referenced_limit_size = "-";
+   unsigned field_width = 0;
+
+   unit_mode = get_unit_mode_from_arg(, argv, 1);
 
while (1) {
int c;
@@ -1112,6 +1120,44 @@ static int cmd_subvol_show(int argc, char **argv)
btrfs_list_subvols_print(fd, filter_set, NULL, BTRFS_LIST_LAYOUT_RAW,
1, raw_prefix);
 
+   ret = btrfs_qgroup_query(fd, get_ri.root_id, );
+   if (ret < 0) {
+   if (ret == -ENODATA)
+   printf("Quotas must be enabled for per-subvolume 
usage\n");
+   else if (ret != -ENOTTY)
+   fprintf(stderr,
+   "\nERROR: BTRFS_IOC_QUOTA_QUERY failed: %s\n",
+   strerror(errno));
+   goto out;
+   }
+
+   printf("\tQuota Usage:\t\t");
+   fflush(stdout);
+
+   referenced_size = pretty_size_mode(stats.info.referenced, unit_mode);
+   if (stats.limit.max_referenced)
+  referenced_limit_size = pretty_size_mode(
+   stats.limit.max_referenced,
+   unit_mode);
+   field_width = max(strlen(referenced_size),
+ strlen(referenced_limit_size));
+
+   printf("%-*s referenced, %s exclusive\n ", field_width,
+  referenced_size,
+  pretty_size_mode(stats.info.exclusive, unit_mode));
+
+   printf("\tQuota Limits:\t\t");
+   if (stats.limit.max_referenced || stats.limit.max_exclusive) {
+   const char *excl = "-";
+
+   if (stats.limit.max_exclusive)
+  excl = pretty_size_mode(stats.limit.max_exclusive,
+  unit_mode);
+   printf("%-*s referenced, %s exclusive\n", field_width,
+  referenced_limit_size, excl);
+   } else
+   printf("None\n");
+
 out:
/* clean up */
free(get_ri.path);
-- 
2.15.1

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 4/8] btrfs-progs: qgroups: add pathname to show output

[jeffm]

From: Jeff Mahoney 

The btrfs qgroup show command currently only exports qgroup IDs,
forcing the user to resolve which subvolume each corresponds to.

This patch adds pathname resolution to qgroup show so that when
the -P option is used, the last column contains the pathname of
the root of the subvolume it describes.  In the case of nested
qgroups, it will show the number of member qgroups or the paths
of the members if the -v option is used.

Pathname can also be used as a sort parameter.

Signed-off-by: Jeff Mahoney 
---
 Documentation/btrfs-qgroup.asciidoc |   4 +
 cmds-qgroup.c   |  17 -
 kerncompat.h|   1 +
 qgroup.c| 142 
 qgroup.h|   4 +-
 utils.c |  22 --
 utils.h |   2 +
 7 files changed, 166 insertions(+), 26 deletions(-)

diff --git a/Documentation/btrfs-qgroup.asciidoc 
b/Documentation/btrfs-qgroup.asciidoc
index 3108457c..360b3269 100644
--- a/Documentation/btrfs-qgroup.asciidoc
+++ b/Documentation/btrfs-qgroup.asciidoc
@@ -97,10 +97,14 @@ print child qgroup id.
 print limit of referenced size of qgroup.
 -e
 print limit of exclusive size of qgroup.
+-P
+print pathname to the root of the subvolume managed by qgroup.  For nested 
qgroups, the number of members will be printed unless -v is specified.
 -F
 list all qgroups which impact the given path(include ancestral qgroups)
 -f
 list all qgroups which impact the given path(exclude ancestral qgroups)
+-v
+Be more verbose.  Print pathnames of member qgroups when nested.
 --raw
 raw numbers in bytes, without the 'B' suffix.
 --human-readable
diff --git a/cmds-qgroup.c b/cmds-qgroup.c
index 48686436..94cd0fd3 100644
--- a/cmds-qgroup.c
+++ b/cmds-qgroup.c
@@ -280,8 +280,10 @@ static const char * const cmd_qgroup_show_usage[] = {
"   (including ancestral qgroups)",
"-f list all qgroups which impact the given path",
"   (excluding ancestral qgroups)",
+   "-P print first-level qgroups using pathname",
+   "-v verbose, prints all nested subvolumes",
HELPINFO_UNITS_LONG,
-   "--sort=qgroupid,rfer,excl,max_rfer,max_excl",
+   "--sort=qgroupid,rfer,excl,max_rfer,max_excl,pathname",
"   list qgroups sorted by specified items",
"   you can use '+' or '-' in front of each item.",
"   (+:ascending, -:descending, ascending default)",
@@ -299,6 +301,7 @@ static int cmd_qgroup_show(int argc, char **argv)
int filter_flag = 0;
unsigned unit_mode;
int sync = 0;
+   bool verbose = false;
 
struct btrfs_qgroup_comparer_set *comparer_set;
struct btrfs_qgroup_filter_set *filter_set;
@@ -316,10 +319,11 @@ static int cmd_qgroup_show(int argc, char **argv)
static const struct option long_options[] = {
{"sort", required_argument, NULL, GETOPT_VAL_SORT},
{"sync", no_argument, NULL, GETOPT_VAL_SYNC},
+   {"verbose", no_argument, NULL, 'v'},
{ NULL, 0, NULL, 0 }
};
 
-   c = getopt_long(argc, argv, "pcreFf", long_options, NULL);
+   c = getopt_long(argc, argv, "pPcreFfv", long_options, NULL);
if (c < 0)
break;
switch (c) {
@@ -327,6 +331,10 @@ static int cmd_qgroup_show(int argc, char **argv)
btrfs_qgroup_setup_print_column(
BTRFS_QGROUP_PARENT);
break;
+   case 'P':
+   btrfs_qgroup_setup_print_column(
+   BTRFS_QGROUP_PATHNAME);
+   break;
case 'c':
btrfs_qgroup_setup_print_column(
BTRFS_QGROUP_CHILD);
@@ -354,6 +362,9 @@ static int cmd_qgroup_show(int argc, char **argv)
case GETOPT_VAL_SYNC:
sync = 1;
break;
+   case 'v':
+   verbose = true;
+   break;
default:
usage(cmd_qgroup_show_usage);
}
@@ -394,7 +405,7 @@ static int cmd_qgroup_show(int argc, char **argv)
BTRFS_QGROUP_FILTER_PARENT,
qgroupid);
}
-   ret = btrfs_show_qgroups(fd, filter_set, comparer_set);
+   ret = btrfs_show_qgroups(fd, filter_set, comparer_set, verbose);
close_file_or_dir(fd, dirstream);
free(filter_set);
free(comparer_set);
diff --git a/kerncompat.h b/kerncompat.h
index fa96715f..f97495ee 100644
--- a/kerncompat.h
+++ 

[PATCH 2/8] btrfs-progs: qgroups: fix misleading index check

[jeffm]

From: Jeff Mahoney 

In print_single_qgroup_table we check the loop index against
BTRFS_QGROUP_CHILD, but what we really mean is "last column."  Since
we have an enum value to indicate the last value, use that instead
of assuming that BTRFS_QGROUP_CHILD is always last.

Signed-off-by: Jeff Mahoney 
---
 qgroup.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/qgroup.c b/qgroup.c
index 11659e83..67bc0738 100644
--- a/qgroup.c
+++ b/qgroup.c
@@ -267,7 +267,7 @@ static void print_single_qgroup_table(struct btrfs_qgroup 
*qgroup)
continue;
print_qgroup_column(qgroup, i);
 
-   if (i != BTRFS_QGROUP_CHILD)
+   if (i != BTRFS_QGROUP_ALL - 1)
printf(" ");
}
printf("\n");
-- 
2.15.1

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 1/8] btrfs-progs: quota: Add -W option to rescan to wait without starting rescan

[jeffm]

From: Jeff Mahoney 

This patch adds a new -W option to wait for a rescan without starting a
new operation.  This is useful for things like xfstests where we want
do to do a "btrfs quota enable" and not continue until the subsequent
rescan has finished.

In addition to documenting the new option in the man page, I've cleaned
up the rescan entry to document the -w option a bit better.

Signed-off-by: Jeff Mahoney 
---
 Documentation/btrfs-quota.asciidoc | 10 +++---
 cmds-quota.c   | 21 +++--
 2 files changed, 22 insertions(+), 9 deletions(-)

diff --git a/Documentation/btrfs-quota.asciidoc 
b/Documentation/btrfs-quota.asciidoc
index 85ebf729..0b64a69b 100644
--- a/Documentation/btrfs-quota.asciidoc
+++ b/Documentation/btrfs-quota.asciidoc
@@ -238,15 +238,19 @@ Disable subvolume quota support for a filesystem.
 *enable* ::
 Enable subvolume quota support for a filesystem.
 
-*rescan* [-s] ::
+*rescan* [-s|-w|-W] ::
 Trash all qgroup numbers and scan the metadata again with the current config.
 +
 `Options`
 +
 -s
-show status of a running rescan operation.
+Show status of a running rescan operation.
+
 -w
-wait for rescan operation to finish(can be already in progress).
+Start rescan operation and wait until it has finished before exiting.  If a 
rescan is already running, wait until it finishes and then exit without 
starting a new one.
+
+-W
+Wait for rescan operation to finish and then exit.  If a rescan is not already 
running, exit silently.
 
 EXIT STATUS
 ---
diff --git a/cmds-quota.c b/cmds-quota.c
index 745889d1..fe6376ac 100644
--- a/cmds-quota.c
+++ b/cmds-quota.c
@@ -120,14 +120,20 @@ static int cmd_quota_rescan(int argc, char **argv)
int wait_for_completion = 0;
 
while (1) {
-   int c = getopt(argc, argv, "sw");
+   int c = getopt(argc, argv, "swW");
if (c < 0)
break;
switch (c) {
case 's':
ioctlnum = BTRFS_IOC_QUOTA_RESCAN_STATUS;
break;
+   case 'W':
+   ioctlnum = 0;
+   wait_for_completion = 1;
+   break;
case 'w':
+   /* Reset it in case the user did both -W and -w */
+   ioctlnum = BTRFS_IOC_QUOTA_RESCAN;
wait_for_completion = 1;
break;
default:
@@ -135,8 +141,9 @@ static int cmd_quota_rescan(int argc, char **argv)
}
}
 
-   if (ioctlnum != BTRFS_IOC_QUOTA_RESCAN && wait_for_completion) {
-   error("switch -w cannot be used with -s");
+   if (ioctlnum == BTRFS_IOC_QUOTA_RESCAN_STATUS && wait_for_completion) {
+   error("switch -%c cannot be used with -s",
+ ioctlnum ? 'w' : 'W');
return 1;
}
 
@@ -150,8 +157,10 @@ static int cmd_quota_rescan(int argc, char **argv)
if (fd < 0)
return 1;
 
-   ret = ioctl(fd, ioctlnum, );
-   e = errno;
+   if (ioctlnum) {
+   ret = ioctl(fd, ioctlnum, );
+   e = errno;
+   }
 
if (ioctlnum == BTRFS_IOC_QUOTA_RESCAN_STATUS) {
close_file_or_dir(fd, dirstream);
@@ -167,7 +176,7 @@ static int cmd_quota_rescan(int argc, char **argv)
return 0;
}
 
-   if (ret == 0) {
+   if (ioctlnum == BTRFS_IOC_QUOTA_RESCAN && ret == 0) {
printf("quota rescan started\n");
fflush(stdout);
} else if (ret < 0 && (!wait_for_completion || e != EINPROGRESS)) {
-- 
2.15.1

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 0/8] btrfs-progs: qgroups usability [corrected]

[jeffm]

From: Jeff Mahoney 

Hi all -

The following series addresses some usability issues with the qgroups UI.

1) Adds -W option so we can wait on a rescan completing without starting one.
2) Adds qgroup information to 'btrfs subvolume show'
3) Adds a -P option to show pathnames for first-level qgroups (or member
   of nested qgroups with -v)
4) Allows exporting the qgroup table in JSON format for use by external
   programs/scripts.

-Jeff

Jeff Mahoney (8):
  btrfs-progs: quota: Add -W option to rescan to wait without starting
rescan
  btrfs-progs: qgroups: fix misleading index check
  btrfs-progs: constify pathnames passed as arguments
  btrfs-progs: qgroups: add pathname to show output
  btrfs-progs: qgroups: introduce and use info and limit structures
  btrfs-progs: qgroups: introduce btrfs_qgroup_query
  btrfs-progs: subvolume: add quota info to btrfs sub show
  btrfs-progs: qgroups: export qgroups usage information as JSON

 Documentation/btrfs-qgroup.asciidoc |   8 +
 Documentation/btrfs-quota.asciidoc  |  10 +-
 Makefile.inc.in |   4 +-
 chunk-recover.c |   4 +-
 cmds-device.c   |   2 +-
 cmds-fi-usage.c |   6 +-
 cmds-qgroup.c   |  49 +++-
 cmds-quota.c|  21 +-
 cmds-rescue.c   |   4 +-
 cmds-subvolume.c|  46 
 configure.ac|   6 +
 kerncompat.h|   1 +
 qgroup.c| 526 ++--
 qgroup.h|  22 +-
 send-utils.c|   4 +-
 utils.c |  22 +-
 utils.h |   2 +
 17 files changed, 621 insertions(+), 116 deletions(-)

-- 
2.15.1

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 4/8] btrfs-progs: qgroups: add pathname to show output

[jeffm]

From: Jeff Mahoney 

The btrfs qgroup show command currently only exports qgroup IDs,
forcing the user to resolve which subvolume each corresponds to.

This patch adds pathname resolution to qgroup show so that when
the -P option is used, the last column contains the pathname of
the root of the subvolume it describes.  In the case of nested
qgroups, it will show the number of member qgroups or the paths
of the members if the -v option is used.

Pathname can also be used as a sort parameter.

Signed-off-by: Jeff Mahoney 
---
 Documentation/btrfs-qgroup.asciidoc |   4 +
 cmds-qgroup.c   |  17 -
 kerncompat.h|   1 +
 qgroup.c| 142 
 qgroup.h|   4 +-
 utils.c |  22 --
 utils.h |   2 +
 7 files changed, 166 insertions(+), 26 deletions(-)

diff --git a/Documentation/btrfs-qgroup.asciidoc 
b/Documentation/btrfs-qgroup.asciidoc
index 3108457c..360b3269 100644
--- a/Documentation/btrfs-qgroup.asciidoc
+++ b/Documentation/btrfs-qgroup.asciidoc
@@ -97,10 +97,14 @@ print child qgroup id.
 print limit of referenced size of qgroup.
 -e
 print limit of exclusive size of qgroup.
+-P
+print pathname to the root of the subvolume managed by qgroup.  For nested 
qgroups, the number of members will be printed unless -v is specified.
 -F
 list all qgroups which impact the given path(include ancestral qgroups)
 -f
 list all qgroups which impact the given path(exclude ancestral qgroups)
+-v
+Be more verbose.  Print pathnames of member qgroups when nested.
 --raw
 raw numbers in bytes, without the 'B' suffix.
 --human-readable
diff --git a/cmds-qgroup.c b/cmds-qgroup.c
index 48686436..94cd0fd3 100644
--- a/cmds-qgroup.c
+++ b/cmds-qgroup.c
@@ -280,8 +280,10 @@ static const char * const cmd_qgroup_show_usage[] = {
"   (including ancestral qgroups)",
"-f list all qgroups which impact the given path",
"   (excluding ancestral qgroups)",
+   "-P print first-level qgroups using pathname",
+   "-v verbose, prints all nested subvolumes",
HELPINFO_UNITS_LONG,
-   "--sort=qgroupid,rfer,excl,max_rfer,max_excl",
+   "--sort=qgroupid,rfer,excl,max_rfer,max_excl,pathname",
"   list qgroups sorted by specified items",
"   you can use '+' or '-' in front of each item.",
"   (+:ascending, -:descending, ascending default)",
@@ -299,6 +301,7 @@ static int cmd_qgroup_show(int argc, char **argv)
int filter_flag = 0;
unsigned unit_mode;
int sync = 0;
+   bool verbose = false;
 
struct btrfs_qgroup_comparer_set *comparer_set;
struct btrfs_qgroup_filter_set *filter_set;
@@ -316,10 +319,11 @@ static int cmd_qgroup_show(int argc, char **argv)
static const struct option long_options[] = {
{"sort", required_argument, NULL, GETOPT_VAL_SORT},
{"sync", no_argument, NULL, GETOPT_VAL_SYNC},
+   {"verbose", no_argument, NULL, 'v'},
{ NULL, 0, NULL, 0 }
};
 
-   c = getopt_long(argc, argv, "pcreFf", long_options, NULL);
+   c = getopt_long(argc, argv, "pPcreFfv", long_options, NULL);
if (c < 0)
break;
switch (c) {
@@ -327,6 +331,10 @@ static int cmd_qgroup_show(int argc, char **argv)
btrfs_qgroup_setup_print_column(
BTRFS_QGROUP_PARENT);
break;
+   case 'P':
+   btrfs_qgroup_setup_print_column(
+   BTRFS_QGROUP_PATHNAME);
+   break;
case 'c':
btrfs_qgroup_setup_print_column(
BTRFS_QGROUP_CHILD);
@@ -354,6 +362,9 @@ static int cmd_qgroup_show(int argc, char **argv)
case GETOPT_VAL_SYNC:
sync = 1;
break;
+   case 'v':
+   verbose = true;
+   break;
default:
usage(cmd_qgroup_show_usage);
}
@@ -394,7 +405,7 @@ static int cmd_qgroup_show(int argc, char **argv)
BTRFS_QGROUP_FILTER_PARENT,
qgroupid);
}
-   ret = btrfs_show_qgroups(fd, filter_set, comparer_set);
+   ret = btrfs_show_qgroups(fd, filter_set, comparer_set, verbose);
close_file_or_dir(fd, dirstream);
free(filter_set);
free(comparer_set);
diff --git a/kerncompat.h b/kerncompat.h
index fa96715f..f97495ee 100644
--- a/kerncompat.h
+++ 

[PATCH 8/8] btrfs-progs: qgroups: export qgroups usage information as JSON

[jeffm]

From: Jeff Mahoney 

One of the common requests I receive is for 'df' like facilities
for subvolume usage.  Really, the request is for monitoring tools to be
able to understand when subvolumes may be approaching quota in the same
manner traditional file systems approach ENOSPC.

This patch allows us to export the qgroups data in a machine-readable
format so that monitoring tools can parse it easily.

There are two modes since JSON can technically handle 64-bit numbers
but JavaScript proper cannot.  show -j enables JSON mode using 64-bit
integers directly.  --json-compat presents 64-bit numbers as an array
of two 32-bit numbers (high followed by low).

Signed-off-by: Jeff Mahoney 
---
 Documentation/btrfs-qgroup.asciidoc |   4 +
 Makefile.inc.in |   4 +-
 cmds-qgroup.c   |  36 +-
 configure.ac|   6 +
 qgroup.c| 211 
 qgroup.h|   3 +
 6 files changed, 258 insertions(+), 6 deletions(-)

diff --git a/Documentation/btrfs-qgroup.asciidoc 
b/Documentation/btrfs-qgroup.asciidoc
index 360b3269..22a9c2a7 100644
--- a/Documentation/btrfs-qgroup.asciidoc
+++ b/Documentation/btrfs-qgroup.asciidoc
@@ -105,6 +105,10 @@ list all qgroups which impact the given path(include 
ancestral qgroups)
 list all qgroups which impact the given path(exclude ancestral qgroups)
 -v
 Be more verbose.  Print pathnames of member qgroups when nested.
+-j
+If enabled, export qgroup usage information in JSON format.  This implies 
--raw.
+--json-compat
+By default, JSON output contains full 64-bit integers, which may be 
incompatible with some JSON parsers.  This option exports those values as an 
array of 32-bit numbers in [high, low] format.
 --raw
 raw numbers in bytes, without the 'B' suffix.
 --human-readable
diff --git a/Makefile.inc.in b/Makefile.inc.in
index 56271903..68bddbed 100644
--- a/Makefile.inc.in
+++ b/Makefile.inc.in
@@ -18,9 +18,9 @@ BTRFSRESTORE_ZSTD = @BTRFSRESTORE_ZSTD@
 SUBST_CFLAGS = @CFLAGS@
 SUBST_LDFLAGS = @LDFLAGS@
 
-LIBS_BASE = @UUID_LIBS@ @BLKID_LIBS@ -L. -pthread
+LIBS_BASE = @UUID_LIBS@ @BLKID_LIBS@ @JSON_LIBS@ -L. -pthread
 LIBS_COMP = @ZLIB_LIBS@ @LZO2_LIBS@ @ZSTD_LIBS@
-STATIC_LIBS_BASE = @UUID_LIBS_STATIC@ @BLKID_LIBS_STATIC@ -L. -pthread
+STATIC_LIBS_BASE = @UUID_LIBS_STATIC@ @BLKID_LIBS_STATIC@ @JSON_LIBS_STATIC@ 
-L. -pthread
 STATIC_LIBS_COMP = @ZLIB_LIBS_STATIC@ @LZO2_LIBS_STATIC@ @ZSTD_LIBS_STATIC@
 
 prefix ?= @prefix@
diff --git a/cmds-qgroup.c b/cmds-qgroup.c
index 94cd0fd3..eee15ef1 100644
--- a/cmds-qgroup.c
+++ b/cmds-qgroup.c
@@ -282,6 +282,10 @@ static const char * const cmd_qgroup_show_usage[] = {
"   (excluding ancestral qgroups)",
"-P print first-level qgroups using pathname",
"-v verbose, prints all nested subvolumes",
+#ifdef HAVE_JSON
+   "-j export in JSON format",
+   "--json-compat  export in JSON compatibility mode",
+#endif
HELPINFO_UNITS_LONG,
"--sort=qgroupid,rfer,excl,max_rfer,max_excl,pathname",
"   list qgroups sorted by specified items",
@@ -302,6 +306,8 @@ static int cmd_qgroup_show(int argc, char **argv)
unsigned unit_mode;
int sync = 0;
bool verbose = false;
+   bool export_json = false;
+   bool compat_json = false;
 
struct btrfs_qgroup_comparer_set *comparer_set;
struct btrfs_qgroup_filter_set *filter_set;
@@ -314,16 +320,26 @@ static int cmd_qgroup_show(int argc, char **argv)
int c;
enum {
GETOPT_VAL_SORT = 256,
-   GETOPT_VAL_SYNC
+   GETOPT_VAL_SYNC,
+   GETOPT_VAL_JSCOMPAT,
};
static const struct option long_options[] = {
{"sort", required_argument, NULL, GETOPT_VAL_SORT},
{"sync", no_argument, NULL, GETOPT_VAL_SYNC},
{"verbose", no_argument, NULL, 'v'},
+#ifdef HAVE_JSON
+   {"json-compat", no_argument, NULL, GETOPT_VAL_JSCOMPAT},
+#endif
{ NULL, 0, NULL, 0 }
};
-
-   c = getopt_long(argc, argv, "pPcreFfv", long_options, NULL);
+   const char getopt_chars[] = {
+   'p', 'P', 'c', 'r', 'e', 'F', 'f', 'v',
+#ifdef HAVE_JSON
+   'j',
+#endif
+   '\0' };
+
+   c = getopt_long(argc, argv, getopt_chars, long_options, NULL);
if (c < 0)
break;
switch (c) {
@@ -353,6 +369,14 @@ static int cmd_qgroup_show(int argc, char **argv)
case 'f':
filter_flag |= 0x2;
break;
+#ifdef HAVE_JSON
+   case 

[PATCH 1/8] btrfs-progs: quota: Add -W option to rescan to wait without starting rescan

[jeffm]

From: Jeff Mahoney 

This patch adds a new -W option to wait for a rescan without starting a
new operation.  This is useful for things like xfstests where we want
do to do a "btrfs quota enable" and not continue until the subsequent
rescan has finished.

In addition to documenting the new option in the man page, I've cleaned
up the rescan entry to document the -w option a bit better.

Signed-off-by: Jeff Mahoney 
---
 Documentation/btrfs-quota.asciidoc | 10 +++---
 cmds-quota.c   | 21 +++--
 2 files changed, 22 insertions(+), 9 deletions(-)

diff --git a/Documentation/btrfs-quota.asciidoc 
b/Documentation/btrfs-quota.asciidoc
index 85ebf729..0b64a69b 100644
--- a/Documentation/btrfs-quota.asciidoc
+++ b/Documentation/btrfs-quota.asciidoc
@@ -238,15 +238,19 @@ Disable subvolume quota support for a filesystem.
 *enable* ::
 Enable subvolume quota support for a filesystem.
 
-*rescan* [-s] ::
+*rescan* [-s|-w|-W] ::
 Trash all qgroup numbers and scan the metadata again with the current config.
 +
 `Options`
 +
 -s
-show status of a running rescan operation.
+Show status of a running rescan operation.
+
 -w
-wait for rescan operation to finish(can be already in progress).
+Start rescan operation and wait until it has finished before exiting.  If a 
rescan is already running, wait until it finishes and then exit without 
starting a new one.
+
+-W
+Wait for rescan operation to finish and then exit.  If a rescan is not already 
running, exit silently.
 
 EXIT STATUS
 ---
diff --git a/cmds-quota.c b/cmds-quota.c
index 745889d1..fe6376ac 100644
--- a/cmds-quota.c
+++ b/cmds-quota.c
@@ -120,14 +120,20 @@ static int cmd_quota_rescan(int argc, char **argv)
int wait_for_completion = 0;
 
while (1) {
-   int c = getopt(argc, argv, "sw");
+   int c = getopt(argc, argv, "swW");
if (c < 0)
break;
switch (c) {
case 's':
ioctlnum = BTRFS_IOC_QUOTA_RESCAN_STATUS;
break;
+   case 'W':
+   ioctlnum = 0;
+   wait_for_completion = 1;
+   break;
case 'w':
+   /* Reset it in case the user did both -W and -w */
+   ioctlnum = BTRFS_IOC_QUOTA_RESCAN;
wait_for_completion = 1;
break;
default:
@@ -135,8 +141,9 @@ static int cmd_quota_rescan(int argc, char **argv)
}
}
 
-   if (ioctlnum != BTRFS_IOC_QUOTA_RESCAN && wait_for_completion) {
-   error("switch -w cannot be used with -s");
+   if (ioctlnum == BTRFS_IOC_QUOTA_RESCAN_STATUS && wait_for_completion) {
+   error("switch -%c cannot be used with -s",
+ ioctlnum ? 'w' : 'W');
return 1;
}
 
@@ -150,8 +157,10 @@ static int cmd_quota_rescan(int argc, char **argv)
if (fd < 0)
return 1;
 
-   ret = ioctl(fd, ioctlnum, );
-   e = errno;
+   if (ioctlnum) {
+   ret = ioctl(fd, ioctlnum, );
+   e = errno;
+   }
 
if (ioctlnum == BTRFS_IOC_QUOTA_RESCAN_STATUS) {
close_file_or_dir(fd, dirstream);
@@ -167,7 +176,7 @@ static int cmd_quota_rescan(int argc, char **argv)
return 0;
}
 
-   if (ret == 0) {
+   if (ioctlnum == BTRFS_IOC_QUOTA_RESCAN && ret == 0) {
printf("quota rescan started\n");
fflush(stdout);
} else if (ret < 0 && (!wait_for_completion || e != EINPROGRESS)) {
-- 
2.15.1

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 8/8] btrfs-progs: add quota info to btrfs sub show

[jeffm]

From: Jeff Mahoney 

This patch reports on the first-level qgroup, if any, associated with
a particular subvolume.  It displays the usage and limit, subject
to the usual unit parameters.

Signed-off-by: Jeff Mahoney 
---
 cmds-subvolume.c | 46 ++
 1 file changed, 46 insertions(+)

diff --git a/cmds-subvolume.c b/cmds-subvolume.c
index 8a473f7a..29d0e0e5 100644
--- a/cmds-subvolume.c
+++ b/cmds-subvolume.c
@@ -972,6 +972,7 @@ static const char * const cmd_subvol_show_usage[] = {
"Show more information about the subvolume",
"-r|--rootid   rootid of the subvolume",
"-u|--uuid uuid of the subvolume",
+   HELPINFO_UNITS_SHORT_LONG,
"",
"If no option is specified,  will be shown, otherwise",
"the rootid or uuid are resolved relative to the  path.",
@@ -993,6 +994,13 @@ static int cmd_subvol_show(int argc, char **argv)
int by_uuid = 0;
u64 rootid_arg;
u8 uuid_arg[BTRFS_UUID_SIZE];
+   struct btrfs_qgroup_stats stats;
+   unsigned int unit_mode;
+   const char *referenced_size;
+   const char *referenced_limit_size = "-";
+   unsigned field_width = 0;
+
+   unit_mode = get_unit_mode_from_arg(, argv, 1);
 
while (1) {
int c;
@@ -1112,6 +1120,44 @@ static int cmd_subvol_show(int argc, char **argv)
btrfs_list_subvols_print(fd, filter_set, NULL, BTRFS_LIST_LAYOUT_RAW,
1, raw_prefix);
 
+   ret = btrfs_qgroup_query(fd, get_ri.root_id, );
+   if (ret < 0) {
+   if (ret == -ENODATA)
+   printf("Quotas must be enabled for per-subvolume 
usage\n");
+   else if (ret != -ENOTTY)
+   fprintf(stderr,
+   "\nERROR: BTRFS_IOC_QUOTA_QUERY failed: %s\n",
+   strerror(errno));
+   goto out;
+   }
+
+   printf("\tQuota Usage:\t\t");
+   fflush(stdout);
+
+   referenced_size = pretty_size_mode(stats.info.referenced, unit_mode);
+   if (stats.limit.max_referenced)
+  referenced_limit_size = pretty_size_mode(
+   stats.limit.max_referenced,
+   unit_mode);
+   field_width = max(strlen(referenced_size),
+ strlen(referenced_limit_size));
+
+   printf("%-*s referenced, %s exclusive\n ", field_width,
+  referenced_size,
+  pretty_size_mode(stats.info.exclusive, unit_mode));
+
+   printf("\tQuota Limits:\t\t");
+   if (stats.limit.max_referenced || stats.limit.max_exclusive) {
+   const char *excl = "-";
+
+   if (stats.limit.max_exclusive)
+  excl = pretty_size_mode(stats.limit.max_exclusive,
+  unit_mode);
+   printf("%-*s referenced, %s exclusive\n", field_width,
+  referenced_limit_size, excl);
+   } else
+   printf("None\n");
+
 out:
/* clean up */
free(get_ri.path);
-- 
2.15.1

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 5/8] btrfs-progs: qgroups: export qgroups usage information as JSON

[jeffm]

From: Jeff Mahoney 

One of the common requests I receive is for 'df' like facilities
for subvolume usage.  Really, the request is for monitoring tools to be
able to understand when subvolumes may be approaching quota in the same
manner traditional file systems approach ENOSPC.

This patch allows us to export the qgroups data in a machine-readable
format so that monitoring tools can parse it easily.

There are two modes since JSON can technically handle 64-bit numbers
but JavaScript proper cannot.  show -j enables JSON mode using 64-bit
integers directly.  --json-compat presents 64-bit numbers as an array
of two 32-bit numbers (high followed by low).

Signed-off-by: Jeff Mahoney 
---
 Documentation/btrfs-qgroup.asciidoc |   4 +
 Makefile.inc.in |   4 +-
 cmds-qgroup.c   |  36 +-
 configure.ac|   6 +
 qgroup.c| 211 
 qgroup.h|   3 +
 6 files changed, 258 insertions(+), 6 deletions(-)

diff --git a/Documentation/btrfs-qgroup.asciidoc 
b/Documentation/btrfs-qgroup.asciidoc
index 360b3269..22a9c2a7 100644
--- a/Documentation/btrfs-qgroup.asciidoc
+++ b/Documentation/btrfs-qgroup.asciidoc
@@ -105,6 +105,10 @@ list all qgroups which impact the given path(include 
ancestral qgroups)
 list all qgroups which impact the given path(exclude ancestral qgroups)
 -v
 Be more verbose.  Print pathnames of member qgroups when nested.
+-j
+If enabled, export qgroup usage information in JSON format.  This implies 
--raw.
+--json-compat
+By default, JSON output contains full 64-bit integers, which may be 
incompatible with some JSON parsers.  This option exports those values as an 
array of 32-bit numbers in [high, low] format.
 --raw
 raw numbers in bytes, without the 'B' suffix.
 --human-readable
diff --git a/Makefile.inc.in b/Makefile.inc.in
index 56271903..68bddbed 100644
--- a/Makefile.inc.in
+++ b/Makefile.inc.in
@@ -18,9 +18,9 @@ BTRFSRESTORE_ZSTD = @BTRFSRESTORE_ZSTD@
 SUBST_CFLAGS = @CFLAGS@
 SUBST_LDFLAGS = @LDFLAGS@
 
-LIBS_BASE = @UUID_LIBS@ @BLKID_LIBS@ -L. -pthread
+LIBS_BASE = @UUID_LIBS@ @BLKID_LIBS@ @JSON_LIBS@ -L. -pthread
 LIBS_COMP = @ZLIB_LIBS@ @LZO2_LIBS@ @ZSTD_LIBS@
-STATIC_LIBS_BASE = @UUID_LIBS_STATIC@ @BLKID_LIBS_STATIC@ -L. -pthread
+STATIC_LIBS_BASE = @UUID_LIBS_STATIC@ @BLKID_LIBS_STATIC@ @JSON_LIBS_STATIC@ 
-L. -pthread
 STATIC_LIBS_COMP = @ZLIB_LIBS_STATIC@ @LZO2_LIBS_STATIC@ @ZSTD_LIBS_STATIC@
 
 prefix ?= @prefix@
diff --git a/cmds-qgroup.c b/cmds-qgroup.c
index 94cd0fd3..eee15ef1 100644
--- a/cmds-qgroup.c
+++ b/cmds-qgroup.c
@@ -282,6 +282,10 @@ static const char * const cmd_qgroup_show_usage[] = {
"   (excluding ancestral qgroups)",
"-P print first-level qgroups using pathname",
"-v verbose, prints all nested subvolumes",
+#ifdef HAVE_JSON
+   "-j export in JSON format",
+   "--json-compat  export in JSON compatibility mode",
+#endif
HELPINFO_UNITS_LONG,
"--sort=qgroupid,rfer,excl,max_rfer,max_excl,pathname",
"   list qgroups sorted by specified items",
@@ -302,6 +306,8 @@ static int cmd_qgroup_show(int argc, char **argv)
unsigned unit_mode;
int sync = 0;
bool verbose = false;
+   bool export_json = false;
+   bool compat_json = false;
 
struct btrfs_qgroup_comparer_set *comparer_set;
struct btrfs_qgroup_filter_set *filter_set;
@@ -314,16 +320,26 @@ static int cmd_qgroup_show(int argc, char **argv)
int c;
enum {
GETOPT_VAL_SORT = 256,
-   GETOPT_VAL_SYNC
+   GETOPT_VAL_SYNC,
+   GETOPT_VAL_JSCOMPAT,
};
static const struct option long_options[] = {
{"sort", required_argument, NULL, GETOPT_VAL_SORT},
{"sync", no_argument, NULL, GETOPT_VAL_SYNC},
{"verbose", no_argument, NULL, 'v'},
+#ifdef HAVE_JSON
+   {"json-compat", no_argument, NULL, GETOPT_VAL_JSCOMPAT},
+#endif
{ NULL, 0, NULL, 0 }
};
-
-   c = getopt_long(argc, argv, "pPcreFfv", long_options, NULL);
+   const char getopt_chars[] = {
+   'p', 'P', 'c', 'r', 'e', 'F', 'f', 'v',
+#ifdef HAVE_JSON
+   'j',
+#endif
+   '\0' };
+
+   c = getopt_long(argc, argv, getopt_chars, long_options, NULL);
if (c < 0)
break;
switch (c) {
@@ -353,6 +369,14 @@ static int cmd_qgroup_show(int argc, char **argv)
case 'f':
filter_flag |= 0x2;
break;
+#ifdef HAVE_JSON
+   case 

[PATCH 6/8] btrfs-progs: qgroups: introduce btrfs_qgroup_query

[jeffm]

From: Jeff Mahoney 

The only mechanism we have in the progs for searching qgroups is to load
all of them and filter the results.  This works for qgroup show but
to add quota information to 'btrfs subvoluem show' it's pretty wasteful.

This patch splits out setting up the search and performing the search so
we can search for a single qgroupid more easily.

Signed-off-by: Jeff Mahoney 
---
 qgroup.c | 98 +---
 qgroup.h |  7 +
 2 files changed, 77 insertions(+), 28 deletions(-)

diff --git a/qgroup.c b/qgroup.c
index b1be3311..2d0a6947 100644
--- a/qgroup.c
+++ b/qgroup.c
@@ -1146,11 +1146,11 @@ static inline void print_status_flag_warning(u64 flags)
warning("qgroup data inconsistent, rescan recommended");
 }
 
-static int __qgroups_search(int fd, struct qgroup_lookup *qgroup_lookup)
+static int __qgroups_search(int fd, struct btrfs_ioctl_search_args *args,
+   struct qgroup_lookup *qgroup_lookup)
 {
int ret;
-   struct btrfs_ioctl_search_args args;
-   struct btrfs_ioctl_search_key *sk = 
+   struct btrfs_ioctl_search_key *sk = >key;
struct btrfs_ioctl_search_header *sh;
unsigned long off = 0;
unsigned int i;
@@ -1161,30 +1161,12 @@ static int __qgroups_search(int fd, struct 
qgroup_lookup *qgroup_lookup)
u64 qgroupid;
u64 qgroupid1;
 
-   memset(, 0, sizeof(args));
-
-   sk->tree_id = BTRFS_QUOTA_TREE_OBJECTID;
-   sk->max_type = BTRFS_QGROUP_RELATION_KEY;
-   sk->min_type = BTRFS_QGROUP_STATUS_KEY;
-   sk->max_objectid = (u64)-1;
-   sk->max_offset = (u64)-1;
-   sk->max_transid = (u64)-1;
-   sk->nr_items = 4096;
-
qgroup_lookup_init(qgroup_lookup);
 
while (1) {
-   ret = ioctl(fd, BTRFS_IOC_TREE_SEARCH, );
+   ret = ioctl(fd, BTRFS_IOC_TREE_SEARCH, args);
if (ret < 0) {
-   if (errno == ENOENT) {
-   error("can't list qgroups: quotas not enabled");
-   ret = -ENOTTY;
-   } else {
-   error("can't list qgroups: %s",
-  strerror(errno));
-   ret = -errno;
-   }
-
+   ret = -errno;
break;
}
 
@@ -1198,14 +1180,14 @@ static int __qgroups_search(int fd, struct 
qgroup_lookup *qgroup_lookup)
 * read the root_ref item it contains
 */
for (i = 0; i < sk->nr_items; i++) {
-   sh = (struct btrfs_ioctl_search_header *)(args.buf +
+   sh = (struct btrfs_ioctl_search_header *)(args->buf +
  off);
off += sizeof(*sh);
 
switch (btrfs_search_header_type(sh)) {
case BTRFS_QGROUP_STATUS_KEY:
si = (struct btrfs_qgroup_status_item *)
-(args.buf + off);
+(args->buf + off);
flags = btrfs_stack_qgroup_status_flags(si);
 
print_status_flag_warning(flags);
@@ -1213,7 +1195,7 @@ static int __qgroups_search(int fd, struct qgroup_lookup 
*qgroup_lookup)
case BTRFS_QGROUP_INFO_KEY:
qgroupid = btrfs_search_header_offset(sh);
info = (struct btrfs_qgroup_info_item *)
-  (args.buf + off);
+  (args->buf + off);
 
ret = update_qgroup_info(fd, qgroup_lookup,
 qgroupid, info);
@@ -1221,7 +1203,7 @@ static int __qgroups_search(int fd, struct qgroup_lookup 
*qgroup_lookup)
case BTRFS_QGROUP_LIMIT_KEY:
qgroupid = btrfs_search_header_offset(sh);
limit = (struct btrfs_qgroup_limit_item *)
-   (args.buf + off);
+   (args->buf + off);
 
ret = update_qgroup_limit(fd, qgroup_lookup,
  qgroupid, limit);
@@ -1267,6 +1249,66 @@ static int __qgroups_search(int fd, struct qgroup_lookup 
*qgroup_lookup)
return ret;
 }
 
+static int qgroups_search_all(int fd, struct qgroup_lookup *qgroup_lookup)
+{
+   struct btrfs_ioctl_search_args args = {
+   .key = {
+   .tree_id = BTRFS_QUOTA_TREE_OBJECTID,
+   .max_type = BTRFS_QGROUP_RELATION_KEY,
+   .min_type = 

[PATCH 7/8] btrfs-progs: subvolume: add quota info to btrfs sub show

[jeffm]

From: Jeff Mahoney 

This patch reports on the first-level qgroup, if any, associated with
a particular subvolume.  It displays the usage and limit, subject
to the usual unit parameters.

Signed-off-by: Jeff Mahoney 
---
 cmds-subvolume.c | 46 ++
 1 file changed, 46 insertions(+)

diff --git a/cmds-subvolume.c b/cmds-subvolume.c
index 8a473f7a..29d0e0e5 100644
--- a/cmds-subvolume.c
+++ b/cmds-subvolume.c
@@ -972,6 +972,7 @@ static const char * const cmd_subvol_show_usage[] = {
"Show more information about the subvolume",
"-r|--rootid   rootid of the subvolume",
"-u|--uuid uuid of the subvolume",
+   HELPINFO_UNITS_SHORT_LONG,
"",
"If no option is specified,  will be shown, otherwise",
"the rootid or uuid are resolved relative to the  path.",
@@ -993,6 +994,13 @@ static int cmd_subvol_show(int argc, char **argv)
int by_uuid = 0;
u64 rootid_arg;
u8 uuid_arg[BTRFS_UUID_SIZE];
+   struct btrfs_qgroup_stats stats;
+   unsigned int unit_mode;
+   const char *referenced_size;
+   const char *referenced_limit_size = "-";
+   unsigned field_width = 0;
+
+   unit_mode = get_unit_mode_from_arg(, argv, 1);
 
while (1) {
int c;
@@ -1112,6 +1120,44 @@ static int cmd_subvol_show(int argc, char **argv)
btrfs_list_subvols_print(fd, filter_set, NULL, BTRFS_LIST_LAYOUT_RAW,
1, raw_prefix);
 
+   ret = btrfs_qgroup_query(fd, get_ri.root_id, );
+   if (ret < 0) {
+   if (ret == -ENODATA)
+   printf("Quotas must be enabled for per-subvolume 
usage\n");
+   else if (ret != -ENOTTY)
+   fprintf(stderr,
+   "\nERROR: BTRFS_IOC_QUOTA_QUERY failed: %s\n",
+   strerror(errno));
+   goto out;
+   }
+
+   printf("\tQuota Usage:\t\t");
+   fflush(stdout);
+
+   referenced_size = pretty_size_mode(stats.info.referenced, unit_mode);
+   if (stats.limit.max_referenced)
+  referenced_limit_size = pretty_size_mode(
+   stats.limit.max_referenced,
+   unit_mode);
+   field_width = max(strlen(referenced_size),
+ strlen(referenced_limit_size));
+
+   printf("%-*s referenced, %s exclusive\n ", field_width,
+  referenced_size,
+  pretty_size_mode(stats.info.exclusive, unit_mode));
+
+   printf("\tQuota Limits:\t\t");
+   if (stats.limit.max_referenced || stats.limit.max_exclusive) {
+   const char *excl = "-";
+
+   if (stats.limit.max_exclusive)
+  excl = pretty_size_mode(stats.limit.max_exclusive,
+  unit_mode);
+   printf("%-*s referenced, %s exclusive\n", field_width,
+  referenced_limit_size, excl);
+   } else
+   printf("None\n");
+
 out:
/* clean up */
free(get_ri.path);
-- 
2.15.1

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 6/8] btrfs-progs: qgroups: introduce and use info and limit structures

[jeffm]

From: Jeff Mahoney 

We use structures to pass the info and limit from the kernel as items
but store the individual values separately in btrfs_qgroup.  We already
have a btrfs_qgroup_limit structure that's used for setting the limit.

This patch introduces a btrfs_qgroup_info structure and uses that and
btrfs_qgroup_limit in btrfs_qgroup.

Signed-off-by: Jeff Mahoney 
---
 qgroup.c | 83 +++-
 qgroup.h |  8 +++
 2 files changed, 48 insertions(+), 43 deletions(-)

diff --git a/qgroup.c b/qgroup.c
index 5a7a8530..7ec12ec1 100644
--- a/qgroup.c
+++ b/qgroup.c
@@ -50,20 +50,12 @@ struct btrfs_qgroup {
/*
 * info_item
 */
-   u64 generation;
-   u64 rfer;   /*referenced*/
-   u64 rfer_cmpr;  /*referenced compressed*/
-   u64 excl;   /*exclusive*/
-   u64 excl_cmpr;  /*exclusive compressed*/
+   struct btrfs_qgroup_info info;
 
/*
 *limit_item
 */
-   u64 flags;  /*which limits are set*/
-   u64 max_rfer;
-   u64 max_excl;
-   u64 rsv_rfer;
-   u64 rsv_excl;
+   struct btrfs_qgroup_limit limit;
 
/*qgroups this group is member of*/
struct list_head qgroups;
@@ -276,24 +268,24 @@ static void print_qgroup_column(struct btrfs_qgroup 
*qgroup,
print_qgroup_column_add_blank(BTRFS_QGROUP_QGROUPID, len);
break;
case BTRFS_QGROUP_RFER:
-   len = printf("%*s", max_len, pretty_size_mode(qgroup->rfer, 
unit_mode));
+   len = printf("%*s", max_len, 
pretty_size_mode(qgroup->info.referenced, unit_mode));
break;
case BTRFS_QGROUP_EXCL:
-   len = printf("%*s", max_len, pretty_size_mode(qgroup->excl, 
unit_mode));
+   len = printf("%*s", max_len, 
pretty_size_mode(qgroup->info.exclusive, unit_mode));
break;
case BTRFS_QGROUP_PARENT:
len = print_parent_column(qgroup);
print_qgroup_column_add_blank(BTRFS_QGROUP_PARENT, len);
break;
case BTRFS_QGROUP_MAX_RFER:
-   if (qgroup->flags & BTRFS_QGROUP_LIMIT_MAX_RFER)
-   len = printf("%*s", max_len, 
pretty_size_mode(qgroup->max_rfer, unit_mode));
+   if (qgroup->limit.flags & BTRFS_QGROUP_LIMIT_MAX_RFER)
+   len = printf("%*s", max_len, 
pretty_size_mode(qgroup->limit.max_referenced, unit_mode));
else
len = printf("%*s", max_len, "none");
break;
case BTRFS_QGROUP_MAX_EXCL:
-   if (qgroup->flags & BTRFS_QGROUP_LIMIT_MAX_EXCL)
-   len = printf("%*s", max_len, 
pretty_size_mode(qgroup->max_excl, unit_mode));
+   if (qgroup->limit.flags & BTRFS_QGROUP_LIMIT_MAX_EXCL)
+   len = printf("%*s", max_len, 
pretty_size_mode(qgroup->limit.max_exclusive, unit_mode));
else
len = printf("%*s", max_len, "none");
break;
@@ -436,9 +428,9 @@ static int comp_entry_with_rfer(struct btrfs_qgroup *entry1,
 {
int ret;
 
-   if (entry1->rfer > entry2->rfer)
+   if (entry1->info.referenced > entry2->info.referenced)
ret = 1;
-   else if (entry1->rfer < entry2->rfer)
+   else if (entry1->info.referenced < entry2->info.referenced)
ret = -1;
else
ret = 0;
@@ -452,9 +444,9 @@ static int comp_entry_with_excl(struct btrfs_qgroup *entry1,
 {
int ret;
 
-   if (entry1->excl > entry2->excl)
+   if (entry1->info.exclusive > entry2->info.exclusive)
ret = 1;
-   else if (entry1->excl < entry2->excl)
+   else if (entry1->info.exclusive < entry2->info.exclusive)
ret = -1;
else
ret = 0;
@@ -468,9 +460,9 @@ static int comp_entry_with_max_rfer(struct btrfs_qgroup 
*entry1,
 {
int ret;
 
-   if (entry1->max_rfer > entry2->max_rfer)
+   if (entry1->limit.max_referenced > entry2->limit.max_referenced)
ret = 1;
-   else if (entry1->max_rfer < entry2->max_rfer)
+   else if (entry1->limit.max_referenced < entry2->limit.max_referenced)
ret = -1;
else
ret = 0;
@@ -484,9 +476,9 @@ static int comp_entry_with_max_excl(struct btrfs_qgroup 
*entry1,
 {
int ret;
 
-   if (entry1->max_excl > entry2->max_excl)
+   if (entry1->limit.max_exclusive > entry2->limit.max_exclusive)
ret = 1;
-   else if (entry1->max_excl < entry2->max_excl)
+   else if (entry1->limit.max_exclusive < entry2->limit.max_exclusive)
ret = -1;
else
ret = 0;
@@ -743,11 +735,13 @@ static int update_qgroup_info(int fd, struct 
qgroup_lookup *qgroup_lookup,
if (IS_ERR_OR_NULL(bq))
return 

[PATCH 2/8] btrfs-progs: qgroups: fix misleading index check

[jeffm]

From: Jeff Mahoney 

In print_single_qgroup_table we check the loop index against
BTRFS_QGROUP_CHILD, but what we really mean is "last column."  Since
we have an enum value to indicate the last value, use that instead
of assuming that BTRFS_QGROUP_CHILD is always last.

Signed-off-by: Jeff Mahoney 
---
 qgroup.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/qgroup.c b/qgroup.c
index 11659e83..67bc0738 100644
--- a/qgroup.c
+++ b/qgroup.c
@@ -267,7 +267,7 @@ static void print_single_qgroup_table(struct btrfs_qgroup 
*qgroup)
continue;
print_qgroup_column(qgroup, i);
 
-   if (i != BTRFS_QGROUP_CHILD)
+   if (i != BTRFS_QGROUP_ALL - 1)
printf(" ");
}
printf("\n");
-- 
2.15.1

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 5/8] btrfs-progs: qgroups: introduce and use info and limit structures

[jeffm]

From: Jeff Mahoney 

We use structures to pass the info and limit from the kernel as items
but store the individual values separately in btrfs_qgroup.  We already
have a btrfs_qgroup_limit structure that's used for setting the limit.

This patch introduces a btrfs_qgroup_info structure and uses that and
btrfs_qgroup_limit in btrfs_qgroup.

Signed-off-by: Jeff Mahoney 
---
 qgroup.c | 73 +++-
 qgroup.h |  8 +++
 2 files changed, 43 insertions(+), 38 deletions(-)

diff --git a/qgroup.c b/qgroup.c
index 83918134..b1be3311 100644
--- a/qgroup.c
+++ b/qgroup.c
@@ -46,20 +46,12 @@ struct btrfs_qgroup {
/*
 * info_item
 */
-   u64 generation;
-   u64 rfer;   /*referenced*/
-   u64 rfer_cmpr;  /*referenced compressed*/
-   u64 excl;   /*exclusive*/
-   u64 excl_cmpr;  /*exclusive compressed*/
+   struct btrfs_qgroup_info info;
 
/*
 *limit_item
 */
-   u64 flags;  /*which limits are set*/
-   u64 max_rfer;
-   u64 max_excl;
-   u64 rsv_rfer;
-   u64 rsv_excl;
+   struct btrfs_qgroup_limit limit;
 
/*qgroups this group is member of*/
struct list_head qgroups;
@@ -272,24 +264,24 @@ static void print_qgroup_column(struct btrfs_qgroup 
*qgroup,
print_qgroup_column_add_blank(BTRFS_QGROUP_QGROUPID, len);
break;
case BTRFS_QGROUP_RFER:
-   len = printf("%*s", max_len, pretty_size_mode(qgroup->rfer, 
unit_mode));
+   len = printf("%*s", max_len, 
pretty_size_mode(qgroup->info.referenced, unit_mode));
break;
case BTRFS_QGROUP_EXCL:
-   len = printf("%*s", max_len, pretty_size_mode(qgroup->excl, 
unit_mode));
+   len = printf("%*s", max_len, 
pretty_size_mode(qgroup->info.exclusive, unit_mode));
break;
case BTRFS_QGROUP_PARENT:
len = print_parent_column(qgroup);
print_qgroup_column_add_blank(BTRFS_QGROUP_PARENT, len);
break;
case BTRFS_QGROUP_MAX_RFER:
-   if (qgroup->flags & BTRFS_QGROUP_LIMIT_MAX_RFER)
-   len = printf("%*s", max_len, 
pretty_size_mode(qgroup->max_rfer, unit_mode));
+   if (qgroup->limit.flags & BTRFS_QGROUP_LIMIT_MAX_RFER)
+   len = printf("%*s", max_len, 
pretty_size_mode(qgroup->limit.max_referenced, unit_mode));
else
len = printf("%*s", max_len, "none");
break;
case BTRFS_QGROUP_MAX_EXCL:
-   if (qgroup->flags & BTRFS_QGROUP_LIMIT_MAX_EXCL)
-   len = printf("%*s", max_len, 
pretty_size_mode(qgroup->max_excl, unit_mode));
+   if (qgroup->limit.flags & BTRFS_QGROUP_LIMIT_MAX_EXCL)
+   len = printf("%*s", max_len, 
pretty_size_mode(qgroup->limit.max_exclusive, unit_mode));
else
len = printf("%*s", max_len, "none");
break;
@@ -432,9 +424,9 @@ static int comp_entry_with_rfer(struct btrfs_qgroup *entry1,
 {
int ret;
 
-   if (entry1->rfer > entry2->rfer)
+   if (entry1->info.referenced > entry2->info.referenced)
ret = 1;
-   else if (entry1->rfer < entry2->rfer)
+   else if (entry1->info.referenced < entry2->info.referenced)
ret = -1;
else
ret = 0;
@@ -448,9 +440,9 @@ static int comp_entry_with_excl(struct btrfs_qgroup *entry1,
 {
int ret;
 
-   if (entry1->excl > entry2->excl)
+   if (entry1->info.exclusive > entry2->info.exclusive)
ret = 1;
-   else if (entry1->excl < entry2->excl)
+   else if (entry1->info.exclusive < entry2->info.exclusive)
ret = -1;
else
ret = 0;
@@ -464,9 +456,9 @@ static int comp_entry_with_max_rfer(struct btrfs_qgroup 
*entry1,
 {
int ret;
 
-   if (entry1->max_rfer > entry2->max_rfer)
+   if (entry1->limit.max_referenced > entry2->limit.max_referenced)
ret = 1;
-   else if (entry1->max_rfer < entry2->max_rfer)
+   else if (entry1->limit.max_referenced < entry2->limit.max_referenced)
ret = -1;
else
ret = 0;
@@ -480,9 +472,9 @@ static int comp_entry_with_max_excl(struct btrfs_qgroup 
*entry1,
 {
int ret;
 
-   if (entry1->max_excl > entry2->max_excl)
+   if (entry1->limit.max_exclusive > entry2->limit.max_exclusive)
ret = 1;
-   else if (entry1->max_excl < entry2->max_excl)
+   else if (entry1->limit.max_exclusive < entry2->limit.max_exclusive)
ret = -1;
else
ret = 0;
@@ -739,11 +731,13 @@ static int update_qgroup_info(int fd, struct 
qgroup_lookup *qgroup_lookup,
if (IS_ERR_OR_NULL(bq))
return 

[PATCH 3/8] btrfs-progs: constify pathnames passed as arguments

[jeffm]

From: Jeff Mahoney 

It's unlikely we're going to modify a pathname argument, so codify that
and use const.

Signed-off-by: Jeff Mahoney 
---
 chunk-recover.c | 4 ++--
 cmds-device.c   | 2 +-
 cmds-fi-usage.c | 6 +++---
 cmds-rescue.c   | 4 ++--
 send-utils.c| 4 ++--
 5 files changed, 10 insertions(+), 10 deletions(-)

diff --git a/chunk-recover.c b/chunk-recover.c
index 705bcf52..1d30db51 100644
--- a/chunk-recover.c
+++ b/chunk-recover.c
@@ -1492,7 +1492,7 @@ out:
return ERR_PTR(ret);
 }
 
-static int recover_prepare(struct recover_control *rc, char *path)
+static int recover_prepare(struct recover_control *rc, const char *path)
 {
int ret;
int fd;
@@ -2296,7 +2296,7 @@ static void validate_rebuild_chunks(struct 
recover_control *rc)
 /*
  * Return 0 when successful, < 0 on error and > 0 if aborted by user
  */
-int btrfs_recover_chunk_tree(char *path, int verbose, int yes)
+int btrfs_recover_chunk_tree(const char *path, int verbose, int yes)
 {
int ret = 0;
struct btrfs_root *root = NULL;
diff --git a/cmds-device.c b/cmds-device.c
index 86459d1b..a49c9d9d 100644
--- a/cmds-device.c
+++ b/cmds-device.c
@@ -526,7 +526,7 @@ static const char * const cmd_device_usage_usage[] = {
NULL
 };
 
-static int _cmd_device_usage(int fd, char *path, unsigned unit_mode)
+static int _cmd_device_usage(int fd, const char *path, unsigned unit_mode)
 {
int i;
int ret = 0;
diff --git a/cmds-fi-usage.c b/cmds-fi-usage.c
index de7ad668..9a1c76ab 100644
--- a/cmds-fi-usage.c
+++ b/cmds-fi-usage.c
@@ -227,7 +227,7 @@ static int cmp_btrfs_ioctl_space_info(const void *a, const 
void *b)
 /*
  * This function load all the information about the space usage
  */
-static struct btrfs_ioctl_space_args *load_space_info(int fd, char *path)
+static struct btrfs_ioctl_space_args *load_space_info(int fd, const char *path)
 {
struct btrfs_ioctl_space_args *sargs = NULL, *sargs_orig = NULL;
int ret, count;
@@ -305,7 +305,7 @@ static void get_raid56_used(struct chunk_info *chunks, int 
chunkcount,
 #defineMIN_UNALOCATED_THRESH   SZ_16M
 static int print_filesystem_usage_overall(int fd, struct chunk_info *chunkinfo,
int chunkcount, struct device_info *devinfo, int devcount,
-   char *path, unsigned unit_mode)
+   const char *path, unsigned unit_mode)
 {
struct btrfs_ioctl_space_args *sargs = NULL;
int i;
@@ -931,7 +931,7 @@ static void _cmd_filesystem_usage_linear(unsigned unit_mode,
 static int print_filesystem_usage_by_chunk(int fd,
struct chunk_info *chunkinfo, int chunkcount,
struct device_info *devinfo, int devcount,
-   char *path, unsigned unit_mode, int tabular)
+   const char *path, unsigned unit_mode, int tabular)
 {
struct btrfs_ioctl_space_args *sargs;
int ret = 0;
diff --git a/cmds-rescue.c b/cmds-rescue.c
index c40088ad..c61145bc 100644
--- a/cmds-rescue.c
+++ b/cmds-rescue.c
@@ -32,8 +32,8 @@ static const char * const rescue_cmd_group_usage[] = {
NULL
 };
 
-int btrfs_recover_chunk_tree(char *path, int verbose, int yes);
-int btrfs_recover_superblocks(char *path, int verbose, int yes);
+int btrfs_recover_chunk_tree(const char *path, int verbose, int yes);
+int btrfs_recover_superblocks(const char *path, int verbose, int yes);
 
 static const char * const cmd_rescue_chunk_recover_usage[] = {
"btrfs rescue chunk-recover [options] ",
diff --git a/send-utils.c b/send-utils.c
index b5289e76..8ce94de1 100644
--- a/send-utils.c
+++ b/send-utils.c
@@ -28,8 +28,8 @@
 #include "ioctl.h"
 #include "btrfs-list.h"
 
-static int btrfs_subvolid_resolve_sub(int fd, char *path, size_t *path_len,
- u64 subvol_id);
+static int btrfs_subvolid_resolve_sub(int fd, char *path,
+ size_t *path_len, u64 subvol_id);
 
 static int btrfs_get_root_id_by_sub_path(int mnt_fd, const char *sub_path,
 u64 *root_id)
-- 
2.15.1

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 7/8] btrfs-progs: qgroups: introduce btrfs_qgroup_query

[jeffm]

From: Jeff Mahoney 

The only mechanism we have in the progs for searching qgroups is to load
all of them and filter the results.  This works for qgroup show but
to add quota information to 'btrfs subvoluem show' it's pretty wasteful.

This patch splits out setting up the search and performing the search so
we can search for a single qgroupid more easily.

Signed-off-by: Jeff Mahoney 
---
 qgroup.c | 100 +--
 qgroup.h |   7 +
 2 files changed, 78 insertions(+), 29 deletions(-)

diff --git a/qgroup.c b/qgroup.c
index 7ec12ec1..f632a45c 100644
--- a/qgroup.c
+++ b/qgroup.c
@@ -1150,11 +1150,11 @@ static inline void print_status_flag_warning(u64 flags)
warning("qgroup data inconsistent, rescan recommended");
 }
 
-static int __qgroups_search(int fd, struct qgroup_lookup *qgroup_lookup)
+static int __qgroups_search(int fd, struct btrfs_ioctl_search_args *args,
+   struct qgroup_lookup *qgroup_lookup)
 {
int ret;
-   struct btrfs_ioctl_search_args args;
-   struct btrfs_ioctl_search_key *sk = 
+   struct btrfs_ioctl_search_key *sk = >key;
struct btrfs_ioctl_search_header *sh;
unsigned long off = 0;
unsigned int i;
@@ -1165,30 +1165,12 @@ static int __qgroups_search(int fd, struct 
qgroup_lookup *qgroup_lookup)
u64 qgroupid;
u64 qgroupid1;
 
-   memset(, 0, sizeof(args));
-
-   sk->tree_id = BTRFS_QUOTA_TREE_OBJECTID;
-   sk->max_type = BTRFS_QGROUP_RELATION_KEY;
-   sk->min_type = BTRFS_QGROUP_STATUS_KEY;
-   sk->max_objectid = (u64)-1;
-   sk->max_offset = (u64)-1;
-   sk->max_transid = (u64)-1;
-   sk->nr_items = 4096;
-
qgroup_lookup_init(qgroup_lookup);
 
while (1) {
-   ret = ioctl(fd, BTRFS_IOC_TREE_SEARCH, );
+   ret = ioctl(fd, BTRFS_IOC_TREE_SEARCH, args);
if (ret < 0) {
-   if (errno == ENOENT) {
-   error("can't list qgroups: quotas not enabled");
-   ret = -ENOTTY;
-   } else {
-   error("can't list qgroups: %s",
-  strerror(errno));
-   ret = -errno;
-   }
-
+   ret = -errno;
break;
}
 
@@ -1202,14 +1184,14 @@ static int __qgroups_search(int fd, struct 
qgroup_lookup *qgroup_lookup)
 * read the root_ref item it contains
 */
for (i = 0; i < sk->nr_items; i++) {
-   sh = (struct btrfs_ioctl_search_header *)(args.buf +
+   sh = (struct btrfs_ioctl_search_header *)(args->buf +
  off);
off += sizeof(*sh);
 
switch (btrfs_search_header_type(sh)) {
case BTRFS_QGROUP_STATUS_KEY:
si = (struct btrfs_qgroup_status_item *)
-(args.buf + off);
+(args->buf + off);
flags = btrfs_stack_qgroup_status_flags(si);
 
print_status_flag_warning(flags);
@@ -1217,7 +1199,7 @@ static int __qgroups_search(int fd, struct qgroup_lookup 
*qgroup_lookup)
case BTRFS_QGROUP_INFO_KEY:
qgroupid = btrfs_search_header_offset(sh);
info = (struct btrfs_qgroup_info_item *)
-  (args.buf + off);
+  (args->buf + off);
 
ret = update_qgroup_info(fd, qgroup_lookup,
 qgroupid, info);
@@ -1225,7 +1207,7 @@ static int __qgroups_search(int fd, struct qgroup_lookup 
*qgroup_lookup)
case BTRFS_QGROUP_LIMIT_KEY:
qgroupid = btrfs_search_header_offset(sh);
limit = (struct btrfs_qgroup_limit_item *)
-   (args.buf + off);
+   (args->buf + off);
 
ret = update_qgroup_limit(fd, qgroup_lookup,
  qgroupid, limit);
@@ -1271,6 +1253,66 @@ static int __qgroups_search(int fd, struct qgroup_lookup 
*qgroup_lookup)
return ret;
 }
 
+static int qgroups_search_all(int fd, struct qgroup_lookup *qgroup_lookup)
+{
+   struct btrfs_ioctl_search_args args = {
+   .key = {
+   .tree_id = BTRFS_QUOTA_TREE_OBJECTID,
+   .max_type = BTRFS_QGROUP_RELATION_KEY,
+   .min_type = 

[PATCH 0/8] btrfs-progs: qgroups usability

[jeffm]

From: Jeff Mahoney 

Hi all -

The following series addresses some usability issues with the qgroups UI.

1) Adds -W option so we can wait on a rescan completing without starting one.
2) Adds qgroup information to 'btrfs subvolume show'
3) Adds a -P option to show pathnames for first-level qgroups (or member
   of nested qgroups with -v)
4) Allows exporting the qgroup table in JSON format for use by external
   programs/scripts.

-Jeff

Jeff Mahoney (8):
  btrfs-progs: quota: Add -W option to rescan to wait without starting
rescan
  btrfs-progs: qgroups: fix misleading index check
  btrfs-progs: constify pathnames passed as arguments
  btrfs-progs: qgroups: add pathname to show output
  btrfs-progs: qgroups: introduce and use info and limit structures
  btrfs-progs: qgroups: introduce btrfs_qgroup_query
  btrfs-progs: subvolume: add quota info to btrfs sub show
  btrfs-progs: qgroups: export qgroups usage information as JSON

 Documentation/btrfs-qgroup.asciidoc |   8 +
 Documentation/btrfs-quota.asciidoc  |  10 +-
 Makefile.inc.in |   4 +-
 chunk-recover.c |   4 +-
 cmds-device.c   |   2 +-
 cmds-fi-usage.c |   6 +-
 cmds-qgroup.c   |  49 +++-
 cmds-quota.c|  21 +-
 cmds-rescue.c   |   4 +-
 cmds-subvolume.c|  46 
 configure.ac|   6 +
 kerncompat.h|   1 +
 qgroup.c| 526 ++--
 qgroup.h|  22 +-
 send-utils.c|   4 +-
 utils.c |  22 +-
 utils.h |   2 +
 17 files changed, 621 insertions(+), 116 deletions(-)

-- 
2.15.1

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH] btrfs: qgroups, properly handle no reservations

[jeffm]

From: Jeff Mahoney 

There are several places where we call btrfs_qgroup_reserve_meta and
assume that a return value of 0 means that the reservation was successful.

Later, we use the original bytes value passed to that call to free
bytes during error handling or to pass the number of bytes reserved to
the caller.

This patch returns -ENODATA when we don't perform a reservation so that
callers can make the distinction.  This also lets call sites not
necessarily care whether qgroups are enabled.

Signed-off-by: Jeff Mahoney 
---
 fs/btrfs/extent-tree.c | 33 -
 fs/btrfs/qgroup.c  |  4 ++--
 fs/btrfs/transaction.c |  5 -
 3 files changed, 22 insertions(+), 20 deletions(-)

diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c
index c1618ab9fecf..2d5e963fae88 100644
--- a/fs/btrfs/extent-tree.c
+++ b/fs/btrfs/extent-tree.c
@@ -5988,20 +5988,18 @@ int btrfs_subvolume_reserve_metadata(struct btrfs_root 
*root,
 u64 *qgroup_reserved,
 bool use_global_rsv)
 {
-   u64 num_bytes;
int ret;
struct btrfs_fs_info *fs_info = root->fs_info;
struct btrfs_block_rsv *global_rsv = _info->global_block_rsv;
+   /* One for parent inode, two for dir entries */
+   u64 num_bytes = 3 * fs_info->nodesize;
 
-   if (test_bit(BTRFS_FS_QUOTA_ENABLED, _info->flags)) {
-   /* One for parent inode, two for dir entries */
-   num_bytes = 3 * fs_info->nodesize;
-   ret = btrfs_qgroup_reserve_meta(root, num_bytes, true);
-   if (ret)
-   return ret;
-   } else {
+   ret = btrfs_qgroup_reserve_meta(root, num_bytes, true);
+   if (ret == -ENODATA) {
num_bytes = 0;
-   }
+   ret = 0;
+   } else if (ret)
+   return ret;
 
*qgroup_reserved = num_bytes;
 
@@ -6057,6 +6055,7 @@ int btrfs_delalloc_reserve_metadata(struct btrfs_inode 
*inode, u64 num_bytes)
enum btrfs_reserve_flush_enum flush = BTRFS_RESERVE_FLUSH_ALL;
int ret = 0;
bool delalloc_lock = true;
+   u64 qgroup_reserved;
 
/* If we are a free space inode we need to not flush since we will be in
 * the middle of a transaction commit.  We also don't need the delalloc
@@ -6090,17 +6089,17 @@ int btrfs_delalloc_reserve_metadata(struct btrfs_inode 
*inode, u64 num_bytes)
btrfs_calculate_inode_block_rsv_size(fs_info, inode);
spin_unlock(>lock);
 
-   if (test_bit(BTRFS_FS_QUOTA_ENABLED, _info->flags)) {
-   ret = btrfs_qgroup_reserve_meta(root,
-   nr_extents * fs_info->nodesize, true);
-   if (ret)
-   goto out_fail;
-   }
+   qgroup_reserved = nr_extents * fs_info->nodesize;
+   ret = btrfs_qgroup_reserve_meta(root, qgroup_reserved, true);
+   if (ret == -ENODATA) {
+   ret = 0;
+   qgroup_reserved = 0;
+   } if (ret)
+   goto out_fail;
 
ret = btrfs_inode_rsv_refill(inode, flush);
if (unlikely(ret)) {
-   btrfs_qgroup_free_meta(root,
-  nr_extents * fs_info->nodesize);
+   btrfs_qgroup_free_meta(root, qgroup_reserved);
goto out_fail;
}
 
diff --git a/fs/btrfs/qgroup.c b/fs/btrfs/qgroup.c
index aa259d6986e1..5d9e011243c6 100644
--- a/fs/btrfs/qgroup.c
+++ b/fs/btrfs/qgroup.c
@@ -3025,7 +3025,7 @@ int btrfs_qgroup_reserve_meta(struct btrfs_root *root, 
int num_bytes,
 
if (!test_bit(BTRFS_FS_QUOTA_ENABLED, _info->flags) ||
!is_fstree(root->objectid) || num_bytes == 0)
-   return 0;
+   return -ENODATA;
 
BUG_ON(num_bytes != round_down(num_bytes, fs_info->nodesize));
trace_qgroup_meta_reserve(root, (s64)num_bytes);
@@ -3057,7 +3057,7 @@ void btrfs_qgroup_free_meta(struct btrfs_root *root, int 
num_bytes)
struct btrfs_fs_info *fs_info = root->fs_info;
 
if (!test_bit(BTRFS_FS_QUOTA_ENABLED, _info->flags) ||
-   !is_fstree(root->objectid))
+   !is_fstree(root->objectid) || num_bytes == 0)
return;
 
BUG_ON(num_bytes != round_down(num_bytes, fs_info->nodesize));
diff --git a/fs/btrfs/transaction.c b/fs/btrfs/transaction.c
index 04f07144b45c..ab67b73bd7fa 100644
--- a/fs/btrfs/transaction.c
+++ b/fs/btrfs/transaction.c
@@ -510,7 +510,10 @@ start_transaction(struct btrfs_root *root, unsigned int 
num_items,
qgroup_reserved = num_items * fs_info->nodesize;
ret = btrfs_qgroup_reserve_meta(root, qgroup_reserved,
enforce_qgroups);
-   if (ret)
+   if (ret == -ENODATA) {
+   ret = 0;
+   qgroup_reserved = 0;
+   } else if (ret)
   

[PATCH] btrfs: use kvzalloc to allocate btrfs_fs_info

[jeffm]

From: Jeff Mahoney 

The srcu_struct in btrfs_fs_infoa scales in size with NR_CPUS.  On
kernels built with NR_CPUS=8192, this can result in kmalloc failures
that prevent mounting.

There is work in progress to try to resolve this for every user of
srcu_struct but using kvzalloc will work around the failures until
that is complete.

Signed-off-by: Jeff Mahoney 
---
 fs/btrfs/ctree.h | 2 +-
 fs/btrfs/super.c | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/fs/btrfs/ctree.h b/fs/btrfs/ctree.h
index 1a462ab85c49..0f521ba5f2f9 100644
--- a/fs/btrfs/ctree.h
+++ b/fs/btrfs/ctree.h
@@ -2974,7 +2974,7 @@ static inline void free_fs_info(struct btrfs_fs_info 
*fs_info)
kfree(fs_info->super_copy);
kfree(fs_info->super_for_commit);
security_free_mnt_opts(_info->security_opts);
-   kfree(fs_info);
+   kvfree(fs_info);
 }
 
 /* tree mod log functions from ctree.c */
diff --git a/fs/btrfs/super.c b/fs/btrfs/super.c
index 6e71a2a78363..4b817947e00f 100644
--- a/fs/btrfs/super.c
+++ b/fs/btrfs/super.c
@@ -1545,7 +1545,7 @@ static struct dentry *btrfs_mount_root(struct 
file_system_type *fs_type,
 * it for searching for existing supers, so this lets us do that and
 * then open_ctree will properly initialize everything later.
 */
-   fs_info = kzalloc(sizeof(struct btrfs_fs_info), GFP_KERNEL);
+   fs_info = kvzalloc(sizeof(struct btrfs_fs_info), GFP_KERNEL);
if (!fs_info) {
error = -ENOMEM;
goto error_sec_opts;
-- 
2.15.1

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH] btrfs: fix missing error return in btrfs_drop_snapshot

[jeffm]

From: Jeff Mahoney 

If btrfs_del_root fails in btrfs_drop_snapshot, we'll pick up the
error but then return 0 anyway due to mixing err and ret.

Fixes: 79787eaab4612 ("btrfs: replace many BUG_ONs with proper error handling")
Cc:  # v3.4+
Signed-off-by: Jeff Mahoney 
---
 fs/btrfs/extent-tree.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c
index 4497f937e8fb..2f4328511ac8 100644
--- a/fs/btrfs/extent-tree.c
+++ b/fs/btrfs/extent-tree.c
@@ -9206,6 +9206,7 @@ int btrfs_drop_snapshot(struct btrfs_root *root,
ret = btrfs_del_root(trans, fs_info, >root_key);
if (ret) {
btrfs_abort_transaction(trans, ret);
+   err = ret;
goto out_end_trans;
}
 
-- 
2.14.2

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH] btrfs: handle errors while updating refcounts in update_ref_for_cow

[jeffm]

From: Jeff Mahoney 

Since commit fb235dc06fa (btrfs: qgroup: Move half of the qgroup
accounting time out of commit trans) the assumption that
btrfs_add_delayed_{data,tree}_ref can only return 0 or -ENOMEM has
been false.  The qgroup operations call into btrfs_search_slot
and friends and can now return the full spectrum of error codes.

Fortunately, the fix here is easy since update_ref_for_cow failing
is already handled so we just need to bail early with the error
code.

Fixes: fb235dc06fa (btrfs: qgroup: Move half of the qgroup accounting ...)
Cc:  # v4.11+
Signed-off-by: Jeff Mahoney 
---
 fs/btrfs/ctree.c | 18 --
 1 file changed, 12 insertions(+), 6 deletions(-)

diff --git a/fs/btrfs/ctree.c b/fs/btrfs/ctree.c
index 531e0a8645b0..1e74cf826532 100644
--- a/fs/btrfs/ctree.c
+++ b/fs/btrfs/ctree.c
@@ -1032,14 +1032,17 @@ static noinline int update_ref_for_cow(struct 
btrfs_trans_handle *trans,
 root->root_key.objectid == BTRFS_TREE_RELOC_OBJECTID) &&
!(flags & BTRFS_BLOCK_FLAG_FULL_BACKREF)) {
ret = btrfs_inc_ref(trans, root, buf, 1);
-   BUG_ON(ret); /* -ENOMEM */
+   if (ret)
+   return ret;
 
if (root->root_key.objectid ==
BTRFS_TREE_RELOC_OBJECTID) {
ret = btrfs_dec_ref(trans, root, buf, 0);
-   BUG_ON(ret); /* -ENOMEM */
+   if (ret)
+   return ret;
ret = btrfs_inc_ref(trans, root, cow, 1);
-   BUG_ON(ret); /* -ENOMEM */
+   if (ret)
+   return ret;
}
new_flags |= BTRFS_BLOCK_FLAG_FULL_BACKREF;
} else {
@@ -1049,7 +1052,8 @@ static noinline int update_ref_for_cow(struct 
btrfs_trans_handle *trans,
ret = btrfs_inc_ref(trans, root, cow, 1);
else
ret = btrfs_inc_ref(trans, root, cow, 0);
-   BUG_ON(ret); /* -ENOMEM */
+   if (ret)
+   return ret;
}
if (new_flags != 0) {
int level = btrfs_header_level(buf);
@@ -1068,9 +1072,11 @@ static noinline int update_ref_for_cow(struct 
btrfs_trans_handle *trans,
ret = btrfs_inc_ref(trans, root, cow, 1);
else
ret = btrfs_inc_ref(trans, root, cow, 0);
-   BUG_ON(ret); /* -ENOMEM */
+   if (ret)
+   return ret;
ret = btrfs_dec_ref(trans, root, buf, 1);
-   BUG_ON(ret); /* -ENOMEM */
+   if (ret)
+   return ret;
}
clean_tree_block(fs_info, buf);
*last_ref = 1;
-- 
2.14.2

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH] btrfs: pass fs_info to routines that always take tree_root

[jeffm]

From: Jeff Mahoney 

btrfs_find_root and btrfs_del_root always use the tree_root.  Let's pass
fs_info instead.

Signed-off-by: Jeff Mahoney 
---
 fs/btrfs/ctree.h   |  7 ---
 fs/btrfs/disk-io.c |  2 +-
 fs/btrfs/extent-tree.c |  4 ++--
 fs/btrfs/free-space-tree.c |  2 +-
 fs/btrfs/qgroup.c  |  3 +--
 fs/btrfs/root-tree.c   | 15 +--
 6 files changed, 18 insertions(+), 15 deletions(-)

diff --git a/fs/btrfs/ctree.h b/fs/btrfs/ctree.h
index 3f3eb7b17cac..eed7cc991a80 100644
--- a/fs/btrfs/ctree.h
+++ b/fs/btrfs/ctree.h
@@ -2973,8 +2973,8 @@ int btrfs_del_root_ref(struct btrfs_trans_handle *trans,
   struct btrfs_fs_info *fs_info,
   u64 root_id, u64 ref_id, u64 dirid, u64 *sequence,
   const char *name, int name_len);
-int btrfs_del_root(struct btrfs_trans_handle *trans, struct btrfs_root *root,
-  const struct btrfs_key *key);
+int btrfs_del_root(struct btrfs_trans_handle *trans,
+  struct btrfs_fs_info *fs_info, const struct btrfs_key *key);
 int btrfs_insert_root(struct btrfs_trans_handle *trans, struct btrfs_root 
*root,
  const struct btrfs_key *key,
  struct btrfs_root_item *item);
@@ -2982,7 +2982,8 @@ int __must_check btrfs_update_root(struct 
btrfs_trans_handle *trans,
   struct btrfs_root *root,
   struct btrfs_key *key,
   struct btrfs_root_item *item);
-int btrfs_find_root(struct btrfs_root *root, const struct btrfs_key 
*search_key,
+int btrfs_find_root(struct btrfs_fs_info *fs_info,
+   const struct btrfs_key *search_key,
struct btrfs_path *path, struct btrfs_root_item *root_item,
struct btrfs_key *root_key);
 int btrfs_find_orphan_roots(struct btrfs_fs_info *fs_info);
diff --git a/fs/btrfs/disk-io.c b/fs/btrfs/disk-io.c
index 080e2ebb8aa0..ea1959937875 100644
--- a/fs/btrfs/disk-io.c
+++ b/fs/btrfs/disk-io.c
@@ -1581,7 +1581,7 @@ static struct btrfs_root *btrfs_read_tree_root(struct 
btrfs_root *tree_root,
 
__setup_root(root, fs_info, key->objectid);
 
-   ret = btrfs_find_root(tree_root, key, path,
+   ret = btrfs_find_root(fs_info, key, path,
  >root_item, >root_key);
if (ret) {
if (ret > 0)
diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c
index 82d53a7b6652..12fa33accdcc 100644
--- a/fs/btrfs/extent-tree.c
+++ b/fs/btrfs/extent-tree.c
@@ -9192,14 +9192,14 @@ int btrfs_drop_snapshot(struct btrfs_root *root,
if (err)
goto out_end_trans;
 
-   ret = btrfs_del_root(trans, tree_root, >root_key);
+   ret = btrfs_del_root(trans, fs_info, >root_key);
if (ret) {
btrfs_abort_transaction(trans, ret);
goto out_end_trans;
}
 
if (root->root_key.objectid != BTRFS_TREE_RELOC_OBJECTID) {
-   ret = btrfs_find_root(tree_root, >root_key, path,
+   ret = btrfs_find_root(fs_info, >root_key, path,
  NULL, NULL);
if (ret < 0) {
btrfs_abort_transaction(trans, ret);
diff --git a/fs/btrfs/free-space-tree.c b/fs/btrfs/free-space-tree.c
index a5e34de06c2f..684f12247db7 100644
--- a/fs/btrfs/free-space-tree.c
+++ b/fs/btrfs/free-space-tree.c
@@ -1257,7 +1257,7 @@ int btrfs_clear_free_space_tree(struct btrfs_fs_info 
*fs_info)
if (ret)
goto abort;
 
-   ret = btrfs_del_root(trans, tree_root, _space_root->root_key);
+   ret = btrfs_del_root(trans, fs_info, _space_root->root_key);
if (ret)
goto abort;
 
diff --git a/fs/btrfs/qgroup.c b/fs/btrfs/qgroup.c
index 4ce351efe281..ba60523a443c 100644
--- a/fs/btrfs/qgroup.c
+++ b/fs/btrfs/qgroup.c
@@ -946,7 +946,6 @@ int btrfs_quota_enable(struct btrfs_trans_handle *trans,
 int btrfs_quota_disable(struct btrfs_trans_handle *trans,
struct btrfs_fs_info *fs_info)
 {
-   struct btrfs_root *tree_root = fs_info->tree_root;
struct btrfs_root *quota_root;
int ret = 0;
 
@@ -968,7 +967,7 @@ int btrfs_quota_disable(struct btrfs_trans_handle *trans,
if (ret)
goto out;
 
-   ret = btrfs_del_root(trans, tree_root, _root->root_key);
+   ret = btrfs_del_root(trans, fs_info, _root->root_key);
if (ret)
goto out;
 
diff --git a/fs/btrfs/root-tree.c b/fs/btrfs/root-tree.c
index 460db0cb2d07..31c0e7265f44 100644
--- a/fs/btrfs/root-tree.c
+++ b/fs/btrfs/root-tree.c
@@ -62,7 +62,7 @@ static void btrfs_read_root_item(struct extent_buffer *eb, 
int slot,
 
 /*
  * btrfs_find_root - lookup the root by the key.
- * root: the root of the root tree
+ * fs_info: the fs_info for the file system to search
  * search_key: the key to 

[PATCH 0/5 v2] btrfs-progs: convert fixes + reiserfs support

[jeffm]

From: Jeff Mahoney 

Changes since v1:
- reiserfs conversion:
  - use bool instead of int
  - catch 'impossible' condition of multiple discontiguous tails
  - properly handle hole followed by tail
  - add testing for combinations of real blocks, tails, and holes
  - print error indicating filename (and key) that caused a failure
  - constify buffer arg to btrfs_insert_inline_extent
  - add tails=on to reiserfs mount options
  - fixed absence of libreiserfscore to be non-fatal unless specificially
enabled

- btrfs-progs: convert: use search_cache_extent in migrate_one_reserved_range
  - In testing, we would not be able to roll back to part of the 0-1MB range
not being migrated.

- btrfs-progs: tests: fix typo in convert-tests/008-readonly-image
  - The test used ext2_save instead of ext2_saved as the filename

---

Jeff Mahoney (5):
  btrfs-progs: convert: properly handle reserved ranges while iterating
files
  btrfs-progs: convert: add missing newlines for printfs
  btrfs-progs: convert: use search_cache_extent in
migrate_one_reserved_range
  btrfs-progs: tests: fix typo in convert-tests/008-readonly-image
  btrfs-progs: convert: add support for converting reiserfs

 Makefile   |3 +-
 Makefile.inc.in|3 +-
 configure.ac   |   15 +-
 convert/main.c |   21 +-
 convert/source-fs.c|   25 +-
 convert/source-reiserfs.c  | 1015 
 convert/source-reiserfs.h  |  105 ++
 ctree.h|2 +-
 file-item.c|2 +-
 tests/common.convert   |   14 +-
 tests/convert-tests/008-readonly-image/test.sh |4 +-
 tests/convert-tests/010-reiserfs-basic/test.sh |   16 +
 .../011-reiserfs-delete-all-rollback/test.sh   |   67 ++
 .../012-reiserfs-large-hole-extent/test.sh |   23 +
 .../013-reiserfs-common-inode-flags/test.sh|   35 +
 .../convert-tests/014-reiserfs-tail-handling/input |1 +
 .../014-reiserfs-tail-handling/input2  |1 +
 .../014-reiserfs-tail-handling/test.sh |   69 ++
 18 files changed, 1391 insertions(+), 30 deletions(-)
 create mode 100644 convert/source-reiserfs.c
 create mode 100644 convert/source-reiserfs.h
 create mode 100755 tests/convert-tests/010-reiserfs-basic/test.sh
 create mode 100755 tests/convert-tests/011-reiserfs-delete-all-rollback/test.sh
 create mode 100755 tests/convert-tests/012-reiserfs-large-hole-extent/test.sh
 create mode 100755 tests/convert-tests/013-reiserfs-common-inode-flags/test.sh
 create mode 100644 tests/convert-tests/014-reiserfs-tail-handling/input
 create mode 100644 tests/convert-tests/014-reiserfs-tail-handling/input2
 create mode 100755 tests/convert-tests/014-reiserfs-tail-handling/test.sh

-- 
2.11.0

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 4/5] btrfs-progs: tests: fix typo in convert-tests/008-readonly-image

[jeffm]

From: Jeff Mahoney 

The dd in convert-tests/008-readonly-image is expected to fail, so
there being a typo in the file name has gone unnoticed.

Signed-off-by: Jeff Mahoney 
---
 tests/convert-tests/008-readonly-image/test.sh | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/tests/convert-tests/008-readonly-image/test.sh 
b/tests/convert-tests/008-readonly-image/test.sh
index 27c9373..cc846fa 100755
--- a/tests/convert-tests/008-readonly-image/test.sh
+++ b/tests/convert-tests/008-readonly-image/test.sh
@@ -14,9 +14,10 @@ convert_test_prep_fs $default_mke2fs
 run_check_umount_test_dev
 convert_test_do_convert
 run_check_mount_test_dev
+run_check e2fsck -n "$TEST_MNT/ext2_saved/image"
 
 # It's expected to fail
-$SUDO_HELPER dd if=/dev/zero of="$TEST_MNT/ext2_save/image" bs=1M count=1 \
+$SUDO_HELPER dd if=/dev/zero of="$TEST_MNT/ext2_saved/image" bs=1M count=1 \
&> /dev/null
 if [ $? -ne 1 ]; then
echo "after convert ext2_save/image is not read-only"
@@ -24,3 +25,4 @@ if [ $? -ne 1 ]; then
 fi
 run_check_umount_test_dev
 convert_test_post_rollback
+
-- 
2.11.0

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 5/5] btrfs-progs: convert: add support for converting reiserfs

[jeffm]

From: Jeff Mahoney 

This patch adds support to convert reiserfs file systems in-place to btrfs.

It will convert extended attribute files to btrfs extended attributes,
translate ACLs, coalesce tails that consist of multiple items into one item,
and convert tails that are too big into indirect files.

This requires that libreiserfscore 3.6.27 be available.

Many of the test cases for convert apply regardless of what the source
file system is and using ext4 is sufficient.  I've included several
test cases that are reiserfs-specific.

Signed-off-by: Jeff Mahoney 
---
 Makefile   |3 +-
 Makefile.inc.in|3 +-
 configure.ac   |   15 +-
 convert/main.c |   13 +-
 convert/source-reiserfs.c  | 1015 
 convert/source-reiserfs.h  |  105 ++
 ctree.h|2 +-
 file-item.c|2 +-
 tests/common.convert   |   14 +-
 tests/convert-tests/010-reiserfs-basic/test.sh |   16 +
 .../011-reiserfs-delete-all-rollback/test.sh   |   67 ++
 .../012-reiserfs-large-hole-extent/test.sh |   23 +
 .../013-reiserfs-common-inode-flags/test.sh|   35 +
 .../014-reiserfs-tail-handling/test.sh |   74 ++
 14 files changed, 1375 insertions(+), 12 deletions(-)
 create mode 100644 convert/source-reiserfs.c
 create mode 100644 convert/source-reiserfs.h
 create mode 100755 tests/convert-tests/010-reiserfs-basic/test.sh
 create mode 100755 tests/convert-tests/011-reiserfs-delete-all-rollback/test.sh
 create mode 100755 tests/convert-tests/012-reiserfs-large-hole-extent/test.sh
 create mode 100755 tests/convert-tests/013-reiserfs-common-inode-flags/test.sh
 create mode 100755 tests/convert-tests/014-reiserfs-tail-handling/test.sh

diff --git a/Makefile b/Makefile
index 81598df..f7f6dab 100644
--- a/Makefile
+++ b/Makefile
@@ -111,7 +111,7 @@ libbtrfs_headers = send-stream.h send-utils.h send.h 
kernel-lib/rbtree.h btrfs-l
   kernel-lib/radix-tree.h kernel-lib/sizes.h extent-cache.h \
   extent_io.h ioctl.h ctree.h btrfsck.h version.h
 convert_objects = convert/main.o convert/common.o convert/source-fs.o \
- convert/source-ext2.o
+ convert/source-ext2.o convert/source-reiserfs.o
 mkfs_objects = mkfs/main.o mkfs/common.o
 
 TESTS = fsck-tests.sh convert-tests.sh
@@ -188,6 +188,7 @@ endif
 # external libs required by various binaries; for btrfs-foo,
 # specify btrfs_foo_libs = ; see $($(subst...)) rules below
 btrfs_convert_cflags = -DBTRFSCONVERT_EXT2=$(BTRFSCONVERT_EXT2)
+btrfs_convert_cflags += -DBTRFSCONVERT_REISERFS=$(BTRFSCONVERT_REISERFS)
 btrfs_fragments_libs = -lgd -lpng -ljpeg -lfreetype
 btrfs_debug_tree_objects = cmds-inspect-dump-tree.o
 btrfs_show_super_objects = cmds-inspect-dump-super.o
diff --git a/Makefile.inc.in b/Makefile.inc.in
index 4e1b68c..3c7bc03 100644
--- a/Makefile.inc.in
+++ b/Makefile.inc.in
@@ -12,6 +12,7 @@ INSTALL = @INSTALL@
 DISABLE_DOCUMENTATION = @DISABLE_DOCUMENTATION@
 DISABLE_BTRFSCONVERT = @DISABLE_BTRFSCONVERT@
 BTRFSCONVERT_EXT2 = @BTRFSCONVERT_EXT2@
+BTRFSCONVERT_REISERFS = @BTRFSCONVERT_REISERFS@
 
 SUBST_CFLAGS = @CFLAGS@
 SUBST_LDFLAGS = @LDFLAGS@
@@ -31,6 +32,6 @@ udevruledir = ${udevdir}/rules.d
 
 # external libs required by various binaries; for btrfs-foo,
 # specify btrfs_foo_libs = ; see $($(subst...)) rules in Makefile
-btrfs_convert_libs = @EXT2FS_LIBS@ @COM_ERR_LIBS@
+btrfs_convert_libs = @EXT2FS_LIBS@ @COM_ERR_LIBS@ @REISERFS_LIBS@
 
 MAKEFILE_INC_INCLUDED = yes
diff --git a/configure.ac b/configure.ac
index 30055f8..27d875b 100644
--- a/configure.ac
+++ b/configure.ac
@@ -120,6 +120,7 @@ fi
 
 convertfs=
 BTRFSCONVERT_EXT2=0
+BTRFSCONVERT_REISERFS=0
 if test "x$enable_convert" = xyes; then
if test "x$with_convert" = "xauto" || echo "$with_convert" | grep -q 
"ext2"; then
PKG_CHECK_MODULES(EXT2FS, [ext2fs >= 1.42],,
@@ -131,11 +132,23 @@ if test "x$enable_convert" = xyes; then
convertfs="${convertfs:+$convertfs,}ext2"
BTRFSCONVERT_EXT2=1
fi
+   if test "x$with_convert" = "xauto"; then
+   PKG_CHECK_MODULES(REISERFS, [reiserfscore >= 3.6.27],
+ [BTRFSCONVERT_REISERFS=1],
+ [BTRFSCONVERT_REISERFS=0])
+   elif echo "$with_convert" | grep -q "reiserfs"; then
+   PKG_CHECK_MODULES(REISERFS, [reiserfscore >= 3.6.27],
+ [BTRFSCONVERT_REISERFS=1],[])
+   fi
+   if test "$BTRFSCONVERT_REISERFS" = 1; then
+   convertfs="${convertfs:+$convertfs,}reiserfs"
+   fi
 fi
 AC_SUBST([BTRFSCONVERT_EXT2])
+AC_SUBST([BTRFSCONVERT_REISERFS])
 
 # catch 

[PATCH 3/5] btrfs-progs: convert: use search_cache_extent in migrate_one_reserved_range

[jeffm]

From: Jeff Mahoney 

When we are looking for extents in migrate_one_reserved_range, it's likely
that there will be multiple extents that fall into the 0-1MB range.

If lookup_cache_extent is called with a range that covers multiple cache
entries, it will return the first entry it encounters while searching
from the top of the tree that happens to fall in that range.  That
means that we can end up skipping regions within that range, resulting
in a file system image that can't be rolled back since it wasn't
all migrated properly.

This is reproducible using convert-tests/008-readonly-image.  There was
a range from 0-160kB, but the only entry that was returned began at
~ 280kB.

The fix is to use search_cache_extent to iterate through multiple regions
within that range.

Signed-off-by: Jeff Mahoney 
---
 convert/main.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/convert/main.c b/convert/main.c
index 01657a6..24ed1de 100644
--- a/convert/main.c
+++ b/convert/main.c
@@ -340,10 +340,12 @@ static int migrate_one_reserved_range(struct 
btrfs_trans_handle *trans,
 * migrate ranges that covered by old fs data.
 */
while (cur_off < range_end(range)) {
-   cache = lookup_cache_extent(used, cur_off, cur_len);
+   cache = search_cache_extent(used, cur_off);
if (!cache)
break;
cur_off = max(cache->start, cur_off);
+   if (cur_off >= range_end(range))
+   break;
cur_len = min(cache->start + cache->size, range_end(range)) -
  cur_off;
BUG_ON(cur_len < root->sectorsize);
-- 
2.11.0

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 1/5] btrfs-progs: convert: properly handle reserved ranges while iterating files

[jeffm]

From: Jeff Mahoney 

Commit 522ef705e38 (btrfs-progs: convert: Introduce function to calculate
the available space) changed how we handle migrating file data so that
we never have btrfs space associated with the reserved ranges.  This
works pretty well and when we iterate over the file blocks, the
associations are redirected to the migrated locations.

This commit missed the case in block_iterate_proc where we just check
for intersection with a superblock location before looking up a block
group.  intersect_with_sb checks to see if the range intersects with
a stripe containing a superblock but, in fact, we've reserved the
full 0-1MB range at the start of the disk.  So a file block located
at e.g. 160kB will fall in the reserved region but won't be excepted
in block_iterate_block.  We ultimately hit a BUG_ON when we fail
to look up the block group for that location.

This is reproducible using convert-tests/003-ext4-basic.

The fix is to have intersect_with_sb and block_iterate_proc understand
the full size of the reserved ranges.  Since we use the range to
determine the boundary for the block iterator, let's just return the
boundary.  0 isn't a valid boundary and means that we proceed normally
with block group lookup.

Signed-off-by: Jeff Mahoney 
---
 convert/source-fs.c | 25 +++--
 1 file changed, 11 insertions(+), 14 deletions(-)

diff --git a/convert/source-fs.c b/convert/source-fs.c
index 80e4e41..09f6995 100644
--- a/convert/source-fs.c
+++ b/convert/source-fs.c
@@ -28,18 +28,16 @@ const struct simple_range btrfs_reserved_ranges[3] = {
{ BTRFS_SB_MIRROR_OFFSET(2), SZ_64K }
 };
 
-static int intersect_with_sb(u64 bytenr, u64 num_bytes)
+static u64 intersect_with_reserved(u64 bytenr, u64 num_bytes)
 {
int i;
-   u64 offset;
 
-   for (i = 0; i < BTRFS_SUPER_MIRROR_MAX; i++) {
-   offset = btrfs_sb_offset(i);
-   offset &= ~((u64)BTRFS_STRIPE_LEN - 1);
+   for (i = 0; i < ARRAY_SIZE(btrfs_reserved_ranges); i++) {
+   const struct simple_range *range = _reserved_ranges[i];
 
-   if (bytenr < offset + BTRFS_STRIPE_LEN &&
-   bytenr + num_bytes > offset)
-   return 1;
+   if (bytenr < range_end(range) &&
+   bytenr + num_bytes >= range->start)
+   return range_end(range);
}
return 0;
 }
@@ -64,14 +62,14 @@ int block_iterate_proc(u64 disk_block, u64 file_block,
  struct blk_iterate_data *idata)
 {
int ret = 0;
-   int sb_region;
+   u64 reserved_boundary;
int do_barrier;
struct btrfs_root *root = idata->root;
struct btrfs_block_group_cache *cache;
u64 bytenr = disk_block * root->sectorsize;
 
-   sb_region = intersect_with_sb(bytenr, root->sectorsize);
-   do_barrier = sb_region || disk_block >= idata->boundary;
+   reserved_boundary = intersect_with_reserved(bytenr, root->sectorsize);
+   do_barrier = reserved_boundary || disk_block >= idata->boundary;
if ((idata->num_blocks > 0 && do_barrier) ||
(file_block > idata->first_block + idata->num_blocks) ||
(disk_block != idata->disk_block + idata->num_blocks)) {
@@ -91,9 +89,8 @@ int block_iterate_proc(u64 disk_block, u64 file_block,
goto fail;
}
 
-   if (sb_region) {
-   bytenr += BTRFS_STRIPE_LEN - 1;
-   bytenr &= ~((u64)BTRFS_STRIPE_LEN - 1);
+   if (reserved_boundary) {
+   bytenr = reserved_boundary;
} else {
cache = btrfs_lookup_block_group(root->fs_info, bytenr);
BUG_ON(!cache);
-- 
2.11.0

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 2/5] btrfs-progs: convert: add missing newlines for printfs

[jeffm]

From: Jeff Mahoney 

There are two printfs with missing newlines that end up making the
output wonky.

Signed-off-by: Jeff Mahoney 
---
 convert/main.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/convert/main.c b/convert/main.c
index c56382e..01657a6 100644
--- a/convert/main.c
+++ b/convert/main.c
@@ -1285,7 +1285,7 @@ static int do_convert(const char *devname, u32 
convert_flags, u32 nodesize,
goto fail;
}
 
-   printf("creating btrfs metadata");
+   printf("creating btrfs metadata\n");
ctx.max_copy_inodes = (cctx.inodes_count - cctx.free_inodes_count);
ctx.cur_copy_inodes = 0;
 
@@ -1348,7 +1348,7 @@ static int do_convert(const char *devname, u32 
convert_flags, u32 nodesize,
close_ctree(root);
close(fd);
 
-   printf("conversion complete");
+   printf("conversion complete\n");
return 0;
 fail:
clean_convert_context();
-- 
2.11.0

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 1/3] btrfs-progs: convert: properly handle reserved ranges while iterating files

[jeffm]

From: Jeff Mahoney 

Commit 522ef705e38 (btrfs-progs: convert: Introduce function to calculate
the available space) changed how we handle migrating file data so that
we never have btrfs space associated with the reserved ranges.  This
works pretty well and when we iterate over the file blocks, the
associations are redirected to the migrated locations.

This commit missed the case in block_iterate_proc where we just check
for intersection with a superblock location before looking up a block
group.  intersect_with_sb checks to see if the range intersects with
a stripe containing a superblock but, in fact, we've reserved the
full 0-1MB range at the start of the disk.  So a file block located
at e.g. 160kB will fall in the reserved region but won't be excepted
in block_iterate_block.  We ultimately hit a BUG_ON when we fail
to look up the block group for that location.

This is reproducible using convert-tests/003-ext4-basic.

The fix is to have intersect_with_sb and block_iterate_proc understand
the full size of the reserved ranges.  Since we use the range to
determine the boundary for the block iterator, let's just return the
boundary.  0 isn't a valid boundary and means that we proceed normally
with block group lookup.

Cc: Qu Wenruo 
Signed-off-by: Jeff Mahoney 
---
 convert/source-fs.c | 25 +++--
 1 file changed, 11 insertions(+), 14 deletions(-)

diff --git a/convert/source-fs.c b/convert/source-fs.c
index 80e4e41..09f6995 100644
--- a/convert/source-fs.c
+++ b/convert/source-fs.c
@@ -28,18 +28,16 @@ const struct simple_range btrfs_reserved_ranges[3] = {
{ BTRFS_SB_MIRROR_OFFSET(2), SZ_64K }
 };
 
-static int intersect_with_sb(u64 bytenr, u64 num_bytes)
+static u64 intersect_with_reserved(u64 bytenr, u64 num_bytes)
 {
int i;
-   u64 offset;
 
-   for (i = 0; i < BTRFS_SUPER_MIRROR_MAX; i++) {
-   offset = btrfs_sb_offset(i);
-   offset &= ~((u64)BTRFS_STRIPE_LEN - 1);
+   for (i = 0; i < ARRAY_SIZE(btrfs_reserved_ranges); i++) {
+   const struct simple_range *range = _reserved_ranges[i];
 
-   if (bytenr < offset + BTRFS_STRIPE_LEN &&
-   bytenr + num_bytes > offset)
-   return 1;
+   if (bytenr < range_end(range) &&
+   bytenr + num_bytes >= range->start)
+   return range_end(range);
}
return 0;
 }
@@ -64,14 +62,14 @@ int block_iterate_proc(u64 disk_block, u64 file_block,
  struct blk_iterate_data *idata)
 {
int ret = 0;
-   int sb_region;
+   u64 reserved_boundary;
int do_barrier;
struct btrfs_root *root = idata->root;
struct btrfs_block_group_cache *cache;
u64 bytenr = disk_block * root->sectorsize;
 
-   sb_region = intersect_with_sb(bytenr, root->sectorsize);
-   do_barrier = sb_region || disk_block >= idata->boundary;
+   reserved_boundary = intersect_with_reserved(bytenr, root->sectorsize);
+   do_barrier = reserved_boundary || disk_block >= idata->boundary;
if ((idata->num_blocks > 0 && do_barrier) ||
(file_block > idata->first_block + idata->num_blocks) ||
(disk_block != idata->disk_block + idata->num_blocks)) {
@@ -91,9 +89,8 @@ int block_iterate_proc(u64 disk_block, u64 file_block,
goto fail;
}
 
-   if (sb_region) {
-   bytenr += BTRFS_STRIPE_LEN - 1;
-   bytenr &= ~((u64)BTRFS_STRIPE_LEN - 1);
+   if (reserved_boundary) {
+   bytenr = reserved_boundary;
} else {
cache = btrfs_lookup_block_group(root->fs_info, bytenr);
BUG_ON(!cache);
-- 
2.11.0

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 3/3] btrfs-progs: convert: add support for converting reiserfs

[jeffm]

From: Jeff Mahoney 

This patch adds support to convert reiserfs file systems in-place to btrfs.

It will convert extended attribute files to btrfs extended attributes,
translate ACLs, coalesce tails that consist of multiple items into one item,
and convert tails that are too big into indirect files.

This requires that libreiserfscore 3.6.27 be available.

Many of the test cases for convert apply regardless of what the source
file system is and using ext4 is sufficient.  I've included several
test cases that are reiserfs-specific.

Signed-off-by: Jeff Mahoney 
---
 Makefile   |3 +-
 Makefile.inc.in|3 +-
 configure.ac   |   10 +-
 convert/main.c |   13 +-
 convert/source-reiserfs.c  | 1011 
 convert/source-reiserfs.h  |  105 ++
 tests/common.convert   |   14 +-
 tests/convert-tests/010-reiserfs-basic/test.sh |   16 +
 .../011-reiserfs-delete-all-rollback/test.sh   |   67 ++
 .../012-reiserfs-large-hole-extent/test.sh |   23 +
 .../013-reiserfs-common-inode-flags/test.sh|   35 +
 11 files changed, 1290 insertions(+), 10 deletions(-)
 create mode 100644 convert/source-reiserfs.c
 create mode 100644 convert/source-reiserfs.h
 create mode 100755 tests/convert-tests/010-reiserfs-basic/test.sh
 create mode 100755 tests/convert-tests/011-reiserfs-delete-all-rollback/test.sh
 create mode 100755 tests/convert-tests/012-reiserfs-large-hole-extent/test.sh
 create mode 100755 tests/convert-tests/013-reiserfs-common-inode-flags/test.sh

diff --git a/Makefile b/Makefile
index 81598df..f7f6dab 100644
--- a/Makefile
+++ b/Makefile
@@ -111,7 +111,7 @@ libbtrfs_headers = send-stream.h send-utils.h send.h 
kernel-lib/rbtree.h btrfs-l
   kernel-lib/radix-tree.h kernel-lib/sizes.h extent-cache.h \
   extent_io.h ioctl.h ctree.h btrfsck.h version.h
 convert_objects = convert/main.o convert/common.o convert/source-fs.o \
- convert/source-ext2.o
+ convert/source-ext2.o convert/source-reiserfs.o
 mkfs_objects = mkfs/main.o mkfs/common.o
 
 TESTS = fsck-tests.sh convert-tests.sh
@@ -188,6 +188,7 @@ endif
 # external libs required by various binaries; for btrfs-foo,
 # specify btrfs_foo_libs = ; see $($(subst...)) rules below
 btrfs_convert_cflags = -DBTRFSCONVERT_EXT2=$(BTRFSCONVERT_EXT2)
+btrfs_convert_cflags += -DBTRFSCONVERT_REISERFS=$(BTRFSCONVERT_REISERFS)
 btrfs_fragments_libs = -lgd -lpng -ljpeg -lfreetype
 btrfs_debug_tree_objects = cmds-inspect-dump-tree.o
 btrfs_show_super_objects = cmds-inspect-dump-super.o
diff --git a/Makefile.inc.in b/Makefile.inc.in
index 4e1b68c..3c7bc03 100644
--- a/Makefile.inc.in
+++ b/Makefile.inc.in
@@ -12,6 +12,7 @@ INSTALL = @INSTALL@
 DISABLE_DOCUMENTATION = @DISABLE_DOCUMENTATION@
 DISABLE_BTRFSCONVERT = @DISABLE_BTRFSCONVERT@
 BTRFSCONVERT_EXT2 = @BTRFSCONVERT_EXT2@
+BTRFSCONVERT_REISERFS = @BTRFSCONVERT_REISERFS@
 
 SUBST_CFLAGS = @CFLAGS@
 SUBST_LDFLAGS = @LDFLAGS@
@@ -31,6 +32,6 @@ udevruledir = ${udevdir}/rules.d
 
 # external libs required by various binaries; for btrfs-foo,
 # specify btrfs_foo_libs = ; see $($(subst...)) rules in Makefile
-btrfs_convert_libs = @EXT2FS_LIBS@ @COM_ERR_LIBS@
+btrfs_convert_libs = @EXT2FS_LIBS@ @COM_ERR_LIBS@ @REISERFS_LIBS@
 
 MAKEFILE_INC_INCLUDED = yes
diff --git a/configure.ac b/configure.ac
index 30055f8..3a8bd3f 100644
--- a/configure.ac
+++ b/configure.ac
@@ -120,6 +120,7 @@ fi
 
 convertfs=
 BTRFSCONVERT_EXT2=0
+BTRFSCONVERT_REISERFS=0
 if test "x$enable_convert" = xyes; then
if test "x$with_convert" = "xauto" || echo "$with_convert" | grep -q 
"ext2"; then
PKG_CHECK_MODULES(EXT2FS, [ext2fs >= 1.42],,
@@ -131,11 +132,18 @@ if test "x$enable_convert" = xyes; then
convertfs="${convertfs:+$convertfs,}ext2"
BTRFSCONVERT_EXT2=1
fi
+   if test "x$with_convert" = "xauto" || echo "$with_convert" | grep -q 
"reiserfs"; then
+   PKG_CHECK_MODULES(REISERFS, [reiserfscore >= 3.6.27],
+ [BTRFSCONVERT_REISERFS=1],[])
+   convertfs="${convertfs:+$convertfs,}reiserfs"
+   BTRFSCONVERT_REISERFS=1
+   fi
 fi
 AC_SUBST([BTRFSCONVERT_EXT2])
+AC_SUBST([BTRFSCONVERT_REISERFS])
 
 # catch typos
-tmp=$(echo "$with_convert" | sed -e 's/auto//' | sed -e 's/ext2//' | sed -e 
's/,\+/,/')
+tmp=$(echo "$with_convert" | sed -e 's/auto//' | sed -e 's/ext2//' | sed -e 
's/reiserfs//' | sed -e 's/,\+/,/')
 if ! test "x$tmp" = "x"; then
AC_MSG_ERROR([unknown tokens for --with-convert: $tmp])
 fi
diff --git a/convert/main.c b/convert/main.c
index 01657a6..bc459a4 100644
--- a/convert/main.c
+++ b/convert/main.c
@@ -102,12 +102,16 @@
 #include "convert/source-fs.h"
 #include 

[PATCH 2/3] btrfs-progs: convert: add missing newlines for printfs

[jeffm]

From: Jeff Mahoney 

There are two printfs with missing newlines that end up making the
output wonky.

Signed-off-by: Jeff Mahoney 
---
 convert/main.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/convert/main.c b/convert/main.c
index c56382e..01657a6 100644
--- a/convert/main.c
+++ b/convert/main.c
@@ -1285,7 +1285,7 @@ static int do_convert(const char *devname, u32 
convert_flags, u32 nodesize,
goto fail;
}
 
-   printf("creating btrfs metadata");
+   printf("creating btrfs metadata\n");
ctx.max_copy_inodes = (cctx.inodes_count - cctx.free_inodes_count);
ctx.cur_copy_inodes = 0;
 
@@ -1348,7 +1348,7 @@ static int do_convert(const char *devname, u32 
convert_flags, u32 nodesize,
close_ctree(root);
close(fd);
 
-   printf("conversion complete");
+   printf("conversion complete\n");
return 0;
 fail:
clean_convert_context();
-- 
2.11.0

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 1/7] btrfs-progs: check: supplement extent backref list with rbtree

[jeffm]

From: Jeff Mahoney 

For the pathlogical case, like xfstests generic/297 that creates a
large file consisting of one, repeating reflinked extent, fsck can
take hours.  The root cause is that calling find_data_backref while
iterating the extent records is an O(n^2) algorithm.  For my
example test run, n was 2*2^20 and fsck was at 8 hours and counting.

This patch supplements the list with an rbtree and drops the runtime
of that testcase to about 20 seconds.

A previous version of this patch introduced a regression that would
have corrupted file systems during repair.  It was traced to the
compare algorithm honoring ->bytes regardless of whether the
reference had been found and a failure to reinsert nodes after
the target reference was found.

Signed-off-by: Jeff Mahoney 
---
 cmds-check.c | 184 +--
 1 file changed, 180 insertions(+), 4 deletions(-)

diff --git a/cmds-check.c b/cmds-check.c
index 23adc03..6a04553 100644
--- a/cmds-check.c
+++ b/cmds-check.c
@@ -87,6 +87,7 @@ static enum btrfs_check_mode check_mode = CHECK_MODE_DEFAULT;
 
 struct extent_backref {
struct list_head list;
+   struct rb_node node;
unsigned int is_data:1;
unsigned int found_extent_tree:1;
unsigned int full_backref:1;
@@ -99,6 +100,11 @@ static inline struct extent_backref* 
to_extent_backref(struct list_head *entry)
return list_entry(entry, struct extent_backref, list);
 }
 
+static inline struct extent_backref* rb_node_to_extent_backref(struct rb_node 
*node)
+{
+   return rb_entry(node, struct extent_backref, node);
+}
+
 struct data_backref {
struct extent_backref node;
union {
@@ -137,6 +143,51 @@ static inline struct data_backref* to_data_backref(struct 
extent_backref *back)
return container_of(back, struct data_backref, node);
 }
 
+static int compare_data_backref(struct rb_node *node1, struct rb_node *node2)
+{
+   struct extent_backref *ext1 = rb_node_to_extent_backref(node1);
+   struct extent_backref *ext2 = rb_node_to_extent_backref(node2);
+   struct data_backref *back1 = to_data_backref(ext1);
+   struct data_backref *back2 = to_data_backref(ext2);
+
+   WARN_ON(!ext1->is_data);
+   WARN_ON(!ext2->is_data);
+
+   /* parent and root are a union, so this covers both */
+   if (back1->parent > back2->parent)
+   return 1;
+   if (back1->parent < back2->parent)
+   return -1;
+
+   /* This is a full backref and the parents match. */
+   if (back1->node.full_backref)
+   return 0;
+
+   if (back1->owner > back2->owner)
+   return 1;
+   if (back1->owner < back2->owner)
+   return -1;
+
+   if (back1->offset > back2->offset)
+   return 1;
+   if (back1->offset < back2->offset)
+   return -1;
+
+   if (back1->found_ref && back2->found_ref) {
+   if (back1->disk_bytenr > back2->disk_bytenr)
+   return 1;
+   if (back1->disk_bytenr < back2->disk_bytenr)
+   return -1;
+
+   if (back1->bytes > back2->bytes)
+   return 1;
+   if (back1->bytes < back2->bytes)
+   return -1;
+   }
+
+   return 0;
+}
+
 /*
  * Much like data_backref, just removed the undetermined members
  * and change it to use list_head.
@@ -165,12 +216,54 @@ static inline struct tree_backref* to_tree_backref(struct 
extent_backref *back)
return container_of(back, struct tree_backref, node);
 }
 
+static int compare_tree_backref(struct rb_node *node1, struct rb_node *node2)
+{
+   struct extent_backref *ext1 = rb_node_to_extent_backref(node1);
+   struct extent_backref *ext2 = rb_node_to_extent_backref(node2);
+   struct tree_backref *back1 = to_tree_backref(ext1);
+   struct tree_backref *back2 = to_tree_backref(ext2);
+
+   WARN_ON(ext1->is_data);
+   WARN_ON(ext2->is_data);
+
+   /* parent and root are a union, so this covers both */
+   if (back1->parent > back2->parent)
+   return 1;
+   if (back1->parent < back2->parent)
+   return -1;
+
+   return 0;
+}
+
+static int compare_extent_backref(struct rb_node *node1, struct rb_node *node2)
+{
+   struct extent_backref *ext1 = rb_node_to_extent_backref(node1);
+   struct extent_backref *ext2 = rb_node_to_extent_backref(node2);
+
+   if (ext1->is_data > ext2->is_data)
+   return 1;
+
+   if (ext1->is_data < ext2->is_data)
+   return -1;
+
+   if (ext1->full_backref > ext2->full_backref)
+   return 1;
+   if (ext1->full_backref < ext2->full_backref)
+   return -1;
+
+   if (ext1->is_data)
+   return compare_data_backref(node1, node2);
+   else
+   return compare_tree_backref(node1, node2);
+}
+
 /*