Re: [PATCH 08/20] btrfs-progs: qgroups: introduce btrfs_qgroup_query
On 2018年03月08日 23:21, Jeff Mahoney wrote:
> On 3/8/18 12:54 AM, Qu Wenruo wrote:
>>
>>
>> On 2018年03月08日 10:40, je...@suse.com wrote:
>>> From: Jeff Mahoney
>>>
>>> The only mechanism we have in the progs for searching qgroups is to load
>>> all of them and filter the results. This works for qgroup show but
>>> to add quota information to 'btrfs subvoluem show' it's pretty wasteful.
>>>
>>> This patch splits out setting up the search and performing the search so
>>> we can search for a single qgroupid more easily. Since TREE_SEARCH
>>> will give results that don't strictly match the search terms, we add
>>> a filter to match only the results we care about.
>>>
>>> Signed-off-by: Jeff Mahoney
>>> ---
>>> qgroup.c | 143
>>> ---
>>> qgroup.h | 7
>>> 2 files changed, 116 insertions(+), 34 deletions(-)
>>>
>>> diff --git a/qgroup.c b/qgroup.c
>>> index 57815718..d076b1de 100644
>>> --- a/qgroup.c
>>> +++ b/qgroup.c
>>> @@ -1165,11 +1165,30 @@ static inline void print_status_flag_warning(u64
>>> flags)
>>> warning("qgroup data inconsistent, rescan recommended");
>>> }
>>>
>>> -static int __qgroups_search(int fd, struct qgroup_lookup *qgroup_lookup)
>>> +static bool key_in_range(const struct btrfs_key *key,
>>> +const struct btrfs_ioctl_search_key *sk)
>>> +{
>>> + if (key->objectid < sk->min_objectid ||
>>> + key->objectid > sk->max_objectid)
>>> + return false;
>>> +
>>> + if (key->type < sk->min_type ||
>>> + key->type > sk->max_type)
>>> + return false;
>>> +
>>> + if (key->offset < sk->min_offset ||
>>> + key->offset > sk->max_offset)
>>> + return false;
>>> +
>>> + return true;
>>> +}
>>
>> Even with the key_in_range() check here, we are still following the tree
>> search slice behavior:
>>
>> tree search will still gives us all the items in key range from
>> (min_objectid, min_type, min_offset) to
>> (max_objectid, max_type, max_offset).
>>
>> I don't see much different between the tree search ioctl and this one.
>
> It's fundamentally different.
>
> The one in the kernel has a silly interface. It should be min_key and
> max_key since the components aren't evaluated independently. It
> effectively treats min_key and max_key as 136-bit values and returns
> everything between them, inclusive. That's the slice behavior, as you
> call it.
>
> This key_in_range treats each component separately and acts as a filter
> on the slice returned from the kernel. If we request min/max_offset =
> 259, the caller will not get anything without offset = 259.
You're right.
I'm just an idiot, the separate check on objectid/type/offset breaks the
slice behavior so we won't found any thing unrelated.
So the current design is completely fine, even we try to query something
doesn't exist we won't have any noise.
Thanks for the filter and this really helps a lot.
Reviewed-by: Qu Wenruo
Thanks,
Qu
>
> I suppose, ultimately, this could also be done using a filter on the
> rbtree using the existing interface but that seems even more wasteful.
>
> -Jeff
>
>>> +
>>> +static int __qgroups_search(int fd, struct btrfs_ioctl_search_args *args,
>>> + struct qgroup_lookup *qgroup_lookup)
>>> {
>>> int ret;
>>> - struct btrfs_ioctl_search_args args;
>>> - struct btrfs_ioctl_search_key *sk = &args.key;
>>> + struct btrfs_ioctl_search_key *sk = &args->key;
>>> + struct btrfs_ioctl_search_key filter_key = args->key;
>>> struct btrfs_ioctl_search_header *sh;
>>> unsigned long off = 0;
>>> unsigned int i;
>>> @@ -1180,30 +1199,15 @@ static int __qgroups_search(int fd, struct
>>> qgroup_lookup *qgroup_lookup)
>>> u64 qgroupid;
>>> u64 qgroupid1;
>>>
>>> - memset(&args, 0, sizeof(args));
>>> -
>>> - sk->tree_id = BTRFS_QUOTA_TREE_OBJECTID;
>>> - sk->max_type = BTRFS_QGROUP_RELATION_KEY;
>>> - sk->min_type = BTRFS_QGROUP_STATUS_KEY;
>>> - sk->max_objectid = (u64)-1;
>>> - sk->max_offset = (u64)-1;
>>> - sk->max_transid = (u64)-1;
>>> - sk->nr_items = 4096;
>>> -
>>> qgroup_lookup_init(qgroup_lookup);
>>>
>>> while (1) {
>>> - ret = ioctl(fd, BTRFS_IOC_TREE_SEARCH, &args);
>>> + ret = ioctl(fd, BTRFS_IOC_TREE_SEARCH, args);
>>> if (ret < 0) {
>>> - if (errno == ENOENT) {
>>> - error("can't list qgroups: quotas not enabled");
>>> + if (errno == ENOENT)
>>> ret = -ENOTTY;
>>> - } else {
>>> - error("can't list qgroups: %s",
>>> - strerror(errno));
>>> + else
>>> ret = -errno;
>>> - }
>>> -
>>> break;
>>> }
>>>
>>> @@ -1217,37 +1221,46 @@ static int __qgroups_search(int fd, struct
>>> qgroup_loo
Re: [PATCH 08/20] btrfs-progs: qgroups: introduce btrfs_qgroup_query
On 3/8/18 12:54 AM, Qu Wenruo wrote:
>
>
> On 2018年03月08日 10:40, je...@suse.com wrote:
>> From: Jeff Mahoney
>>
>> The only mechanism we have in the progs for searching qgroups is to load
>> all of them and filter the results. This works for qgroup show but
>> to add quota information to 'btrfs subvoluem show' it's pretty wasteful.
>>
>> This patch splits out setting up the search and performing the search so
>> we can search for a single qgroupid more easily. Since TREE_SEARCH
>> will give results that don't strictly match the search terms, we add
>> a filter to match only the results we care about.
>>
>> Signed-off-by: Jeff Mahoney
>> ---
>> qgroup.c | 143
>> ---
>> qgroup.h | 7
>> 2 files changed, 116 insertions(+), 34 deletions(-)
>>
>> diff --git a/qgroup.c b/qgroup.c
>> index 57815718..d076b1de 100644
>> --- a/qgroup.c
>> +++ b/qgroup.c
>> @@ -1165,11 +1165,30 @@ static inline void print_status_flag_warning(u64
>> flags)
>> warning("qgroup data inconsistent, rescan recommended");
>> }
>>
>> -static int __qgroups_search(int fd, struct qgroup_lookup *qgroup_lookup)
>> +static bool key_in_range(const struct btrfs_key *key,
>> + const struct btrfs_ioctl_search_key *sk)
>> +{
>> +if (key->objectid < sk->min_objectid ||
>> +key->objectid > sk->max_objectid)
>> +return false;
>> +
>> +if (key->type < sk->min_type ||
>> +key->type > sk->max_type)
>> +return false;
>> +
>> +if (key->offset < sk->min_offset ||
>> +key->offset > sk->max_offset)
>> +return false;
>> +
>> +return true;
>> +}
>
> Even with the key_in_range() check here, we are still following the tree
> search slice behavior:
>
> tree search will still gives us all the items in key range from
> (min_objectid, min_type, min_offset) to
> (max_objectid, max_type, max_offset).
>
> I don't see much different between the tree search ioctl and this one.
It's fundamentally different.
The one in the kernel has a silly interface. It should be min_key and
max_key since the components aren't evaluated independently. It
effectively treats min_key and max_key as 136-bit values and returns
everything between them, inclusive. That's the slice behavior, as you
call it.
This key_in_range treats each component separately and acts as a filter
on the slice returned from the kernel. If we request min/max_offset =
259, the caller will not get anything without offset = 259.
I suppose, ultimately, this could also be done using a filter on the
rbtree using the existing interface but that seems even more wasteful.
-Jeff
>> +
>> +static int __qgroups_search(int fd, struct btrfs_ioctl_search_args *args,
>> +struct qgroup_lookup *qgroup_lookup)
>> {
>> int ret;
>> -struct btrfs_ioctl_search_args args;
>> -struct btrfs_ioctl_search_key *sk = &args.key;
>> +struct btrfs_ioctl_search_key *sk = &args->key;
>> +struct btrfs_ioctl_search_key filter_key = args->key;
>> struct btrfs_ioctl_search_header *sh;
>> unsigned long off = 0;
>> unsigned int i;
>> @@ -1180,30 +1199,15 @@ static int __qgroups_search(int fd, struct
>> qgroup_lookup *qgroup_lookup)
>> u64 qgroupid;
>> u64 qgroupid1;
>>
>> -memset(&args, 0, sizeof(args));
>> -
>> -sk->tree_id = BTRFS_QUOTA_TREE_OBJECTID;
>> -sk->max_type = BTRFS_QGROUP_RELATION_KEY;
>> -sk->min_type = BTRFS_QGROUP_STATUS_KEY;
>> -sk->max_objectid = (u64)-1;
>> -sk->max_offset = (u64)-1;
>> -sk->max_transid = (u64)-1;
>> -sk->nr_items = 4096;
>> -
>> qgroup_lookup_init(qgroup_lookup);
>>
>> while (1) {
>> -ret = ioctl(fd, BTRFS_IOC_TREE_SEARCH, &args);
>> +ret = ioctl(fd, BTRFS_IOC_TREE_SEARCH, args);
>> if (ret < 0) {
>> -if (errno == ENOENT) {
>> -error("can't list qgroups: quotas not enabled");
>> +if (errno == ENOENT)
>> ret = -ENOTTY;
>> -} else {
>> -error("can't list qgroups: %s",
>> - strerror(errno));
>> +else
>> ret = -errno;
>> -}
>> -
>> break;
>> }
>>
>> @@ -1217,37 +1221,46 @@ static int __qgroups_search(int fd, struct
>> qgroup_lookup *qgroup_lookup)
>> * read the root_ref item it contains
>> */
>> for (i = 0; i < sk->nr_items; i++) {
>> -sh = (struct btrfs_ioctl_search_header *)(args.buf +
>> +struct btrfs_key key;
>> +
>> +sh = (struct btrfs_ioctl_search_header *)(args->buf +
>>off);
>> off += sizeof(*sh);
>>
>> -
Re: [PATCH 08/20] btrfs-progs: qgroups: introduce btrfs_qgroup_query
On 2018年03月08日 10:40, je...@suse.com wrote:
> From: Jeff Mahoney
>
> The only mechanism we have in the progs for searching qgroups is to load
> all of them and filter the results. This works for qgroup show but
> to add quota information to 'btrfs subvoluem show' it's pretty wasteful.
>
> This patch splits out setting up the search and performing the search so
> we can search for a single qgroupid more easily. Since TREE_SEARCH
> will give results that don't strictly match the search terms, we add
> a filter to match only the results we care about.
>
> Signed-off-by: Jeff Mahoney
> ---
> qgroup.c | 143
> ---
> qgroup.h | 7
> 2 files changed, 116 insertions(+), 34 deletions(-)
>
> diff --git a/qgroup.c b/qgroup.c
> index 57815718..d076b1de 100644
> --- a/qgroup.c
> +++ b/qgroup.c
> @@ -1165,11 +1165,30 @@ static inline void print_status_flag_warning(u64
> flags)
> warning("qgroup data inconsistent, rescan recommended");
> }
>
> -static int __qgroups_search(int fd, struct qgroup_lookup *qgroup_lookup)
> +static bool key_in_range(const struct btrfs_key *key,
> + const struct btrfs_ioctl_search_key *sk)
> +{
> + if (key->objectid < sk->min_objectid ||
> + key->objectid > sk->max_objectid)
> + return false;
> +
> + if (key->type < sk->min_type ||
> + key->type > sk->max_type)
> + return false;
> +
> + if (key->offset < sk->min_offset ||
> + key->offset > sk->max_offset)
> + return false;
> +
> + return true;
> +}
Even with the key_in_range() check here, we are still following the tree
search slice behavior:
tree search will still gives us all the items in key range from
(min_objectid, min_type, min_offset) to
(max_objectid, max_type, max_offset).
I don't see much different between the tree search ioctl and this one.
> +
> +static int __qgroups_search(int fd, struct btrfs_ioctl_search_args *args,
> + struct qgroup_lookup *qgroup_lookup)
> {
> int ret;
> - struct btrfs_ioctl_search_args args;
> - struct btrfs_ioctl_search_key *sk = &args.key;
> + struct btrfs_ioctl_search_key *sk = &args->key;
> + struct btrfs_ioctl_search_key filter_key = args->key;
> struct btrfs_ioctl_search_header *sh;
> unsigned long off = 0;
> unsigned int i;
> @@ -1180,30 +1199,15 @@ static int __qgroups_search(int fd, struct
> qgroup_lookup *qgroup_lookup)
> u64 qgroupid;
> u64 qgroupid1;
>
> - memset(&args, 0, sizeof(args));
> -
> - sk->tree_id = BTRFS_QUOTA_TREE_OBJECTID;
> - sk->max_type = BTRFS_QGROUP_RELATION_KEY;
> - sk->min_type = BTRFS_QGROUP_STATUS_KEY;
> - sk->max_objectid = (u64)-1;
> - sk->max_offset = (u64)-1;
> - sk->max_transid = (u64)-1;
> - sk->nr_items = 4096;
> -
> qgroup_lookup_init(qgroup_lookup);
>
> while (1) {
> - ret = ioctl(fd, BTRFS_IOC_TREE_SEARCH, &args);
> + ret = ioctl(fd, BTRFS_IOC_TREE_SEARCH, args);
> if (ret < 0) {
> - if (errno == ENOENT) {
> - error("can't list qgroups: quotas not enabled");
> + if (errno == ENOENT)
> ret = -ENOTTY;
> - } else {
> - error("can't list qgroups: %s",
> -strerror(errno));
> + else
> ret = -errno;
> - }
> -
> break;
> }
>
> @@ -1217,37 +1221,46 @@ static int __qgroups_search(int fd, struct
> qgroup_lookup *qgroup_lookup)
>* read the root_ref item it contains
>*/
> for (i = 0; i < sk->nr_items; i++) {
> - sh = (struct btrfs_ioctl_search_header *)(args.buf +
> + struct btrfs_key key;
> +
> + sh = (struct btrfs_ioctl_search_header *)(args->buf +
> off);
> off += sizeof(*sh);
>
> - switch (btrfs_search_header_type(sh)) {
> + key.objectid = btrfs_search_header_objectid(sh);
> + key.type = btrfs_search_header_type(sh);
> + key.offset = btrfs_search_header_offset(sh);
> +
> + if (!key_in_range(&key, &filter_key))
> + goto next;
It still looks like that most other qgroup info will get calculated.
> +
> + switch (key.type) {
> case BTRFS_QGROUP_STATUS_KEY:
> si = (struct btrfs_qgroup_status_item *)
> - (args.buf + off);
> + (args->buf + off);
> flags = btrfs
