Re: [PATCH] btrfs-progs: Check factor out root parsing from check_chunks_and_extents

2018-06-18 Thread Nikolay Borisov
On 19.06.2018 05:18, Su Yue wrote: > > > On 06/18/2018 07:10 PM, Nikolay Borisov wrote: >> check_chunks_and_extents does quite a number of distinct things. The >> first of those is going through all root items in the root tree and >> classify every root depending on whether they have a

Re: [PATCH 2/3] btrfs: Document __btrfs_inc_extent_ref

2018-06-18 Thread Qu Wenruo
On 2018年06月18日 19:59, Nikolay Borisov wrote: > Here is a doc-only patch which tires to deobfuscate the terra-incognita > that arguments for delayed refs are. > > Signed-off-by: Nikolay Borisov Reviewed-by: Qu Wenruo > --- > fs/btrfs/extent-tree.c | 34 ++ >

Re: [PATCH 1/3] btrfs: Remove fs_info argument from __btrfs_inc_extent_ref

2018-06-18 Thread Qu Wenruo
On 2018年06月18日 19:59, Nikolay Borisov wrote: > This function already takes a transaction which holds a reference to > the fs_info struct. Use that reference and remove the extra arg. No > functional changes. > > Signed-off-by: Nikolay Borisov Reviewed-by: Qu Wenruo Thanks, Qu > --- >

[PATCH] btrfs-progs: Fix wrong optind re-initialization to allow mixed option and non-option

2018-06-18 Thread Qu Wenruo
In function handle_global_options(), we reset @optind to 1. However according to man page of getopt(3) NOTES section, if we need to rescan options later, @optind should be reset to 0 to initialize the internal variables correctly. This explains the reason why in cmd_check(), getopt_long() doesn't

Re: About more loose parameter sequence requirement

2018-06-18 Thread Qu Wenruo
On 2018年06月19日 12:34, Qu Wenruo wrote: > > > On 2018年06月18日 20:02, David Sterba wrote: >> On Mon, Jun 18, 2018 at 07:40:44PM +0800, Qu Wenruo wrote: >>> >>> >>> On 2018年06月18日 19:34, David Sterba wrote: On Thu, Jun 14, 2018 at 03:17:45PM +0800, Qu Wenruo wrote: > I understand that

Re: About more loose parameter sequence requirement

2018-06-18 Thread Qu Wenruo
On 2018年06月18日 20:02, David Sterba wrote: > On Mon, Jun 18, 2018 at 07:40:44PM +0800, Qu Wenruo wrote: >> >> >> On 2018年06月18日 19:34, David Sterba wrote: >>> On Thu, Jun 14, 2018 at 03:17:45PM +0800, Qu Wenruo wrote: I understand that btrfs-progs introduced restrict parameter/option order

Re: [PATCH] btrfs-progs: Check factor out root parsing from check_chunks_and_extents

2018-06-18 Thread Su Yue
On 06/18/2018 07:10 PM, Nikolay Borisov wrote: check_chunks_and_extents does quite a number of distinct things. The first of those is going through all root items in the root tree and classify every root depending on whether they have a dropping operation in progress or not. Lets factor out

Re: [PATCH] btrfs-progs: check: Fix wrong root parameter of btrfs_next_leaf call

2018-06-18 Thread Su Yue
On 06/18/2018 07:10 PM, Nikolay Borisov wrote: The first thing that check_chunks_and_extents does is to iterate all the root items in the root tree and link them to either the "normal_list" or "dropping_trees" list. If a leaf has to be crossed during this operation btrfs_next_leaf is called

Re: About more loose parameter sequence requirement

2018-06-18 Thread Qu Wenruo
On 2018年06月18日 20:02, David Sterba wrote: > On Mon, Jun 18, 2018 at 07:40:44PM +0800, Qu Wenruo wrote: >> >> >> On 2018年06月18日 19:34, David Sterba wrote: >>> On Thu, Jun 14, 2018 at 03:17:45PM +0800, Qu Wenruo wrote: I understand that btrfs-progs introduced restrict parameter/option order

btrfs balance did not progress after 12H

2018-06-18 Thread Marc MERLIN
So, I ran this: gargamel:/mnt/btrfs_pool2# btrfs balance start -dusage=60 -v . & [1] 24450 Dumping filters: flags 0x1, state 0x0, force is off DATA (flags 0x2): balancing, usage=60 gargamel:/mnt/btrfs_pool2# while :; do btrfs balance status .; sleep 60; done 0 out of about 0 chunks balanced (0

RE: About more loose parameter sequence requirement

2018-06-18 Thread Paul Jones
> -Original Message- > From: linux-btrfs-ow...@vger.kernel.org ow...@vger.kernel.org> On Behalf Of Hugo Mills > Sent: Monday, 18 June 2018 9:44 PM > To: dste...@suse.cz; Qu Wenruo ; linux- > bt...@vger.kernel.org > Subject: Re: About more loose parameter sequence requirement > > On Mon,

Re: About more loose parameter sequence requirement

2018-06-18 Thread Hugo Mills
On Mon, Jun 18, 2018 at 01:34:32PM +0200, David Sterba wrote: > On Thu, Jun 14, 2018 at 03:17:45PM +0800, Qu Wenruo wrote: > > I understand that btrfs-progs introduced restrict parameter/option order > > to distinguish global and sub-command parameter/option. > > > > However it's really annoying

Re: About more loose parameter sequence requirement

2018-06-18 Thread David Sterba
On Mon, Jun 18, 2018 at 07:40:44PM +0800, Qu Wenruo wrote: > > > On 2018年06月18日 19:34, David Sterba wrote: > > On Thu, Jun 14, 2018 at 03:17:45PM +0800, Qu Wenruo wrote: > >> I understand that btrfs-progs introduced restrict parameter/option order > >> to distinguish global and sub-command

[PATCH 2/3] btrfs: Document __btrfs_inc_extent_ref

2018-06-18 Thread Nikolay Borisov
Here is a doc-only patch which tires to deobfuscate the terra-incognita that arguments for delayed refs are. Signed-off-by: Nikolay Borisov --- fs/btrfs/extent-tree.c | 34 ++ 1 file changed, 34 insertions(+) diff --git a/fs/btrfs/extent-tree.c

[PATCH 1/3] btrfs: Remove fs_info argument from __btrfs_inc_extent_ref

2018-06-18 Thread Nikolay Borisov
This function already takes a transaction which holds a reference to the fs_info struct. Use that reference and remove the extra arg. No functional changes. Signed-off-by: Nikolay Borisov --- fs/btrfs/extent-tree.c | 15 ++- 1 file changed, 6 insertions(+), 9 deletions(-) diff

[PATCH 3/3] btrfs: Fix comment in lookup_inline_extent_backref

2018-06-18 Thread Nikolay Borisov
The comment wrongfully states that the owner parameter is the level of the parent block. In fact owner is the level of the current block and by adding 1 to it we can eventually get to the parent/root. Signed-off-by: Nikolay Borisov --- fs/btrfs/extent-tree.c | 2 +- 1 file changed, 1

[PATCH 0/3] Cleanups around extent increment

2018-06-18 Thread Nikolay Borisov
Hello, This series improves the functions involved around extent reference increment. The first patch just removes a redundant argument, the second one documents the parameters of __btrfs_inc_extent_ref. It can be considered v2 of the standalone version which Jeff had some input to. The final

Re: About more loose parameter sequence requirement

2018-06-18 Thread Qu Wenruo
On 2018年06月18日 19:34, David Sterba wrote: > On Thu, Jun 14, 2018 at 03:17:45PM +0800, Qu Wenruo wrote: >> I understand that btrfs-progs introduced restrict parameter/option order >> to distinguish global and sub-command parameter/option. >> >> However it's really annoying if one just want to

Re: About more loose parameter sequence requirement

2018-06-18 Thread David Sterba
On Thu, Jun 14, 2018 at 03:17:45PM +0800, Qu Wenruo wrote: > I understand that btrfs-progs introduced restrict parameter/option order > to distinguish global and sub-command parameter/option. > > However it's really annoying if one just want to append some new options > to previous command: > >

[PATCH] btrfs: Deduplicate extent_buffer init code

2018-06-18 Thread Nikolay Borisov
When a new extent buffer is allocated there are a few mandatory fields which need to be set in order for the buffer to be sane: level, generation, bytenr, backref_rev, owner and FSID/UUID. Currently this is open coded in the callers of btrfs_alloc_tree_block, meaning it's fairly high in the

[PATCH] btrfs-progs: Check factor out root parsing from check_chunks_and_extents

2018-06-18 Thread Nikolay Borisov
check_chunks_and_extents does quite a number of distinct things. The first of those is going through all root items in the root tree and classify every root depending on whether they have a dropping operation in progress or not. Lets factor out this code and move the variables specific to this in

[PATCH] btrfs-progs: check: Fix wrong root parameter of btrfs_next_leaf call

2018-06-18 Thread Nikolay Borisov
The first thing that check_chunks_and_extents does is to iterate all the root items in the root tree and link them to either the "normal_list" or "dropping_trees" list. If a leaf has to be crossed during this operation btrfs_next_leaf is called to do that. However, currently it's called with a

Re: [PATCH 2/2] Btrfs: sync log after logging new name

2018-06-18 Thread David Sterba
On Fri, Jun 15, 2018 at 05:19:07PM +0100, Filipe Manana wrote: > On Fri, Jun 15, 2018 at 4:54 PM, David Sterba wrote: > > On Mon, Jun 11, 2018 at 07:24:28PM +0100, fdman...@kernel.org wrote: > >> From: Filipe Manana > >> Fixes: 12fcfd22fe5b ("Btrfs: tree logging unlink/rename fixes") > >>

Re: [PATCH] btrfs: Fix a C compliance issue

2018-06-18 Thread Nikolay Borisov
On 18.06.2018 12:26, David Sterba wrote: > On Sat, Jun 16, 2018 at 01:28:13PM +0300, Nikolay Borisov wrote: >> >> >> On 16.06.2018 01:36, Bart Van Assche wrote: >>> The C programming language does not allow to use preprocessor statements >>> inside macro arguments (pr_info() is defined as a

Re: [PATCH] btrfs: Fix a C compliance issue

2018-06-18 Thread David Sterba
On Sat, Jun 16, 2018 at 01:28:13PM +0300, Nikolay Borisov wrote: > > > On 16.06.2018 01:36, Bart Van Assche wrote: > > The C programming language does not allow to use preprocessor statements > > inside macro arguments (pr_info() is defined as a macro). Hence rework > > the pr_info() statement

[PATCH v2 03/20] btrfs-progs: libbtrfsutil: Factor out btrfs_util_subvolume_info_fd()

2018-06-18 Thread Misono Tomohiro
Factor out main logic of btrfs_util_subvolume_info_fd(). This is a preparation work to relax the root privilege of this function. No functional change happens. Signed-off-by: Misono Tomohiro --- libbtrfsutil/subvolume.c | 45 ++--- 1 file changed, 26

[PATCH v2 10/20] btrfs-progs: sub list: Add -A option to output path in absolute path

2018-06-18 Thread Misono Tomohiro
By default, the printed path is relative to the specified path. Let's add option to print absolute path with -A option. [Example] $ mkfs.btrfs -f $DEV $ mount $DEV /mnt $ btrfs subvolume create /mnt/AAA $ btrfs subvolume list /mnt ID 256 gen 6 top level 5 path AAA $ btrfs subvolume list

[PATCH v2 00/20] btrfs-progs: Rework of "subvolume list/show" and relax the root privileges of them

2018-06-18 Thread Misono Tomohiro
Changelog v1 -> v2: generally update whole patch set, especially: - rebased to progs 4.17 - Improve error handling - Update man/help/commit message - Add/Update several options of sub list: -f ... follow mounted subvolumes -a ... remove meaningless filter -A ...

[PATCH v2 04/20] btrfs-porgs: libbtrfsutil: Relax the privileges of util_subvolume_info()

2018-06-18 Thread Misono Tomohiro
This commit relaxes the privileges of util_subvolume_info() if kernel supports new ioctl (BTRFS_IOC_GET_SUBVOL_INFO) and @id is zero (i.e. when getting the information of the given path/fd). For older kernel (< 4.18), the behavior is the same. Signed-off-by: Misono Tomohiro ---

[PATCH v2 12/20] btrfs-progs: sub list: Add --nosort option to output incrementally without sort

2018-06-18 Thread Misono Tomohiro
Currently, "subvolume list" loads all the subvolume information into memory first in order to sort them. This may cause a performance problem if there are a lot of subvolumes. This commit adds --nosort option to output subvolume information incrementally without sort to avoid consuming memory.

[PATCH v2 05/20] btrfs-progs: libbtrfsuitl: Factor out btrfs_util_subvolume_iterator_next()

2018-06-18 Thread Misono Tomohiro
Factor out the main logic of btrfs_util_subvolume_iterator_next(). This is a preparation work to relax the root privilege of this function. No functional change happens. Signed-off-by: Misono Tomohiro --- libbtrfsutil/subvolume.c | 13 ++--- 1 file changed, 10 insertions(+), 3

[PATCH v2 13/20] btrfs-progs: sub list: Update -a option and remove meaningless filter

2018-06-18 Thread Misono Tomohiro
Currently, -a option add filter and change subvolume path as follows: - If a subvolume is a child of the specified path, nothing changes - otherwise, adds to head This is rather meaningless, so let's remove this filter. As a result, the behavior of -a option becomes the same as default

[PATCH v2 01/20] btrfs-progs: doc: Update man btrfs subvolume

2018-06-18 Thread Misono Tomohiro
Some information is obsolete and updated as follows: - Add missing explanations of some options - Remove outdated explanation of "subvolrootid" mount option - Reorder/group options of "sub list" to corresponds help message - Add explanation about different meaning of parent in "parent ID/UUID"

[PATCH v2 02/20] btrfs-progs: ioctl/libbtrfsutil: Add 3 definitions of new unprivileged ioctl

2018-06-18 Thread Misono Tomohiro
Copy and add 3 definitions of new unprivileged ioctl (BTRFS_IOC_GET_SUBVOL_INFO, BTRFS_IOC_GET_SUBVOL_ROOTREF and BTRFS_IOC_INO_LOOKUP_USER) from kernel. They will be used to implement the user version of "btrfs subvolume list/show" etc. Signed-off-by: Misono Tomohiro --- ioctl.h |

[PATCH v2 11/20] btrfs-progs: sub list: Add -f option to follow mounted subvolumes below the path

2018-06-18 Thread Misono Tomohiro
Add -f option to follow mounted subvolumes below the specified path, only if it is the same filesystem. [Example] $ mkfs.btrfs -f $DEV $ mkfs.btrfs -f $DEV2 $ mount $DEV /mnt $ btrfs subvolume create /mnt/AAA $ btrfs subvolume create /mnt/BBB $ btrfs subvolume create /mnt/CCC $ mkdir

[PATCH v2 15/20] btrfs-progs: sub list: Update help message of -d option

2018-06-18 Thread Misono Tomohiro
Explicitly states that -d requires root privileges. Also, update some option handling with regard to -d option. Signed-off-by: Misono Tomohiro --- Documentation/btrfs-subvolume.asciidoc | 3 ++- cmds-subvolume.c | 8 2 files changed, 10 insertions(+), 1

[PATCH v2 06/20] btrfs-progs: libbtrfsutil: Relax the privileges of subvolume iterator

2018-06-18 Thread Misono Tomohiro
This commit relax the privileges of subvolume iterator when the kernel supports new ioctls (BTRFS_IOC_GET_SUBVOL_ROOTREF and BTRFS_IOC_INO_LOOKUP_USER). For older kernel (< 4.18), the behavior is the same. Non-privileged user can use subvolume iterator only if new ioctls are supported and

[PATCH v2 07/20] btrfs-progs: sub list: Use libbtrfsuitl for subvolume list

2018-06-18 Thread Misono Tomohiro
This is a copy of non-merged following patch originally written by Omar Sandoval: btrfs-progs: use libbtrfsutil for subvolume list expect this commit keeps libbtrfs implementation which above commit tries to remove (therefore this adds suffix _v2 for struct/function). Original Author: Omar

[PATCH v2 17/20] btrfs-progs: sub show: Allow non-privileged user to call "subvolume show"

2018-06-18 Thread Misono Tomohiro
Allow non-privileged user to call subvolume show if new ioctls (BTRFS_IOC_GET_SUBVOL_INFO/BTRFS_IOC_GET_SUBVOL_ROOTREF, BTRFS_IOC_INO_LOOKUP_USER, from kernel 4.18) are available. Non-privileged user still cannot use -r or -u option. The behavior for root user is the same as before. There are

[PATCH v2 09/20] btrfs-progs: sub list: Change the default behavior of "subvolume list" and allow non-privileged user to call it

2018-06-18 Thread Misono Tomohiro
Change the default behavior of "subvolume list" and allow non-privileged user to call it as well. >From this commit, by default it only lists subvolumes under the specified path (incl. the path itself except top-level subvolume). Also, if kernel supports new ioctls

[PATCH v2 14/20] btrfs-progs: sub list: Update help message of -o option

2018-06-18 Thread Misono Tomohiro
Currently "sub list -o" lists only child subvolumes of the specified path. So, update help message and variable name more appropriately. Signed-off-by: Misono Tomohiro --- Documentation/btrfs-subvolume.asciidoc | 2 +- cmds-subvolume.c | 10 +- 2 files changed, 6

[PATCH v2 18/20] btrfs-progs: test: Add helper function to check if test user exists

2018-06-18 Thread Misono Tomohiro
Test user 'progs-test' will be used to test the behavior of normal user. In order to pass this check, add the user by "useradd -M progs-test". Note that progs-test should not have root privileges. Signed-off-by: Misono Tomohiro --- tests/common | 10 ++ 1 file changed, 10 insertions(+)

[PATCH v2 20/20] btrfs-progs: test: Add cli-test/010 to check "subvolume list -f" option

2018-06-18 Thread Misono Tomohiro
Signed-off-by: Misono Tomohiro --- tests/cli-tests/010-subvolume-list-follow/test.sh | 86 +++ 1 file changed, 86 insertions(+) create mode 100755 tests/cli-tests/010-subvolume-list-follow/test.sh diff --git a/tests/cli-tests/010-subvolume-list-follow/test.sh

[PATCH v2 19/20] btrfs-porgs: test: Add cli-test/009 to check subvolume list for both root and normal user

2018-06-18 Thread Misono Tomohiro
Signed-off-by: Misono Tomohiro --- tests/cli-tests/009-subvolume-list/test.sh | 134 + 1 file changed, 134 insertions(+) create mode 100755 tests/cli-tests/009-subvolume-list/test.sh diff --git a/tests/cli-tests/009-subvolume-list/test.sh

[PATCH v2 16/20] btrfs-progs: utils: Fallback to open without O_NOATIME flag in find_mount_root():

2018-06-18 Thread Misono Tomohiro
O_NOATIME flag requires effective UID of process matches file's owner or has CAP_FOWNER capabilities. Fallback to open without O_NOATIME flag so that non-privileged user can also call find_mount_root(). This is a preparation work to allow non-privileged user to call "subvolume show".

[PATCH v2 08/20] btrfs-progs: sub list: factor out main part of btrfs_list_subvols

2018-06-18 Thread Misono Tomohiro
No functional changes. This is a preparation work for reworking "subvolume list". Signed-off-by: Misono Tomohiro --- cmds-subvolume.c | 50 ++ 1 file changed, 30 insertions(+), 20 deletions(-) diff --git a/cmds-subvolume.c b/cmds-subvolume.c