Re: [PATCH v3] btrfs: qgroup: Remove qgroup items along with subvolume deletion

2018-08-07 Thread Qu Wenruo
On 2018年08月08日 10:56, Misono Tomohiro wrote: > On 2018/08/08 1:23, Lu Fengqi wrote: >> On Mon, Aug 06, 2018 at 01:53:28PM +0900, Misono Tomohiro wrote: >>> When qgroup is on, subvolume deletion does not remove qgroup items >>> of the subvolume (qgroup info, limit, relation) from quota tree and

RE: [PATCH 2/5] btrfs: use a separate variable to store the num_bytes of the qgroup_reserve

2018-08-07 Thread Gu, Jinxiang
> -Original Message- > From: David Sterba [mailto:dste...@suse.cz] > Sent: Wednesday, August 08, 2018 12:19 AM > To: Lu, Fengqi/陆 丰麒 > Cc: linux-btrfs@vger.kernel.org; Gu, Jinxiang/顾 金香 > Subject: Re: [PATCH 2/5] btrfs: use a separate variable to store the > num_bytes of the

Re: [PATCH 2/5] btrfs: use a separate variable to store the num_bytes of the qgroup_reserve

2018-08-07 Thread Lu Fengqi
On Tue, Aug 07, 2018 at 06:19:12PM +0200, David Sterba wrote: >On Sat, Aug 04, 2018 at 09:10:54PM +0800, Lu Fengqi wrote: >> After btrfs_qgroup_reserve_meta_prealloc(), num_bytes will be assigned >> again by btrfs_calc_trans_metadata_size(). Therefore, once block_rsv >> fails, we cannot properly

Re: [PATCH v3] btrfs: qgroup: Remove qgroup items along with subvolume deletion

2018-08-07 Thread Misono Tomohiro
On 2018/08/08 1:23, Lu Fengqi wrote: > On Mon, Aug 06, 2018 at 01:53:28PM +0900, Misono Tomohiro wrote: >> When qgroup is on, subvolume deletion does not remove qgroup items >> of the subvolume (qgroup info, limit, relation) from quota tree and >> they need to get removed manually by "btrfs qgroup

Re: [RFC PATCH 0/4] undelete subvolume online version

2018-08-07 Thread Lu Fengqi
On Tue, Aug 07, 2018 at 06:39:50PM +0200, David Sterba wrote: >On Sun, Aug 05, 2018 at 06:39:57PM +0800, Lu Fengqi wrote: >> This patchset will add the BTRFS_IOC_SUBVOL_UNDELETE ioctl for online >> btrfs subvolume undelete. >> >> And btrfs subvolume undelete subcommand was added to btrfs-progs.

Re: [PATCH RESEND v8] Add cli and ioctl to forget scanned device(s)

2018-08-07 Thread Anand Jain
On 08/08/2018 01:51 AM, David Sterba wrote: On Mon, Aug 06, 2018 at 09:09:47AM +0800, Anand Jain wrote: Adds cli and ioctl to forget a scanned device or forget all stale devices in the kernel. Please provide more details about your idea of the usecase, ie. how excactly and when the command

Re: [PATCH] btrfs: handle the BUG_ON in btrfs_num_devices()

2018-08-07 Thread Anand Jain
On 08/08/2018 01:09 AM, David Sterba wrote: On Mon, Aug 06, 2018 at 04:57:45PM +0800, Anand Jain wrote: On 08/03/2018 09:33 PM, Nikolay Borisov wrote: On 3.08.2018 15:45, Anand Jain wrote: Its a logical bug if we hit fs_devices::num_devices == 1 and if the replace is running because,

Re: [PATCH v3 4/4] btrfs: add helper btrfs_num_devices() to deduce num_devices

2018-08-07 Thread Anand Jain
On 08/07/2018 11:02 PM, David Sterba wrote: On Thu, Aug 02, 2018 at 09:07:00PM +0800, Anand Jain wrote: - num_devices = fs_devices->num_devices; - btrfs_dev_replace_read_lock(_info->dev_replace); - if (btrfs_dev_replace_is_ongoing(_info->dev_replace)) { -

Re: [RESEND][PATCH v5 0/2] vfs: better dedupe permission check

2018-08-07 Thread Adam Borowski
On Tue, Aug 07, 2018 at 02:49:47PM -0700, Mark Fasheh wrote: > Hi Andrew, > > Could I please have these patches upstreamed or at least put in a tree for > more public testing? They've hit fsdevel a few times now, I have links to > the discussions in the change log below. > The first patch

[PATCH v5 2/2] vfs: dedupe should return EPERM if permission is not granted

2018-08-07 Thread Mark Fasheh
Right now we return EINVAL if a process does not have permission to dedupe a file. This was an oversight on my part. EPERM gives a true description of the nature of our error, and EINVAL is already used for the case that the filesystem does not support dedupe. Signed-off-by: Mark Fasheh

[RESEND][PATCH v5 0/2] vfs: better dedupe permission check

2018-08-07 Thread Mark Fasheh
Hi Andrew, Could I please have these patches upstreamed or at least put in a tree for more public testing? They've hit fsdevel a few times now, I have links to the discussions in the change log below. The following patches fix a couple of issues with the permission check we do in

[PATCH v5 1/2] vfs: allow dedupe of user owned read-only files

2018-08-07 Thread Mark Fasheh
The permission check in vfs_dedupe_file_range() is too coarse - We only allow dedupe of the destination file if the user is root, or they have the file open for write. This effectively limits a non-root user from deduping their own read-only files. In addition, the write file descriptor that the

Re: [PATCH RESEND v8] Add cli and ioctl to forget scanned device(s)

2018-08-07 Thread David Sterba
On Mon, Aug 06, 2018 at 09:09:47AM +0800, Anand Jain wrote: > Adds cli and ioctl to forget a scanned device or forget all stale > devices in the kernel. Please provide more details about your idea of the usecase, ie. how excactly and when the command is supposed to be used. I vaguely recollect

Re: [PATCH] btrfs: handle the BUG_ON in btrfs_num_devices()

2018-08-07 Thread David Sterba
On Mon, Aug 06, 2018 at 04:57:45PM +0800, Anand Jain wrote: > > > On 08/03/2018 09:33 PM, Nikolay Borisov wrote: > > > > > > On 3.08.2018 15:45, Anand Jain wrote: > >> Its a logical bug if we hit fs_devices::num_devices == 1 and if the > >> replace is running because, as

Re: [RFC PATCH 0/4] undelete subvolume online version

2018-08-07 Thread David Sterba
On Sun, Aug 05, 2018 at 06:39:57PM +0800, Lu Fengqi wrote: > This patchset will add the BTRFS_IOC_SUBVOL_UNDELETE ioctl for online > btrfs subvolume undelete. > > And btrfs subvolume undelete subcommand was added to btrfs-progs. > > So user can use the following command to recover all the

Re: [PATCH v3] btrfs: qgroup: Remove qgroup items along with subvolume deletion

2018-08-07 Thread Lu Fengqi
On Mon, Aug 06, 2018 at 01:53:28PM +0900, Misono Tomohiro wrote: >When qgroup is on, subvolume deletion does not remove qgroup items >of the subvolume (qgroup info, limit, relation) from quota tree and >they need to get removed manually by "btrfs qgroup destroy". > >Since level 0 qgroup cannot be

Re: [PATCH 2/5] btrfs: use a separate variable to store the num_bytes of the qgroup_reserve

2018-08-07 Thread David Sterba
On Sat, Aug 04, 2018 at 09:10:54PM +0800, Lu Fengqi wrote: > After btrfs_qgroup_reserve_meta_prealloc(), num_bytes will be assigned > again by btrfs_calc_trans_metadata_size(). Therefore, once block_rsv > fails, we cannot properly free the num_bytes of the previous > qgroup_reserve. This does not

Re: [PATCH 1/5] btrfs: simplify the send_in_progress check in btrfs_delete_subvolume()

2018-08-07 Thread David Sterba
On Sat, Aug 04, 2018 at 09:10:53PM +0800, Lu Fengqi wrote: > Only when send_in_progress, we have to do something different such as > btrfs_warn() and return -EPERM. Therefore, we could check > send_in_progress first and process error handling, after the > root_item_lock has been got. > > Just for

Re: [PATCH 3/5] btrfs: switch update_size to bool in both of btrfs_block_rsv_migrate and btrfs_rsv_add_bytes

2018-08-07 Thread David Sterba
On Sat, Aug 04, 2018 at 09:10:55PM +0800, Lu Fengqi wrote: > Using true and false here is more semantic than using 0 and 1. > > No functional change. > > Signed-off-by: Lu Fengqi Reviewed-by: David Sterba -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of

Re: [PATCH 5/5] btrfs: Remove root parameter from btrfs_insert_dir_item

2018-08-07 Thread David Sterba
On Sat, Aug 04, 2018 at 09:10:57PM +0800, Lu Fengqi wrote: > All callers pass the root tree of dir, we can push that down to the > function itself. > > Signed-off-by: Lu Fengqi Reviewed-by: David Sterba -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a

Re: [PATCH] Btrfs: fix unexpected failure of nocow buffered writes after snapshotting when low on space

2018-08-07 Thread David Sterba
On Mon, Aug 06, 2018 at 10:30:30AM +0800, robbieko wrote: > From: Robbie Ko > > Commit e9894fd3e3b3 ("Btrfs: fix snapshot vs nocow writting") > forced nocow writes to fallback to COW, during writeback, > when a snapshot is created. This resulted in writes made before > creating the snapshot to

Re: [PATCH v3 4/4] btrfs: add helper btrfs_num_devices() to deduce num_devices

2018-08-07 Thread David Sterba
On Thu, Aug 02, 2018 at 09:07:00PM +0800, Anand Jain wrote: > >>> - num_devices = fs_devices->num_devices; > >>> - btrfs_dev_replace_read_lock(_info->dev_replace); > >>> - if (btrfs_dev_replace_is_ongoing(_info->dev_replace)) { > >>> - BUG_ON(num_devices < 1); > >>> -

Re: [PATCH v2 2/4] btrfs: fix race between free_stale_devices and close_fs_devices

2018-08-07 Thread David Sterba
On Thu, Aug 02, 2018 at 05:29:12PM +0800, Anand Jain wrote: > > > On 08/01/2018 10:29 PM, David Sterba wrote: > > On Thu, Jul 26, 2018 at 02:53:32PM +0800, Anand Jain wrote: > >> From: Anand Jain > >> > >> %fs_devices can be free-ed by btrfs_free_stale_devices() when the > >> close_fs_devices()

Re: [PATCH 1/2] Install uncompressed man pages

2018-08-07 Thread David Sterba
On Mon, Aug 06, 2018 at 04:54:31PM -0400, Mike Gilbert wrote: > Build systems do not typically compress man pages when installing them. > This is generally left to distro packaging mechanisms, which may end up > recompressing them using a different compressor. Makes sense. I've tested this on

Re: [PATCH 2/2] Install btrfs-ioctl.3 man page

2018-08-07 Thread David Sterba
On Mon, Aug 06, 2018 at 04:54:32PM -0400, Mike Gilbert wrote: > --- > Documentation/Makefile.in | 2 ++ > 1 file changed, 2 insertions(+) > > diff --git a/Documentation/Makefile.in b/Documentation/Makefile.in > index c4c843cb..5bc0d80d 100644 > --- a/Documentation/Makefile.in > +++

[PATCH 11/11] btrfs-progs: test/mkfs: Add test case for --rootdir and --quota

2018-08-07 Thread Qu Wenruo
Nothing interesting, since such combination can be handled easily by qgroup-verify. Signed-off-by: Qu Wenruo --- .../mkfs-tests/018-rootdir-with-quota/test.sh | 51 +++ 1 file changed, 51 insertions(+) create mode 100755 tests/mkfs-tests/018-rootdir-with-quota/test.sh diff

[PATCH 10/11] btrfs-progs: test/mkfs: Add test case for -Q|--quota option

2018-08-07 Thread Qu Wenruo
Only test if btrfs check (which will check qgroup by default) and kernel mount success. Comprehensive qgroup test cases still belongs to fstests. Signed-off-by: Qu Wenruo --- tests/mkfs-tests/001-basic-profiles/test.sh | 10 ++ 1 file changed, 10 insertions(+) diff --git

[PATCH 09/11] btrfs-progs: mkfs: Introduce mkfs time quota support

2018-08-07 Thread Qu Wenruo
The result fs will has quota enabled, with consistent qgroup accounting. This is quite handy to test quota with fstests, which doesn't support to call ioctl for btrfs at mount time. Signed-off-by: Qu Wenruo --- Documentation/mkfs.btrfs.asciidoc | 5 + mkfs/main.c |

[PATCH 08/11] btrfs-progs: mkfs: Introduce function to setup quota root and rescan

2018-08-07 Thread Qu Wenruo
Introduce a new function, setup_quota_root(), which will create quota root, and do an offline rescan to ensure all quota accounting numbers are correct. Signed-off-by: Qu Wenruo --- mkfs/main.c | 86 + 1 file changed, 86 insertions(+) diff

[PATCH v3 00/11] btrfs-progs: mkfs: Quota support through -Q|--quota

2018-08-07 Thread Qu Wenruo
Can be fetched from github: https://github.com/adam900710/btrfs-progs/tree/mkfs_qgroup Which is based on v4.17.1 tag. This patchset adds quota support, which means the result fs will have quota enabled by default, and its accounting is already consistent, no manually rescan or quota enable is

[PATCH 03/11] btrfs-progs: qgroup-verify: Use fs_info->readonly to check if we should repair qgroups

2018-08-07 Thread Qu Wenruo
In fact qgroup-verify is just kind of offline qgroup rescan, and later mkfs qgroup support will reuse it. So qgroup-verify doesn't really need to rely the global variable @repair to check if it should repair qgroups. Instead check fs_info->readonly to do the repair. Signed-off-by: Qu Wenruo

[PATCH 06/11] btrfs-progs: ctree: Introduce function to create an empty tree

2018-08-07 Thread Qu Wenruo
Introduce a new function, btrfs_create_tree(), to create an empty tree. Currently, there is only one caller to create new tree, namely data reloc tree in mkfs. However it's copying fs tree to create a new root. This copy fs tree method is not a good idea if we only need an empty tree. So here

[PATCH 05/11] btrfs-progs: qgroup-verify: Allow repair_qgroups function to do silent repair

2018-08-07 Thread Qu Wenruo
Allow repair_qgroups() to do silent repair, so it can acts as offline qgroup rescan. This provides the basis for later mkfs quota support. Signed-off-by: Qu Wenruo --- check/main.c| 2 +- qgroup-verify.c | 19 +++ qgroup-verify.h | 2 +- 3 files changed, 13 insertions(+),

[PATCH 07/11] btrfs-progs: mkfs: Introduce function to insert qgroup info and limit items

2018-08-07 Thread Qu Wenruo
Introduce a new function, insert_qgroup_items(), to insert qgroup info item and qgroup limit item for later mkfs qgroup support. Signed-off-by: Qu Wenruo --- mkfs/main.c | 34 ++ 1 file changed, 34 insertions(+) diff --git a/mkfs/main.c b/mkfs/main.c index

[PATCH 04/11] btrfs-progs: qgroup-verify: Move qgroup classification out of report_qgroups

2018-08-07 Thread Qu Wenruo
The original qgroup-verify integrates qgroup classification into report_qgroups(). This behavior makes silent qgroup repair (or offline rescan) impossible. To repair qgroup, we must call report_qgroups() to trigger bad qgroup classification, which will output error message. This patch moves bad

[PATCH 01/11] btrfs-progs: qgroup-verify: Avoid NULL pointer dereference for later silent qgroup repair

2018-08-07 Thread Qu Wenruo
Commit 078e9a1cc973 ("btrfs-progs: check: enhanced progress indicator") introduced @qgroup_item_count for progress indicator. However since we will later introduce silent qgroup rescan functionality, the @qgroup_item_count pointer can be NULL. So check if @qgroup_item_count is NULL before

[PATCH 02/11] btrfs-progs: qgroup-verify: Also repair qgroup status version

2018-08-07 Thread Qu Wenruo
Current kernel only supports qgroup version 1. Make qgroup-verify to follow this standard. Signed-off-by: Qu Wenruo --- ctree.h | 1 + qgroup-verify.c | 2 ++ 2 files changed, 3 insertions(+) diff --git a/ctree.h b/ctree.h index 4719962df67d..61c3624ff5db 100644 --- a/ctree.h +++

Re: [PATCH v2 00/12] mkfs: Quota support through -R|--runtime quota

2018-08-07 Thread Qu Wenruo
On 2018年08月07日 15:02, Lu Fengqi wrote: > On Mon, Jul 30, 2018 at 01:03:00PM +0800, Qu Wenruo wrote: >> Ping the 3rd time? >> >> Or should I just rebase the patchset? > > Hi Qu > > Could you rebase this patchset? Because I want to test existing test cases > with enabled quota, mkfs_qgroup seems

Re: [PATCH v2 00/12] mkfs: Quota support through -R|--runtime quota

2018-08-07 Thread Lu Fengqi
On Mon, Jul 30, 2018 at 01:03:00PM +0800, Qu Wenruo wrote: >Ping the 3rd time? > >Or should I just rebase the patchset? Hi Qu Could you rebase this patchset? Because I want to test existing test cases with enabled quota, mkfs_qgroup seems to ease my workload. -- Thanks, Lu > >Thanks, >Qu >