Re: [PATCH v4] btrfs: qgroup: Remove qgroup items along with subvolume deletion

2018-08-09 Thread Misono Tomohiro
On 2018/08/09 14:47, Qu Wenruo wrote: > > > On 8/9/18 12:12 PM, Misono Tomohiro wrote: >> When qgroup is on, subvolume deletion does not remove qgroup items >> of the subvolume (qgroup info, limit, relation) from quota tree and >> they need to get removed manually by "btrfs qgroup destroy". >>

[PATCH] btrfs: qgroup: Dirty all qgroups before rescan

2018-08-09 Thread Qu Wenruo
[BUG] In the following case, rescan won't zero out the number of qgroup 1/0: -- $ mkfs.btrfs -fq $DEV $ mount $DEV /mnt $ btrfs quota enable /mnt $ btrfs qgroup create 1/0 /mnt $ btrfs sub create /mnt/sub $ btrfs qgroup assign 0/257 1/0 /mnt $ dd if=/dev/urandom of=/mnt/sub/file bs=1k

Re: [PATCH v2 00/20] btrfs-progs: Rework of "subvolume list/show" and relax the root privileges of them

2018-08-09 Thread Misono Tomohiro
On 2018/08/03 22:46, David Sterba wrote: > On Wed, Jul 04, 2018 at 05:14:59PM +0900, Misono Tomohiro wrote: >> Gentle ping, as this is related to the new ioctls merged in 4.18-rc1. > > Due to me spending more time than expected on kernel, this patchset will > be merged partially or the 4.18 will

Re: [PATCH v2] btrfs: qgroup: Dirty all qgroups before rescan

2018-08-09 Thread Filipe Manana
On Thu, Aug 9, 2018 at 8:08 AM, Qu Wenruo wrote: > [BUG] > In the following case, rescan won't zero out the number of qgroup 1/0: > -- > $ mkfs.btrfs -fq $DEV > $ mount $DEV /mnt > > $ btrfs quota enable /mnt > $ btrfs qgroup create 1/0 /mnt > $ btrfs sub create /mnt/sub > $ btrfs qgroup

Re: [PATCH] fstests: btrfs: Add test for corrupted orphan qgroup numbers

2018-08-09 Thread Filipe Manana
On Thu, Aug 9, 2018 at 8:45 AM, Qu Wenruo wrote: > This bug is exposed by populating a high level qgroup, and then make it > orphan (high level qgroup without child) Same comment as in the kernel patch: "That sentence is confusing. An orphan, by definition [1], is someone (or something in this

[PATCH] fstests: btrfs: Add test for corrupted orphan qgroup numbers

2018-08-09 Thread Qu Wenruo
This bug is exposed by populating a high level qgroup, and then make it orphan (high level qgroup without child) with old qgroup numbers, and finally do rescan. Normally rescan should zero out all qgroups' accounting number, but due to a kernel bug which won't mark orphan qgroups dirty, their

Re: [PATCH v4] btrfs: qgroup: Remove qgroup items along with subvolume deletion

2018-08-09 Thread Misono Tomohiro
On 2018/08/09 15:14, Qu Wenruo wrote: > > > On 8/9/18 2:05 PM, Misono Tomohiro wrote: >> On 2018/08/09 14:47, Qu Wenruo wrote: >>> >>> >>> On 8/9/18 12:12 PM, Misono Tomohiro wrote: When qgroup is on, subvolume deletion does not remove qgroup items of the subvolume (qgroup info, limit,

Re: [PATCH v2] btrfs: qgroup: Dirty all qgroups before rescan

2018-08-09 Thread Misono Tomohiro
On 2018/08/09 16:08, Qu Wenruo wrote: > [BUG] > In the following case, rescan won't zero out the number of qgroup 1/0: > -- > $ mkfs.btrfs -fq $DEV > $ mount $DEV /mnt > > $ btrfs quota enable /mnt > $ btrfs qgroup create 1/0 /mnt > $ btrfs sub create /mnt/sub > $ btrfs qgroup assign 0/257

[PATCH v5] btrfs: qgroup: Remove qgroup items along with subvolume deletion

2018-08-09 Thread Misono Tomohiro
When qgroup is on, subvolume deletion does not remove qgroup items of the subvolume (qgroup info, limit, relation) from quota tree and they need to get removed manually by "btrfs qgroup destroy". Since level 0 qgroup cannot be used/inherited by any other subvolume, let's remove them automatically

Re: [PATCH] btrfs: qgroup: Dirty all qgroups before rescan

2018-08-09 Thread Nikolay Borisov
On 9.08.2018 10:03, Qu Wenruo wrote: > [BUG] > In the following case, rescan won't zero out the number of qgroup 1/0: > -- > $ mkfs.btrfs -fq $DEV > $ mount $DEV /mnt > > $ btrfs quota enable /mnt > $ btrfs qgroup create 1/0 /mnt > $ btrfs sub create /mnt/sub > $ btrfs qgroup assign 0/257

Re: [PATCH v4] btrfs: qgroup: Remove qgroup items along with subvolume deletion

2018-08-09 Thread Qu Wenruo
On 8/9/18 2:05 PM, Misono Tomohiro wrote: > On 2018/08/09 14:47, Qu Wenruo wrote: >> >> >> On 8/9/18 12:12 PM, Misono Tomohiro wrote: >>> When qgroup is on, subvolume deletion does not remove qgroup items >>> of the subvolume (qgroup info, limit, relation) from quota tree and >>> they need to

Re: [PATCH] btrfs: qgroup: Dirty all qgroups before rescan

2018-08-09 Thread Qu Wenruo
On 8/9/18 3:05 PM, Nikolay Borisov wrote: > > > On 9.08.2018 10:03, Qu Wenruo wrote: >> [BUG] >> In the following case, rescan won't zero out the number of qgroup 1/0: >> -- >> $ mkfs.btrfs -fq $DEV >> $ mount $DEV /mnt >> >> $ btrfs quota enable /mnt >> $ btrfs qgroup create 1/0 /mnt >>

[PATCH v2] btrfs: qgroup: Dirty all qgroups before rescan

2018-08-09 Thread Qu Wenruo
[BUG] In the following case, rescan won't zero out the number of qgroup 1/0: -- $ mkfs.btrfs -fq $DEV $ mount $DEV /mnt $ btrfs quota enable /mnt $ btrfs qgroup create 1/0 /mnt $ btrfs sub create /mnt/sub $ btrfs qgroup assign 0/257 1/0 /mnt $ dd if=/dev/urandom of=/mnt/sub/file bs=1k

Re: recover broken partition on external HDD

2018-08-09 Thread Marijn Stollenga
Thanks for the reply! Indeed I should back up more properly, that was actually what I was in the process of doing but yeah. I'll check out the pointers, and I guess I'll just read the papers describing the whole btrfs system to see how it works. I would like to make an automatic scrubbing

Re: [PATCH v2] btrfs: qgroup: Dirty all qgroups before rescan

2018-08-09 Thread Qu Wenruo
On 8/9/18 5:17 PM, Filipe Manana wrote: > On Thu, Aug 9, 2018 at 8:08 AM, Qu Wenruo wrote: >> [BUG] >> In the following case, rescan won't zero out the number of qgroup 1/0: >> -- >> $ mkfs.btrfs -fq $DEV >> $ mount $DEV /mnt >> >> $ btrfs quota enable /mnt >> $ btrfs qgroup create 1/0 /mnt

Re: btrfs-convert missing in btrfs-tools v4.15.1

2018-08-09 Thread David Sterba
On Sat, Jul 28, 2018 at 05:34:49PM -0400, Nicholas D Steeves wrote: > On 28 July 2018 at 16:50, jkexcel wrote: > > I'm an end user trying to use btrfs-convert but when I installed > > btrfs-tools and its dependency btrfs-progs on kubuntu 18.04, the > > installation was successful, and it shows

[COMMAND HANGS] The command 'btrfs subvolume sync -s 2 xyz' can hangs.

2018-08-09 Thread Giuseppe Della Bianca
Hi. My system: - Fedora 28 x86_64 - kernel-4.17.7-200 - btrfs-progs-4.15.1-1 The command 'btrfs subvolume sync -s 2 xyz' hangs in this case: - Run command 'btrfs subvolume sync -s 2 xyz' . - After some time the kernel reports an error on the filesystem. (error that existed before the command

Re: Report correct filesystem usage / limits on BTRFS subvolumes with quota

2018-08-09 Thread Tomasz Pala
On Tue, Jul 31, 2018 at 22:32:07 +0800, Qu Wenruo wrote: > 2) Different limitations on exclusive/shared bytes >Btrfs can set different limit on exclusive/shared bytes, further >complicating the problem. > > 3) Btrfs quota only accounts data/metadata used by the subvolume >It lacks

[PATCH v2 0/3] Misc volume patch set part2

2018-08-09 Thread Anand Jain
The patch set (2/3 and 3/3) adds helper function to deduce the num_device (which is the number of the devices when device is mounted, this excludes the seed device however includes the replacing target). We need to know the num_device that actually belongs to the FSID without the replacing target

[PATCH 1/2] btrfs: assert for num_devices below 0

2018-08-09 Thread Anand Jain
In preparation to add helper function to deduce the num_devices with replace running, use assert instead of bug_on and warn_on. Signed-off-by: Anand Jain --- fs/btrfs/volumes.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c index

[PATCH v4 1/3] btrfs: drop uuid_mutex in btrfs_free_extra_devids()

2018-08-09 Thread Anand Jain
btrfs_free_extra_devids() is called only in the mount context which traverses through the fs_devices::devices and frees the orphan devices devices in the given %fs_devices if any. As the search for the orphan device is limited to fs_devices::devices so we don't need the global uuid_mutex. There

[PATCH v5 2/2] btrfs: add helper btrfs_num_devices() to deduce num_devices

2018-08-09 Thread Anand Jain
When the replace is running the fs_devices::num_devices also includes the replace device, however in some operations like device delete and balance it needs the actual num_devices without the repalce devices, so now the function btrfs_num_devices() just provides that. And here is a scenario how

[PATCH v2] fstests: btrfs: Add test for corrupted childless qgroup numbers

2018-08-09 Thread Qu Wenruo
This bug is exposed by populating a high level qgroup, and then make it childless with old qgroup numbers, and finally do rescan. Normally rescan should zero out all qgroups' accounting number, but due to a kernel bug which won't mark childless qgroups dirty, their on-disk data is never updated,

[PATCH v3] btrfs: qgroup: Dirty all qgroups before rescan

2018-08-09 Thread Qu Wenruo
[BUG] In the following case, rescan won't zero out the number of qgroup 1/0: -- $ mkfs.btrfs -fq $DEV $ mount $DEV /mnt $ btrfs quota enable /mnt $ btrfs qgroup create 1/0 /mnt $ btrfs sub create /mnt/sub $ btrfs qgroup assign 0/257 1/0 /mnt $ dd if=/dev/urandom of=/mnt/sub/file bs=1k

[PATCH] btrfs-progs: qgroup: Don't return 1 if qgroup is marked inconsistent during relationship assignment

2018-08-09 Thread Qu Wenruo
BTRFS_IOC_QGROUP_ASSIGN ioctl could return >0 if qgroup is marked inconsistent after successful relationship assignment/removal. We leak the return value as the final return value of btrfs command. But according to the man page, return value other than 0 means failure. Fix this by resetting the

List of known BTRFS Raid 5/6 Bugs?

2018-08-09 Thread erenthetitan
I am searching for more information regarding possible bugs related to BTRFS Raid 5/6. All sites i could find are incomplete and information contradicts itself: The Wiki Raid 5/6 Page (https://btrfs.wiki.kernel.org/index.php/RAID56) warns of the write hole bug, stating that your data remains

mount shows incorrect subvol when backed by bind mount

2018-08-09 Thread Chris Murphy
I've got another example of bind mounts resulting in confusing (incorrect) information in the mount command with Btrfs. In this case, it's Docker using bind mounts. Full raw version (expires in 7 days) https://paste.fedoraproject.org/paste/r8tr-3nuvoycwxf0bPUrmA/raw Relevant portion: mount

Re: [COMMAND HANGS] The command 'btrfs subvolume sync -s 2 xyz' can hangs.

2018-08-09 Thread Jeff Mahoney
On 8/9/18 11:15 AM, Giuseppe Della Bianca wrote: > Hi. > > My system: > - Fedora 28 x86_64 > - kernel-4.17.7-200 > - btrfs-progs-4.15.1-1 > > The command 'btrfs subvolume sync -s 2 xyz' hangs in this case: > > - Run command 'btrfs subvolume sync -s 2 xyz' . > - After some time the kernel

Re: Report correct filesystem usage / limits on BTRFS subvolumes with quota

2018-08-09 Thread Qu Wenruo
On 8/10/18 1:48 AM, Tomasz Pala wrote: > On Tue, Jul 31, 2018 at 22:32:07 +0800, Qu Wenruo wrote: > >> 2) Different limitations on exclusive/shared bytes >>Btrfs can set different limit on exclusive/shared bytes, further >>complicating the problem. >> >> 3) Btrfs quota only accounts