[PATCH] btrfs-progs: libbtrfsutil: python: use top=0 as default value in SubvolumeIterator

2018-04-17 Thread Misono Tomohiro
Document of SubvolumeIterator says: SubvolumeIterator(path, top=0, info=False, post_order=False) -> new subvolume iterator ... top -- if not zero, instead of only listing subvolumes beneath the given path, list subvolumes beneath the subvolume with this ID; passing

[PATCH] btrfs: delayed-inode: Remove wrong qgroup meta reservation calls

2018-04-17 Thread Qu Wenruo
Commit 4f5427ccce5d ("btrfs: delayed-inode: Use new qgroup meta rsv for delayed inode and item") merged into mainline is not the updated version submitted to the mail list in Dec 2017. Which lacks the following fixes: 1) Remove btrfs_qgroup_convert_reserved_meta() call in

Re: [PATCH] btrfs: Do super block verification before writing it to disk

2018-04-17 Thread Anand Jain
v3: Update commit message to show the corruption in details. Modify the kernel error message to show corruption is detected before transaction commitment. Nice. Thanks. more below. @@ -3310,6 +3311,27 @@ static int write_dev_supers(struct btrfs_device *device,

Re: [PATCH] btrfs: Do super block verification before writing it to disk

2018-04-17 Thread Qu Wenruo
On 2018年04月17日 17:05, Anand Jain wrote: > >> v3: >>    Update commit message to show the corruption in details. >>    Modify the kernel error message to show corruption is detected before >>    transaction commitment. >  Nice. Thanks. more below. > >> @@ -3310,6 +3311,27 @@ static int

Re: Add udev-md-raid-safe-timeouts.rules

2018-04-17 Thread Austin S. Hemmelgarn
On 2018-04-16 11:02, Wol's lists wrote: On 16/04/18 12:43, Austin S. Hemmelgarn wrote: On 2018-04-15 21:04, Chris Murphy wrote: I just ran into this: https://github.com/neilbrown/mdadm/pull/32/commits/af1ddca7d5311dfc9ed60a5eb6497db1296f1bec This solution is inadequate, can it be made more

Re: Add udev-md-raid-safe-timeouts.rules

2018-04-17 Thread Austin S. Hemmelgarn
On 2018-04-16 13:10, Chris Murphy wrote: Adding linux-usb@ and linux-scsi@ (This email does contain the thread initiating email, but some replies are on the other lists.) On Mon, Apr 16, 2018 at 5:43 AM, Austin S. Hemmelgarn wrote: On 2018-04-15 21:04, Chris Murphy

Re: Kernel unaligned access at ... btrfs_real_readdir+0x51c/0x718 [btrfs]

2018-04-17 Thread René Rebe
Hi, On 16 Apr 2018, at 21:15, David Sterba wrote: > On Mon, Apr 16, 2018 at 09:55:45PM +0200, René Rebe wrote: >> Hi, >> >> On 04/16/2018 06:48 PM, David Sterba wrote: >>> The warnings are valid, there's unaligned access introduced by patch >>> >>>

[PATCH] btrfs: Fix wrong btrfs_delalloc_release_extents parameter

2018-04-17 Thread Qu Wenruo
Commit 43b18595d660 ("btrfs: qgroup: Use separate meta reservation type for delalloc") merged into mainline is not the latest version submitted to mail list in Dec 2017. It has a fatal wrong @qgroup_free parameter, which results increasing qgroup metadata pertrans reserved space, and causing a

[PATCH 1/1] fstests: btrfs/130 fix Invalid argument

2018-04-17 Thread Anand Jain
btrfs-progs patch[1] replaced read(2) write(2) with splice(2) and caused the append-redirect to stop working. Before: btrfs send /btrfs/ro_send > /dev/null At subvol /btrfs/ro_snap btrfs send /btrfs/ro_send >> /dev/null At subvol /btrfs/ro_snap After: btrfs send /btrfs/ro_send > /dev/null

Re: [PATCH] btrfs: Do super block verification before writing it to disk

2018-04-17 Thread Qu Wenruo
On 2018年04月17日 22:32, Anand Jain wrote: > > > On 04/17/2018 05:58 PM, Qu Wenruo wrote: >> >> >> On 2018年04月17日 17:05, Anand Jain wrote: >>> v3:     Update commit message to show the corruption in details.     Modify the kernel error message to show corruption is detected

[PATCH] fstests: btrfs/130 make it workable on small systems

2018-04-17 Thread Anand Jain
This test case takes a long time to complete at the default LOAD_FACTOR=1, so reduce the nr_extents to 256, so for larger systems it can still use higher LOAD_FACTOR. Signed-off-by: Anand Jain --- tests/btrfs/130 | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff

Re: [PATCH] Btrfs: clean up resources during umount after trans is aborted

2018-04-17 Thread David Sterba
On Sat, Mar 31, 2018 at 06:11:56AM +0800, Liu Bo wrote: > Currently if some fatal errors occur, like all IO get -EIO, resources > would be cleaned up when > a) transaction is being committed or > b) BTRFS_FS_STATE_ERROR is set > > However, in some rare cases, resources may be left alone after

Re: [PATCH] btrfs: Do super block verification before writing it to disk

2018-04-17 Thread Anand Jain
On 04/17/2018 05:58 PM, Qu Wenruo wrote: On 2018年04月17日 17:05, Anand Jain wrote: v3:    Update commit message to show the corruption in details.    Modify the kernel error message to show corruption is detected before    transaction commitment.  Nice. Thanks. more below. @@ -3310,6

Filesystem full with unallocated space?

2018-04-17 Thread Matthew Lai
Hello! I am getting ENOSPC on my root filesystem with plenty of unallocated space according to "fi usage". Any idea why that may be? This is a root partition for Debian Stable. Not doing anything unusual that I'm aware of. No snapshots. Thanks! Matthew uname -a: Linux bigfoot

Re: [PATCH] btrfs: delayed-inode: Remove wrong qgroup meta reservation calls

2018-04-17 Thread David Sterba
On Tue, Apr 17, 2018 at 04:52:45PM +0800, Qu Wenruo wrote: > Commit 4f5427ccce5d ("btrfs: delayed-inode: Use new qgroup meta rsv for > delayed inode and item") merged into mainline is not the updated version > submitted to the mail list in Dec 2017. > > Which lacks the following fixes: > > 1)

Re: [PATCH] btrfs: Fix wrong btrfs_delalloc_release_extents parameter

2018-04-17 Thread David Sterba
On Tue, Apr 17, 2018 at 06:43:58PM +0800, Qu Wenruo wrote: > Commit 43b18595d660 ("btrfs: qgroup: Use separate meta reservation type > for delalloc") merged into mainline is not the latest version submitted > to mail list in Dec 2017. > > It has a fatal wrong @qgroup_free parameter, which results

[no subject]

2018-04-17 Thread Timo Nentwig
unsubscribe linux-kernel -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: [PATCH v2 05/10] btrfs-progs: undelete-subvol: introduce recover_dead_root

2018-04-17 Thread Qu Wenruo
On 2018年03月27日 15:06, Lu Fengqi wrote: > The function will find the root_item specified by the subvol_id, > clear the BTRFS_ROOT_SUBVOL_DEAD flag and set root_refs to one. Sorry I didn't point it out in btrfs_link_subvol() patch, but at least to me, refs should only be modified by

Re: [PATCH v2 04/10] btrfs-progs: undelete-subvol: introduce is_subvol_intact

2018-04-17 Thread Qu Wenruo
On 2018年03月27日 15:06, Lu Fengqi wrote: > The function is used to determine whether the subvolume is intact. > > Signed-off-by: Lu Fengqi > --- > Makefile | 3 ++- > undelete-subvol.c | 53 + >

Re: [PATCH v2 03/10] btrfs-progs: use btrfs_find_free_dir_index to find free inode index

2018-04-17 Thread Qu Wenruo
On 2018年03月27日 15:06, Lu Fengqi wrote: > Since we have an existing function to find free inode index, we can > reuse it here. > > Signed-off-by: Lu Fengqi Looks good. Reviewed-by: Qu Wenruo Thanks, Qu > --- > inode.c | 27

Re: [PATCH v2 08/10] btrfs-progs: undelete-subvol: add undelete-subvol subcommand

2018-04-17 Thread Qu Wenruo
On 2018年03月27日 15:06, Lu Fengqi wrote: > Add the undelete-subvol subcommand for btrfs rescue. This subcommand is > used to recover deleted subvolume left intact on the device. > > Signed-off-by: Lu Fengqi > --- > v2: add -s option to specify subvol_id. > >

Re: [PATCH v2 06/10] btrfs-progs: undelete-subvol: introduce link_subvol_to_lostfound

2018-04-17 Thread Qu Wenruo
On 2018年03月27日 15:06, Lu Fengqi wrote: > The function will create lost+found directory, link the deleted > subvolume specified by the subvol_id to the directory, update the > information of root_item and cleanup the associated orphan item. > > Signed-off-by: Lu Fengqi

Re: [PATCH v2 09/10] btrfs-progs: tests: add testcase for undelete-subvol

2018-04-17 Thread Qu Wenruo
On 2018年03月27日 15:06, Lu Fengqi wrote: > The testcase checks the functionality of "btrfs rescue undelete-subvol", > including recovering an intact subvolume, and handling correctly > incomplete subvolume. > > Signed-off-by: Lu Fengqi > --- > v2: add shell quoting to

Re: [PATCH v2 02/10] btrfs-progs: extract btrfs_link_subvol from btrfs_mksubvol

2018-04-17 Thread Qu Wenruo
On 2018年03月27日 15:06, Lu Fengqi wrote: > The original btrfs_mksubvol is too specific to specify the directory that > the subvolume will link to. Furthermore, in this transaction, we don't only > need to create root_ref/dir-item, but also update the refs or flags of > root_item. Extract a generic

Re: [PATCH v2 07/10] btrfs-progs: undelete-subvol: introduce btrfs_undelete_intact_subvols

2018-04-17 Thread Qu Wenruo
On 2018年03月27日 15:06, Lu Fengqi wrote: > The function default will traverse the all orphan items on the tree root, > and recover the all intact subvolumes. If subvol_id is specified, then only > the corresponding subvolume will be recovered. > > Signed-off-by: Lu Fengqi

[RFC] Add support for BTRFS raid5/6 to GRUB

2018-04-17 Thread Goffredo Baroncelli
Hi All, Below you can find a patch to add support for accessing files from grub in a RAID5/6 btrfs filesystem. This is a RFC because it is missing the support for recovery (i.e. if some devices are missed). In the next days (weeks ?) I will extend this patch to support also this case.

Re: How to dump/find parity of RAID-5 file?

2018-04-17 Thread Goffredo Baroncelli
On 02/06/2017 09:40 PM, Goffredo Baroncelli wrote: > On 2017-02-03 11:44, Lakshmipathi.G wrote: >> Hi. >> >> Came across this thread >> https://www.mail-archive.com/linux-btrfs@vger.kernel.org/msg55161.html >> Exploring possibility of adding test-scripts around these area using >> dump-tree &

Re: [PATCH] btrfs: add comment about BTRFS_FS_EXCL_OP

2018-04-17 Thread David Sterba
On Mon, Apr 16, 2018 at 06:28:31PM +0800, Anand Jain wrote: > Adds comments about BTRFS_FS_EXCL_OP to existing comments > about the device locks. Thanks, a few comments below. > Signed-off-by: Anand Jain > --- > fs/btrfs/volumes.c | 16 > 1 file changed,

Re: [PATCH] Btrfs: clean up resources during umount after trans is aborted

2018-04-17 Thread Liu Bo
On Tue, Apr 17, 2018 at 03:34:04PM +0200, David Sterba wrote: > On Sat, Mar 31, 2018 at 06:11:56AM +0800, Liu Bo wrote: > > Currently if some fatal errors occur, like all IO get -EIO, resources > > would be cleaned up when > > a) transaction is being committed or > > b) BTRFS_FS_STATE_ERROR is set

Re: [PATCH 08/16] btrfs: add sanity check when resuming balance after mount

2018-04-17 Thread David Sterba
On Mon, Apr 16, 2018 at 02:10:08PM +0800, Anand Jain wrote: > > > On 04/04/2018 02:34 AM, David Sterba wrote: > > Replace a WARN_ON with a proper check and message in case something goes > > really wrong and resumed balance cannot set up its exclusive status. > > > The check is a user friendly

[PATCH] btrfs: push relocation recovery into a helper thread

2018-04-17 Thread Jeff Mahoney
On a file system with many snapshots and qgroups enabled, an interrupted balance can end up taking a long time to mount due to recovering the relocations during mount. It does this in the task performing the mount, which can't be interrupted and may prevent mounting (and systems booting) for a

Re: [PATCH v4 3/4] btrfs: Refactor btrfs_ioctl_snap_destroy() by using btrfs_delete_subvolume()

2018-04-17 Thread David Sterba
On Tue, Apr 17, 2018 at 09:21:16AM +0900, Misono Tomohiro wrote: > On 2018/04/17 2:53, David Sterba wrote: > > On Wed, Apr 11, 2018 at 02:20:49PM +0900, Misono Tomohiro wrote: > >> Use btrfs_delete_subvolume() to refactor btrfs_ioctl_snap_destroy(). > >> The permission check is still done in

Re: [PATCH 13/16] btrfs: remove redundant read-only check from btrfs_cancel_balance

2018-04-17 Thread David Sterba
On Mon, Apr 16, 2018 at 05:43:46PM +0800, Anand Jain wrote: > On 04/04/2018 02:34 AM, David Sterba wrote: > > Balance cannot be started on a read-only filesystem and will have to > > finish/exit before eg. going to read-only via remount. Cancelling does > > not need to check for that. > > > > In

[PATCH] btrfs: optimize free space tree bitmap conversion

2018-04-17 Thread Howard McLauchlan
Presently, convert_free_space_to_extents() does a linear scan of the bitmap. We can speed this up with find_next_{bit,zero_bit}_le(). This patch replaces the linear scan with find_next_{bit,zero_bit}_le(). Testing shows a 20-33% decrease in execution time for convert_free_space_to_extents().

[PATCH v5 0/3] Allow rmdir(2) to delete a subvolume

2018-04-17 Thread Misono Tomohiro
cangelog: v4 -> v5 ... Merge 2nd and 3rd patches and update commit log No code change in total v3 -> v4 ... Reorganize patches and update commit log No code change in total v2 -> v3 ... Use if-else block instead of two if blocks and add Tested-by

[PATCH v5 3/3] btrfs: Allow rmdir(2) to delete an empty subvolume

2018-04-17 Thread Misono Tomohiro
Change the behavior of rmdir(2) and allow it to delete an empty subvolume by using btrfs_delete_subvolume() which is used by btrfs_ioctl_snap_destroy(). The required lock for @dir and inode of @dentry is already acquired in vfs layer. We need some check before deleting a subvolume. Permission

[PATCH v5 2/3] btrfs: Factor out the main deletion process from btrfs_ioctl_snap_destroy()

2018-04-17 Thread Misono Tomohiro
Factor out the secand half of btrfs_ioctl_snap_destroy() as btrfs_delete_subvolume(), which performes some subvolume specific checks before deletion (1. send is not in progress. 2. the subvolume is not the default subvolume. 3. the subvolume does not contain other subvolumes) and actual deletion

[PATCH v5 1/3] btrfs: Move may_destroy_subvol() from ioctl.c to inode.c

2018-04-17 Thread Misono Tomohiro
This is a preparation work to refactor btrfs_ioctl_snap_destroy() and to allow rmdir(2) to delete an empty subvolume. Signed-off-by: Tomohiro Misono --- fs/btrfs/ctree.h | 1 + fs/btrfs/inode.c | 54 ++

[PATCH] btrfs-progs: add chattr support for send/receive

2018-04-17 Thread Howard McLauchlan
Presently, btrfs send/receive does not propagate inode attribute flags; all chattr operations are effectively discarded upon transmission. This patch adds userspace support for inode attribute flags. Kernel support can be found under the commit: btrfs: add chattr support for send/receive An

[PATCH] btrfs: add chattr support for send/receive

2018-04-17 Thread Howard McLauchlan
Presently btrfs send/receive does not propagate inode attribute flags; all chattr operations are effectively discarded upon transmission. This patch adds kernel support for inode attribute flags. Userspace support can be found under the commit: btrfs-progs: add chattr support for

Re: [PATCH] btrfs: add chattr support for send/receive

2018-04-17 Thread Howard McLauchlan
On 04/17/2018 04:39 PM, Howard McLauchlan wrote: > Presently btrfs send/receive does not propagate inode attribute flags; > all chattr operations are effectively discarded upon transmission. > > This patch adds kernel support for inode attribute flags. Userspace > support can be found under the

Re: Hard link not persisted on fsync

2018-04-17 Thread Jayashree Mohan
Hi, A gentle reminder on the crash consistency bug that we found on btrfs: Link count of a file is not persisted even after a fsync. We believe a filesystem that ensures strictly ordered metadata behaviour should be able to persist the hard link after a fsync on the original file. Could you

Re: [PATCH] btrfs: delayed-inode: Remove wrong qgroup meta reservation calls

2018-04-17 Thread Qu Wenruo
On 2018年04月17日 23:18, David Sterba wrote: > On Tue, Apr 17, 2018 at 04:52:45PM +0800, Qu Wenruo wrote: >> Commit 4f5427ccce5d ("btrfs: delayed-inode: Use new qgroup meta rsv for >> delayed inode and item") merged into mainline is not the updated version >> submitted to the mail list in Dec 2017.

Re: [PATCH v2 00/10] undelete subvolume offline version

2018-04-17 Thread Lu Fengqi
On Tue, Mar 27, 2018 at 03:06:48PM +0800, Lu Fengqi wrote: >This patchset will add undelete-subvol subcommand for btrfs rescue. This >enhancement allows undeleting a subvolume on an unmounted filesystem. Gentle ping. Any advice about this patchset will be welcomed. -- Thanks, Lu > >Patchset