Re: [PATCH 1/3] btrfs: Add incompat flags check for btrfs_check_super_valid()

On 04/21/2018 10:43 AM, Qu Wenruo wrote: On 2018年04月21日 10:38, Anand Jain wrote: On 04/20/2018 11:15 PM, David Sterba wrote: On Fri, Apr 20, 2018 at 10:32:03PM +0800, Anand Jain wrote: On 04/19/2018 05:38 PM, Qu Wenruo wrote: Although we have already checked incompat flags manually

Re: [PATCH 1/3] btrfs: Add incompat flags check for btrfs_check_super_valid()

On 2018年04月21日 10:38, Anand Jain wrote: > > > On 04/20/2018 11:15 PM, David Sterba wrote: >> On Fri, Apr 20, 2018 at 10:32:03PM +0800, Anand Jain wrote: >>> >>> >>> On 04/19/2018 05:38 PM, Qu Wenruo wrote: Although we have already checked incompat flags manually before really

Re: [PATCH 1/3] btrfs: Add incompat flags check for btrfs_check_super_valid()

On 04/20/2018 11:15 PM, David Sterba wrote: On Fri, Apr 20, 2018 at 10:32:03PM +0800, Anand Jain wrote: On 04/19/2018 05:38 PM, Qu Wenruo wrote: Although we have already checked incompat flags manually before really mounting it, we could still enhance btrfs_check_super_valid() to check

Directory entry not persisted on a fsync

Hi, We came across a scenario where inspite of fsync-ing the directory, the entry was not persisted - the file created under this directory was lost. Consider the following workload : 1. creat test/foo 2. mkdir test/A 3. creat test/A/foo 4. fsync test/A/foo 5. fsync test -crash- When

Re: [PATCH] btrfs: add chattr support for send/receive

On 04/18/2018 01:15 AM, Filipe Manana wrote: > On Wed, Apr 18, 2018 at 12:39 AM, Howard McLauchlan > wrote: >> Presently btrfs send/receive does not propagate inode attribute flags; >> all chattr operations are effectively discarded upon transmission. >> >> This patch adds

[PATCH 0/7] Add ioctl to support extended inode flags

The patchset implements the existing VFS ioctls for reading extended ioctl flags by btrfs. There are many flags/attributes/extended/combined, the naming is confusing, so let's recap what we have: * generic VFS inode flags (i_flags) - S_* namespace

[PATCH 1/7] btrfs: rename btrfs_update_iflags to reflect which flags it touches

The btrfs inode flag flavour is now simply called 'inode flags' and the vfs inode are i_flags. Also rename the internal variable to something less confusing than 'ip'. Signed-off-by: David Sterba --- fs/btrfs/ctree.h | 2 +- fs/btrfs/inode.c | 4 ++-- fs/btrfs/ioctl.c | 16

[PATCH 2/7] btrfs: rename btrfs_mask_flags to reflect which flags it touches

The FS_*_FL flags cannot be easily identified by a variable name prefix but we still need to recognize them so the 'fsflags' should be closer to the naming scheme but again the 'fs' part sounds like it's a filesystem flag. I don't have a better idea for now. Signed-off-by: David Sterba

[PATCH 5/7] btrfs: add helpers for FS_XFLAG_* conversion

Preparatory work for the FS_IOC_FSGETXATTR ioctl, basic conversions and checking helpers. Signed-off-by: David Sterba --- fs/btrfs/ioctl.c | 32 1 file changed, 32 insertions(+) diff --git a/fs/btrfs/ioctl.c b/fs/btrfs/ioctl.c index

[PATCH 6/7] btrfs: add FS_IOC_FSGETXATTR ioctl

The new ioctl is an extension to the FS_IOC_GETFLAGS and adds new flags and is extensible. This patch allows to return the xflags portion of the fsxattr structure, other items have no meaning for btrfs or can be added later. The original patch was written by Chandan Jay Sharma but was incomplete

[PATCH 7/7] btrfs: add FS_IOC_FSSETXATTR ioctl

The new ioctl is an extension to the FS_IOC_SETFLAGS and adds new flags and is extensible. Don't get fooled by the XATTR in the name, it does not have anything in common with the extended attributes, incidentally also abbreviated as XATTRs. This patch allows to set the xflags portion of the

[PATCH 4/7] btrfs: rename btrfs_flags_to_ioctl to reflect which flags it touches

Converts btrfs_inode::flags to the FS_*_FL flags. Signed-off-by: David Sterba --- fs/btrfs/ioctl.c | 9 + 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/fs/btrfs/ioctl.c b/fs/btrfs/ioctl.c index 953473f2a136..f0e6074233fa 100644 --- a/fs/btrfs/ioctl.c

[PATCH 3/7] btrfs: rename check_flags to reflect which flags it touches

The FS_*_FL flags cannot be easily identified by a prefix but we still need to recognize them so the 'fsflags' should be closer to the naming scheme but again the 'fs' part sounds like it's a filesystem flag. I don't have a better idea for now. Signed-off-by: David Sterba ---

Re: [PATCH 1/3] btrfs: Add incompat flags check for btrfs_check_super_valid()

On Fri, Apr 20, 2018 at 10:32:03PM +0800, Anand Jain wrote: > > > On 04/19/2018 05:38 PM, Qu Wenruo wrote: > > Although we have already checked incompat flags manually before really > > mounting it, we could still enhance btrfs_check_super_valid() to check > > incompat flags for later write time

Re: [PATCH 0/6] Add ioctl to clear unused space

On 2018-04-20 10:21, David Sterba wrote: This patchset adds new ioctl similar to TRIM, that provides several other ways how to clear the unused space. The changelogs are incomplete, for preview not for inclusion yet. +1 for the idea. This will be insanely useful for certain VM setups. It

Re: [PATCH 3/3] btrfs: Do super block verification before writing it to disk

@@ -3563,6 +3565,12 @@ int write_all_supers(struct btrfs_fs_info *fs_info, int max_mirrors) sb = fs_info->super_for_commit; dev_item = >dev_item; + if (btrfs_check_super_valid(fs_info, sb, -1)) { + btrfs_err(fs_info, + "superblock corruption

Re: [PATCH 1/3] btrfs: Add incompat flags check for btrfs_check_super_valid()

On 04/19/2018 05:38 PM, Qu Wenruo wrote: Although we have already checked incompat flags manually before really mounting it, we could still enhance btrfs_check_super_valid() to check incompat flags for later write time super block validation check. This patch adds such incompat flags check

[PATCH 4/6] btrfs: add new ioctl BTRFS_IOC_CLEAR_FREE

A new ioctl that will clear the free space (by writing zeros) given by the user range. Similar to the TRIM ioctl. We need a new ioctl for that because struct fstrim_range does not provide any existing or reserved member for extensions. The new ioctl also supports TRIM as the operation type.

[PATCH 6/6] btrfs: add more zeroout modes for clearing unused space

Signed-off-by: David Sterba --- fs/btrfs/extent-tree.c | 12 +++- include/uapi/linux/btrfs.h | 5 + 2 files changed, 16 insertions(+), 1 deletion(-) diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c index 285bace8e2c6..bd2ac5779998 100644 ---

[PATCH 2/6] btrfs: add wrapper to switch clearing operation

Signed-off-by: David Sterba --- fs/btrfs/extent-tree.c | 25 ++--- 1 file changed, 14 insertions(+), 11 deletions(-) diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c index ffdd3aba508c..b317f8ee42a9 100644 --- a/fs/btrfs/extent-tree.c +++

[PATCH 5/6] btrfs: add discard secure to clear unused space

Signed-off-by: David Sterba --- fs/btrfs/extent-tree.c | 14 -- include/uapi/linux/btrfs.h | 1 + 2 files changed, 13 insertions(+), 2 deletions(-) diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c index 21a24fff32dd..285bace8e2c6 100644 ---

[PATCH 3/6] btrfs: add zeroing clear operation

Signed-off-by: David Sterba --- fs/btrfs/ctree.h | 1 + fs/btrfs/extent-tree.c | 3 +++ 2 files changed, 4 insertions(+) diff --git a/fs/btrfs/ctree.h b/fs/btrfs/ctree.h index 7cde72683b8e..772cb4ccc5f7 100644 --- a/fs/btrfs/ctree.h +++ b/fs/btrfs/ctree.h @@ -679,6

[PATCH 0/6] Add ioctl to clear unused space

This patchset adds new ioctl similar to TRIM, that provides several other ways how to clear the unused space. The changelogs are incomplete, for preview not for inclusion yet. It compiles and has been tested lightly, the clearing modes depend on hw capabilities (secure discard, sector unmapping

[PATCH 1/6] btrfs: extend trim callchain to pass the operation type

This is preparatory work to implement clearing free space by zeroing, much in the same way the FITRIM ioctl works. This patch simply defines a symbolic name for the current clearing operation and passes the parameter around. No functional change. Signed-off-by: David Sterba

Re: [PATCH v2 12/16] btrfs: track running balance in a simpler way

Ok not that simple, the running status is checked outside of balance_mutex and there's one more assertion that does not expect the balance_ctl to exist: @@ -4031,16 +4032,16 @@ int btrfs_pause_balance(struct btrfs_fs_info *fs_info) return -ENOTCONN; } -

Re: [PATCH 1/3] btrfs: Add incompat flags check for btrfs_check_super_valid()

On Thu, Apr 19, 2018 at 07:24:48PM +0300, Nikolay Borisov wrote: > I agree with David, just make it btrfs_validate_super Ack. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at

Re: [PATCH v2 12/16] btrfs: track running balance in a simpler way

On Fri, Apr 20, 2018 at 01:58:11PM +0200, David Sterba wrote: > On Fri, Apr 20, 2018 at 03:52:24PM +0800, Anand Jain wrote: > > > > > > On 04/20/2018 12:33 AM, David Sterba wrote: > > > Currently fs_info::balance_running is 0 or 1 and does not use the > > > semantics of atomics. The pause and

[PATCH v2.1 13/16] btrfs: move and comment read-only check in btrfs_cancel_balance

Balance cannot be started on a read-only filesystem and will have to finish/exit before eg. going to read-only via remount. In case the filesystem is forcibly set to read-only after an error, balance will finish anyway and if the cancel call is too fast it will just wait for that to happen. The

Re: [PATCH v2 13/16] btrfs: move and comment read-only check in btrfs_cancel_balance

On Fri, Apr 20, 2018 at 05:13:02PM +0800, Anand Jain wrote: > > > On 04/20/2018 12:33 AM, David Sterba wrote: > > Balance cannot be started on a read-only filesystem and will have to > > finish/exit before eg. going to read-only via remount. > > It can be paused as well. > btrfs balance

Re: [PATCH v2 12/16] btrfs: track running balance in a simpler way

On Fri, Apr 20, 2018 at 03:52:24PM +0800, Anand Jain wrote: > > > On 04/20/2018 12:33 AM, David Sterba wrote: > > Currently fs_info::balance_running is 0 or 1 and does not use the > > semantics of atomics. The pause and cancel check for 0, that can happen > > only after __btrfs_balance exits for

Re: Hard link not persisted on fsync

Hi Zygo, Thanks for the reply. Here are few responses about btrfs behavior that you had questions about in your email. On Thu, Apr 19, 2018 at 4:41 PM, Zygo Blaxell wrote: > > On Mon, Apr 16, 2018 at 09:35:24AM -0500, Jayashree Mohan wrote: > > Hi, > > > > The

Re: [PATCH v2 09/16] btrfs: cleanup helpers that reset balance state

On Fri, Apr 20, 2018 at 10:07:17AM +0300, Nikolay Borisov wrote: > > > On 19.04.2018 19:33, David Sterba wrote: > > + /* reset_balance_state needs volume_mutex */ > > Does it make sense to codify this invariant as lockdep_assert_held in > reset_balance_state ? No, the comment and the

Re: [bug report] btrfs fail when run xfstests-dev/btrfs/007

On Fri, Apr 20, 2018 at 10:23 AM, Gu, Jinxiang wrote: > Hi, > > Here's a bug report. > Kernel v4.17-rc1 failed xfstests-dev/btrfs/007. > It is not always happens.( occurred times/test times: 1/20) You don't need to check how many times it fails... You have a fsstress seed

[PATCH] btrfs: fix BUG trying to resume balance without resume flag

We set the BTRFS_BALANCE_RESUME flag in the btrfs_recover_balance(), which is not called during the remount. So when resuming from the paused balance we hit BUG. kernel: kernel BUG at fs/btrfs/volumes.c:3890! :: kernel: balance_kthread+0x51/0x60 [btrfs] kernel: kthread+0x111/0x130 ::

Re: [PATCH v2 15/16] btrfs: use mutex in btrfs_resume_balance_async

On 04/20/2018 12:33 AM, David Sterba wrote: While the spinlock does not cause problems, using the mutex is more correct and consistent with others. The global status of balance is eg. checked from btrfs_pause_balance or btrfs_cancel_balance with mutex. Resuming balance happens during mount or

[bug report] btrfs fail when run xfstests-dev/btrfs/007

Hi, Here's a bug report. Kernel v4.17-rc1 failed xfstests-dev/btrfs/007. It is not always happens.( occurred times/test times: 1/20) And I confirmed this test using kernel v4.16-rc1. It also occurred sometimes. LOG when using kernel v4.17-rc1. btrfs/007[failed, exit status 1] - output

Re: [PATCH v2 14/16] btrfs: drop lock parameter from update_ioctl_balance_args and rename

On 04/20/2018 12:33 AM, David Sterba wrote: The parameter controls locking of the stats part but we can lock it unconditionally, as this only happens once when balance starts. This is not performance critical. Add the prefix for an exported function. Signed-off-by: David Sterba

Re: Recovery from full metadata with all device space consumed?

Timofey Titovets posted on Fri, 20 Apr 2018 01:32:42 +0300 as excerpted: > 2018-04-20 1:08 GMT+03:00 Drew Bloechl : >> I've got a btrfs filesystem that I can't seem to get back to a useful >> state. The symptom I started with is that rename() operations started >> dying with

Re: [PATCH v2 13/16] btrfs: move and comment read-only check in btrfs_cancel_balance

On 04/20/2018 12:33 AM, David Sterba wrote: Balance cannot be started on a read-only filesystem and will have to finish/exit before eg. going to read-only via remount. It can be paused as well. btrfs balance pause /btrfs && mount -o remount,ro /dev/sdb /btrfs @@ -4053,15 +4053,20 @@

Re: [PATCH v2 09/16] btrfs: cleanup helpers that reset balance state

On 04/20/2018 12:33 AM, David Sterba wrote: The function __cancel_balance name is confusing with the cancel operation of balance and it really resets the state of balance back to zero. The unset_balance_control helper is called only from one place and simple enough to be inlined.

Re: [PATCH v2 12/16] btrfs: track running balance in a simpler way

On 04/20/2018 12:33 AM, David Sterba wrote: Currently fs_info::balance_running is 0 or 1 and does not use the semantics of atomics. The pause and cancel check for 0, that can happen only after __btrfs_balance exits for whatever reason. Parallel calls to balance ioctl may enter

Re: [PATCH v2 08/16] btrfs: add sanity check when resuming balance after mount

On 04/20/2018 12:33 AM, David Sterba wrote: Replace a WARN_ON with a proper check and message in case something goes really wrong and resumed balance cannot set up its exclusive status. The check is a user friendly assertion, I don't expect to ever happen under normal circumstances. Also

Re: [PATCH v2 07/16] btrfs: add proper safety check before resuming dev-replace

On 04/20/2018 12:33 AM, David Sterba wrote: The device replace is paused by unmount or read only remount, and resumed on next mount or write remount. The exclusive status should be checked properly as it's a global invariant and we must not allow 2 operations run. In this case, the balance

Re: [PATCH v2 06/16] btrfs: move clearing of EXCL_OP out of __cancel_balance

On 04/20/2018 12:33 AM, David Sterba wrote: Make the clearning visible in the callers so we can pair it with the test_and_set part. Signed-off-by: David Sterba Reviewed-by: Anand Jain Thanks, Anand --- fs/btrfs/ioctl.c | 2 +-

Re: [PATCH v2 09/16] btrfs: cleanup helpers that reset balance state

On 19.04.2018 19:33, David Sterba wrote: > + /* reset_balance_state needs volume_mutex */ Does it make sense to codify this invariant as lockdep_assert_held in reset_balance_state ? -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to

Re: [PATCH v2 07/16] btrfs: add proper safety check before resuming dev-replace

On 19.04.2018 19:33, David Sterba wrote: > The device replace is paused by unmount or read only remount, and > resumed on next mount or write remount. > > The exclusive status should be checked properly as it's a global > invariant and we must not allow 2 operations run. In this case, the >

Re: [PATCH v2 06/16] btrfs: move clearing of EXCL_OP out of __cancel_balance

On 19.04.2018 19:33, David Sterba wrote: > Make the clearning visible in the callers so we can pair it with the > test_and_set part. > > Signed-off-by: David Sterba Reviewed-by: Nikolay Borisov > --- > fs/btrfs/ioctl.c | 2 +- > fs/btrfs/volumes.c |

Re: [PATCH RESEND v4 0/4] device_list_add() peparation to add reappearing missing device

On 04/20/2018 02:22 PM, Gu, Jinxiang wrote: Hi, I reproduced this using kernel v4.17-rc1. It is not always happens.( occurred times/test times: 1/20) Though it was reported here, its not related to this patch set. Instead its about the send receive. Pls post it as a separate thread so it

RE: [PATCH RESEND v4 0/4] device_list_add() peparation to add reappearing missing device

Hi, I reproduced this using kernel v4.17-rc1. It is not always happens.( occurred times/test times: 1/20) > -Original Message- > From: linux-btrfs-ow...@vger.kernel.org > [mailto:linux-btrfs-ow...@vger.kernel.org] On Behalf Of Anand Jain > Sent: Tuesday, January 23, 2018 5:53 AM > To: