Re: [PATCH] btrfs: Fix suspicious RCU usage warning in device_list_add

2018-11-14 Thread Anand Jain
On 11/14/2018 03:24 PM, Lu Fengqi wrote: = WARNING: suspicious RCU usage 4.20.0-rc2+ #23 Tainted: G O - fs/btrfs/volumes.c:886 suspicious rcu_dereference_check() usage! Use btrfs_info_in_rcu instead of pr_info for the required

Re: [PATCH v4.1b 2/3] btrfs: balance: add args info during start and resume

2018-11-14 Thread Anand Jain
On 05/31/2018 05:47 PM, David Sterba wrote: On Fri, May 25, 2018 at 11:05:47AM +0800, Anand Jain wrote: Balance arg info is an important information to be reviewed for the system audit. So this patch adds them to the kernel log. Example: ->btrfs bal start -f

Re: [PATCH] btrfs: introduce feature to forget a btrfs device

2018-11-14 Thread Filipe Manana
On Wed, Nov 14, 2018 at 11:15 AM Filipe Manana wrote: > > On Wed, Nov 14, 2018 at 9:14 AM Anand Jain wrote: > > > > Support for a new command 'btrfs dev forget [dev]' is proposed here > > to undo the effects of 'btrfs dev scan [dev]'. For this purpose > > this patch proposes to use ioctl #5 as

Re: [PATCH] btrfs: introduce feature to forget a btrfs device

2018-11-14 Thread Filipe Manana
On Wed, Nov 14, 2018 at 9:14 AM Anand Jain wrote: > > Support for a new command 'btrfs dev forget [dev]' is proposed here > to undo the effects of 'btrfs dev scan [dev]'. For this purpose > this patch proposes to use ioctl #5 as it was empty. > IOW(BTRFS_IOCTL_MAGIC, 5, ..) > This patch

Re: [PATCH] btrfs: Fix suspicious RCU usage warning in device_list_add

2018-11-13 Thread Nikolay Borisov
On 14.11.18 г. 9:24 ч., Lu Fengqi wrote: > = > WARNING: suspicious RCU usage > 4.20.0-rc2+ #23 Tainted: G O > - > fs/btrfs/volumes.c:886 suspicious rcu_dereference_check() usage! > > Use btrfs_info_in_rcu instead of pr_info for

Re: [PATCH v15.1 00/13] Btrfs In-band De-duplication

2018-11-13 Thread Lu Fengqi
On Tue, Nov 13, 2018 at 02:45:45PM +0100, David Sterba wrote: >On Tue, Nov 06, 2018 at 02:41:09PM +0800, Lu Fengqi wrote: >> This patchset can be fetched from github: >> https://github.com/littleroad/linux.git dedupe_latest >> >> Now the new base is v4.20-rc1. > >Before anybody spends more time

Re: [PATCH 4/9] btrfs: fix UAF due to race between replace start and cancel

2018-11-13 Thread Anand Jain
On 11/14/2018 01:24 AM, David Sterba wrote: On Sun, Nov 11, 2018 at 10:22:19PM +0800, Anand Jain wrote: replace cancel thread can race with the replace start thread and if fs_info::scrubs_running is not yet set the btrfs_scrub_cancel() will fail to stop the scrub thread, so the scrub thread

Re: [PATCH V7] Btrfs: enhance raid1/10 balance heuristic

2018-11-13 Thread Anand Jain
I am ok with the least used path approach here for the IO routing that's probably most reasonable in generic configurations. It can be default read mirror policy as well. But as I mentioned. Not all configurations would agree to the heuristic approach here. For example: To make use of the SAN

Re: [PATCH v2 0/6] btrfs: qgroup: Delay subtree scan to reduce overhead

2018-11-13 Thread Qu Wenruo
On 2018/11/14 上午1:58, Filipe Manana wrote: > On Tue, Nov 13, 2018 at 5:08 PM David Sterba wrote: >> >> On Mon, Nov 12, 2018 at 10:33:33PM +0100, David Sterba wrote: >>> On Thu, Nov 08, 2018 at 01:49:12PM +0800, Qu Wenruo wrote: This patchset can be fetched from github:

Re: [PATCH V7] Btrfs: enhance raid1/10 balance heuristic

2018-11-13 Thread Goffredo Baroncelli
On 12/11/2018 12.58, Timofey Titovets wrote: > From: Timofey Titovets > > Currently btrfs raid1/10 balancer bаlance requests to mirrors, > based on pid % num of mirrors. [...] > v6 -> v7: > - Fixes based on Nikolay Borisov review: > * Assume num == 2 > * Remove "for" loop based

Re: [PATCH v2 0/6] btrfs: qgroup: Delay subtree scan to reduce overhead

2018-11-13 Thread Filipe Manana
On Tue, Nov 13, 2018 at 5:08 PM David Sterba wrote: > > On Mon, Nov 12, 2018 at 10:33:33PM +0100, David Sterba wrote: > > On Thu, Nov 08, 2018 at 01:49:12PM +0800, Qu Wenruo wrote: > > > This patchset can be fetched from github: > > >

Re: [PATCH 0/9 v2] fix replace-start and replace-cancel racing

2018-11-13 Thread David Sterba
On Sun, Nov 11, 2018 at 10:22:15PM +0800, Anand Jain wrote: > v1->v2: > 2/9: Drop writeback required > 3/9: Drop writeback required > 7/9: Use the condition within the WARN_ON() > 6/9: Use the condition within the ASSERT() > > Replace-start and replace-cancel threads can race to create a

Re: [PATCH 4/9] btrfs: fix UAF due to race between replace start and cancel

2018-11-13 Thread David Sterba
On Sun, Nov 11, 2018 at 10:22:19PM +0800, Anand Jain wrote: > replace cancel thread can race with the replace start thread and if > fs_info::scrubs_running is not yet set the btrfs_scrub_cancel() will fail > to stop the scrub thread, so the scrub thread continue with the scrub for > replace which

Re: [PATCH v2 0/6] btrfs: qgroup: Delay subtree scan to reduce overhead

2018-11-13 Thread David Sterba
On Mon, Nov 12, 2018 at 10:33:33PM +0100, David Sterba wrote: > On Thu, Nov 08, 2018 at 01:49:12PM +0800, Qu Wenruo wrote: > > This patchset can be fetched from github: > > https://github.com/adam900710/linux/tree/qgroup_delayed_subtree_rebased > > > > Which is based on v4.20-rc1. > > Thanks,

Re: [PATCH 0/6] Chunk allocator DUP fix and cleanups

2018-11-13 Thread Hans van Kranenburg
On 11/13/18 4:03 PM, David Sterba wrote: > On Thu, Oct 11, 2018 at 07:40:22PM +, Hans van Kranenburg wrote: >> On 10/11/2018 05:13 PM, David Sterba wrote: >>> On Thu, Oct 04, 2018 at 11:24:37PM +0200, Hans van Kranenburg wrote: This patch set contains an additional fix for a newly exposed

Re: [PATCH RFC] btrfs: harden agaist duplicate fsid

2018-11-13 Thread Anand Jain
On 11/13/2018 11:31 PM, David Sterba wrote: On Mon, Oct 01, 2018 at 09:31:04PM +0800, Anand Jain wrote: +    /* + * we are going to replace the device path, make sure its the + * same device if the device mounted + */ +    if (device->bdev) { +   

Re: [PATCH RFC RESEND] btrfs: harden agaist duplicate fsid

2018-11-13 Thread Anand Jain
On 11/13/2018 11:21 PM, David Sterba wrote: On Mon, Oct 15, 2018 at 10:45:17AM +0800, Anand Jain wrote: (Thanks for the comments on requiring to warn_on if we fail the device change.) (This fixes an ugly bug, I appreciate if you have any further comments). Its not that impossible to imagine

Re: [PATCH RFC] btrfs: harden agaist duplicate fsid

2018-11-13 Thread Austin S. Hemmelgarn
On 11/13/2018 10:31 AM, David Sterba wrote: On Mon, Oct 01, 2018 at 09:31:04PM +0800, Anand Jain wrote: +    /* + * we are going to replace the device path, make sure its the + * same device if the device mounted + */ +    if (device->bdev) { +    struct

Re: [PATCH RFC] btrfs: harden agaist duplicate fsid

2018-11-13 Thread David Sterba
On Mon, Oct 01, 2018 at 09:31:04PM +0800, Anand Jain wrote: > >> +    /* > >> + * we are going to replace the device path, make sure its the > >> + * same device if the device mounted > >> + */ > >> +    if (device->bdev) { > >> +    struct block_device

Re: [PATCH RFC RESEND] btrfs: harden agaist duplicate fsid

2018-11-13 Thread David Sterba
On Mon, Oct 15, 2018 at 10:45:17AM +0800, Anand Jain wrote: > (Thanks for the comments on requiring to warn_on if we fail the device > change.) > (This fixes an ugly bug, I appreciate if you have any further comments). > > Its not that impossible to imagine that a device OR a btrfs image is >

Re: [PATCH 0/6] Chunk allocator DUP fix and cleanups

2018-11-13 Thread David Sterba
On Thu, Oct 11, 2018 at 07:40:22PM +, Hans van Kranenburg wrote: > On 10/11/2018 05:13 PM, David Sterba wrote: > > On Thu, Oct 04, 2018 at 11:24:37PM +0200, Hans van Kranenburg wrote: > >> This patch set contains an additional fix for a newly exposed bug after > >> the previous attempt to fix

Re: [PATCH] Btrfs: do not set log for full commit when creating non-data block groups

2018-11-13 Thread Filipe Manana
On Tue, Nov 13, 2018 at 2:31 PM David Sterba wrote: > > On Thu, Nov 08, 2018 at 02:48:29PM +, Filipe Manana wrote: > > On Thu, Nov 8, 2018 at 2:37 PM Filipe Manana wrote: > > > > > > On Thu, Nov 8, 2018 at 2:35 PM Qu Wenruo wrote: > > > > > > > > > > > > > > > > On 2018/11/8 下午9:17,

Re: [PATCH v4] Btrfs: fix deadlock on tree root leaf when finding free extent

2018-11-13 Thread David Sterba
On Mon, Nov 05, 2018 at 04:36:34PM +, Filipe Manana wrote: > On Mon, Nov 5, 2018 at 4:34 PM David Sterba wrote: > > On Mon, Nov 05, 2018 at 04:30:35PM +, Filipe Manana wrote: > > > On Mon, Nov 5, 2018 at 4:29 PM David Sterba wrote: > > > > On Wed, Oct 24, 2018 at 01:48:40PM +0100, Filipe

Re: [PATCH] Btrfs: do not set log for full commit when creating non-data block groups

2018-11-13 Thread David Sterba
On Thu, Nov 08, 2018 at 02:48:29PM +, Filipe Manana wrote: > On Thu, Nov 8, 2018 at 2:37 PM Filipe Manana wrote: > > > > On Thu, Nov 8, 2018 at 2:35 PM Qu Wenruo wrote: > > > > > > > > > > > > On 2018/11/8 下午9:17, fdman...@kernel.org wrote: > > > > From: Filipe Manana > > > > > > > > When

Re: [PATCH v15.1 00/13] Btrfs In-band De-duplication

2018-11-13 Thread David Sterba
On Tue, Nov 06, 2018 at 02:41:09PM +0800, Lu Fengqi wrote: > This patchset can be fetched from github: > https://github.com/littleroad/linux.git dedupe_latest > > Now the new base is v4.20-rc1. Before anybody spends more time with this patchset: this is a big feature and quite intrusive to

Re: [PATCH v2] btrfs: add zstd compression level support

2018-11-13 Thread Timofey Titovets
вт, 13 нояб. 2018 г. в 04:52, Nick Terrell : > > > > > On Nov 12, 2018, at 4:33 PM, David Sterba wrote: > > > > On Wed, Oct 31, 2018 at 11:11:08AM -0700, Nick Terrell wrote: > >> From: Jennifer Liu > >> > >> Adds zstd compression level support to btrfs. Zstd requires > >> different amounts of

Re: [PATCH] Btrfs: fix rare chances for data loss when doing a fast fsync

2018-11-13 Thread David Sterba
On Mon, Nov 12, 2018 at 10:23:58AM +, fdman...@kernel.org wrote: > From: Filipe Manana > > After the simplification of the fast fsync patch done recently by commit > b5e6c3e170b7 ("btrfs: always wait on ordered extents at fsync time") and > commit e7175a692765 ("btrfs: remove the wait

Re: [PATCH 1/3] bitops: Fix big endian compilation

2018-11-13 Thread David Sterba
On Mon, Nov 05, 2018 at 11:06:41AM -0800, Rosen Penev wrote: > Replaced bswap with _ variants. While it's a glibc extension, all of the > popular libc implementations (glibc, uClibc, musl, BIONIC) seem to support > it. > > Added static inline to two functions to match little endian variants. This

Re: [RFC] BTRFS_DEV_REPLACE_ITEM_STATE_* doesn't match with on disk

2018-11-13 Thread Anand Jain
David, Gentle ping. Thanks, Anand On 11/12/2018 03:50 PM, Nikolay Borisov wrote: On 12.11.18 г. 6:58 ч., Anand Jain wrote: The dev_replace_state defines are miss matched between the BTRFS_IOCTL_DEV_REPLACE_STATE_* and BTRFS_DEV_REPLACE_ITEM_STATE_* [1]. [1] -

Re: [PATCH] btrfs: fix computation of max fs size for multiple device fs tests

2018-11-13 Thread Anand Jain
On 11/06/2018 11:34 PM, fdman...@kernel.org wrote: From: Filipe Manana We were sorting numerical values with the 'sort' tool without telling it that we are sorting numbers, giving us unexpected ordering. So just pass the '-n' option to the 'sort' tool. Example: $ echo -e "11\n9\n20" |

Re: [PATCH] btrfs: qgroups: Prepare to deprecate BTRFS_QGROUP_LIMIT_RSV_RFER/EXCL flags

2018-11-13 Thread Qu Wenruo
On 2018/11/13 下午5:48, Nikolay Borisov wrote: > > > On 13.11.18 г. 8:09 ч., Qu Wenruo wrote: >> These two flags are only used to set btrfs_qgroup::rsv_rfer/excl number, >> but then are never used. >> >> Nor these flags are really used by btrfs-progs, so it's pretty safe just >> to deprecate

Re: [PATCH 1/2] btrfs-progs: ctree.h: Deprecate unused limit flags

2018-11-13 Thread Nikolay Borisov
On 13.11.18 г. 8:09 ч., Qu Wenruo wrote: > BTRFS_QGROUP_LIMIT_RSV_RFER/EXCL are never used by btrfs-progs. > And kernel only do the basic assignment but don't really make use of > them. > > So deprecate these two flags in btrfs-progs. > Also, add some comment for all the used flags. > >

Re: [PATCH] btrfs: qgroup: Remove duplicated trace points for qgroup_rsv_add/release()

2018-11-13 Thread Nikolay Borisov
On 13.11.18 г. 9:05 ч., Qu Wenruo wrote: > Inside qgroup_rsv_add/release(), we have trace events > trace_qgroup_update_reserve() to catch reserved space update. > > However we still have two manual trace_qgroup_update_reserve() calls > just outside these functions. > > Remove these duplicated

Re: [PATCH] btrfs: qgroups: Prepare to deprecate BTRFS_QGROUP_LIMIT_RSV_RFER/EXCL flags

2018-11-13 Thread Nikolay Borisov
On 13.11.18 г. 8:09 ч., Qu Wenruo wrote: > These two flags are only used to set btrfs_qgroup::rsv_rfer/excl number, > but then are never used. > > Nor these flags are really used by btrfs-progs, so it's pretty safe just > to deprecate them in next kernel release. > > This patch will mark

Re: [PATCH 2/2] btrfs-progs: gitignore: Ignore compressed man pages

2018-11-12 Thread Qu Wenruo
On 2018/11/13 下午2:09, Qu Wenruo wrote: > Just to Make "git status" more clean. > > Signed-off-by: Qu Wenruo Please discard this one patch, as now we don't build .gz man pages anymore. > --- > .gitignore | 1 + > 1 file changed, 1 insertion(+) > > diff --git a/.gitignore b/.gitignore >

Re: [PATCH v2] btrfs: add zstd compression level support

2018-11-12 Thread Nick Terrell
> On Nov 12, 2018, at 4:33 PM, David Sterba wrote: > > On Wed, Oct 31, 2018 at 11:11:08AM -0700, Nick Terrell wrote: >> From: Jennifer Liu >> >> Adds zstd compression level support to btrfs. Zstd requires >> different amounts of memory for each level, so the design had >> to be modified to

Re: [PATCH v2] btrfs: add zstd compression level support

2018-11-12 Thread David Sterba
On Wed, Oct 31, 2018 at 11:11:08AM -0700, Nick Terrell wrote: > From: Jennifer Liu > > Adds zstd compression level support to btrfs. Zstd requires > different amounts of memory for each level, so the design had > to be modified to allow set_level() to allocate memory. We > preallocate one

Re: [PATCH v2] btrfs: add zstd compression level support

2018-11-12 Thread David Sterba
On Wed, Oct 31, 2018 at 11:11:08AM -0700, Nick Terrell wrote: > From: Jennifer Liu > > Adds zstd compression level support to btrfs. Zstd requires > different amounts of memory for each level, so the design had > to be modified to allow set_level() to allocate memory. We > preallocate one

Re: [PATCH v9 0/6] Btrfs: implement swap file support

2018-11-12 Thread David Sterba
On Fri, Nov 09, 2018 at 08:12:39PM -0800, Omar Sandoval wrote: > On Wed, Nov 07, 2018 at 04:28:10PM +0100, David Sterba wrote: > > On Wed, Nov 07, 2018 at 05:07:00PM +0200, Nikolay Borisov wrote: > > > > > > > > > On 7.11.18 г. 16:49 ч., David Sterba wrote: > > > > On Tue, Nov 06, 2018 at

Re: [PATCH v2 0/6] btrfs: qgroup: Delay subtree scan to reduce overhead

2018-11-12 Thread David Sterba
On Thu, Nov 08, 2018 at 01:49:12PM +0800, Qu Wenruo wrote: > This patchset can be fetched from github: > https://github.com/adam900710/linux/tree/qgroup_delayed_subtree_rebased > > Which is based on v4.20-rc1. Thanks, I'll add it to for-next soon.

Re: [PATCH] btrfs: Always try all copies when reading extent buffers

2018-11-12 Thread David Sterba
24 which would > spuriously fail its balance operations. The reason was that when balance > was run following re-introduction of the missing raid1 disk > __btrfs_map_block would map the read request to stripe 0, which > corresponded to devid 2 (the disk which is being removed in the test)

Re: [PATCH v2] btrfs: use tagged writepage to mitigate livelock of snapshot

2018-11-12 Thread David Sterba
On Fri, Nov 02, 2018 at 05:06:07PM +0800, Ethan Lien wrote: > Snapshot is expected to be fast. But if there are writers steadily > create dirty pages in our subvolume, the snapshot may take a very long > time to complete. To fix the problem, we use tagged writepage for > snapshot flusher as we do

Re: [PATCH 3/3] btrfs: Remove unused extent_state argument from btrfs_writepage_endio_finish_ordered

2018-11-12 Thread David Sterba
On Thu, Nov 08, 2018 at 10:18:08AM +0200, Nikolay Borisov wrote: > This parameter was never used, yet was part of the interface of the > function ever since its introduction as extent_io_ops::writepage_end_io_hook > in e6dcd2dc9c48 ("Btrfs: New data=ordered implementation"). Now that > NULL is

Re: [PATCH 0/3] Cleanups following optional extent_io_ops callbacks removal

2018-11-12 Thread David Sterba
On Thu, Nov 08, 2018 at 10:18:05AM +0200, Nikolay Borisov wrote: > Here are 3 minor patches that further clean up writepage_delalloc. The first > one moves the extent locked check in the caller of writepage_delalloc since > this seems more natural. This paves the way for the second patch which

Re: [PATCH] btrfs: Check for missing device before bio submission in btrfs_map_bio

2018-11-12 Thread David Sterba
On Thu, Nov 08, 2018 at 04:16:38PM +0200, Nikolay Borisov wrote: > Before btrfs_map_bio submits all stripe bio it does a number of checks > to ensure the device for every stripe is present. However, it doesn't > do a DEV_STATE_MISSING check, instead this is relegated to the lower > level

Re: [PATCH] btrfs: remove redundant replace_state init

2018-11-12 Thread David Sterba
On Mon, Nov 12, 2018 at 01:05:15PM +0800, Anand Jain wrote: > dev_replace::replace_state has been set to > BTRFS_DEV_REPLACE_ITEM_STATE_NEVER_STARTED (0) in the same function, > So delete the line which sets replace_state = 0; > > Signed-off-by: Anand Jain Added to misc-next, thanks.

Re: [PATCH] Btrfs: remove no longer used io_err from btrfs_log_ctx

2018-11-12 Thread David Sterba
On Mon, Nov 12, 2018 at 10:24:30AM +, fdman...@kernel.org wrote: > From: Filipe Manana > > The io_err field of struct btrfs_log_ctx is no longer used after the > recent simplification of the fast fsync path, where we now wait for > ordered extents to complete before logging the inode. We did

Re: [PATCH] Btrfs: simpler and more efficient cleanup of a log tree's extent io tree

2018-11-12 Thread David Sterba
On Fri, Nov 09, 2018 at 10:43:08AM +, fdman...@kernel.org wrote: > From: Filipe Manana > > We currently are in a loop finding each range (corresponding to a btree > node/leaf) in a log root's extent io tree and then clean it up. This is a > waste of time since we are traversing the extent io

Re: [PATCHv2 0/9] Removal of optionsl extent_io_ops callbacks

2018-11-12 Thread David Sterba
On Fri, Nov 09, 2018 at 04:12:20PM +0200, Nikolay Borisov wrote: > > > On 9.11.18 г. 16:08 ч., Nikolay Borisov wrote: > > Here is the 2nd revision of the extent_io_ops optinal callbacks removal. > > What > > prompted this version was that I identified a flaw in the initial version - > > while

Re: [PATCH] Btrfs: remove no longer used io_err from btrfs_log_ctx

2018-11-12 Thread Josef Bacik
On Mon, Nov 12, 2018 at 10:24:30AM +, fdman...@kernel.org wrote: > From: Filipe Manana > > The io_err field of struct btrfs_log_ctx is no longer used after the > recent simplification of the fast fsync path, where we now wait for > ordered extents to complete before logging the inode. We did

Re: [PATCH] Btrfs: fix rare chances for data loss when doing a fast fsync

2018-11-12 Thread Josef Bacik
On Mon, Nov 12, 2018 at 10:23:58AM +, fdman...@kernel.org wrote: > From: Filipe Manana > > After the simplification of the fast fsync patch done recently by commit > b5e6c3e170b7 ("btrfs: always wait on ordered extents at fsync time") and > commit e7175a692765 ("btrfs: remove the wait

Re: [Regression bisected] btrfs: always wait on ordered extents at fsync time

2018-11-12 Thread Josef Bacik
On Fri, Nov 09, 2018 at 10:56:42PM +, Mel Gorman wrote: > On Fri, Nov 09, 2018 at 09:51:48PM +, Mel Gorman wrote: > > Unfortunately, as > > I'm about to travel, I didn't attempt a revert and a test comparing 4.18, > > 4.19 and 4.20-rc1 is a few hours away so this could potentially be fixed

Re: why /dev/ptmx is in the snapshots?

2018-11-12 Thread Marc Pignat
On 11/12/18 1:50 PM, Remi Gauvin wrote: > On 2018-11-12 07:26 AM, Marc Pignat wrote: ... > fuser /mnt/sda3/@/dev/ptmx Done the mount somewhere else thing, still no user of this file... > > Depending on how Ubuntu was installed, you might not have the @ > subvolume. I think then, root is just

Re: [PATCH V6] Btrfs: enhance raid1/10 balance heuristic

2018-11-12 Thread Timofey Titovets
пн, 12 нояб. 2018 г. в 10:28, Nikolay Borisov : > > > > On 25.09.18 г. 21:38 ч., Timofey Titovets wrote: > > Currently btrfs raid1/10 balancer bаlance requests to mirrors, > > based on pid % num of mirrors. > > > > Make logic understood: > > - if one of underline devices are non rotational > > -

Re: [PATCH] Btrfs: remove no longer used io_err from btrfs_log_ctx

2018-11-12 Thread Nikolay Borisov
On 12.11.18 г. 12:24 ч., fdman...@kernel.org wrote: > From: Filipe Manana > > The io_err field of struct btrfs_log_ctx is no longer used after the > recent simplification of the fast fsync path, where we now wait for > ordered extents to complete before logging the inode. We did this in >

Re: [RFC] BTRFS_DEV_REPLACE_ITEM_STATE_* doesn't match with on disk

2018-11-11 Thread Nikolay Borisov
On 12.11.18 г. 6:58 ч., Anand Jain wrote: > > The dev_replace_state defines are miss matched between the > BTRFS_IOCTL_DEV_REPLACE_STATE_* and BTRFS_DEV_REPLACE_ITEM_STATE_* [1]. > > [1] > - > btrfs.h:#define BTRFS_IOCTL_DEV_REPLACE_STATE_FINISHED    2 >

Re: [PATCH] btrfs: remove redundant replace_state init

2018-11-11 Thread Nikolay Borisov
On 12.11.18 г. 7:05 ч., Anand Jain wrote: > dev_replace::replace_state has been set to > BTRFS_DEV_REPLACE_ITEM_STATE_NEVER_STARTED (0) in the same function, > So delete the line which sets replace_state = 0; > > Signed-off-by: Anand Jain Reviewed-by: Nikolay Borisov > --- >

Re: [PATCH V6] Btrfs: enhance raid1/10 balance heuristic

2018-11-11 Thread Nikolay Borisov
On 25.09.18 г. 21:38 ч., Timofey Titovets wrote: > Currently btrfs raid1/10 balancer bаlance requests to mirrors, > based on pid % num of mirrors. > > Make logic understood: > - if one of underline devices are non rotational > - Queue length to underline devices > > By default try use pid %

Re: Full filesystem btrfs rebalance kernel panic to read-only lock

2018-11-11 Thread Qu Wenruo
On 2018/11/12 下午1:30, Anand Jain wrote: [snip] Btrfs-progs could do it with some extra dirty work. (I purposed offline device resize idea, but didn't implement it yet) You could use this branch: https://github.com/adam900710/btrfs-progs/tree/dirty_fix >>> >>> Qu, >>> >>>

Re: Full filesystem btrfs rebalance kernel panic to read-only lock

2018-11-11 Thread Anand Jain
On 11/12/2018 10:12 AM, Qu Wenruo wrote: On 2018/11/12 上午9:35, Anand Jain wrote: On 11/09/2018 09:21 AM, Qu Wenruo wrote: On 2018/11/9 上午6:40, Pieter Maes wrote: Hello, So, I've had the full disk issue, so when I tried re-balancing, I got a panic, that pushed filesystem read-only

Re: Full filesystem btrfs rebalance kernel panic to read-only lock

2018-11-11 Thread Qu Wenruo
On 2018/11/12 上午9:35, Anand Jain wrote: > > > On 11/09/2018 09:21 AM, Qu Wenruo wrote: >> >> >> On 2018/11/9 上午6:40, Pieter Maes wrote: >>> Hello, >>> >>> So, I've had the full disk issue, so when I tried re-balancing, >>>

Re: Full filesystem btrfs rebalance kernel panic to read-only lock

2018-11-11 Thread Anand Jain
On 11/09/2018 09:21 AM, Qu Wenruo wrote: On 2018/11/9 上午6:40, Pieter Maes wrote: Hello, So, I've had the full disk issue, so when I tried re-balancing, I got a panic, that pushed filesystem read-only and I'm unable to balance or grow the filesystem now. fs info: btrfs fi show / Label

Re: [PATCH V6] Btrfs: enhance raid1/10 balance heuristic

2018-11-11 Thread Timofey Titovets
Gentle ping. вт, 25 сент. 2018 г. в 21:38, Timofey Titovets : > > Currently btrfs raid1/10 balancer bаlance requests to mirrors, > based on pid % num of mirrors. > > Make logic understood: > - if one of underline devices are non rotational > - Queue length to underline devices > > By default try

Re: [PATCH 0/7] fstests: test Btrfs swapfile support

2018-11-11 Thread Eryu Guan
On Tue, Nov 06, 2018 at 02:06:30PM +0100, David Sterba wrote: > On Mon, Nov 05, 2018 at 12:09:31AM +0800, Eryu Guan wrote: > > On Fri, Nov 02, 2018 at 02:29:35PM -0700, Omar Sandoval wrote: > > > From: Omar Sandoval > > > > > > This series fixes a couple of generic swapfile tests and adds some >

Re: [PATCH 2/9] btrfs: replace go back to suspended if target missing

2018-11-11 Thread Anand Jain
On 11/07/2018 08:35 PM, Nikolay Borisov wrote: On 7.11.18 г. 13:43 ч., Anand Jain wrote: At the time of forced unmount we place the running replace to BTRFS_IOCTL_DEV_REPLACE_STATE_SUSPENDED state, so when the system comes back and suppose the target device is missing, then let the replace

Re: [PATCH] btrfs: Check for missing device before bio submission in btrfs_map_bio

2018-11-10 Thread Anand Jain
On 11/08/2018 10:16 PM, Nikolay Borisov wrote: Before btrfs_map_bio submits all stripe bio it does a number of checks to ensure the device for every stripe is present. However, it doesn't do a DEV_STATE_MISSING check, instead this is relegated to the lower level btrfs_schedule_bio (in the

Re: [PATCH v15.1 00/13] Btrfs In-band De-duplication

2018-11-09 Thread Anand Jain
De-duplication must also let use cases to enable de-duplication on per subvolume level, using the subvolume properties. Similar to compression and future-encryption. Thanks, Anand

Re: [PATCH v9 0/6] Btrfs: implement swap file support

2018-11-09 Thread Omar Sandoval
On Wed, Nov 07, 2018 at 04:28:10PM +0100, David Sterba wrote: > On Wed, Nov 07, 2018 at 05:07:00PM +0200, Nikolay Borisov wrote: > > > > > > On 7.11.18 г. 16:49 ч., David Sterba wrote: > > > On Tue, Nov 06, 2018 at 10:54:51AM +0100, David Sterba wrote: > > >> On Thu, Sep 27, 2018 at 11:17:32AM

Re: [PATCH v2] Btrfs: fix missing delayed iputs on unmount

2018-11-09 Thread Omar Sandoval
On Wed, Nov 07, 2018 at 05:01:19PM +0100, David Sterba wrote: > On Wed, Oct 31, 2018 at 10:06:08AM -0700, Omar Sandoval wrote: > > From: Omar Sandoval > > > > There's a race between close_ctree() and cleaner_kthread(). > > close_ctree() sets btrfs_fs_closing(), and the cleaner stops when it > >

Re: [Regression bisected] btrfs: always wait on ordered extents at fsync time

2018-11-09 Thread Mel Gorman
On Fri, Nov 09, 2018 at 09:51:48PM +, Mel Gorman wrote: > Unfortunately, as > I'm about to travel, I didn't attempt a revert and a test comparing 4.18, > 4.19 and 4.20-rc1 is a few hours away so this could potentially be fixed > already but I didn't spot any obvious Fixes commit. > Still

Re: [PATCH v15.1 04/13] btrfs: dedupe: Introduce function to remove hash from in-memory tree

2018-11-09 Thread Josef Bacik
On Tue, Nov 06, 2018 at 02:41:13PM +0800, Lu Fengqi wrote: > From: Wang Xiaoguang > > Introduce static function inmem_del() to remove hash from in-memory > dedupe tree. > And implement btrfs_dedupe_del() and btrfs_dedup_disable() interfaces. > > Also for btrfs_dedupe_disable(), add new

Re: [PATCH v15.1 01/13] btrfs: dedupe: Introduce dedupe framework and its header

2018-11-09 Thread Josef Bacik
On Tue, Nov 06, 2018 at 02:41:10PM +0800, Lu Fengqi wrote: > From: Wang Xiaoguang > > Introduce the header for btrfs in-band(write time) de-duplication > framework and needed header. > > The new de-duplication framework is going to support 2 different dedupe > methods and 1 dedupe hash. > >

Re: [PATCH] btrfs: Check for missing device before bio submission in btrfs_map_bio

2018-11-09 Thread Josef Bacik
On Thu, Nov 08, 2018 at 04:16:38PM +0200, Nikolay Borisov wrote: > Before btrfs_map_bio submits all stripe bio it does a number of checks > to ensure the device for every stripe is present. However, it doesn't > do a DEV_STATE_MISSING check, instead this is relegated to the lower > level

Re: [PATCH] Btrfs: simpler and more efficient cleanup of a log tree's extent io tree

2018-11-09 Thread Josef Bacik
On Fri, Nov 09, 2018 at 10:43:08AM +, fdman...@kernel.org wrote: > From: Filipe Manana > > We currently are in a loop finding each range (corresponding to a btree > node/leaf) in a log root's extent io tree and then clean it up. This is a > waste of time since we are traversing the extent io

Re: unable to mount btrfs after upgrading from 4.16.1 to 4.19.1

2018-11-09 Thread Tomasz Chmielewski
On 2018-11-10 04:20, Tomasz Chmielewski wrote: On 2018-11-10 04:15, Tomasz Chmielewski wrote: On 2018-11-10 03:20, Roman Mamedov wrote: On Sat, 10 Nov 2018 03:08:01 +0900 Tomasz Chmielewski wrote: After upgrading from kernel 4.16.1 to 4.19.1 and a clean restart, the fs no longer mounts:

Re: unable to mount btrfs after upgrading from 4.16.1 to 4.19.1

2018-11-09 Thread Tomasz Chmielewski
On 2018-11-10 04:15, Tomasz Chmielewski wrote: On 2018-11-10 03:20, Roman Mamedov wrote: On Sat, 10 Nov 2018 03:08:01 +0900 Tomasz Chmielewski wrote: After upgrading from kernel 4.16.1 to 4.19.1 and a clean restart, the fs no longer mounts: Did you try rebooting back to 4.16.1 to see if

Re: unable to mount btrfs after upgrading from 4.16.1 to 4.19.1

2018-11-09 Thread Tomasz Chmielewski
On 2018-11-10 03:20, Roman Mamedov wrote: On Sat, 10 Nov 2018 03:08:01 +0900 Tomasz Chmielewski wrote: After upgrading from kernel 4.16.1 to 4.19.1 and a clean restart, the fs no longer mounts: Did you try rebooting back to 4.16.1 to see if it still mounts there? Yes, just did.

Re: unable to mount btrfs after upgrading from 4.16.1 to 4.19.1

2018-11-09 Thread Roman Mamedov
On Sat, 10 Nov 2018 03:08:01 +0900 Tomasz Chmielewski wrote: > After upgrading from kernel 4.16.1 to 4.19.1 and a clean restart, the fs > no longer mounts: Did you try rebooting back to 4.16.1 to see if it still mounts there? -- With respect, Roman

Re: Full filesystem btrfs rebalance kernel panic to read-only lock

2018-11-09 Thread Pieter Maes
tps://github.com/adam900710/btrfs-progs/tree/dirty_fix > > It's a quick and dirty fix to allow "btrfs-corrupt-block -X " to > extent device size to max. > > Please try above command to see if it solves your problem. > > Thanks, > Qu > > This worked! thank!s :D

Re: [PATCHv2 0/9] Removal of optionsl extent_io_ops callbacks

2018-11-09 Thread Nikolay Borisov
On 9.11.18 г. 16:08 ч., Nikolay Borisov wrote: > Here is the 2nd revision of the extent_io_ops optinal callbacks removal. What > prompted this version was that I identified a flaw in the initial version - > while > it did check for the presence of an inode in extent_io_tree::private_data, i

Re: [PATCH] Btrfs: do not set log for full commit when creating non-data block groups

2018-11-09 Thread Qu Wenruo
On 2018/11/9 下午6:21, Filipe Manana wrote: > On Fri, Nov 9, 2018 at 12:27 AM Qu Wenruo wrote: >> >> >> >> On 2018/11/8 下午10:48, Filipe Manana wrote: >>> On Thu, Nov 8, 2018 at 2:37 PM Filipe Manana wrote: On Thu, Nov 8, 2018 at 2:35 PM Qu Wenruo wrote: > > > > On

Re: [PATCH] Btrfs: simpler and more efficient cleanup of a log tree's extent io tree

2018-11-09 Thread Nikolay Borisov
On 9.11.18 г. 12:43 ч., fdman...@kernel.org wrote: > From: Filipe Manana > > We currently are in a loop finding each range (corresponding to a btree > node/leaf) in a log root's extent io tree and then clean it up. This is a > waste of time since we are traversing the extent io tree's rb_tree

Re: [PATCH] Btrfs: do not set log for full commit when creating non-data block groups

2018-11-09 Thread Filipe Manana
On Fri, Nov 9, 2018 at 12:27 AM Qu Wenruo wrote: > > > > On 2018/11/8 下午10:48, Filipe Manana wrote: > > On Thu, Nov 8, 2018 at 2:37 PM Filipe Manana wrote: > >> > >> On Thu, Nov 8, 2018 at 2:35 PM Qu Wenruo wrote: > >>> > >>> > >>> > >>> On 2018/11/8 下午9:17, fdman...@kernel.org wrote: >

Re: [PATCH] Btrfs: incremental send, fix infinite loop when apply children dir moves

2018-11-09 Thread Filipe Manana
On Fri, Nov 9, 2018 at 1:21 AM robbieko wrote: > > robbieko 於 2018-11-06 20:23 寫到: > > Hi, > > > > I can reproduce the infinite loop, the following will describe the > > reason and example. > > > > Example: > > tree --inodes parent/ send/ > > parent/ > > `-- [261] 261 > > `-- [271]

Re: Full filesystem btrfs rebalance kernel panic to read-only lock

2018-11-08 Thread Qu Wenruo
On 2018/11/9 上午6:40, Pieter Maes wrote: > Hello, > > So, I've had the full disk issue, so when I tried re-balancing, > I got a panic, that pushed filesystem read-only and I'm unable to > balance or grow the filesystem now. > > fs info: > btrfs fi show / > Label: non

Re: [PATCH] Btrfs: incremental send, fix infinite loop when apply children dir moves

2018-11-08 Thread robbieko
robbieko 於 2018-11-06 20:23 寫到: Hi, I can reproduce the infinite loop, the following will describe the reason and example. Example: tree --inodes parent/ send/ parent/ `-- [261] 261 `-- [271] 271 `-- [266] 266 |-- [259] 259 |-- [260]

Re: [PATCH] Btrfs: do not set log for full commit when creating non-data block groups

2018-11-08 Thread Qu Wenruo
On 2018/11/8 下午10:48, Filipe Manana wrote: > On Thu, Nov 8, 2018 at 2:37 PM Filipe Manana wrote: >> >> On Thu, Nov 8, 2018 at 2:35 PM Qu Wenruo wrote: >>> >>> >>> >>> On 2018/11/8 下午9:17, fdman...@kernel.org wrote: From: Filipe Manana When creating a block group we don't need

Re: [PATCH v15.1 03/13] btrfs: dedupe: Introduce function to add hash into in-memory tree

2018-11-08 Thread Timofey Titovets
вт, 6 нояб. 2018 г. в 9:41, Lu Fengqi : > > From: Wang Xiaoguang > > Introduce static function inmem_add() to add hash into in-memory tree. > And now we can implement the btrfs_dedupe_add() interface. > > Signed-off-by: Qu Wenruo > Signed-off-by: Wang Xiaoguang > Reviewed-by: Josef Bacik >

Re: Where is my disk space ?

2018-11-08 Thread Chris Murphy
On Thu, Nov 8, 2018 at 2:27 AM, Barbet Alain wrote: > Hi ! > Just to give you end of the story: > I move my /var/lib/docker to my home (other partition), and my space > come back ... I'm not sure why that would matter. Both btrfs du and regular du showed only ~350M used in /var which is about

Re: [PATCH] Btrfs: do not set log for full commit when creating non-data block groups

2018-11-08 Thread Filipe Manana
On Thu, Nov 8, 2018 at 2:37 PM Filipe Manana wrote: > > On Thu, Nov 8, 2018 at 2:35 PM Qu Wenruo wrote: > > > > > > > > On 2018/11/8 下午9:17, fdman...@kernel.org wrote: > > > From: Filipe Manana > > > > > > When creating a block group we don't need to set the log for full commit > > > if the new

Re: [PATCH] Btrfs: do not set log for full commit when creating non-data block groups

2018-11-08 Thread Filipe Manana
On Thu, Nov 8, 2018 at 2:35 PM Qu Wenruo wrote: > > > > On 2018/11/8 下午9:17, fdman...@kernel.org wrote: > > From: Filipe Manana > > > > When creating a block group we don't need to set the log for full commit > > if the new block group is not used for data. Logged items can only point > > to

Re: [PATCH] Btrfs: do not set log for full commit when creating non-data block groups

2018-11-08 Thread Qu Wenruo
On 2018/11/8 下午9:17, fdman...@kernel.org wrote: > From: Filipe Manana > > When creating a block group we don't need to set the log for full commit > if the new block group is not used for data. Logged items can only point > to logical addresses of data block groups (through file extent items)

Re: [PATCH v15.1 02/13] btrfs: dedupe: Introduce function to initialize dedupe info

2018-11-08 Thread Timofey Titovets
if (!limit_nr) > + limit_nr = (u64)-1; > + > + dargs->limit_nr = min(tmp, limit_nr); > + } > + } > + if (backend == BTRFS_DEDUPE_BACKEND_ONDISK) > + dargs->limit_nr = 0; > +

Re: [PATCH -next] btrfs: remove set but not used variable 'tree'

2018-11-08 Thread David Sterba
On Thu, Nov 08, 2018 at 02:14:43AM +, YueHaibing wrote: > Fixes gcc '-Wunused-but-set-variable' warning: > > fs/btrfs/extent_io.c: In function 'end_extent_writepage': > fs/btrfs/extent_io.c:2406:25: warning: > variable 'tree' set but not used [-Wunused-but-set-variable] > > It not used any

Re: [PATCH v15.1 01/13] btrfs: dedupe: Introduce dedupe framework and its header

2018-11-08 Thread Timofey Titovets
вт, 6 нояб. 2018 г. в 9:41, Lu Fengqi : > > From: Wang Xiaoguang > > Introduce the header for btrfs in-band(write time) de-duplication > framework and needed header. > > The new de-duplication framework is going to support 2 different dedupe > methods and 1 dedupe hash. > > Signed-off-by: Qu

Re: Where is my disk space ?

2018-11-08 Thread Barbet Alain
Hi ! Just to give you end of the story: I move my /var/lib/docker to my home (other partition), and my space come back ... I let docker here & don't try to put it again on / to see if problem come back. Le mer. 31 oct. 2018 à 08:34, Barbet Alain a écrit : > > > Also, since you don't have any

Re: [PATCH 6/9] btrfs: replace's scrub must not be running in replace suspended state

2018-11-08 Thread Anand Jain
On 11/08/2018 04:52 PM, Nikolay Borisov wrote: On 8.11.18 г. 10:33 ч., Anand Jain wrote: On 11/07/2018 08:19 PM, Nikolay Borisov wrote: On 7.11.18 г. 13:43 ч., Anand Jain wrote: +    /* scrub for replace must not be running in suspended state */ +    if

Re: [PATCH 6/9] btrfs: replace's scrub must not be running in replace suspended state

2018-11-08 Thread Nikolay Borisov
On 8.11.18 г. 10:33 ч., Anand Jain wrote: > > > On 11/07/2018 08:19 PM, Nikolay Borisov wrote: >> >> >> On 7.11.18 г. 13:43 ч., Anand Jain wrote: >>> +    /* scrub for replace must not be running in suspended state */ >>> +    if (btrfs_scrub_cancel(fs_info) != -ENOTCONN) >>> +

Re: [PATCH 6/9] btrfs: replace's scrub must not be running in replace suspended state

2018-11-08 Thread Anand Jain
On 11/07/2018 08:19 PM, Nikolay Borisov wrote: On 7.11.18 г. 13:43 ч., Anand Jain wrote: + /* scrub for replace must not be running in suspended state */ + if (btrfs_scrub_cancel(fs_info) != -ENOTCONN) + ASSERT(0);

Re: [PATCH 7/9] btrfs: quiten warn if the replace is canceled at finish

2018-11-08 Thread Anand Jain
On 11/07/2018 08:17 PM, Nikolay Borisov wrote: On 7.11.18 г. 13:43 ч., Anand Jain wrote: - WARN_ON(ret); + if (ret != -ECANCELED) + WARN_ON(ret); WARN_ON(ret && ret != -ECANCELED) Will fix. Thanks, Anand

<    1   2   3   4   5   6   7   8   9   10   >