[PATCH v2 1/2] btrfs-progs: fi-usage: Fix wrong RAID10 used and unallocated space

2018-01-09 Thread Qu Wenruo
[BUG] For a very basic RAID10 with 4 disks, "fi usage" and "fi show" are outputting conflicting result: -- # btrfs fi show /mnt/btrfs/ Label: none uuid: 6d0229db-28d1-4696-ac20-e828cc45dc40 Total devices 4 FS bytes used 1.12MiB devid1 size 5.00GiB used 2.01GiB path

[PATCH v2 2/2] btrfs-progs: misc-test: Add test case to check if fi usage and show report consistent unallocated space

2018-01-09 Thread Qu Wenruo
Signed-off-by: Qu Wenruo --- v2: Don't create unnecessary temporary dir. Use variable directly to save output Use run_check_mount_test_dev() macro with TEST_DEV set manually. --- tests/misc-tests/028-fi-usage-cross-check/test.sh | 44 +++ 1 file changed,

[PATCH] btrfs-progs: Fix wrong chunk allocation type in btrfs_reserve_extent

2018-01-09 Thread Qu Wenruo
btrfs_reserve_extent() will try to allocate new chunk if there is not enough space (mostly meta space). However when it tries to allocate new meta chunk, it will always try to allocate SINGLE meta chunk, and if the fs is using other profile, it will cause dead allocation which can't be really

[PATCH 1/4] btrfs: move pr_info into device_list_add

2018-01-09 Thread Anand Jain
Commit 60999ca4b403 ("btrfs: make device scan less noisy") adds return value 1 to device_list_add(), so that parent function can call pr_info only when new device is added. Move the pr_info() part into device_list_add() so that this function can be kept simple. Signed-off-by: Anand Jain

[PATCH 4/4] btrfs: drop devid as device_list_add() arg

2018-01-09 Thread Anand Jain
As struct btrfs_disk_super is being passed, so it can get devid the same way its parent does. Signed-off-by: Anand Jain Reviewed-by: Josef Bacik --- fs/btrfs/volumes.c | 8 +++- 1 file changed, 3 insertions(+), 5 deletions(-) diff --git

[PATCH 3/4] btrfs: get device pointer from device_list_add()

2018-01-09 Thread Anand Jain
Instead of pointer to btrfs_fs_devices as an arg in device_list_add() better to get pointer to btrfs_device as return value, then we have both, pointer to btrfs_device and btrfs_fs_devices. btrfs_device is needed to handle reappearing missing device. Signed-off-by: Anand Jain

[PATCH v4 0/6] preparatory work to add device forget

2018-01-09 Thread Anand Jain
v3->v4: Mainly fix as per comments from Josef. @3/6: rename btrfs_free_stale_device() to btrfs_free_stale_devices() @4/6: reorg logic, init not_found = 0; drop else part @5/6: added new in v4. Renames arg cur_dev to skip_dev @6/6: v3:5/6 is merged to v4:6/6 checkpath error fixes. v2->v3: @

[PATCH 6/6] btrfs: make btrfs_free_stale_device() to match the path

2018-01-09 Thread Anand Jain
From: Anand Jain The btrfs_free_stale_device() is updated to match for the given device path and delete it. (It searches for only unmounted list of devices.) Also drop the comment about different path being used for the same device, since now we will have cli to clean any

[PATCH 2/4] btrfs: set the total_devices in device_list_add()

2018-01-09 Thread Anand Jain
There is no other parent for device_list_add() except for btrfs_scan_one_device(), which would set btrfs_fs_devices::total_devices if device_list_add is successful and this can be done with in device_list_add() itself. Signed-off-by: Anand Jain Reviewed-by: Josef Bacik

[PATCH 5/6] btrfs: rename btrfs_free_stale_devices() arg to skip_dev

2018-01-09 Thread Anand Jain
No functional changes. Rename btrfs_free_stale_devices() arg to skip_dev, so that it reflects what that arg for. Signed-off-by: Anand Jain --- fs/btrfs/volumes.c | 8 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/fs/btrfs/volumes.c

[PATCH v3 0/4] device_list_add() peparation to add reappearing missing device

2018-01-09 Thread Anand Jain
(Apply on top of my patchset [PATCH v4 0/6] preparatory work to add device forget for conflict free apply. They don't actually depend on each other though). v2->v3: Fix device_list_add() fn description which was still referring to the previous return values. v1->v2: Drop patch 5/5 for

[PATCH 4/6] btrfs: make btrfs_free_stale_device() argument optional

2018-01-09 Thread Anand Jain
From: Anand Jain This updates btrfs_free_stale_device() helper function to delete all unmouted devices, when arg is NULL. Signed-off-by: Anand Jain --- fs/btrfs/volumes.c | 14 +- 1 file changed, 5 insertions(+), 9 deletions(-) diff

[PATCH 2/6] btrfs: no need to check for btrfs_fs_devices::seeding

2018-01-09 Thread Anand Jain
There is no need to check for btrfs_fs_devices::seeding when we have checked for btrfs_fs_devices::opened, because we can't sprout without its seed FS being opened. Signed-off-by: Anand Jain Reviewed-by: Josef Bacik --- fs/btrfs/volumes.c | 2 -- 1 file

[PATCH 3/6] btrfs: make btrfs_free_stale_device() to iterate all stales

2018-01-09 Thread Anand Jain
From: Anand Jain Let the list iterator iterate further and find other stale devices and delete it. This is in preparation to add support for user land request-able stale devices cleanup. Also rename btrfs_free_stale_device() to btrfs_free_stale_devices(). Signed-off-by:

[PATCH 1/6] btrfs: cleanup btrfs_free_stale_device() usage

2018-01-09 Thread Anand Jain
We call btrfs_free_stale_device() only when we alloc a new struct btrfs_device (ret=1), so move it closer to where we alloc the new device. Also drop the comments. Signed-off-by: Anand Jain Reviewed-by: Josef Bacik --- fs/btrfs/volumes.c | 8 +--- 1

Re: [PATCH] generic/015: Change the test filesystem size to 101mb

2018-01-09 Thread Eryu Guan
On Mon, Jan 08, 2018 at 10:43:30AM +0200, Nikolay Borisov wrote: > This test has been failing for btrfs for quite some time, > at least since 4.7. There are 2 implementation details of btrfs that > it exposes: > > 1. Currently btrfs filesystem under 100mb are created in Mixed block > group mode.

Re: Recommendations for balancing as part of regular maintenance?

2018-01-09 Thread Duncan
Austin S. Hemmelgarn posted on Tue, 09 Jan 2018 07:46:48 -0500 as excerpted: >> On 08/01/18 23:29, Martin Raiber wrote: >>> There have been reports of (rare) corruption caused by balance (won't >>> be detected by a scrub) here on the mailing list. So I would stay a >>> away from btrfs balance

Re: btrfs scrub not repair corruption

2018-01-09 Thread Duncan
Wolf posted on Mon, 08 Jan 2018 23:27:27 +0100 as excerpted: > I'm running btrfs scrub on my raid each week (is that too often?) and > I'm having a problem that it reports corruption, says it's repaired but > next week reports it again. I won't attempt to answer the larger question, but on the

[PATCH v2] Btrfs: raid56: fix race between merge_bio and rbio_orig_end_io

2018-01-09 Thread Liu Bo
Before rbio_orig_end_io() goes to free rbio, rbio may get merged with more bios from other rbios and rbio->bio_list becomes non-empty, in that case, these newly merged bios don't end properly. Once unlock_stripe() is done, rbio->bio_list will not be updated any more and we can call bio_endio() on

Re: [RFC PATCH] btrfs: Fix slight df misreporting when listing mixed block groups fs

2018-01-09 Thread Qu Wenruo
On 2018年01月10日 09:30, Qu Wenruo wrote: > > > On 2018年01月10日 00:04, Nikolay Borisov wrote: >> Currently when df (resp. statfs) is executed on a btrfs instance the >> free space is calculated as the freespace of the block groups + >> any unallocated space on the devices, constituting this

[PATCH 4/4] btrfs: drop devid as device_list_add() arg

2018-01-09 Thread Anand Jain
As struct btrfs_disk_super is being passed, so it can get devid the same way its parent does. Signed-off-by: Anand Jain Reviewed-by: Josef Bacik --- fs/btrfs/volumes.c | 8 +++- 1 file changed, 3 insertions(+), 5 deletions(-) diff --git

[PATCH V2 0/4] device_list_add() peparation to add reappearing missing device

2018-01-09 Thread Anand Jain
v1->v2: Drop patch 5/5 for uuid_mutex optimize. That was wrong. Thanks Josef. In patch 3/5 make btrfs_device * as return. Cleanup of device_list_add(), mainly in preparation to handle reappearing missing device which its next reroll will be sent separately. Anand Jain (4): btrfs: move

[PATCH 2/4] btrfs: set the total_devices in device_list_add()

2018-01-09 Thread Anand Jain
There is no other parent for device_list_add() except for btrfs_scan_one_device(), which would set btrfs_fs_devices::total_devices if device_list_add is successful and this can be done with in device_list_add() itself. Signed-off-by: Anand Jain Reviewed-by: Josef Bacik

[PATCH 1/4] btrfs: move pr_info into device_list_add

2018-01-09 Thread Anand Jain
Commit 60999ca4b403 ("btrfs: make device scan less noisy") adds return value 1 to device_list_add(), so that parent function can call pr_info only when new device is added. Move the pr_info() part into device_list_add() so that this function can be kept simple. Signed-off-by: Anand Jain

[PATCH 3/4] btrfs: get device pointer from device_list_add()

2018-01-09 Thread Anand Jain
Instead of pointer to btrfs_fs_devices as an arg in device_list_add() better to get pointer to btrfs_device as return value, then we have both, pointer to btrfs_device and btrfs_fs_devices. btrfs_device is needed to handle reappearing missing device. Signed-off-by: Anand Jain

Re: [PATCH 5/5] btrfs: move uuid_mutex closer to exclusivity

2018-01-09 Thread Anand Jain
On 01/10/2018 12:15 AM, Josef Bacik wrote: On Tue, Jan 09, 2018 at 10:46:25PM +0800, Anand Jain wrote: move uuid_mutex with in device_list_add(). Signed-off-by: Anand Jain This isn't going to work, there's a bunch of places we return errors so this just deadlocks

Re: [RFC PATCH] btrfs: Fix slight df misreporting when listing mixed block groups fs

2018-01-09 Thread Qu Wenruo
On 2018年01月10日 00:04, Nikolay Borisov wrote: > Currently when df (resp. statfs) is executed on a btrfs instance the > free space is calculated as the freespace of the block groups + > any unallocated space on the devices, constituting this filesystem. > There is a catch, however, in that the

Re: [PATCH 3/5] btrfs: get device pointer from device_list_add()

2018-01-09 Thread Anand Jain
On 01/10/2018 12:13 AM, Josef Bacik wrote: On Tue, Jan 09, 2018 at 10:46:23PM +0800, Anand Jain wrote: Instead of pointer to btrfs_fs_devices from device_list_add() its better to get pointer to btrfs_device, then we have both, pointer to btrfs_device and btrfs_fs_devices. This is needed in

Re: [PATCH v5 19/19] fs: handle inode->i_version more efficiently

2018-01-09 Thread Dave Chinner
On Tue, Jan 09, 2018 at 09:10:59AM -0500, Jeff Layton wrote: > From: Jeff Layton > > Since i_version is mostly treated as an opaque value, we can exploit that > fact to avoid incrementing it when no one is watching. With that change, > we can avoid incrementing the counter on

Re: [PATCH v5 17/19] xfs: avoid setting XFS_ILOG_CORE if i_version doesn't need incrementing

2018-01-09 Thread Dave Chinner
On Tue, Jan 09, 2018 at 09:10:57AM -0500, Jeff Layton wrote: > From: Jeff Layton > > If XFS_ILOG_CORE is already set then go ahead and increment it. > > Signed-off-by: Jeff Layton > Acked-by: Darrick J. Wong Acked-by: Dave

Re: [PATCH v5 14/19] xfs: convert to new i_version API

2018-01-09 Thread Dave Chinner
On Tue, Jan 09, 2018 at 09:10:54AM -0500, Jeff Layton wrote: > From: Jeff Layton > > Signed-off-by: Jeff Layton > Acked-by: Darrick J. Wong Looks ok, but I haven't tested it at all. Acked-by: Dave Chinner

Re: [PATCH] btrfs: use correct string length

2018-01-09 Thread David Sterba
On Mon, Jan 08, 2018 at 07:51:22PM +0800, Xiongfeng Wang wrote: > From: Xiongfeng Wang > > gcc-8 reports > > fs/btrfs/ioctl.c: In function 'btrfs_ioctl': > ./include/linux/string.h:245:9: warning: '__builtin_strncpy' specified > bound 1024 equals destination size

Re: [PATCH v2 10/10] Btrfs: noinline merge_extent_mapping

2018-01-09 Thread Josef Bacik
On Fri, Jan 05, 2018 at 12:51:17PM -0700, Liu Bo wrote: > In order to debug subtle bugs around merge_extent_mapping(), perf probe > can be used to check the arguments, but sometimes merge_extent_mapping() > got inlined by compiler and couldn't be probed. > > This is adding noinline attribute to

Re: [PATCH v2 09/10] Btrfs: add tracepoint for em's EEXIST case

2018-01-09 Thread Josef Bacik
On Fri, Jan 05, 2018 at 12:51:16PM -0700, Liu Bo wrote: > This is adding a tracepoint 'btrfs_handle_em_exist' to help debug the > subtle bugs around merge_extent_mapping. > > Signed-off-by: Liu Bo Reviewed-by: Josef Bacik Thanks, Josef -- To unsubscribe

Re: [PATCH v2 08/10] Btrfs: add WARN_ONCE to detect unexpected error from merge_extent_mapping

2018-01-09 Thread Josef Bacik
On Fri, Jan 05, 2018 at 12:51:15PM -0700, Liu Bo wrote: > This is a subtle case, so in order to understand the problem, it'd be good > to know the content of existing and em when any error occurs. > > Signed-off-by: Liu Bo Reviewed-by: Josef Bacik Thanks,

Re: [PATCH v2 07/10] Btrfs: extent map selftest: dio write vs dio read

2018-01-09 Thread Josef Bacik
On Fri, Jan 05, 2018 at 12:51:14PM -0700, Liu Bo wrote: > This test case simulates the racy situation of dio write vs dio read, > and see if btrfs_get_extent() would return -EEXIST. > > Signed-off-by: Liu Bo Reviewed-by: Josef Bacik Thanks, Josef -- To

Re: [PATCH v2 06/10] Btrfs: extent map selftest: buffered write vs dio read

2018-01-09 Thread Josef Bacik
On Fri, Jan 05, 2018 at 12:51:13PM -0700, Liu Bo wrote: > This test case simulates the racy situation of buffered write vs dio > read, and see if btrfs_get_extent() would return -EEXIST. > > Signed-off-by: Liu Bo Reviewed-by: Josef Bacik Thanks, Josef --

Re: [PATCH v2 05/10] Btrfs: add extent map selftests

2018-01-09 Thread Josef Bacik
On Fri, Jan 05, 2018 at 12:51:12PM -0700, Liu Bo wrote: > We've observed that btrfs_get_extent() and merge_extent_mapping() could > return -EEXIST in several cases, and they are caused by some racy > condition, e.g dio read vs dio write, which makes the problem very tricky > to reproduce. > >

Re: [PATCH v2 04/10] Btrfs: move extent map specific code to extent_map.c

2018-01-09 Thread Josef Bacik
On Fri, Jan 05, 2018 at 12:51:11PM -0700, Liu Bo wrote: > These helpers are extent map specific, move them to extent_map.c. > > Signed-off-by: Liu Bo Reviewed-by: Josef Bacik Thanks, Josef -- To unsubscribe from this list: send the line "unsubscribe

Re: [PATCH v2 02/10] Btrfs: fix unexpected EEXIST from btrfs_get_extent

2018-01-09 Thread Josef Bacik
On Fri, Jan 05, 2018 at 12:51:09PM -0700, Liu Bo wrote: > This fixes a corner case that is caused by a race of dio write vs dio > read/write. > > Here is how the race could happen. > > Suppose that no extent map has been loaded into memory yet. > There is a file extent [0, 32K), two jobs are

Re: [PATCH v2 03/10] Btrfs: add helper for em merge logic

2018-01-09 Thread Josef Bacik
On Fri, Jan 05, 2018 at 12:51:10PM -0700, Liu Bo wrote: > This is a prepare work for the following extent map selftest, which > runs tests against em merge logic. > > Signed-off-by: Liu Bo Reviewed-by: Josef Bacik Thanks, Josef -- To unsubscribe from this

Re: [PATCH v2 01/10] Btrfs: fix incorrect block_len in merge_extent_mapping

2018-01-09 Thread Josef Bacik
On Fri, Jan 05, 2018 at 12:51:08PM -0700, Liu Bo wrote: > %block_len could be checked on deciding if two em are mergeable. > > merge_extent_mapping() has only added the front pad if the front part > of em gets truncated, but it's possible that the end part gets > truncated. > > For both

Re: [PATCH 2/3] btrfs: add support for SUPER_FLAG_CHANGING_FSID in btrfs.ko

2018-01-09 Thread David Sterba
On Tue, Jan 09, 2018 at 09:05:42AM +0800, Anand Jain wrote: > Userland sets SUPER_FLAG_CHANGING_FSID and resets it only when changing > fsid is complete. Its not a good idea to mount the device anything in > between. > This patch doesn't add SUPER_FLAG_CHANGING_FSID into BTRFS_SUPER_FLAG_SUPP >

Re: [PATCH 3/3] btrfs: fail mount when sb flag is not in BTRFS_SUPER_FLAG_SUPP

2018-01-09 Thread David Sterba
On Tue, Jan 09, 2018 at 09:05:43AM +0800, Anand Jain wrote: > It appear from the original commit [1] that there isn't any design > specific reason not to fail the mount instead of just warning. This > patch will change it to fail. > > [1] > commit 319e4d0661e5323c9f9945f0f8fb5905e5fe74c3 >

Re: [PATCH 1/3] btrfs: define SUPER_FLAG_METADUMP_V2

2018-01-09 Thread David Sterba
On Tue, Jan 09, 2018 at 09:05:41AM +0800, Anand Jain wrote: > btrfs-progs uses super flag bit BTRFS_SUPER_FLAG_METADUMP_V2 (1ULL << 34). > So just define that in kernel so that we know its been used. > > Signed-off-by: Anand Jain Reviewed-by: David Sterba

Re: [PATCHSET v5] blk-mq: reimplement timeout handling

2018-01-09 Thread Jens Axboe
On 1/9/18 9:29 AM, Tejun Heo wrote: > Hello, > > Changes from [v4] > > - Comments added. Patch description updated. > > Changes from [v3] > > - Rebased on top of for-4.16/block. > > - Integrated Jens's hctx_[un]lock() factoring patch and refreshed the > patches accordingly. > > - Added

[PATCHSET v5] blk-mq: reimplement timeout handling

2018-01-09 Thread Tejun Heo
Hello, Changes from [v4] - Comments added. Patch description updated. Changes from [v3] - Rebased on top of for-4.16/block. - Integrated Jens's hctx_[un]lock() factoring patch and refreshed the patches accordingly. - Added comment explaining the use of hctx_lock() instead of

[PATCH 1/8] blk-mq: move hctx lock/unlock into a helper

2018-01-09 Thread Tejun Heo
From: Jens Axboe Move the RCU vs SRCU logic into lock/unlock helpers, which makes the actual functional bits within the locked region much easier to read. tj: Reordered in front of timeout revamp patches and added the missing blk_mq_run_hw_queue() conversion.

[PATCH 2/8] blk-mq: protect completion path with RCU

2018-01-09 Thread Tejun Heo
Currently, blk-mq protects only the issue path with RCU. This patch puts the completion path under the same RCU protection. This will be used to synchronize issue/completion against timeout by later patches, which will also add the comments. Signed-off-by: Tejun Heo ---

[PATCH 3/8] blk-mq: replace timeout synchronization with a RCU and generation based scheme

2018-01-09 Thread Tejun Heo
Currently, blk-mq timeout path synchronizes against the usual issue/completion path using a complex scheme involving atomic bitflags, REQ_ATOM_*, memory barriers and subtle memory coherence rules. Unfortunately, it contains quite a few holes. There's a complex dancing around REQ_ATOM_STARTED and

[PATCH 5/8] blk-mq: make blk_abort_request() trigger timeout path

2018-01-09 Thread Tejun Heo
With issue/complete and timeout paths now using the generation number and state based synchronization, blk_abort_request() is the only one which depends on REQ_ATOM_COMPLETE for arbitrating completion. There's no reason for blk_abort_request() to be a completely separate path. This patch makes

[PATCH 4/8] blk-mq: use blk_mq_rq_state() instead of testing REQ_ATOM_COMPLETE

2018-01-09 Thread Tejun Heo
blk_mq_check_inflight() and blk_mq_poll_hybrid_sleep() test REQ_ATOM_COMPLETE to determine the request state. Both uses are speculative and we can test REQ_ATOM_STARTED and blk_mq_rq_state() for equivalent results. Replace the tests. This will allow removing REQ_ATOM_COMPLETE usages from

[PATCH 6/8] blk-mq: remove REQ_ATOM_COMPLETE usages from blk-mq

2018-01-09 Thread Tejun Heo
After the recent updates to use generation number and state based synchronization, blk-mq no longer depends on REQ_ATOM_COMPLETE except to avoid firing the same timeout multiple times. Remove all REQ_ATOM_COMPLETE usages and use a new rq_flags flag RQF_MQ_TIMEOUT_EXPIRED to avoid firing the same

[PATCH 7/8] blk-mq: remove REQ_ATOM_STARTED

2018-01-09 Thread Tejun Heo
After the recent updates to use generation number and state based synchronization, we can easily replace REQ_ATOM_STARTED usages by adding an extra state to distinguish completed but not yet freed state. Add MQ_RQ_COMPLETE and replace REQ_ATOM_STARTED usages with blk_mq_rq_state() tests.

Re: [PATCH 4/6] btrfs: make btrfs_free_stale_device() argument optional

2018-01-09 Thread Josef Bacik
On Tue, Jan 09, 2018 at 10:13:12PM +0800, Anand Jain wrote: > From: Anand Jain > > This updates btrfs_free_stale_device() helper function to delete all > unmouted devices, when arg is NULL. > > Signed-off-by: Anand Jain > --- > fs/btrfs/volumes.c

Re: [PATCH 2/8] blk-mq: protect completion path with RCU

2018-01-09 Thread Jens Axboe
On 1/9/18 9:19 AM, t...@kernel.org wrote: > Hello, Bart. > > On Tue, Jan 09, 2018 at 04:12:40PM +, Bart Van Assche wrote: >> I'm concerned about the additional CPU cycles needed for the new >> blk_mq_map_queue() >> call, although I know this call is cheap. Would the timeout code really get

Re: [PATCH 3/6] btrfs: make btrfs_free_stale_device() to iterate all stales

2018-01-09 Thread Josef Bacik
On Tue, Jan 09, 2018 at 10:13:11PM +0800, Anand Jain wrote: > From: Anand Jain > > Let the list iterator iterate further and find other stale > devices and delete it. This is in preparation to add support > for user land request-able stale devices cleanup. > >

Re: [PATCH 2/8] blk-mq: protect completion path with RCU

2018-01-09 Thread t...@kernel.org
Hello, Bart. On Tue, Jan 09, 2018 at 04:12:40PM +, Bart Van Assche wrote: > I'm concerned about the additional CPU cycles needed for the new > blk_mq_map_queue() > call, although I know this call is cheap. Would the timeout code really get > that So, if that is really a concern, let's

Re: [PATCH 2/6] btrfs: no need to check for btrfs_fs_devices::seeding

2018-01-09 Thread Josef Bacik
On Tue, Jan 09, 2018 at 10:13:10PM +0800, Anand Jain wrote: > There is no need to check for btrfs_fs_devices::seeding when we > have checked for btrfs_fs_devices::opened, because we can't sprout > without its seed FS being opened. > > Signed-off-by: Anand Jain

Re: [PATCH 1/6] btrfs: cleanup btrfs_free_stale_device() usage

2018-01-09 Thread Josef Bacik
On Tue, Jan 09, 2018 at 10:13:09PM +0800, Anand Jain wrote: > We call btrfs_free_stale_device() only when we alloc a new > struct btrfs_device (ret=1), so move it closer to where we > alloc the new device. Also drop the comments. > > Signed-off-by: Anand Jain Reviewed-by:

Re: [PATCH 5/5] btrfs: move uuid_mutex closer to exclusivity

2018-01-09 Thread Josef Bacik
On Tue, Jan 09, 2018 at 10:46:25PM +0800, Anand Jain wrote: > move uuid_mutex with in device_list_add(). > > Signed-off-by: Anand Jain This isn't going to work, there's a bunch of places we return errors so this just deadlocks the box. Leave it like it is, it's not

Re: [PATCH 3/5] btrfs: get device pointer from device_list_add()

2018-01-09 Thread Josef Bacik
On Tue, Jan 09, 2018 at 10:46:23PM +0800, Anand Jain wrote: > Instead of pointer to btrfs_fs_devices from device_list_add() its > better to get pointer to btrfs_device, then we have both, pointer > to btrfs_device and btrfs_fs_devices. This is needed in preparation > to add handling reappearing

Re: [PATCH 2/8] blk-mq: protect completion path with RCU

2018-01-09 Thread Bart Van Assche
On Mon, 2018-01-08 at 11:15 -0800, Tejun Heo wrote: > Currently, blk-mq protects only the issue path with RCU. This patch > puts the completion path under the same RCU protection. This will be > used to synchronize issue/completion against timeout by later patches, > which will also add the

Re: [PATCH 1/5] btrfs: move pr_info into device_list_add

2018-01-09 Thread Josef Bacik
On Tue, Jan 09, 2018 at 10:46:21PM +0800, Anand Jain wrote: > Commit 60999ca4b403 ("btrfs: make device scan less noisy") > adds return value 1 to device_list_add(), so that parent function can > call pr_info only when new device is added. Move the pr_info() part > into device_list_add() so that

Re: [RFC PATCH] btrfs: Fix slight df misreporting when listing mixed block groups fs

2018-01-09 Thread Josef Bacik
On Tue, Jan 09, 2018 at 06:04:05PM +0200, Nikolay Borisov wrote: > Currently when df (resp. statfs) is executed on a btrfs instance the > free space is calculated as the freespace of the block groups + > any unallocated space on the devices, constituting this filesystem. > There is a catch,

Re: [PATCH 5/8] blk-mq: make blk_abort_request() trigger timeout path

2018-01-09 Thread t...@kernel.org
On Mon, Jan 08, 2018 at 10:10:01PM +, Bart Van Assche wrote: > Other req->deadline writes are protected by preempt_disable(), > write_seqcount_begin(>gstate_seq), write_seqcount_end(>gstate_seq) > and preempt_enable(). I think it's fine that the above req->deadline store > does not have that

[RFC PATCH] btrfs: Fix slight df misreporting when listing mixed block groups fs

2018-01-09 Thread Nikolay Borisov
Currently when df (resp. statfs) is executed on a btrfs instance the free space is calculated as the freespace of the block groups + any unallocated space on the devices, constituting this filesystem. There is a catch, however, in that the unallocated space from the devices has 1 mb subtracted

Re: [PATCH 3/8] blk-mq: replace timeout synchronization with a RCU and generation based scheme

2018-01-09 Thread t...@kernel.org
On Mon, Jan 08, 2018 at 11:29:11PM +, Bart Van Assche wrote: > Does "gstate" perhaps stand for "generation number and state"? If so, please > mention this in one of the above comments. Yeah, will do. Thanks. -- tejun -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs"

Re: [PATCH 3/8] blk-mq: replace timeout synchronization with a RCU and generation based scheme

2018-01-09 Thread t...@kernel.org
On Mon, Jan 08, 2018 at 09:06:55PM +, Bart Van Assche wrote: > On Mon, 2018-01-08 at 11:15 -0800, Tejun Heo wrote: > > +static void blk_mq_rq_update_aborted_gstate(struct request *rq, u64 gstate) > > +{ > > + unsigned long flags; > > + > > + local_irq_save(flags); > > +

Re: [PATCH 2/8] blk-mq: protect completion path with RCU

2018-01-09 Thread Jens Axboe
On 1/9/18 12:08 AM, Hannes Reinecke wrote: > On 01/08/2018 08:15 PM, Tejun Heo wrote: >> Currently, blk-mq protects only the issue path with RCU. This patch >> puts the completion path under the same RCU protection. This will be >> used to synchronize issue/completion against timeout by later

Re: [PATCH v5 02/19] fs: don't take the i_lock in inode_inc_iversion

2018-01-09 Thread Jan Kara
On Tue 09-01-18 09:10:42, Jeff Layton wrote: > From: Jeff Layton > > The rationale for taking the i_lock when incrementing this value is > lost in antiquity. The readers of the field don't take it (at least > not universally), so my assumption is that it was only done here to

[PATCH 5/5] btrfs: move uuid_mutex closer to exclusivity

2018-01-09 Thread Anand Jain
move uuid_mutex with in device_list_add(). Signed-off-by: Anand Jain --- fs/btrfs/volumes.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c index f942e8193862..283417bf3b00 100644 --- a/fs/btrfs/volumes.c

Re: [PATCH] generic/015: Change the test filesystem size to 101mb

2018-01-09 Thread Nikolay Borisov
On 8.01.2018 10:43, Nikolay Borisov wrote: > This test has been failing for btrfs for quite some time, > at least since 4.7. There are 2 implementation details of btrfs that > it exposes: > > 1. Currently btrfs filesystem under 100mb are created in Mixed block > group mode. Freespace

[PATCH 0/5] device_list_add() peparation to add reappearing missing device

2018-01-09 Thread Anand Jain
Cleanup of device_list_add(), mainly in preparation to handle reappearing missing device which its next reroll will be sent separately. Anand Jain (5): btrfs: move pr_info into device_list_add btrfs: set the total_devices in device_list_add() btrfs: get device pointer from device_list_add()

[PATCH 2/5] btrfs: set the total_devices in device_list_add()

2018-01-09 Thread Anand Jain
There is no other parent for device_list_add() except for btrfs_scan_one_device(), which would set btrfs_fs_devices::total_devices if device_list_add is successful and this can be done with in device_list_add() itself. Signed-off-by: Anand Jain --- fs/btrfs/volumes.c | 6

[PATCH 3/5] btrfs: get device pointer from device_list_add()

2018-01-09 Thread Anand Jain
Instead of pointer to btrfs_fs_devices from device_list_add() its better to get pointer to btrfs_device, then we have both, pointer to btrfs_device and btrfs_fs_devices. This is needed in preparation to add handling reappearing missing device feature. Signed-off-by: Anand Jain

[PATCH 1/5] btrfs: move pr_info into device_list_add

2018-01-09 Thread Anand Jain
Commit 60999ca4b403 ("btrfs: make device scan less noisy") adds return value 1 to device_list_add(), so that parent function can call pr_info only when new device is added. Move the pr_info() part into device_list_add() so that this function can be kept simple. Signed-off-by: Anand Jain

[PATCH 4/5] btrfs: drop devid as device_list_add() arg

2018-01-09 Thread Anand Jain
As struct btrfs_disk_super is being passed, so it can get devid the same way its parent does. Signed-off-by: Anand Jain --- fs/btrfs/volumes.c | 8 +++- 1 file changed, 3 insertions(+), 5 deletions(-) diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c index

[PATCH 3/6] btrfs: make btrfs_free_stale_device() to iterate all stales

2018-01-09 Thread Anand Jain
From: Anand Jain Let the list iterator iterate further and find other stale devices and delete it. This is in preparation to add support for user land request-able stale devices cleanup. Signed-off-by: Anand Jain --- fs/btrfs/volumes.c | 20

[PATCH v5 00/19] fs: rework and optimize i_version handling in filesystems

2018-01-09 Thread Jeff Layton
From: Jeff Layton v5: - don't corrupt refcounts stashed in i_version of ext4 xattr inodes - add raw variants of inc and cmp functions, and have nfs use them v4: - fix SB_LAZYTIME handling in generic_update_time - add memory barriers to patch to convert i_version field to

[PATCH v5 01/19] fs: new API for handling inode->i_version

2018-01-09 Thread Jeff Layton
From: Jeff Layton Add a documentation blob that explains what the i_version field is, how it is expected to work, and how it is currently implemented by various filesystems. We already have inode_inc_iversion. Add several other functions for manipulating and accessing the

[PATCH v5 02/19] fs: don't take the i_lock in inode_inc_iversion

2018-01-09 Thread Jeff Layton
From: Jeff Layton The rationale for taking the i_lock when incrementing this value is lost in antiquity. The readers of the field don't take it (at least not universally), so my assumption is that it was only done here to serialize incrementors. If that is indeed the case,

[PATCH v5 03/19] fat: convert to new i_version API

2018-01-09 Thread Jeff Layton
From: Jeff Layton Signed-off-by: Jeff Layton --- fs/fat/dir.c | 3 ++- fs/fat/inode.c | 9 + fs/fat/namei_msdos.c | 7 --- fs/fat/namei_vfat.c | 22 +++--- 4 files changed, 22 insertions(+), 19 deletions(-)

[PATCH v5 04/19] affs: convert to new i_version API

2018-01-09 Thread Jeff Layton
From: Jeff Layton Signed-off-by: Jeff Layton --- fs/affs/amigaffs.c | 5 +++-- fs/affs/dir.c | 5 +++-- fs/affs/super.c| 3 ++- 3 files changed, 8 insertions(+), 5 deletions(-) diff --git a/fs/affs/amigaffs.c b/fs/affs/amigaffs.c index

[PATCH v5 05/19] afs: convert to new i_version API

2018-01-09 Thread Jeff Layton
From: Jeff Layton For AFS, it's generally treated as an opaque value, so we use the *_raw variants of the API here. Note that AFS has quite a different definition for this counter. AFS only increments it on changes to the data to the data in regular files and contents of the

[PATCH v5 06/19] btrfs: convert to new i_version API

2018-01-09 Thread Jeff Layton
From: Jeff Layton Signed-off-by: Jeff Layton Acked-by: David Sterba --- fs/btrfs/delayed-inode.c | 7 +-- fs/btrfs/inode.c | 6 -- fs/btrfs/tree-log.c | 4 +++- 3 files changed, 12 insertions(+), 5 deletions(-)

[PATCH v5 08/19] ext2: convert to new i_version API

2018-01-09 Thread Jeff Layton
From: Jeff Layton Signed-off-by: Jeff Layton Reviewed-by: Jan Kara --- fs/ext2/dir.c | 9 + fs/ext2/super.c | 5 +++-- 2 files changed, 8 insertions(+), 6 deletions(-) diff --git a/fs/ext2/dir.c b/fs/ext2/dir.c index

[PATCH v5 07/19] exofs: switch to new i_version API

2018-01-09 Thread Jeff Layton
From: Jeff Layton Signed-off-by: Jeff Layton --- fs/exofs/dir.c | 9 + fs/exofs/super.c | 3 ++- 2 files changed, 7 insertions(+), 5 deletions(-) diff --git a/fs/exofs/dir.c b/fs/exofs/dir.c index 98233a97b7b8..c5a53fcc43ea 100644 ---

[PATCH 5/6] btrfs: make btrfs_free_stale_device() to match the path

2018-01-09 Thread Anand Jain
From: Anand Jain The btrfs_free_stale_device() is updated to match for the given device path and delete it. (It searchs for only unmounted list of devices.) Signed-off-by: Anand Jain --- fs/btrfs/volumes.c | 6 -- 1 file changed, 4

[PATCH 4/6] btrfs: make btrfs_free_stale_device() argument optional

2018-01-09 Thread Anand Jain
From: Anand Jain This updates btrfs_free_stale_device() helper function to delete all unmouted devices, when arg is NULL. Signed-off-by: Anand Jain --- fs/btrfs/volumes.c | 14 ++ 1 file changed, 6 insertions(+), 8 deletions(-) diff

[PATCH 1/6] btrfs: cleanup btrfs_free_stale_device() usage

2018-01-09 Thread Anand Jain
We call btrfs_free_stale_device() only when we alloc a new struct btrfs_device (ret=1), so move it closer to where we alloc the new device. Also drop the comments. Signed-off-by: Anand Jain --- fs/btrfs/volumes.c | 8 +--- 1 file changed, 1 insertion(+), 7

[PATCH v5 09/19] ext4: convert to new i_version API

2018-01-09 Thread Jeff Layton
From: Jeff Layton Signed-off-by: Jeff Layton Acked-by: Theodore Ts'o --- fs/ext4/dir.c| 9 + fs/ext4/inline.c | 7 --- fs/ext4/inode.c | 12 fs/ext4/ioctl.c | 3 ++- fs/ext4/namei.c | 4 ++--

[PATCH v5 10/19] nfs: convert to new i_version API

2018-01-09 Thread Jeff Layton
From: Jeff Layton For NFS, we just use the "raw" API since the i_version is mostly managed by the server. The exception there is when the client holds a write delegation, but we only need to bump it once there anyway to handle CB_GETATTR. Tested-by: Krzysztof Kozlowski

[PATCH v5 13/19] ufs: use new i_version API

2018-01-09 Thread Jeff Layton
From: Jeff Layton Signed-off-by: Jeff Layton --- fs/ufs/dir.c | 9 + fs/ufs/inode.c | 3 ++- fs/ufs/super.c | 3 ++- 3 files changed, 9 insertions(+), 6 deletions(-) diff --git a/fs/ufs/dir.c b/fs/ufs/dir.c index 2edc1755b7c5..50dfce000864

Re: Recommendations for balancing as part of regular maintenance?

2018-01-09 Thread Tom Worster
On 9 Jan 2018, at 7:23, Austin S. Hemmelgarn wrote: On 2018-01-08 16:43, Tom Worster wrote: Given the documentation and the usage stats, I did not know what options to use with balance. I spent some time reading and researching and trying to understand the filters and how they should

[PATCH v5 11/19] nfsd: convert to new i_version API

2018-01-09 Thread Jeff Layton
From: Jeff Layton Mostly just making sure we use the "get" wrappers so we know when it is being fetched for later use. Signed-off-by: Jeff Layton --- fs/nfsd/nfsfh.h | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/fs/nfsd/nfsfh.h

[PATCH v5 12/19] ocfs2: convert to new i_version API

2018-01-09 Thread Jeff Layton
From: Jeff Layton Signed-off-by: Jeff Layton Reviewed-by: Jan Kara --- fs/ocfs2/dir.c | 15 --- fs/ocfs2/inode.c| 3 ++- fs/ocfs2/namei.c| 3 ++- fs/ocfs2/quota_global.c | 3 ++- 4 files changed, 14

[PATCH v5 15/19] IMA: switch IMA over to new i_version API

2018-01-09 Thread Jeff Layton
From: Jeff Layton Signed-off-by: Jeff Layton --- security/integrity/ima/ima_api.c | 3 ++- security/integrity/ima/ima_main.c | 3 ++- 2 files changed, 4 insertions(+), 2 deletions(-) diff --git a/security/integrity/ima/ima_api.c

[PATCH v5 14/19] xfs: convert to new i_version API

2018-01-09 Thread Jeff Layton
From: Jeff Layton Signed-off-by: Jeff Layton Acked-by: Darrick J. Wong --- fs/xfs/libxfs/xfs_inode_buf.c | 7 +-- fs/xfs/xfs_icache.c | 5 +++-- fs/xfs/xfs_inode.c| 3 ++- fs/xfs/xfs_inode_item.c

[PATCH v5 17/19] xfs: avoid setting XFS_ILOG_CORE if i_version doesn't need incrementing

2018-01-09 Thread Jeff Layton
From: Jeff Layton If XFS_ILOG_CORE is already set then go ahead and increment it. Signed-off-by: Jeff Layton Acked-by: Darrick J. Wong --- fs/xfs/xfs_trans_inode.c | 14 -- 1 file changed, 8 insertions(+), 6

  1   2   >