Re: [PATCH 2/8] btrfs: extent-tree: Open-code process_func in __btrfs_mod_ref

2018-12-07 Thread Nikolay Borisov
ned-off-by: Qu Wenruo Reviewed-by: Nikolay Borisov > --- > fs/btrfs/extent-tree.c | 33 ++--- > 1 file changed, 18 insertions(+), 15 deletions(-) > > diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c > index ea2c3d5220f0..ea68d288d761 100644 &g

Re: [PATCH 05/10] btrfs: introduce delayed_refs_rsv

2018-12-07 Thread Nikolay Borisov
On 3.12.18 г. 17:20 ч., Josef Bacik wrote: > From: Josef Bacik > > Traditionally we've had voodoo in btrfs to account for the space that > delayed refs may take up by having a global_block_rsv. This works most > of the time, except when it doesn't. We've had issues reported and seen > in

Re: [PATCH 04/10] btrfs: only track ref_heads in delayed_ref_updates

2018-12-07 Thread Nikolay Borisov
thing along the lines of: In addition to using this number to limit the number of delayed refs run, a future patch is also going to use it to calculate the amount of space required for delayed refs space reservation. > > Reviewed-by: Nikolay Borisov > Signed-off-by: Josef Bacik >

Re: [PATCH 08/10] btrfs: rework btrfs_check_space_for_delayed_refs

2018-12-07 Thread Nikolay Borisov
On 7.12.18 г. 9:09 ч., Nikolay Borisov wrote: > > > On 6.12.18 г. 19:54 ч., David Sterba wrote: >> On Thu, Dec 06, 2018 at 06:52:21PM +0200, Nikolay Borisov wrote: >>> >>> >>> On 3.12.18 г. 17:20 ч., Josef Bacik wrote: >>>> Now with

Re: [PATCH 08/10] btrfs: rework btrfs_check_space_for_delayed_refs

2018-12-06 Thread Nikolay Borisov
On 6.12.18 г. 19:54 ч., David Sterba wrote: > On Thu, Dec 06, 2018 at 06:52:21PM +0200, Nikolay Borisov wrote: >> >> >> On 3.12.18 г. 17:20 ч., Josef Bacik wrote: >>> Now with the delayed_refs_rsv we can now know exactly how much pending >>> delayed

Re: [PATCH 08/10] btrfs: rework btrfs_check_space_for_delayed_refs

2018-12-06 Thread Nikolay Borisov
ond that amount then we > know it's time to commit the transaction and stop any more delayed refs > from being generated. > > Signed-off-by: Josef Bacik Reviewed-by: Nikolay Borisov > --- > fs/btrfs/ctree.h | 2 +- > fs/btrfs/extent-tree.c | 48 ++--

Re: [PATCH 09/10] btrfs: don't run delayed refs in the end transaction logic

2018-12-06 Thread Nikolay Borisov
On 3.12.18 г. 17:20 ч., Josef Bacik wrote: > Over the years we have built up a lot of infrastructure to keep delayed > refs in check, mostly by running them at btrfs_end_transaction() time. > We have a lot of different maths we do to figure out how much, if we > should do it inline or async,

Re: [PATCH 06/10] btrfs: update may_commit_transaction to use the delayed refs rsv

2018-12-06 Thread Nikolay Borisov
ecision. > > Signed-off-by: Josef Bacik Reviewed-by: Nikolay Borisov However, look below for one suggestion: > --- > fs/btrfs/extent-tree.c | 24 +++- > 1 file changed, 15 insertions(+), 9 deletions(-) > > diff --git a/fs/btrfs/extent-tree.c

Re: [PATCH 02/10] btrfs: add cleanup_ref_head_accounting helper

2018-12-06 Thread Nikolay Borisov
f_cleanup mean that every time the last reference for a block was freed we were leaking bytes in total_bytes_pinned? Shouldn't this have lead to eventually total_bytes_pinned dominating the usage in a space_info ? Codewise lgtm: Reviewed-by: Nikolay Borisov > -

Re: [PATCH 01/10] btrfs: add btrfs_delete_ref_head helper

2018-12-06 Thread Nikolay Borisov
On 3.12.18 г. 17:20 ч., Josef Bacik wrote: > From: Josef Bacik > > We do this dance in cleanup_ref_head and check_ref_cleanup, unify it > into a helper and cleanup the calling functions. > > Signed-off-by: Josef Bacik > Reviewed-by: Omar Sandoval Reviewed-by: Nikolay

Re: [PATCH 1/3] btrfs: use offset_in_page instead of open-coding it

2018-12-05 Thread Nikolay Borisov
check. > > Signed-off-by: Johannes Thumshirn Reviewed-by: Nikolay Borisov > --- > fs/btrfs/check-integrity.c | 12 +-- > fs/btrfs/compression.c | 2 +- > fs/btrfs/extent_io.c | 53 > +- > fs/btrfs/file.c

Re: [PATCH 2/3] btrfs: use PAGE_ALIGNED instead of open-coding it

2018-12-05 Thread Nikolay Borisov
(). > > Signed-off-by: Johannes Thumshirn Reviewed-by: Nikolay Borisov > --- > fs/btrfs/check-integrity.c | 8 > fs/btrfs/compression.c | 2 +- > fs/btrfs/inode.c | 2 +- > 3 files changed, 6 insertions(+), 6 deletions(-) > > diff --git a/fs/btrfs/

Re: [PATCH 04/10] Rename __endio_write_update_ordered() to btrfs_update_ordered_extent()

2018-12-05 Thread Nikolay Borisov
On 5.12.18 г. 14:28 ч., Goldwyn Rodrigues wrote: > From: Goldwyn Rodrigues > > Since we will be using it in another part of the code, use a > better name to declare it non-static > > Signed-off-by: Goldwyn Rodrigues > --- > fs/btrfs/ctree.h | 7 +-- > fs/btrfs/inode.c | 14

Re: [PATCH 03/10] btrfs: dax: read zeros from holes

2018-12-05 Thread Nikolay Borisov
On 5.12.18 г. 14:28 ч., Goldwyn Rodrigues wrote: > From: Goldwyn Rodrigues > > Signed-off-by: Goldwyn Rodrigues > --- > fs/btrfs/dax.c | 7 ++- > 1 file changed, 6 insertions(+), 1 deletion(-) > > diff --git a/fs/btrfs/dax.c b/fs/btrfs/dax.c > index d614bf73bf8e..5a297674adec 100644 >

Re: [PATCH 02/10] btrfs: basic dax read

2018-12-05 Thread Nikolay Borisov
On 5.12.18 г. 14:28 ч., Goldwyn Rodrigues wrote: > From: Goldwyn Rodrigues > > Signed-off-by: Goldwyn Rodrigues > --- > fs/btrfs/Makefile | 1 + > fs/btrfs/ctree.h | 5 > fs/btrfs/dax.c| 68 > +++ > fs/btrfs/file.c | 13

Re: [PATCH 01/10] btrfs: create a mount option for dax

2018-12-05 Thread Nikolay Borisov
On 5.12.18 г. 14:28 ч., Goldwyn Rodrigues wrote: > From: Goldwyn Rodrigues > > Also, set the inode->i_flags to S_DAX > > Signed-off-by: Goldwyn Rodrigues Reviewed-by: Nikolay Borisov One question below though . > --- > fs/btrfs/ctree.h | 1 + > fs/btrfs/ioc

[PATCH v2] btrfs: Remove unnecessary code from __btrfs_rebalance

2018-12-05 Thread Nikolay Borisov
ving heavy use of the btree. Signed-off-by: Nikolay Borisov Suggested-by: Josef Bacik Reviewed-by: Josef Bacik --- Changes since v1: * Improved changelog by adding information about reduced runtimes and explaining where they would come from. I did measurements of btrfs balance with and without the

Re: BTRFS Mount Delay Time Graph

2018-12-04 Thread Nikolay Borisov
On 4.12.18 г. 22:14 ч., Wilson, Ellis wrote: > On 12/4/18 8:07 AM, Nikolay Borisov wrote: >> On 3.12.18 г. 20:20 ч., Wilson, Ellis wrote: >>> With 14TB drives available today, it doesn't take more than a handful of >>> drives to result in a filesystem that tak

Re: [PATCHv3] btrfs: Fix error handling in btrfs_cleanup_ordered_extents

2018-12-04 Thread Nikolay Borisov
On 21.11.18 г. 17:10 ч., Nikolay Borisov wrote: > Running btrfs/124 in a loop hung up on me sporadically with the > following call trace: > btrfs D0 5760 5324 0x > Call Trace: >? __schedule+0x243/0x800 >

Re: [PATCH 2/2] btrfs: scrub: move scrub_setup_ctx allocation out of device_list_mutex

2018-12-04 Thread Nikolay Borisov
On 4.12.18 г. 17:11 ч., David Sterba wrote: > The scrub context is allocated with GFP_KERNEL and called from > btrfs_scrub_dev under the fs_info::device_list_mutex. This is not safe > regarding reclaim that could try to flush filesystem data in order to > get the memory. And the

Re: [PATCH 1/2] btrfs: scrub: pass fs_info to scrub_setup_ctx

2018-12-04 Thread Nikolay Borisov
On 4.12.18 г. 17:11 ч., David Sterba wrote: > We can pass fs_info directly as this is the only member of btrfs_device > that's bing used inside scrub_setup_ctx. > > Signed-off-by: David Sterba Reviewed-by: Nikolay Borisov > --- > fs/btrfs/scrub.c | 9 - &g

Re: BTRFS Mount Delay Time Graph

2018-12-04 Thread Nikolay Borisov
On 3.12.18 г. 20:20 ч., Wilson, Ellis wrote: > Hi all, > > Many months ago I promised to graph how long it took to mount a BTRFS > filesystem as it grows. I finally had (made) time for this, and the > attached is the result of my testing. The image is a fairly > self-explanatory graph,

Re: [PATCH 7/9] btrfs-progs: Fix Wmaybe-uninitialized warning

2018-12-04 Thread Nikolay Borisov
On 4.12.18 г. 14:22 ч., Adam Borowski wrote: > On Tue, Dec 04, 2018 at 01:17:04PM +0100, David Sterba wrote: >> On Fri, Nov 16, 2018 at 03:54:24PM +0800, Qu Wenruo wrote: >>> The only location is the following code: >>> >>> int level = path->lowest_level + 1; >>>

Re: [PATCH 3/3] btrfs: replace cleaner_delayed_iput_mutex with a waitqueue

2018-12-04 Thread Nikolay Borisov
On 3.12.18 г. 18:06 ч., Josef Bacik wrote: > The throttle path doesn't take cleaner_delayed_iput_mutex, which means > we could think we're done flushing iputs in the data space reservation > path when we could have a throttler doing an iput. There's no real > reason to serialize the delayed

Re: [PATCH 2/3] btrfs: wakeup cleaner thread when adding delayed iput

2018-12-04 Thread Nikolay Borisov
On 3.12.18 г. 18:06 ч., Josef Bacik wrote: > The cleaner thread usually takes care of delayed iputs, with the > exception of the btrfs_end_transaction_throttle path. The cleaner > thread only gets woken up every 30 seconds, so instead wake it up to do > it's work so that we can free up that

Re: [PATCH 1/3] btrfs: run delayed iputs before committing

2018-12-04 Thread Nikolay Borisov
yed iputs so that any potential space is free'd up. > If there is and we freed enough we can then commit the transaction and > potentially be able to make our reservation. > > Signed-off-by: Josef Bacik > Reviewed-by: Omar Sandoval Reviewed-by: Nikolay Borisov > ---

Re: [PATCH 2/5] btrfs: Refactor btrfs_can_relocate

2018-12-03 Thread Nikolay Borisov
On 3.12.18 г. 19:25 ч., David Sterba wrote: > On Sat, Nov 17, 2018 at 09:29:27AM +0800, Anand Jain wrote: >>> - ret = find_free_dev_extent(trans, device, min_free, >>> - _offset, NULL); >>> - if (!ret) >>> +

Re: [RFC PATCH] btrfs: Remove __extent_readpages

2018-12-03 Thread Nikolay Borisov
On 3.12.18 г. 12:25 ч., Nikolay Borisov wrote: > When extent_readpages is called from the generic readahead code it first > builds a batch of 16 pages (which might or might not be consecutive, > depending on whether add_to_page_cache_lru failed) and submits them to > __exte

[RFC PATCH] btrfs: Remove __extent_readpages

2018-12-03 Thread Nikolay Borisov
to __do_contiguous_readpages. Also simplify the name of the function to contiguous_readpages. Signed-off-by: Nikolay Borisov --- So this patch looks like a very nice cleanup, however when doing performance measurements with fio I was shocked to see that it actually is detrimental to performance

Re: btrfs development - question about crypto api integration

2018-11-30 Thread Nikolay Borisov
On 30.11.18 г. 17:22 ч., Chris Mason wrote: > On 29 Nov 2018, at 12:37, Nikolay Borisov wrote: > >> On 29.11.18 г. 18:43 ч., Jean Fobe wrote: >>> Hi all, >>> I've been studying LZ4 and other compression algorithms on the >>> kernel, and seen othe

Re: btrfs development - question about crypto api integration

2018-11-29 Thread Nikolay Borisov
On 29.11.18 г. 18:43 ч., Jean Fobe wrote: > Hi all, > I've been studying LZ4 and other compression algorithms on the > kernel, and seen other projects such as zram and ubifs using the > crypto api. Is there a technical reason for not using the crypto api > for compression (and possibly for

[PATCH] btrfs: Refactor main loop in extent_readpages

2018-11-29 Thread Nikolay Borisov
extent_readpages 4761136+660 __extent_readpages 820 --820 Total: Before=44315, After=44155, chg -0.36% Signed-off-by: Nikolay Borisov --- fs/btrfs/extent_io.c | 37

Re: [PATCH v2 1/3] btrfs: remove always true if branch in find_delalloc_range

2018-11-28 Thread Nikolay Borisov
On 29.11.18 г. 5:33 ч., Lu Fengqi wrote: > The @found is always false when it comes to the if branch. Besides, the > bool type is more suitable for @found. Change the return value of the > function and its caller to bool as well. > > Signed-off-by: Lu Fengqi Reviewed-by:

Re: [PATCH] btrfs: Refactor btrfs_merge_bio_hook

2018-11-28 Thread Nikolay Borisov
On 28.11.18 г. 18:46 ч., David Sterba wrote: > On Tue, Nov 27, 2018 at 08:57:58PM +0200, Nikolay Borisov wrote: >> This function really checks whether adding more data to the bio will >> straddle a stripe/chunk. So first let's give it a more appropraite >> name - btr

Re: [PATCH] Fix typos

2018-11-28 Thread Nikolay Borisov
On 28.11.18 г. 15:24 ч., Brendan Hide wrote: > > > On 11/28/18 1:23 PM, Nikolay Borisov wrote: >> >> >> On 28.11.18 г. 13:05 ч., Andrea Gelmini wrote: >>> Signed-off-by: Andrea Gelmini >>> --- > > > >>> >>>

Re: [PATCH] Fix typos

2018-11-28 Thread Nikolay Borisov
On 28.11.18 г. 13:05 ч., Andrea Gelmini wrote: > Signed-off-by: Andrea Gelmini > --- > > Stupid fixes. Made on 4.20-rc4, and ported on linux-next (next-20181128). > > > fs/btrfs/backref.c | 4 ++-- > fs/btrfs/check-integrity.c | 2 +- > fs/btrfs/compression.c | 4 ++-- >

Re: [PATCH v2 3/3] btrfs: document extent mapping assumptions in checksum

2018-11-28 Thread Nikolay Borisov
e current extent buffer allocator always guarantees page aligned > extends, so the above condition can't be true. > > Signed-off-by: Johannes Thumshirn Reviewed-by: Nikolay Borisov > > --- > Changes to v1: > * Changed wording of the commit message according to Noah's sugge

Re: [RFC PATCH 03/17] btrfs: priority alloc: introduce compute_block_group_priority/usage

2018-11-28 Thread Nikolay Borisov
On 28.11.18 г. 5:11 ч., Su Yue wrote: > Introduce compute_block_group_usage() and compute_block_group_usage(). > And call the latter in btrfs_make_block_group() and > btrfs_read_block_groups(). > > compute_priority_level use ilog2(free) to compute priority level. > > Signed-off-by: Su Yue >

Re: [RFC PATCH 01/17] btrfs: priority alloc: prepare of priority aware allocator

2018-11-28 Thread Nikolay Borisov
On 28.11.18 г. 5:11 ч., Su Yue wrote: > To implement priority aware allocator, this patch: > Introduces struct btrfs_priority_tree which contains block groups > in same level. > Adds member priority to struct btrfs_block_group_cache and pointer > points to the priority tree it's located. > >

Re: [PATCH 5/8] btrfs: don't enospc all tickets on flush failure

2018-11-28 Thread Nikolay Borisov
On 27.11.18 г. 21:46 ч., Josef Bacik wrote: > On Mon, Nov 26, 2018 at 02:25:52PM +0200, Nikolay Borisov wrote: >> >> >> On 21.11.18 г. 21:03 ч., Josef Bacik wrote: >>> With the introduction of the per-inode block_rsv it became possible to >>> have really

Re: [RFC PATCH] btrfs: drop file privileges in btrfs_clone_files

2018-11-27 Thread Nikolay Borisov
On 28.11.18 г. 9:46 ч., Christoph Hellwig wrote: > On Wed, Nov 28, 2018 at 09:44:59AM +0200, Nikolay Borisov wrote: >> >> >> On 28.11.18 г. 5:07 ч., Lu Fengqi wrote: >>> The generic/513 tell that cloning into a file did not strip security >>> privileges (s

Re: [RFC PATCH] btrfs: drop file privileges in btrfs_clone_files

2018-11-27 Thread Nikolay Borisov
On 28.11.18 г. 5:07 ч., Lu Fengqi wrote: > The generic/513 tell that cloning into a file did not strip security > privileges (suid, capabilities) like a regular write would. > > Signed-off-by: Lu Fengqi > --- > The xfs and ocfs2 call generic_remap_file_range_prep to drop file > privileges,

Re: [PATCH] btrfs: adjust order of unlocks in do_trimming()

2018-11-27 Thread Nikolay Borisov
On 28.11.18 г. 5:21 ч., Su Yue wrote: > In function do_trimming(), block_group->lock should be unlocked first. > > Fixes: 7fe1e6415026 ("Btrfs: rewrite btrfs_trim_block_group()") > Signed-off-by: Su Yue Reviewed-by: Nikolay Borisov > --- > fs/btrfs/free

Re: [PATCH 3/3] btrfs: remove redundant nowait check for buffered_write

2018-11-27 Thread Nikolay Borisov
ct, so: Reviewed-by: Nikolay Borisov > > Signed-off-by: Lu Fengqi > --- > fs/btrfs/file.c | 4 > 1 file changed, 4 deletions(-) > > diff --git a/fs/btrfs/file.c b/fs/btrfs/file.c > index 3835bb8c146d..190db9a685a2 100644 > --- a/fs/btrfs/file.c > +++ b/fs/btrfs/fi

Re: [PATCH 1/3] btrfs: remove always true if branch in find_delalloc_range

2018-11-27 Thread Nikolay Borisov
On 28.11.18 г. 5:21 ч., Lu Fengqi wrote: > The @found is always false when it comes to the if branch. Besides, the > bool type is more suitable for @found. Well if you are ranging the type of found variable it also makes sense to change the return value of the function to bool as well. > >

Re: [PATCH 2/3] btrfs: cleanup the useless DEFINE_WAIT in cleanup_transaction

2018-11-27 Thread Nikolay Borisov
On 28.11.18 г. 5:22 ч., Lu Fengqi wrote: > When it is introduced at commit f094ac32aba3 ("Btrfs: fix NULL pointer > after aborting a transaction"), it's useless. > > Signed-off-by: Lu Fengqi Reviewed-by: Nikolay Borisov > --- > fs/btrfs/transaction.c | 1 - &

Re: [PATCH 3/3] btrfs: document extent mapping assumptions in checksum

2018-11-27 Thread Nikolay Borisov
On 27.11.18 г. 21:08 ч., Noah Massey wrote: > On Tue, Nov 27, 2018 at 11:43 AM Nikolay Borisov wrote: >> >> On 27.11.18 г. 18:00 ч., Johannes Thumshirn wrote: >>> Document why map_private_extent_buffer() cannot return '1' (i.e. the map >>> spans two page

[PATCH] btrfs: Refactor btrfs_merge_bio_hook

2018-11-27 Thread Nikolay Borisov
it's guaranteed that tree->ops is set so replace the check with an ASSERT. Finally, document the parameters of the function. No functional changes. Signed-off-by: Nikolay Borisov --- fs/btrfs/compression.c | 7 --- fs/btrfs/ctree.h | 5 ++--- fs/btrfs/extent_io.c | 4 ++-- fs/bt

Re: [PATCH 3/3] btrfs: document extent mapping assumptions in checksum

2018-11-27 Thread Nikolay Borisov
than PAGE_SIZE. > > We always pass BTRFS_CSUM_SIZE (32) as offset and a minimal length of 32 > and the current extent buffer allocator always guarantees page aligned > extends, so the above condition can't be true. > > Signed-off-by: Johannes Thumshirn With that word

Re: [PATCH 2/3] btrfs: use offset_in_page for start_offset in map_private_extent_buffer()

2018-11-27 Thread Nikolay Borisov
On 27.11.18 г. 18:00 ч., Johannes Thumshirn wrote: > In map_private_extent_buffer() use offset_in_page() to initialize > 'start_offset' instead of open-coding it. > > Signed-off-by: Johannes Thumshirn Reviewed-by: Nikolay Borisov > --- > fs/btrfs/extent_io.c | 2 +- &g

Re: [PATCH 1/3] btrfs: don't initialize 'offset' in map_private_extent_buffer()

2018-11-27 Thread Nikolay Borisov
> > So get rid of the initial initialization. > > Signed-off-by: Johannes Thumshirn You know, the fastest/most clean code is the one which is deleted so : Reviewed-by: Nikolay Borisov > --- > fs/btrfs/extent_io.c | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > &

Re: [PATCH] Proposal for more detail in scrub doc

2018-11-27 Thread Nikolay Borisov
On 27.11.18 г. 16:14 ч., Andrea Gelmini wrote: > Wise words from Qu: > https://www.mail-archive.com/linux-btrfs@vger.kernel.org/msg82557.html > --- > Documentation/btrfs-scrub.asciidoc | 3 ++- > 1 file changed, 2 insertions(+), 1 deletion(-) > > diff --git

Re: [PATCH] Fix typos in docs (second try...)

2018-11-27 Thread Nikolay Borisov
On 27.11.18 г. 15:56 ч., Andrea Gelmini wrote: > Signed-off-by: Andrea Gelmini Reviewed-by: Nikolay Borisov > --- > Documentation/DocConventions | 4 ++-- > Documentation/ReleaseChecklist| 4 ++-- > Documentation/btrfs-man5.asciidoc | 8 > D

Re: [PATCH] Fix typos in docs

2018-11-27 Thread Nikolay Borisov
On 27.11.18 г. 15:37 ч., Andrea Gelmini wrote: > Signed-off-by: Andrea Gelmini > --- > Documentation/DocConventions | 4 ++-- > Documentation/ReleaseChecklist| 4 ++-- > Documentation/btrfs-man5.asciidoc | 8 > Documentation/btrfs-property.asciidoc | 2 +- >

Re: [PATCH v2 1/5] btrfs-progs: image: Refactor fixup_devices() to fixup_chunks_and_devices()

2018-11-27 Thread Nikolay Borisov
On 27.11.18 г. 10:50 ч., Qu Wenruo wrote: > > > On 2018/11/27 下午4:46, Nikolay Borisov wrote: >> >> >> On 27.11.18 г. 10:38 ч., Qu Wenruo wrote: >>> Current fixup_devices() will only remove DEV_ITEMs and reset DEV_ITEM >>> size. >>> La

Re: [PATCH v2 5/5] btrfs-progs: misc-tests/021: Do extra btrfs check before mounting

2018-11-27 Thread Nikolay Borisov
t pass btrfs check for chunk tree will not > mount. > > So do extra btrfs check before mount, this will also help us to locate > the problem in btrfs-image easier. > > Signed-off-by: Qu Wenruo Reviewed-by: Nikolay Borisov > --- > tests/misc-tests/021-image-multi-devices/test.

Re: [PATCH v2 1/5] btrfs-progs: image: Refactor fixup_devices() to fixup_chunks_and_devices()

2018-11-27 Thread Nikolay Borisov
(). > > Signed-off-by: Qu Wenruo Reviewed-by: Nikolay Borisov However, one minor nit below. > --- > image/main.c | 52 > 1 file changed, 36 insertions(+), 16 deletions(-) > > diff --git a/image/main.c b/image/main.

Re: [PATCH v2 3/5] btrfs-progs: volumes: Refactor btrfs_alloc_dev_extent() into two functions

2018-11-27 Thread Nikolay Borisov
will call > btrfs_insert_dev_extent() anyway, so no duplicated code. > > This removes the need of @convert parameter, and make > btrfs_insert_dev_extent() public for later usage. > > Signed-off-by: Qu Wenruo This looks much better:

Re: [PATCH 3/3] btrfs: replace cleaner_delayed_iput_mutex with a waitqueue

2018-11-27 Thread Nikolay Borisov
On 21.11.18 г. 21:09 ч., Josef Bacik wrote: > The throttle path doesn't take cleaner_delayed_iput_mutex, which means Which one is the throttle path? btrfs_end_transaction_throttle is only called during snapshot drop and relocation? What scenario triggered your observation and prompted this

Re: [PATCH 2/3] btrfs: wakeup cleaner thread when adding delayed iput

2018-11-27 Thread Nikolay Borisov
On 21.11.18 г. 21:09 ч., Josef Bacik wrote: > The cleaner thread usually takes care of delayed iputs, with the > exception of the btrfs_end_transaction_throttle path. The cleaner > thread only gets woken up every 30 seconds, so instead wake it up to do > it's work so that we can free up that

Re: [PATCH 4/5] btrfs-progs: image: Rebuild dev extents using chunk tree

2018-11-26 Thread Nikolay Borisov
On 27.11.18 г. 9:30 ч., Qu Wenruo wrote: > > > On 2018/11/27 下午3:28, Nikolay Borisov wrote: >> >> >> On 27.11.18 г. 4:33 ч., Qu Wenruo wrote: >>> With existing dev extents cleaned up, now we can rebuild dev extents >>> using the correct chun

Re: [PATCH 5/5] btrfs-progs: misc-tests/021: Do extra btrfs check before mounting

2018-11-26 Thread Nikolay Borisov
On 27.11.18 г. 4:33 ч., Qu Wenruo wrote: > Test case misc/021 is testing if we could mount a single disk btrfs > image recovered from multi disk fs. > > The problem is, current kernel has extra check for block group, chunk > and dev extent. > This means any image can't pass btrfs check for

Re: [PATCH 4/5] btrfs-progs: image: Rebuild dev extents using chunk tree

2018-11-26 Thread Nikolay Borisov
On 27.11.18 г. 4:33 ч., Qu Wenruo wrote: > With existing dev extents cleaned up, now we can rebuild dev extents > using the correct chunk tree. > > Since new dev extents are all rebuild from scratch, even we're restoring > image from multi-device fs to single disk, we won't have any problem >

Re: [PATCH 3/5] btrfs-progs: image: Remove all existing dev extents for later rebuild

2018-11-26 Thread Nikolay Borisov
On 27.11.18 г. 4:33 ч., Qu Wenruo wrote: > This patch will remove all existing dev extents for later rebuild. > > Signed-off-by: Qu Wenruo > --- > image/main.c | 68 > 1 file changed, 68 insertions(+) > > diff --git a/image/main.c

Re: [PATCH 2/5] btrfs-progs: image: Fix block group item flags when restoring multi-device image to single device

2018-11-26 Thread Nikolay Borisov
On 27.11.18 г. 4:33 ч., Qu Wenruo wrote: > Since btrfs-image is just restoring tree blocks without really check if > that tree block contents makes sense, for multi-device image, block > group items will keep that incorrect block group flags. > > For example, for a metadata RAID1 data RAID0

Re: [PATCH 1/5] btrfs-progs: image: Refactor fixup_devices() to fixup_chunks_and_devices()

2018-11-26 Thread Nikolay Borisov
On 27.11.18 г. 4:33 ч., Qu Wenruo wrote: > Current fixup_devices() will only remove DEV_ITEMs and reset DEV_ITEM > size. > Later we need to do more fixup works, so change the name to > fixup_chunks_and_devices() and refactor the original device size fixup > to fixup_device_size(). > >

Re: [PATCH 1/3] btrfs: run delayed iputs before committing

2018-11-26 Thread Nikolay Borisov
On 21.11.18 г. 21:09 ч., Josef Bacik wrote: > Delayed iputs means we can have final iputs of deleted inodes in the > queue, which could potentially generate a lot of pinned space that could > be free'd. So before we decide to commit the transaction for ENOPSC > reasons, run the delayed iputs

Re: [PATCH 7/8] btrfs: be more explicit about allowed flush states

2018-11-26 Thread Nikolay Borisov
On 26.11.18 г. 14:41 ч., Nikolay Borisov wrote: > > > On 21.11.18 г. 21:03 ч., Josef Bacik wrote: >> For FLUSH_LIMIT flushers we really can only allocate chunks and flush >> delayed inode items, everything else is problematic. I added a bunch of >> new stat

Re: [PATCH 7/8] btrfs: be more explicit about allowed flush states

2018-11-26 Thread Nikolay Borisov
On 21.11.18 г. 21:03 ч., Josef Bacik wrote: > For FLUSH_LIMIT flushers we really can only allocate chunks and flush > delayed inode items, everything else is problematic. I added a bunch of > new states and it lead to weirdness in the FLUSH_LIMIT case because I > forgot about how it worked.

Re: [PATCH 5/8] btrfs: don't enospc all tickets on flush failure

2018-11-26 Thread Nikolay Borisov
On 21.11.18 г. 21:03 ч., Josef Bacik wrote: > With the introduction of the per-inode block_rsv it became possible to > have really really large reservation requests made because of data > fragmentation. Since the ticket stuff assumed that we'd always have > relatively small reservation

Re: [PATCH 4/8] btrfs: add ALLOC_CHUNK_FORCE to the flushing code

2018-11-26 Thread Nikolay Borisov
e likely need it to make progress. This > resolves the issues I was seeing with the mixed bg tests in xfstests > with my previous patch. > > Signed-off-by: Josef Bacik Reviewed-by: Nikolay Borisov Still, my observation is that the metadata reclaim code is increasing in complexity for r

Re: [PATCH 3/8] btrfs: don't use global rsv for chunk allocation

2018-11-26 Thread Nikolay Borisov
On 21.11.18 г. 21:03 ч., Josef Bacik wrote: > We've done this forever because of the voodoo around knowing how much > space we have. However we have better ways of doing this now, and on > normal file systems we'll easily have a global reserve of 512MiB, and > since metadata chunks are usually

Re: [PATCH 1/8] btrfs: check if free bgs for commit

2018-11-26 Thread Nikolay Borisov
d-by: Omar Sandoval Reviewed-by: Nikolay Borisov > --- > fs/btrfs/extent-tree.c | 33 +++-- > 1 file changed, 19 insertions(+), 14 deletions(-) > > diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c > index 8af68b13fa27..0dca250dc02e 100644 > -

Re: [PATCH 2/2] btrfs: scrub: fix circular locking dependency warning

2018-11-26 Thread Nikolay Borisov
On 26.11.18 г. 11:07 ч., Anand Jain wrote: > Circular locking dependency check reports warning[1], that's because > the btrfs_scrub_dev() calls the stack #0 below with, the > fs_info::scrub_lock held. The test case leading to this warning.. > > mkfs.btrfs -fq /dev/sdb && mount /dev/sdb

Re: [PATCH 1/2] btrfs: scrub: maintain the unlock order in scrub thread

2018-11-26 Thread Nikolay Borisov
On 26.11.18 г. 11:07 ч., Anand Jain wrote: > The fs_info::device_list_mutex and fs_info::scrub_lock creates a > nested locks in btrfs_scrub_dev(). During the lock acquire the > hierarchy is fs_info::device_list_mutex and then fs_info::scrub_lock, > so following the same reverse order during

Re: [PATCH 6/6] btrfs: fix truncate throttling

2018-11-26 Thread Nikolay Borisov
d-off-by: Josef Bacik Reviewed-by: Nikolay Borisov > --- > fs/btrfs/inode.c | 79 > > 1 file changed, 17 insertions(+), 62 deletions(-) > > diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c > index 8532a2eb56d1..

Re: [PATCH 5/6] btrfs: introduce delayed_refs_rsv

2018-11-26 Thread Nikolay Borisov
On 21.11.18 г. 20:59 ч., Josef Bacik wrote: > From: Josef Bacik > > Traditionally we've had voodoo in btrfs to account for the space that > delayed refs may take up by having a global_block_rsv. This works most > of the time, except when it doesn't. We've had issues reported and seen > in

Re: [PATCH v4] Btrfs: fix deadlock with memory reclaim during scrub

2018-11-25 Thread Nikolay Borisov
locations because at any > time a scrub pause request can happen from another task that started to > commit a transaction. > > Fixes: 58c4e173847a ("btrfs: scrub: use GFP_KERNEL on the submission path") > Signed-off-by: Filipe Manana Reviewed-by: Nikolay Borisov > -

Re: [PATCH v2] Btrfs: fix deadlock with memory reclaim during scrub

2018-11-23 Thread Nikolay Borisov
On 23.11.18 г. 18:05 ч., fdman...@kernel.org wrote: > From: Filipe Manana > > When a transaction commit starts, it attempts to pause scrub and it blocks > until the scrub is paused. So while the transaction is blocked waiting for > scrub to pause, we can not do memory allocation with

Re: [PATCH 1/6] btrfs: add btrfs_delete_ref_head helper

2018-11-23 Thread Nikolay Borisov
On 23.11.18 г. 15:45 ч., David Sterba wrote: > On Thu, Nov 22, 2018 at 11:42:28AM +0200, Nikolay Borisov wrote: >> >> >> On 22.11.18 г. 11:12 ч., Nikolay Borisov wrote: >>> >>> >>> On 21.11.18 г. 20:59 ч., Josef Bacik wrote: >>>> From:

Re: [PATCH] btrfs: remove btrfs_bio_end_io_t

2018-11-23 Thread Nikolay Borisov
rfs_bio' which is only needed for btrfs_bio_end_io_t. > > Remove both as they're not needed anywhere. > > Signed-off-by: Johannes Thumshirn Reviewed-by: Nikolay Borisov > --- > fs/btrfs/volumes.h | 3 --- > 1 file changed, 3 deletions(-) > > diff --git a/fs/btrfs/vo

Re: [PATCH 4/4] btrfs: replace btrfs_io_bio::end_io with a simple helper

2018-11-22 Thread Nikolay Borisov
indirection and call a helper that will free the > csums only in the right case. > > This shrinks struct btrfs_io_bio by 8 bytes. > > Signed-off-by: David Sterba Reviewed-by: Nikolay Borisov > --- > fs/btrfs/extent_io.c | 3 +-- > fs/btrfs/file-item.c | 9 --

Re: [PATCH 3/4] btrfs: remove redundant csum buffer in btrfs_io_bio

2018-11-22 Thread Nikolay Borisov
er and check that the inline buffer is not > accidentally freed. > > This shrinks struct btrfs_io_bio by 8 bytes. > > Signed-off-by: David Sterba Reviewed-by: Nikolay Borisov > --- > fs/btrfs/file-item.c | 12 +++- > fs/btrfs/volumes.h | 1 - > 2 file

Re: [PATCH 2/4] btrfs: replace async_cow::root with fs_info

2018-11-22 Thread Nikolay Borisov
On 22.11.18 г. 18:16 ч., David Sterba wrote: > The async_cow::root is used to propagate fs_info to async_cow_submit. > We can't use inode to reach it because it could become NULL after > write without compression in async_cow_start. > > Signed-off-by: David Sterba Revie

Re: [PATCH 1/4] btrfs: merge btrfs_submit_bio_done to its caller

2018-11-22 Thread Nikolay Borisov
On 22.11.18 г. 18:16 ч., David Sterba wrote: > There's one caller and its code is simple, we can open code it in > run_one_async_done. The errors are passed through bio. > > Signed-off-by: David Sterba Reviewed-by: Nikolay Borisov > --- > fs/btrfs/disk-io.c | 18 ++

Re: btrfs-cleaner 100% busy on an idle filesystem with 4.19.3

2018-11-22 Thread Nikolay Borisov
On 22.11.18 г. 14:31 ч., Tomasz Chmielewski wrote: > Yet another system upgraded to 4.19 and showing strange issues. > > btrfs-cleaner is showing as ~90-100% busy in iotop: > >    TID  PRIO  USER DISK READ  DISK WRITE  SWAPIN IO>    COMMAND >   1340 be/4 root    0.00 B/s    0.00

Re: [PATCH 4/6] btrfs: only track ref_heads in delayed_ref_updates

2018-11-22 Thread Nikolay Borisov
o > matter what the number of items on it. Fix the accounting to only be > adjusted when we add/remove a ref head. LGTM: Reviewed-by: Nikolay Borisov However, what if we kill delayed_ref_updates since the name is a bit ambiguous and instead migrate num_heads_ready from delayed_

Re: [PATCH 3/6] btrfs: cleanup extent_op handling

2018-11-22 Thread Nikolay Borisov
On 21.11.18 г. 20:59 ч., Josef Bacik wrote: > From: Josef Bacik > > The cleanup_extent_op function actually would run the extent_op if it > needed running, which made the name sort of a misnomer. Change it to > run_and_cleanup_extent_op, and move the actual cleanup work to >

Re: [PATCH 1/6] btrfs: add btrfs_delete_ref_head helper

2018-11-22 Thread Nikolay Borisov
On 22.11.18 г. 11:12 ч., Nikolay Borisov wrote: > > > On 21.11.18 г. 20:59 ч., Josef Bacik wrote: >> From: Josef Bacik >> >> We do this dance in cleanup_ref_head and check_ref_cleanup, unify it >> into a helper and cleanup the calling functions. >> >

Re: [PATCH 1/6] btrfs: add btrfs_delete_ref_head helper

2018-11-22 Thread Nikolay Borisov
On 21.11.18 г. 20:59 ч., Josef Bacik wrote: > From: Josef Bacik > > We do this dance in cleanup_ref_head and check_ref_cleanup, unify it > into a helper and cleanup the calling functions. > > Signed-off-by: Josef Bacik > Reviewed-by: Omar Sandoval > --- > fs/btrfs/delayed-ref.c | 14

[PATCH] btrfs: Remove extent_io_ops::readpage_io_failed_hook

2018-11-22 Thread Nikolay Borisov
inode, both of which are guaranteed to have their extent_io_tree::ops set. Signed-off-by: Nikolay Borisov --- fs/btrfs/disk-io.c | 16 -- fs/btrfs/extent_io.c | 69 ++-- fs/btrfs/extent_io.h | 1 - fs/btrfs/inode.c | 7 - 4 files changed

[PATCH] btrfs: Remove unnecessary code from __btrfs_rebalance

2018-11-22 Thread Nikolay Borisov
eal practical benefits to the relocation process. No functional changes. Signed-off-by: Nikolay Borisov Suggested-by: Josef Bacik --- fs/btrfs/volumes.c | 53 -- 1 file changed, 53 deletions(-) diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c

Re: [PATCH] btrfs: only run delayed refs if we're committing

2018-11-21 Thread Nikolay Borisov
Omar Sandoval > Signed-off-by: Josef Bacik Wohooo, we no longer run delayed refs at arbitrary points during transaction commit. Reviewed-by: Nikolay Borisov > --- > fs/btrfs/transaction.c | 24 +--- > 1 file changed, 9 insertions(+), 15 deletions(-) > > d

[PATCHv3] btrfs: Fix error handling in btrfs_cleanup_ordered_extents

2018-11-21 Thread Nikolay Borisov
the current page belongs to the range being instantiated and if so adjsut the range parameters passed for cleaning up. If it doesn't, then just clean the whole OE range directly. Signed-off-by: Nikolay Borisov Reviewed-by: Josef Bacik --- V3: * Re-worded the commit for easier comprehension * Added

Re: [PATCH v2] btrfs: Fix error handling in btrfs_cleanup_ordered_extents

2018-11-20 Thread Nikolay Borisov
On 20.11.18 г. 21:00 ч., Josef Bacik wrote: > On Fri, Oct 26, 2018 at 02:41:55PM +0300, Nikolay Borisov wrote: >> Running btrfs/124 in a loop hung up on me sporadically with the >> following call trace: >> btrfs D0 5760 5324 0x00

Re: [PATCH] btrfs: qgroup: Skip delayed data ref for reloc trees

2018-11-20 Thread Nikolay Borisov
On 20.11.18 г. 11:07 ч., Qu Wenruo wrote: > > > On 2018/11/20 下午4:51, Nikolay Borisov wrote: >> I'm beginning to wonder, should we document >> btrfs_add_delayed_data_ref/btrfs_add_tree_ref arguments separate for >> each function, or should only the differences be d

Re: [PATCH] btrfs: qgroup: Skip delayed data ref for reloc trees

2018-11-20 Thread Nikolay Borisov
On 20.11.18 г. 10:46 ч., Qu Wenruo wrote: > Currently we only check @ref_root in btrfs_add_delayed_data_ref() to > determine whether a data delayed ref is for reloc tree. > > Such check is insufficient as for relocation we could pass @ref_root > as the source file tree, causing qgroup to trace

[PATCH] btrfs: Add sysfs support for metadata_uuid feature

2018-11-19 Thread Nikolay Borisov
. Signed-off-by: Nikolay Borisov --- I completely forgot sysfs also needs to be hooked so here it is. fs/btrfs/sysfs.c | 14 ++ 1 file changed, 14 insertions(+) diff --git a/fs/btrfs/sysfs.c b/fs/btrfs/sysfs.c index 3717c864ba23..5a5930e3d32b 100644 --- a/fs/btrfs/sysfs.c +++ b/fs/btrfs

Re: [PATCH v2] Btrfs: fix deadlock when enabling quotas due to concurrent snapshot creation

2018-11-19 Thread Nikolay Borisov
On 19.11.18 г. 16:48 ч., Qu Wenruo wrote: > There may be some qgroup reserved space related problem in such case, > but I'm not 100% sure to foresee such problem. But why is this a problem - we always queue quota rescan following QUOTA enable, that should take care of proper accounting, no? >

Re: [PATCH] Btrfs: fix deadlock when enabling quotas due to concurrent snapshot creation

2018-11-19 Thread Nikolay Borisov
On 19.11.18 г. 11:48 ч., fdman...@kernel.org wrote: > From: Filipe Manana > > If the quota enable and snapshot creation ioctls are called concurrently > we can get into a deadlock where the task enabling quotas will deadlock > on the fs_info->qgroup_ioctl_lock mutex because it attempts to

  1   2   3   4   5   6   7   8   9   10   >