Re: [PATCH v2 0/6] btrfs: qgroup: Delay subtree scan to reduce overhead

2018-12-08 Thread David Sterba
On Sat, Dec 08, 2018 at 08:50:32AM +0800, Qu Wenruo wrote: > > I've adapted a stress tests that unpacks a large tarball, snaphosts > > every 20 seconds, deletes a random snapshot every 50 seconds, deletes > > file from the original subvolume, now enhanced with qgroups just for the > > new

Re: [PATCH v2 0/6] btrfs: qgroup: Delay subtree scan to reduce overhead

2018-12-07 Thread David Sterba
On Fri, Dec 07, 2018 at 06:51:21AM +0800, Qu Wenruo wrote: > > > On 2018/12/7 上午3:35, David Sterba wrote: > > On Mon, Nov 12, 2018 at 10:33:33PM +0100, David Sterba wrote: > >> On Thu, Nov 08, 2018 at 01:49:12PM +0800, Qu Wenruo wrote: > >>> This patchset can

Re: [PATCH] libbtrfsutil: fix unprivileged tests if kernel lacks support

2018-12-07 Thread David Sterba
On Thu, Dec 06, 2018 at 04:29:32PM -0800, Omar Sandoval wrote: > From: Omar Sandoval > > I apparently didn't test this on a pre-4.18 kernel. > test_subvolume_info_unprivileged() checks for an ENOTTY, but this > doesn't seem to work correctly with subTest(). >

Re: [PATCH v2 0/6] btrfs: qgroup: Delay subtree scan to reduce overhead

2018-12-06 Thread David Sterba
On Mon, Nov 12, 2018 at 10:33:33PM +0100, David Sterba wrote: > On Thu, Nov 08, 2018 at 01:49:12PM +0800, Qu Wenruo wrote: > > This patchset can be fetched from github: > > https://github.com/adam900710/linux/tree/qgroup_delayed_subtree_rebased > > > > Which is base

Re: [PATCH][v2] btrfs: run delayed items before dropping the snapshot

2018-12-06 Thread David Sterba
On Wed, Dec 05, 2018 at 12:12:21PM -0500, Josef Bacik wrote: > From: Josef Bacik > > With my delayed refs patches in place we started seeing a large amount > of aborts in __btrfs_free_extent > > BTRFS error (device sdb1): unable to find ref byte nr 91947008 parent 0 root > 35964 owner 1

Re: [PATCH 1/2] btrfs: catch cow on deleting snapshots

2018-12-06 Thread David Sterba
On Fri, Nov 30, 2018 at 12:19:18PM -0500, Josef Bacik wrote: > On Fri, Nov 30, 2018 at 05:14:54PM +, Filipe Manana wrote: > > On Fri, Nov 30, 2018 at 4:53 PM Josef Bacik wrote: > > > > > > From: Josef Bacik > > > > > > When debugging some weird extent reference bug I suspected that we were >

Re: [PATCH 08/10] btrfs: rework btrfs_check_space_for_delayed_refs

2018-12-06 Thread David Sterba
On Thu, Dec 06, 2018 at 06:52:21PM +0200, Nikolay Borisov wrote: > > > On 3.12.18 г. 17:20 ч., Josef Bacik wrote: > > Now with the delayed_refs_rsv we can now know exactly how much pending > > delayed refs space we need. This means we can drastically simplify > > IMO it will be helpful if

Re: [PATCH 00/10][V2] Delayed refs rsv

2018-12-06 Thread David Sterba
On Mon, Dec 03, 2018 at 10:20:28AM -0500, Josef Bacik wrote: > v1->v2: > - addressed the comments from the various reviewers. > - split "introduce delayed_refs_rsv" into 5 patches. The patches are the same > together as they were, just split out more logically. They can't really be >

Re: [PATCH 0/3] btrfs: use offset_in_page and PAGE_ALIGNED

2018-12-06 Thread David Sterba
On Wed, Dec 05, 2018 at 03:23:02PM +0100, Johannes Thumshirn wrote: > Use the offset_in_page() and PAGE_ALIGNED() macros instead of open-coding them > throughout btrfs. > > This series also includes a patch for 'make coccicheck' which is marked as an > RFC and I've CCed Julia in the hoping to get

Re: [PATCHv3] btrfs: Fix error handling in btrfs_cleanup_ordered_extents

2018-12-05 Thread David Sterba
On Wed, Nov 21, 2018 at 05:10:52PM +0200, Nikolay Borisov wrote: > Running btrfs/124 in a loop hung up on me sporadically with the > following call trace: > btrfs D0 5760 5324 0x > Call Trace: >? __schedule+0x243/0x800 >schedule+0x33/0x90 >

Btrfs progs release 4.19.1

2018-12-05 Thread David Sterba
of typos fixed * warning cleanups * doc formatting updates * CI tests against zstd 1.3.7 Tarballs: https://www.kernel.org/pub/linux/kernel/people/kdave/btrfs-progs/ Git: git://git.kernel.org/pub/scm/linux/kernel/git/kdave/btrfs-progs.git Shortlog: David Sterba (7): btrfs-progs

Re: [PATCH v2 07/13] btrfs-progs: Fix Wmaybe-uninitialized warning

2018-12-05 Thread David Sterba
On Wed, Dec 05, 2018 at 02:40:12PM +0800, Qu Wenruo wrote: > GCC 8.2.1 will report the following warning with "make W=1": > > ctree.c: In function 'btrfs_next_sibling_tree_block': > ctree.c:2990:21: warning: 'slot' may be used uninitialized in this function > [-Wmaybe-uninitialized] >

Re: [PATCH 2/2] btrfs: scrub: move scrub_setup_ctx allocation out of device_list_mutex

2018-12-04 Thread David Sterba
On Tue, Dec 04, 2018 at 05:22:19PM +0200, Nikolay Borisov wrote: > > @@ -3874,16 +3882,9 @@ int btrfs_scrub_dev(struct btrfs_fs_info *fs_info, > > u64 devid, u64 start, > > if (ret) { > > mutex_unlock(_info->scrub_lock); > >

[PATCH 0/2] Scrub allocations vs reclaim fix

2018-12-04 Thread David Sterba
in the generic code so I'm posting only results. The patches are in branch dev/locks-not-held in my devel repos. David Sterba (2): btrfs: scrub: pass fs_info to scrub_setup_ctx btrfs: scrub: move scrub_setup_ctx allocation out of device_list_mutex fs/btrfs/scrub.c | 37

[PATCH 2/2] btrfs: scrub: move scrub_setup_ctx allocation out of device_list_mutex

2018-12-04 Thread David Sterba
cause a lockup. Move the alocation and initialization before any changes that require the mutex. Signed-off-by: David Sterba --- fs/btrfs/scrub.c | 30 ++ 1 file changed, 18 insertions(+), 12 deletions(-) diff --git a/fs/btrfs/scrub.c b/fs/btrfs/scrub.c index

[PATCH 1/2] btrfs: scrub: pass fs_info to scrub_setup_ctx

2018-12-04 Thread David Sterba
We can pass fs_info directly as this is the only member of btrfs_device that's bing used inside scrub_setup_ctx. Signed-off-by: David Sterba --- fs/btrfs/scrub.c | 9 - 1 file changed, 4 insertions(+), 5 deletions(-) diff --git a/fs/btrfs/scrub.c b/fs/btrfs/scrub.c index bbd1b36f4918

Re: [PATCH v4] Btrfs: fix deadlock with memory reclaim during scrub

2018-12-04 Thread David Sterba
On Wed, Nov 28, 2018 at 02:40:07PM +, Filipe Manana wrote: > > > Well, the worker tasks can also not use gfp_kernel, since the scrub > > > task waits for them to complete before pausing. > > > I missed this, and 2 reviewers as well, so perhaps it wasn't that > > > trivial and I shouldn't feel

Re: [PATCH v2 1/3] btrfs: remove always true if branch in find_delalloc_range

2018-12-04 Thread David Sterba
On Thu, Nov 29, 2018 at 11:33:38AM +0800, Lu Fengqi wrote: > The @found is always false when it comes to the if branch. Besides, the > bool type is more suitable for @found. Change the return value of the > function and its caller to bool as well. > > Signed-off-by: Lu Fengqi Added to

Re: [PATCH] btrfs: skip file_extent generation check for free_space_inode in run_delalloc_nocow

2018-12-04 Thread David Sterba
On Thu, Nov 29, 2018 at 05:31:32PM +0800, Lu Fengqi wrote: > The btrfs/001 with inode_cache mount option will encounter the following > warning: > > WARNING: CPU: 1 PID: 23700 at fs/btrfs/inode.c:956 > cow_file_range.isra.19+0x32b/0x430 [btrfs] > CPU: 1 PID: 23700 Comm: btrfs Kdump: loaded

Re: [PATCH v2] Btrfs: fix fsync of files with multiple hard links in new directories

2018-12-04 Thread David Sterba
On Wed, Nov 28, 2018 at 02:54:28PM +, fdman...@kernel.org wrote: > From: Filipe Manana > > The log tree has a long standing problem that when a file is fsync'ed we > only check for new ancestors, created in the current transaction, by > following only the hard link for which the fsync was

Re: [PATCH] btrfs: tree-checker: Don't check max block group size as current max chunk size limit is unreliable

2018-12-04 Thread David Sterba
On Tue, Dec 04, 2018 at 06:15:13PM +0800, Qu Wenruo wrote: > Gentle ping. > > Please put this patch into current release as the new block group size > limit check introduced in v4.19 is causing at least 2 reports in mail list. BTW, if there's an urgent fix or patch that should be considered for

Re: [PATCH] btrfs: tree-checker: Don't check max block group size as current max chunk size limit is unreliable

2018-12-04 Thread David Sterba
On Tue, Dec 04, 2018 at 06:15:13PM +0800, Qu Wenruo wrote: > Gentle ping. > > Please put this patch into current release as the new block group size > limit check introduced in v4.19 is causing at least 2 reports in mail list. I see, on the way to 4.20-rc with stable tags. Thanks.

Re: btrfs development - question about crypto api integration

2018-12-04 Thread David Sterba
On Fri, Nov 30, 2018 at 06:27:58PM +0200, Nikolay Borisov wrote: > > > On 30.11.18 г. 17:22 ч., Chris Mason wrote: > > On 29 Nov 2018, at 12:37, Nikolay Borisov wrote: > > > >> On 29.11.18 г. 18:43 ч., Jean Fobe wrote: > >>> Hi all, > >>> I've been studying LZ4 and other compression

Re: [PATCH 2/5] btrfs: Refactor btrfs_can_relocate

2018-12-04 Thread David Sterba
On Tue, Dec 04, 2018 at 08:34:15AM +0200, Nikolay Borisov wrote: > > > On 3.12.18 г. 19:25 ч., David Sterba wrote: > > On Sat, Nov 17, 2018 at 09:29:27AM +0800, Anand Jain wrote: > >>> - ret = find_free_dev_ext

Re: [PATCH v1.1 9/9] btrfs-progs: Cleanup warning reported by -Wmissing-prototypes

2018-12-04 Thread David Sterba
On Tue, Dec 04, 2018 at 08:24:38PM +0800, Qu Wenruo wrote: > > > On 2018/12/4 下午8:22, David Sterba wrote: > > On Fri, Nov 16, 2018 at 04:04:51PM +0800, Qu Wenruo wrote: > >> The following missing prototypes will be fixed: > >> 1) btrfs.c::handle_special_globa

Re: [PATCH v1.1 9/9] btrfs-progs: Cleanup warning reported by -Wmissing-prototypes

2018-12-04 Thread David Sterba
On Fri, Nov 16, 2018 at 04:04:51PM +0800, Qu Wenruo wrote: > The following missing prototypes will be fixed: > 1) btrfs.c::handle_special_globals() > 2) check/mode-lowmem.c::repair_ternary_lowmem() > 3) extent-tree.c::btrfs_search_overlap_extent() > Above 3 can be fixed by making them

Re: [PATCH 7/9] btrfs-progs: Fix Wmaybe-uninitialized warning

2018-12-04 Thread David Sterba
On Fri, Nov 16, 2018 at 03:54:24PM +0800, Qu Wenruo wrote: > The only location is the following code: > > int level = path->lowest_level + 1; > BUG_ON(path->lowest_level + 1 >= BTRFS_MAX_LEVEL); > while(level < BTRFS_MAX_LEVEL) { > slot = path->slots[level] + 1; >

Re: [PATCH v3 1/2] btrfs: scrub: fix circular locking dependency warning

2018-12-04 Thread David Sterba
On Fri, Nov 30, 2018 at 01:15:23PM +0800, Anand Jain wrote: > @@ -3757,10 +3757,13 @@ static noinline_for_stack int > scrub_workers_get(struct btrfs_fs_info *fs_info, > > static noinline_for_stack void scrub_workers_put(struct btrfs_fs_info > *fs_info) > { > +

Re: [PATCH v1.1 3/9] btrfs-progs: Makefile.extrawarn: Don't warn on sign compare

2018-12-04 Thread David Sterba
On Fri, Nov 16, 2018 at 04:00:40PM +0800, Qu Wenruo wrote: > Under most case, we are just using 'int' for 'unsigned int', and doesn't > care about the sign. > > The Wsign-compare is causing tons of false alerts. > Suppressing it would make W=1 less noisy. > > Signed-off-by: Qu Wenruo > --- >

Re: [PATCH 2/9] btrfs-progs: fix gcc8 default build warning caused by '-Wformat-truncation'

2018-12-04 Thread David Sterba
On Fri, Nov 16, 2018 at 03:54:19PM +0800, Qu Wenruo wrote: > From: Su Yanjun > > When using gcc8 compiles utils.c, it complains as below: > > utils.c:852:45: warning: '%s' directive output may be truncated writing > up to 4095 bytes into a region of size 4084 [-Wformat-truncation=] >

Re: [PATCH 1/9] btrfs-progs: Makefile.extrawarn: Import cc-disable-warning

2018-12-04 Thread David Sterba
On Fri, Nov 16, 2018 at 03:54:18PM +0800, Qu Wenruo wrote: > We imported cc-option but forgot to import cc-disable-warning. > > Fixes: b556a992c3ad ("btrfs-progs: build: allow to build with various > compiler warnings") > Signed-off-by: Qu Wenruo > --- > Makefile.extrawarn | 6 ++ > 1 file

Re: [PATCH v2 0/5] btrfs-progs: image: Fix error when restoring multi-disk image to single disk

2018-12-04 Thread David Sterba
On Tue, Nov 27, 2018 at 04:38:23PM +0800, Qu Wenruo wrote: > This patchset can be fetched from github: > https://github.com/adam900710/btrfs-progs/tree/image_recover > > The base commit is as usual, the latest stable tag, v4.19. > > > Test case misc/021 will fail if using latest upstream

Re: [PATCH v2 1/5] btrfs-progs: image: Refactor fixup_devices() to fixup_chunks_and_devices()

2018-12-04 Thread David Sterba
On Tue, Nov 27, 2018 at 04:38:24PM +0800, Qu Wenruo wrote: > +error: > + error( > +"failed to fix chunks and devices mapping, the fs may not be mountable: %s", > + strerror(-ret)); Recently the sterror(error code) have been converted to %m, so I changed this to errno =

Re: [PATCH v2 1/5] btrfs-progs: image: Refactor fixup_devices() to fixup_chunks_and_devices()

2018-12-04 Thread David Sterba
On Tue, Nov 27, 2018 at 04:50:57PM +0800, Qu Wenruo wrote: > >> -static int fixup_devices(struct btrfs_fs_info *fs_info, > >> - struct mdrestore_struct *mdres, off_t dev_size) > >> +static int fixup_device_size(struct btrfs_trans_handle *trans, > >> + struct

Re: [PATCH] btrfs-progs: trivial fix about line breaker in repair_inode_nbytes_lowmem()

2018-12-03 Thread David Sterba
On Sun, Dec 02, 2018 at 03:08:36PM +, damenly...@gmail.com wrote: > From: Su Yue > > Move "\n" at end of the sentence to print. > > Fixes: 281eec7a9ddf ("btrfs-progs: check: repair inode nbytes in lowmem mode") > Signed-off-by: Su Yue Applied, thanks.

Re: [PATCH] btrfs-progs: fsck-tests: Move reloc tree images to 020-extent-ref-cases

2018-12-03 Thread David Sterba
On Mon, Dec 03, 2018 at 12:39:57PM +0800, Qu Wenruo wrote: > For reloc tree, despite of its short lifespan, it's still the backref, > where reloc tree root backref points back to itself, makes it special. > > So it's more approriate to put them into 020-extent-ref-cases. > > Signed-off-by: Qu

Re: [PATCH 2/5] btrfs: Refactor btrfs_can_relocate

2018-12-03 Thread David Sterba
On Sat, Nov 17, 2018 at 09:29:27AM +0800, Anand Jain wrote: > > - ret = find_free_dev_extent(trans, device, min_free, > > - _offset, NULL); > > - if (!ret) > > + if (!find_free_dev_extent(trans,

Re: [PATCH 0/4] Replace custom device-replace locking with rwsem

2018-12-03 Thread David Sterba
On Tue, Nov 20, 2018 at 01:50:54PM +0100, David Sterba wrote: > The first cleanup part went to 4.19, the actual switch from the custom > locking to rswem was postponed as I found performance degradation. This > turned out to be related to VM cache settings, so I'm resending the >

Btrfs progs pre-release 4.19.1-rc1

2018-12-03 Thread David Sterba
, API docs * other * lot of typos fixed * warning cleanups * doc formatting updates * CI tests against zstd 1.3.7 Tarballs: https://www.kernel.org/pub/linux/kernel/people/kdave/btrfs-progs/ Git: git://git.kernel.org/pub/scm/linux/kernel/git/kdave/btrfs-progs.git Shortlog: David

Re: [PATCH] btrfs: Refactor btrfs_merge_bio_hook

2018-11-29 Thread David Sterba
On Wed, Nov 28, 2018 at 06:51:59PM +0200, Nikolay Borisov wrote: > > Got me curious if we could get rid of the size parameter, it's 2x > > PAGE_SIZE and could be something else in one case but it's not obvious > > if it really happens. > > > > Another thing I noticed is lack of proper error

Re: [PATCH] Btrfs: bring back key search optimization to btrfs_search_old_slot()

2018-11-29 Thread David Sterba
On Mon, Nov 26, 2018 at 04:53:11PM +, Filipe Manana wrote: > On Fri, Nov 16, 2018 at 11:09 AM wrote: > > > > From: Filipe Manana > > > > Commit d7396f07358a ("Btrfs: optimize key searches in btrfs_search_slot"), > > dated from August 2013, introduced an optimization to search for keys in a >

Re: [PATCH 2/2] btrfs: scrub: fix circular locking dependency warning

2018-11-29 Thread David Sterba
On Wed, Nov 28, 2018 at 04:47:27PM +0800, Anand Jain wrote: > > 2. scrub_workers_refcnt must eventually be converted to refcount_t type > > ok. Added in v2 patch set. No such thing is in v2 and this would actually get rid of the need to hold scrub_lock in scrub_workers_put. Which in turn can

Re: [PATCH 2/3] btrfs: wakeup cleaner thread when adding delayed iput

2018-11-28 Thread David Sterba
On Tue, Nov 27, 2018 at 03:08:08PM -0500, Josef Bacik wrote: > On Tue, Nov 27, 2018 at 07:59:42PM +, Chris Mason wrote: > > On 27 Nov 2018, at 14:54, Josef Bacik wrote: > > > > > On Tue, Nov 27, 2018 at 10:26:15AM +0200, Nikolay Borisov wrote: > > >> > > >> > > >> On 21.11.18 г. 21:09 ч.,

Re: [PATCH] btrfs: Remove unnecessary code from __btrfs_rebalance

2018-11-28 Thread David Sterba
On Thu, Nov 22, 2018 at 10:17:33AM +0200, Nikolay Borisov wrote: > The first step fo the rebalance process, ensuring there is 1mb free on > each device. This number seems rather small. And in fact when talking > to the original authors their opinions were: > > "man that's a little bonkers" > "i

Re: [PATCH v2] btrfs: improve error handling of btrfs_add_link()

2018-11-28 Thread David Sterba
On Tue, Nov 27, 2018 at 09:56:43AM +0100, Johannes Thumshirn wrote: > On 26/11/2018 19:37, David Sterba wrote: > > I though a transaction abort would be here, as the state is not > > consistent. Also I'm not sure what I as a user would get from such error > > messa

Re: [PATCH] Fix typos

2018-11-28 Thread David Sterba
On Wed, Nov 28, 2018 at 05:17:09PM +0100, Andrea Gelmini wrote: > On Wed, Nov 28, 2018 at 04:56:20PM +0100, David Sterba wrote: > > Thanks, such patches are accepted once in a while when the amount of new > > typo fixes becomes noticeable. > > About this patch (and the others

Re: [PATCH] btrfs: Refactor btrfs_merge_bio_hook

2018-11-28 Thread David Sterba
used to just remove it. Thirdly, pages are submitted to either > btree or data inodes so it's guaranteed that tree->ops is set so replace > the check with an ASSERT. Finally, document the parameters of the > function. No functional changes. > > Signed-off-by: Nikolay Boris

Re: [PATCH 2/3] btrfs: cleanup the useless DEFINE_WAIT in cleanup_transaction

2018-11-28 Thread David Sterba
On Wed, Nov 28, 2018 at 11:22:54AM +0800, Lu Fengqi wrote: > When it is introduced at commit f094ac32aba3 ("Btrfs: fix NULL pointer > after aborting a transaction"), it's useless. > > Signed-off-by: Lu Fengqi Added to misc-next, thanks.

Re: [PATCH 3/3] btrfs: remove redundant nowait check for buffered_write

2018-11-28 Thread David Sterba
On Wed, Nov 28, 2018 at 09:04:31AM +0200, Nikolay Borisov wrote: > On 28.11.18 г. 5:23 ч., Lu Fengqi wrote: > > The generic_write_checks will check the combination of IOCB_NOWAIT and > > !IOCB_DIRECT. > > True, however btrfs will return ENOSUPP whereas the generic code returns > EINVAL. I guess

Re: [PATCH] Fix typos

2018-11-28 Thread David Sterba
On Wed, Nov 28, 2018 at 12:05:13PM +0100, Andrea Gelmini wrote: > Signed-off-by: Andrea Gelmini > --- > > Stupid fixes. Made on 4.20-rc4, and ported on linux-next (next-20181128). Thanks, such patches are accepted once in a while when the amount of new typo fixes becomes noticeable. >

Re: [PATCH v2 0/3] Misc cosmetic changes for map_private_extent_buffer

2018-11-28 Thread David Sterba
On Wed, Nov 28, 2018 at 09:54:53AM +0100, Johannes Thumshirn wrote: > While trying to understand the checksum code I came across some oddities > regarding map_private_extent_buffer() and it's interaction with > csum_tree_block(). > > These patches address them but are either purely cosmetic or

Re: [PATCH v2 2/3] btrfs: use offset_in_page for start_offset in map_private_extent_buffer()

2018-11-28 Thread David Sterba
On Wed, Nov 28, 2018 at 09:54:55AM +0100, Johannes Thumshirn wrote: > In map_private_extent_buffer() use offset_in_page() to initialize > 'start_offset' instead of open-coding it. Can you please fix all instances where it's opencoded? Grepping for 'PAGE_SIZE - 1' finds a number of them. Thanks.

Re: [PATCH 1/9] btrfs: switch BTRFS_FS_STATE_* to enums

2018-11-28 Thread David Sterba
On Tue, Nov 27, 2018 at 04:24:03PM -0800, Omar Sandoval wrote: > On Tue, Nov 27, 2018 at 08:53:41PM +0100, David Sterba wrote: > > We can use simple enum for values that are not part of on-disk format: > > global filesystem states. > > Reviewed-by: Omar Sandoval &g

Re: [PATCH v4] Btrfs: fix deadlock with memory reclaim during scrub

2018-11-28 Thread David Sterba
On Mon, Nov 26, 2018 at 08:10:30PM +, Filipe Manana wrote: > On Mon, Nov 26, 2018 at 6:17 PM David Sterba wrote: > > > > On Fri, Nov 23, 2018 at 06:25:40PM +, fdman...@kernel.org wrote: > > > From: Filipe Manana > > > > > > When a transaction

Re: [PATCH] btrfs: adjust order of unlocks in do_trimming()

2018-11-28 Thread David Sterba
On Wed, Nov 28, 2018 at 11:21:12AM +0800, Su Yue wrote: > In function do_trimming(), block_group->lock should be unlocked first. Please also write why this is correct and if there are any bad consequences of the current behaviour. Thanks.

Re: [PATCH 0/9] Switch defines to enums

2018-11-28 Thread David Sterba
On Wed, Nov 28, 2018 at 09:33:50AM +0800, Qu Wenruo wrote: > On 2018/11/28 上午3:53, David Sterba wrote: > > This is motivated by a merging mistake that happened a few releases ago. > > Two patches updated BTRFS_FS_* flags independently to the same value, > > git did not

Re: [PATCH 7/9] btrfs: switch BTRFS_*_LOCK to enums

2018-11-28 Thread David Sterba
On Tue, Nov 27, 2018 at 04:37:16PM -0800, Omar Sandoval wrote: > On Tue, Nov 27, 2018 at 08:53:55PM +0100, David Sterba wrote: > > We can use simple enum for values that are not part of on-disk format: > > tree lock types. > > > > Signed-off-by: David Sterba > >

[PATCH 9/9] btrfs: drop extra enum initialization where using defaults

2018-11-27 Thread David Sterba
The first auto-assigned value to enum is 0, we can use that and not initialize all members where the auto-increment does the same. This is used for values that are not part of on-disk format. Signed-off-by: David Sterba --- fs/btrfs/btrfs_inode.h | 2 +- fs/btrfs/ctree.h | 28

[PATCH 4/9] btrfs: switch BTRFS_ROOT_* to enums

2018-11-27 Thread David Sterba
We can use simple enum for values that are not part of on-disk format: root tree flags. Signed-off-by: David Sterba --- fs/btrfs/ctree.h | 33 + 1 file changed, 17 insertions(+), 16 deletions(-) diff --git a/fs/btrfs/ctree.h b/fs/btrfs/ctree.h index 7176b95b40e7

[PATCH 3/9] btrfs: switch BTRFS_FS_* to enums

2018-11-27 Thread David Sterba
We can use simple enum for values that are not part of on-disk format: internal filesystem states. Signed-off-by: David Sterba --- fs/btrfs/ctree.h | 63 1 file changed, 31 insertions(+), 32 deletions(-) diff --git a/fs/btrfs/ctree.h b/fs/btrfs

[PATCH 5/9] btrfs: swtich EXTENT_BUFFER_* to enums

2018-11-27 Thread David Sterba
We can use simple enum for values that are not part of on-disk format: extent buffer flags; Signed-off-by: David Sterba --- fs/btrfs/extent_io.h | 28 1 file changed, 16 insertions(+), 12 deletions(-) diff --git a/fs/btrfs/extent_io.h b/fs/btrfs/extent_io.h index

[PATCH 8/9] btrfs: switch BTRFS_ORDERED_* to enums

2018-11-27 Thread David Sterba
We can use simple enum for values that are not part of on-disk format: ordered extent flags. Signed-off-by: David Sterba --- fs/btrfs/ordered-data.h | 45 +++-- 1 file changed, 25 insertions(+), 20 deletions(-) diff --git a/fs/btrfs/ordered-data.h b/fs/btrfs

[PATCH 7/9] btrfs: switch BTRFS_*_LOCK to enums

2018-11-27 Thread David Sterba
We can use simple enum for values that are not part of on-disk format: tree lock types. Signed-off-by: David Sterba --- fs/btrfs/locking.h | 10 ++ 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/fs/btrfs/locking.h b/fs/btrfs/locking.h index 29135def468e..684d0ef4faa4

[PATCH 6/9] btrfs: switch EXTENT_FLAG_* to enums

2018-11-27 Thread David Sterba
We can use simple enum for values that are not part of on-disk format: extent map flags. Signed-off-by: David Sterba --- fs/btrfs/extent_map.h | 21 ++--- 1 file changed, 14 insertions(+), 7 deletions(-) diff --git a/fs/btrfs/extent_map.h b/fs/btrfs/extent_map.h index

[PATCH 1/9] btrfs: switch BTRFS_FS_STATE_* to enums

2018-11-27 Thread David Sterba
We can use simple enum for values that are not part of on-disk format: global filesystem states. Signed-off-by: David Sterba --- fs/btrfs/ctree.h | 25 +++-- 1 file changed, 19 insertions(+), 6 deletions(-) diff --git a/fs/btrfs/ctree.h b/fs/btrfs/ctree.h index a98507fa9192

[PATCH 2/9] btrfs: switch BTRFS_BLOCK_RSV_* to enums

2018-11-27 Thread David Sterba
We can use simple enum for values that are not part of on-disk format: block reserve types. Signed-off-by: David Sterba --- fs/btrfs/ctree.h | 19 --- 1 file changed, 12 insertions(+), 7 deletions(-) diff --git a/fs/btrfs/ctree.h b/fs/btrfs/ctree.h index f82ec5e41b0c

[PATCH 0/9] Switch defines to enums

2018-11-27 Thread David Sterba
problem would not happen as there would be a conflict and the enum value auto-increment would prevent duplicated values anyway. David Sterba (9): btrfs: switch BTRFS_FS_STATE_* to enums btrfs: switch BTRFS_BLOCK_RSV_* to enums btrfs: switch BTRFS_FS_* to enums btrfs: switch BTRFS_ROOT_

Re: [PATCH 5/6] btrfs: introduce delayed_refs_rsv

2018-11-27 Thread David Sterba
On Mon, Nov 26, 2018 at 11:14:12AM +0200, Nikolay Borisov wrote: > > - if (global_rsv->space_info->full) { > > - num_dirty_bgs_bytes <<= 1; > > - num_bytes <<= 1; > > - } > > + struct btrfs_block_rsv *global_rsv = _info->global_block_rsv; > > + struct btrfs_block_rsv

Re: [PATCH v2] btrfs: improve error handling of btrfs_add_link()

2018-11-26 Thread David Sterba
On Fri, Nov 23, 2018 at 09:42:43AM +0100, Johannes Thumshirn wrote: > err holds the return value of either btrfs_del_root_ref() or > btrfs_del_inode_ref() but it hasn't been checked since it's > introduction with commit fe66a05a0679 (Btrfs: improve error handling > for btrfs_insert_dir_item

Re: [PATCH v4] Btrfs: fix deadlock with memory reclaim during scrub

2018-11-26 Thread David Sterba
before incrementing fs_info->scrubs_running). This hilights why there's perhaps no point in trying to make the nofs section smaller, handling all the interactions between scrub and transaction would be too complex. Reviewed-by: David Sterba

Re: [PATCH 00/10] btrfs-progs: my libbtrfsutil patch queue

2018-11-26 Thread David Sterba
On Tue, Nov 13, 2018 at 11:46:55PM -0800, Omar Sandoval wrote: > From: Omar Sandoval > > Hi, > > This series contains my backlog of libbtrfsutil changes which I've been > collecting over the past few weeks. > > Patches 1-4 are fixes. Patches 5-6 add functionality to the unit tests > which is

Re: [PATCH] btrfs: Remove extent_io_ops::readpage_io_failed_hook

2018-11-26 Thread David Sterba
end_bio_extent_readpage. Also eliminate some needless checks since IO is > always > performed on either data inode or btree inode, both of which are guaranteed > to > have their extent_io_tree::ops set. > > Signed-off-by: Nikolay Borisov Reviewed-by: David Sterba

Re: [PATCH 0/6] Delayed refs rsv

2018-11-23 Thread David Sterba
On Wed, Nov 21, 2018 at 01:59:06PM -0500, Josef Bacik wrote: > This patchset changes how we do space reservations for delayed refs. We were > hitting probably 20-40 enospc abort's per day in production while running > delayed refs at transaction commit time. This means we ran out of space in >

Re: [PATCH 3/6] btrfs: cleanup extent_op handling

2018-11-23 Thread David Sterba
On Wed, Nov 21, 2018 at 01:59:09PM -0500, Josef Bacik wrote: > From: Josef Bacik > > The cleanup_extent_op function actually would run the extent_op if it > needed running, which made the name sort of a misnomer. Change it to > run_and_cleanup_extent_op, and move the actual cleanup work to >

Re: [PATCH 2/6] btrfs: add cleanup_ref_head_accounting helper

2018-11-23 Thread David Sterba
On Thu, Nov 22, 2018 at 09:06:30AM +0800, Qu Wenruo wrote: > > > On 2018/11/22 上午2:59, Josef Bacik wrote: > > From: Josef Bacik > > > > We were missing some quota cleanups in check_ref_cleanup, so break the > > ref head accounting cleanup into a helper and call that from both > >

Re: [PATCH 1/6] btrfs: add btrfs_delete_ref_head helper

2018-11-23 Thread David Sterba
On Thu, Nov 22, 2018 at 11:42:28AM +0200, Nikolay Borisov wrote: > > > On 22.11.18 г. 11:12 ч., Nikolay Borisov wrote: > > > > > > On 21.11.18 г. 20:59 ч., Josef Bacik wrote: > >> From: Josef Bacik > >> > >> We do this dance in cleanup_ref_head and check_ref_cleanup, unify it > >> into a

Re: [PATCH] btrfs: remove btrfs_bio_end_io_t

2018-11-23 Thread David Sterba
On Fri, Nov 23, 2018 at 09:42:27AM +0100, Johannes Thumshirn wrote: > The btrfs_bio_end_io_t typedef was introduced with commit > a1d3c4786a4b ("btrfs: btrfs_multi_bio replaced with btrfs_bio") > but never used anywhere. This commit also introduced a forward declaration > of 'struct btrfs_bio'

[PATCH 2/4] btrfs: replace async_cow::root with fs_info

2018-11-22 Thread David Sterba
The async_cow::root is used to propagate fs_info to async_cow_submit. We can't use inode to reach it because it could become NULL after write without compression in async_cow_start. Signed-off-by: David Sterba --- fs/btrfs/inode.c | 9 +++-- 1 file changed, 3 insertions(+), 6 deletions

[PATCH 3/4] btrfs: remove redundant csum buffer in btrfs_io_bio

2018-11-22 Thread David Sterba
. This shrinks struct btrfs_io_bio by 8 bytes. Signed-off-by: David Sterba --- fs/btrfs/file-item.c | 12 +++- fs/btrfs/volumes.h | 1 - 2 files changed, 7 insertions(+), 6 deletions(-) diff --git a/fs/btrfs/file-item.c b/fs/btrfs/file-item.c index ba74827beb32..1f2d0a6ab634 100644 --- a/fs

[PATCH 4/4] btrfs: replace btrfs_io_bio::end_io with a simple helper

2018-11-22 Thread David Sterba
in the right case. This shrinks struct btrfs_io_bio by 8 bytes. Signed-off-by: David Sterba --- fs/btrfs/extent_io.c | 3 +-- fs/btrfs/file-item.c | 9 - fs/btrfs/inode.c | 7 ++- fs/btrfs/volumes.h | 10 -- 4 files changed, 11 insertions(+), 18 deletions(-) diff --git a/fs

[PATCH 0/4] Minor helper and struct member cleanups

2018-11-22 Thread David Sterba
Assorted updates to code vaguely related to the bio callbacks and structures. Remove call indirection, remove redundant struct members, etc. David Sterba (4): btrfs: merge btrfs_submit_bio_done to its caller btrfs: replace async_cow::root with fs_info btrfs: remove redundant csum buffer

[PATCH 1/4] btrfs: merge btrfs_submit_bio_done to its caller

2018-11-22 Thread David Sterba
There's one caller and its code is simple, we can open code it in run_one_async_done. The errors are passed through bio. Signed-off-by: David Sterba --- fs/btrfs/disk-io.c | 18 +- fs/btrfs/inode.c | 23 --- 2 files changed, 17 insertions(+), 24 deletions

Re: [PATCH v2] Btrfs: fix deadlock when enabling quotas due to concurrent snapshot creation

2018-11-22 Thread David Sterba
On Thu, Nov 22, 2018 at 09:46:44PM +0800, Qu Wenruo wrote: > >>> it and after > >>> we started (or joined) a transaction, a lot could of modifications may > >>> have happened. > >>> Nevertheless I don't think it's unexpected for anyone to have the > >>> accounting happening > >>> only after the

Re: [PATCH] btrfs: improve error handling of btrfs_add_link()

2018-11-22 Thread David Sterba
On Thu, Nov 22, 2018 at 02:44:35PM +0100, Johannes Thumshirn wrote: > On 22/11/2018 14:35, David Sterba wrote: > > On Thu, Nov 22, 2018 at 02:15:28PM +0100, Johannes Thumshirn wrote: > >> err holds the return value of either btrfs_del_root_ref() or > >> btrfs_del_ino

Re: [PATCH] btrfs: improve error handling of btrfs_add_link()

2018-11-22 Thread David Sterba
On Thu, Nov 22, 2018 at 02:15:28PM +0100, Johannes Thumshirn wrote: > err holds the return value of either btrfs_del_root_ref() or > btrfs_del_inode_ref() but it hasn't been checked since it's > introduction with commit fe66a05a0679 (Btrfs: improve error handling > for btrfs_insert_dir_item

Re: [PATCH v2] Btrfs: fix deadlock when enabling quotas due to concurrent snapshot creation

2018-11-22 Thread David Sterba
On Tue, Nov 20, 2018 at 08:32:42AM +0800, Qu Wenruo wrote: > >>> @@ -1013,16 +1013,22 @@ int btrfs_quota_enable(struct btrfs_fs_info > >>> *fs_info) > >>> btrfs_abort_transaction(trans, ret); > >>> goto out_free_path; > >>> } > >>> -

Re: [PATCH] Btrfs: fix access to available allocation bits when starting balance

2018-11-21 Thread David Sterba
On Mon, Nov 19, 2018 at 09:48:12AM +, fdman...@kernel.org wrote: > From: Filipe Manana > > The available allocation bits members from struct btrfs_fs_info are > protected by a sequence lock, and when starting balance we access them > incorrectly in two different ways: > > 1) In the read

Re: [PATCH] Btrfs: fix race between enabling quotas and subvolume creation

2018-11-21 Thread David Sterba
qgroup_ioctl_lock >-> ioctl returns -EINVAL > > Returning -EINVAL to user space will be confusing if all the arguments > passed to the subvolume creation ioctl were valid. > > Fix it by grabbing the value from fs_info->quota_root after acquiring > the mutex. > > Signed-off-by: Filipe Manana Reviewed-by: David Sterba Added to misc-next, thanks.

Re: [PATCH v6 0/3] btrfs: balance: improve kernel logs

2018-11-20 Thread David Sterba
On Tue, Nov 20, 2018 at 04:12:54PM +0800, Anand Jain wrote: > v5->v6: > Mostly the defines non functional changes. > Use goto instead of return in middle of the define. > Pls ref individual patches 1/3 and 2/3 for more info. > > v4->v5: > Mainly address David review comment [1]. > [1] >

Re: [PATCH RESEND 0/9 v2] fix warn_on for replace cancel

2018-11-20 Thread David Sterba
On Tue, Nov 20, 2018 at 07:56:14PM +0800, Anand Jain wrote: > These two patches were sent as part of >[PATCH 0/9 v2] fix replace-start and replace-cancel racing > before but as these aren't integrated so I am sending these again. > > The patch [1] which is in misc-next, calls

Re: [PATCH RESEND v2 1/2] btrfs: quieten warn if replace is canceled at finish

2018-11-20 Thread David Sterba
ent function. As of now only user can cancel the replace-scrub, > so its ok to quieten the warn here. > > Signed-off-by: Anand Jain Ok for getting rid if the ECANCELED warning, though it would be better to replace the WARN_ON too. Reviewed-by: David Sterba

[PATCH 4/4] btrfs: dev-replace: open code trivial locking helpers

2018-11-20 Thread David Sterba
The dev-replace locking functions are now trivial wrappers around rw semaphore that can be used directly everywhere. No functional change. Signed-off-by: David Sterba --- fs/btrfs/dev-replace.c | 81 -- fs/btrfs/dev-replace.h | 4 --- fs/btrfs/reada.c

[PATCH 1/4] btrfs: reada: reorder dev-replace locks before radix tree preload

2018-11-20 Thread David Sterba
The read lock is going to use rw semaphore that might sleep, this is not possible in the radix tree preload section. The lock nesting is now: * device replace * radix tree preload * readahead spinlock Signed-off-by: David Sterba --- fs/btrfs/reada.c | 12 +++- 1 file changed, 7

[PATCH 2/4] btrfs: dev-replace: swich locking to rw semaphore

2018-11-20 Thread David Sterba
-by: David Sterba --- fs/btrfs/ctree.h | 2 +- fs/btrfs/dev-replace.c | 12 ++-- fs/btrfs/disk-io.c | 2 +- 3 files changed, 8 insertions(+), 8 deletions(-) diff --git a/fs/btrfs/ctree.h b/fs/btrfs/ctree.h index b58c5e73e458..01efe3849968 100644 --- a/fs/btrfs/ctree.h +++ b/fs

[PATCH 0/4] Replace custom device-replace locking with rwsem

2018-11-20 Thread David Sterba
, waitqueues, which essentially is what the readwrite semaphore does. Previous patchset: https://lore.kernel.org/linux-btrfs/cover.1536331604.git.dste...@suse.com/ Patches correspond to 8/11-11/11 and there's no change besides refreshing on top of current misc-next. David Sterba (4): btrfs: reada

[PATCH 3/4] btrfs: dev-replace: remove custom read/write blocking scheme

2018-11-20 Thread David Sterba
that waited until ::blocking_readers was zero. Signed-off-by: David Sterba --- fs/btrfs/ctree.h | 2 -- fs/btrfs/dev-replace.c | 16 fs/btrfs/dev-replace.h | 1 - fs/btrfs/disk-io.c | 2 -- fs/btrfs/volumes.c | 13 ++--- 5 files changed, 6 insertions(+), 28

Re: [PATCH v5 1/3] btrfs: add helper function describe_block_group()

2018-11-19 Thread David Sterba
On Wed, Nov 14, 2018 at 09:17:10PM +0800, Anand Jain wrote: > Improve on describe_relocation() add a common helper function to describe > the block groups. > > Signed-off-by: Anand Jain > Reviewed-by: David Sterba > --- > v4.1->v5: Initialize buf[128] to null. > v4-&g

Re: [PATCH v5 2/3] btrfs: balance: add args info during start and resume

2018-11-19 Thread David Sterba
On Wed, Nov 14, 2018 at 09:17:11PM +0800, Anand Jain wrote: > Balance arg info is an important information to be reviewed for the > system audit. So this patch adds them to the kernel log. > > Example: > ->btrfs bal start -f -mprofiles=raid1,convert=single,soft > -dlimit=10..20,usage=50 /btrfs >

Re: [PATCH] btrfs: Add sysfs support for metadata_uuid feature

2018-11-19 Thread David Sterba
s/btrfs/UUID/metadata_uuid attribute which allows one to read > the current metadata_uuid. > > Signed-off-by: Nikolay Borisov > --- > > I completely forgot sysfs also needs to be hooked so here it is. Thanks, Reviewed-by: David Sterba I'll add it right after the commit that introduces the METADATA_UUID bit.

Re: [PATCH v2 0/5] btrfs: fix compiler warning with make W=1

2018-11-19 Thread David Sterba
On Mon, Nov 19, 2018 at 10:38:12AM +0100, Johannes Thumshirn wrote: > This patchset fixes most of the compiler warnings encountered when building > btrfs with make W=1. > > There are two more compiler warnings left in raid56.c: > CC [M] fs/btrfs/raid56.o > fs/btrfs/raid56.c: In function

  1   2   3   4   5   6   7   8   9   10   >