[PATCH v2] Btrfs: fix fsync of files with multiple hard links in new directories

2018-11-28 Thread fdmanana
From: Filipe Manana The log tree has a long standing problem that when a file is fsync'ed we only check for new ancestors, created in the current transaction, by following only the hard link for which the fsync was issued. We follow the ancestors using the VFS' dget_parent() API. This means that

Re: [PATCH v4] Btrfs: fix deadlock with memory reclaim during scrub

2018-11-28 Thread Filipe Manana
On Wed, Nov 28, 2018 at 2:22 PM David Sterba wrote: > > On Mon, Nov 26, 2018 at 08:10:30PM +, Filipe Manana wrote: > > On Mon, Nov 26, 2018 at 6:17 PM David Sterba wrote: > > > > > > On Fri, Nov 23, 2018 at 06:25:40PM +, fdman...@kernel.org wrote: > > > > From: Filipe Manana > > > > > >

[PATCH] Btrfs: fix fsync of files with multiple hard links in new directories

2018-11-28 Thread fdmanana
From: Filipe Manana The log tree has a long standing problem that when a file is fsync'ed we only check for new ancestors, created in the current transaction, by following only the hard link for which the fsync was issued. We follow the ancestors using the VFS' dget_parent() API. This means that

Re: [PATCH v4] Btrfs: fix deadlock with memory reclaim during scrub

2018-11-28 Thread David Sterba
On Mon, Nov 26, 2018 at 08:10:30PM +, Filipe Manana wrote: > On Mon, Nov 26, 2018 at 6:17 PM David Sterba wrote: > > > > On Fri, Nov 23, 2018 at 06:25:40PM +, fdman...@kernel.org wrote: > > > From: Filipe Manana > > > > > > When a transaction commit starts, it attempts to pause scrub and

Re: [PATCH] btrfs: adjust order of unlocks in do_trimming()

2018-11-28 Thread David Sterba
On Wed, Nov 28, 2018 at 11:21:12AM +0800, Su Yue wrote: > In function do_trimming(), block_group->lock should be unlocked first. Please also write why this is correct and if there are any bad consequences of the current behaviour. Thanks.

Re: Balance: invalid convert data profile raid10

2018-11-28 Thread Qu Wenruo
On 2018/11/28 下午3:20, Mikko Merikivi wrote: > Well, excuse me for thinking it wouldn't since in md-raid it worked. > https://wiki.archlinux.org/index.php/RAID#RAID_level_comparison > > Anyway, the error message is truly confusing for someone who doesn't > know about btrfs's implementation. I

Re: [PATCH 0/9] Switch defines to enums

2018-11-28 Thread Qu Wenruo
On 2018/11/28 下午9:25, David Sterba wrote: > On Wed, Nov 28, 2018 at 09:33:50AM +0800, Qu Wenruo wrote: >> On 2018/11/28 上午3:53, David Sterba wrote: >>> This is motivated by a merging mistake that happened a few releases ago. >>> Two patches updated BTRFS_FS_* flags independently to the same

Re: [PATCH] Fix typos

2018-11-28 Thread Nikolay Borisov
On 28.11.18 г. 15:24 ч., Brendan Hide wrote: > > > On 11/28/18 1:23 PM, Nikolay Borisov wrote: >> >> >> On 28.11.18 г. 13:05 ч., Andrea Gelmini wrote: >>> Signed-off-by: Andrea Gelmini >>> --- > > > >>> >>> diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c >>> index

Re: [PATCH] Fix typos

2018-11-28 Thread Brendan Hide
On 11/28/18 1:23 PM, Nikolay Borisov wrote: On 28.11.18 г. 13:05 ч., Andrea Gelmini wrote: Signed-off-by: Andrea Gelmini --- diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c index bab2f1983c07..babbd75d91d2 100644 --- a/fs/btrfs/inode.c +++ b/fs/btrfs/inode.c @@ -104,7 +104,7 @@

Re: [PATCH 3/3] btrfs: remove redundant nowait check for buffered_write

2018-11-28 Thread Johannes Thumshirn
Reviewed-by: Johannes Thumshirn -- Johannes ThumshirnSUSE Labs Filesystems jthumsh...@suse.de+49 911 74053 689 SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg GF: Felix Imendörffer, Jane Smithard, Graham Norton HRB 21284 (AG Nürnberg)

Re: [PATCH 2/3] btrfs: cleanup the useless DEFINE_WAIT in cleanup_transaction

2018-11-28 Thread Johannes Thumshirn
Reviewed-by: Johannes Thumshirn -- Johannes ThumshirnSUSE Labs Filesystems jthumsh...@suse.de+49 911 74053 689 SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg GF: Felix Imendörffer, Jane Smithard, Graham Norton HRB 21284 (AG Nürnberg)

Re: [PATCH 9/9] btrfs: drop extra enum initialization where using defaults

2018-11-28 Thread Johannes Thumshirn
Reviewed-by: Johannes Thumshirn -- Johannes ThumshirnSUSE Labs Filesystems jthumsh...@suse.de+49 911 74053 689 SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg GF: Felix Imendörffer, Jane Smithard, Graham Norton HRB 21284 (AG Nürnberg)

Re: [PATCH 8/9] btrfs: switch BTRFS_ORDERED_* to enums

2018-11-28 Thread Johannes Thumshirn
Reviewed-by: Johannes Thumshirn -- Johannes ThumshirnSUSE Labs Filesystems jthumsh...@suse.de+49 911 74053 689 SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg GF: Felix Imendörffer, Jane Smithard, Graham Norton HRB 21284 (AG Nürnberg)

Re: [PATCH 6/9] btrfs: switch EXTENT_FLAG_* to enums

2018-11-28 Thread Johannes Thumshirn
Reviewed-by: Johannes Thumshirn -- Johannes ThumshirnSUSE Labs Filesystems jthumsh...@suse.de+49 911 74053 689 SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg GF: Felix Imendörffer, Jane Smithard, Graham Norton HRB 21284 (AG Nürnberg)

Re: [PATCH 0/9] Switch defines to enums

2018-11-28 Thread David Sterba
On Wed, Nov 28, 2018 at 09:33:50AM +0800, Qu Wenruo wrote: > On 2018/11/28 上午3:53, David Sterba wrote: > > This is motivated by a merging mistake that happened a few releases ago. > > Two patches updated BTRFS_FS_* flags independently to the same value, > > git did not see any direct merge

Re: [PATCH 7/9] btrfs: switch BTRFS_*_LOCK to enums

2018-11-28 Thread David Sterba
On Tue, Nov 27, 2018 at 04:37:16PM -0800, Omar Sandoval wrote: > On Tue, Nov 27, 2018 at 08:53:55PM +0100, David Sterba wrote: > > We can use simple enum for values that are not part of on-disk format: > > tree lock types. > > > > Signed-off-by: David Sterba > > --- > > fs/btrfs/locking.h | 10

Re: [PATCH 5/9] btrfs: swtich EXTENT_BUFFER_* to enums

2018-11-28 Thread Johannes Thumshirn
Reviewed-by: Johannes Thumshirn -- Johannes ThumshirnSUSE Labs Filesystems jthumsh...@suse.de+49 911 74053 689 SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg GF: Felix Imendörffer, Jane Smithard, Graham Norton HRB 21284 (AG Nürnberg)

Re: [PATCH 4/9] btrfs: switch BTRFS_ROOT_* to enums

2018-11-28 Thread Johannes Thumshirn
Looks good, Reviewed-by: Johannes Thumshirn -- Johannes ThumshirnSUSE Labs Filesystems jthumsh...@suse.de+49 911 74053 689 SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg GF: Felix Imendörffer, Jane Smithard, Graham Norton HRB 21284 (AG

Re: [PATCH 2/9] btrfs: switch BTRFS_BLOCK_RSV_* to enums

2018-11-28 Thread Johannes Thumshirn
Looks good, Reviewed-by: Johannes Thumshirn -- Johannes ThumshirnSUSE Labs jthumsh...@suse.de+49 911 74053 689 SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg GF: Felix Imendörffer, Jane Smithard, Graham Norton HRB 21284 (AG

Re: [PATCH 1/9] btrfs: switch BTRFS_FS_STATE_* to enums

2018-11-28 Thread Johannes Thumshirn
Looks good, Reviewed-by: Johannes Thumshirn -- Johannes ThumshirnSUSE Labs jthumsh...@suse.de+49 911 74053 689 SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg GF: Felix Imendörffer, Jane Smithard, Graham Norton HRB 21284 (AG

Re: [PATCH] Fix typos

2018-11-28 Thread Nikolay Borisov
On 28.11.18 г. 13:05 ч., Andrea Gelmini wrote: > Signed-off-by: Andrea Gelmini > --- > > Stupid fixes. Made on 4.20-rc4, and ported on linux-next (next-20181128). > > > fs/btrfs/backref.c | 4 ++-- > fs/btrfs/check-integrity.c | 2 +- > fs/btrfs/compression.c | 4 ++-- >

[PATCH] Fix typos

2018-11-28 Thread Andrea Gelmini
Signed-off-by: Andrea Gelmini --- Stupid fixes. Made on 4.20-rc4, and ported on linux-next (next-20181128). fs/btrfs/backref.c | 4 ++-- fs/btrfs/check-integrity.c | 2 +- fs/btrfs/compression.c | 4 ++-- fs/btrfs/ctree.c | 4 ++-- fs/btrfs/dev-replace.c | 2 +-

Re: [RFC PATCH] btrfs: drop file privileges in btrfs_clone_files

2018-11-28 Thread Filipe Manana
On Wed, Nov 28, 2018 at 9:26 AM Lu Fengqi wrote: > > On Wed, Nov 28, 2018 at 09:48:07AM +0200, Nikolay Borisov wrote: > > > > > >On 28.11.18 г. 9:46 ч., Christoph Hellwig wrote: > >> On Wed, Nov 28, 2018 at 09:44:59AM +0200, Nikolay Borisov wrote: > >>> > >>> > >>> On 28.11.18 г. 5:07 ч., Lu

Re: [PATCH 1/3] btrfs: remove always true if branch in find_delalloc_range

2018-11-28 Thread Lu Fengqi
On Wed, Nov 28, 2018 at 09:01:42AM +0200, Nikolay Borisov wrote: > > >On 28.11.18 г. 5:21 ч., Lu Fengqi wrote: >> The @found is always false when it comes to the if branch. Besides, the >> bool type is more suitable for @found. > >Well if you are ranging the type of found variable it also makes

Re: [RFC PATCH] btrfs: drop file privileges in btrfs_clone_files

2018-11-28 Thread Lu Fengqi
On Wed, Nov 28, 2018 at 09:48:07AM +0200, Nikolay Borisov wrote: > > >On 28.11.18 г. 9:46 ч., Christoph Hellwig wrote: >> On Wed, Nov 28, 2018 at 09:44:59AM +0200, Nikolay Borisov wrote: >>> >>> >>> On 28.11.18 г. 5:07 ч., Lu Fengqi wrote: The generic/513 tell that cloning into a file did not

Re: [RFC PATCH 01/17] btrfs: priority alloc: prepare of priority aware allocator

2018-11-28 Thread Su Yue
On 11/28/18 4:24 PM, Nikolay Borisov wrote: On 28.11.18 г. 5:11 ч., Su Yue wrote: To implement priority aware allocator, this patch: Introduces struct btrfs_priority_tree which contains block groups in same level. Adds member priority to struct btrfs_block_group_cache and pointer points to

Re: [PATCH v2 3/3] btrfs: document extent mapping assumptions in checksum

2018-11-28 Thread Nikolay Borisov
On 28.11.18 г. 10:54 ч., Johannes Thumshirn wrote: > Document why map_private_extent_buffer() cannot return '1' (i.e. the map > spans two pages) for the csum_tree_block() case. > > The current algorithm for detecting a page boundary crossing in > map_private_extent_buffer() will return a '1'

Re: [RFC PATCH 03/17] btrfs: priority alloc: introduce compute_block_group_priority/usage

2018-11-28 Thread Nikolay Borisov
On 28.11.18 г. 5:11 ч., Su Yue wrote: > Introduce compute_block_group_usage() and compute_block_group_usage(). > And call the latter in btrfs_make_block_group() and > btrfs_read_block_groups(). > > compute_priority_level use ilog2(free) to compute priority level. > > Signed-off-by: Su Yue >

[PATCH v2 3/3] btrfs: document extent mapping assumptions in checksum

2018-11-28 Thread Johannes Thumshirn
Document why map_private_extent_buffer() cannot return '1' (i.e. the map spans two pages) for the csum_tree_block() case. The current algorithm for detecting a page boundary crossing in map_private_extent_buffer() will return a '1' *IFF* the extent buffer's offset in the page + the offset passed

[PATCH v2 1/3] btrfs: don't initialize 'offset' in map_private_extent_buffer()

2018-11-28 Thread Johannes Thumshirn
In map_private_extent_buffer() the 'offset' variable is initialized to a page aligned version of the 'start' parameter. But later on it is overwritten with either the offset from the extent buffer's start or 0. So get rid of the initial initialization. Signed-off-by: Johannes Thumshirn

[PATCH v2 2/3] btrfs: use offset_in_page for start_offset in map_private_extent_buffer()

2018-11-28 Thread Johannes Thumshirn
In map_private_extent_buffer() use offset_in_page() to initialize 'start_offset' instead of open-coding it. Signed-off-by: Johannes Thumshirn Reviewed-by: Nikolay Borisov --- fs/btrfs/extent_io.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/fs/btrfs/extent_io.c

[PATCH v2 0/3] Misc cosmetic changes for map_private_extent_buffer

2018-11-28 Thread Johannes Thumshirn
While trying to understand the checksum code I came across some oddities regarding map_private_extent_buffer() and it's interaction with csum_tree_block(). These patches address them but are either purely cosmetic or only add a comment documenting behaviour. Changes to v1: * Add Reviewed-by tags

Re: [PATCH 2/2] btrfs: scrub: fix circular locking dependency warning

2018-11-28 Thread Anand Jain
On 11/26/2018 05:59 PM, Nikolay Borisov wrote: On 26.11.18 г. 11:07 ч., Anand Jain wrote: Circular locking dependency check reports warning[1], that's because the btrfs_scrub_dev() calls the stack #0 below with, the fs_info::scrub_lock held. The test case leading to this warning..

Re: [PATCH 3/3] btrfs: document extent mapping assumptions in checksum

2018-11-28 Thread Johannes Thumshirn
On 27/11/2018 17:36, Nikolay Borisov wrote: > > > On 27.11.18 г. 18:00 ч., Johannes Thumshirn wrote: >> Document why map_private_extent_buffer() cannot return '1' (i.e. the map >> spans two pages) for the csum_tree_block() case. >> >> The current algorithm for detecting a page boundary crossing

Re: [RFC PATCH 01/17] btrfs: priority alloc: prepare of priority aware allocator

2018-11-28 Thread Nikolay Borisov
On 28.11.18 г. 5:11 ч., Su Yue wrote: > To implement priority aware allocator, this patch: > Introduces struct btrfs_priority_tree which contains block groups > in same level. > Adds member priority to struct btrfs_block_group_cache and pointer > points to the priority tree it's located. > >

Re: [PATCH 5/8] btrfs: don't enospc all tickets on flush failure

2018-11-28 Thread Nikolay Borisov
On 27.11.18 г. 21:46 ч., Josef Bacik wrote: > On Mon, Nov 26, 2018 at 02:25:52PM +0200, Nikolay Borisov wrote: >> >> >> On 21.11.18 г. 21:03 ч., Josef Bacik wrote: >>> With the introduction of the per-inode block_rsv it became possible to >>> have really really large reservation requests made

Re: [RFC PATCH] btrfs: drop file privileges in btrfs_clone_files

2018-11-27 Thread Nikolay Borisov
On 28.11.18 г. 9:46 ч., Christoph Hellwig wrote: > On Wed, Nov 28, 2018 at 09:44:59AM +0200, Nikolay Borisov wrote: >> >> >> On 28.11.18 г. 5:07 ч., Lu Fengqi wrote: >>> The generic/513 tell that cloning into a file did not strip security >>> privileges (suid, capabilities) like a regular write

Re: [RFC PATCH] btrfs: drop file privileges in btrfs_clone_files

2018-11-27 Thread Christoph Hellwig
On Wed, Nov 28, 2018 at 09:44:59AM +0200, Nikolay Borisov wrote: > > > On 28.11.18 г. 5:07 ч., Lu Fengqi wrote: > > The generic/513 tell that cloning into a file did not strip security > > privileges (suid, capabilities) like a regular write would. > > > > Signed-off-by: Lu Fengqi > > --- > >

Re: [RFC PATCH] btrfs: drop file privileges in btrfs_clone_files

2018-11-27 Thread Nikolay Borisov
On 28.11.18 г. 5:07 ч., Lu Fengqi wrote: > The generic/513 tell that cloning into a file did not strip security > privileges (suid, capabilities) like a regular write would. > > Signed-off-by: Lu Fengqi > --- > The xfs and ocfs2 call generic_remap_file_range_prep to drop file > privileges,

Re: Balance: invalid convert data profile raid10

2018-11-27 Thread Mikko Merikivi
Well, excuse me for thinking it wouldn't since in md-raid it worked. https://wiki.archlinux.org/index.php/RAID#RAID_level_comparison Anyway, the error message is truly confusing for someone who doesn't know about btrfs's implementation. I suppose in md-raid the near layout is actually RAID 1 and

Re: [PATCH] btrfs: adjust order of unlocks in do_trimming()

2018-11-27 Thread Nikolay Borisov
On 28.11.18 г. 5:21 ч., Su Yue wrote: > In function do_trimming(), block_group->lock should be unlocked first. > > Fixes: 7fe1e6415026 ("Btrfs: rewrite btrfs_trim_block_group()") > Signed-off-by: Su Yue Reviewed-by: Nikolay Borisov > --- > fs/btrfs/free-space-cache.c | 2 +- > 1 file

Re: [PATCH 3/3] btrfs: remove redundant nowait check for buffered_write

2018-11-27 Thread Nikolay Borisov
On 28.11.18 г. 5:23 ч., Lu Fengqi wrote: > The generic_write_checks will check the combination of IOCB_NOWAIT and > !IOCB_DIRECT. True, however btrfs will return ENOSUPP whereas the generic code returns EINVAL. I guess this is not a big deal and it's likely generic code is correct, so:

Re: [PATCH 1/3] btrfs: remove always true if branch in find_delalloc_range

2018-11-27 Thread Nikolay Borisov
On 28.11.18 г. 5:21 ч., Lu Fengqi wrote: > The @found is always false when it comes to the if branch. Besides, the > bool type is more suitable for @found. Well if you are ranging the type of found variable it also makes sense to change the return value of the function to bool as well. > >

Re: [PATCH 2/3] btrfs: cleanup the useless DEFINE_WAIT in cleanup_transaction

2018-11-27 Thread Nikolay Borisov
On 28.11.18 г. 5:22 ч., Lu Fengqi wrote: > When it is introduced at commit f094ac32aba3 ("Btrfs: fix NULL pointer > after aborting a transaction"), it's useless. > > Signed-off-by: Lu Fengqi Reviewed-by: Nikolay Borisov > --- > fs/btrfs/transaction.c | 1 - > 1 file changed, 1 deletion(-)

[PATCH] btrfs: qgroup: Introduce more accurate early reloc tree detection

2018-11-27 Thread Qu Wenruo
The biggest challenge for qgroup to skip reloc tree extents is to detect correct owner of reloc tree blocks owner. Unlike most data operations, the root of tree reloc tree can't be easily detected. For example, for relocation we call btrfs_copy_root to init reloc tree: btrfs_copy_root(root=257,

Re: [RFC PATCH 00/17] btrfs: implementation of priority aware allocator

2018-11-27 Thread Qu Wenruo
On 2018/11/28 上午11:11, Su Yue wrote: > This patchset can be fetched from repo: > https://github.com/Damenly/btrfs-devel/commits/priority_aware_allocator. > Since patchset 'btrfs: Refactor find_free_extent()' does a nice work > to simplify find_free_extent(). This patchset dependents on the

[PATCH 3/3] btrfs: remove redundant nowait check for buffered_write

2018-11-27 Thread Lu Fengqi
The generic_write_checks will check the combination of IOCB_NOWAIT and !IOCB_DIRECT. Signed-off-by: Lu Fengqi --- fs/btrfs/file.c | 4 1 file changed, 4 deletions(-) diff --git a/fs/btrfs/file.c b/fs/btrfs/file.c index 3835bb8c146d..190db9a685a2 100644 --- a/fs/btrfs/file.c +++

[PATCH 2/3] btrfs: cleanup the useless DEFINE_WAIT in cleanup_transaction

2018-11-27 Thread Lu Fengqi
When it is introduced at commit f094ac32aba3 ("Btrfs: fix NULL pointer after aborting a transaction"), it's useless. Signed-off-by: Lu Fengqi --- fs/btrfs/transaction.c | 1 - 1 file changed, 1 deletion(-) diff --git a/fs/btrfs/transaction.c b/fs/btrfs/transaction.c index

[PATCH 1/3] btrfs: remove always true if branch in find_delalloc_range

2018-11-27 Thread Lu Fengqi
The @found is always false when it comes to the if branch. Besides, the bool type is more suitable for @found. Signed-off-by: Lu Fengqi --- fs/btrfs/extent_io.c | 7 +++ 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/fs/btrfs/extent_io.c b/fs/btrfs/extent_io.c index

[PATCH] btrfs: adjust order of unlocks in do_trimming()

2018-11-27 Thread Su Yue
In function do_trimming(), block_group->lock should be unlocked first. Fixes: 7fe1e6415026 ("Btrfs: rewrite btrfs_trim_block_group()") Signed-off-by: Su Yue --- fs/btrfs/free-space-cache.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/fs/btrfs/free-space-cache.c

[RFC PATCH] btrfs: drop file privileges in btrfs_clone_files

2018-11-27 Thread Lu Fengqi
The generic/513 tell that cloning into a file did not strip security privileges (suid, capabilities) like a regular write would. Signed-off-by: Lu Fengqi --- The xfs and ocfs2 call generic_remap_file_range_prep to drop file privileges, I'm not sure whether btrfs should do the same thing. Any

[RFC PATCH 12/17] btrfs: priority alloc: introduce find_free_extent_search()

2018-11-27 Thread Su Yue
In origin, find_free_extent() just searches block groups in space_info one by one. In priority aware allocator, we first search block groups in higher priority tree than in lower priority tree. This helper unify above two ways for further use. Signed-off-by: Su Yue --- fs/btrfs/extent-tree.c

[RFC PATCH 16/17] btrfs: priority alloc: write bg->priority_tree->groups_sem to avoid race in btrfs_delete_unused_bgs()

2018-11-27 Thread Su Yue
If use priority aware allocator, bg->priority_tree->groups_sem should be written instead of space_info->groups_sem. Signed-off-by: Su Yue --- fs/btrfs/extent-tree.c | 60 +++--- 1 file changed, 44 insertions(+), 16 deletions(-) diff --git

[RFC PATCH 08/17] btrfs: priority alloc: add btrfs_update_block_group_priority()

2018-11-27 Thread Su Yue
Introduce btrfs_update_block_group_priority() to update block_groups::priority. It will move block group from old tree to new tree if need. Signed-off-by: Su Yue --- fs/btrfs/extent-tree.c | 76 ++ 1 file changed, 76 insertions(+) diff --git

[RFC PATCH 04/17] btrfs: priority alloc: add functions to create/remove priority trees

2018-11-27 Thread Su Yue
Introduce create_priority_trees() to create priority trees in space_info. Introduce remove_priority_trees() to remove priority trees in space_info. Signed-off-by: Su Yue --- fs/btrfs/extent-tree.c | 94 ++ 1 file changed, 94 insertions(+) diff --git

[RFC PATCH 17/17] btrfs: add mount option "priority_alloc=%s"

2018-11-27 Thread Su Yue
Add mount option "priority_alloc=%s", now %s only supports "usage" and "off". The latter is used for remount. "priority_alloc=usage" will active priority aware allocator. This mount option changes the way of find_free_extent to search block groups and may cost more time. Signed-off-by: Su Yue

[RFC PATCH 13/17] btrfs: priority alloc: modify find_free_extent() to fit priority allocator

2018-11-27 Thread Su Yue
Add member priority_tree to find_free_extent_ctl to represents the tree using. Modify find_free_extent to use find_free_extent_search, so it can work in default mount option and priorit aware allocator. Signed-off-by: Su Yue --- fs/btrfs/extent-tree.c | 114

[RFC PATCH 06/17] btrfs: priority alloc: introduce three macros to mark block group status

2018-11-27 Thread Su Yue
In origin design, the order of space_info::block_groups is changed only when add/remove block group(s). In design of priority aware allocator, block groups may be moved from one priority tree to another one. What the operation is usually that 1) lock block_group down_write first_tree

[RFC PATCH 03/17] btrfs: priority alloc: introduce compute_block_group_priority/usage

2018-11-27 Thread Su Yue
Introduce compute_block_group_usage() and compute_block_group_usage(). And call the latter in btrfs_make_block_group() and btrfs_read_block_groups(). compute_priority_level use ilog2(free) to compute priority level. Signed-off-by: Su Yue --- fs/btrfs/extent-tree.c | 60

[RFC PATCH 11/17] btrfs: priority alloc: remove block group from priority tree while removing block group

2018-11-27 Thread Su Yue
Export btrfs_remove_block_group_priority() to header ctree.h. Call btrfs_remove_block_group_priority() while deleting transaction->deleted_bgs, btrfs_free_block_groups() and btrfs_remove_block_group(). Signed-off-by: Su Yue --- fs/btrfs/ctree.h | 1 + fs/btrfs/extent-tree.c | 3 +++

[RFC PATCH 02/17] btrfs: add mount definition BTRFS_MOUNT_PRIORITY_USAGE

2018-11-27 Thread Su Yue
Now implementation of priority allocator only support usage option. Add BTRFS_MOUNT_PRIORITY_USAGE for further commits. Signed-off-by: Su Yue --- fs/btrfs/ctree.h | 1 + 1 file changed, 1 insertion(+) diff --git a/fs/btrfs/ctree.h b/fs/btrfs/ctree.h index 5c4651d8a524..4c56baf9f7cf 100644 ---

[RFC PATCH 15/17] btrfs: priority alloc: write bg->priority_groups_sem while waiting reservation

2018-11-27 Thread Su Yue
Since if use priority alloc, we should down/up_write() bg->priority_groups_sem. Signed-off-by: Su Yue --- fs/btrfs/extent-tree.c | 10 -- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c index f530a4344368..6627bbe56ad5 100644

[RFC PATCH 10/17] btrfs: priority alloc: call add_block_group_priority while reading or making block group

2018-11-27 Thread Su Yue
Add block group to priority tree in btrfs_read_block_groups() and btrfs_make_block_groups(). Signed-off-by: Su Yue --- fs/btrfs/extent-tree.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c index 2dec02782df1..fc40901b4772 100644 ---

[RFC PATCH 14/17] btrfs: priority alloc: introduce btrfs_set_bg_updating and call btrfs_update_block_group_prioriy

2018-11-27 Thread Su Yue
For usage as priority, the varaiables in block groups we concered are reserved, bytes_super and btrfs_block_group_used(>item). This patch calls btrfs_set_bg_updating() in locations where above three varaiables changed to mark block groups needs to be updated, then calls btrfs_update_block_group()

[RFC PATCH 05/17] btrfs: priority alloc: introduce functions to add block group to priority tree

2018-11-27 Thread Su Yue
Introduce compute_priority_level() to compute priority level according priority, now just divides PRIORITY_USAGE_FACOTR. Introduce add_block_group_priority() to add block groups to priority tree. Signed-off-by: Su Yue --- fs/btrfs/extent-tree.c | 76 ++

[RFC PATCH 07/17] btrfs: priority alloc: add functions to remove block group from priority tree

2018-11-27 Thread Su Yue
Introduce btrfs_remove_block_group_priority() to remove block group from priority tree. Signed-off-by: Su Yue --- fs/btrfs/extent-tree.c | 37 + 1 file changed, 37 insertions(+) diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c index

[RFC PATCH 09/17] btrfs: priority alloc: call create/remove_priority_trees in space_info

2018-11-27 Thread Su Yue
Call create_priority_trees() in create_space_info(). Call remove_priority_trees() before free of space_info. Signed-off-by: Su Yue --- fs/btrfs/extent-tree.c | 21 + 1 file changed, 17 insertions(+), 4 deletions(-) diff --git a/fs/btrfs/extent-tree.c

[RFC PATCH 01/17] btrfs: priority alloc: prepare of priority aware allocator

2018-11-27 Thread Su Yue
To implement priority aware allocator, this patch: Introduces struct btrfs_priority_tree which contains block groups in same level. Adds member priority to struct btrfs_block_group_cache and pointer points to the priority tree it's located. Adds member priority_trees to struct btrfs_space_info to

[RFC PATCH 00/17] btrfs: implementation of priority aware allocator

2018-11-27 Thread Su Yue
This patchset can be fetched from repo: https://github.com/Damenly/btrfs-devel/commits/priority_aware_allocator. Since patchset 'btrfs: Refactor find_free_extent()' does a nice work to simplify find_free_extent(). This patchset dependents on the refactor. The base is the commit in kdave/misc-next:

Re: [PATCH 9/9] btrfs: drop extra enum initialization where using defaults

2018-11-27 Thread Qu Wenruo
On 2018/11/28 上午3:53, David Sterba wrote: > The first auto-assigned value to enum is 0, we can use that and not > initialize all members where the auto-increment does the same. This is > used for values that are not part of on-disk format. > > Signed-off-by: David Sterba Reviewed-by: Qu

Re: [PATCH 0/9] Switch defines to enums

2018-11-27 Thread Qu Wenruo
On 2018/11/28 上午3:53, David Sterba wrote: > This is motivated by a merging mistake that happened a few releases ago. > Two patches updated BTRFS_FS_* flags independently to the same value, > git did not see any direct merge conflict. The two values got mixed at > runtime and caused crash. > >

Re: [PATCH 8/9] btrfs: switch BTRFS_ORDERED_* to enums

2018-11-27 Thread Qu Wenruo
On 2018/11/28 上午3:53, David Sterba wrote: > We can use simple enum for values that are not part of on-disk format: > ordered extent flags. > > Signed-off-by: David Sterba Reviewed-by: Qu Wenruo Thanks, Qu > --- > fs/btrfs/ordered-data.h | 45 +++-- > 1

RE: [PATCH RESEND 0/8] btrfs-progs: sub: Relax the privileges of "subvolume list/show"

2018-11-27 Thread misono.tomoh...@fujitsu.com
Hi, > -Original Message- > From: Martin Steigerwald [mailto:mar...@lichtvoll.de] > Sent: Tuesday, November 27, 2018 6:48 PM > To: Misono, Tomohiro > Cc: linux-btrfs@vger.kernel.org > Subject: Re: [PATCH RESEND 0/8] btrfs-progs: sub: Relax the privileges > of "subvolume list/show" > >

Re: [PATCH 7/9] btrfs: switch BTRFS_*_LOCK to enums

2018-11-27 Thread Qu Wenruo
On 2018/11/28 上午3:53, David Sterba wrote: > We can use simple enum for values that are not part of on-disk format: > tree lock types. > > Signed-off-by: David Sterba Reviewed-by: Qu Wenruo Thanks, Qu > --- > fs/btrfs/locking.h | 10 ++ > 1 file changed, 6 insertions(+), 4

Re: [PATCH 6/9] btrfs: switch EXTENT_FLAG_* to enums

2018-11-27 Thread Qu Wenruo
On 2018/11/28 上午3:53, David Sterba wrote: > We can use simple enum for values that are not part of on-disk format: > extent map flags. > > Signed-off-by: David Sterba Reviewed-by: Qu Wenruo Thanks, Qu > --- > fs/btrfs/extent_map.h | 21 ++--- > 1 file changed, 14

Re: [PATCH 5/9] btrfs: swtich EXTENT_BUFFER_* to enums

2018-11-27 Thread Qu Wenruo
On 2018/11/28 上午3:53, David Sterba wrote: > We can use simple enum for values that are not part of on-disk format: > extent buffer flags; > > Signed-off-by: David Sterba Reviewed-by: Qu Wenruo Thanks, Qu > --- > fs/btrfs/extent_io.h | 28 > 1 file changed, 16

Re: [PATCH 4/9] btrfs: switch BTRFS_ROOT_* to enums

2018-11-27 Thread Qu Wenruo
On 2018/11/28 上午3:53, David Sterba wrote: > We can use simple enum for values that are not part of on-disk format: > root tree flags. > > Signed-off-by: David Sterba Reviewed-by: Qu Wenruo Thanks, Qu > --- > fs/btrfs/ctree.h | 33 + > 1 file changed, 17

Re: [PATCH 3/9] btrfs: switch BTRFS_FS_* to enums

2018-11-27 Thread Qu Wenruo
On 2018/11/28 上午3:53, David Sterba wrote: > We can use simple enum for values that are not part of on-disk format: > internal filesystem states. > > Signed-off-by: David Sterba Reviewed-by: Qu Wenruo Thanks, Qu > --- > fs/btrfs/ctree.h | 63

Re: [PATCH 2/9] btrfs: switch BTRFS_BLOCK_RSV_* to enums

2018-11-27 Thread Qu Wenruo
On 2018/11/28 上午3:53, David Sterba wrote: > We can use simple enum for values that are not part of on-disk format: > block reserve types. > > Signed-off-by: David Sterba Reviewed-by: Qu Wenruo However more comment will always be a good thing. Thanks, Qu > --- > fs/btrfs/ctree.h | 19

Re: [PATCH 1/9] btrfs: switch BTRFS_FS_STATE_* to enums

2018-11-27 Thread Qu Wenruo
On 2018/11/28 上午3:53, David Sterba wrote: > We can use simple enum for values that are not part of on-disk format: > global filesystem states. > > Signed-off-by: David Sterba Good comment. Reviewed-by: Qu Wenruo Thanks, Qu > --- > fs/btrfs/ctree.h | 25 +++-- > 1 file

Re: Balance: invalid convert data profile raid10

2018-11-27 Thread Qu Wenruo
On 2018/11/28 上午5:16, Mikko Merikivi wrote: > I seem unable to convert an existing btrfs device array to RAID 10. > Since it's pretty much RAID 0 and 1 combined, and 5 and 6 are > unstable, it's what I would like to use. > > After I did tried this with 4.19.2-arch1-1-ARCH and btrfs-progs v4.19,

Re: [PATCH 9/9] btrfs: drop extra enum initialization where using defaults

2018-11-27 Thread Omar Sandoval
On Tue, Nov 27, 2018 at 08:53:59PM +0100, David Sterba wrote: > The first auto-assigned value to enum is 0, we can use that and not > initialize all members where the auto-increment does the same. This is > used for values that are not part of on-disk format. Reviewed-by: Omar Sandoval >

Re: [PATCH 8/9] btrfs: switch BTRFS_ORDERED_* to enums

2018-11-27 Thread Omar Sandoval
On Tue, Nov 27, 2018 at 08:53:57PM +0100, David Sterba wrote: > We can use simple enum for values that are not part of on-disk format: > ordered extent flags. Reviewed-by: Omar Sandoval > Signed-off-by: David Sterba > --- > fs/btrfs/ordered-data.h | 45

Re: [PATCH 7/9] btrfs: switch BTRFS_*_LOCK to enums

2018-11-27 Thread Omar Sandoval
On Tue, Nov 27, 2018 at 08:53:55PM +0100, David Sterba wrote: > We can use simple enum for values that are not part of on-disk format: > tree lock types. > > Signed-off-by: David Sterba > --- > fs/btrfs/locking.h | 10 ++ > 1 file changed, 6 insertions(+), 4 deletions(-) > > diff --git

Re: [PATCH 6/9] btrfs: switch EXTENT_FLAG_* to enums

2018-11-27 Thread Omar Sandoval
On Tue, Nov 27, 2018 at 08:53:52PM +0100, David Sterba wrote: > We can use simple enum for values that are not part of on-disk format: > extent map flags. Reviewed-by: Omar Sandoval > Signed-off-by: David Sterba > --- > fs/btrfs/extent_map.h | 21 ++--- > 1 file changed, 14

Re: [PATCH 5/9] btrfs: swtich EXTENT_BUFFER_* to enums

2018-11-27 Thread Omar Sandoval
On Tue, Nov 27, 2018 at 08:53:50PM +0100, David Sterba wrote: > We can use simple enum for values that are not part of on-disk format: > extent buffer flags; This one has a "swtich" typo in the subject. Otherwise, Reviewed-by: Omar Sandoval > Signed-off-by: David Sterba > --- >

Re: [PATCH 4/9] btrfs: switch BTRFS_ROOT_* to enums

2018-11-27 Thread Omar Sandoval
On Tue, Nov 27, 2018 at 08:53:48PM +0100, David Sterba wrote: > We can use simple enum for values that are not part of on-disk format: > root tree flags. Reviewed-by: Omar Sandoval > Signed-off-by: David Sterba > --- > fs/btrfs/ctree.h | 33 + > 1 file changed,

Re: [PATCH 3/9] btrfs: switch BTRFS_FS_* to enums

2018-11-27 Thread Omar Sandoval
On Tue, Nov 27, 2018 at 08:53:45PM +0100, David Sterba wrote: > We can use simple enum for values that are not part of on-disk format: > internal filesystem states. Hah, looks like we never had a bit 0 ;) Reviewed-by: Omar Sandoval > Signed-off-by: David Sterba > --- > fs/btrfs/ctree.h | 63

Re: [PATCH 2/9] btrfs: switch BTRFS_BLOCK_RSV_* to enums

2018-11-27 Thread Omar Sandoval
On Tue, Nov 27, 2018 at 08:53:43PM +0100, David Sterba wrote: > We can use simple enum for values that are not part of on-disk format: > block reserve types. Reviewed-by: Omar Sandoval > Signed-off-by: David Sterba > --- > fs/btrfs/ctree.h | 19 --- > 1 file changed, 12

Re: [PATCH 1/9] btrfs: switch BTRFS_FS_STATE_* to enums

2018-11-27 Thread Omar Sandoval
On Tue, Nov 27, 2018 at 08:53:41PM +0100, David Sterba wrote: > We can use simple enum for values that are not part of on-disk format: > global filesystem states. Reviewed-by: Omar Sandoval Some typos/wording suggestions below. > Signed-off-by: David Sterba > --- > fs/btrfs/ctree.h | 25

Balance: invalid convert data profile raid10

2018-11-27 Thread Mikko Merikivi
I seem unable to convert an existing btrfs device array to RAID 10. Since it's pretty much RAID 0 and 1 combined, and 5 and 6 are unstable, it's what I would like to use. After I did tried this with 4.19.2-arch1-1-ARCH and btrfs-progs v4.19, I updated my system and tried btrfs balance again with

Re: [PATCH 3/3] btrfs: document extent mapping assumptions in checksum

2018-11-27 Thread Noah Massey
On Tue, Nov 27, 2018 at 2:32 PM Nikolay Borisov wrote: > > On 27.11.18 г. 21:08 ч., Noah Massey wrote: > > On Tue, Nov 27, 2018 at 11:43 AM Nikolay Borisov wrote: > >> > >> On 27.11.18 г. 18:00 ч., Johannes Thumshirn wrote: > >>> Document why map_private_extent_buffer() cannot return '1' (i.e.

Re: [PATCH 2/3] btrfs: wakeup cleaner thread when adding delayed iput

2018-11-27 Thread Josef Bacik
On Tue, Nov 27, 2018 at 07:59:42PM +, Chris Mason wrote: > On 27 Nov 2018, at 14:54, Josef Bacik wrote: > > > On Tue, Nov 27, 2018 at 10:26:15AM +0200, Nikolay Borisov wrote: > >> > >> > >> On 21.11.18 г. 21:09 ч., Josef Bacik wrote: > >>> The cleaner thread usually takes care of delayed

Re: [PATCH 3/3] btrfs: replace cleaner_delayed_iput_mutex with a waitqueue

2018-11-27 Thread Josef Bacik
On Tue, Nov 27, 2018 at 10:29:57AM +0200, Nikolay Borisov wrote: > > > On 21.11.18 г. 21:09 ч., Josef Bacik wrote: > > The throttle path doesn't take cleaner_delayed_iput_mutex, which means > > Which one is the throttle path? btrfs_end_transaction_throttle is only > called during snapshot drop

Re: [PATCH 2/3] btrfs: wakeup cleaner thread when adding delayed iput

2018-11-27 Thread Chris Mason
On 27 Nov 2018, at 14:54, Josef Bacik wrote: > On Tue, Nov 27, 2018 at 10:26:15AM +0200, Nikolay Borisov wrote: >> >> >> On 21.11.18 г. 21:09 ч., Josef Bacik wrote: >>> The cleaner thread usually takes care of delayed iputs, with the >>> exception of the btrfs_end_transaction_throttle path. The

Re: [PATCH] btrfs: only run delayed refs if we're committing

2018-11-27 Thread Josef Bacik
On Tue, Nov 27, 2018 at 07:43:39PM +, Filipe Manana wrote: > On Tue, Nov 27, 2018 at 7:22 PM Josef Bacik wrote: > > > > On Fri, Nov 23, 2018 at 04:59:32PM +, Filipe Manana wrote: > > > On Thu, Nov 22, 2018 at 12:35 AM Josef Bacik wrote: > > > > > > > > I noticed in a giant dbench run

Re: [PATCH 2/3] btrfs: wakeup cleaner thread when adding delayed iput

2018-11-27 Thread Josef Bacik
On Tue, Nov 27, 2018 at 10:26:15AM +0200, Nikolay Borisov wrote: > > > On 21.11.18 г. 21:09 ч., Josef Bacik wrote: > > The cleaner thread usually takes care of delayed iputs, with the > > exception of the btrfs_end_transaction_throttle path. The cleaner > > thread only gets woken up every 30

[PATCH 9/9] btrfs: drop extra enum initialization where using defaults

2018-11-27 Thread David Sterba
The first auto-assigned value to enum is 0, we can use that and not initialize all members where the auto-increment does the same. This is used for values that are not part of on-disk format. Signed-off-by: David Sterba --- fs/btrfs/btrfs_inode.h | 2 +- fs/btrfs/ctree.h | 28

[PATCH 4/9] btrfs: switch BTRFS_ROOT_* to enums

2018-11-27 Thread David Sterba
We can use simple enum for values that are not part of on-disk format: root tree flags. Signed-off-by: David Sterba --- fs/btrfs/ctree.h | 33 + 1 file changed, 17 insertions(+), 16 deletions(-) diff --git a/fs/btrfs/ctree.h b/fs/btrfs/ctree.h index

[PATCH 3/9] btrfs: switch BTRFS_FS_* to enums

2018-11-27 Thread David Sterba
We can use simple enum for values that are not part of on-disk format: internal filesystem states. Signed-off-by: David Sterba --- fs/btrfs/ctree.h | 63 1 file changed, 31 insertions(+), 32 deletions(-) diff --git a/fs/btrfs/ctree.h

<    1   2   3   4   5   6   7   8   9   10   >