[PATCH] btrfs-progs: Eliminate remaining uses of strerror(errno)

2018-09-12 Thread Rosen Penev
%m allows a smaller filesize. Useful on embedded systems. Signed-off-by: Rosen Penev --- build/Documentation/Makefile | 142 + build/Makefile.inc | 42 ++ build/config.h | 139 + build/config.log | 822 ++

Re: [PATCH] Btrfs: remove level==0 check in balance_level

2018-09-12 Thread Liu Bo
On Wed, Sep 12, 2018 at 02:52:38PM +0200, David Sterba wrote: > On Wed, Sep 12, 2018 at 06:06:23AM +0800, Liu Bo wrote: > > btrfs_search_slot() > >if (level != 0) > > setup_nodes_for_search() > > balance_level() > > > > It is just impossible to have level=0 in balance_level. >

Re: [PATCH] Btrfs: assert page dirty bit

2018-09-12 Thread Liu Bo
On Wed, Sep 12, 2018 at 09:38:49AM +0300, Nikolay Borisov wrote: > > > On 12.09.2018 01:06, Liu Bo wrote: > > Just in case that someone breaks the rule that pages are dirty as long > > as eb is dirty. > > > > Signed-off-by: Liu Bo > > --- > > fs/btrfs/extent_io.c | 5 + > > 1 file

Re: [PATCH] Btrfs: skip set_page_dirty if eb is dirty

2018-09-12 Thread Liu Bo
On Wed, Sep 12, 2018 at 09:37:20AM +0300, Nikolay Borisov wrote: > > > On 12.09.2018 01:06, Liu Bo wrote: > > As long as @eb is marked with EXTENT_BUFFER_DIRTY, all of its pages > > are dirty, so no need to set pages dirty again. > > > > Signed-off-by: Liu Bo > > Does make it any performance

Re: [PATCH] btrfs: wait on caching when putting the bg cache

2018-09-12 Thread David Sterba
On Wed, Sep 12, 2018 at 10:45:45AM -0400, Josef Bacik wrote: > While testing my backport I noticed there was a panic if I ran > generic/416 generic/417 generic/418 all in a row. This just happened to > uncover a race where we had outstanding IO after we destroy all of our > workqueues, and then

Re: [PATCH] btrfs: wait on caching when putting the bg cache

2018-09-12 Thread Omar Sandoval
On Wed, Sep 12, 2018 at 10:45:45AM -0400, Josef Bacik wrote: > While testing my backport I noticed there was a panic if I ran > generic/416 generic/417 generic/418 all in a row. This just happened to > uncover a race where we had outstanding IO after we destroy all of our > workqueues, and then

Re: [PATCH 00/15] Add delayed-refs support to btrfs-progs

2018-09-12 Thread David Sterba
On Wed, Sep 12, 2018 at 07:51:39PM +0800, Su Yue wrote: > Actually, now kdave/devel still fails at fsck-tests/020 due to > version 1st of this patchset. See the thread please > https://www.spinics.net/lists/linux-btrfs/msg81675.html > > Nikolay's V2 patchset should slove the problem. > You may

Re: [PATCH 05/36] btrfs: only count ref heads run in __btrfs_run_delayed_refs

2018-09-12 Thread David Sterba
On Tue, Sep 11, 2018 at 04:07:30PM -0700, Omar Sandoval wrote: > On Tue, Sep 11, 2018 at 01:57:36PM -0400, Josef Bacik wrote: > > We pick the number of ref's to run based on the number of ref heads, and > > only make the decision to stop once we've processed entire ref heads, so > > only count the

Re: [PATCH 08/36] btrfs: dump block_rsv whe dumping space info

2018-09-12 Thread David Sterba
On Tue, Sep 11, 2018 at 04:11:45PM -0700, Omar Sandoval wrote: > On Tue, Sep 11, 2018 at 01:57:39PM -0400, Josef Bacik wrote: > > For enospc_debug having the block rsvs is super helpful to see if we've > > done something wrong. > > > > Signed-off-by: Josef Bacik > > --- > >

Re: [PATCH] btrfs: wait on caching when putting the bg cache

2018-09-12 Thread Josef Bacik
On Wed, Sep 12, 2018 at 06:15:41PM +0300, Nikolay Borisov wrote: > > > On 12.09.2018 17:45, Josef Bacik wrote: > > While testing my backport I noticed there was a panic if I ran > > generic/416 generic/417 generic/418 all in a row. This just happened to > > uncover a race where we had

Re: [PATCH] btrfs: wait on caching when putting the bg cache

2018-09-12 Thread Nikolay Borisov
On 12.09.2018 17:45, Josef Bacik wrote: > While testing my backport I noticed there was a panic if I ran > generic/416 generic/417 generic/418 all in a row. This just happened to > uncover a race where we had outstanding IO after we destroy all of our > workqueues, and then we'd go to queue

Re: [RFC PATCH 0/6] btrfs-progs: build distinct binaries for specific btrfs subcommands

2018-09-12 Thread Axel Burri
On 30/08/2018 19.23, Austin S. Hemmelgarn wrote: > On 2018-08-30 13:13, Axel Burri wrote: >> On 29/08/2018 21.02, Austin S. Hemmelgarn wrote: >>> On 2018-08-29 13:24, Axel Burri wrote: This patch allows to build distinct binaries for specific btrfs subcommands, e.g.

[RFC PATCH v2 3/4] btrfs-progs: Makefile: add extra objects definitions for separated binaries

2018-09-12 Thread Axel Burri
Some separated binaries have references to specific command objects ($cmds_objects). Add these dependencies in the Makefile, and use them in the linker target (as in target "btrfs-%:"). Fixes linkage errors for these subcommands. The "make separated" target now builds without errors.

[RFC PATCH v2 2/4] btrfs-progs: remove unneeded dependencies on separated build (-DBTRFS_SEPARATED_BUILD)

2018-09-12 Thread Axel Burri
Remove references to unneeded symbols when building separated subcommands: "btrfs-*.separated.o". Note that this patch still leaves unreferenced static symbols, which in turn makes the compiler warn about "unused function/variable". Stripping all symbols would imply adaptions in the source code

[RFC PATCH v2 0/4] btrfs-progs: build distinct binaries for specific btrfs subcommands

2018-09-12 Thread Axel Burri
This patch allows to build distinct binaries for specific btrfs subcommands, e.g. "btrfs-subvolume-show" which would be identical to "btrfs subvolume show". Changes from v1 [1]: - No more need of generated c-file for each separated commands (all functionality has moved to Makefile). - More

[RFC PATCH v2 4/4] btrfs-progs: build: add --enable-setcap-install, --enable-setuid-install, --enable-btrfs-separated

2018-09-12 Thread Axel Burri
Adds Makefile target "install-separated": install all "btrfs-*.separated" binaries and rename to "btrfs-*". If configured with --enable-setcap-install, also sets linux capabilities(7) using setcap(8). If configured with "--enable-setuid-install", sets setuid bit while installing. Use

[RFC PATCH v2 1/4] btrfs-progs: Makefile: create separated binaries for "btrfs" subcommands; add fscaps declarations

2018-09-12 Thread Axel Burri
Create separate binaries "btrfs-xxx-yyy.separated" for each subcommand "btrfs xxx yyy". Also declares fscaps for (supported) subcommands. This is useful for admins to provide specific subcommand binaries with elevated privileges (capabilities(7), suid). Example: # make separated-fscaps #

[PATCH] btrfs: wait on caching when putting the bg cache

2018-09-12 Thread Josef Bacik
While testing my backport I noticed there was a panic if I ran generic/416 generic/417 generic/418 all in a row. This just happened to uncover a race where we had outstanding IO after we destroy all of our workqueues, and then we'd go to queue the endio work on those free'd workqueues. This is

Re: [PATCH] Btrfs: do not wait after queue async work for delaye refs

2018-09-12 Thread Josef Bacik
On Wed, Sep 12, 2018 at 10:19:23AM +0300, Nikolay Borisov wrote: > [Adding Josef to CC] > > On 12.09.2018 01:06, Liu Bo wrote: > > If metadata space is hungry, how fast flush_space() can run determines > > the latency we spend in reserve_metadata_space(). > > > > flush_space() > >case

Re: [PATCH] Btrfs: remove level==0 check in balance_level

2018-09-12 Thread David Sterba
On Wed, Sep 12, 2018 at 06:06:23AM +0800, Liu Bo wrote: > btrfs_search_slot() >if (level != 0) > setup_nodes_for_search() > balance_level() > > It is just impossible to have level=0 in balance_level. While this is true, what do you think about adding ASSERT(level > 0) ? This

[PATCH v2 7/7] btrfs-progs: fsck-tests: add test case inode_extref without dir_item and dir_index

2018-09-12 Thread damenly . su
From: Su Yue This case contains an inode_extref: == ... item 1 key (257 INODE_EXTREF 3460996356) itemoff 3947 itemsize 24 index 257 parent 256 namelen 6 name: foo255 ... == The related dir_item and

[PATCH v2 6/7] btrfs-progs: lowmem: improve check_inode_extref()

2018-09-12 Thread damenly . su
From: Su Yue inode_extref is much similar with inode_ref, most codes are reused in check_inode_extref(). Exception: There is no need to check root directory, so remove it. Make check_inode_extref() verify hash value with key offset now. And lowmem check can detect errors about inode_extref and

[PATCH v2 5/7] btrfs-progs: lowmem: continue to check item in last slot while checking inodes

2018-09-12 Thread damenly . su
From: Su Yue After call of check_inode_item(), path may point to the last unchecked slot of the leaf. The outer walk_up_tree() always treats the position as checked item then skips to next item. If the last item was an inode item, yes, it was unchecked. Then walk_up_tree() will think the leaf

[PATCH v2 0/7] btrfs-progs: lowmem: bug fixes and inode_extref repair

2018-09-12 Thread damenly . su
From: Su Yue This patchset can be fetched from my github(based on v4.17.1): https://github.com/Damenly/btrfs-progs/tree/lowmem_extref The patchset aims to support check and repair errors about inode_extref in lowmem mode. patch[1-2] let btrfs_unlink() detect inode_extref. patch[3] fixes a

[PATCH v2 3/7] btrfs-progs: lowmem check: find dir_item by di_key in check_dir_item()

2018-09-12 Thread damenly . su
From: Su Yue In check_dir_item, we are going to search corresponding dir_item/index. Commit 564901eac7a4 ("btrfs-progs: check: introduce print_dir_item_err()") Changed argument name from key to di_key but forgot to change the key name for dir_item search. So @key shouldn't be used here. It

[PATCH v2 2/7] btrfs-progs: make btrfs_unlink() lookup inode_extref

2018-09-12 Thread damenly . su
From: Su Yue btrfs_unlink() uses btrfs_lookup_inode_ref() to look up inode_ref but forget inode_extref case. Let btrfs_unlink() call btrfs_lookup_inode_extref() if inode_ref is found and EXTENDED_IREF feature is enabled. Fixes: 0cc75eddd093 ("btrfs-progs: Add btrfs_unlink() and

[PATCH v2 4/7] btrfs-progs: lowmem: search key of root again after check_fs_root() under repair

2018-09-12 Thread damenly . su
From: Su Yue In check_fs_roots_lowmem(), we do search and follow the resulted path to call check_fs_root(), then call btrfs_next_item() to check next root. However, if repair is enabled, the root tree can be cowed, the existed path can cause strange errors. Solution: If repair, save the key

[PATCH v2 1/7] btrfs-progs: adjust arguments of btrfs_lookup_inode_extref()

2018-09-12 Thread damenly . su
From: Su Yue The argument index is not used in btrfs_lookup_inode_extref(), so remove it. And adjust positions its arguments to make it consistent with kernel part. No functional change. Fixes: 260675657767 ("btrfs-progs: Import btrfs_insert/del/lookup_extref() functions.") Signed-off-by: Su

Re: [PATCH] Btrfs: unify error handling of btrfs_lookup_dir_item

2018-09-12 Thread David Sterba
On Wed, Sep 12, 2018 at 06:06:26AM +0800, Liu Bo wrote: > Unify the error handling part with IS_ERR_OR_NULL. > > No functional changes. > > Signed-off-by: Liu Bo Reviewed-by: David Sterba

Re: [PATCH 0/6] btrfs-progs: lowmem: bug fixes and inode_extref repair

2018-09-12 Thread Su Yue
Sorry for the noise, Please ignore this patchset. Will send v2. On 2018/9/13 3:20 AM, damenly...@gmail.com wrote: From: Su Yue This patchset can be fetched from my github(based on v4.17.1): https://github.com/Damenly/btrfs-progs/tree/lowmem_extref The patchset aims to support check and

Re: [PATCH 00/15] Add delayed-refs support to btrfs-progs

2018-09-12 Thread Su Yue
On 2018/7/16 11:39 PM, David Sterba wrote: On Fri, Jun 08, 2018 at 03:47:43PM +0300, Nikolay Borisov wrote: Hello, Here is a series which adds support for delayed refs. This is needed to enable later work on

[PATCH 5/7] btrfs-progs: lowmem: improve check_inode_extref()

2018-09-12 Thread damenly . su
From: Su Yue inode_extref is much similar with inode_ref, most codes are reused in check_inode_extref(). Exception: There is no need to check root directory, so remove it. Make check_inode_extref() verify hash value with key offset now. And lowmem check can detect errors about inode_extref and

[PATCH 5/6] btrfs-progs: lowmem: continue to check item in last slot while checking inodes

2018-09-12 Thread damenly . su
From: Su Yue After call of check_inode_item(), path may point to the last unchecked slot of the leaf. The outer walk_up_tree() always treats the position as checked item then skips to next item. If the last item was an inode item, yes, it was unchecked. Then walk_up_tree() will think the leaf

[PATCH 6/7] btrfs-progs: lowmem: continue to check item in last slot while checking inodes

2018-09-12 Thread damenly . su
From: Su Yue After call of check_inode_item(), path may point to the last unchecked slot of the leaf. The outer walk_up_tree() always treats the position as checked item then skips to next item. If the last item was an inode item, yes, it was unchecked. Then walk_up_tree() will think the leaf

[PATCH 7/7] btrfs-progs: fsck-tests: add test case inode_extref without dir_item and dir_index

2018-09-12 Thread damenly . su
From: Su Yue This case contains an inode_extref: == ... item 1 key (257 INODE_EXTREF 3460996356) itemoff 3947 itemsize 24 index 257 parent 256 namelen 6 name: foo255 ... == The related dir_item and

[PATCH 6/6] btrfs-progs: lowmem: optimization and repair for check_inode_extref()

2018-09-12 Thread damenly . su
From: Su Yue inode_extref is much similar with inode_ref, most codes are reused in check_inode_extref(). Exception: There is no need to check root directory, so remove it. Make check_inode_extref() verify hash value with key offset now. And lowmem check can detect errors about inode_extref and

[PATCH 4/6] btrfs-progs: lowmem: search key of root again after check_fs_root() under repair

2018-09-12 Thread damenly . su
From: Su Yue In check_fs_roots_lowmem(), we do search and follow the resulted path to call check_fs_root(), then call btrfs_next_item() to check next root. However, if repair is enabled, the root tree can be cowed, the existed path can cause strange errors. Solution: If repair, save the key

[PATCH 2/6] btrfs-progs: make btrfs_unlink() lookup inode_extref

2018-09-12 Thread damenly . su
From: Su Yue btrfs_unlink() uses btrfs_lookup_inode_ref() to look up inode_ref but forget inode_extref case. Let btrfs_unlink() call btrfs_lookup_inode_extref() if inode_ref is found and EXTENDED_IREF feature is enabled. Fixes: 0cc75eddd093 ("btrfs-progs: Add btrfs_unlink() and

[PATCH 1/6] btrfs-progs: adjust arguments of btrfs_lookup_inode_extref()

2018-09-12 Thread damenly . su
From: Su Yue The argument index is not used in btrfs_lookup_inode_extref(), so remove it. And adjust positions its arguments to make it consistent with kernel part. No functional change. Fixes: 260675657767 ("btrfs-progs: Import btrfs_insert/del/lookup_extref() functions.") Signed-off-by: Su

[PATCH 3/6] btrfs-progs: lowmem check: find dir_item by di_key in check_dir_item()

2018-09-12 Thread damenly . su
From: Su Yue In check_dir_item, we are going to search corresponding dir_item/index. Commit 564901eac7a4 ("btrfs-progs: check: introduce print_dir_item_err()") Changed argument name from key to di_key but forgot to change the key name for dir_item search. So @key shouldn't be used here. It

[PATCH 0/6] btrfs-progs: lowmem: bug fixes and inode_extref repair

2018-09-12 Thread damenly . su
From: Su Yue This patchset can be fetched from my github(based on v4.17.1): https://github.com/Damenly/btrfs-progs/tree/lowmem_extref The patchset aims to support check and repair errors about inode_extref in lowmem mode. patch[1-2] let btrfs_unlink() detect inode_extref. patch[3] fixes a

[PATCH] btrfs-progs: change filename limit to 255 when creating subvolume

2018-09-12 Thread Su Yanjun
Modify the file name length limit to meet the Linux naming convention. In addition, the file name length is always bigger than 0, no need to compare with 0 again. Issue: #145 Signed-off-by: Su Yanjun --- cmds-subvolume.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git

Re: [PATCH] Btrfs: do not wait after queue async work for delaye refs

2018-09-12 Thread Nikolay Borisov
[Adding Josef to CC] On 12.09.2018 01:06, Liu Bo wrote: > If metadata space is hungry, how fast flush_space() can run determines > the latency we spend in reserve_metadata_space(). > > flush_space() >case FLUSH_DELAYED_ITEMS: > ... > btrfs_end_transaction() >case ALLOC_CHUNK:

Re: [PATCH] Btrfs: unify error handling of btrfs_lookup_dir_item

2018-09-12 Thread Nikolay Borisov
On 12.09.2018 01:06, Liu Bo wrote: > Unify the error handling part with IS_ERR_OR_NULL. > > No functional changes. > > Signed-off-by: Liu Bo Reviewed-by: Nikolay Borisov > --- > fs/btrfs/inode.c | 21 + > fs/btrfs/send.c | 8 ++-- > 2 files changed, 7

Re: [PATCH] Btrfs: remove unused code in __btrfs_unlink_inode

2018-09-12 Thread Nikolay Borisov
On 12.09.2018 01:06, Liu Bo wrote: > It might get @leaf and @key in order to do some sanity checks on key's > fields, but since we don't check it any more, it's fine to remove the > code. This is actually false, leaf and key weren't used for sanity checks. Instead this code was first

Re: [PATCH] Btrfs: skip setting path to blocking mode if balance is not needed

2018-09-12 Thread Nikolay Borisov
On 12.09.2018 01:06, Liu Bo wrote: > balance_level() may return early in some cases, but these checks don't > have to be done with blocking write lock. > > This puts together these checks into a helper and the benefit is to > avoid switching spinning locks to blocking locks (in these paticular

Re: [PATCH] Btrfs: remove level==0 check in balance_level

2018-09-12 Thread Nikolay Borisov
On 12.09.2018 01:06, Liu Bo wrote: > btrfs_search_slot() >if (level != 0) > setup_nodes_for_search() > balance_level() > > It is just impossible to have level=0 in balance_level. > > Signed-off-by: Liu Bo I concur with the analysis: Reviewed-by: Nikolay Borisov > ---

Re: [PATCH] Btrfs: assert page dirty bit

2018-09-12 Thread Nikolay Borisov
On 12.09.2018 01:06, Liu Bo wrote: > Just in case that someone breaks the rule that pages are dirty as long > as eb is dirty. > > Signed-off-by: Liu Bo > --- > fs/btrfs/extent_io.c | 5 + > 1 file changed, 5 insertions(+) > > diff --git a/fs/btrfs/extent_io.c b/fs/btrfs/extent_io.c >

Re: [PATCH] Btrfs: skip set_page_dirty if eb is dirty

2018-09-12 Thread Nikolay Borisov
On 12.09.2018 01:06, Liu Bo wrote: > As long as @eb is marked with EXTENT_BUFFER_DIRTY, all of its pages > are dirty, so no need to set pages dirty again. > > Signed-off-by: Liu Bo Does make it any performance difference, numbers? Reviewed-by: Nikolay Borisov > --- > fs/btrfs/extent_io.c