[PATCH 09/25] btrfs: temporarily export fragment_free_space

2019-06-20 Thread Josef Bacik
This is used in caching and reading block groups, so export it while we move these chunks independently. Signed-off-by: Josef Bacik --- fs/btrfs/block-group.h | 1 + fs/btrfs/extent-tree.c | 8 2 files changed, 5 insertions(+), 4 deletions(-) diff --git a/fs/btrfs/block-group.h b/fs

[PATCH 23/25] btrfs: migrate the alloc_profile helpers

2019-06-20 Thread Josef Bacik
These feel more at home in block-group.c. Signed-off-by: Josef Bacik --- fs/btrfs/block-group.c | 100 ++ fs/btrfs/block-group.h | 16 +++ fs/btrfs/ctree.h | 4 -- fs/btrfs/extent-tree.c | 115

[PATCH 01/25] btrfs: move btrfs_add_free_space out of a header file

2019-06-20 Thread Josef Bacik
This is prep work for moving block_group_cache around. Having this in the header file makes the header file include need to be in a certain order, which is awkward, so just move it into free-space-cache.c and then we can re-arrange later. Signed-off-by: Josef Bacik --- fs/btrfs/free-space

[PATCH 00/25] btrfs: migrate the block group code

2019-06-20 Thread Josef Bacik
This is the series to migrate the block group code out of extent-tree.c. This is a much larger series than the previous two series because things were much more intertwined than block_rsv's and space_info. There is one code change patch in this series, it is btrfs: make caching_thread use btrfs_

[PATCH 19/25] btrfs: export block group accounting helpers

2019-06-20 Thread Josef Bacik
Want to move these functions into block-group.c, so export them. Signed-off-by: Josef Bacik --- fs/btrfs/block-group.h | 6 ++ fs/btrfs/extent-tree.c | 21 ++--- 2 files changed, 16 insertions(+), 11 deletions(-) diff --git a/fs/btrfs/block-group.h b/fs/btrfs/block-group.h

[PATCH 24/25] btrfs: migrate the block group cleanup code

2019-06-20 Thread Josef Bacik
This can now be easily migrated as well. Signed-off-by: Josef Bacik --- fs/btrfs/block-group.c | 140 + fs/btrfs/block-group.h | 2 + fs/btrfs/ctree.h | 2 - fs/btrfs/extent-tree.c | 140

[PATCH 25/25] btrfs: unexport the temporary exported functions

2019-06-20 Thread Josef Bacik
These were renamed and exported to facilitate logical migration of different code chunks into block-group.c. Now that all the users are in one file go ahead and rename them back, move the code around, and make them static. Signed-off-by: Josef Bacik --- fs/btrfs/block-group.c | 140

[PATCH 15/25] btrfs: migrate the block group read/creation code

2019-06-20 Thread Josef Bacik
All of the prep work has been done so we can now cleanly move this chunk over. Signed-off-by: Josef Bacik --- fs/btrfs/block-group.c | 613 + fs/btrfs/block-group.h | 5 + fs/btrfs/ctree.h | 5 - fs/btrfs/extent-tree.c | 611

[PATCH 02/25] btrfs: move basic block_group definitions to their own header

2019-06-20 Thread Josef Bacik
This is prep work for moving all of the block group cache code into its own file. Signed-off-by: Josef Bacik --- fs/btrfs/block-group.h | 156 + fs/btrfs/ctree.h | 148 +-- fs/btrfs/disk-io.c

[PATCH 21/25] btrfs: migrate btrfs_can_relocate

2019-06-20 Thread Josef Bacik
This belongs to block-group.c, move it over. Signed-off-by: Josef Bacik --- fs/btrfs/block-group.c | 141 + fs/btrfs/block-group.h | 1 + fs/btrfs/ctree.h | 1 - fs/btrfs/extent-tree.c | 141

[PATCH 18/25] btrfs: migrate the dirty bg writeout code

2019-06-20 Thread Josef Bacik
This can be easily migrated over now. Signed-off-by: Josef Bacik --- fs/btrfs/block-group.c | 520 + fs/btrfs/block-group.h | 3 + fs/btrfs/ctree.h | 3 - fs/btrfs/extent-tree.c | 518 4

[PATCH 13/25] btrfs: migrate the block group removal code

2019-06-20 Thread Josef Bacik
This is the removal code and the unused bgs code. Signed-off-by: Josef Bacik --- fs/btrfs/block-group.c | 509 + fs/btrfs/block-group.h | 7 + fs/btrfs/ctree.h | 7 - fs/btrfs/extent-tree.c | 505

[PATCH 06/25] btrfs: export the block group caching helpers

2019-06-20 Thread Josef Bacik
This will make it so we can move them easily. Signed-off-by: Josef Bacik --- fs/btrfs/block-group.h | 14 +++ fs/btrfs/extent-tree.c | 65 ++ 2 files changed, 43 insertions(+), 36 deletions(-) diff --git a/fs/btrfs/block-group.h b/fs

[PATCH 11/25] btrfs: migrate the block group caching code

2019-06-20 Thread Josef Bacik
We can now just copy it over to block-group.c. Signed-off-by: Josef Bacik --- fs/btrfs/block-group.c | 463 + fs/btrfs/block-group.h | 2 + fs/btrfs/ctree.h | 2 - fs/btrfs/extent-tree.c | 461

[PATCH 22/25] btrfs: migrate the chunk allocation code

2019-06-20 Thread Josef Bacik
This feels more at home in block-group.c than in extent-tree.c. Signed-off-by: Josef Bacik --- fs/btrfs/block-group.c| 247 ++ fs/btrfs/block-group.h| 24 + fs/btrfs/ctree.h | 24 - fs/btrfs/delalloc-space.c | 1 + fs

[PATCH 10/25] btrfs: make caching_thread use btrfs_find_next_key

2019-06-20 Thread Josef Bacik
o fix btrfs_find_next_key to deal with ->skip_locking and then it works exactly the same as the private find_next_key helper. Signed-off-by: Josef Bacik --- fs/btrfs/ctree.c | 4 ++-- fs/btrfs/extent-tree.c | 3 ++- 2 files changed, 4 insertions(+), 3 deletions(-) diff --git a/fs/btrfs/c

[PATCH 12/25] btrfs: temporarily export inc_block_group_ro

2019-06-20 Thread Josef Bacik
This is used in a few logical parts of the block group code, temporarily export it so we can move things in pieces. Signed-off-by: Josef Bacik --- fs/btrfs/block-group.h | 2 ++ fs/btrfs/extent-tree.c | 15 --- 2 files changed, 10 insertions(+), 7 deletions(-) diff --git a/fs

[PATCH 14/25] btrfs: export get_alloc_profile

2019-06-20 Thread Josef Bacik
This gets used directly by a bunch of the block group code, export it to make it easier to move things around. Signed-off-by: Josef Bacik --- fs/btrfs/ctree.h | 1 + fs/btrfs/extent-tree.c | 16 2 files changed, 9 insertions(+), 8 deletions(-) diff --git a/fs/btrfs

Re: [PATCH 04/11] btrfs: move the space_info handling code to space-info.c

2019-06-25 Thread Josef Bacik
On Tue, Jun 25, 2019 at 01:58:43PM +0200, David Sterba wrote: > On Tue, Jun 18, 2019 at 04:09:19PM -0400, Josef Bacik wrote: > > --- /dev/null > > +++ b/fs/btrfs/space-info.c > > @@ -0,0 +1,177 @@ > > +/* SPDX-License-Identifier: GPL-2.0 */ > > +/* > > + * C

Need help with a lockdep splat, possibly perf related?

2019-07-03 Thread Josef Bacik
Hello, I've been seeing a variation of the following splat recently and I have no earthly idea what it's trying to tell me. I either get this one, or I get one that tells me the same thing except it's complaining about &cpuctx_mutex instead of sb_pagefaults. There is no place we take the reloc_m

Re: Need help with a lockdep splat, possibly perf related?

2019-07-05 Thread Josef Bacik
On Wed, Jul 03, 2019 at 11:12:10PM +0200, Peter Zijlstra wrote: > On Wed, Jul 03, 2019 at 09:54:06AM -0400, Josef Bacik wrote: > > Hello, > > > > I've been seeing a variation of the following splat recently and I have no > > earthly idea what it's trying

Re: [PATCH] Btrfs: fix race leading to fs corruption after transaction abortion

2019-07-29 Thread Josef Bacik
17 @@ int btrfs_commit_transaction(struct btrfs_trans_handle > *trans) > } > } else { > spin_unlock(&fs_info->trans_lock); > + /* > + * The previous transaction was aborted and was already removed > + * from the list of transactions at fs_info->trans_list. So we > + * abort to prevent writing a new superblock that reflects a > + * corrupt state (pointing to trees with unwritten nodes/leafs). > + */ > + if (test_bit(BTRFS_FS_STATE_TRANS_ABORTED, > + &fs_info->fs_state)) { > + ret = -EROFS; > + goto cleanup_transaction; > + } > } > > extwriter_counter_dec(cur_trans, trans->type); Reviewed-by: Josef Bacik Thanks, Josef

[PATCH 5/5] btrfs: introduce an evict flushing state

2019-08-01 Thread Josef Bacik
s first. Signed-off-by: Josef Bacik --- fs/btrfs/ctree.h | 1 + fs/btrfs/inode.c | 81 +++ fs/btrfs/space-info.c | 27 +-- 3 files changed, 62 insertions(+), 47 deletions(-) diff --git a/fs/btrfs/ctree.h b/fs/btrfs/ctree.

[PATCH 0/5] Rework eviction space flushing

2019-08-01 Thread Josef Bacik
This is a set of patches to address how we do space flushing for inode evictions. Historically we've only been allowed to do a few things to reclaim space for inode evictions, mostly because we'd deadlock with iput. But we have delayed iputs in place to make sure we're always doing iput where it'

[PATCH 4/5] btrfs: refactor priority_reclaim_metadata_space

2019-08-01 Thread Josef Bacik
ction flushing. Signed-off-by: Josef Bacik --- fs/btrfs/space-info.c | 17 +++-- 1 file changed, 11 insertions(+), 6 deletions(-) diff --git a/fs/btrfs/space-info.c b/fs/btrfs/space-info.c index 71749b355136..03556e411b11 100644 --- a/fs/btrfs/space-info.c +++ b/fs/btrfs/space-i

[PATCH 2/5] btrfs: unify error handling for ticket flushing

2019-08-01 Thread Josef Bacik
Currently we handle the cleanup of errored out tickets in both the priority flush path and the normal flushing path. This is the same code in both places, so just refactor so we don't duplicate the cleanup work. Signed-off-by: Josef Bacik --- fs/btrfs/space-info.c

[PATCH 1/5] btrfs: add a flush step for delayed iputs

2019-08-01 Thread Josef Bacik
Delayed iputs could very well free up enough space without needing to commit the transaction, so make this step it's own step. This will allow us to skip the step for evictions in a later patch. Signed-off-by: Josef Bacik --- fs/btrfs/ctree.h | 3 ++- fs/btrfs/space-i

[PATCH 3/5] btrfs: factor out the ticket flush handling

2019-08-01 Thread Josef Bacik
We're going to make this logic a little more complicated for evict, so factor the ticket flushing/waiting code out of __reserve_metadata_bytes. This has no functional change. Signed-off-by: Josef Bacik --- fs/btrfs/space-info.c | 64 --- 1 file ch

[PATCH] btrfs-progs: add a --check-bg-usage option to fsck

2019-08-02 Thread Josef Bacik
7;t trip up currently, so this will just keep me from regressing us. Thanks, Signed-off-by: Josef Bacik --- btrfsck.h| 1 + check/main.c | 52 ++-- 2 files changed, 51 insertions(+), 2 deletions(-) diff --git a/btrfsck.h b/btrfsck.h index ac7f5d4

Re: [PATCH] btrfs-progs: add a --check-bg-usage option to fsck

2019-08-02 Thread Josef Bacik
On Fri, Aug 02, 2019 at 09:54:12PM +0800, Qu Wenruo wrote: > > > On 2019/8/2 下午9:06, Josef Bacik wrote: > > Sometimes when messing with the chunk allocator code we can end up > > over-allocating chunks. Generally speaking I'll notice this when a > > random x

Re: [PATCH 00/25] btrfs: migrate the block group code

2019-08-02 Thread Josef Bacik
On Fri, Aug 02, 2019 at 03:56:38PM +0200, David Sterba wrote: > On Thu, Jun 20, 2019 at 03:37:42PM -0400, Josef Bacik wrote: > > This is the series to migrate the block group code out of extent-tree.c. > > This > > is a much larger series than the previous two serie

[PATCH][v2] btrfs-progs: add a --check-bg-usage option to fsck

2019-08-02 Thread Josef Bacik
7;t trip up currently, so this will just keep me from regressing us. Thanks, Signed-off-by: Josef Bacik --- v1->v2: - tested with my alloc chunk ioctl, realized the chunk checker removes the bg recs from the list, so this wasn't actually doing anything. Moved the check so now it properly

[PATCH] btrfs: add an ioctl to force chunk allocation

2019-08-02 Thread Josef Bacik
In testing block group removal it's sometimes handy to be able to create block groups on demand. Add an ioctl to allow us to force allocation from userspace. Signed-off-by: Josef Bacik --- fs/btrfs/ioctl.c | 30 ++ include/uapi/linux/btrfs.h | 1

[PATCH] btrfs-progs: add a alloc-chunk command

2019-08-02 Thread Josef Bacik
This is so we can force chunk allocation to test various parts of the fs. I used this to test my btrfsck patch for checking for empty block groups, and a weird block group removal issue. Signed-off-by: Josef Bacik --- cmds/filesystem.c | 48

[PATCH][v2] btrfs: add an ioctl to force chunk allocation

2019-08-05 Thread Josef Bacik
In testing block group removal it's sometimes handy to be able to create block groups on demand. Add an ioctl to allow us to force allocation from userspace. Signed-off-by: Josef Bacik --- v1->v2: - I noticed last week when backporting this that btrfs_chunk_alloc doesn't figure

Re: [PATCH][v2] btrfs: add an ioctl to force chunk allocation

2019-08-05 Thread Josef Bacik
On Mon, Aug 05, 2019 at 10:14:15PM +0800, Qu Wenruo wrote: > > > On 2019/8/5 下午9:19, Josef Bacik wrote: > > In testing block group removal it's sometimes handy to be able to create > > block groups on demand. Add an ioctl to allow us to force allocation > > fro

[PATCH][v3] btrfs: add a force_chunk_alloc to space_info's sysfs

2019-08-05 Thread Josef Bacik
nable us to add testcases in the future to test these various features easier. Signed-off-by: Josef Bacik --- v2->v3: - as per Qu's suggestion, moved this to sysfs where it's easier to mess with and makes more sense. - added side-effect is mixed bg forced allocation works with thi

[PATCH 01/15] btrfs: migrate the block group caching code

2019-08-06 Thread Josef Bacik
We can now just copy it over to block-group.c. Signed-off-by: Josef Bacik --- fs/btrfs/block-group.c | 463 + fs/btrfs/block-group.h | 2 + fs/btrfs/ctree.h | 2 - fs/btrfs/extent-tree.c | 459 4 files

[PATCH 03/15] btrfs: migrate the block group removal code

2019-08-06 Thread Josef Bacik
This is the removal code and the unused bgs code. Signed-off-by: Josef Bacik --- fs/btrfs/block-group.c | 541 + fs/btrfs/block-group.h | 7 + fs/btrfs/ctree.h | 7 - fs/btrfs/extent-tree.c | 537 4 files

[PATCH 02/15] btrfs: temporarily export inc_block_group_ro

2019-08-06 Thread Josef Bacik
This is used in a few logical parts of the block group code, temporarily export it so we can move things in pieces. Signed-off-by: Josef Bacik --- fs/btrfs/block-group.h | 2 ++ fs/btrfs/extent-tree.c | 15 --- 2 files changed, 10 insertions(+), 7 deletions(-) diff --git a/fs

[PATCH 09/15] btrfs: export block group accounting helpers

2019-08-06 Thread Josef Bacik
Want to move these functions into block-group.c, so export them. Signed-off-by: Josef Bacik --- fs/btrfs/block-group.h | 6 ++ fs/btrfs/extent-tree.c | 21 ++--- 2 files changed, 16 insertions(+), 11 deletions(-) diff --git a/fs/btrfs/block-group.h b/fs/btrfs/block-group.h

[PATCH 08/15] btrfs: migrate the dirty bg writeout code

2019-08-06 Thread Josef Bacik
This can be easily migrated over now. Signed-off-by: Josef Bacik --- fs/btrfs/block-group.c | 520 + fs/btrfs/block-group.h | 3 + fs/btrfs/ctree.h | 3 - fs/btrfs/extent-tree.c | 518 4 files changed

[PATCH 06/15] btrfs: temporarily export btrfs_get_restripe_target

2019-08-06 Thread Josef Bacik
This gets used by a few different logical chunks of the block group code, export it while we move things around. Signed-off-by: Josef Bacik --- fs/btrfs/block-group.h | 2 ++ fs/btrfs/extent-tree.c | 6 +++--- 2 files changed, 5 insertions(+), 3 deletions(-) diff --git a/fs/btrfs/block-group.h

[PATCH 00/15] Migrate the block group code into it's own file

2019-08-06 Thread Josef Bacik
This is the rebased set of the much larger group of patches I sent last month. The first 10 patches are already merged, these just didn't apply cleanly. I went through and applied each one, deleted and re-copied anything that didn't merge cleanly, and compiled between each patch to make sure every

[PATCH 15/15] btrfs: remove comment and leftover cruft

2019-08-06 Thread Josef Bacik
Commit "btrfs: convert snapshot/nocow exlcusion to drw lock" removed this code, but didn't remove the comment or the definitions, do that now. Signed-off-by: Josef Bacik --- fs/btrfs/ctree.h | 3 --- fs/btrfs/extent-tree.c | 9 - 2 files changed, 12 deletions(-)

[PATCH 04/15] btrfs: export get_alloc_profile

2019-08-06 Thread Josef Bacik
This gets used directly by a bunch of the block group code, export it to make it easier to move things around. Signed-off-by: Josef Bacik --- fs/btrfs/ctree.h | 1 + fs/btrfs/extent-tree.c | 16 2 files changed, 9 insertions(+), 8 deletions(-) diff --git a/fs/btrfs

[PATCH 10/15] btrfs: migrate the block group space accounting helpers

2019-08-06 Thread Josef Bacik
We can now easily migrate this code as well. Signed-off-by: Josef Bacik --- fs/btrfs/block-group.c | 173 + fs/btrfs/extent-tree.c | 173 - 2 files changed, 173 insertions(+), 173 deletions(-) diff --git a/fs/btrfs

[PATCH 07/15] btrfs: migrate inc/dec_block_group_ro code

2019-08-06 Thread Josef Bacik
This can easily be moved now. Signed-off-by: Josef Bacik --- fs/btrfs/block-group.c | 213 + fs/btrfs/block-group.h | 2 + fs/btrfs/ctree.h | 2 - fs/btrfs/extent-tree.c | 213 - 4 files changed, 215

[PATCH 05/15] btrfs: migrate the block group read/creation code

2019-08-06 Thread Josef Bacik
All of the prep work has been done so we can now cleanly move this chunk over. Signed-off-by: Josef Bacik --- fs/btrfs/block-group.c | 598 + fs/btrfs/block-group.h | 5 + fs/btrfs/ctree.h | 5 - fs/btrfs/extent-tree.c | 596

[PATCH 13/15] btrfs: migrate the block group cleanup code

2019-08-06 Thread Josef Bacik
This can now be easily migrated as well. Signed-off-by: Josef Bacik --- fs/btrfs/block-group.c | 128 + fs/btrfs/block-group.h | 2 + fs/btrfs/ctree.h | 2 - fs/btrfs/extent-tree.c | 128 - 4 files changed

[PATCH 14/15] btrfs: unexport the temporary exported functions

2019-08-06 Thread Josef Bacik
These were renamed and exported to facilitate logical migration of different code chunks into block-group.c. Now that all the users are in one file go ahead and rename them back, move the code around, and make them static. Signed-off-by: Josef Bacik --- fs/btrfs/block-group.c | 176

[PATCH 12/15] btrfs: migrate the alloc_profile helpers

2019-08-06 Thread Josef Bacik
These feel more at home in block-group.c. Signed-off-by: Josef Bacik --- fs/btrfs/block-group.c | 100 +++ fs/btrfs/block-group.h | 16 ++ fs/btrfs/ctree.h | 4 -- fs/btrfs/extent-tree.c | 115 - 4 files

[PATCH 11/15] btrfs: migrate the chunk allocation code

2019-08-06 Thread Josef Bacik
This feels more at home in block-group.c than in extent-tree.c. Signed-off-by: Josef Bacik --- fs/btrfs/block-group.c| 245 ++ fs/btrfs/block-group.h| 24 fs/btrfs/ctree.h | 24 fs/btrfs/delalloc-space.c | 1 + fs/btrfs/extent

[PATCH] btrfs-progs: don't check nbytes on unlinked files

2019-08-09 Thread Josef Bacik
We don't update the inode when evicting it, so the nbytes will be wrong in between transaction commits. This isn't a problem, stop complaining about it to make generic/269 stop randomly failing. Signed-off-by: Josef Bacik --- check/main.c | 2 +- 1 file changed, 1 insertion(+),

[PATCH 4/7] btrfs: rework btrfs_space_info_add_old_bytes

2019-08-09 Thread Josef Bacik
e normal check to see if we can satisfy the whole reservation. If we can then we wake up the ticket and carry on. Signed-off-by: Josef Bacik --- fs/btrfs/space-info.c | 43 --- 1 file changed, 16 insertions(+), 27 deletions(-) diff --git a/fs/btrfs/space-

[PATCH 3/7] btrfs: add space reservation tracepoint for reserved bytes

2019-08-09 Thread Josef Bacik
on to have the tracepoint there. But now there is because we always emit the unreserve for the bytes_may_use side, and this would have broken if compression was on anyway. Add a tracepoint to cover the bytes_reserved counter so the math still comes out right. Signed-off-by: Josef Bacik --- fs/

[PATCH 0/7] Rework reserve ticket handling

2019-08-09 Thread Josef Bacik
While cleaning up some things around the global reserve and can_overcommit I started getting ENOSPC's with plenty of space to make reservations. The root cause of the problem has to do with how we satisfy ticket reservations. Previously we would add any space we were returning to the space info t

[PATCH 7/7] btrfs: remove orig_bytes from reserve_ticket

2019-08-09 Thread Josef Bacik
Now that we do not do partial filling of tickets simply remove orig_bytes, it is no longer needed. Signed-off-by: Josef Bacik --- fs/btrfs/space-info.c | 15 --- fs/btrfs/space-info.h | 1 - 2 files changed, 16 deletions(-) diff --git a/fs/btrfs/space-info.c b/fs/btrfs/space

[PATCH 6/7] btrfs: rework wake_all_tickets

2019-08-09 Thread Josef Bacik
Now that we no longer partially fill tickets we need to rework wake_all_tickets to call btrfs_try_to_wakeup_tickets() in order to see if any subsequent tickets are able to be satisfied. If our tickets_id changes we know something happened and we can keep flushing. Signed-off-by: Josef Bacik

[PATCH 5/7] btrfs: refactor the ticket wakeup code

2019-08-09 Thread Josef Bacik
n call the wakeup helper, and replace all calls to btrfs_space_info_add_new_bytes() with the wakeup helper. Signed-off-by: Josef Bacik --- fs/btrfs/extent-tree.c | 4 ++-- fs/btrfs/space-info.c | 53 +++--- fs/btrfs/space-info.h | 19 ++- 3 files c

[PATCH 1/7] btrfs: do not allow reservations if we have pending tickets

2019-08-09 Thread Josef Bacik
If we already have tickets on the list we don't want to steal their reservations. This is a preparation patch for upcoming changes, technically this shouldn't happen today because of the way we add bytes to tickets before adding them to the space_info in most cases. Signed-off-by: J

[PATCH 2/7] btrfs: roll tracepoint into btrfs_space_info_update helper

2019-08-09 Thread Josef Bacik
We duplicate this tracepoint everywhere we call these helpers, so update the helper to have the tracepoint as well. Signed-off-by: Josef Bacik --- fs/btrfs/block-group.c| 3 --- fs/btrfs/block-rsv.c | 5 - fs/btrfs/delalloc-space.c | 4 fs/btrfs/extent-tree.c| 9

Re: [PATCH] btrfs-progs: don't check nbytes on unlinked files

2019-08-09 Thread Josef Bacik
On Fri, Aug 09, 2019 at 06:27:14PM +0300, Nikolay Borisov wrote: > > > On 9.08.19 г. 16:18 ч., Josef Bacik wrote: > > We don't update the inode when evicting it, so the nbytes will be wrong > > in between transaction commits. This isn't a problem, stop complaini

[PATCH 1/2] fstests: make generic/500 xfs+ext4 only

2019-08-15 Thread Josef Bacik
rovisioned device is out of space this gives us an EIO, and we flip read only. We didn't remove the file, so the fstrim doesn't recover space anyway, so we can't even fstrim and remount. Make this test for ext4/xfs only, it just simply won't work right for btrfs in it's

[PATCH 2/2] fstests: move generic/500 -> shared/001

2019-08-15 Thread Josef Bacik
Now that generic/500 is not for all file systems, move it to shared. Signed-off-by: Josef Bacik --- tests/generic/group | 1 - tests/{generic/500 => shared/001} | 0 tests/{generic/500.out => shared/001.out} | 0 tests/shared/group|

Re: [PATCH 1/2] fstests: make generic/500 xfs+ext4 only

2019-08-15 Thread Josef Bacik
On Thu, Aug 15, 2019 at 08:24:25AM -0700, Darrick J. Wong wrote: > On Thu, Aug 15, 2019 at 11:00:31AM -0400, Josef Bacik wrote: > > I recently fixed some bugs in btrfs's enospc handling that made it start > > failing generic/500. > > > > The point of this test

[PATCH] fstests: generic/500 doesn't work for btrfs

2019-08-15 Thread Josef Bacik
skip this test for btrfs. Signed-off-by: Josef Bacik --- tests/generic/500 | 6 ++ 1 file changed, 6 insertions(+) diff --git a/tests/generic/500 b/tests/generic/500 index 201d8b9f..5cd7126f 100755 --- a/tests/generic/500 +++ b/tests/generic/500 @@ -49,6 +49,12 @@ _supported_os

[PATCH 0/8][v2] Rework reserve ticket handling

2019-08-16 Thread Josef Bacik
Hello, Just some minor tweaks that needed to be added to fix issues introduced by the next series of enospc fixes. v1->v2: - added "btrfs: fix may_commit_transaction to deal with no partial filling" - fixed "btrfs: refactor the ticket wakeup code" to return true if we find a smaller ticket than

[PATCH 5/8] btrfs: refactor the ticket wakeup code

2019-08-16 Thread Josef Bacik
n call the wakeup helper, and replace all calls to btrfs_space_info_add_new_bytes() with the wakeup helper. Signed-off-by: Josef Bacik --- fs/btrfs/extent-tree.c | 4 ++-- fs/btrfs/space-info.c | 53 +++--- fs/btrfs/space-info.h | 19 ++- 3 files c

[PATCH 1/8] btrfs: do not allow reservations if we have pending tickets

2019-08-16 Thread Josef Bacik
If we already have tickets on the list we don't want to steal their reservations. This is a preparation patch for upcoming changes, technically this shouldn't happen today because of the way we add bytes to tickets before adding them to the space_info in most cases. Signed-off-by: J

[PATCH 3/8] btrfs: add space reservation tracepoint for reserved bytes

2019-08-16 Thread Josef Bacik
on to have the tracepoint there. But now there is because we always emit the unreserve for the bytes_may_use side, and this would have broken if compression was on anyway. Add a tracepoint to cover the bytes_reserved counter so the math still comes out right. Signed-off-by: Josef Bacik --- fs/

[PATCH 7/8] btrfs: fix may_commit_transaction to deal with no partial filling

2019-08-16 Thread Josef Bacik
space then we are truly out of space. Signed-off-by: Josef Bacik --- fs/btrfs/space-info.c | 19 +++ 1 file changed, 19 insertions(+) diff --git a/fs/btrfs/space-info.c b/fs/btrfs/space-info.c index bd485be783b8..f79afdc04925 100644 --- a/fs/btrfs/space-info.c +++ b/fs/btrfs/

[PATCH 4/8] btrfs: rework btrfs_space_info_add_old_bytes

2019-08-16 Thread Josef Bacik
e normal check to see if we can satisfy the whole reservation. If we can then we wake up the ticket and carry on. Signed-off-by: Josef Bacik --- fs/btrfs/space-info.c | 43 --- 1 file changed, 16 insertions(+), 27 deletions(-) diff --git a/fs/btrfs/space-

[PATCH 6/8] btrfs: rework wake_all_tickets

2019-08-16 Thread Josef Bacik
is smaller than the first ticket in our queue then we want to retry the flushing loop again in case may_commit_transaction() decides we could satisfy the ticket by committing the transaction. Signed-off-by: Josef Bacik --- fs/btrfs/space-info.c | 34 +++--- 1 file

[PATCH 2/8] btrfs: roll tracepoint into btrfs_space_info_update helper

2019-08-16 Thread Josef Bacik
We duplicate this tracepoint everywhere we call these helpers, so update the helper to have the tracepoint as well. Signed-off-by: Josef Bacik --- fs/btrfs/block-group.c| 3 --- fs/btrfs/block-rsv.c | 5 - fs/btrfs/delalloc-space.c | 4 fs/btrfs/extent-tree.c| 9

[PATCH 8/8] btrfs: remove orig_bytes from reserve_ticket

2019-08-16 Thread Josef Bacik
Now that we do not do partial filling of tickets simply remove orig_bytes, it is no longer needed. Signed-off-by: Josef Bacik --- fs/btrfs/space-info.c | 15 --- fs/btrfs/space-info.h | 1 - 2 files changed, 16 deletions(-) diff --git a/fs/btrfs/space-info.c b/fs/btrfs/space

[PATCH 0/3] Rework the worst case calculations for space reservation

2019-08-16 Thread Josef Bacik
We have two worst case calculations for space reservation, one that takes into account splitting at every level when cow'ing down the btree, and another that doesn't account for splitting at all. The first is used everywhere, and the second is used mostly for truncate. However we also do not spli

[PATCH 2/3] btrfs: only reserve metadata_size for inodes

2019-08-16 Thread Josef Bacik
. This allows us to always reserve only the metadata_size amount for inode updates rather than the insert_metadata_size amount. Signed-off-by: Josef Bacik --- fs/btrfs/delalloc-space.c | 15 --- fs/btrfs/delayed-inode.c | 2 +- 2 files changed, 13 insertions(+), 4 deletions(-) diff

[PATCH 3/3] btrfs: global reserve fallback should use metadata_size

2019-08-16 Thread Josef Bacik
We only use the global reserve fallback for truncates, so use calc_metadata_size instead of calc_insert_metadata_size. Signed-off-by: Josef Bacik --- fs/btrfs/transaction.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/fs/btrfs/transaction.c b/fs/btrfs/transaction.c index

[PATCH 1/3] btrfs: rename the btrfs_calc_*_metadata_size helpers

2019-08-16 Thread Josef Bacik
_size. Also btrfs_calc_trans_metadata_size is purely for inserting items, so rename this to btrfs_calc_insert_metadata_size. Making these clearer will help when I start using them differently in upcoming patches. Signed-off-by: Josef Bacik --- fs/btrfs/block-group.c | 4 ++-- fs/btrfs/ct

[PATCH 3/5] btrfs: use add_old_bytes when updating global reserve

2019-08-16 Thread Josef Bacik
_can_ be using overcommitted space, but the add_old_bytes checks this and won't add the reservation if we're still overcommitted, so we are safe in this regard. Signed-off-by: Josef Bacik --- fs/btrfs/block-rsv.c | 8 +--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git

[PATCH 0/5] Fix global reserve size and can overcommit

2019-08-16 Thread Josef Bacik
We hit a pretty crappy corner case in production that resulted in boxes slowing down to a crawl. can_overcommit() will not allow us to overcommit if there is not enough "real" space to satisfy the global reserve. This is for hysterical raisins, we used to not be able to allocate block groups a tr

[PATCH 5/5] btrfs: add enospc debug messages for ticket failure

2019-08-16 Thread Josef Bacik
When debugging weird enospc problems it's handy to be able to dump the space info when we wake up all tickets, and see what the ticket values are. This helped me figure out cases where we were enospc'ing when we shouldn't have been. Signed-off-by: Josef Bacik --- fs/btrfs/sp

[PATCH 2/5] btrfs: always reserve our entire size for the global reserve

2019-08-16 Thread Josef Bacik
by always taking our full reservation amount for the global reserve. Signed-off-by: Josef Bacik --- fs/btrfs/block-rsv.c | 13 - 1 file changed, 4 insertions(+), 9 deletions(-) diff --git a/fs/btrfs/block-rsv.c b/fs/btrfs/block-rsv.c index 657675eef443..18a0af20ee5a 100644 --- a/fs/btrfs/b

[PATCH 4/5] btrfs: do not account global reserve in can_overcommit

2019-08-16 Thread Josef Bacik
This solves the problem we were seeing in production and keeps us from flushing as aggressively as we approach our actual metadata size usage. Signed-off-by: Josef Bacik --- fs/btrfs/space-info.c | 19 +-- 1 file changed, 1 insertion(+), 18 deletions(-) diff --git a/fs/btrf

[PATCH 1/5] btrfs: change the minimum global reserve size

2019-08-16 Thread Josef Bacik
this doesn't affect normal file systems, but for xfstests where we do things like fill up a fs and then rm * it can fall over in weird ways. This enables us for more sane behavior at extremely small file system sizes. Signed-off-by: Josef Bacik --- fs/btrfs/block-rsv.c | 22

Re: [PATCH 3/3] btrfs: global reserve fallback should use metadata_size

2019-08-16 Thread Josef Bacik
On Fri, Aug 16, 2019 at 04:35:42PM +0100, Filipe Manana wrote: > On Fri, Aug 16, 2019 at 4:08 PM Josef Bacik wrote: > > > > We only use the global reserve fallback for truncates, so use > > For truncates? > I would say only for unlinks, rmdir and removing empty block

Re: [PATCH 1/5] Btrfs: use correct count in btrfs_file_write_iter()

2019-08-16 Thread Josef Bacik
before something more > important trips on it. > > Fixes: edf064e7c6fe ("btrfs: nowait aio support") > Signed-off-by: Omar Sandoval Reviewed-by: Josef Bacik Thanks, Josef

Re: [PATCH 2/5] Btrfs: treat RWF_{,D}SYNC writes as sync for CRCs

2019-08-16 Thread Josef Bacik
the file isn't by > default. Make sure we bump the sync_writers counter in that case, too, > so that we'll do the CRCs synchronously. > > Signed-off-by: Omar Sandoval Reviewed-by: Josef Bacik Thanks, Josef

Re: [PATCH 3/5] Btrfs: stop clearing EXTENT_DIRTY in inode I/O tree

2019-08-16 Thread Josef Bacik
clear it. > > Signed-off-by: Omar Sandoval Ship this, dear lord, Reviewed-by: Josef Bacik Thanks, Josef

Re: [RFC PATCH 4/5] fs: export rw_verify_area()

2019-08-16 Thread Josef Bacik
On Thu, Aug 15, 2019 at 02:04:05PM -0700, Omar Sandoval wrote: > From: Omar Sandoval > > I'm adding a Btrfs ioctl to write compressed data, and rather than > duplicating the checks in rw_verify_area(), let's just export it. > > Signed-off-by: Omar Sandoval Revie

Re: [PATCH 1/3] btrfs: rename the btrfs_calc_*_metadata_size helpers

2019-08-19 Thread Josef Bacik
On Mon, Aug 19, 2019 at 11:30:16AM +0300, Nikolay Borisov wrote: > > > On 16.08.19 г. 18:05 ч., Josef Bacik wrote: > > btrfs_calc_trunc_metadata_size differs from trans_metadata_size in that > > it doesn't take into account any splitting at the levels, because > >

Re: [PATCH 2/3] btrfs: only reserve metadata_size for inodes

2019-08-19 Thread Josef Bacik
On Mon, Aug 19, 2019 at 12:17:07PM +0300, Nikolay Borisov wrote: > > > On 16.08.19 г. 18:05 ч., Josef Bacik wrote: > > Historically we reserved worst case for every btree operation, and > > generally speaking we want to do that in cases where it could be the > > worst

Re: [PATCH 1/8] btrfs: do not allow reservations if we have pending tickets

2019-08-19 Thread Josef Bacik
On Mon, Aug 19, 2019 at 03:54:29PM +0300, Nikolay Borisov wrote: > > > On 16.08.19 г. 17:19 ч., Josef Bacik wrote: > > If we already have tickets on the list we don't want to steal their > > reservations. This is a preparation patch for upcoming changes, > > t

Re: [PATCH 6/8] btrfs: rework wake_all_tickets

2019-08-19 Thread Josef Bacik
On Mon, Aug 19, 2019 at 05:49:45PM +0300, Nikolay Borisov wrote: > > > On 16.08.19 г. 17:19 ч., Josef Bacik wrote: > > Now that we no longer partially fill tickets we need to rework > > wake_all_tickets to call btrfs_try_to_wakeup_tickets() in order to see > > if any

Re: [PATCH 1/5] btrfs: change the minimum global reserve size

2019-08-20 Thread Josef Bacik
On Tue, Aug 20, 2019 at 04:45:15PM +0300, Nikolay Borisov wrote: > > > On 16.08.19 г. 18:20 ч., Josef Bacik wrote: > > It made sense to have the global reserve set at 16M in the past, but > > since it is used less nowadays set the minimum size to the number of > > ite

Re: [PATCH 2/5] btrfs: always reserve our entire size for the global reserve

2019-08-20 Thread Josef Bacik
On Tue, Aug 20, 2019 at 05:23:29PM +0300, Nikolay Borisov wrote: > > > On 16.08.19 г. 18:20 ч., Josef Bacik wrote: > > While messing with the overcommit logic I noticed that sometimes we'd > > ENOSPC out when really we should have run out of space much earlier. It

Re: [PATCH v2 0/2] Btrfs: workqueue cleanups

2019-08-21 Thread Josef Bacik
On Wed, Aug 21, 2019 at 04:14:53PM +0200, David Sterba wrote: > On Wed, Aug 21, 2019 at 03:24:46PM +0200, David Sterba wrote: > > On Wed, Aug 21, 2019 at 03:20:21PM +0200, David Sterba wrote: > > > On Tue, Aug 13, 2019 at 10:33:42AM -0700, Omar Sandoval wrote: > > > > From: Omar Sandoval > > > >

[PATCH][RESEND] btrfs: add a force_chunk_alloc to space_info's sysfs

2019-08-22 Thread Josef Bacik
nable us to add testcases in the future to test these various features easier. Signed-off-by: Josef Bacik --- fs/btrfs/sysfs.c | 64 1 file changed, 64 insertions(+) diff --git a/fs/btrfs/sysfs.c b/fs/btrfs/sysfs.c index f6d3c80f2e28..c290a0c

[PATCH 8/9] btrfs: remove orig_bytes from reserve_ticket

2019-08-22 Thread Josef Bacik
Now that we do not do partial filling of tickets simply remove orig_bytes, it is no longer needed. Signed-off-by: Josef Bacik Reviewed-by: Nikolay Borisov --- fs/btrfs/space-info.c | 8 fs/btrfs/space-info.h | 1 - 2 files changed, 9 deletions(-) diff --git a/fs/btrfs/space-info.c b

<    1   2   3   4   5   6   7   8   9   10   >