On 21.11.18 г. 20:59 ч., Josef Bacik wrote:
> We have a bunch of magic to make sure we're throttling delayed refs when
> truncating a file. Now that we have a delayed refs rsv and a mechanism
> for refilling that reserve simply use that instead of all of this magic.
>
> Signed-off-by: Josef
On 21.11.18 г. 20:59 ч., Josef Bacik wrote:
> From: Josef Bacik
>
> Traditionally we've had voodoo in btrfs to account for the space that
> delayed refs may take up by having a global_block_rsv. This works most
> of the time, except when it doesn't. We've had issues reported and seen
> in
Circular locking dependency check reports warning[1], that's because
the btrfs_scrub_dev() calls the stack #0 below with, the
fs_info::scrub_lock held. The test case leading to this warning..
mkfs.btrfs -fq /dev/sdb && mount /dev/sdb /btrfs
btrfs scrub start -B /btrfs
In fact we have
The fs_info::device_list_mutex and fs_info::scrub_lock creates a
nested locks in btrfs_scrub_dev(). During the lock acquire the
hierarchy is fs_info::device_list_mutex and then fs_info::scrub_lock,
so following the same reverse order during unlock, that is
fs_info::scrub_lock and then
Anand Jain (2):
btrfs: scrub: maintain the unlock order in scrub thread
btrfs: scrub: fix circular locking dependency warning
fs/btrfs/scrub.c | 19 ++-
1 file changed, 10 insertions(+), 9 deletions(-)
--
1.8.3.1
On 2018/11/26 下午3:19, Alexander Fieroch wrote:
> Hi,
>
> My data partition with btrfs RAID 0 (/dev/sdc0 and /dev/sdd0) shows
> errors in syslog:
>
> BTRFS error (device sdc): cleaner transaction attach returned -30
> BTRFS info (device sdc): disk space caching is enabled
> BTRFS info (device
On 23.11.18 г. 20:25 ч., fdman...@kernel.org wrote:
> From: Filipe Manana
>
> When a transaction commit starts, it attempts to pause scrub and it blocks
> until the scrub is paused. So while the transaction is blocked waiting for
> scrub to pause, we can not do memory allocation with
Hi,
My data partition with btrfs RAID 0 (/dev/sdc0 and /dev/sdd0) shows
errors in syslog:
BTRFS error (device sdc): cleaner transaction attach returned -30
BTRFS info (device sdc): disk space caching is enabled
BTRFS info (device sdc): has skinny extents
BTRFS info (device sdc): bdev /dev/sdc
Hi David,
Please merge this pull request:
https://github.com/kdave/btrfs-progs/pull/155
This patch can be fetch from the following branch:
https://github.com/adam900710/btrfs-progs/tree/check-next
The base commit is:
commit 5d64c40240135cc22f4ba2b902bfe20418a599ea (david/devel)
Author: David
From: Filipe Manana
When a transaction commit starts, it attempts to pause scrub and it blocks
until the scrub is paused. So while the transaction is blocked waiting for
scrub to pause, we can not do memory allocation with GFP_KERNEL from scrub,
otherwise we risk getting into a deadlock with
On Fri, Nov 23, 2018 at 3:34 PM Amir Goldstein wrote:
>
> On Fri, Nov 23, 2018 at 2:52 PM Jan Kara wrote:
> >
> > Changed subject to better match what we discuss and added btrfs list to CC.
> >
> > On Thu 22-11-18 17:18:25, Amir Goldstein wrote:
> > > On Thu, Nov 22, 2018 at 3:26 PM Jan Kara
On Thu, Nov 22, 2018 at 12:35 AM Josef Bacik wrote:
>
> I noticed in a giant dbench run that we spent a lot of time on lock
> contention while running transaction commit. This is because dbench
> results in a lot of fsync()'s that do a btrfs_transaction_commit(), and
> they all run the delayed
From: Filipe Manana
When a transaction commit starts, it attempts to pause scrub and it blocks
until the scrub is paused. So while the transaction is blocked waiting for
scrub to pause, we can not do memory allocation with GFP_KERNEL while scrub
is running, we must use GFP_NOS to avoid deadlock
On 23.11.18 г. 18:05 ч., fdman...@kernel.org wrote:
> From: Filipe Manana
>
> When a transaction commit starts, it attempts to pause scrub and it blocks
> until the scrub is paused. So while the transaction is blocked waiting for
> scrub to pause, we can not do memory allocation with
From: Filipe Manana
When a transaction commit starts, it attempts to pause scrub and it blocks
until the scrub is paused. So while the transaction is blocked waiting for
scrub to pause, we can not do memory allocation with GFP_KERNEL while scrub
is running, we must use GFP_NOS to avoid deadlock
On Wed, Nov 21, 2018 at 01:59:06PM -0500, Josef Bacik wrote:
> This patchset changes how we do space reservations for delayed refs. We were
> hitting probably 20-40 enospc abort's per day in production while running
> delayed refs at transaction commit time. This means we ran out of space in
>
On Wed, Nov 21, 2018 at 01:59:09PM -0500, Josef Bacik wrote:
> From: Josef Bacik
>
> The cleanup_extent_op function actually would run the extent_op if it
> needed running, which made the name sort of a misnomer. Change it to
> run_and_cleanup_extent_op, and move the actual cleanup work to
>
On Thu, Nov 22, 2018 at 09:06:30AM +0800, Qu Wenruo wrote:
>
>
> On 2018/11/22 上午2:59, Josef Bacik wrote:
> > From: Josef Bacik
> >
> > We were missing some quota cleanups in check_ref_cleanup, so break the
> > ref head accounting cleanup into a helper and call that from both
> >
On 23.11.18 г. 15:45 ч., David Sterba wrote:
> On Thu, Nov 22, 2018 at 11:42:28AM +0200, Nikolay Borisov wrote:
>>
>>
>> On 22.11.18 г. 11:12 ч., Nikolay Borisov wrote:
>>>
>>>
>>> On 21.11.18 г. 20:59 ч., Josef Bacik wrote:
From: Josef Bacik
We do this dance in cleanup_ref_head
On Thu, Nov 22, 2018 at 11:42:28AM +0200, Nikolay Borisov wrote:
>
>
> On 22.11.18 г. 11:12 ч., Nikolay Borisov wrote:
> >
> >
> > On 21.11.18 г. 20:59 ч., Josef Bacik wrote:
> >> From: Josef Bacik
> >>
> >> We do this dance in cleanup_ref_head and check_ref_cleanup, unify it
> >> into a
From: Filipe Manana
When a transaction commit starts, it attempts to pause scrub and it blocks
until the scrub is paused. So while the transaction is blocked waiting for
scrub to pause, we can not do memory allocation with GFP_KERNEL while scrub
is running, we must use GFP_NOS to avoid deadlock
On Fri, Nov 23, 2018 at 2:52 PM Jan Kara wrote:
>
> Changed subject to better match what we discuss and added btrfs list to CC.
>
> On Thu 22-11-18 17:18:25, Amir Goldstein wrote:
> > On Thu, Nov 22, 2018 at 3:26 PM Jan Kara wrote:
> > >
> > > On Thu 22-11-18 14:36:35, Amir Goldstein wrote:
> >
Changed subject to better match what we discuss and added btrfs list to CC.
On Thu 22-11-18 17:18:25, Amir Goldstein wrote:
> On Thu, Nov 22, 2018 at 3:26 PM Jan Kara wrote:
> >
> > On Thu 22-11-18 14:36:35, Amir Goldstein wrote:
> > > > > Regardless, IIUC, btrfs_statfs() returns an fsid which
On Fri, Nov 23, 2018 at 09:42:27AM +0100, Johannes Thumshirn wrote:
> The btrfs_bio_end_io_t typedef was introduced with commit
> a1d3c4786a4b ("btrfs: btrfs_multi_bio replaced with btrfs_bio")
> but never used anywhere. This commit also introduced a forward declaration
> of 'struct btrfs_bio'
On 2018/11/23 下午2:41, Andy Leadbetter wrote:
> I have a failing 2TB disk that is part of a 4 disk RAID 6 system. I
> have added a new 2TB disk to the computer, and started a BTRFS replace
> for the old and new disk. The process starts correctly however some
> hours into the job, there is an
Will capture all of that this evening, and try it with the latest
kernel and tools. Thanks for the input on what info is relevant, with
gather it asap.
On Fri, 23 Nov 2018 at 07:53, Chris Murphy wrote:
>
> On Thu, Nov 22, 2018 at 11:41 PM Andy Leadbetter
> wrote:
> >
> > I have a failing 2TB
On 23.11.18 г. 10:42 ч., Johannes Thumshirn wrote:
> The btrfs_bio_end_io_t typedef was introduced with commit
> a1d3c4786a4b ("btrfs: btrfs_multi_bio replaced with btrfs_bio")
> but never used anywhere. This commit also introduced a forward declaration
> of 'struct btrfs_bio' which is only
err holds the return value of either btrfs_del_root_ref() or
btrfs_del_inode_ref() but it hasn't been checked since it's
introduction with commit fe66a05a0679 (Btrfs: improve error handling
for btrfs_insert_dir_item callers) in 2012.
Instead of silently ignoring the return values, print a message
The btrfs_bio_end_io_t typedef was introduced with commit
a1d3c4786a4b ("btrfs: btrfs_multi_bio replaced with btrfs_bio")
but never used anywhere. This commit also introduced a forward declaration
of 'struct btrfs_bio' which is only needed for btrfs_bio_end_io_t.
Remove both as they're not needed
Looks good,
Reviewed-by: Johannes Thumshirn
--
Johannes ThumshirnSUSE Labs
jthumsh...@suse.de+49 911 74053 689
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg
GF: Felix Imendörffer, Jane Smithard, Graham Norton
HRB 21284 (AG
Looks good,
Reviewed-by: Johannes Thumshirn
--
Johannes ThumshirnSUSE Labs
jthumsh...@suse.de+49 911 74053 689
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg
GF: Felix Imendörffer, Jane Smithard, Graham Norton
HRB 21284 (AG
Looks good,
Reviewed-by: Johannes Thumshirn
--
Johannes ThumshirnSUSE Labs
jthumsh...@suse.de+49 911 74053 689
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg
GF: Felix Imendörffer, Jane Smithard, Graham Norton
HRB 21284 (AG
Looks good,
Reviewed-by: Johannes Thumshirn
--
Johannes ThumshirnSUSE Labs
jthumsh...@suse.de+49 911 74053 689
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg
GF: Felix Imendörffer, Jane Smithard, Graham Norton
HRB 21284 (AG
On Thu, Nov 22, 2018 at 11:41 PM Andy Leadbetter
wrote:
>
> I have a failing 2TB disk that is part of a 4 disk RAID 6 system. I
> have added a new 2TB disk to the computer, and started a BTRFS replace
> for the old and new disk. The process starts correctly however some
> hours into the job,
On Thu, Nov 22, 2018 at 6:07 AM Tomasz Chmielewski wrote:
>
> On 2018-11-22 21:46, Nikolay Borisov wrote:
>
> >> # echo w > /proc/sysrq-trigger
> >>
> >> # dmesg -c
> >> [ 931.585611] sysrq: SysRq : Show Blocked State
> >> [ 931.585715] taskPC stack pid father
> >> [
I have a failing 2TB disk that is part of a 4 disk RAID 6 system. I
have added a new 2TB disk to the computer, and started a BTRFS replace
for the old and new disk. The process starts correctly however some
hours into the job, there is an error and kernel oops. relevant log
below.
The disks are
[BUG]
A completely valid btrfs will refuse to mount, with error message like:
BTRFS critical (device sdb2): corrupt leaf: root=2 block=239681536 slot=172 \
bg_start=12018974720 bg_len=10888413184, invalid block group size, \
have 10888413184 expect (0, 10737418240]
Btrfs check returns
On 22.11.18 г. 18:16 ч., David Sterba wrote:
> The end_io callback implemented as btrfs_io_bio_endio_readpage only
> calls kfree. Also the callback is set only in case the csum buffer is
> allocated and not pointing to the inline buffer. We can use that
> information to drop the indirection and
On 22.11.18 г. 18:16 ч., David Sterba wrote:
> The io_bio tracks checksums and has an inline buffer or an allocated
> one. And there's a third member that points to the right one, but we
> don't need to use an extra pointer for that. Let btrfs_io_bio::csum
> point to the right buffer and check
On 22.11.18 г. 18:16 ч., David Sterba wrote:
> The async_cow::root is used to propagate fs_info to async_cow_submit.
> We can't use inode to reach it because it could become NULL after
> write without compression in async_cow_start.
>
> Signed-off-by: David Sterba
Reviewed-by: Nikolay
On 22.11.18 г. 18:16 ч., David Sterba wrote:
> There's one caller and its code is simple, we can open code it in
> run_one_async_done. The errors are passed through bio.
>
> Signed-off-by: David Sterba
Reviewed-by: Nikolay Borisov
> ---
> fs/btrfs/disk-io.c | 18 +-
>
The async_cow::root is used to propagate fs_info to async_cow_submit.
We can't use inode to reach it because it could become NULL after
write without compression in async_cow_start.
Signed-off-by: David Sterba
---
fs/btrfs/inode.c | 9 +++--
1 file changed, 3 insertions(+), 6 deletions(-)
The io_bio tracks checksums and has an inline buffer or an allocated
one. And there's a third member that points to the right one, but we
don't need to use an extra pointer for that. Let btrfs_io_bio::csum
point to the right buffer and check that the inline buffer is not
accidentally freed.
This
The end_io callback implemented as btrfs_io_bio_endio_readpage only
calls kfree. Also the callback is set only in case the csum buffer is
allocated and not pointing to the inline buffer. We can use that
information to drop the indirection and call a helper that will free the
csums only in the
Assorted updates to code vaguely related to the bio callbacks and
structures. Remove call indirection, remove redundant struct members,
etc.
David Sterba (4):
btrfs: merge btrfs_submit_bio_done to its caller
btrfs: replace async_cow::root with fs_info
btrfs: remove redundant csum buffer in
There's one caller and its code is simple, we can open code it in
run_one_async_done. The errors are passed through bio.
Signed-off-by: David Sterba
---
fs/btrfs/disk-io.c | 18 +-
fs/btrfs/inode.c | 23 ---
2 files changed, 17 insertions(+), 24
On Thu, Nov 22, 2018 at 09:46:44PM +0800, Qu Wenruo wrote:
> >>> it and after
> >>> we started (or joined) a transaction, a lot could of modifications may
> >>> have happened.
> >>> Nevertheless I don't think it's unexpected for anyone to have the
> >>> accounting happening
> >>> only after the
On Thu, Nov 22, 2018 at 02:44:35PM +0100, Johannes Thumshirn wrote:
> On 22/11/2018 14:35, David Sterba wrote:
> > On Thu, Nov 22, 2018 at 02:15:28PM +0100, Johannes Thumshirn wrote:
> >> err holds the return value of either btrfs_del_root_ref() or
> >> btrfs_del_inode_ref() but it hasn't been
On 2018/11/22 下午10:03, Roman Mamedov wrote:
> On Thu, 22 Nov 2018 22:07:25 +0900
> Tomasz Chmielewski wrote:
>
>> Spot on!
>>
>> Removed "discard" from fstab and added "ssd", rebooted - no more
>> btrfs-cleaner running.
>
> Recently there has been a bugfix for TRIM in Btrfs:
>
> btrfs:
On Thu, 22 Nov 2018 22:07:25 +0900
Tomasz Chmielewski wrote:
> Spot on!
>
> Removed "discard" from fstab and added "ssd", rebooted - no more
> btrfs-cleaner running.
Recently there has been a bugfix for TRIM in Btrfs:
btrfs: Ensure btrfs_trim_fs can trim the whole fs
On 2018/11/22 下午9:12, David Sterba wrote:
> On Tue, Nov 20, 2018 at 08:32:42AM +0800, Qu Wenruo wrote:
> @@ -1013,16 +1013,22 @@ int btrfs_quota_enable(struct btrfs_fs_info
> *fs_info)
> btrfs_abort_transaction(trans, ret);
> goto out_free_path;
>
On 22/11/2018 14:35, David Sterba wrote:
> On Thu, Nov 22, 2018 at 02:15:28PM +0100, Johannes Thumshirn wrote:
>> err holds the return value of either btrfs_del_root_ref() or
>> btrfs_del_inode_ref() but it hasn't been checked since it's
>> introduction with commit fe66a05a0679 (Btrfs: improve
On Thu, Nov 22, 2018 at 02:15:28PM +0100, Johannes Thumshirn wrote:
> err holds the return value of either btrfs_del_root_ref() or
> btrfs_del_inode_ref() but it hasn't been checked since it's
> introduction with commit fe66a05a0679 (Btrfs: improve error handling
> for btrfs_insert_dir_item
err holds the return value of either btrfs_del_root_ref() or
btrfs_del_inode_ref() but it hasn't been checked since it's
introduction with commit fe66a05a0679 (Btrfs: improve error handling
for btrfs_insert_dir_item callers) in 2012.
The first attempt in removing the variable was rejected, so
On Tue, Nov 20, 2018 at 08:32:42AM +0800, Qu Wenruo wrote:
> >>> @@ -1013,16 +1013,22 @@ int btrfs_quota_enable(struct btrfs_fs_info
> >>> *fs_info)
> >>> btrfs_abort_transaction(trans, ret);
> >>> goto out_free_path;
> >>> }
> >>> -
On 2018-11-22 21:46, Nikolay Borisov wrote:
# echo w > /proc/sysrq-trigger
# dmesg -c
[ 931.585611] sysrq: SysRq : Show Blocked State
[ 931.585715] task PC stack pid father
[ 931.590168] btrfs-cleaner D 0 1340 2 0x8000
[ 931.590175] Call Trace:
[
On 22.11.18 г. 14:31 ч., Tomasz Chmielewski wrote:
> Yet another system upgraded to 4.19 and showing strange issues.
>
> btrfs-cleaner is showing as ~90-100% busy in iotop:
>
> TID PRIO USER DISK READ DISK WRITE SWAPIN IO> COMMAND
> 1340 be/4 root 0.00 B/s 0.00
Yet another system upgraded to 4.19 and showing strange issues.
btrfs-cleaner is showing as ~90-100% busy in iotop:
TID PRIO USER DISK READ DISK WRITE SWAPIN IO>COMMAND
1340 be/4 root0.00 B/s0.00 B/s 0.00 % 92.88 %
[btrfs-cleaner]
Note disk read, disk write
On Wed 21-11-18 19:07:06, Phillip Potter wrote:
> Deduplicate the btrfs file type conversion implementation - file systems
> that use the same file types as defined by POSIX do not need to define
> their own versions and can use the common helper functions decared in
> fs_types.h and implemented
On 21.11.18 г. 20:59 ч., Josef Bacik wrote:
> From: Josef Bacik
>
> We use this number to figure out how many delayed refs to run, but
> __btrfs_run_delayed_refs really only checks every time we need a new
> delayed ref head, so we always run at least one ref head completely no
> matter what
On 21.11.18 г. 20:59 ч., Josef Bacik wrote:
> From: Josef Bacik
>
> The cleanup_extent_op function actually would run the extent_op if it
> needed running, which made the name sort of a misnomer. Change it to
> run_and_cleanup_extent_op, and move the actual cleanup work to
>
On 22.11.18 г. 11:12 ч., Nikolay Borisov wrote:
>
>
> On 21.11.18 г. 20:59 ч., Josef Bacik wrote:
>> From: Josef Bacik
>>
>> We do this dance in cleanup_ref_head and check_ref_cleanup, unify it
>> into a helper and cleanup the calling functions.
>>
>> Signed-off-by: Josef Bacik
>>
On 21.11.18 г. 20:59 ч., Josef Bacik wrote:
> From: Josef Bacik
>
> We do this dance in cleanup_ref_head and check_ref_cleanup, unify it
> into a helper and cleanup the calling functions.
>
> Signed-off-by: Josef Bacik
> Reviewed-by: Omar Sandoval
> ---
> fs/btrfs/delayed-ref.c | 14
On Wed, Nov 21, 2018 at 01:59:09PM -0500, Josef Bacik wrote:
>From: Josef Bacik
>
>The cleanup_extent_op function actually would run the extent_op if it
>needed running, which made the name sort of a misnomer. Change it to
>run_and_cleanup_extent_op, and move the actual cleanup work to
For data inodes this hook does nothing but to return -EAGAIN which is
used to signal to the endio routines that this bio belongs to a data
inode. If this is the case the actual retrying is handled by
bio_readpage_error. Alternatively, if this bio belongs to the btree
inode then
The first step fo the rebalance process, ensuring there is 1mb free on
each device. This number seems rather small. And in fact when talking
to the original authors their opinions were:
"man that's a little bonkers"
"i don't think we even need that code anymore"
"I think it was there to make sure
On 21.11.18 г. 21:10 ч., Josef Bacik wrote:
> I noticed in a giant dbench run that we spent a lot of time on lock
> contention while running transaction commit. This is because dbench
> results in a lot of fsync()'s that do a btrfs_transaction_commit(), and
> they all run the delayed refs
v12:
Fixed coding style - leave space between " : ".
v11:
btrfs-progs: Bring the code into the else part of if(forget).
Use strerror to print the erorr instead of ret.
v10:
Make btrfs-progs changes more readable.
With an effort to keep the known bug [1] as it is..
[1]
The
[I will send this the xfstest ML after kernel and progs patch
has been integrated].
btrfs reload was introduced to cleanup the device list inside the btrfs
kernel module.
The problem with the reload approach is that you can't run btrfs test
cases 124,125, 154 and 164 on the system with btrfs as
Support for a new command 'btrfs dev forget [dev]' is proposed here
to undo the effects of 'btrfs dev scan [dev]'. For this purpose
this patch proposes to use ioctl #5 as it was empty.
IOW(BTRFS_IOCTL_MAGIC, 5, ..)
This patch adds new ioctl BTRFS_IOC_FORGET_DEV which can be sent from
the
This patch adds cli
btrfs device forget [dev]
to remove the given device structure in the kernel if the device
is unmounted. If no argument is given it shall remove all stale
(device which are not mounted) from the kernel.
Signed-off-by: Anand Jain
Reviewed-by: Nikolay Borisov
---
v11->v12:
On 2018/11/22 上午2:59, Josef Bacik wrote:
> From: Josef Bacik
>
> We were missing some quota cleanups in check_ref_cleanup, so break the
> ref head accounting cleanup into a helper and call that from both
> check_ref_cleanup and cleanup_ref_head. This will hopefully ensure that
> we don't
Deduplicate the btrfs file type conversion implementation - file systems
that use the same file types as defined by POSIX do not need to define
their own versions and can use the common helper functions decared in
fs_types.h and implemented in fs_types.c
Acked-by: David Sterba
Signed-off-by:
Delayed iputs means we can have final iputs of deleted inodes in the
queue, which could potentially generate a lot of pinned space that could
be free'd. So before we decide to commit the transaction for ENOPSC
reasons, run the delayed iputs so that any potential space is free'd up.
If there is
The cleaner thread usually takes care of delayed iputs, with the
exception of the btrfs_end_transaction_throttle path. The cleaner
thread only gets woken up every 30 seconds, so instead wake it up to do
it's work so that we can free up that space as quickly as possible.
Reviewed-by: Filipe
I noticed in a giant dbench run that we spent a lot of time on lock
contention while running transaction commit. This is because dbench
results in a lot of fsync()'s that do a btrfs_transaction_commit(), and
they all run the delayed refs first thing, so they all contend with
each other. This
Here are some delayed iput fixes. Delayed iputs can hold reservations for a
while and there's no real good way to make sure they were gone for good, which
means we could early enospc when in reality if we had just waited for the iput
we would have had plenty of space. So fix this up by making us
The throttle path doesn't take cleaner_delayed_iput_mutex, which means
we could think we're done flushing iputs in the data space reservation
path when we could have a throttler doing an iput. There's no real
reason to serialize the delayed iput flushing, so instead of taking the
The first thing we do is loop through the list, this
if (!list_empty())
btrfs_create_pending_block_groups();
thing is just wasted space.
Reviewed-by: Nikolay Borisov
Signed-off-by: Josef Bacik
---
fs/btrfs/extent-tree.c | 3 +--
fs/btrfs/transaction.c | 6 ++
2 files changed, 3
We may abort the transaction during a commit and not have a chance to
run the pending bgs stuff, which will leave block groups on our list and
cause us accounting issues and leaked memory. Fix this by running the
pending bgs when we cleanup a transaction.
Reviewed-by: Omar Sandoval
We still need to do all of the accounting cleanup for pending block
groups if we abort. So set the ret to trans->aborted so if we aborted
the cleanup happens and everybody is happy.
Reviewed-by: Omar Sandoval
Signed-off-by: Josef Bacik
---
fs/btrfs/extent-tree.c | 8 +++-
1 file changed,
If we flip read-only before we initiate writeback on all dirty pages for
ordered extents we've created then we'll have ordered extents left over
on umount, which results in all sorts of bad things happening. Fix this
by making sure we wait on ordered extents if we have to do the aborted
A new xfstests that really hammers on transaction aborts (generic/495 I think?)
uncovered a lot of random issues. Some of these were introduced with the new
delayed refs rsv patches, others were just exposed by them, such as the pending
bg stuff. With these patches in place I stopped getting all
We weren't doing any of the accounting cleanup when we aborted
transactions. Fix this by making cleanup_ref_head_accounting global and
calling it from the abort code, this fixes the issue where our
accounting was all wrong after the fs aborts.
Signed-off-by: Josef Bacik
---
fs/btrfs/ctree.h
Instead of open coding this stuff use the helper instead.
Reviewed-by: Nikolay Borisov
Signed-off-by: Josef Bacik
---
fs/btrfs/disk-io.c | 7 +--
1 file changed, 1 insertion(+), 6 deletions(-)
diff --git a/fs/btrfs/disk-io.c b/fs/btrfs/disk-io.c
index f062fb0487cd..7d02748cf3f6 100644
---
We have this open coded in btrfs_destroy_delayed_refs, use the helper
instead.
Reviewed-by: Nikolay Borisov
Signed-off-by: Josef Bacik
---
fs/btrfs/disk-io.c | 11 ++-
1 file changed, 2 insertions(+), 9 deletions(-)
diff --git a/fs/btrfs/disk-io.c b/fs/btrfs/disk-io.c
index
We've done this forever because of the voodoo around knowing how much
space we have. However we have better ways of doing this now, and on
normal file systems we'll easily have a global reserve of 512MiB, and
since metadata chunks are usually 1GiB that means we'll allocate
metadata chunks more
With my change to no longer take into account the global reserve for
metadata allocation chunks we have this side-effect for mixed block
group fs'es where we are no longer allocating enough chunks for the
data/metadata requirements. To deal with this add a ALLOC_CHUNK_FORCE
step to the flushing
With the introduction of the per-inode block_rsv it became possible to
have really really large reservation requests made because of data
fragmentation. Since the ticket stuff assumed that we'd always have
relatively small reservation requests it just killed all tickets if we
were unable to
For FLUSH_LIMIT flushers we really can only allocate chunks and flush
delayed inode items, everything else is problematic. I added a bunch of
new states and it lead to weirdness in the FLUSH_LIMIT case because I
forgot about how it worked. So instead explicitly declare the states
that are ok for
The delayed refs rsv patches exposed a bunch of issues in our enospc
infrastructure that needed to be addressed. These aren't really one coherent
group, but they are all around flushing and reservations.
may_commit_transaction() needed to be updated a little bit, and we needed to add
a new state
With severe fragmentation we can end up with our inode rsv size being
huge during writeout, which would cause us to need to make very large
metadata reservations. However we may not actually need that much once
writeout is complete. So instead try to make our reservation, and if we
couldn't make
For enospc_debug having the block rsvs is super helpful to see if we've
done something wrong.
Signed-off-by: Josef Bacik
Reviewed-by: Omar Sandoval
Reviewed-by: David Sterba
---
fs/btrfs/extent-tree.c | 15 +++
1 file changed, 15 insertions(+)
diff --git a/fs/btrfs/extent-tree.c
We could generate a lot of delayed refs in evict but never have any left
over space from our block rsv to make up for that fact. So reserve some
extra space and give it to the transaction so it can be used to refill
the delayed refs rsv every loop through the truncate path.
Signed-off-by: Josef
may_commit_transaction will skip committing the transaction if we don't
have enough pinned space or if we're trying to find space for a SYSTEM
chunk. However if we have pending free block groups in this transaction
we still want to commit as we may be able to allocate a chunk to make
our
We have a bunch of magic to make sure we're throttling delayed refs when
truncating a file. Now that we have a delayed refs rsv and a mechanism
for refilling that reserve simply use that instead of all of this magic.
Signed-off-by: Josef Bacik
---
fs/btrfs/inode.c | 79
From: Josef Bacik
Traditionally we've had voodoo in btrfs to account for the space that
delayed refs may take up by having a global_block_rsv. This works most
of the time, except when it doesn't. We've had issues reported and seen
in production where sometimes the global reserve is exhausted
This patchset changes how we do space reservations for delayed refs. We were
hitting probably 20-40 enospc abort's per day in production while running
delayed refs at transaction commit time. This means we ran out of space in the
global reserve and couldn't easily get more space in
From: Josef Bacik
We use this number to figure out how many delayed refs to run, but
__btrfs_run_delayed_refs really only checks every time we need a new
delayed ref head, so we always run at least one ref head completely no
matter what the number of items on it. Fix the accounting to only be
From: Josef Bacik
We were missing some quota cleanups in check_ref_cleanup, so break the
ref head accounting cleanup into a helper and call that from both
check_ref_cleanup and cleanup_ref_head. This will hopefully ensure that
we don't screw up accounting in the future for other things that we
501 - 600 of 77199 matches
Mail list logo
