On 30.08.2018 20:42, Josef Bacik wrote:
> We were not handling the reserved byte accounting properly for data
> references. Metadata was fine, if it errored out the error paths would
> free the bytes_reserved count and pin the extent, but it even missed one
> of the error cases. So instead
On 30.08.2018 20:42, Josef Bacik wrote:
> If we flip read-only before we initiate writeback on all dirty pages for
> ordered extents we've created then we'll have ordered extents left over
> on umount, which results in all sorts of bad things happening. Fix this
> by making sure we wait on
From: Omar Sandoval
Hi,
This series implements swap file support for Btrfs.
Compared to v5 [1], this is pretty much feature-complete. It now
supports:
- Balance (skips block groups containing an active swap file)
- Resize (error if trying to shrink past a block group containing an
active
From: Omar Sandoval
Implement the swap file a_ops on Btrfs. Activation needs to make sure
that the file can be used as a swap file, which currently means it must
be fully allocated as nocow with no compression on one device. It must
also do the proper tracking so that ioctls will not interfere
From: Omar Sandoval
The SWP_FILE flag serves two purposes: to make swap_{read,write}page()
go through the filesystem, and to make swapoff() call
->swap_deactivate(). For Btrfs, we want the latter but not the former,
so split this flag into two. This makes us always call
->swap_deactivate() if
From: Omar Sandoval
The documentation for these functions is wrong in several ways:
- swap_activate() is called with the inode locked
- swap_activate() takes a swap_info_struct * and a sector_t *
- swap_activate() can also return a positive number of extents it added
itself
-
From: Omar Sandoval
When a swap file is active, we must make sure that the extents of the
file are not moved and that they don't become shared. That means that
the following are not safe:
- chattr +c (enable compression)
- reflink
- dedupe
- snapshot
- defrag
Don't allow those to happen on an
From: Omar Sandoval
The Btrfs swap code is going to need it, so give it a btrfs_ prefix and
make it non-static.
Signed-off-by: Omar Sandoval
---
fs/btrfs/volumes.c | 22 +++---
fs/btrfs/volumes.h | 9 +
2 files changed, 20 insertions(+), 11 deletions(-)
diff --git
From: Omar Sandoval
Btrfs will need this for swap file support.
Signed-off-by: Omar Sandoval
---
mm/swapfile.c | 1 +
1 file changed, 1 insertion(+)
diff --git a/mm/swapfile.c b/mm/swapfile.c
index d3f95833d12e..51cb30de17bc 100644
--- a/mm/swapfile.c
+++ b/mm/swapfile.c
@@ -2365,6 +2365,7
Well... It seems it's not the hardware.
I ran a long SMART check which ran through without errors and
reallocation count is still 0.
So I used clonezilla (partclone.btrfs) to mirror the drive to another
drive (same model).
Everything copied over just fine. No I/O error im dmesg.
The new disk
On Fri, Sep 7, 2018 at 3:56 AM, Jürgen Herrmann wrote:
> Hello!
>
> I'm having a problem with btrfs send which stops after several seconds.
> The process hangs with 100% cpu time on one cpu. The system is still
> responsive to input but no io is happening anymore so the system
> basically becomes
On Fri, Sep 07, 2018 at 11:39:25AM +0300, Nikolay Borisov wrote:
>
>
> On 7.09.2018 10:39, Omar Sandoval wrote:
> > From: Omar Sandoval
> >
> > Implement the swap file a_ops on Btrfs. Activation needs to make sure
> > that the file can be used as a swap file, which currently means it must
> >
On Sat, Sep 08, 2018 at 01:59:50AM +0800, Liu Bo wrote:
> kcalloc is defined as
> kcalloc(size_t n, size_t size, gfp_t flags)
>
> Although this won't cause problems in practice, btrfsic_read_block()
> has switched n with size.
>
> This updates btrfsic_read_block() to be correct in using kcalloc.
On Fri, Sep 7, 2018 at 11:21 AM, Omar Sandoval wrote:
> On Sat, Sep 08, 2018 at 01:59:50AM +0800, Liu Bo wrote:
>> kcalloc is defined as
>> kcalloc(size_t n, size_t size, gfp_t flags)
>>
>> Although this won't cause problems in practice, btrfsic_read_block()
>> has switched n with size.
>>
>>
On Fri, Sep 7, 2018 at 6:47 AM, Stefan Loewen wrote:
> Well... It seems it's not the hardware.
> I ran a long SMART check which ran through without errors and
> reallocation count is still 0.
That only checks the drive, it's an internal test. It doesn't check
anything else, including
kcalloc is defined as
kcalloc(size_t n, size_t size, gfp_t flags)
Although this won't cause problems in practice, btrfsic_read_block()
has switched n with size.
This updates btrfsic_read_block() to be correct in using kcalloc.
Signed-off-by: Liu Bo
---
fs/btrfs/check-integrity.c | 6 +++---
1
On Fri, Sep 07, 2018 at 11:04:28AM -0700, Omar Sandoval wrote:
> On Fri, Sep 07, 2018 at 11:39:25AM +0300, Nikolay Borisov wrote:
> >
> >
> > On 7.09.2018 10:39, Omar Sandoval wrote:
> > > From: Omar Sandoval
> > >
> > > Implement the swap file a_ops on Btrfs. Activation needs to make sure
>
List of steps:
- 3.8G iso lays in read-only subvol A
- I create subvol B and reflink-copy the iso into it.
- I create a read-only snapshot C of B
- I "btrfs send --no-data C > /somefile"
So you got that right, yes.
Unfortunately I don't have any way to connect the drive to a SATA port
directly
On Fri, Sep 07, 2018 at 12:51:54PM +0200, Jürgen Herrmann wrote:
> Am 7.9.2018 12:38, schrieb David Sterba:
> > On Fri, Sep 07, 2018 at 11:56:18AM +0200, Jürgen Herrmann wrote:
> >> I'm on linux mint 19, freshly installed on a new laptop about 6 weeks
> >> ago. So i wanted to compile latest
The series peels off the custom locking that's used for dev-replace and
uses read-write semaphore in the end.
I've mainly focused on correctness and haven't measured the performance
effects. There should be none as the blocking and waiting was merely
open coding what the rw semaphore does, but
There's a single caller and the function name does not say it's actually
taking the lock, so open coding makes it more explicit.
For now, btrfs_dev_replace_read_lock is used instead of read_lock so
it's paired with the unlocking wrapper in the same block.
Signed-off-by: David Sterba
---
Too trivial, the purpose can be simply documented in a comment.
Signed-off-by: David Sterba
---
fs/btrfs/dev-replace.c | 8
fs/btrfs/dev-replace.h | 1 -
fs/btrfs/transaction.c | 5 -
3 files changed, 4 insertions(+), 10 deletions(-)
diff --git a/fs/btrfs/dev-replace.c
The exit sequence in btrfs_dev_replace_start does not allow to simply
add a label to the right place so the error handling after starting
transaction failure jumps there. Currently there's a lock that pairs
with the unlock in the section, which is unnecessary and only raises
questions. Add a
On Fri, Sep 07, 2018 at 09:27:28AM +0530, Lakshmipathi.G wrote:
> > One question:
> > Why not ioctl_fideduperange?
> > i.e. you kill most of benefits from that ioctl - atomicity.
> >
> I plan to add fideduperange as an option too. User can
> choose between fideduperange and ficlonerange call.
>
On Thu, Sep 06, 2018 at 09:25:45PM +0800, Qu Wenruo wrote:
> > The deprecation should be done in a few steps. First issue a warning
> > that the feature is deprecated and will be removed in release X. Then
> > wait until somebody complains (or not) and remove the code in release X.
> >
> > The X
This member seems to be copied from the extent_buffer locking scheme and
is at least used to assert that the read lock/unlock is properly nested.
In some way. While the _inc/_dec are called inside the read lock
section, the asserts are both inside and outside, so the ordering is not
guaranteed and
The wrapper is too trivial, open coding does not make it less readable.
Signed-off-by: David Sterba
---
fs/btrfs/dev-replace.h | 5 -
fs/btrfs/scrub.c | 11 ---
2 files changed, 4 insertions(+), 12 deletions(-)
diff --git a/fs/btrfs/dev-replace.h b/fs/btrfs/dev-replace.h
The read lock is going to use rw semaphore that might sleep, this is not
possible in the radix tree preload section. The lock nesting is now:
* device replace
* radix tree preload
* readahead spinlock
Signed-off-by: David Sterba
---
fs/btrfs/reada.c | 12 +++-
1 file changed, 7
After the rw semaphore has been added, the custom blocking using
::blocking_readers and ::read_lock_wq is redundant.
The blocking logic in __btrfs_map_block is replaced by extending the
time the semaphore is held, that has the same blocking effect on writes
as the previous custom scheme that
The value of blocking_readers is increased only when the lock is taken
for read, no way we can fail the condition with the write lock.
Signed-off-by: David Sterba
---
fs/btrfs/dev-replace.c | 1 -
1 file changed, 1 deletion(-)
diff --git a/fs/btrfs/dev-replace.c b/fs/btrfs/dev-replace.c
index
The replace_wait and bio_counter were mistakenly added to fs_info in
commit c404e0dc2c843b154f ("Btrfs: fix use-after-free in the finishing
procedure of the device replace"), but they logically belong to
fs_info::dev_replace. Besides, bio_counter is a very generic name and is
confusing in bare
The dev-replace locking functions are now trivial wrappers around rw
semaphore that can be used directly everywhere. No functional change.
Signed-off-by: David Sterba
---
fs/btrfs/dev-replace.c | 73 --
fs/btrfs/dev-replace.h | 4 ---
fs/btrfs/reada.c
On 9/7/18 8:00 AM, David Sterba wrote:
> On Thu, Sep 06, 2018 at 04:59:33PM -0400, je...@suse.com wrote:
>> From: Jeff Mahoney
>
> If this is a resend, I can't find the previous postings, same or similar
> subject.
I had tagged it as submitted in March, but I can't find any posting of
it
sorry for disturb this discussion,
are there any plans/dates to fix the raid5/6 issue? Is somebody working on this
issue? Cause this is for me one of the most important things for a fileserver,
with a raid1 config I loose to much diskspace.
best regards
Stefan
Am 7.9.2018 13:13, schrieb Qu Wenruo:
On 2018/9/7 下午6:45, Jürgen Herrmann wrote:
Am 7.9.2018 12:15, schrieb Qu Wenruo:
On 2018/9/7 下午5:56, Jürgen Herrmann wrote:
Hello!
I'm having a problem with btrfs send which stops after several
seconds.
The process hangs with 100% cpu time on one cpu.
On 7.09.2018 10:39, Omar Sandoval wrote:
> From: Omar Sandoval
>
> The Btrfs swap code is going to need it, so give it a btrfs_ prefix and
> make it non-static.
>
> Signed-off-by: Omar Sandoval
One minor nit but otherwise:
Reviewed-by: Nikolay Borisov
> ---
> fs/btrfs/volumes.c | 22
On 7.09.2018 10:39, Omar Sandoval wrote:
> From: Omar Sandoval
>
> Implement the swap file a_ops on Btrfs. Activation needs to make sure
> that the file can be used as a swap file, which currently means it must
> be fully allocated as nocow with no compression on one device. It must
> also
Introduce new function, qgroup_trace_new_subtree_blocks(), to iterate
all new tree blocks in a reloc tree.
So that qgroup could skip unrelated tree blocks during balance, which
should hugely speedup balance speed when quota is enabled.
The function qgroup_trace_new_subtree_blocks() itself only
This patchset can be fetched from github:
https://github.com/adam900710/linux/tree/qgroup_balance_skip_trees
The base commit is v4.19-rc1 tag.
There are a lot of reports of system hang for balance on quota enabled
fs.
It's most obvious for large fs.
The hang is caused by tons of unmodified
Before this patch, for quota enabled balance, btrfs needs to mark the
whole subtree dirty for quota.
E.g.
OO = Old tree blocks (from file tree)
NN = New tree blocks (from reloc tree)
File tree (src) Reloc tree (dst)
OO (a) NN (a)
Introduce a new function, qgroup_trace_extent_swap(), which will be used
later for balance qgroup speedup.
The basis idea of balance is swapping tree blocks between reloc tree and
the real file tree.
The swap will happen in highest tree block, but there may be a lot of
tree blocks involved.
For
Number of qgroup dirty extents is directly linked to the performance
overhead, so add a new trace event, trace_qgroup_num_dirty_extents(), to
record how many dirty extents is processed in
btrfs_qgroup_account_extents().
This will be pretty handy to analyse later balance performance
improvement.
Reloc tree doesn't contribute to qgroup numbers, as we have
accounted them at balance time (check replace_path()).
Skip such unneeded subtree trace should reduce some performance
overhead.
Signed-off-by: Qu Wenruo
---
fs/btrfs/extent-tree.c | 8 +++-
1 file changed, 7 insertions(+), 1
This patchset can be fetched from github:
https://github.com/adam900710/linux/tree/trim_fix
Which is based on v4.19-rc1 tag.
This patchset introduces 2 enhancement, one to output better error
messages during trim, the other one is to ensure we could really trim
block groups if logical bytenr of
Function btrfs_trim_fs() doesn't handle errors in a consistent way, if
error happens when trimming existing block groups, it will skip the
remaining blocks and continue to trim unallocated space for each device.
And the return value will only reflect the final error from device
trimming.
This
[BUG]
fstrim on some btrfs only trims the unallocated space, not trimming any
space in existing block groups.
[CAUSE]
Before fstrim_range passed to btrfs_trim_fs(), it get truncated to
range [0, super->total_bytes).
So later btrfs_trim_fs() will only be able to trim block groups in range
[0,
On 2018/9/7 下午5:56, Jürgen Herrmann wrote:
> Hello!
>
> I'm having a problem with btrfs send which stops after several seconds.
> The process hangs with 100% cpu time on one cpu. The system is still
> responsive to input but no io is happening anymore so the system
> basically becomes
On Fri, Sep 07, 2018 at 11:56:18AM +0200, Jürgen Herrmann wrote:
> I'm on linux mint 19, freshly installed on a new laptop about 6 weeks
> ago. So i wanted to compile latest btrfs-progs and latest stable kernel
> first before reporting that here, BUT: I cannot compile btrfs-progs
> 4.17.1
>
Am 7.9.2018 12:15, schrieb Qu Wenruo:
On 2018/9/7 下午5:56, Jürgen Herrmann wrote:
Hello!
I'm having a problem with btrfs send which stops after several
seconds.
The process hangs with 100% cpu time on one cpu. The system is still
responsive to input but no io is happening anymore so the
Am 7.9.2018 12:38, schrieb David Sterba:
On Fri, Sep 07, 2018 at 11:56:18AM +0200, Jürgen Herrmann wrote:
I'm on linux mint 19, freshly installed on a new laptop about 6 weeks
ago. So i wanted to compile latest btrfs-progs and latest stable
kernel
first before reporting that here, BUT: I
Am 7.9.2018 12:15, schrieb Qu Wenruo:
On 2018/9/7 下午5:56, Jürgen Herrmann wrote:
Hello!
I'm having a problem with btrfs send which stops after several
seconds.
The process hangs with 100% cpu time on one cpu. The system is still
responsive to input but no io is happening anymore so the
Hello!
I'm having a problem with btrfs send which stops after several seconds.
The process hangs with 100% cpu time on one cpu. The system is still
responsive to input but no io is happening anymore so the system
basically becomes unuseable.
I'm on linux mint 19, freshly installed on a new
On Fri, Sep 07, 2018 at 10:54:52AM +0300, Nikolay Borisov wrote:
>
>
> On 7.09.2018 10:39, Omar Sandoval wrote:
> > From: Omar Sandoval
> >
> > The Btrfs swap code is going to need it, so give it a btrfs_ prefix and
> > make it non-static.
> >
> > Signed-off-by: Omar Sandoval
>
> One minor
Prepare to deprecate the following btrfs_qgroup_inherit features:
1) Flags
There is only one flag supported, BTRFS_QGROUP_INHERIT_SET_LIMITS,
however it's never used by btrfs-progs.
2) num_excl|ref_copies
These two features are going to copy excl/rfer numbers from one
qgroup to
btrfs_qgroup_inherit structure doesn't goes through much validation
check.
Now do a comprehensive check for it, including:
1) Inherit size
Should not exceeding SZ_4K and its num_qgroups should not
exceed its size passed in btrfs_ioctl_vol_args_v2.
2) Flags
Should not include any unknown
This patchset can be fetched from github:
https://github.com/adam900710/linux/tree/qgroup_inherit_check
Which is based on v4.19-rc1 tag.
This patchset will firstly set btrfs_qgroup_inherit structure size limit
from PAGE_SIZE to fixed SZ_4K.
I understand this normally will cause compatibility
Change btrfs_qgroup_inherit maximum size from PAGE_SIZE to SZ_4K to make
it consistent across different architectures.
This change will only affect architectures whose page size is not
(larger than) 4K, and will only affect how many qgroups can exist in
btrfs_qgroup_inherit structure:
Before:
On Fri, Aug 31, 2018 at 04:00:29PM -0700, Omar Sandoval wrote:
> On Thu, Aug 30, 2018 at 01:41:53PM -0400, Josef Bacik wrote:
> > From: Josef Bacik
> >
> > Unify the extent_op handling as well, just add a flag so we don't
> > actually run the extent op from check_ref_cleanup and instead return a
On Fri, Sep 7, 2018 at 11:07 AM, Stefan Loewen wrote:
> List of steps:
> - 3.8G iso lays in read-only subvol A
> - I create subvol B and reflink-copy the iso into it.
> - I create a read-only snapshot C of B
> - I "btrfs send --no-data C > /somefile"
> So you got that right, yes.
OK I can't
kcalloc is defined as
kcalloc(size_t n, size_t size, gfp_t flags)
Although this won't cause problems in practice, btrfsic_read_block()
has switched n with size.
This updates btrfsic_read_block() to be correct in using kcalloc.
Reviewed-by: Omar Sandoval
Signed-off-by: Liu Bo
---
v2: use
On Fri, Aug 31, 2018 at 10:29:30AM +0800, Qu Wenruo wrote:
> Since btrfs_validate_inherit() will not allow features like copy
> rfer/excl and limit set, remove these dead code.
Reviewed-by: Omar Sandoval
> Signed-off-by: Qu Wenruo
> ---
> fs/btrfs/qgroup.c | 57
On Fri, Sep 07, 2018 at 04:55:06PM +0200, David Sterba wrote:
> The wrapper is too trivial, open coding does not make it less readable.
Reviewed-by: Omar Sandoval
> Signed-off-by: David Sterba
> ---
> fs/btrfs/dev-replace.h | 5 -
> fs/btrfs/scrub.c | 11 ---
> 2 files
On Fri, Sep 07, 2018 at 04:55:09PM +0200, David Sterba wrote:
> Too trivial, the purpose can be simply documented in a comment.
Reviewed-by: Omar Sandoval
> Signed-off-by: David Sterba
> ---
> fs/btrfs/dev-replace.c | 8
> fs/btrfs/dev-replace.h | 1 -
> fs/btrfs/transaction.c | 5
On Fri, Aug 31, 2018 at 10:29:29AM +0800, Qu Wenruo wrote:
> btrfs_qgroup_inherit structure doesn't goes through much validation
> check.
>
> Now do a comprehensive check for it, including:
> 1) inherit size
>Should not exceeding SZ_4K and its num_qgroups should not
>exceed its size
On Fri, Sep 07, 2018 at 04:55:11PM +0200, David Sterba wrote:
> The exit sequence in btrfs_dev_replace_start does not allow to simply
> add a label to the right place so the error handling after starting
> transaction failure jumps there. Currently there's a lock that pairs
> with the unlock in
On Fri, Sep 07, 2018 at 04:55:13PM +0200, David Sterba wrote:
> The replace_wait and bio_counter were mistakenly added to fs_info in
> commit c404e0dc2c843b154f ("Btrfs: fix use-after-free in the finishing
> procedure of the device replace"), but they logically belong to
> fs_info::dev_replace.
On 2018/9/8 上午4:50, Omar Sandoval wrote:
> On Fri, Aug 31, 2018 at 10:29:29AM +0800, Qu Wenruo wrote:
>> btrfs_qgroup_inherit structure doesn't goes through much validation
>> check.
>>
>> Now do a comprehensive check for it, including:
>> 1) inherit size
>>Should not exceeding SZ_4K and its
On Fri, Sep 07, 2018 at 09:27:28AM +0530, Lakshmipathi.G wrote:
> >
> > One question:
> > Why not ioctl_fideduperange?
> > i.e. you kill most of benefits from that ioctl - atomicity.
> >
> I plan to add fideduperange as an option too. User can
> choose between fideduperange and ficlonerange
On 2018/9/7 下午6:45, Jürgen Herrmann wrote:
> Am 7.9.2018 12:15, schrieb Qu Wenruo:
>> On 2018/9/7 下午5:56, Jürgen Herrmann wrote:
>>> Hello!
>>>
>>> I'm having a problem with btrfs send which stops after several seconds.
>>> The process hangs with 100% cpu time on one cpu. The system is still
>>>
On Fri, Sep 07, 2018 at 12:01:21PM +0900, Misono Tomohiro wrote:
> On 2018/09/07 0:57, David Sterba wrote:
> > On Thu, Aug 30, 2018 at 10:59:16AM +0900, Misono Tomohiro wrote:
> >> Since commit d7df2c796d7e ("Btrfs attach delayed ref updates to
> >> delayed ref heads"), check_delaed_ref() won't
On Thu, Sep 06, 2018 at 04:59:33PM -0400, je...@suse.com wrote:
> From: Jeff Mahoney
If this is a resend, I can't find the previous postings, same or similar
subject.
> When we hit an I/O error in free_log_tree->walk_log_tree during file system
> shutdown we can crash due to there not being a
71 matches
Mail list logo
