Hi!
I got a reproducible infinite hang, reliably triggered by the testsuite of
"flatpak"; fails on at least 4.15-rc6, 4.9.75, and on another machine with
Debian's 4.14.2-1.
[580632.355107] INFO: task kworker/u8:2:11105 blocked for more than 120 seconds.
[580632.355120] Not tainted 4.14.0-1-a
I created a subvolume on a btrfs, set a limit and the quota is enforced -
dumping too much data into the subvolume results in a 'quota exceeded' message
(from dd, for example). But when I am trying to get netlink socket
notifications, nothing arrives on the socket (I am using pyroute2 which is
Although
commit e6c4efd87ab0 ("btrfs: Fix and enhance merge_extent_mapping() to insert
best fitted extent map")
fixed up the negetive em->len, it has introduced several regressions, several
has been fixed by
commit 32be3a1ac6d0 ("btrfs: Fix the wrong condition judgment about subset
extent map")
These helpers are extent map specific, move them to extent_map.c.
Signed-off-by: Liu Bo
---
fs/btrfs/ctree.h | 2 -
fs/btrfs/extent_map.c | 125 ++
fs/btrfs/extent_map.h | 2 +
fs/btrfs/inode.c | 107 --
This is a subtle case, so in order to understand the problem, it'd be good
to know the content of existing and em when any error occurs.
Signed-off-by: Liu Bo
---
v2: Remove unnecessary KERN_INFO.
fs/btrfs/extent_map.c | 8 +++-
1 file changed, 7 insertions(+), 1 deletion(-)
diff --git a/f
In order to debug subtle bugs around merge_extent_mapping(), perf probe
can be used to check the arguments, but sometimes merge_extent_mapping()
got inlined by compiler and couldn't be probed.
This is adding noinline attribute to merge_extent_mapping().
Signed-off-by: Liu Bo
---
fs/btrfs/extent
This test case simulates the racy situation of dio write vs dio read,
and see if btrfs_get_extent() would return -EEXIST.
Signed-off-by: Liu Bo
---
fs/btrfs/tests/extent-map-tests.c | 88 +++
1 file changed, 88 insertions(+)
diff --git a/fs/btrfs/tests/extent
This test case simulates the racy situation of buffered write vs dio
read, and see if btrfs_get_extent() would return -EEXIST.
Signed-off-by: Liu Bo
---
fs/btrfs/tests/extent-map-tests.c | 73 +++
1 file changed, 73 insertions(+)
diff --git a/fs/btrfs/tests/e
We've observed that btrfs_get_extent() and merge_extent_mapping() could
return -EEXIST in several cases, and they are caused by some racy
condition, e.g dio read vs dio write, which makes the problem very tricky
to reproduce.
This adds extent map selftests in order to simulate those racy situation
%block_len could be checked on deciding if two em are mergeable.
merge_extent_mapping() has only added the front pad if the front part
of em gets truncated, but it's possible that the end part gets
truncated.
For both compressed extent and inline extent, em->block_len is not
adjusted accordingly,
This fixes a corner case that is caused by a race of dio write vs dio
read/write.
Here is how the race could happen.
Suppose that no extent map has been loaded into memory yet.
There is a file extent [0, 32K), two jobs are running concurrently
against it, t1 is doing dio write to [8K, 32K) and t2
This is adding a tracepoint 'btrfs_handle_em_exist' to help debug the
subtle bugs around merge_extent_mapping.
Signed-off-by: Liu Bo
---
fs/btrfs/extent_map.c| 3 +++
include/trace/events/btrfs.h | 35 +++
2 files changed, 38 insertions(+)
diff --git a/f
This is a prepare work for the following extent map selftest, which
runs tests against em merge logic.
Signed-off-by: Liu Bo
---
fs/btrfs/ctree.h | 2 ++
fs/btrfs/inode.c | 80
2 files changed, 48 insertions(+), 34 deletions(-)
diff --gi
Hi,
btrfs-progs version 4.14.1 have been released.
Changes:
* dump-tree: print times of root items
* check: fix several lowmem mode bugs
* convert: fix rollback after balance
* other
* new and updated tests, enabled lowmem mode in CI
* docs updates
* fix travis CI build
*
On Wed, Nov 15, 2017 at 04:28:11PM -0700, Liu Bo wrote:
> We've avoided data losing raid profile when doing balance, but it
> turns out that deleting a device could also result in the same
> problem.
>
> Say we have 3 disks, and they're created with '-d raid1' profile.
>
> - We have chunk P (the
Hi,
we have two more fixes for 4.15, aimed for stable. The leak fix is
obvious, the second patch fixes a bug revealed by the refcount API, when
it behaves differently than previous atomic_t and reports refs going
from 0 to 1 in one case.
No merge conflicts. Please pull, thanks.
The following cha
09
We now need Invoice.
http://www.petewalker.me/Invoices-attached/
This correspondence and any files transmitted with it are confidential and
intended solely for the use of the intended recipient(s) to whom it is
addressed.
Emma Aiden
On Sat, Nov 04, 2017 at 04:07:47AM +, fdman...@kernel.org wrote:
> From: Filipe Manana
>
> This implements support the zero range operation of fallocate. For now
> at least it's as simple as possible while reusing most of the existing
> fallocate and hole punching infrastructure.
>
> Signed-
On Tue, Dec 05, 2017 at 01:51:43PM +0200, Nikolay Borisov wrote:
> add_pending_csums was added as part of the new data=ordered implementation in
> e6dcd2dc9c48 ("Btrfs: New data=ordered implementation"). Even back then it
> called the btrfs_csum_file_blocks which can fail but it never bothered
> h
On Mon, Dec 04, 2017 at 03:40:34PM -0700, Liu Bo wrote:
> Patch 1 is a simple cleanup.
> Patch 2 fixes a bug in raid56 rbio merging code.
> Patch 3 fixes a bug in raid6 reconstruction process which can end up
> read failure when it can rebuild up good data.
>
> Liu Bo (3):
> Btrfs: remove redund
On Fri, Jan 05, 2018 at 05:24:22PM +0800, Anand Jain wrote:
> btrfs-progs uses additional two super flag bits. So just define
> that so that we know its been used.
>
> Signed-off-by: Anand Jain
> ---
> The btrfs-progs commits (very old) introduced them,
>
> commit 7cc792872a133cabc3467e6ccaf5a2
On Fri, Dec 15, 2017 at 03:40:16PM +0800, Anand Jain wrote:
> Assign ret = -EINVAL where actually its required.
> Remove { } around single line if else code.
>
> Signed-off-by: Anand Jain
Added to next, thanks.
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body
On Fri, Dec 15, 2017 at 03:40:15PM +0800, Anand Jain wrote:
> Move uuid_mutex closer to the exclusion section.
Looks good, there's really something unrelated inside the critical
section so this could potentially speed up scanning devices.
Reviewed-by: David Sterba
--
To unsubscribe from this lis
On Fri, Dec 15, 2017 at 03:40:14PM +0800, Anand Jain wrote:
> No functional change. First set the usual case, writeable then check
> for any special config.
>
> Signed-off-by: Anand Jain
> ---
> fs/btrfs/volumes.c | 8 +++-
> 1 file changed, 3 insertions(+), 5 deletions(-)
>
> diff --git a/
Hi,
My filesytem appears to have become corrupted and btrfs check appears
to get stuck in an infinite loop trying to repair it.
The issue initially manifested itself as a BUG (RIP:
btrfs_set_item_key_safe+0x132/0x190) on v4.14.8 - see attached
dmesg.txt. I do not know whether this is the cause of
On Wed, Jan 3, 2018 at 3:39 PM, Timofey Titovets wrote:
> 2018-01-03 14:40 GMT+03:00 Filipe Manana :
>> On Thu, Dec 28, 2017 at 3:28 PM, Timofey Titovets
>> wrote:
>>> Insert sort are generaly perform better then bubble sort,
>>> by have less iterations on avarage.
>>> That version also try plac
On Wed, Dec 20, 2017 at 10:19:14AM +0200, Nikolay Borisov wrote:
>
>
> On 20.12.2017 08:42, Anand Jain wrote:
> > No functional change rearrange the mutex_unlock.
> >
> > Signed-off-by: Anand Jain
>
> Reviewed-by: Nikolay Borisov
Added to for-next.
--
To unsubscribe from this list: send the
On Wed, Jan 03, 2018 at 04:08:30PM +0800, Anand Jain wrote:
> btrfs_device::scrub_device is not a device which is being scrubbed,
> but it holds the scrub context, so rename to reflect the same. No
> functional changes here.
>
> Signed-off-by: Anand Jain
Reviewed-by: David Sterba
--
To unsubscr
On Thu, Jan 04, 2018 at 02:11:21PM +0200, Nikolay Borisov wrote:
>
>
> On 4.01.2018 12:01, Anand Jain wrote:
> > There is no other consumer for btrfs_handle_error() other than
> > __btrfs_handle_fs_error(), further this function quite small.
> > Merge it into its parent.
> >
> > Signed-off-by:
On Tue, Jan 02, 2018 at 01:36:41PM -0700, Liu Bo wrote:
> There is a scenario that can end up with rebuild process failing to
> return good content, i.e.
> suppose that all disks can be read without problems and if the content
> that was read out doesn't match its checksum, currently for raid6
> bt
On Thu, Jan 04, 2018 at 09:38:46AM +0800, Qu Wenruo wrote:
> >> - uuid_le uuid;
> >> + uuid_le uuid = { 0 };
> >
> > I get a warning with gcc 4.8.5
> >
> > fs/btrfs/disk-io.c:1236:2: warning: missing braces around initializer
> > [-Wmissing-braces]
> >
> > but no warning with gcc 7.2.1 (built
From: Qu Wenruo
Introduce new functions, check/recover_tree_mirror(), to check and
recover mirror-based tree blocks (Single/DUP/RAID0/1/10).
check_tree_mirror() can also be used on in-memory tree blocks using @data
parameter.
This is very handy for RAID5/6 case, either checking the data stripe
t
From: Qu Wenruo
Introduce a new function, scrub_one_extent(), as a wrapper to check one
mirror-based extent.
It will accept a btrfs_path parameter @path, which must point to a
META/EXTENT_ITEM.
And @start, @len, which must be a subset of META/EXTENT_ITEM.
Signed-off-by: Qu Wenruo
Signed-off-by
From: Qu Wenruo
Introuduce new local structures, scrub_full_stripe and scrub_stripe, for
incoming offline RAID56 scrub support.
For pure stripe/mirror based profiles, like raid0/1/10/dup/single, we
will follow the original bytenr and mirror number based iteration, so
they don't need any extra st
From: Qu Wenruo
Introduce new function, verify_parities(), to check whether parities match
with full stripe, whose data stripes match with their csum.
Caller should fill the scrub_full_stripe structure properly before
calling this function.
Signed-off-by: Qu Wenruo
Signed-off-by: Gu Jinxiang
From: Qu Wenruo
Introduce new function, scrub_one_block_group(), to scrub a block group.
For Single/DUP/RAID0/RAID1/RAID10, we use old mirror number based
map_block, and check extent by extent.
For parity based profile (RAID5/6), we use new map_block_v2() and check
full stripe by full stripe.
From: Qu Wenruo
Introduce a new function, btrfs_check_extent_exists(), to check if there
is any extent in the range specified by user.
The parameter can be a large range, and if any extent exists in the
range, it will return >0 (in fact it will return 1).
Or return 0 if no extent is found.
Sign
From: Qu Wenruo
Introduce new function, scrub_one_data_stripe(), to check all data and
tree blocks inside the data stripe.
This function will not try to recovery any error, but only check if any
data/tree blocks has mismatch csum.
If data missing csum, which is completely valid for case like no
From: Qu Wenruo
Introduce a new function: btrfs_read_data_csums(), to read out csums
for sectors in range.
This is quite useful for read out data csum so we don't need to do it
using open code.
Signed-off-by: Qu Wenruo
Signed-off-by: Su Yue
Signed-off-by: Gu Jinxiang
---
Makefile | 2
From: Qu Wenruo
Introduce function, recover_from_parities(), to recover data stripes.
It just wraps raid56_recov() with extra check functions to
scrub_full_stripe structure.
Signed-off-by: Qu Wenruo
---
scrub.c | 51 +++
1 file changed, 51 inser
Add a test for offline-scrub.
The process of this test case:
1)create a filesystem with profile raid10
2)mount the filesystem, create a file in the mount point, and write
some data to the file
3)get the logical address of the file's extent data
4)get the physical address of the logical address
5)ov
From: Qu Wenruo
Introduce a new function, __btrfs_map_block_v2().
Unlike old btrfs_map_block(), which needs different parameter to handle
different RAID profile, this new function uses unified btrfs_map_block
structure to handle all RAID profile in a more meaningful method:
Return physical addr
From: Qu Wenruo
Now, btrfs-progs has a kernel scrub equivalent.
A new option, --offline is added to "btrfs scrub start".
If --offline is given, btrfs scrub will just act like kernel scrub, to
check every copy of extent and do a report on corrupted data and if it's
recoverable.
The advantage com
From: Qu Wenruo
Introduce a internal helper, write_full_stripe() to calculate P/Q and
write the whole full stripe.
This is useful to recover RAID56 stripes.
Signed-off-by: Qu Wenruo
---
scrub.c | 44
1 file changed, 44 insertions(+)
diff --git a/s
From: Qu Wenruo
For READ, caller normally hopes to get what they request, other than
full stripe map.
In this case, we should remove unrelated stripe map, just like the
following case:
32K 96K
|<-request range->|
0 64k 1
From: Qu Wenruo
Introduce a new function, scrub_one_full_stripe(), to check a full
stripe.
It handles the full stripe scrub in the following steps:
0) Check if we need to check full stripe
If full stripe contains no extent, why waste our CPU and IO?
1) Read out full stripe
Then we know ho
From: Qu Wenruo
Introduce new function, check/recover_data_mirror(), to check and recover
mirror based data blocks.
Unlike tree block, data blocks must be recovered sector by sector, so we
introduced corrupted_bitmap for check and recover.
Signed-off-by: Qu Wenruo
Signed-off-by: Su Yue
Signed
For any one who wants to try it, it can be get from my repo:
https://github.com/gujx2017/btrfs-progs/tree/offline-scrub/
In this v6, just rebase to v4.14 and a test for offline-scrub.
Several reports on kernel scrub screwing up good data stripes are in ML for
sometime.
And since kernel scrub wo
On 01/05/2018 04:12 PM, Qu Wenruo wrote:
Before this patch, chunk allocation is split into 2 parts:
1) Chunk allocation
Handled by btrfs_alloc_chunk(), which will insert chunk and device
extent items.
2) Block group allocation
Handled by btrfs_make_block_group(), which will insert
On 2018年01月05日 17:36, Su Yue wrote:
>
>
> On 01/05/2018 04:12 PM, Qu Wenruo wrote:
>> We used to have two chunk allocators, btrfs_alloc_chunk() and
>> btrfs_alloc_data_chunk(), the former is the more generic one, while the
>> later is only used in mkfs and convert, to allocate SINGLE data chunk
On 01/05/2018 05:36 PM, Su Yue wrote:
On 01/05/2018 04:12 PM, Qu Wenruo wrote:
We used to have two chunk allocators, btrfs_alloc_chunk() and
btrfs_alloc_data_chunk(), the former is the more generic one, while the
later is only used in mkfs and convert, to allocate SINGLE data chunk.
Althoug
On 01/05/2018 04:12 PM, Qu Wenruo wrote:
We used to have two chunk allocators, btrfs_alloc_chunk() and
btrfs_alloc_data_chunk(), the former is the more generic one, while the
later is only used in mkfs and convert, to allocate SINGLE data chunk.
Although btrfs_alloc_data_chunk() has some speci
btrfs-progs uses additional two super flag bits. So just define
that so that we know its been used.
Signed-off-by: Anand Jain
---
The btrfs-progs commits (very old) introduced them,
commit 7cc792872a133cabc3467e6ccaf5a2c8ea9e5218
btrfs-progs: Add CHANGING_FSID super flag
commit 797a937e5d
Before this patch, chunk allocation is split into 2 parts:
1) Chunk allocation
Handled by btrfs_alloc_chunk(), which will insert chunk and device
extent items.
2) Block group allocation
Handled by btrfs_make_block_group(), which will insert block group
item and update space info.
How
We used to have two chunk allocators, btrfs_alloc_chunk() and
btrfs_alloc_data_chunk(), the former is the more generic one, while the
later is only used in mkfs and convert, to allocate SINGLE data chunk.
Although btrfs_alloc_data_chunk() has some special hacks to cooperate
with convert, it's quit
This patchset, along with its prerequisite (patchset named:
"[PATCH 0/5] Cleanups for later btrfs_alloc_chunk() rework") can be
fetched from github:
https://github.com/adam900710/btrfs-progs/tree/chunk_alloc_enospc
This patchset can still be treated as cleanup, but brings a much larger
structure m
56 matches
Mail list logo
