On 3.07.2018 12:10, Qu Wenruo wrote:
> A crafted image has empty root tree block, which will cause later NULL
> pointer dereference.
>
> The following trees should never be empty:
> 1) Tree root
>Must contain at least root items for extent tree, device tree and fs
>tree
>
> 2) Chunk
On 3.07.2018 12:10, Qu Wenruo wrote:
> A crafted image with invalid block group items could make free space cache
> code to cause panic.
>
> We could early detect such invalid block group item by checking:
> 1) Item size
>Fixed value.
> 2) Block group size (key.offset)
>We have a up
> -Original Message-
> From: linux-btrfs-ow...@vger.kernel.org
> [mailto:linux-btrfs-ow...@vger.kernel.org] On Behalf Of Qu Wenruo
> Sent: Tuesday, July 03, 2018 5:10 PM
> To: linux-btrfs@vger.kernel.org
> Subject: [PATCH 4/5] btrfs: Check each block group has corresponding chunk at
>
Update the information to reflect the status of 4.18
Main Updates:
- Add explanation of improved compression heuristic algorithm
- Add explanation that norecovery == nologreplay
- Add explanation of nossd_spread mount option
- Add explanation of rmdir_subovl feature
Signed-off-by: Misono
> -Original Message-
> From: linux-btrfs-ow...@vger.kernel.org
> [mailto:linux-btrfs-ow...@vger.kernel.org] On Behalf Of Qu Wenruo
> Sent: Tuesday, July 03, 2018 5:10 PM
> To: linux-btrfs@vger.kernel.org
> Subject: [PATCH 3/5] btrfs: relocation: Only remove reloc rb_trees if reloc
>
> -Original Message-
> From: linux-btrfs-ow...@vger.kernel.org
> [mailto:linux-btrfs-ow...@vger.kernel.org] On Behalf Of Qu Wenruo
> Sent: Tuesday, July 03, 2018 5:10 PM
> To: linux-btrfs@vger.kernel.org
> Subject: [PATCH 2/5] btrfs: tree-checker: Detect invalid empty essential tree
>
> -Original Message-
> From: linux-btrfs-ow...@vger.kernel.org
> [mailto:linux-btrfs-ow...@vger.kernel.org] On Behalf Of Qu Wenruo
> Sent: Tuesday, July 03, 2018 5:10 PM
> To: linux-btrfs@vger.kernel.org
> Subject: [PATCH 1/5] btrfs: tree-checker: Verify block_group_item
>
> A crafted
On 07/04/2018 05:40 AM, Marc MERLIN wrote:
On Tue, Jul 03, 2018 at 03:34:45PM -0600, Chris Murphy wrote:
On Tue, Jul 3, 2018 at 2:34 AM, Su Yue wrote:
Yes, extent tree is the hardest part for lowmem mode. I'm quite
confident the tool can deal well with file trees(which records metadata
Sure, thanks!
-Wen
> On Jul 3, 2018, at 7:38 PM, Qu Wenruo wrote:
>
>
>
> On 2018年07月04日 07:26, Xu, Wen wrote:
>> I see, then does BTRFS have a dev branch for merging into mainline?…
>
> We have, although these fixes are not fully merged yet.
>
>
On 07/03/2018 07:42 PM, David Disseldorp wrote:
The experimental flag is already carried in the manpage, but was removed
from the btrfs check usage message as part of refactoring via
87c1bd13c1fca430c3dbf0da62e9aa33bde609c8. Add it back.
Signed-off-by: David Disseldorp
Reviewed-by: Nikolay
On 2018年07月04日 07:26, Xu, Wen wrote:
> I see, then does BTRFS have a dev branch for merging into mainline?…
We have, although these fixes are not fully merged yet.
https://github.com/kdave/btrfs-devel/tree/misc-next
Thanks,
Qu
>
> Thanks,
> Wen
>
>> On Jul 3, 2018, at 6:36 PM, Qu Wenruo
On 2018年07月04日 06:00, Marc MERLIN wrote:
> On Tue, Jul 03, 2018 at 03:46:59PM -0600, Chris Murphy wrote:
>> On Tue, Jul 3, 2018 at 2:50 AM, Qu Wenruo wrote:
>>>
>>>
>>> There must be something wrong, however due to the size of the fs, and
>>> the complexity of extent tree, I can't tell.
>>
>>
On 2018年07月04日 00:27, Xu, Wen wrote:
> So this is the dev branch of btrfs?…
Just my branch of all the related fixes.
Thanks,
Qu
>
> Thanks,
> -Wen
>
>> On Jul 3, 2018, at 9:36 AM, Qu Wenruo wrote:
>>
>>
>>
>> On 2018年07月03日 20:37, Xu, Wen wrote:
>>> Sure, thanks for your efforts! So as I
On Tue, Jul 03, 2018 at 03:46:59PM -0600, Chris Murphy wrote:
> On Tue, Jul 3, 2018 at 2:50 AM, Qu Wenruo wrote:
> >
> >
> > There must be something wrong, however due to the size of the fs, and
> > the complexity of extent tree, I can't tell.
>
> Right, which is why I'm asking if any of the
On Tue, Jul 3, 2018 at 2:50 AM, Qu Wenruo wrote:
>
>
> There must be something wrong, however due to the size of the fs, and
> the complexity of extent tree, I can't tell.
Right, which is why I'm asking if any of the metadata integrity
checker mask options might reveal what's going wrong?
I
On Tue, Jul 03, 2018 at 03:34:45PM -0600, Chris Murphy wrote:
> On Tue, Jul 3, 2018 at 2:34 AM, Su Yue wrote:
>
> > Yes, extent tree is the hardest part for lowmem mode. I'm quite
> > confident the tool can deal well with file trees(which records metadata
> > about file and directory name,
On Tue, Jul 3, 2018 at 2:34 AM, Su Yue wrote:
> Yes, extent tree is the hardest part for lowmem mode. I'm quite
> confident the tool can deal well with file trees(which records metadata
> about file and directory name, relationships).
> As for extent tree, I have few confidence due to its
Nikolay Borisov - 03.07.18, 11:08:
> On 3.07.2018 11:47, Qu Wenruo wrote:
> > On 2018年07月03日 16:33, Nikolay Borisov wrote:
> >> On 3.07.2018 11:08, Qu Wenruo wrote:
> >>> Reported in https://bugzilla.kernel.org/show_bug.cgi?id=199837, if
> >>> a
> >>> crafted btrfs with incorrect chunk<->block
So this is the dev branch of btrfs?…
Thanks,
-Wen
> On Jul 3, 2018, at 9:36 AM, Qu Wenruo wrote:
>
>
>
> On 2018年07月03日 20:37, Xu, Wen wrote:
>> Sure, thanks for your efforts! So as I suppose, 199833 and 199835 are
>> implicitly fixed by these patches as well?
>>
>> -Wen
>
> Yep.
>
>
On Tue, Jul 03, 2018 at 04:50:48PM +0800, Qu Wenruo wrote:
> > It sounds like there may not be a fix to this problem with the filesystem's
> > design, outside of "do not get there, or else".
> > It would even be useful for btrfs tools to start computing heuristics and
> > output warnings like "you
Don't panic for btrfs_read_chunk_tree() if one device or chunk is
corrupted.
Caller can already handle it pretty well.
Link: https://bugzilla.kernel.org/show_bug.cgi?id=199839
Signed-off-by: Qu Wenruo
---
volumes.c | 6 --
1 file changed, 4 insertions(+), 2 deletions(-)
diff --git
On 2018年07月03日 20:37, Xu, Wen wrote:
> Sure, thanks for your efforts! So as I suppose, 199833 and 199835 are
> implicitly fixed by these patches as well?
>
> -Wen
Yep.
Kernel can know detect all related chunk <-> block group mapping problem
in all 6 images and exit gracefully with error
Sure, thanks for your efforts! So as I suppose, 199833 and 199835 are
implicitly fixed by these patches as well?
-Wen
> On Jul 3, 2018, at 4:32 AM, Qu Wenruo wrote:
>
>
>
> On 2018年07月01日 08:06, Xu, Wen wrote:
>> Dear BTRFS developers,
>>
>> I would like to know if these issues are fixed
On 2018-07-03 03:35, Duncan wrote:
Austin S. Hemmelgarn posted on Mon, 02 Jul 2018 07:49:05 -0400 as
excerpted:
Notably, most Intel systems I've seen have the SATA controllers in the
chipset enumerate after the USB controllers, and the whole chipset
enumerates after add-in cards (so they
The experimental flag is already carried in the manpage, but was removed
from the btrfs check usage message as part of refactoring via
87c1bd13c1fca430c3dbf0da62e9aa33bde609c8. Add it back.
Signed-off-by: David Disseldorp
Reviewed-by: Nikolay Borisov
---
check/main.c | 2 +-
1 file changed, 1
On 2018年07月03日 17:55, Paul Jones wrote:
>> -Original Message-
>> From: linux-btrfs-ow...@vger.kernel.org > ow...@vger.kernel.org> On Behalf Of Marc MERLIN
>> Sent: Tuesday, 3 July 2018 2:16 PM
>> To: Qu Wenruo
>> Cc: Su Yue ; linux-btrfs@vger.kernel.org
>> Subject: Re: how to best
> -Original Message-
> From: linux-btrfs-ow...@vger.kernel.org ow...@vger.kernel.org> On Behalf Of Marc MERLIN
> Sent: Tuesday, 3 July 2018 2:16 PM
> To: Qu Wenruo
> Cc: Su Yue ; linux-btrfs@vger.kernel.org
> Subject: Re: how to best segment a big block device in resizeable btrfs
>
A crafted btrfs with incorrect chunk<->block group mapping, it could leads
to a lot of unexpected behavior.
Although the crafted image can be catched by block group item checker
added in "[PATCH] btrfs: tree-checker: Verify block_group_item", if one
crafted a valid enough block group item which
If a crafted btrfs has missing block group items, it could cause
unexpected behavior and breaks our expectation on 1:1
chunk<->block group mapping.
Although we added block group -> chunk mapping check, we still need
chunk -> block group mapping check.
This patch will do extra check to ensure
A crafted image has empty root tree block, which will cause later NULL
pointer dereference.
The following trees should never be empty:
1) Tree root
Must contain at least root items for extent tree, device tree and fs
tree
2) Chunk tree
Or we can't even bootstrap.
3) Fs tree
At least
Invalid tree reloc tree can cause kernel NULL pointer dereference when
btrfs does some cleanup for reloc roots.
It turns out that fs_info->reloc_ctl can be NULL in
btrfs_recover_relocation() as we allocate relocation control after all
reloc roots are verified.
So when we hit out: tag, we haven't
A crafted image with invalid block group items could make free space cache
code to cause panic.
We could early detect such invalid block group item by checking:
1) Item size
Fixed value.
2) Block group size (key.offset)
We have a up limit on block group item (10G)
3) Chunk objectid
Fixed
Can be fetched from github, which is based on v4.18-rc1 tag:
https://github.com/adam900710/linux/tree/tree_checker_enhance
Reported by Xu Wen , some crafted btrfs image can
cause unexpected kernel behavior.
All of them are related to block group and chunk, so this patchset will
enhance block
On 3.07.2018 11:47, Qu Wenruo wrote:
>
>
> On 2018年07月03日 16:33, Nikolay Borisov wrote:
>>
>>
>> On 3.07.2018 11:08, Qu Wenruo wrote:
>>> Reported in https://bugzilla.kernel.org/show_bug.cgi?id=199837, if a
>>> crafted btrfs with incorrect chunk<->block group mapping, it could leads
>>> to
On 3.07.2018 12:07, Anand Jain wrote:
>
>
> On 07/03/2018 01:56 PM, Nikolay Borisov wrote:
>>
>>
>> On 3.07.2018 08:12, Anand Jain wrote:
>>> When a device is deleted, the btrfs_super_block::number_devices is
>>> reduced by 1, but we do that after the commit transaction, so this
>>> change
When a device is deleted, the btrfs_super_block::number_devices is
reduced by 1, but we do that after the commit transaction, so this
change did not made it to the disk and waits for the next commit
transaction whenever it happens.
This can be easily demonstrated using the following test case
In case of deleting the seed device the %cur_devices (seed)
and the %fs_devices (parent) are different. Now, as the parent
fs_devices::total_devices also maintains the total number of devices
including the seed device, so decrement its in-memory value for the
successful seed delete. We are already
On 07/03/2018 01:56 PM, Nikolay Borisov wrote:
On 3.07.2018 08:12, Anand Jain wrote:
When a device is deleted, the btrfs_super_block::number_devices is
reduced by 1, but we do that after the commit transaction, so this
change did not made it to the disk and waits for the next commit
On 2.07.2018 18:40, David Sterba wrote:
> On Mon, Jul 02, 2018 at 02:00:34PM +0300, Nikolay Borisov wrote:
>> Commit 5d23515be669 ("btrfs: Move qgroup rescan on quota enable to
>> btrfs_quota_enable") not only resulted in an easier to follow code but
>> it also introduced a subtle bug. It
On 2018年07月03日 12:22, Marc MERLIN wrote:
> On Mon, Jul 02, 2018 at 06:31:43PM -0600, Chris Murphy wrote:
>> So the idea behind journaled file systems is that journal replay
>> enabled mount time "repair" that's faster than an fsck. Already Btrfs
>> use cases with big, but not huge, file systems
On 2018年07月03日 16:33, Nikolay Borisov wrote:
>
>
> On 3.07.2018 11:08, Qu Wenruo wrote:
>> Reported in https://bugzilla.kernel.org/show_bug.cgi?id=199837, if a
>> crafted btrfs with incorrect chunk<->block group mapping, it could leads
>> to a lot of unexpected behavior.
>>
>> Although the
This test case verifies if the device ready return success after the
device delete.
Signed-off-by: Anand Jain
---
tests/btrfs/168 | 68 +
tests/btrfs/168.out | 2 ++
tests/btrfs/group | 1 +
3 files changed, 71 insertions(+)
create
On 3.07.2018 11:33, Nikolay Borisov wrote:
>
>
> On 3.07.2018 11:08, Qu Wenruo wrote:
>> Reported in https://bugzilla.kernel.org/show_bug.cgi?id=199837, if a
>> crafted btrfs with incorrect chunk<->block group mapping, it could leads
>> to a lot of unexpected behavior.
>>
>> Although the
On 2018年07月01日 08:06, Xu, Wen wrote:
> Dear BTRFS developers,
>
> I would like to know if these issues are fixed or handled? Thanks.
>
> -Wen
All these images should be addressed by the following patches now:
https://patchwork.kernel.org/patch/10503415/
On 3.07.2018 11:08, Qu Wenruo wrote:
> Reported in https://bugzilla.kernel.org/show_bug.cgi?id=199837, if a
> crafted btrfs with incorrect chunk<->block group mapping, it could leads
> to a lot of unexpected behavior.
>
> Although the crafted image can be catched by block group item checker
>
On 07/03/2018 12:22 PM, Marc MERLIN wrote:
On Mon, Jul 02, 2018 at 06:31:43PM -0600, Chris Murphy wrote:
So the idea behind journaled file systems is that journal replay
enabled mount time "repair" that's faster than an fsck. Already Btrfs
use cases with big, but not huge, file systems makes
On 2018年07月03日 15:58, Nikolay Borisov wrote:
> Hello Qu,
>
> The commit from $SUBJECT breaks btrfs/100. Before that commit this test
> takes around 25 seconds and it succeeds, whereas with this patch applied
> the test is twice as fast but is broken after just a couple of iterations.
> The
The typo 'mutexe' should be 'mutex'.
Signed-off-by: Su Yue
---
fs/btrfs/volumes.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c
index e034ad9e23b4..d9d87deb1160 100644
--- a/fs/btrfs/volumes.c
+++ b/fs/btrfs/volumes.c
@@ -3771,7
Reported in https://bugzilla.kernel.org/show_bug.cgi?id=199847, the
image has empty extent tree.
Although that image also has empty root tree, thus can be catched by
tree checker on empty leaves, but if a crafted btrfs has missing block
group items, it could still cause problem.
This patch will
Reported in https://bugzilla.kernel.org/show_bug.cgi?id=199837, if a
crafted btrfs with incorrect chunk<->block group mapping, it could leads
to a lot of unexpected behavior.
Although the crafted image can be catched by block group item checker
added in "[PATCH] btrfs: tree-checker: Verify
Reported in https://bugzilla.kernel.org/show_bug.cgi?id=199847, where
the crafted image has empty root tree block, which will cause later NULL
pointer dereference.
The following trees should never be empty:
1) Tree root
Must contain at least root items for extent tree, device tree and fs
Hello Qu,
The commit from $SUBJECT breaks btrfs/100. Before that commit this test
takes around 25 seconds and it succeeds, whereas with this patch applied
the test is twice as fast but is broken after just a couple of iterations.
The breake I've observed so far is either lock up of the
Reported in https://bugzilla.kernel.org/show_bug.cgi?id=199847, where
the crafted image has empty root tree block, which will cause later NULL
pointer dereference.
The following trees should never be empty:
1) Tree root
Must contain at least root items for extent tree, device tree and fs
For trees have been balanced, leaves are with flag BTRFS_HEADER_FLAG_RELOC and
extent data backrefs are shared. Like:
=
item 0 key (11927552 EXTENT_ITEM 524288) itemoff 3932 itemsize 63
refs 129 gen 7 flags DATA
shared data backref parent
Btrfs lowmem check reports such false alerts:
=
ERROR: extent[1419709677568, 1703936] referencer count mismatch (root: 2192,
owner: 327635, offset: 0) wanted: 9, have: 13
=
While in extent tree, the extent has:
=
item 98 key (1419709677568 EXTENT_ITEM 1703936)
Austin S. Hemmelgarn posted on Mon, 02 Jul 2018 07:49:05 -0400 as
excerpted:
> Notably, most Intel systems I've seen have the SATA controllers in the
> chipset enumerate after the USB controllers, and the whole chipset
> enumerates after add-in cards (so they almost always have this issue),
>
Andrei Borzenkov posted on Tue, 03 Jul 2018 07:25:14 +0300 as excerpted:
> 02.07.2018 21:35, Austin S. Hemmelgarn пишет:
>> them (trimming blocks on BTRFS gets rid of old root trees, so it's a
>> bit dangerous to do it while writes are happening).
>
> Could you please elaborate? Do you mean
As reported in https://bugzilla.kernel.org/show_bug.cgi?id=199849,
a crafted image with invalid block group items could make free space cache
code to cause panic.
We could early detect such invalid block group item by checking:
1) Item size
Fixed value.
2) Block group size (key.offset)
We
On 2018年07月03日 14:20, Gu Jinxiang wrote:
> Reported in https://bugzilla.kernel.org/show_bug.cgi?id=199839,
> which has a invalid chunk, not return error opportunlly.
>
> Add chunk type check in btrfs_check_chunk_valid,
> to make error be returned in advance.
>
> Reported-by: Xu Wen
>
On 2018/07/02 20:00, Nikolay Borisov wrote:
> Commit 5d23515be669 ("btrfs: Move qgroup rescan on quota enable to
> btrfs_quota_enable") not only resulted in an easier to follow code but
> it also introduced a subtle bug. It changed the timing when the initial
> transaction rescan was happening
Reported in https://bugzilla.kernel.org/show_bug.cgi?id=199839,
which has a invalid chunk, not return error opportunlly.
Add chunk type check in btrfs_check_chunk_valid,
to make error be returned in advance.
Reported-by: Xu Wen
Signed-off-by: Gu Jinxiang
---
changelog:
v2:
deal with comment by
61 matches
Mail list logo
