On 07/10/2018 06:53 PM, Su Yue wrote:
On 07/10/2018 12:10 PM, Marc MERLIN wrote:
On Tue, Jul 10, 2018 at 08:56:15AM +0800, Su Yue wrote:
I'm just not clear if my FS is still damaged and btrfsck was just
hacked to
ignore the damage it can't deal with, or whether it was able to repair
On 07/10/2018 12:55 PM, Qu Wenruo wrote:
On 2018年07月10日 11:50, Marc MERLIN wrote:
On Tue, Jul 10, 2018 at 09:34:36AM +0800, Qu Wenruo wrote:
Ok, this is where I am now:
WARNING: debug: end of checking extent item[18457780273152 169 1]
type: 176 offset: 2
checking extent items
On 2018年07月10日 11:50, Marc MERLIN wrote:
> On Tue, Jul 10, 2018 at 09:34:36AM +0800, Qu Wenruo wrote:
>> Ok, this is where I am now:
>> WARNING: debug: end of checking extent item[18457780273152 169 1]
>> type: 176 offset: 2
>> checking extent items
To fill in for the spectators on the list :)
Su gave me a modified version of btrfsck lowmem that was able to clean
most of my filesystem.
It's not a general case solution since it had some hardcoding specific
to my filesystem problems, but still a great success.
Email quoted below, along with
On Tue, Jul 10, 2018 at 09:34:36AM +0800, Qu Wenruo wrote:
> Ok, this is where I am now:
> WARNING: debug: end of checking extent item[18457780273152 169 1]
> type: 176 offset: 2
> checking extent items [18457780273152/18457780273152]
> ERROR: errors found in extent
On 2018年07月10日 09:37, Su Yue wrote:
> [CC to linux-btrfs]
>
> Here is the log of wrong extent data.
>
> On 07/08/2018 01:21 AM, Marc MERLIN wrote:
>> On Fri, Jul 06, 2018 at 10:56:36AM -0700, Marc MERLIN wrote:
>>> On Fri, Jul 06, 2018 at 09:05:23AM -0700, Marc MERLIN wrote:
Ok, this is
[CC to linux-btrfs]
Here is the log of wrong extent data.
On 07/08/2018 01:21 AM, Marc MERLIN wrote:
On Fri, Jul 06, 2018 at 10:56:36AM -0700, Marc MERLIN wrote:
On Fri, Jul 06, 2018 at 09:05:23AM -0700, Marc MERLIN wrote:
Ok, this is where I am now:
WARNING: debug: end of checking extent
questions in the part forwarded, replies are
always welcome.
Error messages are showed in the last.
Forwarded Message
Subject: Re: So, does btrfs check lowmem take days? weeks?
Date: Mon, 9 Jul 2018 10:48:18 -0700
From: Marc MERLIN
To: Su Yue
CC: quwenruo.bt...@gmx.com, Su Yue
.
Error messages are showed in the last.
Forwarded Message
Subject: Re: So, does btrfs check lowmem take days? weeks?
Date: Mon, 9 Jul 2018 10:48:18 -0700
From: Marc MERLIN
To: Su Yue
CC: quwenruo.bt...@gmx.com, Su Yue
Success!
Well done Su, this is a huge improvement
On 07/04/2018 05:40 AM, Marc MERLIN wrote:
On Tue, Jul 03, 2018 at 03:34:45PM -0600, Chris Murphy wrote:
On Tue, Jul 3, 2018 at 2:34 AM, Su Yue wrote:
Yes, extent tree is the hardest part for lowmem mode. I'm quite
confident the tool can deal well with file trees(which records metadata
On 2018年07月04日 06:00, Marc MERLIN wrote:
> On Tue, Jul 03, 2018 at 03:46:59PM -0600, Chris Murphy wrote:
>> On Tue, Jul 3, 2018 at 2:50 AM, Qu Wenruo wrote:
>>>
>>>
>>> There must be something wrong, however due to the size of the fs, and
>>> the complexity of extent tree, I can't tell.
>>
>>
On Tue, Jul 03, 2018 at 03:46:59PM -0600, Chris Murphy wrote:
> On Tue, Jul 3, 2018 at 2:50 AM, Qu Wenruo wrote:
> >
> >
> > There must be something wrong, however due to the size of the fs, and
> > the complexity of extent tree, I can't tell.
>
> Right, which is why I'm asking if any of the
On Tue, Jul 3, 2018 at 2:50 AM, Qu Wenruo wrote:
>
>
> There must be something wrong, however due to the size of the fs, and
> the complexity of extent tree, I can't tell.
Right, which is why I'm asking if any of the metadata integrity
checker mask options might reveal what's going wrong?
I
On Tue, Jul 03, 2018 at 03:34:45PM -0600, Chris Murphy wrote:
> On Tue, Jul 3, 2018 at 2:34 AM, Su Yue wrote:
>
> > Yes, extent tree is the hardest part for lowmem mode. I'm quite
> > confident the tool can deal well with file trees(which records metadata
> > about file and directory name,
On Tue, Jul 3, 2018 at 2:34 AM, Su Yue wrote:
> Yes, extent tree is the hardest part for lowmem mode. I'm quite
> confident the tool can deal well with file trees(which records metadata
> about file and directory name, relationships).
> As for extent tree, I have few confidence due to its
On Tue, Jul 03, 2018 at 04:50:48PM +0800, Qu Wenruo wrote:
> > It sounds like there may not be a fix to this problem with the filesystem's
> > design, outside of "do not get there, or else".
> > It would even be useful for btrfs tools to start computing heuristics and
> > output warnings like "you
On 2018年07月03日 12:22, Marc MERLIN wrote:
> On Mon, Jul 02, 2018 at 06:31:43PM -0600, Chris Murphy wrote:
>> So the idea behind journaled file systems is that journal replay
>> enabled mount time "repair" that's faster than an fsck. Already Btrfs
>> use cases with big, but not huge, file systems
On 07/03/2018 12:22 PM, Marc MERLIN wrote:
On Mon, Jul 02, 2018 at 06:31:43PM -0600, Chris Murphy wrote:
So the idea behind journaled file systems is that journal replay
enabled mount time "repair" that's faster than an fsck. Already Btrfs
use cases with big, but not huge, file systems makes
On Mon, Jul 02, 2018 at 06:31:43PM -0600, Chris Murphy wrote:
> So the idea behind journaled file systems is that journal replay
> enabled mount time "repair" that's faster than an fsck. Already Btrfs
> use cases with big, but not huge, file systems makes btrfs check a
> problem. Either running
On Mon, Jul 2, 2018 at 8:42 AM, Qu Wenruo wrote:
>
>
> On 2018年07月02日 22:05, Marc MERLIN wrote:
>> On Mon, Jul 02, 2018 at 02:22:20PM +0800, Su Yue wrote:
Ok, that's 29MB, so it doesn't fit on pastebin:
http://marc.merlins.org/tmp/dshelf2_inspect.txt
>>> Sorry Marc. After offline
On Mon, Jul 02, 2018 at 10:33:09PM +0500, Roman Mamedov wrote:
> On Mon, 2 Jul 2018 08:19:03 -0700
> Marc MERLIN wrote:
>
> > I actually have fewer snapshots than this per filesystem, but I backup
> > more than 10 filesystems.
> > If I used as many snapshots as you recommend, that would already
On Mon, 2 Jul 2018 08:19:03 -0700
Marc MERLIN wrote:
> I actually have fewer snapshots than this per filesystem, but I backup
> more than 10 filesystems.
> If I used as many snapshots as you recommend, that would already be 230
> snapshots for 10 filesystems :)
(...once again me with my rsync
On 2018-07-02 11:19, Marc MERLIN wrote:
Hi Qu,
thanks for the detailled and honest answer.
A few comments inline.
On Mon, Jul 02, 2018 at 10:42:40PM +0800, Qu Wenruo wrote:
For full, it depends. (but for most real world case, it's still flawed)
We have small and crafted images as test cases,
Hi Qu,
thanks for the detailled and honest answer.
A few comments inline.
On Mon, Jul 02, 2018 at 10:42:40PM +0800, Qu Wenruo wrote:
> For full, it depends. (but for most real world case, it's still flawed)
> We have small and crafted images as test cases, which btrfs check can
> repair without
On 2018年07月02日 22:05, Marc MERLIN wrote:
> On Mon, Jul 02, 2018 at 02:22:20PM +0800, Su Yue wrote:
>>> Ok, that's 29MB, so it doesn't fit on pastebin:
>>> http://marc.merlins.org/tmp/dshelf2_inspect.txt
>>>
>> Sorry Marc. After offline communication with Qu, both
>> of us think the filesystem
On Mon, Jul 02, 2018 at 02:22:20PM +0800, Su Yue wrote:
> > Ok, that's 29MB, so it doesn't fit on pastebin:
> > http://marc.merlins.org/tmp/dshelf2_inspect.txt
> >
> Sorry Marc. After offline communication with Qu, both
> of us think the filesystem is hard to repair.
> The filesystem is too large
On 07/02/2018 11:22 AM, Marc MERLIN wrote:
On Mon, Jul 02, 2018 at 10:02:33AM +0800, Su Yue wrote:
Could you try follow dumps? They shouldn't cost much time.
#btrfs inspect dump-tree -t 21872 | grep -C 50 "374857
EXTENT_DATA "
#btrfs inspect dump-tree -t 22911 | grep -C 50 "374857
On Mon, Jul 02, 2018 at 10:02:33AM +0800, Su Yue wrote:
> Could you try follow dumps? They shouldn't cost much time.
>
> #btrfs inspect dump-tree -t 21872 | grep -C 50 "374857
> EXTENT_DATA "
>
> #btrfs inspect dump-tree -t 22911 | grep -C 50 "374857
> EXTENT_DATA "
Ok, that's 29MB, so it
On 07/02/2018 07:22 AM, Marc MERLIN wrote:
On Thu, Jun 28, 2018 at 11:43:54PM -0700, Marc MERLIN wrote:
On Fri, Jun 29, 2018 at 02:32:44PM +0800, Su Yue wrote:
https://github.com/Damenly/btrfs-progs/tree/tmp1
Not sure if I undertand that you meant, here.
Sorry for my unclear words.
On Thu, Jun 28, 2018 at 11:43:54PM -0700, Marc MERLIN wrote:
> On Fri, Jun 29, 2018 at 02:32:44PM +0800, Su Yue wrote:
> > > > https://github.com/Damenly/btrfs-progs/tree/tmp1
> > >
> > > Not sure if I undertand that you meant, here.
> > >
> > Sorry for my unclear words.
> > Simply speaking, I
On Sat, Jun 30, 2018 at 10:49:07PM +0800, Qu Wenruo wrote:
> But the last abort looks pretty possible to be the culprit.
>
> Would you try to dump the extent tree?
> # btrfs inspect dump-tree -t extent | grep -A50 156909494272
Sure, there you go:
item 25 key (156909494272 EXTENT_ITEM
On 2018年06月30日 10:44, Marc MERLIN wrote:
> Well, there goes that. After about 18H:
> ERROR: extent[156909494272, 55320576] referencer count mismatch (root: 21872,
> owner: 374857, offset: 235175936) wanted: 1, have: 1452
> backref.c:466: __add_missing_keys: Assertion `ref->root_id` failed,
Well, there goes that. After about 18H:
ERROR: extent[156909494272, 55320576] referencer count mismatch (root: 21872,
owner: 374857, offset: 235175936) wanted: 1, have: 1452
backref.c:466: __add_missing_keys: Assertion `ref->root_id` failed, value 0
btrfs(+0x3a232)[0x56091704f232]
I've got about 1/2 the snapshots and less than 1/10th the data...but
my btrfs check times are much shorter than either: 15 minutes and 65
minutes (lowmem).
[chris@f28s ~]$ sudo btrfs fi us /mnt/first
Overall:
Device size:1024.00GiB
Device allocated: 774.12GiB
Device
On Fri, Jun 29, 2018 at 12:28:31AM -0700, Marc MERLIN wrote:
> So, I rebooted, and will now run Su's btrfs check without repair and
> report back.
As expected, it will likely still take days, here's the start:
gargamel:~# btrfs check --mode=lowmem -p /dev/mapper/dshelf2
Checking filesystem on
Hi,
On 29/06/2018 09:22, Marc MERLIN wrote:
> On Fri, Jun 29, 2018 at 12:09:54PM +0500, Roman Mamedov wrote:
>> On Thu, 28 Jun 2018 23:59:03 -0700
>> Marc MERLIN wrote:
>>
>>> I don't waste a week recreating the many btrfs send/receive relationships.
>> Consider not using send/receive, and
On Fri, 29 Jun 2018 00:22:10 -0700
Marc MERLIN wrote:
> On Fri, Jun 29, 2018 at 12:09:54PM +0500, Roman Mamedov wrote:
> > On Thu, 28 Jun 2018 23:59:03 -0700
> > Marc MERLIN wrote:
> >
> > > I don't waste a week recreating the many btrfs send/receive relationships.
> >
> > Consider not using
On Fri, Jun 29, 2018 at 03:20:42PM +0800, Qu Wenruo wrote:
> If certain btrfs specific operations are involved, it's definitely not OK:
> 1) Balance
> 2) Quota
> 3) Btrfs check
Ok, I understand. I'll try to balance almost never then. My problems did
indeed start because I ran balance and it got
On Fri, Jun 29, 2018 at 12:09:54PM +0500, Roman Mamedov wrote:
> On Thu, 28 Jun 2018 23:59:03 -0700
> Marc MERLIN wrote:
>
> > I don't waste a week recreating the many btrfs send/receive relationships.
>
> Consider not using send/receive, and switching to regular rsync instead.
> Send/receive
On 2018年06月29日 14:59, Marc MERLIN wrote:
> On Fri, Jun 29, 2018 at 02:29:10PM +0800, Qu Wenruo wrote:
>>> If --repair doesn't work, check is useless to me sadly.
>>
>> Not exactly.
>> Although it's time consuming, I have manually patched several users fs,
>> which normally ends pretty well.
>
On Thu, 28 Jun 2018 23:59:03 -0700
Marc MERLIN wrote:
> I don't waste a week recreating the many btrfs send/receive relationships.
Consider not using send/receive, and switching to regular rsync instead.
Send/receive is very limiting and cumbersome, including because of what you
described. And
On Fri, Jun 29, 2018 at 02:29:10PM +0800, Qu Wenruo wrote:
> > If --repair doesn't work, check is useless to me sadly.
>
> Not exactly.
> Although it's time consuming, I have manually patched several users fs,
> which normally ends pretty well.
Ok I understand now.
> > Agreed, I doubt I have
On Fri, Jun 29, 2018 at 02:32:44PM +0800, Su Yue wrote:
> > > https://github.com/Damenly/btrfs-progs/tree/tmp1
> >
> > Not sure if I undertand that you meant, here.
> >
> Sorry for my unclear words.
> Simply speaking, I suggest you to stop current running check.
> Then, clone above branch to
On 2018年06月29日 14:06, Marc MERLIN wrote:
> On Fri, Jun 29, 2018 at 01:48:17PM +0800, Qu Wenruo wrote:
>> Just normal btrfs check, and post the output.
>> If normal check eats up all your memory, btrfs check --mode=lowmem.
>
> Does check without --repair eat less RAM?
Unfortunately, no.
>
>>
On 06/29/2018 02:10 PM, Marc MERLIN wrote:
On Fri, Jun 29, 2018 at 02:02:19PM +0800, Su Yue wrote:
I have figured out the bug is lowmem check can't deal with shared tree block
in reloc tree. The fix is simple, you can try the follow repo:
https://github.com/Damenly/btrfs-progs/tree/tmp1
On Fri, Jun 29, 2018 at 02:02:19PM +0800, Su Yue wrote:
> I have figured out the bug is lowmem check can't deal with shared tree block
> in reloc tree. The fix is simple, you can try the follow repo:
>
> https://github.com/Damenly/btrfs-progs/tree/tmp1
Not sure if I undertand that you meant,
On Fri, Jun 29, 2018 at 01:48:17PM +0800, Qu Wenruo wrote:
> Just normal btrfs check, and post the output.
> If normal check eats up all your memory, btrfs check --mode=lowmem.
Does check without --repair eat less RAM?
> --repair should be considered as the last method.
If --repair doesn't
On 06/29/2018 01:28 PM, Marc MERLIN wrote:
On Fri, Jun 29, 2018 at 01:07:20PM +0800, Qu Wenruo wrote:
lowmem repair seems to be going still, but it's been days and -p seems
to do absolutely nothing.
I'm a afraid you hit a bug in lowmem repair code.
By all means, --repair shouldn't really
On 2018年06月29日 13:28, Marc MERLIN wrote:
> On Fri, Jun 29, 2018 at 01:07:20PM +0800, Qu Wenruo wrote:
>>> lowmem repair seems to be going still, but it's been days and -p seems
>>> to do absolutely nothing.
>>
>> I'm a afraid you hit a bug in lowmem repair code.
>> By all means, --repair
On Fri, Jun 29, 2018 at 01:35:06PM +0800, Su Yue wrote:
> > It's hard to estimate, especially when every cross check involves a lot
> > of disk IO.
> >
> > But at least, we could add such indicator to show we're doing something.
> > Maybe we can account all roots in root tree first, before
On 06/29/2018 01:07 PM, Qu Wenruo wrote:
On 2018年06月29日 12:27, Marc MERLIN wrote:
Regular btrfs check --repair has a nice progress option. It wasn't
perfect, but it showed something.
But then it also takes all your memory quicker than the linux kernel can
defend itself and reliably
On Fri, Jun 29, 2018 at 01:07:20PM +0800, Qu Wenruo wrote:
> > lowmem repair seems to be going still, but it's been days and -p seems
> > to do absolutely nothing.
>
> I'm a afraid you hit a bug in lowmem repair code.
> By all means, --repair shouldn't really be used unless you're pretty
> sure
On 2018年06月29日 12:27, Marc MERLIN wrote:
> Regular btrfs check --repair has a nice progress option. It wasn't
> perfect, but it showed something.
>
> But then it also takes all your memory quicker than the linux kernel can
> defend itself and reliably completely kills my 32GB server quicker
53 matches
Mail list logo
