Re: shall distros run btrfsck on boot?

On Tue, Nov 24, 2015 at 04:26:47PM -0600, Eric Sandeen wrote: > On 11/24/15 2:38 PM, Austin S Hemmelgarn wrote: > > > if the system was > > shut down cleanly, you're fine barring software bugs, but if it > > crashed, you should be running a check on the FS. > > Um, no... > > The *entire point*

Re: shall distros run btrfsck on boot?

On Tue, 2015-11-24 at 22:33 +, Hugo Mills wrote: > whereas a read-only mount of a journalling FS _must_ modify the disk > data after an unclean shitdown, in order to be useful (because the FS > isn't consistent without the journal replay). I've always considered that rather a bug,... or at

Re: subvols and parents - how?

On Tue, 2015-11-24 at 21:55 +, Hugo Mills wrote: >    In practice, new content is checked by a number of people when > it's > put in, so the case of someone putting random poorly-thought-out crap > in the wiki isn't particularly likely to happen. Well... it may work in 99% cases... but there

Re: bad extent [5993525264384, 5993525280768), type mismatch with chunk

Christoph Anton Mitterer wrote on 2015/11/24 19:25 +0100: On Tue, 2015-11-24 at 13:35 +0800, Qu Wenruo wrote: Hopes you didn't wait too long. No worries, didn't hold my breath ;) The fixing patch is CCed to you, or you can get it from patchwork: https://patchwork.kernel.org/patch/7687611/

Re: shall distros run btrfsck on boot?(Off topic, btrfs per-inode tree idea)

Hugo Mills wrote on 2015/11/24 22:33 +: On Tue, Nov 24, 2015 at 04:26:47PM -0600, Eric Sandeen wrote: On 11/24/15 2:38 PM, Austin S Hemmelgarn wrote: if the system was shut down cleanly, you're fine barring software bugs, but if it crashed, you should be running a check on the FS. Um,

btrfs-mount improvemt suggestions

Hey. I though rather than just being around here and complaining all the time about documentation I might help to improve the same a bit. the btrfs-mount manpage could be a good start and I'd propose a more structurised format as I've did it for the first few options: *alloc_start='bytes'*::

Re: bad extent [5993525264384, 5993525280768), type mismatch with chunk

On Wed, 2015-11-25 at 08:59 +0800, Qu Wenruo wrote: > Did you use the complied btrfsck? Or use the system btrfsck by > mistake? I'm pretty sure cause I already did the whole procedure twice, but let me repeat and record it here just to be 100% sure: $ make clean Cleaning $ md5sum cmds-check.c

Re: bad extent [5993525264384, 5993525280768), type mismatch with chunk

Hey again. So it seems that data-b is always fine (well at least three times in a row) and data-old-a always gives errors. including e.g: bad extent [3067663679488, 3067663695872), type mismatch with chunk bad extent [3067663876096, 3067663892480), type mismatch with chunk bad extent

[PATCH v2] btrfs-progs: fsck: Fix a false alert where extent record has wrong metadata flag

In process_extent_item(), it gives 'metadata' initial value 0, but for non-skinny-metadata case, metadata extent can't be judged just from key type and it forgot that case. This causes a lot of false alert in non-skinny-metadata filesystem. Fix it by set correct metadata value before calling

Re: subvols and parents - how?

On Tue, 2015-11-24 at 23:30 +, Hugo Mills wrote: >    Yes, that makes sense. Feel free to shamelessly use my idea (as well as the one to call btrfs' RAID1 replica2 or something else) :-O >    With a recent mv root@heisenberg:/mnt# mv --version mv (GNU coreutils) 8.23 => not recent enough...

Re: subvols and parents - how?

On Wed, Nov 25, 2015 at 12:20:09AM +0100, Christoph Anton Mitterer wrote: > On Tue, 2015-11-24 at 21:55 +, Hugo Mills wrote: > >    In practice, new content is checked by a number of people when > > it's > > put in, so the case of someone putting random poorly-thought-out crap > > in the wiki

Re: shall distros run btrfsck on boot?

On Wed, Nov 25, 2015 at 12:01:49AM +0100, Christoph Anton Mitterer wrote: > On Tue, 2015-11-24 at 22:33 +, Hugo Mills wrote: > > whereas a read-only mount of a journalling FS _must_ modify the disk > > data after an unclean shitdown, in order to be useful (because the FS > > isn't consistent

Re: bad extent [5993525264384, 5993525280768), type mismatch with chunk

On 11/24/2015 09:15 PM, Laurent Bonnaud wrote: On 23/11/2015 02:00, Qu Wenruo wrote: Considering the size, I'd like not to touch the dump, metadata is over 5G, It is only 2GB once compressed :>. The size seems small enough, I'll try to download it as it's super useful to debug it.

Re: btrfs send reproducibly fails for a specific subvolume after sending 15 GiB, scrub reports no errors

On Sun, Nov 22, 2015 at 9:59 PM, Nils Steinger wrote: > Hi, > > I recently ran into a problem while trying to back up some of my btrfs > subvolumes over the network: > `btrfs send` works flawlessly on snapshots of most subvolumes, but keeps > failing on snapshots of a certain

Re: subvols and parents - how?

On Tue, 2015-11-24 at 08:29 +, Duncan wrote: > OK, found it on the wiki.  It wasn't under use-cases, where I > initially > thought to look, but under sysadmin guide.  Specifically, see section > 4.2, managing snapshots, but I'd suggest reading the entire > subvolumes > discussion, section 4,

Re: subvols and parents - how?

On Tue, Nov 24, 2015 at 10:25:50PM +0100, Christoph Anton Mitterer wrote: > On Tue, 2015-11-24 at 08:29 +, Duncan wrote: > > OK, found it on the wiki.  It wasn't under use-cases, where I > > initially > > thought to look, but under sysadmin guide.  Specifically, see section > > 4.2, managing

Re: btrfs send reproducibly fails for a specific subvolume after sending 15 GiB, scrub reports no errors

On 2015-11-24 15:50, Christoph Anton Mitterer wrote: On Tue, 2015-11-24 at 15:44 -0500, Austin S Hemmelgarn wrote: I would say it's currently usable for one-shot stuff, but probably not reliably useable for automated things without some kind of administrative oversight. In theory, it wouldn't

Re: btrfs send reproducibly fails for a specific subvolume after sending 15 GiB, scrub reports no errors

On Tue, 2015-11-24 at 15:58 -0500, Austin S Hemmelgarn wrote: > I had tried using send/receive once with -p, but had numerous issues.   > The incrementals I've been doing have used -c instead, and I hadn't had  > any issues with data loss with that.  The issue outlined here was only a  > small

Re: btrfs check help

On 2015-11-24 12:06, Vincent Olivier wrote: Hi, Woke up this morning with a kernel panic (for which I do not have details). Please find below the output for btrfs check. Is this normal ? What should I do ? Arch Linux 4.2.5. Btrfs-utils 4.3.1. 17x4TB RAID10. You get bonus points for being on a

Re: shall distros run btrfsck on boot?

On 2015-11-24 12:23, Christoph Anton Mitterer wrote: On Tue, 2015-11-24 at 11:14 -0600, Eric Sandeen wrote: In a nutshell, though, I think a filesystem repair should be an admin-initiated action, not something that surprises you on a boot, at least for a journaling filesystem which is designed

Re: btrfs send reproducibly fails for a specific subvolume after sending 15 GiB, scrub reports no errors

On Tue, Nov 24, 2015 at 10:17:13PM +0100, Christoph Anton Mitterer wrote: > On Tue, 2015-11-24 at 15:58 -0500, Austin S Hemmelgarn wrote: > > I had tried using send/receive once with -p, but had numerous issues. >   > > The incrementals I've been doing have used -c instead, and I hadn't had  > >

Re: [PATCH v2 1/5] btrfs-progs: introduce framework to check kernel supported features

On 2015-11-24 09:39, Mike Fleetwood wrote: On 23 November 2015 at 12:56, Anand Jain wrote: In the newer kernel, supported kernel features can be known from /sys/fs/btrfs/features however this interface was introduced only after 3.14, and most the incompatible FS

Re: btrfs send reproducibly fails for a specific subvolume after sending 15 GiB, scrub reports no errors

On Tue, 2015-11-24 at 15:44 -0500, Austin S Hemmelgarn wrote: > I would say it's currently usable for one-shot stuff, but probably > not > reliably useable for automated things without some kind of > administrative oversight.  In theory, it wouldn't be hard to write a > script to automate

Re: btrfs check help

On Tue, Nov 24, 2015 at 03:28:28PM -0500, Austin S Hemmelgarn wrote: > On 2015-11-24 12:06, Vincent Olivier wrote: > >Hi, > > > >Woke up this morning with a kernel panic (for which I do not have details). > >Please find below the output for btrfs check. Is this normal ? What should I > >do ?

Re: btrfs send reproducibly fails for a specific subvolume after sending 15 GiB, scrub reports no errors

On 2015-11-24 13:48, Christoph Anton Mitterer wrote: Hey. All that sounds pretty serious, doesn't it? So in other words, AFAIU, send/receive cannot really be reliably used. I did so far for making incremental backups, but I've also experienced some problems (though not what this is about

Re: btrfs send reproducibly fails for a specific subvolume after sending 15 GiB, scrub reports no errors

On Tue, 2015-11-24 at 21:27 +, Hugo Mills wrote: >    -p only sends the file metadata for the changes from the reference > snapshot to the sent snapshot. -c sends all the file metadata, but > will preserve the reflinks between the sent snapshot and the (one or > more) reference snapshots. Let

Re: Using Btrfs on single drives

On Sun, 15 Nov 2015 03:01:57 PM Duncan wrote: > That looks to me like native drive limitations. > > Due to the fact that a modern hard drive spins at the same speed no > matter where the read/write head is located, when it's reading/writing to > the first part of the drive -- the outside --

Re: shall distros run btrfsck on boot?

On Tue, 2015-11-24 at 11:14 -0600, Eric Sandeen wrote: > In a nutshell, though, I think a filesystem repair should be an > admin-initiated > action, not something that surprises you on a boot, at least for a > journaling > filesystem which is designed to maintain its integrity even in the > face

Re: [Bug] btrfs-progs v4.3.1, mkfs.btrfs manpage, profiles table missing raid1

On Mon, Nov 23, 2015 at 07:07:51PM +0100, David Sterba wrote: > > Also, it calls raid5/6 "copies" rather than "parity". Perhaps add > > another column for parity, change the redundancy column to copies, and > > adjust accordingly? Alternatively, keep the single redundancy column and > > just

Re: [PATCH] btrfs-progs: fsck: Fix a false alert where extent record has wrong metadata flag

On Tue, Nov 24, 2015 at 01:16:51PM +0800, Qu Wenruo wrote: > In process_extent_item(), it gives 'metadata' initial value 0, but for > non-skinny-metadata case, metadata extent can't be judged just from key > type and it forgot that case. > > This causes a lot of false alert in non-skinny-metadata

Re: bad extent [5993525264384, 5993525280768), type mismatch with chunk

On Tue, Nov 24, 2015 at 08:46:03AM +0800, Qu Wenruo wrote: > > > Christoph Anton Mitterer wrote on 2015/11/23 19:12 +0100: > > On Mon, 2015-11-23 at 09:10 +0800, Qu Wenruo wrote: > >> Also, you won't want compiler to do extra optimization > > I did the following: > > $ export CFLAGS="-g -O0

Re: bad extent [5993525264384, 5993525280768), type mismatch with chunk

On Tue, 2015-11-24 at 13:35 +0800, Qu Wenruo wrote: > Hopes you didn't wait too long. No worries, didn't hold my breath ;) > The fixing patch is CCed to you, or you can get it from patchwork: > https://patchwork.kernel.org/patch/7687611/ Unfortunately that doesn't make the error messages go

Re: btrfs send reproducibly fails for a specific subvolume after sending 15 GiB, scrub reports no errors

Hey. All that sounds pretty serious, doesn't it? So in other words, AFAIU, send/receive cannot really be reliably used. I did so far for making incremental backups, but I've also experienced some problems (though not what this is about here). Cheers, Chris. smime.p7s Description: S/MIME

Re: shall distros run btrfsck on boot?

On 11/24/15 12:56 AM, Duncan wrote: > Duncan posted on Tue, 24 Nov 2015 06:46:18 + as excerpted: > >> That wouldn't be entirely uncommon, because as Eric mentions, btrfs >> check is intended to be thorough, where the kernel mount-time check is >> intended to be fast. >> >> But of course, as

btrfs check help

Hi, Woke up this morning with a kernel panic (for which I do not have details). Please find below the output for btrfs check. Is this normal ? What should I do ? Arch Linux 4.2.5. Btrfs-utils 4.3.1. 17x4TB RAID10. Regards, Vincent [root@3dcpc5 ~]# btrfs check /dev/sdk Checking filesystem on

Re: btrfs send reproducibly fails for a specific subvolume after sending 15 GiB, scrub reports no errors

On Tue, Nov 24, 2015 at 10:36:26PM +0100, Christoph Anton Mitterer wrote: > On Tue, 2015-11-24 at 21:27 +, Hugo Mills wrote: > >    -p only sends the file metadata for the changes from the reference > > snapshot to the sent snapshot. -c sends all the file metadata, but > > will preserve the

Re: shall distros run btrfsck on boot?

On 11/24/15 2:38 PM, Austin S Hemmelgarn wrote: > if the system was > shut down cleanly, you're fine barring software bugs, but if it > crashed, you should be running a check on the FS. Um, no... The *entire point* of having a journaling filesystem is that after a crash or power loss, a journal

Re: bad extent [5993525264384, 5993525280768), type mismatch with chunk

On 11/24/2015 09:15 PM, Laurent Bonnaud wrote: On 23/11/2015 02:00, Qu Wenruo wrote: Considering the size, I'd like not to touch the dump, metadata is over 5G, It is only 2GB once compressed :>. and I think it's not related to on-disk data, but runtime problem like I mentioned above.

Re: bad extent [5993525264384, 5993525280768), type mismatch with chunk

On 11/25/2015 02:25 AM, Christoph Anton Mitterer wrote: On Tue, 2015-11-24 at 13:35 +0800, Qu Wenruo wrote: Hopes you didn't wait too long. No worries, didn't hold my breath ;) The fixing patch is CCed to you, or you can get it from patchwork: https://patchwork.kernel.org/patch/7687611/

Re: subvols and parents - how?

Christoph Anton Mitterer posted on Tue, 24 Nov 2015 05:56:00 +0100 as excerpted: > When I use subvolumes than these have always a parent subvolume (except > ID5), so I can basically decide between two ways: > > a) make child subvolumes, e.g. > 5 > | > +-root   (=subvol, mountpoint /) >   +-boot/

Re: [RFC][PATCH 00/12] Enhanced file stat system call

Am Dienstag, 24. November 2015, 00:13:08 CET schrieb Christoph Hellwig: > On Fri, Nov 20, 2015 at 05:19:31PM +0100, Martin Steigerwald wrote: > > I know its mostly relevant for just for FAT32, but on any account rather > > than trying to write 4 GiB and then file, it would be good to at some > >

Re: [PATCH 00/25] Btrfs-convert rework to support native separate

David Sterba wrote on 2015/11/23 18:33 +0100: On Fri, Nov 20, 2015 at 11:24:04AM +0800, Qu Wenruo wrote: Here comes the 1st version of btrfs-convert rework. Any test is welcomed, and it can already pass the convert test from btrfs-progs. (Since the test doesn't test rollback function) I

Re: [RFC][PATCH 00/12] Enhanced file stat system call

On Tue, Nov 24, 2015 at 09:48:22AM +0100, Martin Steigerwald wrote: > It might be interesting for BTRFS as well, to be able to ask what amount of > free space there currently is *at* a given path. Cause with BTRFS and > Subvolumes this may differ between different paths. We can handle this

[PATCH] fstests: speedup generic/027 for new version of btrfs

From: Zhao Lei New version of btrfs create non-mixed blockgroups in all case. For generic/027, the filesystem in test is convert from mixed-blockgroup to non-mixed blockgroup. And test time is changed from 400s -> 2700s in my node. To test btrfs with all mountoptions,

Re: [PATCH v2 1/5] btrfs-progs: introduce framework to check kernel supported features

On 23 November 2015 at 12:56, Anand Jain wrote: > In the newer kernel, supported kernel features can be known from > /sys/fs/btrfs/features > however this interface was introduced only after 3.14, and most the > incompatible FS features were introduce before 3.14. > >

Re: [PATCH v2 0/5] Make btrfs-progs really compatible with any kernel version

Thanks for comments. Distro would also want to use the latest btrfs-progs on older kernel since it will have latest fsck/send/receive fixes, better UI and updated man pages. btrfs-progs which claim backward kernel compatible and it shouldn't fail on the below cmd when the btrfs-progs is

Re: btrfs send reproducibly fails for a specific subvolume after sending 15 GiB, scrub reports no errors

On 2015-11-24 00:42, Duncan wrote: Nils Steinger posted on Mon, 23 Nov 2015 22:10:12 +0100 as excerpted: Do we anything about what might cause a filesystem to enter a state which `send` chokes on? I've only seen a small sample of the corrupted files before growing tired of the process and just

Re: bad extent [5993525264384, 5993525280768), type mismatch with chunk

On 23/11/2015 02:00, Qu Wenruo wrote: > Considering the size, I'd like not to touch the dump, metadata is over 5G, It is only 2GB once compressed :>. > and I think it's not related to on-disk data, but runtime problem like I > mentioned above. To test this hypothesis I did the following: -

Re: [PATCH v2 0/5] Make btrfs-progs really compatible with any kernel version

David Sterba wrote: On Mon, Nov 23, 2015 at 08:56:13PM +0800, Anand Jain wrote: Btrfs-progs is a tool for the btrfs kernel and we hope latest btrfs-progs be compatible w any set of older/newer kernels. So far mkfs.btrfs and btrfs-convert sets the default features, for eg, skinny-metadata