Re: shall distros run btrfsck on boot?

Am Mittwoch, 25. November 2015, 07:32:34 CET schrieb Austin S Hemmelgarn: > On 2015-11-24 17:26, Eric Sandeen wrote: > > On 11/24/15 2:38 PM, Austin S Hemmelgarn wrote: > >> if the system was > >> shut down cleanly, you're fine barring software bugs, but if it > >> crashed, you should be running a

Re: btrfs check help

I should probably point out that there is 64GB of RAM on this machine and it’s a dual Xeon processor (LGA2011-3) system. Also, there is only Btrfs served via Samba and the kernel panic was caused Btrfs (as per what I remember from the log on the screen just before I rebooted) and happened in

Re: [4.3-rc4] scrubbing aborts before finishing

Am Samstag, 31. Oktober 2015, 12:10:37 CET schrieb Martin Steigerwald: > Am Donnerstag, 22. Oktober 2015, 10:41:15 CET schrieb Martin Steigerwald: > > I get this: > > > > merkaba:~> btrfs scrub status -d / > > scrub status for […] > > scrub device /dev/mapper/sata-debian (id 1) history > > > >

Re: [auto-]defrag, nodatacow - general suggestions?(was: btrfs: poor performance on deleting many large files?)

On Thu, Nov 26, 2015 at 01:23:59AM +0100, Christoph Anton Mitterer wrote: > 2) Why does notdatacow imply nodatasum and can that ever be decoupled? Answering the second part first, no, it can't. The issue is that nodatacow bypasses the transactional nature of the FS, making changes to live

Re: [PATCH 00/25] Btrfs-convert rework to support native separate

David Sterba wrote on 2015/11/25 13:42 +0100: On Tue, Nov 24, 2015 at 04:50:00PM +0800, Qu Wenruo wrote: It seems the conflict is quite huge, your reiserfs support is based on the old behavior, just like what old ext2 one do: custom extent allocation. I'm afraid the rebase will take a lot

Re: btrfs check help

[...] > Ca I get a strong confirmation that I should run with the “—repair” option on > each device? Thanks. > > Vincent > > > Checking filesystem on /dev/sdk > UUID: 6a742786-070d-4557-9e67-c73b84967bf5 > checking extents [o] > checking free space cache [.] > root 5 inode 1341670 errors 400,

[PATCH V1.1 2/7] btrfs-progs: add kernel alias for each of the features in the list

We should have maintained feature's name same across progs UI and sysfs UI. For example, progs mixed-bg is /sys/fs/btrfs/features/mixed_groups in sysfs. As these are already released and is UIs, there is nothing much can be done about it, except for creating the alias and making it aware. Add

[RFC PATCH] Btrfs: improve performance on dbench

Kent Overstreet posted some dbench test numbers in the announcement of bcachefs[1], in which btrfs's performance is much worse than that of ext4 and xfs, especially in the case of multiple threads. This difference can be observed on fast storage, I ran 'dbench -t10 64' with 1.6T NVMe disk,

Re: [PATCH 2/7] btrfs-progs: add kernel alias for each of the features in the list

Liu Bo wrote: On Wed, Nov 25, 2015 at 08:08:15PM +0800, Anand Jain wrote: We should have maintained feature's name same across progs UI and sysfs UI. For example, progs mixed-bg is /sys/fs/btrfs/features/mixed_groups in sysfs. As these are already released and is UIs, there is nothing much

Re: [auto-]defrag, nodatacow - general suggestions?(was: btrfs: poor performance on deleting many large files?)

Hey. I've worried before about the topics Mitch has raised. Some questions. 1) AFAIU, the fragmentation problem exists especially for those files that see many random writes, especially, but not limited to, big files. Now that databases and VMs are affected by this, is probably broadly known in

Re: 4.2.6: livelock in recovery (free_reloc_roots)?

On 11/21/2015 10:01 PM, Alexander Fougner wrote as excerpted: > This is fixed in btrfs-progs 4.3.1, that allows you to delete a > device again by the 'missing' keyword. Thanks Alexander! I just found the thread reporting the bug but not the patch with the corresponding btrfs-tools version it was

Re: [PATCH 0/7] Let user specify the kernel version for features

Anand Jain wrote on 2015/11/25 20:08 +0800: Sometimes users may want to have a btrfs to be supported on multiple kernel version. A simple example, USB drive can be used with multiple system running different kernel versions. Or in a data center a SAN LUN could be mounted on any system with

Re: [PATCH] Btrfs: fix error path when failing to submit bio for direct IO write

On Wed, Nov 25, 2015 at 5:58 PM, Liu Bo wrote: > On Tue, Nov 24, 2015 at 05:25:18PM +, fdman...@kernel.org wrote: >> From: Filipe Manana >> >> Commit 61de718fceb6 ("Btrfs: fix memory corruption on failure to submit >> bio for direct IO") fixed

[PATCH v2] Btrfs: fix error path when failing to submit bio for direct IO write

From: Filipe Manana Commit 61de718fceb6 ("Btrfs: fix memory corruption on failure to submit bio for direct IO") fixed problems with the error handling code after we fail to submit a bio for direct IO. However there were 2 problems that it did not address when the failure is

Re: [PATCH] Btrfs: fix error path when failing to submit bio for direct IO write

On Tue, Nov 24, 2015 at 05:25:18PM +, fdman...@kernel.org wrote: > From: Filipe Manana > > Commit 61de718fceb6 ("Btrfs: fix memory corruption on failure to submit > bio for direct IO") fixed problems with the error handling code after we > fail to submit a bio for direct

Re: [PATCH v2] Btrfs: fix error path when failing to submit bio for direct IO write

On Tue, Nov 24, 2015 at 11:35:25PM +, fdman...@kernel.org wrote: > From: Filipe Manana > > Commit 61de718fceb6 ("Btrfs: fix memory corruption on failure to submit > bio for direct IO") fixed problems with the error handling code after we > fail to submit a bio for direct

Re: btrfs: poor performance on deleting many large files

On Mon, 2015-11-23 at 06:29 +, Duncan wrote: > Using subvolumes was the first recommendation I was going to make, too, > so you're on the right track. =:^) > > Also, in case you are using it (you didn't say, but this has been > demonstrated to solve similar issues for others so it's worth

Re: [PATCH 2/7] btrfs-progs: add kernel alias for each of the features in the list

On Wed, Nov 25, 2015 at 08:08:15PM +0800, Anand Jain wrote: > We should have maintained feature's name same across progs UI and sysfs UI. > For example, progs mixed-bg is /sys/fs/btrfs/features/mixed_groups > in sysfs. As these are already released and is UIs, there is nothing much > can be done

Re: Imbalanced RAID1 with three unequal disks

On Wed, Nov 25, 2015 at 12:36:32PM +0100, Mario wrote: > > Hi, > > I pushed a subvolume using send/receive to an 8 TB disk, added > two 4 TB disks and started a balance with conversion to RAID1. > > Afterwards, I got the following: > > Total devices 3 FS bytes used 5.40TiB > devid1

Re: [PATCH 0/7] Let user specify the kernel version for features

Anand Jain wrote on 2015/11/26 14:07 +0800: On 11/26/2015 10:02 AM, Qu Wenruo wrote: Anand Jain wrote on 2015/11/25 20:08 +0800: Sometimes users may want to have a btrfs to be supported on multiple kernel version. A simple example, USB drive can be used with multiple system running

Re: bad extent [5993525264384, 5993525280768), type mismatch with chunk

On 25/11/2015 00:46, Qu Wenruo wrote: > The size seems small enough, I'll try to download it as it's super useful to > debug it. Thanks ! > Nice reproducer. > Is it 100% reproducible or has a chance to reproduce? I tried a second time and got a similar kernel backtrace. > BTW, did you

[PATCH] Btrfs: fix error path when failing to submit bio for direct IO write

From: Filipe Manana Commit 61de718fceb6 ("Btrfs: fix memory corruption on failure to submit bio for direct IO") fixed problems with the error handling code after we fail to submit a bio for direct IO. However there were 2 problems that it did not address when the failure is

Re: shall distros run btrfsck on boot?

On 2015-11-24 17:26, Eric Sandeen wrote: On 11/24/15 2:38 PM, Austin S Hemmelgarn wrote: if the system was shut down cleanly, you're fine barring software bugs, but if it crashed, you should be running a check on the FS. Um, no... The *entire point* of having a journaling filesystem is that

[PATCH 10/12] Fix btrfs/098 to work on non-4k block sized filesystems

This commit makes use of the new _filter_xfs_io_blocks_modified filtering function to print information in terms of file blocks rather than file offset. Signed-off-by: Chandan Rajendra --- tests/btrfs/098 | 65 ++---

[PATCH 05/12] Fix btrfs/056 to work on non-4k block sized filesystems

This commit makes use of the new _filter_xfs_io_blocks_modified and _filter_od filtering functions to print information in terms of file blocks rather than file offset. Signed-off-by: Chandan Rajendra --- tests/btrfs/056 | 51 ++

[PATCH 09/12] Fix btrfs/097 to work on non-4k block sized filesystems

This commit makes use of the new _filter_xfs_io_blocks_modified filtering function to print information in terms of file blocks rather than file offset. Signed-off-by: Chandan Rajendra --- tests/btrfs/097 | 42 +-

[PATCH 07/12] Fix btrfs/095 to work on non-4k block sized filesystems

This commit makes use of the new _filter_xfs_io_blocks_modified filtering function to print information in terms of file blocks rather than file offset. Signed-off-by: Chandan Rajendra --- tests/btrfs/095 | 113 +---

[PATCH 08/12] Fix btrfs/096 to work on non-4k block sized filesystems

This commit makes use of the new _filter_xfs_io_blocks_modified filtering function to print information in terms of file blocks rather than file offset. Signed-off-by: Chandan Rajendra --- tests/btrfs/096 | 45 +

[PATCH 04/12] Fix btrfs/055 to work on non-4k block sized filesystems

This commit makes use of the new _filter_xfs_io_blocks_modified and _filter_od filtering functions to print information in terms of file blocks rather than file offset. Signed-off-by: Chandan Rajendra --- tests/btrfs/055 | 128 ++

[PATCH 11/12] Fix btrfs/103 to work on non-4k block sized filesystems

This commit makes use of the new _filter_xfs_io_blocks_modified filtering function to print information in terms of file blocks rather than file offset. Signed-off-by: Chandan Rajendra --- tests/btrfs/103 | 47 +--

[PATCH 00/12] Fix Btrfs tests to work on non-4k block sized fs instances

This patchset fixes Btrfs tests to work on variable block size. This is based off the RFC patch sent during March of this year (https://www.marc.info/?l=linux-btrfs=142736088310300=2). Currently, some of the tests are written with the assumption that 4k is the block size of the filesystem

[PATCH 01/12] Filter xfs_io and od's output in units of FS block size and the CPU's page size

The helpers will be used to make btrfs tests that assume 4k as the block size to work on non-4k blocksized filesystem instances as well. Signed-off-by: Chandan Rajendra --- common/filter | 52 common/rc | 5

[PATCH 03/12] Fix btrfs/052 to work on non-4k block sized filesystems

This commit makes use of the new _filter_xfs_io_blocks_modified filtering function to print information in terms of file blocks rather than file offset. Signed-off-by: Chandan Rajendra --- tests/btrfs/052 | 127 +++- tests/btrfs/052.out | 546

[PATCH 02/12] Fix btrfs/017 to work on non-4k block sized filesystems

This commit makes use of the new _filter_xfs_io_blocks_modified filtering function to print information in terms of file blocks rather than file offset. Signed-off-by: Chandan Rajendra --- tests/btrfs/017 | 16 tests/btrfs/017.out | 3 +-- 2

Re: [PATCH 06/12] Fix btrfs/094 to work on non-4k block sized filesystems

On Wednesday 25 Nov 2015 11:51:52 Filipe Manana wrote: > On Wed, Nov 25, 2015 at 11:47 AM, Chandan Rajendra > > wrote: > > On Wednesday 25 Nov 2015 11:11:27 Filipe Manana wrote: > >> Hi Chandan, > >> > >> I can't agree with this change. We're no longer checking that

[PATCH 5/7] btrfs-progs: introduce framework version to features

As discussed in the mailing list this provides a framework to introduce the feature where mkfs and btrfs-convert can set the default features as per the given mainline kernel version. Suggested-by: David Sterba Signed-off-by: Anand Jain --- utils.c | 23

[PATCH 1/7] btrfs-progs: show the version for -O list-all

Shows min kernel version in the -O list-all output eg: (version is show with in ()) btrfs-convert -O list-all Filesystem features available: extref - increased hardlink limit per file to 65536 (0x40, 3.7, default) skinny-metadata - reduced-size metadata extent refs (0x100, 3.10,

[PATCH 6/7] btrfs-progs: add -O comp= option for mkfs.btrfs

This provides default feature set by version, for mkfs.btrfs through the new option '-O comp=|', where x.y.z is the minimum kernel version that should be supported. Signed-off-by: Anand Jain --- mkfs.c | 24 ++-- 1 file changed, 22 insertions(+), 2

[PATCH 0/7] Let user specify the kernel version for features

Sometimes users may want to have a btrfs to be supported on multiple kernel version. A simple example, USB drive can be used with multiple system running different kernel versions. Or in a data center a SAN LUN could be mounted on any system with different kernel version. Thanks for providing

[PATCH 2/7] btrfs-progs: add kernel alias for each of the features in the list

We should have maintained feature's name same across progs UI and sysfs UI. For example, progs mixed-bg is /sys/fs/btrfs/features/mixed_groups in sysfs. As these are already released and is UIs, there is nothing much can be done about it, except for creating the alias and making it aware. Add

[PATCH 4/7] btrfs-progs: check for numerical in version_to_code()

As the version is now being passed by user it should be checked if its numerical. We didn't need this before as version wasn't passed by used. So this is not a bug fix. Signed-off-by: Anand Jain --- utils.c | 10 +++--- 1 file changed, 7 insertions(+), 3 deletions(-)

[PATCH 3/7] btrfs-progs: make is_numerical non static

Signed-off-by: Anand Jain --- cmds-replace.c | 11 --- utils.c| 11 +++ utils.h| 1 + 3 files changed, 12 insertions(+), 11 deletions(-) diff --git a/cmds-replace.c b/cmds-replace.c index 9ab8438..86162b6 100644 --- a/cmds-replace.c +++

[PATCH 7/7] btrfs-progs: add -O comp= option for btrfs-convert

User may want to convert the FS to a minimum kernel version. As they may need to use btrfs on a set of known kernel versions. And have the disk layout compatible. Signed-off-by: Anand Jain --- btrfs-convert.c | 21 + 1 file changed, 21 insertions(+)

[PATCH v3 1/5] btrfs-progs: introduce framework to check kernel supported features

In the newer kernel, supported kernel features can be known from /sys/fs/btrfs/features however this interface was introduced only after 3.14, and most the incompatible FS features were introduce before 3.14. This patch proposes to maintain kernel version against the feature list, and so that

[PATCH 12/12] Fix btrfs/106 to work on non-4k block sized filesystems

This commit makes use of the new _filter_xfs_io_pages_modified filtering function to print information in terms of file blocks rather than file offset. Signed-off-by: Chandan Rajendra --- tests/btrfs/106 | 42 --

[PATCH 06/12] Fix btrfs/094 to work on non-4k block sized filesystems

This commit makes use of the new _filter_xfs_io_blocks_modified filtering function to print information in terms of file blocks rather than file offset. Signed-off-by: Chandan Rajendra --- tests/btrfs/094 | 78 +++--

Re: [PATCH 06/12] Fix btrfs/094 to work on non-4k block sized filesystems

On Wed, Nov 25, 2015 at 11:03 AM, Chandan Rajendra wrote: > This commit makes use of the new _filter_xfs_io_blocks_modified filtering > function to print information in terms of file blocks rather than file > offset. > > Signed-off-by: Chandan Rajendra

Imbalanced RAID1 with three unequal disks

Hi, I pushed a subvolume using send/receive to an 8 TB disk, added two 4 TB disks and started a balance with conversion to RAID1. Afterwards, I got the following: Total devices 3 FS bytes used 5.40TiB devid1 size 7.28TiB used 4.54TiB path /dev/mapper/yellow4 devid2 size 3.64TiB

Re: [PATCH 06/12] Fix btrfs/094 to work on non-4k block sized filesystems

On Wed, Nov 25, 2015 at 11:47 AM, Chandan Rajendra wrote: > On Wednesday 25 Nov 2015 11:11:27 Filipe Manana wrote: >> >> Hi Chandan, >> >> I can't agree with this change. We're no longer checking that file >> data is correct after the cloning operations. The md5sum

Re: [PATCH 06/12] Fix btrfs/094 to work on non-4k block sized filesystems

On Wednesday 25 Nov 2015 11:11:27 Filipe Manana wrote: > > Hi Chandan, > > I can't agree with this change. We're no longer checking that file > data is correct after the cloning operations. The md5sum checks were > exactly for that. So essentially the test is only verifying the clone >

Re: [PATCH 00/25] Btrfs-convert rework to support native separate

On Tue, Nov 24, 2015 at 04:50:00PM +0800, Qu Wenruo wrote: > It seems the conflict is quite huge, your reiserfs support is based on > the old behavior, just like what old ext2 one do: custom extent allocation. > I'm afraid the rebase will take a lot of time since I'm completely a > newbie about

Re: [PATCH 0/7] Let user specify the kernel version for features

On 11/26/2015 10:02 AM, Qu Wenruo wrote: Anand Jain wrote on 2015/11/25 20:08 +0800: Sometimes users may want to have a btrfs to be supported on multiple kernel version. A simple example, USB drive can be used with multiple system running different kernel versions. Or in a data center a SAN