Re: [patch] file dedupe (and maybe clone) data corruption (was Re: [PATCH] generic: test for deduplication between different files)

2018-09-20 Thread Zygo Blaxell
On Fri, Sep 21, 2018 at 12:59:31PM +1000, Dave Chinner wrote: > On Wed, Sep 19, 2018 at 12:12:03AM -0400, Zygo Blaxell wrote: [...] > With no DMAPI in the future, people with custom HSM-like interfaces > based on dmapi are starting to turn to fanotify and friends to > provide them with the change

Re: [patch] file dedupe (and maybe clone) data corruption (was Re: [PATCH] generic: test for deduplication between different files)

2018-09-20 Thread Dave Chinner
On Wed, Sep 19, 2018 at 12:12:03AM -0400, Zygo Blaxell wrote: > On Mon, Sep 10, 2018 at 07:06:46PM +1000, Dave Chinner wrote: > > On Thu, Sep 06, 2018 at 11:53:06PM -0400, Zygo Blaxell wrote: > > > On Thu, Sep 06, 2018 at 06:38:09PM +1000, Dave Chinner wrote: > > > > On Fri, Aug 31, 2018 at

Re: btrfs problems

2018-09-20 Thread Remi Gauvin
On 2018-09-20 05:35 PM, Adrian Bastholm wrote: > Thanks a lot for the detailed explanation. > Aabout "stable hardware/no lying hardware". I'm not running any raid > hardware, was planning on just software raid. three drives glued > together with "mkfs.btrfs -d raid5 /dev/sdb /dev/sdc /dev/sdd".

Re: btrfs problems

2018-09-20 Thread Chris Murphy
On Thu, Sep 20, 2018 at 3:36 PM Adrian Bastholm wrote: > > Thanks a lot for the detailed explanation. > Aabout "stable hardware/no lying hardware". I'm not running any raid > hardware, was planning on just software raid. Yep. I'm referring to the drives, their firmware, cables, logic board, its

Re: btrfs problems

2018-09-20 Thread Adrian Bastholm
Thanks a lot for the detailed explanation. Aabout "stable hardware/no lying hardware". I'm not running any raid hardware, was planning on just software raid. three drives glued together with "mkfs.btrfs -d raid5 /dev/sdb /dev/sdc /dev/sdd". Would this be a safer bet, or would You recommend running

Re: btrfs problems

2018-09-20 Thread Chris Murphy
On Thu, Sep 20, 2018 at 11:23 AM, Adrian Bastholm wrote: > On Mon, Sep 17, 2018 at 2:44 PM Qu Wenruo wrote: > >> >> Then I strongly recommend to use the latest upstream kernel and progs >> for btrfs. (thus using Debian Testing) >> >> And if anything went wrong, please report asap to the mail

Re: inline extents

2018-09-20 Thread Theodore Y. Ts'o
On Wed, Sep 19, 2018 at 09:18:16PM -0600, Chris Murphy wrote: > > ext4 has inline data, too, so there's every chance grub will corrupt > > ext4 filesystems with tit's wonderful new feature. I'm not sure if > > the ext4 metadata cksums cover the entire inode and inline data, but > > if they do it's

Re: [PATCH v8 0/6] Btrfs: implement swap file support

2018-09-20 Thread Omar Sandoval
On Thu, Sep 20, 2018 at 07:22:55PM +0200, David Sterba wrote: > On Wed, Sep 19, 2018 at 10:02:11PM -0700, Omar Sandoval wrote: > > From: Omar Sandoval > > Changes from v7 [1]: > > > > - Expanded a few commit messages > > - Added Johannes' acked-by on patches 1 and 2 > > - Rebased on v4.19-rc4 >

Re: [PATCH 0/5] rb_first to rb_first_cached conversion

2018-09-20 Thread Liu Bo
On Thu, Sep 20, 2018 at 06:49:59PM +0200, David Sterba wrote: > On Thu, Aug 23, 2018 at 03:51:48AM +0800, Liu Bo wrote: > > Several structs in btrfs are using rb_first() in a while loop, it'd be > > more efficient to do this with rb_first_cached() which has the O(1) > > complexity. > > > > This

Re: btrfs send hangs after partial transfer and blocks all IO

2018-09-20 Thread Chris Murphy
On Wed, Sep 19, 2018 at 1:41 PM, Jürgen Herrmann wrote: > Am 13.9.2018 14:35, schrieb Nikolay Borisov: >> >> On 13.09.2018 15:30, Jürgen Herrmann wrote: >>> >>> OK, I will install kdump later and perform a dump after the hang. >>> >>> One more noob question beforehand: does this dump contain

Re: btrfs problems

2018-09-20 Thread Adrian Bastholm
On Mon, Sep 17, 2018 at 2:44 PM Qu Wenruo wrote: > > Then I strongly recommend to use the latest upstream kernel and progs > for btrfs. (thus using Debian Testing) > > And if anything went wrong, please report asap to the mail list. > > Especially for fs corruption, that's the ghost I'm always

Re: [PATCH v8 6/6] Btrfs: support swap files

2018-09-20 Thread Omar Sandoval
On Thu, Sep 20, 2018 at 07:15:41PM +0200, David Sterba wrote: > On Wed, Sep 19, 2018 at 10:02:17PM -0700, Omar Sandoval wrote: > > From: Omar Sandoval > > > > Btrfs has not allowed swap files since commit 35054394c4b3 ("Btrfs: stop > > providing a bmap operation to avoid swapfile corruptions").

Re: [PATCH v8 0/6] Btrfs: implement swap file support

2018-09-20 Thread David Sterba
On Wed, Sep 19, 2018 at 10:02:11PM -0700, Omar Sandoval wrote: > From: Omar Sandoval > Changes from v7 [1]: > > - Expanded a few commit messages > - Added Johannes' acked-by on patches 1 and 2 > - Rebased on v4.19-rc4 I've sent my comments, it's mostly about the usability or small enhancements.

Re: [PATCH v8 6/6] Btrfs: support swap files

2018-09-20 Thread David Sterba
On Wed, Sep 19, 2018 at 10:02:17PM -0700, Omar Sandoval wrote: > From: Omar Sandoval > > Btrfs has not allowed swap files since commit 35054394c4b3 ("Btrfs: stop > providing a bmap operation to avoid swapfile corruptions"). However, now > that the proper restrictions are in place, Btrfs can

Re: [PATCH v8 5/6] Btrfs: rename get_chunk_map() and make it non-static

2018-09-20 Thread David Sterba
On Wed, Sep 19, 2018 at 10:02:16PM -0700, Omar Sandoval wrote: > From: Omar Sandoval > > The Btrfs swap code is going to need it, so give it a btrfs_ prefix and > make it non-static. > > Reviewed-by: Nikolay Borisov > Signed-off-by: Omar Sandoval > --- > fs/btrfs/volumes.c | 29

Re: [PATCH v8 4/6] Btrfs: prevent ioctls from interfering with a swap file

2018-09-20 Thread David Sterba
On Wed, Sep 19, 2018 at 10:02:15PM -0700, Omar Sandoval wrote: > --- a/fs/btrfs/dev-replace.c > +++ b/fs/btrfs/dev-replace.c > @@ -414,6 +414,14 @@ int btrfs_dev_replace_start(struct btrfs_fs_info > *fs_info, > if (ret) > return ret; > > + if

Re: [PATCH 0/5] rb_first to rb_first_cached conversion

2018-09-20 Thread David Sterba
On Thu, Aug 23, 2018 at 03:51:48AM +0800, Liu Bo wrote: > Several structs in btrfs are using rb_first() in a while loop, it'd be > more efficient to do this with rb_first_cached() which has the O(1) > complexity. > > This patch set updates five structs which may have a large rb tree in > practice

Re: cannot mount btrfs as root

2018-09-20 Thread Zbigniew 'zibi' Jarosik
On 20 September 2018 at 08:20, Zbigniew 'zibi' Jarosik wrote: > On 19 September 2018 at 15:54, Qu Wenruo wrote: >> >> Does the iniramfs do a "btrfs dev scan" to make populate btrfs devices >> lists? > > This solved problem. Looks like btrfs scans disks to early, before > bcache initializes. I

Re: [PATCH 0/5] rb_first to rb_first_cached conversion

2018-09-20 Thread David Sterba
On Fri, Sep 14, 2018 at 12:14:57PM -0700, Liu Bo wrote: > On Fri, Sep 14, 2018 at 05:11:02PM +0200, David Sterba wrote: > > On Tue, Sep 11, 2018 at 11:31:49AM -0700, Liu Bo wrote: > > > On Tue, Sep 11, 2018 at 05:34:03PM +0200, David Sterba wrote: > > > > On Thu, Aug 23, 2018 at 03:51:48AM +0800,

Re: Punch hole on full fs

2018-09-20 Thread Qu Wenruo
On 2018/9/20 下午5:04, Anand Jain wrote: > > > On 09/20/2018 04:45 PM, Qu Wenruo wrote: >> >> >> On 2018/9/20 下午1:26, anand.j...@oracle.com wrote: >>> >>> >>> Test script [1] tries to punch hole on a full FS and it works fine as >>> long as the hole size and the offset is aligned with the

Re: [PATCH V5 RESEND] Btrfs: enchanse raid1/10 balance heuristic

2018-09-20 Thread Timofey Titovets
чт, 20 сент. 2018 г. в 12:05, Peter Becker : > > i like the idea. > do you have any benchmarks for this change? > > the general logic looks good for me. https://patchwork.kernel.org/patch/10137909/ > > Tested-by: Dmitrii Tcvetkov > > Benchmark summary (arithmetic mean of 3 runs): > Mainline

Re: Punch hole on full fs

2018-09-20 Thread Qu Wenruo
On 2018/9/20 下午5:04, Anand Jain wrote: > > > On 09/20/2018 04:45 PM, Qu Wenruo wrote: >> >> >> On 2018/9/20 下午1:26, anand.j...@oracle.com wrote: >>> >>> >>> Test script [1] tries to punch hole on a full FS and it works fine as >>> long as the hole size and the offset is aligned with the

Re: [PATCH V5 RESEND] Btrfs: enchanse raid1/10 balance heuristic

2018-09-20 Thread Peter Becker
i like the idea. do you have any benchmarks for this change? the general logic looks good for me.

Re: Punch hole on full fs

2018-09-20 Thread Anand Jain
On 09/20/2018 04:45 PM, Qu Wenruo wrote: On 2018/9/20 下午1:26, anand.j...@oracle.com wrote: Test script [1] tries to punch hole on a full FS and it works fine as long as the hole size and the offset is aligned with the sectorsize and the extent, so that it could just drop the relevant

Re: Punch hole on full fs

2018-09-20 Thread Qu Wenruo
On 2018/9/20 下午1:26, anand.j...@oracle.com wrote: > > > Test script [1] tries to punch hole on a full FS and it works fine as > long as the hole size and the offset is aligned with the sectorsize and > the extent, so that it could just drop the relevant extent to create the > hole. > > The

Re: [RFC PATCH v2 0/4] btrfs-progs: build distinct binaries for specific btrfs subcommands

2018-09-20 Thread Duncan
Axel Burri posted on Thu, 20 Sep 2018 00:02:22 +0200 as excerpted: > Now not everybody wants to install these with fscaps or setuid, but it > might also make sense to provide "/usr/bin/btrfs-subvolume-{show,list}", > as they now work for a regular user. Having both root/user binaries >

Re: very poor performance / a lot of writes to disk with space_cache (but not with space_cache=v2)

2018-09-20 Thread Duncan
Tomasz Chmielewski posted on Wed, 19 Sep 2018 10:43:18 +0200 as excerpted: > I have a mysql slave which writes to a RAID-1 btrfs filesystem (with > 4.17.14 kernel) on 3 x ~1.9 TB SSD disks; filesystem is around 40% full. > > The slave receives around 0.5-1 MB/s of data from the master over the >

Re: btrfs filesystem show takes a long time

2018-09-20 Thread Stefan K
Hi, > You may try to run the show command under strace to see where it blocks. any recommendations for strace options? On Friday, September 14, 2018 1:25:30 PM CEST David Sterba wrote: > Hi, > > thanks for the report, I've forwarded it to the issue tracker >

Re: cannot mount btrfs as root

2018-09-20 Thread Zbigniew 'zibi' Jarosik
On 19 September 2018 at 15:54, Qu Wenruo wrote: > > Does the iniramfs do a "btrfs dev scan" to make populate btrfs devices > lists? This solved problem. Looks like btrfs scans disks to early, before bcache initializes. I need to debug initrfamfs scripts. Thanks a lot for hint! -- Zbigniew