Re: [Lsf-pc] LSF/MM 2017: Call for Proposals

On Thu 08-12-16 13:26:19, Michal Hocko wrote: > On Wed 07-12-16 06:57:06, James Bottomley wrote: > [...] > > Just on this point, since there seems to be a lot of confusion: lsf-pc > > is the list for contacting the programme committee, so you cannot > > subscribe to it. > > > > There is no

Re: [PATCH 1/2] btrfs: qgroup: Introduce extent changeset for qgroup reserve functions

On Friday, December 02, 2016 10:03:06 AM Qu Wenruo wrote: > Introduce a new parameter, struct extent_changeset for > btrfs_qgroup_reserved_data() and its callers. > > Such extent_changeset was used in btrfs_qgroup_reserve_data() to record > which range it reserved in current reserve, so it can

Re: [PATCH v2 2/2] fstests: btrfs: Use _require_btrfs_qgroup_report to replace open code

On Thu, Dec 08, 2016 at 04:12:13PM +0800, Qu Wenruo wrote: > Introduce new _require_btrfs_qgroup_report function, which will check > the accessibility to "btrfs check --qgroup-report", then set a global > flag to info _check_scratch_fs() to do extra qgroup check. > > Signed-off-by: Qu Wenruo

Re: [PATCH] btrfs-progs: Fix extents after finding all errors

On Thu, Dec 01, 2016 at 11:28:08AM -0600, Goldwyn Rodrigues wrote: > From: Goldwyn Rodrigues > > Simplifying the logic of fixing. > > Calling fixup_extent_ref() after encountering every error causes > more error messages after the extent is fixed. In case of multiple errors,

[PULL] Btrfs cleanups for 4.10

Hi, now that the regular patches are on the way to 4.10, it's a good time to squeeze the cleanups before we start merging further patches. This pull request contains the series from Jeff, the code changes are all over the map. I think we won't find a better time to merge that so be it now. The

Re: [PATCH 2/6] Btrfs: set single device limit for dax usecase

On Wed, Dec 07, 2016 at 01:45:06PM -0800, Liu Bo wrote: > Dax on btrfs is not ready for multiple device. How about DUP? Technically it's not multi-device but still stores multiple copies, so I don't know if the implementation is ok with that. -- To unsubscribe from this list: send the line

[PATCH 2/2] btrfs-convert: Fix migrate_super_block() to work with 64k sectorsize

migrate_super_block() uses sectorsize to refer to the size of the superblock. Hence on 64k sectorsize filesystems, it ends up computing checksum beyond the super block length (i.e. BTRFS_SUPER_INFO_SIZE). This commit fixes the bug by using BTRFS_SUPER_INFO_SIZE instead of sectorsize of the

[PATCH 1/2] btrfs-progs: btrfs-convert: Prevent accounting blocks beyond end of device

When looping across data block bitmap, __ext2_add_one_block() may add blocks which do not exist on the underlying disk. This commit prevents this from happening by checking the block index against the maximum block count that was present in the ext4 filesystem instance that is being converted.

Re: [PATCH] btrfs-progs: man mkfs: warn about RAID5/6 being experimental

On Mon, Nov 28, 2016 at 07:52:43PM +0100, Adam Borowski wrote: > Signed-off-by: Adam Borowski Applied, thanks. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at

Re: LSF/MM 2017: Call for Proposals

On Wed 07-12-16 06:57:06, James Bottomley wrote: [...] > Just on this point, since there seems to be a lot of confusion: lsf-pc > is the list for contacting the programme committee, so you cannot > subscribe to it. > > There is no -discuss equivalent, like kernel summit has, because we > expect

Re: [PATCH 4/6] Btrfs: add DAX support for nocow btrfs

On Wed 07-12-16 17:15:42, Chris Mason wrote: > On 12/07/2016 04:45 PM, Liu Bo wrote: > >This has implemented DAX support for btrfs with nocow and single-device. > > > >DAX is developed for block devices that are memory-like in order to avoid > >double buffer in both page cache and the storage, so

[PATCH v2 2/2] fstests: btrfs: Use _require_btrfs_qgroup_report to replace open code

Introduce new _require_btrfs_qgroup_report function, which will check the accessibility to "btrfs check --qgroup-report", then set a global flag to info _check_scratch_fs() to do extra qgroup check. Signed-off-by: Qu Wenruo --- v2: Use

[PATCH v2 1/2] fstests: common: rename and enhance _require_btrfs to _require_btrfs_command

Rename _require_btrfs() to _require_btrfs_command() to avoid confusion, as all other _require_btrfs_* has a quite clear suffix, like _require_btrfs_mkfs_feature() or _require_btrfs_fs_feature(). Also enhance _require_btrfs_command() to accept 2nd level commands or options. Options will be

Re: [PATCH v2 2/2] fstests: btrfs: Use _require_btrfs_qgroup_report to replace open code

On Thu, Dec 08, 2016 at 05:20:49PM +0800, Qu Wenruo wrote: > > > At 12/08/2016 04:47 PM, Eryu Guan wrote: > > On Thu, Dec 08, 2016 at 04:12:13PM +0800, Qu Wenruo wrote: > > > Introduce new _require_btrfs_qgroup_report function, which will check > > > the accessibility to "btrfs check

Re: [PATCH v2 2/2] fstests: btrfs: Use _require_btrfs_qgroup_report to replace open code

At 12/08/2016 04:47 PM, Eryu Guan wrote: On Thu, Dec 08, 2016 at 04:12:13PM +0800, Qu Wenruo wrote: Introduce new _require_btrfs_qgroup_report function, which will check the accessibility to "btrfs check --qgroup-report", then set a global flag to info _check_scratch_fs() to do extra qgroup

Re: [PATCH 4/6] Btrfs: add DAX support for nocow btrfs

On Thu, Dec 08, 2016 at 11:47:41AM +0100, Jan Kara wrote: > On Wed 07-12-16 17:15:42, Chris Mason wrote: > > On 12/07/2016 04:45 PM, Liu Bo wrote: > > >This has implemented DAX support for btrfs with nocow and single-device. > > > > > >DAX is developed for block devices that are memory-like in

Re: LSF/MM 2017: Call for Proposals

On Thu 08-12-16 07:30:43, James Bottomley wrote: > On Thu, 2016-12-08 at 13:26 +0100, Michal Hocko wrote: > > On Wed 07-12-16 06:57:06, James Bottomley wrote: > > [...] > > > Just on this point, since there seems to be a lot of confusion: lsf > > > -pc > > > is the list for contacting the

Re: duperemove : some real world figures on BTRFS deduplication

On 2016-12-08 10:11, Swâmi Petaramesh wrote: Hi, Some real world figures about running duperemove deduplication on BTRFS : I have an external 2,5", 5400 RPM, 1 TB HD, USB3, on which I store the BTRFS backups (full rsync) of 5 PCs, using 2 different distros, typically at the same update level,

Re: [PATCH] btrfs-progs: Initialize ret to suppress compiler warning

On Thu, Dec 01, 2016 at 11:28:20AM -0600, Goldwyn Rodrigues wrote: > From: Goldwyn Rodrigues The path that leaves ret unintialized goes through the second if block and requires dback->found_ref to be 0. Quick search leads to several places where it's set according to found

Re: [PATCH] btrfs-progs: mkfs, balance convert: warn about RAID5/6 in fiery letters

On Wed, Nov 30, 2016 at 04:12:46PM +0100, David Sterba wrote: > On Mon, Nov 28, 2016 at 07:51:53PM +0100, Adam Borowski wrote: > > People who don't frequent IRC nor the mailing list tend to believe RAID 5/6 > > are stable; this leads to data loss. Thus, let's do warn them. > > > > At this point,

duperemove : some real world figures on BTRFS deduplication

Hi, Some real world figures about running duperemove deduplication on BTRFS : I have an external 2,5", 5400 RPM, 1 TB HD, USB3, on which I store the BTRFS backups (full rsync) of 5 PCs, using 2 different distros, typically at the same update level, and all of them more of less sharing the

Re: [RESEND][PATCH v2] btrfs-progs: add dev stats returncode option

On Mon, Dec 05, 2016 at 01:35:20PM -0500, Austin S. Hemmelgarn wrote: > Currently, `btrfs device stats` returns non-zero only when there was an > error getting the counter values. This is fine for when it gets run by a > user directly, but is a serious pain when trying to use it in a script or >

Re: [PATCH 2/6] Btrfs: set single device limit for dax usecase

On Thu, Dec 08, 2016 at 02:35:59PM +0100, David Sterba wrote: > On Wed, Dec 07, 2016 at 01:45:06PM -0800, Liu Bo wrote: > > Dax on btrfs is not ready for multiple device. > > How about DUP? Technically it's not multi-device but still stores > multiple copies, so I don't know if the implementation

Re: LSF/MM 2017: Call for Proposals

On Thu, 2016-12-08 at 13:26 +0100, Michal Hocko wrote: > On Wed 07-12-16 06:57:06, James Bottomley wrote: > [...] > > Just on this point, since there seems to be a lot of confusion: lsf > > -pc > > is the list for contacting the programme committee, so you cannot > > subscribe to it. > > > >

Re: [PATCH 0/6] btrfs dax IO

On Thu, Dec 08, 2016 at 07:48:18AM +0100, Xin Zhou wrote: > Hi Liu, >   > From the patch, is the snapshot disabled by disabling the COW in the mounting > path? > It seems the create_snapshot() in ioctl.c does not get changed. Well, I think I made a mistake in this cover letter, snapshot still

out-of-band dedup status?

Hey. I just wondered whether out-of-band/"offline" dedup is safe for general use... https://btrfs.wiki.kernel.org/index.php/Status kinda implies so (it tells about unspecified performance issues), but this seems again already outdated (kernel 4.7)... :-( My intention was to use it with

Re: duperemove : some real world figures on BTRFS deduplication

2016-12-08 18:42 GMT+03:00 Austin S. Hemmelgarn : > On 2016-12-08 10:11, Swâmi Petaramesh wrote: >> >> Hi, Some real world figures about running duperemove deduplication on >> BTRFS : >> >> I have an external 2,5", 5400 RPM, 1 TB HD, USB3, on which I store the >> BTRFS

Re: [RESEND][PATCH v2] btrfs-progs: add dev stats returncode option

On 2016-12-08 12:20, David Sterba wrote: On Mon, Dec 05, 2016 at 01:35:20PM -0500, Austin S. Hemmelgarn wrote: Currently, `btrfs device stats` returns non-zero only when there was an error getting the counter values. This is fine for when it gets run by a user directly, but is a serious pain

Re: duperemove : some real world figures on BTRFS deduplication

On 12/8/16 10:42 AM, Austin S. Hemmelgarn wrote: > On 2016-12-08 10:11, Swâmi Petaramesh wrote: >> Hi, Some real world figures about running duperemove deduplication on >> BTRFS : >> >> I have an external 2,5", 5400 RPM, 1 TB HD, USB3, on which I store the >> BTRFS backups (full rsync) of 5 PCs,

Re: out-of-band dedup status?

On 12/8/16 1:36 PM, Christoph Anton Mitterer wrote: > Hey. > > I just wondered whether out-of-band/"offline" dedup is safe for general > use... https://btrfs.wiki.kernel.org/index.php/Status kinda implies so > (it tells about unspecified performance issues), but this seems again > already

Re: duperemove : some real world figures on BTRFS deduplication

On 2016-12-08 15:07, Jeff Mahoney wrote: On 12/8/16 10:42 AM, Austin S. Hemmelgarn wrote: On 2016-12-08 10:11, Swâmi Petaramesh wrote: Hi, Some real world figures about running duperemove deduplication on BTRFS : I have an external 2,5", 5400 RPM, 1 TB HD, USB3, on which I store the BTRFS

Re: duperemove : some real world figures on BTRFS deduplication

On 12/8/16 10:11 AM, Swâmi Petaramesh wrote: > Hi, Some real world figures about running duperemove deduplication on > BTRFS : > > I have an external 2,5", 5400 RPM, 1 TB HD, USB3, on which I store the > BTRFS backups (full rsync) of 5 PCs, using 2 different distros, > typically at the same

Re: out-of-band dedup status?

Pretty sure it will not dedupe extents that are referenced in a read only subvolume. Chris Murphy -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: out-of-band dedup status?

On Thursday 08 December 2016 13:41:36 Chris Murphy wrote: > Pretty sure it will not dedupe extents that are referenced in a read > only subvolume. I've used duperemove to de-duplicate files in read-only snapshots (of different systems) on my backup drive, so unless you're referencing some

Re: out-of-band dedup status?

On Thu, 2016-12-08 at 13:41 -0700, Chris Murphy wrote: > Pretty sure it will not dedupe extents that are referenced in a read > only subvolume. Oh... hm.. well that would be quite some limitation, cause as soon as one has a snapshot of the full fs (which is probably not so unlikely) i won't work

Re: duperemove : some real world figures on BTRFS deduplication

> 2016-12-08 16:11 GMT+01:00 Swâmi Petaramesh : > > Then it took another 48 hours just for "loading the hashes of duplicate > extents". > This issue i adressing currently with the following patches: https://github.com/Floyddotnet/duperemove/commits/digest_trigger Tested

Re: [PATCH 2/2] btrfs-convert: Fix migrate_super_block() to work with 64k sectorsize

At 12/08/2016 09:56 PM, Chandan Rajendra wrote: migrate_super_block() uses sectorsize to refer to the size of the superblock. Hence on 64k sectorsize filesystems, it ends up computing checksum beyond the super block length (i.e. BTRFS_SUPER_INFO_SIZE). This commit fixes the bug by using

[PATCH v3 2/2] fstests: btrfs: Use _require_btrfs_qgroup_report to replace open code

Introduce new _require_btrfs_qgroup_report function, which will check the accessibility to "btrfs check --qgroup-report", then set a global flag to info _check_scratch_fs() to do extra qgroup check. Signed-off-by: Qu Wenruo --- v2: Use

Re: [PATCH 1/2] btrfs: qgroup: Introduce extent changeset for qgroup reserve functions

At 12/08/2016 10:29 PM, Chandan Rajendra wrote: On Friday, December 02, 2016 10:03:06 AM Qu Wenruo wrote: Introduce a new parameter, struct extent_changeset for btrfs_qgroup_reserved_data() and its callers. Such extent_changeset was used in btrfs_qgroup_reserve_data() to record which range

Re: out-of-band dedup status?

OK something's wrong. Kernel 4.8.12 and duperemove v0.11.beta4. Brand new file system (mkfs.btrfs -dsingle -msingle, default mount options) and two identical files separately copied. [chris@f25s]$ ls -li /mnt/test total 2811904 260 -rw-r--r--. 1 root root 1439694848 Dec 8 17:26

Re: [PATCH 1/2] btrfs-progs: btrfs-convert: Prevent accounting blocks beyond end of device

Hi Chandan, Thanks for the patch. At 12/08/2016 09:56 PM, Chandan Rajendra wrote: When looping across data block bitmap, __ext2_add_one_block() may add blocks which do not exist on the underlying disk. This commit prevents this from happening by checking the block index against the maximum

Re: [PATCH 1/2] btrfs-progs: btrfs-convert: Prevent accounting blocks beyond end of device

Hi Chandan, Thanks for the patch. At 12/08/2016 09:56 PM, Chandan Rajendra wrote: When looping across data block bitmap, __ext2_add_one_block() may add blocks which do not exist on the underlying disk. This commit prevents this from happening by checking the block index against the maximum

WARNING at extent-tree.c:1621 lookup_inline_extent_backref and extent-tree.c:2960 btrfs_run_delayed_refs (Linux 4.8.11)

Basically some sectors on multi-disk btrfs filesystem got corrupted due HDD fault. Now after scrub have finished (with some uncorrectable errors) when doing metadata balance $ btrfs balance start -m /mnt/Data [ 612.683979] WARNING: CPU: 0 PID: 6 at /mnt/linux/fs/btrfs/extent-tree.c:1621

Re: out-of-band dedup status?

On Thu, Dec 08, 2016 at 05:45:40PM -0700, Chris Murphy wrote: > OK something's wrong. > > Kernel 4.8.12 and duperemove v0.11.beta4. Brand new file system > (mkfs.btrfs -dsingle -msingle, default mount options) and two > identical files separately copied. > > [chris@f25s]$ ls -li /mnt/test >

Re: out-of-band dedup status?

On Thu, Dec 8, 2016 at 7:26 PM, Darrick J. Wong wrote: > On Thu, Dec 08, 2016 at 05:45:40PM -0700, Chris Murphy wrote: >> OK something's wrong. >> >> Kernel 4.8.12 and duperemove v0.11.beta4. Brand new file system >> (mkfs.btrfs -dsingle -msingle, default mount options)

[PATCH 00/10] Qgroup fixes for kdave/for-next-20161125 branch

The branch can be fetched from github: https://github.com/adam900710/linux.git for-david-next-qgroup-fixes If David wants to push these fixes to 4.10, then I can rebase these patches to Chris' for-linus branch. Recent qgroup fixes for several problems: 1) Qgroup reserved space underflow

Re: duperemove : some real world figures on BTRFS deduplication

On Thu, Dec 8, 2016 at 8:11 AM, Swâmi Petaramesh wrote: > Well, the damn thing has been running for 15 days uninterrupted ! > ...Until I [Ctrl]-C it this morning as I had to move with the machine (I > wasn't expecting it to last THAT long...). Can you check some bigger

Re: [PATCH 1/2] btrfs-progs: btrfs-convert: Prevent accounting blocks beyond end of device

On Friday, December 09, 2016 09:03:57 AM Qu Wenruo wrote: > Hi Chandan, > > Thanks for the patch. > > At 12/08/2016 09:56 PM, Chandan Rajendra wrote: > > When looping across data block bitmap, __ext2_add_one_block() may add > > blocks which do not exist on the underlying disk. This commit

Re: [PATCH 1/6] Btrfs: add mount option for dax

On Wed, Dec 07, 2016 at 01:45:05PM -0800, Liu Bo wrote: > Signed-off-by: Liu Bo > --- > fs/btrfs/ctree.h | 1 + > fs/btrfs/super.c | 40 +++- > 2 files changed, 40 insertions(+), 1 deletion(-) > > diff --git a/fs/btrfs/ctree.h

Re: [PATCH 2/2] btrfs-convert: Fix migrate_super_block() to work with 64k sectorsize

On Friday, December 09, 2016 09:09:29 AM Qu Wenruo wrote: > > At 12/08/2016 09:56 PM, Chandan Rajendra wrote: > > migrate_super_block() uses sectorsize to refer to the size of the > > superblock. Hence on 64k sectorsize filesystems, it ends up computing > > checksum beyond the super block length

Re: [PATCH 4/6] Btrfs: add DAX support for nocow btrfs

On Wed, Dec 07, 2016 at 01:45:08PM -0800, Liu Bo wrote: > Since I haven't figure out how to map multiple devices to userspace without > pagecache, this DAX support is only for single-device, and I don't think > DAX(Direct Access) can work with cow, this is limited to nocow case. I made > this by