Re: [PATCH v2] Btrfs: fix memory leak of block group cache

On 07/21/2016 03:03 PM, Liu Bo wrote: On Thu, Jul 21, 2016 at 11:32:26AM -0400, Chris Mason wrote: On 07/20/2016 08:44 PM, Liu Bo wrote: While processing delayed refs, we may update block group's statistics and attach it to cur_trans->dirty_bgs, and later writing dirty block groups will

Re: [PATCH v2] Btrfs: fix memory leak of block group cache

On Thu, Jul 21, 2016 at 11:32:26AM -0400, Chris Mason wrote: > > > On 07/20/2016 08:44 PM, Liu Bo wrote: > > While processing delayed refs, we may update block group's statistics > > and attach it to cur_trans->dirty_bgs, and later writing dirty block > > groups will process the list, which

Re: A lot warnings in dmesg while running thunderbird

On 21.07.2016 14:56, Chris Mason wrote: > On 07/20/2016 01:50 PM, Gabriel C wrote: >> >> After 24h of running the program and thundirbird all is still fine here. >> >> I let it run one more day.. But looks very good. >> > > Thanks for your time in helping to track this down. It'll go into the

Re: [PATCH] generic/371: run write(2) and fallocate(2) in parallel

On Thu, Jul 21, 2016 at 06:41:00PM +0800, Eryu Guan wrote: > On Thu, Jul 21, 2016 at 03:30:25PM +0800, Wang Xiaoguang wrote: > > + > > +run_time=$((180 * $TIME_FACTOR)) > > 180s is too long time, I can reproduce it in around 10s on my test vm, > just loop for 100 times for each operation (pwrite

Re: [PATCH 1/2] Btrfs: be more precise on errors when getting an inode from disk

On Fri, Jul 22 2016, J. Bruce Fields wrote: > On Fri, Jul 22, 2016 at 11:08:17AM +1000, NeilBrown wrote: >> On Fri, Jun 10 2016, fdman...@kernel.org wrote: >> >> > From: Filipe Manana >> > >> > When we attempt to read an inode from disk, we end up always returning an >> >

[PATCH 09/17] common/dmerror: fix mount option issues

Calling _mount doesn't work when we want to add mount options such as realtime devices. Since it's just a normal scratch device mount except for the source device, just call _scratch_mount with SCRATCH_DEV set to the dmerror device. Signed-off-by: Darrick J. Wong ---

[PATCH 07/17] xfs/128: cycle_mount the scratch device, not the test device

This test uses the scratch device, so cycle that, not the test dev. This is also a xfs_fsr test, so put it in the fsr group. Signed-off-by: Darrick J. Wong --- tests/xfs/128 |7 --- tests/xfs/group |2 +- 2 files changed, 5 insertions(+), 4 deletions(-)

[PATCH 06/17] xfs: run xfs_repair at the end of each test

Run xfs_repair twice at the end of each test -- once to rebuild the btree indices, and again with -n to check the rebuild work. Signed-off-by: Darrick J. Wong --- README|4 common/rc | 30 ++ 2 files changed, 34 insertions(+)

[PATCH 10/17] xfs/179: use scratch device helpers

Use the helper functions for scratch devices. This fixes a problem where xfs/179 fails when there's a realtime device. Signed-off-by: Darrick J. Wong --- tests/xfs/179 | 10 +- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/tests/xfs/179

[PATCH 01/17] xfs/26[34]: remove duplicate tests

These two tests were accidentally double-added as xfs/30[78], but the newer versions have fixed up helper usage and fewer whitespace problems, so nuke the old tests. Signed-off-by: Darrick J. Wong --- tests/xfs/263 | 179

[PATCH 11/17] xfs/234: use scratch device helpers

Use the helper functions for scratch devices. This fixes a problem where xfs/234 fails when there's a realtime device. Signed-off-by: Darrick J. Wong --- tests/xfs/234 |6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/tests/xfs/234

[PATCH 02/17] xfs: use rmapbt-checking helper

Don't open-code _notrun checks for the rmapbt, just use the helper. Signed-off-by: Darrick J. Wong --- tests/xfs/310 |4 +--- tests/xfs/314 |4 +--- tests/xfs/317 |4 +--- tests/xfs/322 |4 +--- 4 files changed, 4 insertions(+), 12 deletions(-) diff

[PATCH v7 00/17] xfstests: fixes and new tests for rmap/reflink/etc

Hi all, This is the seventh revision of a patchset that adds to xfstests support for testing reverse-mappings of physical blocks to file and metadata (rmap); support for testing multiple file logical blocks to the same physical block (reflink); and implements the beginnings of online metadata

[PATCH 04/17] xfs/122: list the new log redo items

List the new log redo items. These should have stable sizes. Signed-off-by: Darrick J. Wong --- tests/xfs/122.out |8 1 file changed, 8 insertions(+) diff --git a/tests/xfs/122.out b/tests/xfs/122.out index ebc4421..c4ed725 100644 --- a/tests/xfs/122.out

[PATCH 03/17] xfs/310: fix the size calculation for the huge device

Fix the calculation of the dmhuge size. The previous calculation tried to calculate the size correctly, but got it wrong for 1k block sizes. Therefore, clean the whole mess up. Signed-off-by: Darrick J. Wong --- tests/xfs/310 |5 +++-- 1 file changed, 3

[PATCH 05/17] common/reflink: actually test dedupe on scratch device

In _require_scratch_dedupe, test the scratch device, not the testdev. Signed-off-by: Darrick J. Wong --- common/reflink |2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/common/reflink b/common/reflink index 4ec390d..1363971 100644 ---

[PATCH 17/17] xfs: test realtime rmapbt code

Test the realtime rmap btree code by exercising various IO patterns on realtime files. Signed-off-by: Darrick J. Wong --- tests/xfs/878 | 88 +++ tests/xfs/878.out |9 tests/xfs/879 | 67

[PATCH 08/17] xfs/129: fix post-metadump remounting idiocy

Use the standard _scratch_mount to mount the filesystem from the restored image, instead of trying to call mount directly. This is needed in case we had custom mount options (like rtdev). Signed-off-by: Darrick J. Wong --- tests/xfs/129 |6 +++--- 1 file changed, 3

[PATCH 12/17] reflink: test cross-mountpoint reflink and dedupe

Test sharing blocks via reflink and dedupe between two different mountpoints of the same filesystem. This shouldn't work, since we don't allow cross-mountpoint functions. Signed-off-by: Darrick J. Wong --- tests/generic/927 | 88

[PATCH 15/17] xfs: scrub fs (if still mounted) at the end of the test

Teach _check_xfs_filesystem to scrub mounted filesystems before unmounting and fscking them. This is mostly to test the online scrub tool... Signed-off-by: Darrick J. Wong --- README|3 +++ common/config |1 + common/rc |7 +++ 3 files

[PATCH 13/17] xfs: test swapext with reflink

Add a few tests to stress the new swapext code for reflink and rmap. Signed-off-by: Darrick J. Wong --- tests/xfs/873 | 107 + tests/xfs/873.out |6 +++ tests/xfs/874 | 99

[PATCH 14/17] xfs: more rmapbt tests

More tests for the reverse mapping functionality. Signed-off-by: Darrick J. Wong --- tests/xfs/876 | 76 +++ tests/xfs/876.out |4 ++ tests/xfs/877 | 85 +

[PATCH 16/17] xfs/122: add the realtime rmapbt inode and btree fields

Add the on-disk structures added by the realtime rmapbt. Signed-off-by: Darrick J. Wong --- tests/xfs/122.out |6 +- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/tests/xfs/122.out b/tests/xfs/122.out index c4ed725..b20011e 100644 ---

Re: A lot warnings in dmesg while running thunderbird

On Thu, Jul 21, 2016 at 11:25:02PM +0200, Gabriel C wrote: > On 21.07.2016 14:56, Chris Mason wrote: > > On 07/20/2016 01:50 PM, Gabriel C wrote: > >> After 24h of running the program and thundirbird all is still fine here. > >> > >> I let it run one more day.. But looks very good. > >> > > > >

Re: [PATCH 1/2] Btrfs: be more precise on errors when getting an inode from disk

On Fri, Jun 10 2016, fdman...@kernel.org wrote: > From: Filipe Manana > > When we attempt to read an inode from disk, we end up always returning an > -ESTALE error to the caller regardless of the actual failure reason, which > can be an out of memory problem (when allocating a

Re: [PATCH 1/2] Btrfs: be more precise on errors when getting an inode from disk

On Fri, Jul 22, 2016 at 11:08:17AM +1000, NeilBrown wrote: > On Fri, Jun 10 2016, fdman...@kernel.org wrote: > > > From: Filipe Manana > > > > When we attempt to read an inode from disk, we end up always returning an > > -ESTALE error to the caller regardless of the actual

Re: Status of SMR with BTRFS

On Thu, Jul 21, 2016 at 6:46 AM, Austin S. Hemmelgarn wrote: > On 2016-07-20 15:58, Chris Murphy wrote: >> >> On Sun, Jul 17, 2016 at 3:08 AM, Hendrik Friedel >> wrote: >> >>> Well, btrfs does write data very different to many other file systems. On

Re: Status of SMR with BTRFS

On 2016-07-21 09:34, Chris Murphy wrote: On Thu, Jul 21, 2016 at 6:46 AM, Austin S. Hemmelgarn wrote: On 2016-07-20 15:58, Chris Murphy wrote: On Sun, Jul 17, 2016 at 3:08 AM, Hendrik Friedel wrote: Well, btrfs does write data very different to

Re: Status of SMR with BTRFS

On Thu, Jul 21, 2016 at 8:12 AM, Austin S. Hemmelgarn wrote: > This really isn't fake COW, it's COW, just at a higher level than most > programmers would think of it. Alright I'll stop calling it that. Thanks. -- Chris Murphy -- To unsubscribe from this list: send the

Finding only non-snapshots via btrfs subvol list

I'm trying to find non-snapshots, i.e. 'top-level' subvolumes in a filesystem and this seems harder than it IMHO should be. The fs is just like: /mnt/stuff subvolA subvolA-date1 subvolA-date2 subvolB subvolB-date1 subvolB-date2 .. All I want are the subvol{A,B} *without* the snapshots,

Re: [PATCH] btrfs: test whether fallocate(2) can preallocate half of the whole fs space

hello, On 07/21/2016 03:00 AM, Christoph Hellwig wrote: On Wed, Jul 20, 2016 at 03:34:50PM +0800, Wang Xiaoguang wrote: Currently in btrfs, there is something wrong with data space reservation. For example, if we try to preallocate more than haf of whole fs space, ENOSPC will occur, but indeed

[PATCH] generic/371: run write(2) and fallocate(2) in parallel

Currently in btrfs, there is something wrong with fallocate(2)'s data space reservation, it'll temporarily occupy more data space thant it really needs, which in turn will impact other operations' data request. In this test case, it runs write(2) and fallocate(2) in parallel and the total needed

[PATCH v2] btrfs: Update quick and auto tag for btrfs group

Update the following quick/auto tag based on their execution time 007 050 100 101 Two systems are used to determine their execution time. One is backed by an SATA spinning rust, whose maximum R/W speed is about 100MB/s, modern desktop performance. (VM1) Another one is a VM inside a openstack

Re: [PATCH] btrfs: test whether fallocate(2) can preallocate half of the whole fs space

hello, On 07/20/2016 05:25 PM, Filipe Manana wrote: On Wed, Jul 20, 2016 at 8:34 AM, Wang Xiaoguang wrote: Currently in btrfs, there is something wrong with data space reservation. For example, if we try to preallocate more than haf of whole fs space, ENOSPC will

Re: mount btrfs takes 30 minutes, btrfs check runs out of memory

Thanks for the info, pretty helpful. After a simple analysis, the defrag did do a pretty good job. --- | Avg Extent size | Median Extent size | Data Extents |

Re: mount btrfs takes 30 minutes, btrfs check runs out of memory

At 07/21/2016 04:13 PM, John Ettedgui wrote: On Thu, Jul 21, 2016 at 1:10 AM Qu Wenruo > wrote: Thanks for the info, pretty helpful. After a simple analysis, the defrag did do a pretty good job.

Re: [PATCH] btrfs: Update quick and auto tag for btrfs group

On Thu, Jul 21, 2016 at 10:06:07AM +0800, Qu Wenruo wrote: > Update the following quick/auto tag based on their execution time > 007 > 011 > 050 > 100 > 101 > > Two systems are used to determine their execution time. > One is backed by an SATA spinning rust, whose maximum R/W speed is > about

Re: [PATCH] btrfs: Update quick and auto tag for btrfs group

At 07/21/2016 03:47 PM, Eryu Guan wrote: On Thu, Jul 21, 2016 at 10:06:07AM +0800, Qu Wenruo wrote: Update the following quick/auto tag based on their execution time 007 011 050 100 101 Two systems are used to determine their execution time. One is backed by an SATA spinning rust, whose

Re: Status of SMR with BTRFS

On Thu, Jul 21, 2016 at 4:34 PM, Chris Murphy wrote: > > Do programs have a way to communicate what portion of a data file is > modified, so that only changed blocks are COW'd? When I change a > single pixel in a 400MiB image and do a save (to overwrite the > original

[PATCH] Btrfs: fix delalloc accounting after copy_from_user faults

Commit 56244ef151c3cd11 was almost but not quite enough to fix the reservation math after btrfs_copy_from_user returned partial copies. Some users are still seeing warnings in btrfs_destroy_inode, and with a long enough test run I'm able to trigger them as well. This patch fixes the accounting

Re: [PATCH] generic/371: run write(2) and fallocate(2) in parallel

On Thu, Jul 21, 2016 at 03:30:25PM +0800, Wang Xiaoguang wrote: > Currently in btrfs, there is something wrong with fallocate(2)'s data > space reservation, it'll temporarily occupy more data space thant it > really needs, which in turn will impact other operations' data request. > > In this test

[PATCH] Btrfs: avoid deadlocks during reservations in btrfs_truncate_block

From: Josef Bacik The new enospc code makes it possible to deadlock if we don't use FLUSH_LIMIT during reservations inside a transaction. This enforces the correct flush type to avoid both deadlocks and assertions Signed-off-by: Chris Mason Signed-off-by: Josef

Re: Status of SMR with BTRFS

On 2016-07-20 15:58, Chris Murphy wrote: On Sun, Jul 17, 2016 at 3:08 AM, Hendrik Friedel wrote: Well, btrfs does write data very different to many other file systems. On every write the file is copied to another place, even if just one bit is changed. That's special

Re: Status of SMR with BTRFS

> I'd expect the write pattern of Btrfs to be similar to f2fs, with > respect to sequentiality of new writes. Ideally yes - though my tests with a Seagate SMR drive suggest otherwise. Optimizing the write behavior would probably lead to speed improvements for btrfs on spinning disks. --- Matthias

Re: A lot warnings in dmesg while running thunderbird

On 07/20/2016 01:50 PM, Gabriel C wrote: After 24h of running the program and thundirbird all is still fine here. I let it run one more day.. But looks very good. Thanks for your time in helping to track this down. It'll go into the next merge window and be cc'd to stable. -chris -- To

Re: [PATCH 1/4] btrfs: use correct offset for reloc_inode in prealloc_file_extent_cluster()

On 07/20/2016 09:49 PM, Wang Xiaoguang wrote: hello, On 07/20/2016 09:18 PM, Josef Bacik wrote: On 07/20/2016 01:56 AM, Wang Xiaoguang wrote: In prealloc_file_extent_cluster(), btrfs_check_data_free_space() uses wrong file offset for reloc_inode, it uses cluster->start and cluster->end, which

Re: Status of SMR with BTRFS

On 21 July 2016 at 15:34, Chris Murphy wrote: > > Do programs have a way to communicate what portion of a data file is > modified, so that only changed blocks are COW'd? When I change a > single pixel in a 400MiB image and do a save (to overwrite the > original file), it

Re: mount btrfs takes 30 minutes, btrfs check runs out of memory

On 21/07/16 09:19, Qu Wenruo wrote: > We don't usually get such large extent tree dump from a real world use > case. Let us know if you want some more :-) I have a heavily used single disk BTRFS filesystem with about 3.7TB in use and about 9 million extents. I am happy to provide an extent dump

Re: [PATCH v2] Btrfs: fix memory leak of block group cache

On 07/20/2016 08:44 PM, Liu Bo wrote: While processing delayed refs, we may update block group's statistics and attach it to cur_trans->dirty_bgs, and later writing dirty block groups will process the list, which happens during btrfs_commit_transaction(). For whatever reason, the transaction

Re: [PATCH v2] Btrfs: fix memory leak of block group cache

On Thu, Jul 21, 2016 at 03:17:41PM -0400, Chris Mason wrote: > > > On 07/21/2016 03:03 PM, Liu Bo wrote: > > On Thu, Jul 21, 2016 at 11:32:26AM -0400, Chris Mason wrote: > > > > > > > > > On 07/20/2016 08:44 PM, Liu Bo wrote: > > > > While processing delayed refs, we may update block group's

Re: [PATCH] fstests: btrfs: Test send on heavily deduped file

On Thu, Jul 21, 2016 at 10:05:25AM +0800, Qu Wenruo wrote: > > > At 07/21/2016 07:37 AM, Dave Chinner wrote: > >On Wed, Jul 20, 2016 at 03:40:29PM +0800, Qu Wenruo wrote: > >>At 07/20/2016 03:01 PM, Eryu Guan wrote: > >>>On Tue, Jul 19, 2016 at 01:42:03PM +0800, Qu Wenruo wrote: > > >