Re: suspected BTRFS errors resulting in file system becoming unrecovable

On 2016-02-08 11:23, WillIam Thorne wrote: Thanks all for the help. Here’s a bit more info below. Seeing as its possibly related to the USB implementation on the pi, I have cc’d their mailing list. Glad we could be of assistance. On 25 Jan 2016, at 16:43, Austin S. Hemmelgarn

4.4.0 - no space left with >1.7 TB free space left

Linux 4.4.0 - btrfs is mainly used to host lots of test containers, often snapshots, and at times, there is heavy IO in many of them for extended periods of time. btrfs is on HDDs. Every few days I'm getting "no space left" in a container running mongo 3.2.1 database. Interestingly, haven't

Re: 4.4.0 - no space left with >1.7 TB free space left

On 2016-02-08 20:24, Roman Mamedov wrote: On Mon, 08 Feb 2016 18:22:34 +0900 Tomasz Chmielewski wrote: Linux 4.4.0 - btrfs is mainly used to host lots of test containers, often snapshots, and at times, there is heavy IO in many of them for extended periods of time. btrfs is

Re: 4.4.0 - no space left with >1.7 TB free space left

On Mon, 08 Feb 2016 21:15:38 +0900 Tomasz Chmielewski wrote: > With the last error, a snapshot was made at around 06:06 > "no space left" was reported on 06:14. If you mean the log that you have posted in your original message, the ENOSPC happened at 06:06 and 14 seconds, not

Re: Use fast device only for metadata?

On 2016-02-07 15:59, Martin Steigerwald wrote: Am Sonntag, 7. Februar 2016, 21:07:13 CET schrieb Kai Krakow: Am Sun, 07 Feb 2016 11:06:58 -0800 schrieb Nikolaus Rath : Hello, I have a large home directory on a spinning disk that I regularly synchronize between different

Re: Fwd: Unmountable fs after power outage

2016-01-22 16:25 GMT+01:00 Hugo Mills : >Try mounting with -orecovery. That's the main approach for dealing > with transid failures. Tried that, but I still get the same error. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a

Re: 4.4.0 - no space left with >1.7 TB free space left

On Mon, 08 Feb 2016 18:22:34 +0900 Tomasz Chmielewski wrote: > Linux 4.4.0 - btrfs is mainly used to host lots of test containers, > often snapshots, and at times, there is heavy IO in many of them for > extended periods of time. btrfs is on HDDs. > > > Every few days I'm

Re: "layout" of a six drive raid10

Am Tue, 9 Feb 2016 01:42:40 + (UTC) schrieb Duncan <1i5t5.dun...@cox.net>: > Tho I'd consider benchmarking or testing, as I'm not sure btrfs raid1 > on spinning rust will in practice fully saturate the gigabit > Ethernet, particularly as it gets fragmented (which COW filesystems > such as

Re: [PATCH v4.1 00/23] xfstests: test the nfs/cifs/btrfs/xfs reflink/dedupe ioctls

On Mon, Feb 08, 2016 at 05:11:45PM -0800, Darrick J. Wong wrote: > Happy New Year! > > Dave Chinner: I've renumbered the new tests and pushed to github[3] if > you'd like to pull. Can you include the commit ID I should see at the head of the tree so I can confirm I'm pulling the right branch?

Re: [PATCH v4.1 00/23] xfstests: test the nfs/cifs/btrfs/xfs reflink/dedupe ioctls

Aha, /me finds git request-pull. Sorry for the noise. --D The following changes since commit d98149c205559950c03d6b1d539e45fd35b5630e: Fix prerequisite packages to build fstests on Ubuntu (2016-02-08 09:27:15 +1100) are available in the git repository at:

Re: [PATCH 12/23] xfs/122: support refcount/rmap data structures

On Mon, Feb 08, 2016 at 05:13:03PM -0800, Darrick J. Wong wrote: > Include the refcount and rmap structures in the golden output. > > Signed-off-by: Darrick J. Wong > --- > tests/xfs/122 |3 +++ > tests/xfs/122.out |4 > tests/xfs/group |2 +- > 3

Re: [PATCH 10/23] xfs: more reflink tests

On Mon, Feb 08, 2016 at 05:12:50PM -0800, Darrick J. Wong wrote: > Create a couple of XFS-specific tests -- one to check that growing > and shrinking the refcount btree works and a second one to check > what happens when we hit maximum refcount. > > Signed-off-by: Darrick J. Wong

Re: [PATCH 06/23] dio unwritten conversion bug tests

On Mon, Feb 08, 2016 at 05:12:23PM -0800, Darrick J. Wong wrote: > Check that we don't expose old disk contents when a directio write to > an unwritten extent fails due to IO errors. This primarily affects > XFS and ext4. > > Signed-off-by: Darrick J. Wong . > ---

Re: "layout" of a six drive raid10

boli posted on Mon, 08 Feb 2016 23:19:52 +0100 as excerpted: > Hi > > I'm trying to figure out what a six drive btrfs raid10 would look like. > It could mean that stripes are split over two raid1 sets of three > devices each. The sentence "Every stripe is split across to exactly 2 > RAID-1

Re: Use fast device only for metadata?

Am Mon, 08 Feb 2016 13:44:17 -0800 schrieb Nikolaus Rath : > On Feb 07 2016, Martin Steigerwald wrote: > > Am Sonntag, 7. Februar 2016, 21:07:13 CET schrieb Kai Krakow: > >> Am Sun, 07 Feb 2016 11:06:58 -0800 > >> > >> schrieb Nikolaus Rath

Re: [PATCH v4.1 00/23] xfstests: test the nfs/cifs/btrfs/xfs reflink/dedupe ioctls

On Tue, Feb 09, 2016 at 06:21:06PM +1100, Dave Chinner wrote: > On Mon, Feb 08, 2016 at 05:11:45PM -0800, Darrick J. Wong wrote: > > Happy New Year! > > > > Dave Chinner: I've renumbered the new tests and pushed to github[3] if > > you'd like to pull. > > Can you include the commit ID I should

Re: "layout" of a six drive raid10

Am Tue, 9 Feb 2016 08:02:58 +0100 schrieb Kai Krakow : > Am Tue, 9 Feb 2016 01:42:40 + (UTC) > schrieb Duncan <1i5t5.dun...@cox.net>: > > > Tho I'd consider benchmarking or testing, as I'm not sure btrfs > > raid1 on spinning rust will in practice fully saturate the

Re: [PATCH 12/23] xfs/122: support refcount/rmap data structures

On Tue, Feb 09, 2016 at 06:43:30PM +1100, Dave Chinner wrote: > On Mon, Feb 08, 2016 at 05:13:03PM -0800, Darrick J. Wong wrote: > > Include the refcount and rmap structures in the golden output. > > > > Signed-off-by: Darrick J. Wong > > --- > > tests/xfs/122 |

Re: Use fast device only for metadata?

On 02/08/2016 09:29 PM, Austin S. Hemmelgarn wrote: On 2016-02-08 08:20, Qu Wenruo wrote: On 02/08/2016 08:24 PM, Austin S. Hemmelgarn wrote: On 2016-02-07 15:59, Martin Steigerwald wrote: Am Sonntag, 7. Februar 2016, 21:07:13 CET schrieb Kai Krakow: Am Sun, 07 Feb 2016 11:06:58 -0800

Re: Use fast device only for metadata?

On 2016-02-08 08:20, Qu Wenruo wrote: On 02/08/2016 08:24 PM, Austin S. Hemmelgarn wrote: On 2016-02-07 15:59, Martin Steigerwald wrote: Am Sonntag, 7. Februar 2016, 21:07:13 CET schrieb Kai Krakow: Am Sun, 07 Feb 2016 11:06:58 -0800 schrieb Nikolaus Rath : Hello, I have

Re: [PATCH v2 10/10] fs: btrfs: Replace CURRENT_TIME by current_fs_time()

On Sat, Feb 06, 2016 at 11:57:21PM -0800, Deepa Dinamani wrote: > CURRENT_TIME macro is not appropriate for filesystems as it > doesn't use the right granularity for filesystem timestamps. > Use current_fs_time() instead. > > Signed-off-by: Deepa Dinamani > Cc: Chris

Re: Use fast device only for metadata?

On 02/08/2016 08:24 PM, Austin S. Hemmelgarn wrote: On 2016-02-07 15:59, Martin Steigerwald wrote: Am Sonntag, 7. Februar 2016, 21:07:13 CET schrieb Kai Krakow: Am Sun, 07 Feb 2016 11:06:58 -0800 schrieb Nikolaus Rath : Hello, I have a large home directory on a spinning

Re: Fwd: Unmountable fs after power outage

On 01/22/2016 11:25 PM, Hugo Mills wrote: On Fri, Jan 22, 2016 at 04:11:53PM +0100, Radek Sprta wrote: Hello everybody, after a recent power outage attempting to mount a my btrfs partition fails with the following error: mount: wrong fs type, bad option, bad superblock on /dev/sda8,

Re: Use fast device only for metadata?

Nikolaus Rath posted on Mon, 08 Feb 2016 13:44:17 -0800 as excerpted: > Otherwise I'll give bcache a shot. I've avoided it so far because of the > need to reformat and because of rumours that it doesn't work well with > LVM or BTRFS. But it sounds as if that's not the case.. Bcache used to have

"layout" of a six drive raid10

Hi I'm trying to figure out what a six drive btrfs raid10 would look like. The example at seems ambiguous to me. It could mean that stripes are split over two raid1 sets of

Re: Use fast device only for metadata?

On Feb 07 2016, Martin Steigerwald wrote: > Am Sonntag, 7. Februar 2016, 21:07:13 CET schrieb Kai Krakow: >> Am Sun, 07 Feb 2016 11:06:58 -0800 >> >> schrieb Nikolaus Rath : >> > Hello, >> > >> > I have a large home directory on a spinning disk that I

Re: "layout" of a six drive raid10

On Mon, Feb 08, 2016 at 11:19:52PM +0100, boli wrote: > Hi > > I'm trying to figure out what a six drive btrfs raid10 would look like. The > example at > > seems ambiguous

[PATCH 02/23] xfstests: filter whitespace in 128 and 132

Seems either I have a different lsattr version, or different mount points cause differences in the golden output. Send the lsattr output through the whitespaces filter so that it works everywhere. The lsattr output /does/ change depending on mountpoints. Ick. I'd actually changed it to the

[PATCH 06/23] dio unwritten conversion bug tests

Check that we don't expose old disk contents when a directio write to an unwritten extent fails due to IO errors. This primarily affects XFS and ext4. Signed-off-by: Darrick J. Wong --- .gitignore |1 src/aio-dio-regress/aiocp.c | 489

[PATCH v4.1 00/23] xfstests: test the nfs/cifs/btrfs/xfs reflink/dedupe ioctls

Happy New Year! Dave Chinner: I've renumbered the new tests and pushed to github[3] if you'd like to pull. This is a (no longer) small patch set against the reflink/dedupe test cases in xfstests. The first four patches fix errors in the existing reflink tests, some of which are from Christoph

[PATCH 01/23] generic/182: this is a dedupe test, check for dedupe

Since this test examines dedupe behavior, the documentation should say 'dedupe', not 'reflink'. Furthermore, the feature checks must look for working dedupe functionality, not reflink functionality. Signed-off-by: Darrick J. Wong [h...@lst.de: add the test for dedupe

[PATCH 03/23] xfstests: make _scratch_mkfs_blocksized usable

The default mkfs.xfs options contain -b size=4096, so all tests using _scratch_mkfs_blocksized won't actually run unless those options are changed. As we're trying to specificly test 1k blocks we should always override the default option. v2: Move the function to common/rc Signed-off-by:

[PATCH 07/23] reflink: test intersecting CoW and falloc/fpunch/fzero/fcollapse/finsert/ftrunc

Ensure that we correctly handle a CoW operation immediately followed by a truncate, falloc, fpunch, fzero, fcollapse, and finsert operation in the middle of the CoW'd region before any flush can occur. Signed-off-by: Darrick J. Wong --- tests/generic/253 | 93

[PATCH 04/23] reflink: remove redundant filesystem checks from the end of the tests

Turns out that check already runs _check_filesystems after each test, so we don't need to do this at the end of each test. Signed-off-by: Darrick J. Wong --- tests/generic/157 |1 - tests/generic/158 |1 - tests/generic/161 |1 - tests/generic/162 |1 -

[PATCH 05/23] common/dmerror: add some more dmerror routines

Add functions to the dmerror routine so that we can load both the error table and the linear table. This will help us with EIO testing of copy-on-write. Signed-off-by: Darrick J. Wong --- common/dmerror | 27 +-- tests/btrfs/100 |2 +-

[PATCH 10/23] xfs: more reflink tests

Create a couple of XFS-specific tests -- one to check that growing and shrinking the refcount btree works and a second one to check what happens when we hit maximum refcount. Signed-off-by: Darrick J. Wong --- tests/xfs/169 | 90

[PATCH 15/23] reflink: test xfs cow behavior when the filesystem crashes

Use the extent size hint to force leftover CoW reservations then crash the filesystem to see how recovery works. Signed-off-by: Darrick J. Wong --- tests/xfs/212 | 106 + tests/xfs/212.out | 14 +++

[PATCH 13/23] xfs: test fragmentation characteristics of copy-on-write

Perform copy-on-writes at random offsets to stress the CoW allocation system. Assess the effectiveness of the extent size hint at combatting fragmentation via unshare, a rewrite, and no-op after the random writes. Signed-off-by: Darrick J. Wong --- tests/generic/301

[PATCH 16/23] reflink: test quota accounting

Signed-off-by: Darrick J. Wong --- common/reflink|2 - tests/generic/305 | 105 +++ tests/generic/305.out | 23 ++ tests/generic/326 | 105 +++

[PATCH 09/23] reflink: test CoW operations against the source file

Ensure that CoW operations against shared blocks in the source file work correctly. v2: remove filefrag dependencies Signed-off-by: Darrick J. Wong --- tests/generic/196 |2 - tests/generic/197 |2 - tests/generic/284 | 95

[PATCH 08/23] reflink: test CoW behavior with IO errors

Test various scenarios (with dm-flakey) where we simulate write failures during CoW, to see if the FS can get through it without blowing up or corrupting data. Plumb in a FS-generic method to sort out repairing filesystems after they get hit by IO errors. Signed-off-by: Darrick J. Wong

[PATCH 18/23] xfs: test the automatic cowextsize extent garbage collector

Signed-off-by: Darrick J. Wong --- tests/xfs/231 | 135 tests/xfs/231.out | 17 +++ tests/xfs/232 | 137 + tests/xfs/232.out | 17 +++

[PATCH 20/23] reflink: test aio copy on write

Make sure that copy on write works with the AIO path. Signed-off-by: Darrick J. Wong --- tests/generic/329 | 105 tests/generic/329.out | 12 + tests/generic/330 | 96

[PATCH 23/23] reflink: test reflink+cow+enospc all at the same time

Set up an impossibly small filesystem and try to reflink and rewrite a file on it to see what happens when we ENOSPC. Basically generic/16[67] but with a constrained fs size. Signed-off-by: Darrick J. Wong --- tests/generic/166 |6 ++- tests/generic/167 |

[PATCH 17/23] reflink: test CoW across a mixed range of block types with cowextsize set

Signed-off-by: Darrick J. Wong --- tests/xfs/215 | 108 ++ tests/xfs/215.out | 14 + tests/xfs/218 | 108 ++ tests/xfs/218.out | 14 + tests/xfs/219 | 108

[PATCH 14/23] reflink: high offset reflink and dedupe tests

Ensure that we can pass absurdly enormous offsets and lengths to reflink/dedupe and it'll survive. v2: Ask for dedupe in the dedupe test. Signed-off-by: Darrick J. Wong [h...@lst.de: call _require_test_dedupe] Signed-off-by: Christoph Hellwig ---

[PATCH 12/23] xfs/122: support refcount/rmap data structures

Include the refcount and rmap structures in the golden output. Signed-off-by: Darrick J. Wong --- tests/xfs/122 |3 +++ tests/xfs/122.out |4 tests/xfs/group |2 +- 3 files changed, 8 insertions(+), 1 deletion(-) diff --git a/tests/xfs/122

[PATCH 11/23] reflink: ensure that we can handle reflinking a lot of extents

Update the existing stress tests to ensure that we can handle reflinking the same block a million times, and that we can handle reflinking million different extents. Add a couple of tests to ensure that we can ^C and SIGKILL our way out of long-running reflinks. v2: Don't run the signal tests on

[PATCH 19/23] xfs: test rmapbt functionality

Signed-off-by: Darrick J. Wong --- common/xfs| 44 ++ tests/xfs/233 | 78 ++ tests/xfs/233.out |6 +++ tests/xfs/234 | 89 tests/xfs/234.out |

[PATCH 21/23] xfs: aio cow tests

Signed-off-by: Darrick J. Wong --- tests/xfs/237 | 107 tests/xfs/237.out | 12 ++ tests/xfs/239 | 98 tests/xfs/239.out | 11 + tests/xfs/240

[PATCH 22/23] xfs: test xfs_getbmapx behavior with shared extents

Make sure that xfs_getbmapx behaves properly w.r.t. shared extents and CoW fork reporting. Signed-off-by: Darrick J. Wong --- common/xfs| 19 ++ tests/xfs/243 | 169 + tests/xfs/243.out | 27