Re: integration tree updated

2014-08-04 Thread Chris Mason
On 08/01/2014 05:05 AM, Anand Jain wrote: > > > Hi Chris, > > Looks like you missed Miao update (on 17 Jul) to back out the bad > patch (below), and your integration branch (published on 25 Jul) > still contains the same. > > [PATCH 2/2] Btrfs: fix wrong total device counter after removing a

btrfs and iostat - how do I measure the live performance of my btrsf filesystems?

2014-08-04 Thread G. Richard Bellamy
I apologize if this has already been addressed, regardless of whether or not it's a n00b question, I'm stumped and am looking for some guidance. Is iostat a viable measure of btrfs performance? I've got the following environment:

Re: ENOSPC with mkdir and rename

2014-08-04 Thread Chris Samuel
Hi Peter, On Mon, 4 Aug 2014 11:59:19 AM Peter Waller wrote: > On 4 August 2014 11:50, Chris Samuel wrote: > > > To be honest I'm not sure I'd suggest btrfs for production use at all at > > present, it's only recently been unmarked as experimental and to be honest > > I feel that was premature.

btrfs sub list output

2014-08-04 Thread Hugo Mills
The output options of btrfs sub list seem a bit... arbitrary? awkward? unhelpful? Here's my problem: Given a path at some arbitrary point into a mounted btrfs (sub)volume, find all subvolumes visible under that point, and identify their absolute path names. My test btrfs filesystem looks

Re: [PATCH 03/12] btrfs: handle errors from reading the quota tree root

2014-08-04 Thread Eric Sandeen
On 8/4/14, 1:51 PM, Zach Brown wrote: > On Mon, Aug 04, 2014 at 01:42:23PM -0500, Eric Sandeen wrote: >> On 8/4/14, 1:35 PM, Zach Brown wrote: >>> On Fri, Aug 01, 2014 at 06:12:37PM -0500, Eric Sandeen wrote: Reading the quota tree root may fail with ENOENT if there is no quota, which is

Re: [PATCH 03/12] btrfs: handle errors from reading the quota tree root

2014-08-04 Thread Zach Brown
On Mon, Aug 04, 2014 at 01:42:23PM -0500, Eric Sandeen wrote: > On 8/4/14, 1:35 PM, Zach Brown wrote: > > On Fri, Aug 01, 2014 at 06:12:37PM -0500, Eric Sandeen wrote: > >> Reading the quota tree root may fail with ENOENT > >> if there is no quota, which is fine, but the code was > >> ignoring ever

Re: [PATCH 03/12] btrfs: handle errors from reading the quota tree root

2014-08-04 Thread Eric Sandeen
On 8/4/14, 1:35 PM, Zach Brown wrote: > On Fri, Aug 01, 2014 at 06:12:37PM -0500, Eric Sandeen wrote: >> Reading the quota tree root may fail with ENOENT >> if there is no quota, which is fine, but the code was >> ignoring every other error as well, which is not fine. > > Kinda makes you want to w

Re: [PATCH 03/12] btrfs: handle errors from reading the quota tree root

2014-08-04 Thread Zach Brown
On Fri, Aug 01, 2014 at 06:12:37PM -0500, Eric Sandeen wrote: > Reading the quota tree root may fail with ENOENT > if there is no quota, which is fine, but the code was > ignoring every other error as well, which is not fine. Kinda makes you want to write a test that would have caught this. Kinda

Re: [PATCH 1/1] Btrfs: fix sparse warning

2014-08-04 Thread Zach Brown
On Sat, Aug 02, 2014 at 02:24:49PM +0200, Fabian Frederick wrote: > On Thu, 17 Jul 2014 12:01:52 -0700 > Zach Brown wrote: > > > > > > @@ -515,7 +515,8 @@ static int write_buf(struct file *filp, const > > > > > void *buf, > > > > > u32 len, loff_t *off) > > > > > > > > Though this probably wants

[PATCH] Btrfs: make btrfs_search_forward return with nodes unlocked

2014-08-04 Thread Filipe Manana
None of the uses of btrfs_search_forward() need to have the path nodes (level >= 1) read locked, only the leaf needs to be locked while the caller processes it. Therefore make it return a path with all nodes unlocked, except for the leaf. This change is motivated by the observation that during a f

Re: ENOSPC with mkdir and rename

2014-08-04 Thread Austin S Hemmelgarn
On 2014-08-04 06:31, Peter Waller wrote: > Thanks Hugo, this is the most informative e-mail yet! (more inline) > > On 4 August 2014 11:22, Hugo Mills wrote: >> >> * btrfs fi show >> - look at the total and used values. If used < total, you're OK. >> If used == total, then you could pot

Re: Unrecoverable errors when the btrfs file system was modified outside the running OS

2014-08-04 Thread Chris Murphy
On Aug 4, 2014, at 12:29 AM, rocwhite168 wrote: > Hello, > > I just had a very frustrating experience with btrfs, which I was only > able to resolve by rolling back to ext4 using the subvol btrfs-convert > created. The same type of situation occurred before when I was using > the ext file syste

Re: ENOSPC with mkdir and rename

2014-08-04 Thread Mitch Harder
On Mon, Aug 4, 2014 at 9:47 AM, Russell Coker wrote: > If you regularly run a scrub with options such as "-dusage=50 -musage=10" then > the amount of free space in metadata chunks will tend to be a lot greater than > that in data chunks. > Just to clarify for posterity, I'm pretty sure you meant

Re: Scan not being performed properly on boot

2014-08-04 Thread Peter Roberts
On 04/08/2014 04:31, Russell Coker wrote: What is GRUB (or your boot loader) giving as parameters to the kernel? What error messages appear on screen? Sometimes it's helpful to photograph the screen and put the picture on a web server to help people diagnose the problem. Here a screenshot htt

Re: ENOSPC with mkdir and rename

2014-08-04 Thread Russell Coker
On Mon, 4 Aug 2014 14:17:02 Peter Waller wrote: > For anyone else having this problem, this article is fairly useful for > understanding disk full problems and rebalance: > > http://marc.merlins.org/perso/btrfs/post_2014-05-04_Fixing-Btrfs-Filesystem-> > Full-Problems.html > > It actually covers

Re: ENOSPC with mkdir and rename

2014-08-04 Thread Austin S Hemmelgarn
On 2014-08-04 10:11, Peter Waller wrote: > On 4 August 2014 15:02, Austin S Hemmelgarn wrote: >> I really disagree with the statement that adding more storage is >> difficult or expensive, all you need to do is plug in a 2G USB flash >> drive, or allocate a ramdisk, and add the device to the files

Re: ENOSPC with mkdir and rename

2014-08-04 Thread Peter Waller
On 4 August 2014 15:02, Austin S Hemmelgarn wrote: > I really disagree with the statement that adding more storage is > difficult or expensive, all you need to do is plug in a 2G USB flash > drive, or allocate a ramdisk, and add the device to the filesystem only > long enough to do a full balance.

Re: ENOSPC with mkdir and rename

2014-08-04 Thread Austin S Hemmelgarn
On 2014-08-04 09:17, Peter Waller wrote: > For anyone else having this problem, this article is fairly useful for > understanding disk full problems and rebalance: > > http://marc.merlins.org/perso/btrfs/post_2014-05-04_Fixing-Btrfs-Filesystem-Full-Problems.html > > It actually covers the problem

Re: ENOSPC with mkdir and rename

2014-08-04 Thread Hugo Mills
On Mon, Aug 04, 2014 at 02:17:02PM +0100, Peter Waller wrote: > For anyone else having this problem, this article is fairly useful for > understanding disk full problems and rebalance: > > http://marc.merlins.org/perso/btrfs/post_2014-05-04_Fixing-Btrfs-Filesystem-Full-Problems.html > > It actual

Re: Work Queue for btrfs compression writes

2014-08-04 Thread Chris Mason
On 07/29/2014 11:54 PM, Nick Krause wrote: > Hey Guys , > I am new to reading and writing kernel code.I got interested in > writing code for btrfs as it seems to > need more work then other file systems and this seems other then > drivers, a good use of time on my part. > I interested in helpin

Re: ENOSPC with mkdir and rename

2014-08-04 Thread Peter Waller
For anyone else having this problem, this article is fairly useful for understanding disk full problems and rebalance: http://marc.merlins.org/perso/btrfs/post_2014-05-04_Fixing-Btrfs-Filesystem-Full-Problems.html It actually covers the problem that I had, which is that a rebalance can't take pla

Re: btrfs on bcache

2014-08-04 Thread Fábio Pfeifer
After completely loosing my filesystem twice because of this bug, I gave up using btrfs on top of bcache (also writeback). In my case, I used to have some subvolumes and some snapshot of these subvolumes, but not many of them. The btrfs mantra "backup, bakcup and backup" saved me. Best regards, F

Re: [PATCH] Btrfs: fix compressed write corruption on enospc

2014-08-04 Thread Martin Steigerwald
Am Montag, 4. August 2014, 14:50:29 schrieb Martin Steigerwald: > Am Freitag, 25. Juli 2014, 11:54:37 schrieb Martin Steigerwald: > > Am Donnerstag, 24. Juli 2014, 22:48:05 schrieben Sie: > > > When failing to allocate space for the whole compressed extent, we'll > > > fallback to uncompressed IO,

Re: [PATCH] Btrfs: fix compressed write corruption on enospc

2014-08-04 Thread Martin Steigerwald
Am Freitag, 25. Juli 2014, 11:54:37 schrieb Martin Steigerwald: > Am Donnerstag, 24. Juli 2014, 22:48:05 schrieben Sie: > > When failing to allocate space for the whole compressed extent, we'll > > fallback to uncompressed IO, but we've forgotten to redirty the pages > > which belong to this compre

Re: ENOSPC with mkdir and rename

2014-08-04 Thread Hugo Mills
On Mon, Aug 04, 2014 at 01:04:25PM +0200, Clemens Eisserer wrote: > Hi Hugo, > > >On the 3.15+ kernels, the block reserve is split out of metadata > > and reported separately. This helps with the following process: > > Thanks a lot for pointing this out, I hadn't noticed this change until now

Re: ENOSPC with mkdir and rename

2014-08-04 Thread Hugo Mills
On Mon, Aug 04, 2014 at 11:48:17AM +0100, Peter Waller wrote: > On 4 August 2014 11:39, Hugo Mills wrote: > >> > * btrfs fi df > >> > - look at metadata used vs total. If these are close to zero (on > >> > 3.15+) or close to 512 MiB (on <3.15), then you are in danger of > >> > ENO

Re: ENOSPC with mkdir and rename

2014-08-04 Thread Clemens Eisserer
Hi Hugo, >On the 3.15+ kernels, the block reserve is split out of metadata > and reported separately. This helps with the following process: Thanks a lot for pointing this out, I hadn't noticed this change until now. One thing I didn't find any information about is the overhead introduced by

Re: ENOSPC with mkdir and rename

2014-08-04 Thread Peter Waller
On 4 August 2014 11:50, Chris Samuel wrote: > To be honest I'm not sure I'd suggest btrfs for production use at all at > present, it's only recently been unmarked as experimental and to be honest I > feel that was premature. :-( Thanks for the honest answer. There are very positive signals out t

Re: ENOSPC with mkdir and rename

2014-08-04 Thread Chris Samuel
On Mon, 4 Aug 2014 11:09:23 AM Peter Waller wrote: > I accept that. It's all very well if you read the BTRFS list and/or > are a BTRFS developer. But if you're trying to work it out in the heat > of battle, as we have sysadmins who would have to, there is a > combination of things here that makes

Re: ENOSPC with mkdir and rename

2014-08-04 Thread Peter Waller
Thanks Hugo, this is the most informative e-mail yet! (more inline) On 4 August 2014 11:22, Hugo Mills wrote: > > * btrfs fi show > - look at the total and used values. If used < total, you're OK. > If used == total, then you could potentially hit ENOSPC. Another thing which is unclea

Re: ENOSPC with mkdir and rename

2014-08-04 Thread Peter Waller
On 4 August 2014 11:39, Hugo Mills wrote: >> > * btrfs fi df >> > - look at metadata used vs total. If these are close to zero (on >> > 3.15+) or close to 512 MiB (on <3.15), then you are in danger of >> > ENOSPC. >> >> Hmm. It's unfortunate that this could indicate an amount of s

Re: ENOSPC with mkdir and rename

2014-08-04 Thread Hugo Mills
On Mon, Aug 04, 2014 at 11:31:57AM +0100, Peter Waller wrote: > Thanks Hugo, this is the most informative e-mail yet! (more inline) > > On 4 August 2014 11:22, Hugo Mills wrote: > > > > * btrfs fi show > > - look at the total and used values. If used < total, you're OK. > > If used ==

Re: ENOSPC with mkdir and rename

2014-08-04 Thread Chris Samuel
On Mon, 4 Aug 2014 11:56:46 AM Clemens Eisserer wrote: > Which doesn't protect the *average* user from running into issues like this. No, but they need to be aware of it. > Just because it has been discussed, doesn't mean nothing can/should be done > about it Indeed, and a lot of work has been

Re: ENOSPC with mkdir and rename

2014-08-04 Thread Hugo Mills
On Mon, Aug 04, 2014 at 11:09:23AM +0100, Peter Waller wrote: > On 4 August 2014 10:39, Chris Samuel wrote: > > On Mon, 4 Aug 2014 09:14:19 AM Peter Waller wrote: > >> All of this is *very* surprising. > > > > Hmm, it shouldn't be, the ENOSPC issues are well known and have been > > discussed > >

Re: ENOSPC with mkdir and rename

2014-08-04 Thread Peter Waller
On 4 August 2014 10:39, Chris Samuel wrote: > On Mon, 4 Aug 2014 09:14:19 AM Peter Waller wrote: >> All of this is *very* surprising. > > Hmm, it shouldn't be, the ENOSPC issues are well known and have been discussed > here for years. I accept that. It's all very well if you read the BTRFS list a

Re: ENOSPC with mkdir and rename

2014-08-04 Thread Clemens Eisserer
Hi Chris, > Hmm, it shouldn't be, the ENOSPC issues are well known and have been discussed > here for years. Which doesn't protect the *average* user from running into issues like this. Just because it has been discussed, doesn't mean nothing can/should be done about it ;) However, as I am only

Re: ENOSPC with mkdir and rename

2014-08-04 Thread Chris Samuel
On Mon, 4 Aug 2014 09:14:19 AM Peter Waller wrote: > All of this is *very* surprising. Hmm, it shouldn't be, the ENOSPC issues are well known and have been discussed here for years. cheers, Chris -- Chris Samuel : http://www.csamuel.org/ : Melbourne, VIC -- To unsubscribe from this list

Re: Scan not being performed properly on boot

2014-08-04 Thread Chris Samuel
On Mon, 4 Aug 2014 01:31:42 PM Russell Coker wrote: > Is BTRFS supported in that version of Ubuntu? Out of the box a fresh 14.04 install onto btrfs worked fine for me on two different sets of hardware. 13.10 the same on a third piece of hardware. cheers, Chris -- Chris Samuel : http://www.

Re: ENOSPC with mkdir and rename

2014-08-04 Thread Clemens Eisserer
Hi Peter, > All of this is *very* surprising. I'm not new to BTRFS, I've been > using it on my own machines for multiple years. I didn't realise there > was an un-holstered footgun on my lap at this point. How can it be > made clear how to avoid the ENOSPC problem to myself and other > sysadmins?

Re: ENOSPC with mkdir and rename

2014-08-04 Thread Peter Waller
Thanks for responses. All of this is *very* surprising. I'm not new to BTRFS, I've been using it on my own machines for multiple years. I didn't realise there was an un-holstered footgun on my lap at this point. How can it be made clear how to avoid the ENOSPC problem to myself and other sysadmins

[PATCH] btrfs-progs: Documentations: Fix a format error in btrfs-replace.txt

2014-08-04 Thread Qu Wenruo
Missing '+'s cause '-B' option not displayed correctly, add it to fix. Signed-off-by: Qu Wenruo --- Documentation/btrfs-replace.txt | 2 ++ 1 file changed, 2 insertions(+) diff --git a/Documentation/btrfs-replace.txt b/Documentation/btrfs-replace.txt index eecf9b0..067b02d 100644 --- a/Document