Re: ENOSPC errors during raid1 rebalance

2014-03-07 Thread Hugo Mills
On Fri, Mar 07, 2014 at 01:13:53AM +, Michael Russo wrote: Duncan 1i5t5.duncan at cox.net writes: But if you're not using compression, /that/ can't explain it... Ha! Well while that was an interesting discussion of fragmentation, I am only using the default mount options here and

Re: ENOSPC errors during raid1 rebalance

2014-03-07 Thread Duncan
Hugo Mills posted on Fri, 07 Mar 2014 08:02:13 + as excerpted: On Fri, Mar 07, 2014 at 01:13:53AM +, Michael Russo wrote: Duncan 1i5t5.duncan at cox.net writes: But if you're not using compression, /that/ can't explain it... Ha! Well while that was an interesting discussion

RE: ENOSPC errors during raid1 rebalance

2014-03-07 Thread Mike Russo
on the system (since I can't find out what files are in those block groups). -Original Message- From: Hugo Mills [mailto:h...@carfax.org.uk] Sent: Friday, March 07, 2014 3:02 AM To: Mike Russo Cc: linux-btrfs@vger.kernel.org Subject: Re: ENOSPC errors during raid1 rebalance The defrag

RE: ENOSPC errors during raid1 rebalance

2014-03-07 Thread Mike Russo
Alright! After doing: cd /mymedia; find . -type f | while read file; do mv -v $file /dev/shm; f2=`basename $file`; mv -v /dev/shm/$f2 $file; done I finally moved whatever files out of the single allocation and back onto the new RAID1 profile: oot@ossy:~# /usr/src/btrfs-progs/btrfs fi df

Re: ENOSPC errors during raid1 rebalance

2014-03-06 Thread Chris Murphy
On Mar 5, 2014, at 3:13 PM, Michael Russo m...@papersolve.com wrote: Chris Murphy lists at colorremedies.com writes: Did you do a defrag and balance after ext4btrfs conversion, but before data/metadata profile conversion? No I didn't, as I thought it was only optional and didn't realize

Re: ENOSPC errors during raid1 rebalance

2014-03-06 Thread Michael Russo
Duncan 1i5t5.duncan at cox.net writes: But if you're not using compression, /that/ can't explain it... Ha! Well while that was an interesting discussion of fragmentation, I am only using the default mount options here and so no compression. The only reason I'm really even looking at the

Re: ENOSPC errors during raid1 rebalance

2014-03-05 Thread Michael Russo
Chris Murphy lists at colorremedies.com writes: You could also try a full defragment by specifying -r on the mount point with a small -t value to effectively cause everything to be subject to defragmenting. If this still doesn't permit soft rebalance, then maybe filefrag can find files that

Re: ENOSPC errors during raid1 rebalance

2014-03-05 Thread Michael Russo
Chris Murphy lists at colorremedies.com writes: Did you do a defrag and balance after ext4btrfs conversion, but before data/metadata profile conversion? No I didn't, as I thought it was only optional and didn't realize it might later affect my ability to change profiles. -- To

Re: ENOSPC errors during raid1 rebalance

2014-03-05 Thread Duncan
Michael Russo posted on Wed, 05 Mar 2014 22:13:10 + as excerpted: Chris Murphy lists at colorremedies.com writes: You could also try a full defragment by specifying -r on the mount point with a small -t value to effectively cause everything to be subject to defragmenting. If this still

Re: ENOSPC errors during raid1 rebalance

2014-03-04 Thread Michael Russo
Hugo Mills hugo at carfax.org.uk writes: This is just a guess, but you might have some large (1GB) extents in there that span across multiple chunks. I'd suggest running a btrfs defrag on any particularly big files and see if that helps the situation. Doing this is definitely helping,

Re: ENOSPC errors during raid1 rebalance

2014-03-04 Thread Chris Murphy
On Mar 4, 2014, at 8:55 AM, Michael Russo m...@papersolve.com wrote: Hugo Mills hugo at carfax.org.uk writes: This is just a guess, but you might have some large (1GB) extents in there that span across multiple chunks. I'd suggest running a btrfs defrag on any particularly big files

Re: ENOSPC errors during raid1 rebalance

2014-03-04 Thread Michael Russo
Chris Murphy lists at colorremedies.com writes: Based on my reading of the man page, I think it's expected. You either need -s -l or -t. Ok, although the man page uses [ ] instead of and something does happen if I don't add them. But if I use -t 1 wouldn't that get everything? Now I've

Re: ENOSPC errors during raid1 rebalance

2014-03-04 Thread Chris Murphy
On Mar 4, 2014, at 11:54 AM, Michael Russo m...@papersolve.com wrote: Chris Murphy lists at colorremedies.com writes: Based on my reading of the man page, I think it's expected. You either need -s -l or -t. Ok, although the man page uses [ ] instead of and something does happen if I

RE: ENOSPC errors during raid1 rebalance

2014-03-04 Thread Mike Russo
Chris Murphy lists at colorremedies.com writes: How can I find out what file is on the block group that it's having a problem with? I think that's btrfs-debug-tree -b ? But don't hold me to that. I haven't done enough debugging to find files from block numbers. Another that might be

Re: ENOSPC errors during raid1 rebalance

2014-03-04 Thread Chris Murphy
On Mar 4, 2014, at 5:27 PM, Mike Russo m...@papersolve.com wrote: Chris Murphy lists at colorremedies.com writes: How can I find out what file is on the block group that it's having a problem with? I think that's btrfs-debug-tree -b ? But don't hold me to that. I haven't done enough

Re: ENOSPC errors during raid1 rebalance

2014-03-03 Thread Duncan
Mike Russo posted on Mon, 03 Mar 2014 17:23:43 + as excerpted: I'm trying to convert a disk from single (/dev/sdc1) to RAID1 (dev/sdd1), and the filesystem was previously ext4 but the conversion seemed to go just fine, and I have no snapshots. System and metadata convert, and almost all

Re: ENOSPC errors during raid1 rebalance

2014-03-03 Thread Michael Russo
Duncan 1i5t5.duncan at cox.net writes: That allows rollback if desired, but does tie up some some space with the automatically created btrfs snapshot that contains the ext3/4 metadata and untouched data. Nope, I definitely deleted the snapshots, running btrfs sub list gives me nothing

Re: ENOSPC errors during raid1 rebalance

2014-03-03 Thread Hugo Mills
On Mon, Mar 03, 2014 at 05:23:43PM +, Mike Russo wrote: Hi guys - I'm trying to convert a disk from single (/dev/sdc1) to RAID1 (dev/sdd1), and the filesystem was previously ext4 but the conversion seemed to go just fine, and I have no snapshots. System and metadata convert, and almost

Re: ENOSPC errors during raid1 rebalance

2014-03-03 Thread Chris Murphy
On Mar 3, 2014, at 11:24 AM, Michael Russo m...@papersolve.com wrote: Duncan 1i5t5.duncan at cox.net writes: That allows rollback if desired, but does tie up some some space with the automatically created btrfs snapshot that contains the ext3/4 metadata and untouched data. Nope, I

Re: ENOSPC errors during raid1 rebalance

2014-03-03 Thread Michael Russo
Chris Murphy lists at colorremedies.com writes: It might be worth adding enospc_debug as a mount option These messages only appear when I mount with enospc_debug. I included them as examples but I can post the full output later if needed. As for whether btrfs-convert should be recommended, I

Re: ENOSPC errors during raid1 rebalance

2014-03-03 Thread Chris Murphy
On Mar 3, 2014, at 12:24 PM, Michael Russo m...@papersolve.com wrote: Chris Murphy lists at colorremedies.com writes: It might be worth adding enospc_debug as a mount option These messages only appear when I mount with enospc_debug. Gotcha. I think Hugo has the best next step.

Re: ENOSPC errors during raid1 rebalance

2014-03-03 Thread Michael Russo
Chris Murphy lists at colorremedies.com writes: Chris Murphy lists at colorremedies.com writes: Gotcha. I think Hugo has the best next step. Defragment. I think this is going to work. I cancelled a partial defrag and did another move attempt, and this time 5GB got moved! So I'm going to

Re: ENOSPC errors during raid1 rebalance

2014-03-03 Thread Chris Murphy
On Mar 3, 2014, at 1:50 PM, Michael Russo m...@papersolve.com wrote: Oh yeah, it was definitely a problem with either the drives or the external enclosure, which was converting USB to SATA and mirroring the drives internally (it was a WD MyBook Mirror Edition). There was a problem with