Re: ENOSPC errors during raid1 rebalance

2014-03-13 Thread Eugene Crosser
Hello, I want to report that I have the same problem as Michael Russo, except in my case there is definitely *a lot* of free space. I had ext4 on a 1Tb LVM mirror. The filesystem was 96% full, with many multi-Gb files. I successfully converted it into btrfs, removed ext2_saved subvolume, but did

RE: ENOSPC errors during raid1 rebalance

2014-03-07 Thread Mike Russo
Alright! After doing: cd /mymedia; find . -type f | while read file; do mv -v "$file" /dev/shm; f2=`basename "$file"`; mv -v "/dev/shm/$f2" "$file"; done I finally moved whatever files out of the "single" allocation and back onto the new RAID1 profile: oot@ossy:~# /usr/src/btrfs-progs/btrfs f

RE: ENOSPC errors during raid1 rebalance

2014-03-07 Thread Mike Russo
e files I care about probably aren't in this list so I'll do it for all the files on the system (since I can't find out what files are in those block groups). -Original Message- From: Hugo Mills [mailto:h...@carfax.org.uk] Sent: Friday, March 07, 2014 3:02 AM To: M

Re: ENOSPC errors during raid1 rebalance

2014-03-07 Thread Duncan
Hugo Mills posted on Fri, 07 Mar 2014 08:02:13 + as excerpted: > On Fri, Mar 07, 2014 at 01:13:53AM +, Michael Russo wrote: >> Duncan <1i5t5.duncan cox.net> writes: >> >> > But if you're not using compression, /that/ can't explain it... >> > >> > >> Ha! Well while that was an interesti

Re: ENOSPC errors during raid1 rebalance

2014-03-07 Thread Hugo Mills
On Fri, Mar 07, 2014 at 01:13:53AM +, Michael Russo wrote: > Duncan <1i5t5.duncan cox.net> writes: > > > But if you're not using compression, /that/ can't explain it... > > > > Ha! Well while that was an interesting discussion of fragmentation, > I am only using the default mount options he

Re: ENOSPC errors during raid1 rebalance

2014-03-06 Thread Michael Russo
Duncan <1i5t5.duncan cox.net> writes: > But if you're not using compression, /that/ can't explain it... > Ha! Well while that was an interesting discussion of fragmentation, I am only using the default mount options here and so no compression. The only reason I'm really even looking at the fra

Re: ENOSPC errors during raid1 rebalance

2014-03-06 Thread Chris Murphy
On Mar 5, 2014, at 3:13 PM, Michael Russo wrote: > Chris Murphy colorremedies.com> writes: > >> Did you do a defrag and balance after ext4>btrfs conversion, >> but before data/metadata profile conversion? > > No I didn't, as I thought it was only optional and didn't realize > it might later

Re: ENOSPC errors during raid1 rebalance

2014-03-05 Thread Duncan
Michael Russo posted on Wed, 05 Mar 2014 22:13:10 + as excerpted: > Chris Murphy colorremedies.com> writes: > >> You could also try a full defragment by specifying -r on the mount >> point with a small -t value to effectively cause everything to be >> subject to defragmenting. If this still

Re: ENOSPC errors during raid1 rebalance

2014-03-05 Thread Michael Russo
Chris Murphy colorremedies.com> writes: > Did you do a defrag and balance after ext4>btrfs conversion, > but before data/metadata profile conversion? No I didn't, as I thought it was only optional and didn't realize it might later affect my ability to change profiles. -- To unsubscribe fr

Re: ENOSPC errors during raid1 rebalance

2014-03-05 Thread Michael Russo
Chris Murphy colorremedies.com> writes: > You could also try a full defragment by specifying -r on the mount point > with a small -t value to effectively cause everything to be subject > to defragmenting. If this still doesn't permit soft rebalance, then maybe > filefrag can find files that have

Re: ENOSPC errors during raid1 rebalance

2014-03-04 Thread Chris Murphy
On Mar 4, 2014, at 5:27 PM, Mike Russo wrote: > I'm sure this is due to the ext4 conversion, but that means the utility is > making a btrfs filesystem that later can't be converted to another profile > for some reason. https://btrfs.wiki.kernel.org/index.php/Conversion_from_Ext3 "But still,

Re: ENOSPC errors during raid1 rebalance

2014-03-04 Thread Chris Murphy
On Mar 4, 2014, at 5:27 PM, Mike Russo wrote: > Chris Murphy colorremedies.com> writes: > >>> How can I find out what file is on the block group that >>> it's having a problem with? >> >> I think that's btrfs-debug-tree -b ? But don't hold me to that. >> I haven't done enough debugging to f

RE: ENOSPC errors during raid1 rebalance

2014-03-04 Thread Mike Russo
Chris Murphy colorremedies.com> writes: > > How can I find out what file is on the block group that > > it's having a problem with? > > I think that's btrfs-debug-tree -b ? But don't hold me to that. > I haven't done enough debugging to find files > from block numbers. Another that might be re

Re: ENOSPC errors during raid1 rebalance

2014-03-04 Thread Chris Murphy
On Mar 4, 2014, at 11:54 AM, Michael Russo wrote: > Chris Murphy colorremedies.com> writes: >> Based on my reading of the man page, I think it's expected. >> You either need -s -l or -t. > > Ok, although the man page uses [ ] instead of < > and something > does happen if I don't add them. But

Re: ENOSPC errors during raid1 rebalance

2014-03-04 Thread Michael Russo
Chris Murphy colorremedies.com> writes: > Based on my reading of the man page, I think it's expected. >You either need -s -l or -t. Ok, although the man page uses [ ] instead of < > and something does happen if I don't add them. But if I use "-t 1" wouldn't that get everything? > > > Now I've

Re: ENOSPC errors during raid1 rebalance

2014-03-04 Thread Chris Murphy
On Mar 4, 2014, at 8:55 AM, Michael Russo wrote: > Hugo Mills carfax.org.uk> writes: > >> This is just a guess, but you might have some large (>1GB) > extents >> in there that span across multiple chunks. I'd suggest running a > btrfs >> defrag on any particularly big files and see if that

Re: ENOSPC errors during raid1 rebalance

2014-03-04 Thread Michael Russo
Hugo Mills carfax.org.uk> writes: >This is just a guess, but you might have some large (>1GB) extents > in there that span across multiple chunks. I'd suggest running a btrfs > defrag on any particularly big files and see if that helps the situation. > Doing this is definitely helping, b

Re: ENOSPC errors during raid1 rebalance

2014-03-03 Thread Chris Murphy
On Mar 3, 2014, at 1:50 PM, Michael Russo wrote: > Oh yeah, it was definitely a problem with either the drives > or the external enclosure, which was converting USB to SATA > and mirroring the drives internally (it was a WD MyBook > Mirror Edition). There was a problem with one of the drives

Re: ENOSPC errors during raid1 rebalance

2014-03-03 Thread Michael Russo
Chris Murphy colorremedies.com> writes: > > > Chris Murphy colorremedies.com> writes: > Gotcha. I think Hugo has the best next step. Defragment. I think this is going to work. I cancelled a partial defrag and did another move attempt, and this time 5GB got moved! So I'm going to let the whol

Re: ENOSPC errors during raid1 rebalance

2014-03-03 Thread Chris Murphy
On Mar 3, 2014, at 12:24 PM, Michael Russo wrote: > Chris Murphy colorremedies.com> writes: > >> It might be worth adding enospc_debug as a mount option > > These messages only appear when I mount with enospc_debug. Gotcha. I think Hugo has the best next step. Defragment. > As for whether

Re: ENOSPC errors during raid1 rebalance

2014-03-03 Thread Michael Russo
Chris Murphy colorremedies.com> writes: > It might be worth adding enospc_debug as a mount option These messages only appear when I mount with enospc_debug. I included them as examples but I can post the full output later if needed. As for whether btrfs-convert should be recommended, I mean

Re: ENOSPC errors during raid1 rebalance

2014-03-03 Thread Chris Murphy
On Mar 3, 2014, at 11:24 AM, Michael Russo wrote: > Duncan <1i5t5.duncan cox.net> writes: >> >> That allows rollback if desired, but does tie up some some space with the >> automatically created btrfs "snapshot" that contains the ext3/4 metadata >> and untouched data. > > Nope, I definite

Re: ENOSPC errors during raid1 rebalance

2014-03-03 Thread Hugo Mills
On Mon, Mar 03, 2014 at 05:23:43PM +, Mike Russo wrote: > Hi guys - > I'm trying to convert a disk from single (/dev/sdc1) to RAID1 (dev/sdd1), and > the filesystem was previously ext4 but the conversion seemed to go just fine, > and I have no snapshots. System and metadata convert, and almos

Re: ENOSPC errors during raid1 rebalance

2014-03-03 Thread Michael Russo
Duncan <1i5t5.duncan cox.net> writes: > > That allows rollback if desired, but does tie up some some space with the > automatically created btrfs "snapshot" that contains the ext3/4 metadata > and untouched data. Nope, I definitely deleted the snapshots, running btrfs sub list gives me noth

Re: ENOSPC errors during raid1 rebalance

2014-03-03 Thread Duncan
Mike Russo posted on Mon, 03 Mar 2014 17:23:43 + as excerpted: > I'm trying to convert a disk from single (/dev/sdc1) to RAID1 > (dev/sdd1), and the filesystem was previously ext4 but the conversion > seemed to go just fine, and I have no snapshots. System and metadata > convert, and almost al