Re: "disk full" on a 5 GB btrfs filesystem, FAQ outdated?
On Mon, Nov 30, 2015 at 05:44:23AM +, Duncan wrote: > Yes, you can get dup metadata back, but because data and metadata > are now combined in the same blockgroups (aka chunks), they must > both be the same replication type. Thanks for this explanation, it's perfectly clear to me now. Greetings Marc -- - Marc Haber | "I don't trust Computers. They | Mailadresse im Header Leimen, Germany| lose things."Winona Ryder | Fon: *49 6224 1600402 Nordisch by Nature | How to make an American Quilt | Fax: *49 6224 1600421 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: "disk full" on a 5 GB btrfs filesystem, FAQ outdated?
Hi Hugo, On Sun, Nov 29, 2015 at 02:18:06PM +, Hugo Mills wrote: > On Sun, Nov 29, 2015 at 02:07:54PM +0100, Marc Haber wrote: > > However, the FAQ > > https://btrfs.wiki.kernel.org/index.php/FAQ#Help.21_Btrfs_claims_I.27m_out_of_space.2C_but_it_looks_like_I_should_have_lots_left.21 > > suggests that for small filesystems (<16 GB), the best solution would > > be to upgrade to at least 2.6.37 and recreate the filesystem. 2.6.37 > > is ancient, from 2011, so I am pretty sure that the filesystem _was_ > > created at least with a kernel more recent than that. > >You missed the most important thing from that paragraph: Use mixed > block groups. That's "mkfs.btrfs --mixed ..." (which I realise is > missing from the text, and I'll be adding it after I send this email). Yes, that was the important bit of missing information. My filesystem now reads: [26/512]mh@fan:/mnt/tempdisk$ df -h . Filesystem Size Used Avail Use% Mounted on /dev/mapper/banana-root 6,0G 836M 5,2G 14% /mnt/tempdisk [27/513]mh@fan:/mnt/tempdisk$ sudo btrfs fi show . Label: none uuid: b2906231-70a9-46d9-9830-38a13cb73171 Total devices 1 FS bytes used 861.29MiB devid1 size 6.00GiB used 6.00GiB path /dev/mapper/banana-root btrfs-progs v4.3 [28/514]mh@fan:/mnt/tempdisk$ sudo btrfs fi df . System, single: total=4.00MiB, used=4.00KiB Data+Metadata, single: total=6.00GiB, used=861.29MiB GlobalReserve, single: total=20.00MiB, used=0.00B [29/515]mh@fan:/mnt/tempdisk$ Can I somehow get duplicate metadata back? Or is that unnecessary? > > My normal way to recover from this situation is to btrfs add a new > > device, btrfs balance, btrfs --convert=single --force balance, btfs > > device remove, btr balance start -mconvert=dup --force and finally > > balance start again. > > > > Is there any solution to solve this more elegantly? > >Recreate the FS with --mixed, and that should deal with it. Done. Thanks! Greetings Marc -- - Marc Haber | "I don't trust Computers. They | Mailadresse im Header Leimen, Germany| lose things."Winona Ryder | Fon: *49 6224 1600402 Nordisch by Nature | How to make an American Quilt | Fax: *49 6224 1600421 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: "disk full" on a 5 GB btrfs filesystem, FAQ outdated?
On Sun, Nov 29, 2015 at 02:07:54PM +0100, Marc Haber wrote: > Hi, > > I have a banana pi with a btrfs filesystem of 5 GB in size, which > frequently runs out of space (lots of snapshots). This is currently > again the case: > > [27/524]mh@banana:~$ sudo btrfs balance start / > ERROR: error during balancing '/' - No space left on device > There may be more info in syslog - try dmesg | tail > [28/525]mh@banana:~$ sudo btrfs balance start / -dlimit=3 > [sudo] password for mh on banana: > ERROR: error during balancing '/' - No space left on device > There may be more info in syslog - try dmesg | tail > [29/526]mh@banana:~$ sudo btrfs balance start / -dlimit=3 > ERROR: error during balancing '/' - No space left on device > There may be more info in syslog - try dmesg | tail > [30/526]mh@banana:~$ sudo btrfs balance start / -dusage=0 > Done, had to relocate 0 out of 8 chunks > [31/527]mh@banana:~$ sudo btrfs balance start / -dlimit=3 > ERROR: error during balancing '/' - No space left on device > There may be more info in syslog - try dmesg | tail > [32/528]mh@banana:~$ sudo btrfs fi show / > Label: none uuid: ada6b7f5-98d6-4fee-a3a3-b73bd152ff6c > Total devices 1 FS bytes used 3.37GiB > devid1 size 6.89GiB used 4.22GiB path /dev/mapper/banana-root > > btrfs-progs v4.3 > [33/529]mh@banana:~$ sudo btrfs fi df / > Data, single: total=3.41GiB, used=3.25GiB > System, DUP: total=32.00MiB, used=16.00KiB > Metadata, DUP: total=384.00MiB, used=121.75MiB > GlobalReserve, single: total=48.00MiB, used=0.00B > [34/530]mh@banana:~$ uname -a > Linux banana 4.3.0-zgbpi-armmp-lpae+ #2 SMP Sat Nov 7 13:07:34 UTC 2015 > armv7l GNU/Linux > [36/532]mh@banana:~$ df -h / > Filesystem Size Used Avail Use% Mounted on > /dev/mapper/banana-root 6.9G 3.6G 2.9G 56% / > [37/533]mh@banana:~$ > > The first kernel that was ever booted on the device was 4.1, I am > therefore reasonably sure that the filesystem was also created with a > recent kernel. Is there any possibility to find out about the kernel > version that a filesystem was created with? > > However, the FAQ > https://btrfs.wiki.kernel.org/index.php/FAQ#Help.21_Btrfs_claims_I.27m_out_of_space.2C_but_it_looks_like_I_should_have_lots_left.21 > suggests that for small filesystems (<16 GB), the best solution would > be to upgrade to at least 2.6.37 and recreate the filesystem. 2.6.37 > is ancient, from 2011, so I am pretty sure that the filesystem _was_ > created at least with a kernel more recent than that. You missed the most important thing from that paragraph: Use mixed block groups. That's "mkfs.btrfs --mixed ..." (which I realise is missing from the text, and I'll be adding it after I send this email). > My normal way to recover from this situation is to btrfs add a new > device, btrfs balance, btrfs --convert=single --force balance, btfs > device remove, btr balance start -mconvert=dup --force and finally > balance start again. > > Is there any solution to solve this more elegantly? Recreate the FS with --mixed, and that should deal with it. Hugo. -- Hugo Mills | There isn't a noun that can't be verbed. hugo@... carfax.org.uk | http://carfax.org.uk/ | PGP: E2AB1DE4 | signature.asc Description: Digital signature
Re: "disk full" on a 5 GB btrfs filesystem, FAQ outdated?
Marc Haber posted on Sun, 29 Nov 2015 19:06:29 +0100 as excerpted: > Hi Hugo, > > On Sun, Nov 29, 2015 at 02:18:06PM +, Hugo Mills wrote: >> On Sun, Nov 29, 2015 at 02:07:54PM +0100, Marc Haber wrote: >>> However, the FAQ >>> https://btrfs.wiki.kernel.org/index.php/FAQ#Help.21_Btrfs_claims_I.27m_out_of_space.2C_but_it_looks_like_I_should_have_lots_left.21 >>> suggests that for small filesystems (<16 GB), the best solution [...] >> >> You missed the most important thing from that paragraph: Use mixed >> block groups. That's "mkfs.btrfs --mixed ..." [...] > > Yes, that was the important bit of missing information. My filesystem > now reads: > > $ df -h . > Filesystem Size Used Avail Use% Mounted on > /dev/mapper/banana-root 6,0G 836M 5,2G 14% /mnt/tempdisk > [27/513]mh@fan:/mnt/tempdisk$ sudo btrfs fi show . > Label: none uuid: b2906231-70a9-46d9-9830-38a13cb73171 > Total devices 1 FS bytes used 861.29MiB > devid 1 size 6.00GiB used 6.00GiB path /dev/mapper/banana-root > > btrfs-progs v4.3 > $ sudo btrfs fi df . > System, single: total=4.00MiB, used=4.00KiB > Data+Metadata, single: total=6.00GiB, used=861.29MiB > GlobalReserve, single: total=20.00MiB, used=0.00B > $ > > Can I somehow get duplicate metadata back? Or is that unnecessary? Define "necessary"... Yes, you can get dup metadata back, but because data and metadata are now combined in the same blockgroups (aka chunks), they must both be the same replication type. So with --mixed, if you go dup, you get dup data and metadata both, which means halving the data you can actually store, since it's stored twice. Which means... 6 GiB btrfs... less than 3 GiB data storage if you go mixed dup mode. If say 2.5 GiB (you'll probably get a bit more but...) is enough room to do what you need to do with it, I'd say go for dup, as it /should/ be incrementally more reliable for you. Obviously, if it's not going to be enough room, you better stay single data, which means either staying mixed and single metadata as well, or switching back to unmixed so you can do dup metadata and still do single data. Dup, when it works /as/ dup, gives you two copies. That's useful if one copy goes bad while the other is still good, but it's obviously not as reliable as raid1, two copies, each on different devices, would be. Meanwhile, various ssds do firmware level deduplication and turn your two copies back into only one on the physical device. That shouldn't waste space as the device shouldn't actually take space for deduped copies, but it will then uselessly spend time processing that second copy, to no benefit since the firmware second guessed you and deduped, and if that now one copy goes bad, it's bad no matter which of the two filesystem level pointers to it you try to read. So dup /should/ be incrementally more reliable, /provided/ you're actually getting dup, without the firmware or some other level second-guessing you. But tho I'm not personally familiar with the banana-pie, I'd guess that the storage is emmc or the like, and likely doesn't have firmware with that level of sophistication, so you're likely to still have dup. Meanwhile, in theory you could use the compress-force=zlib mount option to squeeze a bit more space out of that dup, letting you store somewhat more than 3 GiB of data, depending on how zlib-compressible it is. But I'd guess the CPU does better on cost and performance per watt than on raw performance, and compression, particularly zlib, is likely to slow things down sufficiently badly that it's not going to be tolerable. Still, you could try it, and if you're close on the data size and performance does end up being tolerable, that just might let you do dup, where you couldn't, otherwise. Of course you can also try compress-force(=lzo the default compression so the =spec isn't required), which should give you slightly better performance than zlib, but also a bit less efficient compression in terms of size saved. -- Duncan - List replies preferred. No HTML msgs. "Every nonfree program has a lord, a master -- and if you use the program, he is your master." Richard Stallman -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: "disk full" on a 5 GB btrfs filesystem, FAQ outdated?
On 11/30/15, Duncan <1i5t5.dun...@cox.net> wrote: > Of course you can also try compress-force(=lzo the default > compression so the =spec isn't required), which should give > you slightly better performance than zlib, but also a bit > less efficient compression in terms of size saved. lzo perf relative to zlib (A very very rough comparison) Compress/Decompress times: 1/3 - 1/2 (which is significant) Output Size: %10-20 larger -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
