Re: btrfs: mixing raid0 and raid1 - How?
Hi Matthias, On Sat, Apr 16, 2016 at 08:15:43PM +0200, Matthias Bodenbinder wrote: > I am using Mint LMDE2 with debian backports. So I do have kernel > 4.4+71~bpo8+1 running. > btrfs tools are from debian stable, which has version 3.17. I am wondering if > it would make sense to also get the tools from the backports repo which has > version 4.4-1~bpo8+1. > > I understand from your message that I should do that. Right? You definitely need btrfs-tools newer than 3.17 owing to a number of bugs that have been fixed. In your position if I was running a kernel from backports then I would also want btrfs-tools from backports, yes. Cheers, Andy -- http://bitfolk.com/ -- No-nonsense VPS hosting
Re: btrfs: mixing raid0 and raid1 - How?
Hello Andy, I am trying to stay with newest versions as long as it does not jeopardize stability. I am using Mint LMDE2 with debian backports. So I do have kernel 4.4+71~bpo8+1 running. btrfs tools are from debian stable, which has version 3.17. I am wondering if it would make sense to also get the tools from the backports repo which has version 4.4-1~bpo8+1. I understand from your message that I should do that. Right? Matthias 4.4+71~bpo8+1 Am 16.04.2016 um 19:57 schrieb Andy Smith: > Hello, > > On Sat, Apr 16, 2016 at 01:35:20PM +0200, Luis Felipe Tabera Alonso wrote: >> Still btrfs is quite young, I am not sure if there are serious issues in >> 3.17, >> I would make some experiments before actual use. > > If you are going to use btrfs I would consider it essential to be > subscribed to the linux-btrfs mailing list. You will also need to > use a much newer kernel than 3.17, and you will need to commit to > continuing to use newer kernels for some time yet. > > I mention this because you need to be aware that you may still hit > issues where you need the help of linux-btrfs and that there is a > need to use newer kernels than you will find packaged in Debian > stable. > > I do myself run btrfs at home and I thought I had a stable > combination of kernel version and userland tools (btrfs-tools), but > when I came to need to replace a dead device I found that subsequent > developments in btrfs meant I needed a newer btrfs-tools, and that > in turn meant I needed a newer kernel. > > If the idea of having to upgrade kernel and some userland tools in > order to recover from a simple situation of a dead device does not > appeal to you then btrfs may not yet be for you. Things like this > are why I do not yet run it in production. > > Cheers, > Andy >
Re: btrfs: mixing raid0 and raid1 - How?
You are right. I did the same test with dd. It took a while ;-) 23# dd if=/dev/zero of=file.zero dd: Schreiben in „file.zero“: Auf dem Gerät ist kein Speicherplatz mehr verfügbar 1462670786+0 Datensätze ein 1462670785+0 Datensätze aus 748887441920 Bytes (749 GB) kopiert, 14831,9 s, 50,5 MB/s 24# ll -h insgesamt 698G -rw-r--r-- 1 root root 698G Apr 16 20:03 file.zero Thank you for helping me! Matthias Am 16.04.2016 um 13:35 schrieb Luis Felipe Tabera Alonso: > On sábado, 16 de abril de 2016 12:05:05 (CEST) Matthias Bodenbinder wrote: > >> 38# df -h /mnt/test/ >> Dateisystem Größe Benutzt Verf. Verw% Eingehängt auf >> /dev/sdg 699G 17M 466G 1% /mnt/test > > df is not reliable for btrfs raids, it is better to use btrfs fi df to check > actual used space. > > Using btrfs-tools and kernel from sid (4.4): > > root@mychabol:/mnt/temporal# dd if=/dev/zero of=disk1 bs=1 count=0 seek=750M > root@mychabol:/mnt/temporal# dd if=/dev/zero of=disk2 bs=1 count=0 seek=500M > root@mychabol:/mnt/temporal# dd if=/dev/zero of=disk3 bs=1 count=0 seek=250M > > root@mychabol:/mnt/temporal# mkfs.btrfs -f -draid1 -mraid1 disk1 disk2 disk3 > btrfs-progs v4.4 > See http://btrfs.wiki.kernel.org for more information. > > Label: (null) > UUID: 4fbfda48-3a10-4e1b-92d8-96be759c2a8c > Node size: 16384 > Sector size:4096 > Filesystem size:1.46GiB > Block group profiles: > Data: RAID183.00MiB > Metadata: RAID183.00MiB > System: RAID112.00MiB > SSD detected: no > Incompat features: extref, skinny-metadata > Number of devices: 3 > Devices: >IDSIZE PATH > 1 750.00MiB disk1 > 2 500.00MiB disk2 > 3 250.00MiB disk3 > > root@mychabol:/mnt/temporal# losetup /dev/loop0 disk1 > root@mychabol:/mnt/temporal# losetup /dev/loop1 disk2 > root@mychabol:/mnt/temporal# losetup /dev/loop2 disk3 > > root@mychabol:/mnt/temporal# mkdir test > root@mychabol:/mnt/temporal# mount /dev/loop0 test > > root@mychabol:/mnt/temporal# btrfs fi df test/ > Data, RAID1: total=75.00MiB, used=0.00B > System, RAID1: total=8.00MiB, used=16.00KiB > Metadata, RAID1: total=75.00MiB, used=112.00KiB > GlobalReserve, single: total=16.00MiB, used=0.00B > > root@mychabol:/mnt/temporal# df -h test/ > S.ficheros Tamaño Usados Disp Uso% Montado en > /dev/loop0 750M17M 491M 4% /mnt/temporal/test > > Like in your case, df thinks that there are only 491M usable. However, if I > try to fill the volume > > root@mychabol:/mnt/temporal# cd test/ > root@mychabol:/mnt/temporal/test# dd if=/dev/zero of=file.zero > dd: escribiendo en 'file.zero': No queda espacio en el dispositivo > 1345402+0 registros leídos > 1345401+0 registros escritos > 688845312 bytes (689 MB, 657 MiB) copied, 3,10508 s, 222 MB/s > > root@mychabol:/mnt/temporal/test# ls -lh > total 657M > -rw-r--r-- 1 root root 657M abr 16 13:24 file.zero > root@mychabol:/mnt/temporal/test# df -h . > S.ficheros Tamaño Usados Disp Uso% Montado en > /dev/loop0 750M 674M 64K 100% /mnt/temporal/test > > With the 75M of metadata, you can check that this fills the three devices. > Moreover > > root@mychabol:/mnt/temporal/test# rm file.zero > root@mychabol:/mnt/temporal/test# df -h . > S.ficheros Tamaño Usados Disp Uso% Montado en > /dev/loop0 750M17M 657M 3% /mnt/temporal/test > > Try to do the same to see if df gets more accurate. > > Still btrfs is quite young, I am not sure if there are serious issues in > 3.17, > I would make some experiments before actual use. > > Bests > > Luis > >
Re: btrfs: mixing raid0 and raid1 - How?
On 04/16/2016 12:00 AM, Matthias Bodenbinder wrote: I have 3 hard drive with 750 GB, 500 GB and 250 GB. I want to use btrfs as filesystem. This will be my first test installation of btrfs. My target is to get redundancy as well as a 750 GB data capacity. So I was thinking to create a raid0 with the 500 and 250 GB drive. This would result in a raid0 with 750 GB capacity. I want to add this raid0 as a mirror in a raid1 with the other 750 GB drive. But how do I do that? 250 GB HDD + 500 GB HDD can yield: - 250 GB RAID0, - 250 GB RAID1, or - 750 GB JBOD. To get 750 GB RAID0 or 750 GB RAID1, you need two 750 GB devices. As other posters have mentioned, btrfs is fairly new on Linux and Debian. I run Oldstable (Wheezy), and btrfs seems to mostly work as a file system (OOTB, btrfs root works, but GRUB can't read btrfs /boot). You may have to run a newer Debian and/or a newer kernel to get a btrfs with volume management and/or better Debian integration. LVM has been stable on Debian for many years. I used it in the past to create a 250 GB HDD + 250 GB HDD + 300 GB HDD => 800 GB JBOD, so LVM can certainly create a 250 GB HDD + 500 GB HDD => 750 GB JBOD. I assume LVM can also create a 750 GB HDD + 750 GB JBOD => 750 GB RAID1. The challenge would be setting everything up so that the 750 GB RAID1 comes up automagically at boot and tears down cleanly at shutdown. mdadm is another stable technology that could work in your case. David
Re: btrfs: mixing raid0 and raid1 - How?
Hello, On Sat, Apr 16, 2016 at 01:35:20PM +0200, Luis Felipe Tabera Alonso wrote: > Still btrfs is quite young, I am not sure if there are serious issues in > 3.17, > I would make some experiments before actual use. If you are going to use btrfs I would consider it essential to be subscribed to the linux-btrfs mailing list. You will also need to use a much newer kernel than 3.17, and you will need to commit to continuing to use newer kernels for some time yet. I mention this because you need to be aware that you may still hit issues where you need the help of linux-btrfs and that there is a need to use newer kernels than you will find packaged in Debian stable. I do myself run btrfs at home and I thought I had a stable combination of kernel version and userland tools (btrfs-tools), but when I came to need to replace a dead device I found that subsequent developments in btrfs meant I needed a newer btrfs-tools, and that in turn meant I needed a newer kernel. If the idea of having to upgrade kernel and some userland tools in order to recover from a simple situation of a dead device does not appeal to you then btrfs may not yet be for you. Things like this are why I do not yet run it in production. Cheers, Andy -- http://bitfolk.com/ -- No-nonsense VPS hosting
Re: btrfs: mixing raid0 and raid1 - How?
On sábado, 16 de abril de 2016 12:05:05 (CEST) Matthias Bodenbinder wrote: > 38# df -h /mnt/test/ > Dateisystem Größe Benutzt Verf. Verw% Eingehängt auf > /dev/sdg 699G 17M 466G 1% /mnt/test df is not reliable for btrfs raids, it is better to use btrfs fi df to check actual used space. Using btrfs-tools and kernel from sid (4.4): root@mychabol:/mnt/temporal# dd if=/dev/zero of=disk1 bs=1 count=0 seek=750M root@mychabol:/mnt/temporal# dd if=/dev/zero of=disk2 bs=1 count=0 seek=500M root@mychabol:/mnt/temporal# dd if=/dev/zero of=disk3 bs=1 count=0 seek=250M root@mychabol:/mnt/temporal# mkfs.btrfs -f -draid1 -mraid1 disk1 disk2 disk3 btrfs-progs v4.4 See http://btrfs.wiki.kernel.org for more information. Label: (null) UUID: 4fbfda48-3a10-4e1b-92d8-96be759c2a8c Node size: 16384 Sector size:4096 Filesystem size:1.46GiB Block group profiles: Data: RAID183.00MiB Metadata: RAID183.00MiB System: RAID112.00MiB SSD detected: no Incompat features: extref, skinny-metadata Number of devices: 3 Devices: IDSIZE PATH 1 750.00MiB disk1 2 500.00MiB disk2 3 250.00MiB disk3 root@mychabol:/mnt/temporal# losetup /dev/loop0 disk1 root@mychabol:/mnt/temporal# losetup /dev/loop1 disk2 root@mychabol:/mnt/temporal# losetup /dev/loop2 disk3 root@mychabol:/mnt/temporal# mkdir test root@mychabol:/mnt/temporal# mount /dev/loop0 test root@mychabol:/mnt/temporal# btrfs fi df test/ Data, RAID1: total=75.00MiB, used=0.00B System, RAID1: total=8.00MiB, used=16.00KiB Metadata, RAID1: total=75.00MiB, used=112.00KiB GlobalReserve, single: total=16.00MiB, used=0.00B root@mychabol:/mnt/temporal# df -h test/ S.ficheros Tamaño Usados Disp Uso% Montado en /dev/loop0 750M17M 491M 4% /mnt/temporal/test Like in your case, df thinks that there are only 491M usable. However, if I try to fill the volume root@mychabol:/mnt/temporal# cd test/ root@mychabol:/mnt/temporal/test# dd if=/dev/zero of=file.zero dd: escribiendo en 'file.zero': No queda espacio en el dispositivo 1345402+0 registros leídos 1345401+0 registros escritos 688845312 bytes (689 MB, 657 MiB) copied, 3,10508 s, 222 MB/s root@mychabol:/mnt/temporal/test# ls -lh total 657M -rw-r--r-- 1 root root 657M abr 16 13:24 file.zero root@mychabol:/mnt/temporal/test# df -h . S.ficheros Tamaño Usados Disp Uso% Montado en /dev/loop0 750M 674M 64K 100% /mnt/temporal/test With the 75M of metadata, you can check that this fills the three devices. Moreover root@mychabol:/mnt/temporal/test# rm file.zero root@mychabol:/mnt/temporal/test# df -h . S.ficheros Tamaño Usados Disp Uso% Montado en /dev/loop0 750M17M 657M 3% /mnt/temporal/test Try to do the same to see if df gets more accurate. Still btrfs is quite young, I am not sure if there are serious issues in 3.17, I would make some experiments before actual use. Bests Luis
Re: btrfs: mixing raid0 and raid1 - How?
And: 39# btrfs filesystem show Label: none uuid: 35db7af4-a116-45fa-8611-97aeb5d4a8d3 Total devices 3 FS bytes used 640.00KiB devid1 size 698.64GiB used 2.02GiB path /dev/sdg devid2 size 465.76GiB used 1.01GiB path /dev/sdh devid3 size 232.88GiB used 1.01GiB path /dev/sdi Am 16.04.2016 um 12:05 schrieb Matthias Bodenbinder: > Hello Luis, > > I followed that link which confirms that a btrfs raid1 out of these three > devices should give 750 GB of data capacity. But his is not what I get. > > This is what I have: > > Disk /dev/sdg: 698,7 GiB, 750156374016 bytes, 1465149168 sectors > Disk /dev/sdh: 465,8 GiB, 500107862016 bytes, 976773168 sectors > Disk /dev/sdi: 232,9 GiB, 250058268160 bytes, 488395055 sectors > > So it is actually not 750 GB but 700. Anyways... > > This is what I did: > > ### > 35# mkfs.btrfs -f -d raid1 -m raid1 /dev/sdg /dev/sdh /dev/sdi > Btrfs v3.17 > See http://btrfs.wiki.kernel.org for more information. > > Turning ON incompat feature 'extref': increased hardlink limit per file to > 65536 > adding device /dev/sdh id 2 > adding device /dev/sdi id 3 > fs created label (null) on /dev/sdg > nodesize 16384 leafsize 16384 sectorsize 4096 size 1.36TiB > > 36# mount /dev/sdg /mnt/test/ > > 37# btrfs fi df /mnt/test/ > Data, RAID1: total=1.00GiB, used=512.00KiB > Data, single: total=8.00MiB, used=0.00B > System, RAID1: total=8.00MiB, used=16.00KiB > System, single: total=4.00MiB, used=0.00B > Metadata, RAID1: total=1.00GiB, used=112.00KiB > Metadata, single: total=8.00MiB, used=0.00B > GlobalReserve, single: total=16.00MiB, used=0.00B > > 38# df -h /mnt/test/ > Dateisystem Größe Benutzt Verf. Verw% Eingehängt auf > /dev/sdg 699G 17M 466G 1% /mnt/test > > ### > > This leaves me with just 466 GB of free disc space. > Something is wrong. Either what I am doing is wrong or the web page is wrong. > > Matthias > > > Am 16.04.2016 um 10:20 schrieb Luis Felipe Tabera Alonso: >> On sábado, 16 de abril de 2016 9:00:00 (CEST) Matthias Bodenbinder wrote: >>> Hello, >>> >>> I have 3 hard drive with 750 GB, 500 GB and 250 GB. I want to use btrfs as >>> filesystem. This will be my first test installation of btrfs. >>> >>> My target is to get redundancy as well as a 750 GB data capacity. So I was >>> thinking to create a raid0 with the 500 and 250 GB drive. This would result >>> in a raid0 with 750 GB capacity. I want to add this raid0 as a mirror in a >>> raid1 with the other 750 GB drive. >>> >>> But how do I do that? >>> >>> Thanks >>> Matthias >> >> Disclaimer: I have necer made anything further than a btrfs raid1 with two >> equally sized devices. >> >> >> If you create a raid1 with the three devices you will get 750GB with >> redundancy. Current Btrfs raid1 means "data and metadata is copied twice in >> different devices". And btrfs is smart enough to not copy the data in the >> two >> smaller disks. I am not sure if this web page is accurate http:// >> carfax.org.uk/btrfs-usage/ but you can take a look. >> >> You can also create a single virtual device from the 250 and 500 devices >> using >> mdadm or lvm and then create a btrfs raid1 out of the 750GB device and the >> virtual device. >> >> With the "pure" btrfs setup, if you lose a single device you will be able to >> mount the device as "degraded". >> >> With the mixed setting, you can lose a single device and read the data. >> Moreover, you can lose the two small devices and still be able to mount the >> volume. >> >> luis >> >> >> > > >
Re: btrfs: mixing raid0 and raid1 - How?
Hello Luis, I followed that link which confirms that a btrfs raid1 out of these three devices should give 750 GB of data capacity. But his is not what I get. This is what I have: Disk /dev/sdg: 698,7 GiB, 750156374016 bytes, 1465149168 sectors Disk /dev/sdh: 465,8 GiB, 500107862016 bytes, 976773168 sectors Disk /dev/sdi: 232,9 GiB, 250058268160 bytes, 488395055 sectors So it is actually not 750 GB but 700. Anyways... This is what I did: ### 35# mkfs.btrfs -f -d raid1 -m raid1 /dev/sdg /dev/sdh /dev/sdi Btrfs v3.17 See http://btrfs.wiki.kernel.org for more information. Turning ON incompat feature 'extref': increased hardlink limit per file to 65536 adding device /dev/sdh id 2 adding device /dev/sdi id 3 fs created label (null) on /dev/sdg nodesize 16384 leafsize 16384 sectorsize 4096 size 1.36TiB 36# mount /dev/sdg /mnt/test/ 37# btrfs fi df /mnt/test/ Data, RAID1: total=1.00GiB, used=512.00KiB Data, single: total=8.00MiB, used=0.00B System, RAID1: total=8.00MiB, used=16.00KiB System, single: total=4.00MiB, used=0.00B Metadata, RAID1: total=1.00GiB, used=112.00KiB Metadata, single: total=8.00MiB, used=0.00B GlobalReserve, single: total=16.00MiB, used=0.00B 38# df -h /mnt/test/ Dateisystem Größe Benutzt Verf. Verw% Eingehängt auf /dev/sdg 699G 17M 466G 1% /mnt/test ### This leaves me with just 466 GB of free disc space. Something is wrong. Either what I am doing is wrong or the web page is wrong. Matthias Am 16.04.2016 um 10:20 schrieb Luis Felipe Tabera Alonso: > On sábado, 16 de abril de 2016 9:00:00 (CEST) Matthias Bodenbinder wrote: >> Hello, >> >> I have 3 hard drive with 750 GB, 500 GB and 250 GB. I want to use btrfs as >> filesystem. This will be my first test installation of btrfs. >> >> My target is to get redundancy as well as a 750 GB data capacity. So I was >> thinking to create a raid0 with the 500 and 250 GB drive. This would result >> in a raid0 with 750 GB capacity. I want to add this raid0 as a mirror in a >> raid1 with the other 750 GB drive. >> >> But how do I do that? >> >> Thanks >> Matthias > > Disclaimer: I have necer made anything further than a btrfs raid1 with two > equally sized devices. > > > If you create a raid1 with the three devices you will get 750GB with > redundancy. Current Btrfs raid1 means "data and metadata is copied twice in > different devices". And btrfs is smart enough to not copy the data in the two > smaller disks. I am not sure if this web page is accurate http:// > carfax.org.uk/btrfs-usage/ but you can take a look. > > You can also create a single virtual device from the 250 and 500 devices > using > mdadm or lvm and then create a btrfs raid1 out of the 750GB device and the > virtual device. > > With the "pure" btrfs setup, if you lose a single device you will be able to > mount the device as "degraded". > > With the mixed setting, you can lose a single device and read the data. > Moreover, you can lose the two small devices and still be able to mount the > volume. > > luis > > >
Re: btrfs: mixing raid0 and raid1 - How?
On sábado, 16 de abril de 2016 9:00:00 (CEST) Matthias Bodenbinder wrote: > Hello, > > I have 3 hard drive with 750 GB, 500 GB and 250 GB. I want to use btrfs as > filesystem. This will be my first test installation of btrfs. > > My target is to get redundancy as well as a 750 GB data capacity. So I was > thinking to create a raid0 with the 500 and 250 GB drive. This would result > in a raid0 with 750 GB capacity. I want to add this raid0 as a mirror in a > raid1 with the other 750 GB drive. > > But how do I do that? > > Thanks > Matthias Disclaimer: I have necer made anything further than a btrfs raid1 with two equally sized devices. If you create a raid1 with the three devices you will get 750GB with redundancy. Current Btrfs raid1 means "data and metadata is copied twice in different devices". And btrfs is smart enough to not copy the data in the two smaller disks. I am not sure if this web page is accurate http:// carfax.org.uk/btrfs-usage/ but you can take a look. You can also create a single virtual device from the 250 and 500 devices using mdadm or lvm and then create a btrfs raid1 out of the 750GB device and the virtual device. With the "pure" btrfs setup, if you lose a single device you will be able to mount the device as "degraded". With the mixed setting, you can lose a single device and read the data. Moreover, you can lose the two small devices and still be able to mount the volume. luis
btrfs: mixing raid0 and raid1 - How?
Hello, I have 3 hard drive with 750 GB, 500 GB and 250 GB. I want to use btrfs as filesystem. This will be my first test installation of btrfs. My target is to get redundancy as well as a 750 GB data capacity. So I was thinking to create a raid0 with the 500 and 250 GB drive. This would result in a raid0 with 750 GB capacity. I want to add this raid0 as a mirror in a raid1 with the other 750 GB drive. But how do I do that? Thanks Matthias
