Re: 800 GByte free, but no space left
Hallo, Hugo, Du meintest am 06.12.10: Kernel 2.6.37-rc4: # btrfs filesystem df /srv/MM Data, RAID0: total=2.39TB, used=2.37TB System, RAID1: total=8.00MB, used=188.00KB System: total=4.00MB, used=0.00 Metadata, RAID1: total=4.25GB, used=3.51GB Metadata, DUP: total=1.00GB, used=2.33MB Hope it helps! Yup. You've got what I've got(*). You have two different RAID types for metadata, which shouldn't happen (but does, due to a bug). Fear I right that balancing tries to reduce the system to something like RAID1? Viele Gruesse! Helmut -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: 800 GByte free, but no space left
Helmut - On Tue, Dec 07, 2010 at 06:05:00PM +0100, Helmut Hullen wrote: Du meintest am 06.12.10: Kernel 2.6.37-rc4: # btrfs filesystem df /srv/MM Data, RAID0: total=2.39TB, used=2.37TB System, RAID1: total=8.00MB, used=188.00KB System: total=4.00MB, used=0.00 Metadata, RAID1: total=4.25GB, used=3.51GB Metadata, DUP: total=1.00GB, used=2.33MB Hope it helps! Yup. You've got what I've got(*). You have two different RAID types for metadata, which shouldn't happen (but does, due to a bug). Fear I right that balancing tries to reduce the system to something like RAID1? It _should_ move all the data on the disk to somewhere else on the disk, whilst honouring the RAID settings for the filesystem. However, since it's got buggered up RAID settings now and has been using some of the space for the wrong RAID type, the balance can't find space with the right RAID parameters to write to, so it runs out of space. (I think I got that right, anyway. I'm working off a conversation with Chris on IRC some weeks ago, about what happened to my filesystem). Hugo. -- === Hugo Mills: h...@... carfax.org.uk | darksatanic.net | lug.org.uk === PGP key: 515C238D from wwwkeys.eu.pgp.net or http://www.carfax.org.uk --- Always be sincere, whether you mean it or not. --- signature.asc Description: Digital signature
Re: 800 GByte free, but no space left
Hallo, Hugo, Du meintest am 06.12.10: But after copying about 300 MByte (part of a 1.5-GByte *.mpg) I got no space left on device. Looks like balancing has stolen about 300 GByte. This sounds exactly like a problem I've had. What output do you get from btrfs fi df /srv/MM? I've just written a script for gathering the (perhaps) interesting data ... # btrfs filesystem show Label: 'MM2' uuid: ad7c0668-316c-4a79-ba00-3b505b9d99b4 Total devices 2 FS bytes used 2.37TB devid2 size 1.35TB used 1.20TB path /dev/sdc3 devid1 size 1.81TB used 1.20TB path /dev/sdf2 Btrfs Btrfs v0.19 # btrfs filesystem df /srv/MM Data: total=2.39TB, used=2.37TB Metadata: total=5.25GB, used=3.51GB System: total=12.00MB, used=188.00KB Can you try that again with either the latest 2.6.37-rc, or with the btrfs-unstable kernel? There's a bug in earlier versions that breaks the reporting of RAID types, which is what I wanted to see here. Do you mean git://git.kernel.org/pub/scm/linux/kernel/git/mason/btrfs-unstable.git as btrfs-unstable kernel? Compiling 2.6.37-rc is no big problem, it only needs som time. Just now I'm using Kernel 2.6.35.8 btrfs-git from 20101117 I've moved about 50 Gbyte away from srv/MM in the meantime, before running the script with this output. And I don't dare running balance again - maybe it reduces the available space again and again. If you've hit the bug I think you have, then yes, it will. Hmm - it can't get worse ... If the error is related to the kernel or to the btrfs version and I try a newer one: can that lead to more free space? Viele Gruesse! Helmut -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: 800 GByte free, but no space left
On Mon, Dec 06, 2010 at 02:13:00PM +0100, Helmut Hullen wrote: Hallo, Hugo, Du meintest am 06.12.10: But after copying about 300 MByte (part of a 1.5-GByte *.mpg) I got no space left on device. Looks like balancing has stolen about 300 GByte. This sounds exactly like a problem I've had. What output do you get from btrfs fi df /srv/MM? I've just written a script for gathering the (perhaps) interesting data ... # btrfs filesystem show Label: 'MM2' uuid: ad7c0668-316c-4a79-ba00-3b505b9d99b4 Total devices 2 FS bytes used 2.37TB devid2 size 1.35TB used 1.20TB path /dev/sdc3 devid1 size 1.81TB used 1.20TB path /dev/sdf2 Btrfs Btrfs v0.19 # btrfs filesystem df /srv/MM Data: total=2.39TB, used=2.37TB Metadata: total=5.25GB, used=3.51GB System: total=12.00MB, used=188.00KB Can you try that again with either the latest 2.6.37-rc, or with the btrfs-unstable kernel? There's a bug in earlier versions that breaks the reporting of RAID types, which is what I wanted to see here. Do you mean git://git.kernel.org/pub/scm/linux/kernel/git/mason/btrfs-unstable.git as btrfs-unstable kernel? Yes. It's 2.6.36, plus the patches that Chris has sent to Linus for inclusion into 2.6.37. Compiling 2.6.37-rc is no big problem, it only needs som time. Just now I'm using Kernel 2.6.35.8 btrfs-git from 20101117 I've moved about 50 Gbyte away from srv/MM in the meantime, before running the script with this output. And I don't dare running balance again - maybe it reduces the available space again and again. If you've hit the bug I think you have, then yes, it will. Hmm - it can't get worse ... If the error is related to the kernel or to the btrfs version and I try a newer one: can that lead to more free space? Not yet. I've taken the whole of December off work (using up my leave allocation for last year), and my plan is to get myself to the point where I can understand enough of the code to fix this particular problem. Hugo. -- === Hugo Mills: h...@... carfax.org.uk | darksatanic.net | lug.org.uk === PGP key: 515C238D from wwwkeys.eu.pgp.net or http://www.carfax.org.uk --- What are we going to do tonight? The same thing we do --- every night, Pinky. Try to take over the world! signature.asc Description: Digital signature
Re: 800 GByte free, but no space left
Hallo, Hugo, Du meintest am 06.12.10: Can you try that again with either the latest 2.6.37-rc, or with the btrfs-unstable kernel? There's a bug in earlier versions that breaks the reporting of RAID types, which is what I wanted to see here. Compiling 2.6.37-rc is no big problem, it only needs som time. Compiling runs .. And I don't dare running balance again - maybe it reduces the available space again and again. If you've hit the bug I think you have, then yes, it will. Hmm - it can't get worse ... If the error is related to the kernel or to the btrfs version and I try a newer one: can that lead to more free space? Not yet. I've taken the whole of December off work (using up my leave allocation for last year), and my plan is to get myself to the point where I can understand enough of the code to fix this particular problem. I love such vacancies planning ... If/when I install 2.6.37-rc4: should I update btrfs (from the 20101117 version)? How can I see that changing the kernel makes things better? It's more and more difficult to externalize (?) btrfs directories to other disks ... Viele Gruesse! Helmut -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: 800 GByte free, but no space left
Helmut - On Mon, Dec 06, 2010 at 03:45:00PM +0100, Helmut Hullen wrote: If/when I install 2.6.37-rc4: should I update btrfs (from the 20101117 version)? I think that's the latest version. How can I see that changing the kernel makes things better? It's more and more difficult to externalize (?) btrfs directories to other disks ... Updating the kernel won't fix the problem I'm thinking of (sorry). It will, however, fix the bug that stops the btrfs tool from reporting what RAID levels you've got. The problem I suspect you may have (because your symptoms seem to be the same as mine) is that there are some circumstances where the filesystem can change RAID levels pretty much arbitrarily. Running btrfs fi df with a kernel that reports RAID levels will show whether that's the case, as you'll have more than one RAID level listed. Hugo. -- === Hugo Mills: h...@... carfax.org.uk | darksatanic.net | lug.org.uk === PGP key: 515C238D from wwwkeys.eu.pgp.net or http://www.carfax.org.uk --- But people have always eaten people, / what else is there to --- eat? / If the Juju had meant us not to eat people / he wouldn't have made us of meat. signature.asc Description: Digital signature
Re: 800 GByte free, but no space left
Hallo, Hugo, Du meintest am 06.12.10: How can I see that changing the kernel makes things better? It's more and more difficult to externalize (?) btrfs directories to other disks ... Updating the kernel won't fix the problem I'm thinking of (sorry). It will, however, fix the bug that stops the btrfs tool from reporting what RAID levels you've got. The problem I suspect you may have (because your symptoms seem to be the same as mine) is that there are some circumstances where the filesystem can change RAID levels pretty much arbitrarily. Running btrfs fi df with a kernel that reports RAID levels will show whether that's the case, as you'll have more than one RAID level listed. Kernel 2.6.35.8: # btrfs filesystem df /srv/MM Data: total=2.39TB, used=2.37TB Metadata: total=5.25GB, used=3.51GB System: total=12.00MB, used=188.00KB Kernel 2.6.37-rc4: # btrfs filesystem df /srv/MM Data, RAID0: total=2.39TB, used=2.37TB System, RAID1: total=8.00MB, used=188.00KB System: total=4.00MB, used=0.00 Metadata, RAID1: total=4.25GB, used=3.51GB Metadata, DUP: total=1.00GB, used=2.33MB Hope it helps! Viele Gruesse! Helmut -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: 800 GByte free, but no space left
On Mon, Dec 06, 2010 at 06:13:00PM +0100, Helmut Hullen wrote: Hallo, Hugo, Du meintest am 06.12.10: How can I see that changing the kernel makes things better? It's more and more difficult to externalize (?) btrfs directories to other disks ... Updating the kernel won't fix the problem I'm thinking of (sorry). It will, however, fix the bug that stops the btrfs tool from reporting what RAID levels you've got. The problem I suspect you may have (because your symptoms seem to be the same as mine) is that there are some circumstances where the filesystem can change RAID levels pretty much arbitrarily. Running btrfs fi df with a kernel that reports RAID levels will show whether that's the case, as you'll have more than one RAID level listed. Kernel 2.6.37-rc4: # btrfs filesystem df /srv/MM Data, RAID0: total=2.39TB, used=2.37TB System, RAID1: total=8.00MB, used=188.00KB System: total=4.00MB, used=0.00 Metadata, RAID1: total=4.25GB, used=3.51GB Metadata, DUP: total=1.00GB, used=2.33MB Hope it helps! Yup. You've got what I've got(*). You have two different RAID types for metadata, which shouldn't happen (but does, due to a bug). Hugo. (*) My .sig fairy is clearly working overtime for appropriate quotations. :) -- === Hugo Mills: h...@... carfax.org.uk | darksatanic.net | lug.org.uk === PGP key: 515C238D from wwwkeys.eu.pgp.net or http://www.carfax.org.uk --- Charting the inexorable advance of Western syphilisation... --- signature.asc Description: Digital signature
Re: 800 GByte free, but no space left
On Sun, Dec 5, 2010 at 1:48 AM, Helmut Hullen hul...@t-online.de wrote: Hallo, Evert, Du meintest am 04.12.10 zum Thema Re: 800 GByte free, but no space left: On Sat, Dec 4, 2010 at 10:17 AM, Helmut Hullen hul...@t-online.de wrote: Hallo, I wrote am 02.12.10: I use 2 disks (1.5 Tbyte and 2.0 TByte) under 1 LABEL (for my video collection, nearly alle files have more than 1 GByte): Label: MM2 uuid: ad7c0668-316c-4a79-ba00-3b505b9d99b4 Total devices 2 FS bytes used 2.38TB devid 2 size 1.35TB used 1.35TB path /dev/sdc3 devid 1 size 1.81TB used 1.35TB path /dev/sdf2 (btrfs-show uses TiByte, it's 10% less than TByte) Btrfs Btrfs v0.19 Filesystem 1K-blocks Used Available Use% Mounted on /dev/sdc3 3400799848 2559596740 841203108 76% /srv/MM When I add some more videos, writing gets slower and slower, and then the system refuses with no space left ... [...] No help? The weekend isn't the best time for I am not an expert on this by a long shot, but it looks like you added these two disks in raid0. This means that the total space cannot exceed the space of the smallest disk. ie: 1.35TB is the max you can use on any of your disks, as that is the size of the smallest disk. In other words, once any of the disks in a btrfs array runs out of space, the whole array is out of space. I don't know if this is intended, but it certainly would appear so. I won't hope that this error is related to RAID0, I haven't installed (as far as I know) RAID0. My installation way: (2-TByte-Disk) mkfs.btrfs /dev/sdf2 mount /dev/sdf2 /srv/MM (1.5-TByte-Disk) btrfs device add /dev/sdc3 /srv/MM btrfs filesystem balance /srv/MM (and then waiting about 1 day ...) Especially: no RAID definition. If the smallest device defines the capacity then I should use 2*1.35 TiByte, but my system tells no space left at about 2.4 TiByte - where are (at least) 300 GiByte hidden? --- Kernel 2.6.35.8 btrfs-git from 20101117 Viele Gruesse! Helmut -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html If it's not a raid1, and there's multiple devices, it's a raid0 (and so available space is the sum of all drives). Your problem however is that metadata is raid1 by default (where everything is duplicated on separate drives). One device has no space unallocated to any block groups, and so if a particular metadata block group is full, there's no place for that device's copy to end up, hence ENOSPC. Adding another device will probably work around this, as will simply running a balance operation (possibly, and you may need to free up some space first anyway). -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: 800 GByte free, but no space left
Hallo, cwillu, Du meintest am 05.12.10: I am not an expert on this by a long shot, but it looks like you added these two disks in raid0. I won't hope that this error is related to RAID0, I haven't installed (as far as I know) RAID0. My installation way: (2-TByte-Disk) mkfs.btrfs /dev/sdf2 mount /dev/sdf2 /srv/MM (1.5-TByte-Disk) btrfs device add /dev/sdc3 /srv/MM btrfs filesystem balance /srv/MM (and then waiting about 1 day ...) Especially: no RAID definition. [...] If it's not a raid1, and there's multiple devices, it's a raid0 (and so available space is the sum of all drives). Your problem however is that metadata is raid1 by default (where everything is duplicated on separate drives). Maybe you're right. But if you're right then I have got the worst of two worlds. I don't want neither RAID0 nor RAID1, I want a bundle of different disks (at least partititions) which seem to be one large disk. And I've hoped btrfs does this job. Adding another device will probably work around this, as will simply running a balance operation (possibly, and you may need to free up some space first anyway). That could lead to the following steps: Buy a 3 GByte disk btrfs device add /dev/sdxy /srv/MM btrfs filesystem balance /srv/MM 1.5 TByte disk: btrfs device delete /dev/sdc3 /srv/MM btrfs filesystem balance /srv/MM and then disconnect the 1.5 TByte disk (and hope that now the 2 TByte disk sets the limits). No nice way ... -- Is there a way to avoid this (presumably) RAID mismatch? By the way: working with TByte disks includes (for home users) that there's no backup ... Viele Gruesse! Helmut -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: 800 GByte free, but no space left
On Sun, Dec 5, 2010 at 3:51 AM, Helmut Hullen hul...@t-online.de wrote: Hallo, cwillu, Du meintest am 05.12.10: I am not an expert on this by a long shot, but it looks like you added these two disks in raid0. I won't hope that this error is related to RAID0, I haven't installed (as far as I know) RAID0. My installation way: (2-TByte-Disk) mkfs.btrfs /dev/sdf2 mount /dev/sdf2 /srv/MM (1.5-TByte-Disk) btrfs device add /dev/sdc3 /srv/MM btrfs filesystem balance /srv/MM (and then waiting about 1 day ...) Especially: no RAID definition. [...] If it's not a raid1, and there's multiple devices, it's a raid0 (and so available space is the sum of all drives). Your problem however is that metadata is raid1 by default (where everything is duplicated on separate drives). Maybe you're right. But if you're right then I have got the worst of two worlds. I don't want neither RAID0 nor RAID1, I want a bundle of different disks (at least partititions) which seem to be one large disk. And I've hoped btrfs does this job. That's what raid0 is, and it's actually the best of both worlds: your metadata (which will be less than 5% of the data) is safely duplicated, such that you always have the checksums even with a disk gone, so you can verify that the data that you still have is good, while not wasting space duplicating every little bit of file data, which you may not care about that much, and which you have backed up anyway (right? right?). Larger files will be striped though, which is more incidental to how data is allocated to block groups, but even so, it's generally not as simple as saying I don't want raid0, I just want a single big disk; you have to figure out how to do that, and it'll generally result in some raid0-like properties. This will almost certainly become much more tunable in the future, but not every feature that people want is done yet. In fact, most of the really cool user-visible features aren't done yet. Btrfs is still pretty young. Adding another device will probably work around this, as will simply running a balance operation (possibly, and you may need to free up some space first anyway). That could lead to the following steps: Buy a 3 GByte disk btrfs device add /dev/sdxy /srv/MM btrfs filesystem balance /srv/MM 1.5 TByte disk: btrfs device delete /dev/sdc3 /srv/MM btrfs filesystem balance /srv/MM and then disconnect the 1.5 TByte disk (and hope that now the 2 TByte disk sets the limits). No nice way ... No, just run the balance without adding another disk. That will probably work (although it _will_ take a while on a large filesystem). I'm not sure that you understand how this all works though; you might want to re-read the wiki articles (which I believe have been freshened up recently). Is there a way to avoid this (presumably) RAID mismatch? Yes, you can specify the raid level for each when you make the filesystem (and will eventually be able to do it with existing filesystems). However, as I described above, you really want metadata to be duplicated. Your problem is more of an unfortunate case of everything not being tuned quite right yet. By the way: working with TByte disks includes (for home users) that there's no backup ... Not sure why you'd think that. It can't be the bandwidth, and if you can't afford a second drive, there's a good case to be made that you can't afford the data you can't afford to lose. -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: 800 GByte free, but no space left
On Sun, Dec 5, 2010 at 12:48 AM, Helmut Hullen hul...@t-online.de wrote: Hallo, Evert, Du meintest am 04.12.10 zum Thema Re: 800 GByte free, but no space left: On Sat, Dec 4, 2010 at 10:17 AM, Helmut Hullen hul...@t-online.de wrote: Hallo, I wrote am 02.12.10: I use 2 disks (1.5 Tbyte and 2.0 TByte) under 1 LABEL (for my video collection, nearly alle files have more than 1 GByte): Label: MM2 uuid: ad7c0668-316c-4a79-ba00-3b505b9d99b4 Total devices 2 FS bytes used 2.38TB devid 2 size 1.35TB used 1.35TB path /dev/sdc3 devid 1 size 1.81TB used 1.35TB path /dev/sdf2 (btrfs-show uses TiByte, it's 10% less than TByte) Btrfs Btrfs v0.19 Filesystem 1K-blocks Used Available Use% Mounted on /dev/sdc3 3400799848 2559596740 841203108 76% /srv/MM When I add some more videos, writing gets slower and slower, and then the system refuses with no space left ... [...] No help? I am not an expert on this by a long shot, but it looks like you added these two disks in raid0. This means that the total space cannot exceed the space of the smallest disk. ie: 1.35TB is the max you can use on any of your disks, as that is the size of the smallest disk. In other words, once any of the disks in a btrfs array runs out of space, the whole array is out of space. I don't know if this is intended, but it certainly would appear so. I won't hope that this error is related to RAID0, I haven't installed (as far as I know) RAID0. My installation way: (2-TByte-Disk) mkfs.btrfs /dev/sdf2 mount /dev/sdf2 /srv/MM (1.5-TByte-Disk) btrfs device add /dev/sdc3 /srv/MM btrfs filesystem balance /srv/MM (and then waiting about 1 day ...) Especially: no RAID definition. If the smallest device defines the capacity then I should use 2*1.35 TiByte, but my system tells no space left at about 2.4 TiByte - where are (at least) 300 GiByte hidden? devid2 size 1.35TB used 1.35TB path /dev/sdc3 devid1 size 1.81TB used 1.35TB path /dev/sdf2 Here devid 2 is at 100%, and hence you are getting the no more space left errors. So, the 300 TB is on the bigger disk, and not usable for you right now. I know of the disk mode you speak.. an old raid card of mine called it Just a bunch of disks and it literally filled up the first disk before carrying on to the second one until that was full under windows... under UNIX it had the effect of just adding all the sectors to each other, and stretching the file system over the disks in a linear fashion. Most UNIX file systems writes files in the middle of the largest contiguous free space, which meant that some files got written on the first disk, and some on the second. As far as I know, btrfs does not support this raid mode. Another thing to keep in mind is that as far as I know you cannot remove devid 1 from a btrfs volume. This is due to be fixed, but I have no idea on the status of that. Lastly, external USB disks are not too expensive, and together with rsync makes a good off-line backup solution. You could, if you really wanted to use all of two differently sized disks in a btrfs, subdivide the disks in equal sized partitions, and just put all of those partitions in a btrfs raid0... ie: Say you have a 1TB disk and 2TB disk... make 1TB partition on first disk, and two 1TB partitions on second disk, then add all three partitions to btrfs raid0 to make one volume of 3TB which will be fully usable. In your case: 1.81 Tb and 1.3 Tb your best use of space would probably be 0.6Tb partitions.ie: 3x0.6Tb partitions on the first disk, and 2x0.6TB partitions on the second. Then add all five partitions to a btrfs raid. This would leave 0.1Tb of wasted space on the smaller device, but at least you can use this partition separately. Kind regards, -Evert Vorster- -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: 800 GByte free, but no space left
On Sun, Dec 05, 2010 at 04:08:26AM -0700, Evert Vorster wrote: On Sun, Dec 5, 2010 at 12:48 AM, Helmut Hullen hul...@t-online.de wrote: Hallo, Evert, Du meintest am 04.12.10 zum Thema Re: 800 GByte free, but no space left: I am not an expert on this by a long shot, but it looks like you added these two disks in raid0. Nope -- btrfs will spread out its allocations across both disks. This means that the total space cannot exceed the space of the smallest disk. [...] Especially: no RAID definition. If the smallest device defines the capacity then I should use 2*1.35 TiByte, but my system tells no space left at about 2.4 TiByte - where are (at least) 300 GiByte hidden? devid2 size 1.35TB used 1.35TB path /dev/sdc3 devid1 size 1.81TB used 1.35TB path /dev/sdf2 Here devid 2 is at 100%, and hence you are getting the no more space left errors. So, the 300 TB is on the bigger disk, and not usable for you right now. I _think_ that a balance is all that's needed at this point. It can't hurt, anyway (other than taking quite a long time). I know of the disk mode you speak.. an old raid card of mine called it Just a bunch of disks and it literally filled up the first disk before carrying on to the second one until that was full under windows... under UNIX it had the effect of just adding all the sectors to each other, and stretching the file system over the disks in a linear fashion. Most UNIX file systems writes files in the middle of the largest contiguous free space, which meant that some files got written on the first disk, and some on the second. As far as I know, btrfs does not support this raid mode. It does support it: that's what the single RAID profile in mkfs.btrfs is. It attempts to use the disk space marginally more intelligently than traditional linear mode, though, as it allocates block groups (in chunks of about 1G) to each disk in turn. This isn't the same as RAID-0, which stripes within block groups with a much smaller stripe size. Another thing to keep in mind is that as far as I know you cannot remove devid 1 from a btrfs volume. This is due to be fixed, but I have no idea on the status of that. I've done it (I have a filesystem with IDs 7, 8, 9, 12, 13, 14). Looks like that particular problem has been fixed. You could, if you really wanted to use all of two differently sized disks in a btrfs, subdivide the disks in equal sized partitions, and just put all of those partitions in a btrfs raid0... [...] That would be a really bad idea, as your disks would thrash horribly, reading stripes from different locations on the disk. Hugo. -- === Hugo Mills: h...@... carfax.org.uk | darksatanic.net | lug.org.uk === PGP key: 515C238D from wwwkeys.eu.pgp.net or http://www.carfax.org.uk --- Nostalgia isn't what it used to be. --- signature.asc Description: Digital signature
Re: 800 GByte free, but no space left
Hallo, cwillu, Du meintest am 05.12.10: Maybe you're right. But if you're right then I have got the worst of two worlds. I don't want neither RAID0 nor RAID1, I want a bundle of different disks (at least partititions) which seem to be one large disk. And I've hoped btrfs does this job. That's what raid0 is, and it's actually the best of both worlds: your metadata (which will be less than 5% of the data) is safely duplicated, such that you always have the checksums even with a disk gone, so you can verify that the data that you still have is good, while not wasting space duplicating every little bit of file data, which you may not care about that much, and which you have backed up anyway (right? right?). Is it really RAID0, or is the btrfs way only similar to RAID0? I don't like RAID0 because I never now on which physical disk the files are. That makes changing disks very risky. [...] This will almost certainly become much more tunable in the future, but not every feature that people want is done yet. In fact, most of the really cool user-visible features aren't done yet. Btrfs is still pretty young. But I still hope btrfs is smarter than RAID0 or LVM ... [...] 1.5 TByte disk: btrfs device delete /dev/sdc3 /srv/MM btrfs filesystem balance /srv/MM and then disconnect the 1.5 TByte disk (and hope that now the 2 TByte disk sets the limits). No nice way ... No, just run the balance without adding another disk. That will probably work (although it _will_ take a while on a large filesystem). I'll try - perhaps it helps for some (few) weeks. I'm not sure that you understand how this all works though; you might want to re-read the wiki articles (which I believe have been freshened up recently). Beg your pardon - my major interest is that the system works. I'm glad when I believe to understand what happens, but this feeling is an add- on, no must. Is there a way to avoid this (presumably) RAID mismatch? Yes, you can specify the raid level for each when you make the filesystem (and will eventually be able to do it with existing filesystems). However, as I described above, you really want metadata to be duplicated. Your problem is more of an unfortunate case of everything not being tuned quite right yet. May be - I thought avoiding the RAID0 definition was a good idea. By the way: working with TByte disks includes (for home users) that there's no backup ... Not sure why you'd think that. It can't be the bandwidth, and if you can't afford a second drive, there's a good case to be made that you can't afford the data you can't afford to lose. The data isn't really valuable - DVB videos. Most of them are copied to disks in a machine about 250 km away. And the TV stations repeat them on and on. Viele Gruesse! Helmut -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: 800 GByte free, but no space left
Hallo, Hugo, Du meintest am 05.12.10: If the smallest device defines the capacity then I should use 2*1.35 TiByte, but my system tells no space left at about 2.4 TiByte - where are (at least) 300 GiByte hidden? devid2 size 1.35TB used 1.35TB path /dev/sdc3 devid1 size 1.81TB used 1.35TB path /dev/sdf2 Here devid 2 is at 100%, and hence you are getting the no more space left errors. So, the 300 TB is on the bigger disk, and not usable for you right now. I _think_ that a balance is all that's needed at this point. It can't hurt, anyway (other than taking quite a long time). It's just running - tomorrow I'll know more. grep relocating /var/log/messages (counting down the groups) tells the proceedings. Viele Gruesse! Helmut -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: 800 GByte free, but no space left
devid 2 size 1.35TB used 1.35TB path /dev/sdc3 devid 1 size 1.81TB used 1.35TB path /dev/sdf2 Here devid 2 is at 100%, and hence you are getting the no more space left errors. So, the 300 TB is on the bigger disk, and not usable for you right now. I _think_ that a balance is all that's needed at this point. It can't hurt, anyway (other than taking quite a long time). I am under the impression that as soon as one of the devices in a btrfs is out of space, and write to that file system returns ENOSPACE This certainly does look like it from this example, and from what I have read elsewhere. Also, I believe that the balance operation tries to equalize the amount of bytes used on each underlying device, rather than try to equalize the percentage of underlying devices filled. With exactly the same amount used on both disks, I doubt that there would be any advantage to a balance operation. I know of the disk mode you speak.. an old raid card of mine called it Just a bunch of disks and it literally filled up the first disk before carrying on to the second one until that was full under windows... under UNIX it had the effect of just adding all the sectors to each other, and stretching the file system over the disks in a linear fashion. Most UNIX file systems writes files in the middle of the largest contiguous free space, which meant that some files got written on the first disk, and some on the second. As far as I know, btrfs does not support this raid mode. It does support it: that's what the single RAID profile in mkfs.btrfs is. It attempts to use the disk space marginally more intelligently than traditional linear mode, though, as it allocates block groups (in chunks of about 1G) to each disk in turn. This isn't the same as RAID-0, which stripes within block groups with a much smaller stripe size. I stand corrected. Which brings us to the question... if you don't specify the raid level when creating the btrfs, what is the default? raid0? single? Another thing to keep in mind is that as far as I know you cannot remove devid 1 from a btrfs volume. This is due to be fixed, but I have no idea on the status of that. I've done it (I have a filesystem with IDs 7, 8, 9, 12, 13, 14). Looks like that particular problem has been fixed. Good to know. I'm outgrowing one of my filesystems as well... and would like to be able to migrate to bigger disks without rebuilding the btrfs from scratch. You could, if you really wanted to use all of two differently sized disks in a btrfs, subdivide the disks in equal sized partitions, and just put all of those partitions in a btrfs raid0... [...] That would be a really bad idea, as your disks would thrash horribly, reading stripes from different locations on the disk. True, it would be slower than a normal raid0... in single mode there should be almost no thrashing, though. This does warrant some testing, though. I'll see what I can do testing wise. Unfortunately, I'm no good at coding so the best I can do is test and report results... :( -Evert Vorster- -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: 800 GByte free, but no space left
Hallo, Evert, Du meintest am 05.12.10: devid 2 size 1.35TB used 1.35TB path /dev/sdc3 devid 1 size 1.81TB used 1.35TB path /dev/sdf2 [...] When I created a file system with mkfs.btrfs -d single /dev/sdb6 /dev/sdb7 I was able to fill the resulting file system to 2.7Gb before it started throwing ENOSPACE this file system was quite a bit faster... So, by far the simplest solution would be to re-create your file system with single mode, Can I add or delete hard disks/partitions to the two devices/partitions in your example? I've studied the man page and the Wiki but didn't find any help. And in my special case I have to add at least yearly new disks and from time to time remove the smallest disks from this bundle. Viele Gruesse! Helmut -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: 800 GByte free, but no space left
Hallo, I wrote am 02.12.10: I use 2 disks (1.5 Tbyte and 2.0 TByte) under 1 LABEL (for my video collection, nearly alle files have more than 1 GByte): Label: MM2 uuid: ad7c0668-316c-4a79-ba00-3b505b9d99b4 Total devices 2 FS bytes used 2.38TB devid2 size 1.35TB used 1.35TB path /dev/sdc3 devid1 size 1.81TB used 1.35TB path /dev/sdf2 (btrfs-show uses TiByte, it's 10% less than TByte) Btrfs Btrfs v0.19 Filesystem 1K-blocks Used Available Use% Mounted on /dev/sdc33400799848 2559596740 841203108 76% /srv/MM When I add some more videos, writing gets slower and slower, and then the system refuses with no space left ... [...] No help? Viele Gruesse! Helmut -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: 800 GByte free, but no space left
Hallo, Evert, Du meintest am 04.12.10 zum Thema Re: 800 GByte free, but no space left: On Sat, Dec 4, 2010 at 10:17 AM, Helmut Hullen hul...@t-online.de wrote: Hallo, I wrote am 02.12.10: I use 2 disks (1.5 Tbyte and 2.0 TByte) under 1 LABEL (for my video collection, nearly alle files have more than 1 GByte): Label: MM2 uuid: ad7c0668-316c-4a79-ba00-3b505b9d99b4 Total devices 2 FS bytes used 2.38TB devid 2 size 1.35TB used 1.35TB path /dev/sdc3 devid 1 size 1.81TB used 1.35TB path /dev/sdf2 (btrfs-show uses TiByte, it's 10% less than TByte) Btrfs Btrfs v0.19 Filesystem 1K-blocks Used Available Use% Mounted on /dev/sdc3 3400799848 2559596740 841203108 76% /srv/MM When I add some more videos, writing gets slower and slower, and then the system refuses with no space left ... [...] No help? I am not an expert on this by a long shot, but it looks like you added these two disks in raid0. This means that the total space cannot exceed the space of the smallest disk. ie: 1.35TB is the max you can use on any of your disks, as that is the size of the smallest disk. In other words, once any of the disks in a btrfs array runs out of space, the whole array is out of space. I don't know if this is intended, but it certainly would appear so. I won't hope that this error is related to RAID0, I haven't installed (as far as I know) RAID0. My installation way: (2-TByte-Disk) mkfs.btrfs /dev/sdf2 mount /dev/sdf2 /srv/MM (1.5-TByte-Disk) btrfs device add /dev/sdc3 /srv/MM btrfs filesystem balance /srv/MM (and then waiting about 1 day ...) Especially: no RAID definition. If the smallest device defines the capacity then I should use 2*1.35 TiByte, but my system tells no space left at about 2.4 TiByte - where are (at least) 300 GiByte hidden? --- Kernel 2.6.35.8 btrfs-git from 20101117 Viele Gruesse! Helmut -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
800 GByte free, but no space left
Hallo, I've new problems. I use 2 disks (1.5 Tbyte and 2.0 TByte) under 1 LABEL (for my video collection, nearly alle files have more than 1 GByte): Label: MM2 uuid: ad7c0668-316c-4a79-ba00-3b505b9d99b4 Total devices 2 FS bytes used 2.38TB devid2 size 1.35TB used 1.35TB path /dev/sdc3 devid1 size 1.81TB used 1.35TB path /dev/sdf2 (btrfs-show uses TiByte, it's 10% less than TByte) Btrfs Btrfs v0.19 Filesystem 1K-blocks Used Available Use% Mounted on /dev/sdc33400799848 2559596740 841203108 76% /srv/MM When I add some more videos, writing gets slower and slower, and then the system refuses with no space left ... Label: MM2 uuid: ad7c0668-316c-4a79-ba00-3b505b9d99b4 Total devices 2 FS bytes used 2.40TB devid2 size 1.35TB used 1.35TB path /dev/sdc3 devid1 size 1.81TB used 1.35TB path /dev/sdf2 Btrfs Btrfs v0.19 Filesystem 1K-blocks Used Available Use% Mounted on /dev/sdc33400799848 2585340332 815459516 77% /srv/MM - When I try to rename files I get no space left When I delete some files and then try again to rename the system doesn't really rename but first copies to the new name and then deletes the old file. Same behaviour when I try to move a file from one directory to another. -- Where is the bottleneck? Viele Gruesse! Helmut -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: 800 GByte free, but no space left
On Thu, Dec 2, 2010 at 10:23 AM, Helmut Hullen hul...@t-online.de wrote: Btrfs Btrfs v0.19 btrfs in the kernel has been version 0.19 for a *long* time. The version number there may never change. How do you encode a feature mask in a version number? Some features may be in one tree but not upstreamed all together and other such minutiae. What you need to do is use a more recent kernel than 2.6.32 if you want to use btrfs (modulo backports, but let's not talk about that right now). So if you're using a kernel older than 2.6.36, then you should probably upgrade. -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: 800 GByte free, but no space left
Hallo, Mike, Du meintest am 02.12.10: Btrfs Btrfs v0.19 btrfs in the kernel has been version 0.19 for a *long* time. The version number there may never change. How do you encode a feature mask in a version number? Some features may be in one tree but not upstreamed all together and other such minutiae. Sorry - I forgot: Kernel 2.6.35.8 btrfs-git from 20101117 Viele Gruesse! Helmut -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
