Re: Provide a better free space estimate on RAID1

2014-02-10 Thread Roman Mamedov
On Mon, 10 Feb 2014 00:02:38 + (UTC) Duncan 1i5t5.dun...@cox.net wrote: Meanwhile, you said it yourself, users aren't normally concerned about this. I think you're being mistaken here, the point that users aren't looking at the free space, hence it is not important to provide a correct

Re: Provide a better free space estimate on RAID1

2014-02-09 Thread Roman Mamedov
On Sun, 9 Feb 2014 06:38:53 + (UTC) Duncan 1i5t5.dun...@cox.net wrote: RAID or multi-device filesystems aren't 1970s features and break 1970s behavior and the assumptions associated with it. If you're not prepared to deal with those broken assumptions, don't. Use mdraid or dmraid or

Re: Provide a better free space estimate on RAID1

2014-02-09 Thread Kai Krakow
Duncan 1i5t5.dun...@cox.net schrieb: Roman Mamedov posted on Sun, 09 Feb 2014 04:10:50 +0600 as excerpted: If you need to perform a btrfs-specific operation, you can easily use the btrfs-specific tools to prepare for it, specifically use btrfs fi df which could give provide every imaginable

Re: Provide a better free space estimate on RAID1

2014-02-09 Thread Kai Krakow
Roman Mamedov r...@romanrm.net schrieb: When I started to use unix, df returned blocks, not bytes. Without your proposed patch, it does that right. With your patch, it does it wrong. It returns total/used/available space that is usable/used/available by/for user data. No, it does not. It

[PATCH][V3] Provide a better free space estimate [was]Re: Provide a better free space estimate on RAID1

2014-02-09 Thread Goffredo Baroncelli
On 02/07/2014 05:40 AM, Roman Mamedov wrote: On Thu, 06 Feb 2014 20:54:19 +0100 Goffredo Baroncelli kreij...@libero.it wrote: [...] As Roman pointed out, df show the raw space available. However when a RAID level is used, the space available to the user is less. This patch try to address this

Re: Provide a better free space estimate on RAID1

2014-02-09 Thread Duncan
Roman Mamedov posted on Sun, 09 Feb 2014 15:20:00 +0600 as excerpted: On Sun, 9 Feb 2014 06:38:53 + (UTC) Duncan 1i5t5.dun...@cox.net wrote: RAID or multi-device filesystems aren't 1970s features and break 1970s behavior and the assumptions associated with it. If you're not prepared

Re: Provide a better free space estimate on RAID1

2014-02-08 Thread Roman Mamedov
On Fri, 7 Feb 2014 12:08:12 +0600 Roman Mamedov r...@romanrm.net wrote: Earlier conventions would have stated Size ~900GB, and Avail ~900GB. But that's not exactly true either, is it? Much better, and matching the user expectations of how RAID1 should behave, without a major gotcha

Re: Provide a better free space estimate on RAID1

2014-02-08 Thread Roman Mamedov
On Fri, 07 Feb 2014 21:32:42 +0100 Kai Krakow hurikhan77+bt...@gmail.com wrote: It should show the raw space available. Btrfs also supports compression and doesn't try to be smart about how much compressed data would fit in the free space of the drive. If one is using RAID1, it's supposed to

Re: Provide a better free space estimate on RAID1

2014-02-08 Thread Hugo Mills
On Sat, Feb 08, 2014 at 05:33:10PM +0600, Roman Mamedov wrote: On Fri, 07 Feb 2014 21:32:42 +0100 Kai Krakow hurikhan77+bt...@gmail.com wrote: It should show the raw space available. Btrfs also supports compression and doesn't try to be smart about how much compressed data would fit in

Re: Provide a better free space estimate on RAID1

2014-02-08 Thread Goffredo Baroncelli
On 02/07/2014 05:40 AM, Roman Mamedov wrote: On Thu, 06 Feb 2014 20:54:19 +0100 Goffredo Baroncelli kreij...@libero.it wrote: [...] Even I am not entirely convinced, I update the Roman's PoC in order to take in account all the RAID levels. I performed some tests with 7 48.8GB disks. Here my

[PATCH][V2] Re: Provide a better free space estimate on RAID1

2014-02-08 Thread Goffredo Baroncelli
On 02/07/2014 05:40 AM, Roman Mamedov wrote: On Thu, 06 Feb 2014 20:54:19 +0100 Goffredo Baroncelli kreij...@libero.it wrote: [...] Even I am not entirely convinced, I update the Roman's PoC in order to take in account all the RAID levels. The filesystem test is composed by 7 51GB disks.

Re: Provide a better free space estimate on RAID1

2014-02-08 Thread Kai Krakow
Hugo Mills h...@carfax.org.uk schrieb: On Sat, Feb 08, 2014 at 05:33:10PM +0600, Roman Mamedov wrote: On Fri, 07 Feb 2014 21:32:42 +0100 Kai Krakow hurikhan77+bt...@gmail.com wrote: It should show the raw space available. Btrfs also supports compression and doesn't try to be smart about

Re: Provide a better free space estimate on RAID1

2014-02-08 Thread Kai Krakow
Chris Murphy li...@colorremedies.com schrieb: On Feb 6, 2014, at 11:08 PM, Roman Mamedov r...@romanrm.net wrote: And what if I am accessing that partition on a server via a network CIFS/NFS share and don't even *have a way to find out* any of that. That's the strongest argument. And

Re: Provide a better free space estimate on RAID1

2014-02-08 Thread Kai Krakow
Martin Steigerwald mar...@lichtvoll.de schrieb: While I understand that there is *never* a guarentee that a given free space can really be allocated by a process cause other processes can allocate space as well in the mean time, and while I understand that its difficult to provide an accurate

Re: Provide a better free space estimate on RAID1

2014-02-08 Thread Roman Mamedov
On Sat, 08 Feb 2014 22:35:40 +0100 Kai Krakow hurikhan77+bt...@gmail.com wrote: Imagine the future: Btrfs supports different RAID levels per subvolume. We need to figure out where to place a new subvolume. I need raw numbers for it. Df won't tell me that now. Things become very difficult

Re: Provide a better free space estimate on RAID1

2014-02-08 Thread cwillu
Everyone who has actually looked at what the statfs syscall returns and how df (and everyone else) uses it, keep talking. Everyone else, go read that source code first. There is _no_ combination of values you can return in statfs which will not be grossly misleading in some common scenario that

Re: Provide a better free space estimate on RAID1

2014-02-08 Thread Kai Krakow
Roman Mamedov r...@romanrm.net schrieb: It should show the raw space available. Btrfs also supports compression and doesn't try to be smart about how much compressed data would fit in the free space of the drive. If one is using RAID1, it's supposed to fill up with a rate of 2:1. If one is

Re: Provide a better free space estimate on RAID1

2014-02-08 Thread Kai Krakow
cwillu cwi...@cwillu.com schrieb: Everyone who has actually looked at what the statfs syscall returns and how df (and everyone else) uses it, keep talking. Everyone else, go read that source code first. There is _no_ combination of values you can return in statfs which will not be grossly

Re: Provide a better free space estimate on RAID1

2014-02-08 Thread Kai Krakow
Roman Mamedov r...@romanrm.net schrieb: UNIX 'df' and the 'statfs' call on the other hand should keep the behavior people are accustomized to rely on since 1970s. When I started to use unix, df returned blocks, not bytes. Without your proposed patch, it does that right. With your patch, it

Re: Provide a better free space estimate on RAID1

2014-02-08 Thread Roman Mamedov
On Sun, 09 Feb 2014 00:32:47 +0100 Kai Krakow hurikhan77+bt...@gmail.com wrote: When I started to use unix, df returned blocks, not bytes. Without your proposed patch, it does that right. With your patch, it does it wrong. It returns total/used/available space that is usable/used/available

Re: Provide a better free space estimate on RAID1

2014-02-08 Thread Roman Mamedov
On Sun, 09 Feb 2014 00:17:29 +0100 Kai Krakow hurikhan77+bt...@gmail.com wrote: Dear employees, Please keep in mind that when you run out of space on the fileserver '\\DepartmentC', when you free up space in the directory '\PublicStorage7' the free space you gain on '\StorageArchive' is

Re: Provide a better free space estimate on RAID1

2014-02-08 Thread Chris Murphy
On Feb 8, 2014, at 6:55 PM, Roman Mamedov r...@romanrm.net wrote: Not sure what exactly becomes problematic if a 2-device RAID1 tells the user they can store 1 TB of their data on it, and is no longer lying about the possibility to store 2 TB on it as currently. Two 1TB disks in RAID1.

Re: Provide a better free space estimate on RAID1

2014-02-08 Thread Chris Murphy
On Feb 8, 2014, at 7:21 PM, Chris Murphy li...@colorremedies.com wrote: we don't have a top level switch for variable raid on a volume yet This isn't good wording. We don't have a controllable way to set variable raid levels. The interrupted convert model I'd consider not controllable.

Re: Provide a better free space estimate on RAID1

2014-02-08 Thread Duncan
Roman Mamedov posted on Sun, 09 Feb 2014 04:10:50 +0600 as excerpted: If you need to perform a btrfs-specific operation, you can easily use the btrfs-specific tools to prepare for it, specifically use btrfs fi df which could give provide every imaginable interpretation of free space estimate

Re: Provide a better free space estimate on RAID1

2014-02-07 Thread Martin Steigerwald
Am Donnerstag, 6. Februar 2014, 22:30:46 schrieb Chris Murphy: On Feb 6, 2014, at 9:40 PM, Roman Mamedov r...@romanrm.net wrote: On Thu, 06 Feb 2014 20:54:19 +0100 Goffredo Baroncelli kreij...@libero.it wrote: I agree with you about the needing of a solution. However your patch to

Re: Provide a better free space estimate on RAID1

2014-02-07 Thread Frank Kingswood
On 06/02/14 19:54, Goffredo Baroncelli wrote: Hi Roman On 02/06/2014 01:45 PM, Roman Mamedov wrote: There's not a lot of code to include (as my 3-line patch demonstrates), it could just as easily be removed when it's obsolete. But I did not have any high hopes of defeating the broken by design

Re: Provide a better free space estimate on RAID1

2014-02-07 Thread Chris Murphy
On Feb 6, 2014, at 11:08 PM, Roman Mamedov r...@romanrm.net wrote: And what if I am accessing that partition on a server via a network CIFS/NFS share and don't even *have a way to find out* any of that. That's the strongest argument. And if the user is using Explorer/Finder/Nautilus to

Re: Provide a better free space estimate on RAID1

2014-02-07 Thread Kai Krakow
Josef Bacik jba...@fb.com schrieb: On 02/05/2014 03:15 PM, Roman Mamedov wrote: Hello, On a freshly-created RAID1 filesystem of two 1TB disks: # df -h /mnt/p2/ Filesystem Size Used Avail Use% Mounted on /dev/sda2 1.8T 1.1M 1.8T 1% /mnt/p2 I cannot write 2TB of user

Re: Provide a better free space estimate on RAID1

2014-02-06 Thread Roman Mamedov
On Thu, 06 Feb 2014 09:38:15 +0200 Brendan Hide bren...@swiftspirit.co.za wrote: This is a known issue: https://btrfs.wiki.kernel.org/index.php/FAQ#Why_does_df_show_incorrect_free_space_for_my_RAID_volume.3F Btrfs is still considered experimental It's long overdue to start tackling these

Re: Provide a better free space estimate on RAID1

2014-02-06 Thread Goffredo Baroncelli
Hi Roman On 02/06/2014 01:45 PM, Roman Mamedov wrote: On Thu, 06 Feb 2014 09:38:15 +0200 [...] There's not a lot of code to include (as my 3-line patch demonstrates), it could just as easily be removed when it's obsolete. But I did not have any high hopes of defeating the broken by design

Re: Provide a better free space estimate on RAID1

2014-02-06 Thread Josef Bacik
On 02/05/2014 03:15 PM, Roman Mamedov wrote: Hello, On a freshly-created RAID1 filesystem of two 1TB disks: # df -h /mnt/p2/ Filesystem Size Used Avail Use% Mounted on /dev/sda2 1.8T 1.1M 1.8T 1% /mnt/p2 I cannot write 2TB of user data to that RAID1, so this estimate is

Re: Provide a better free space estimate on RAID1

2014-02-06 Thread Roman Mamedov
On Thu, 06 Feb 2014 20:54:19 +0100 Goffredo Baroncelli kreij...@libero.it wrote: I agree with you about the needing of a solution. However your patch to me seems even worse than the actual code. For example you cannot take in account the mix of data/linear and metadata/dup (with the

Re: Provide a better free space estimate on RAID1

2014-02-06 Thread Chris Murphy
On Feb 6, 2014, at 9:40 PM, Roman Mamedov r...@romanrm.net wrote: On Thu, 06 Feb 2014 20:54:19 +0100 Goffredo Baroncelli kreij...@libero.it wrote: I agree with you about the needing of a solution. However your patch to me seems even worse than the actual code. For example you cannot

Re: Provide a better free space estimate on RAID1

2014-02-06 Thread Roman Mamedov
On Thu, 6 Feb 2014 22:30:46 -0700 Chris Murphy li...@colorremedies.com wrote: From the original post, context is a 2x 1TB raid volume: Filesystem Size Used Avail Use% Mounted on /dev/sda2 1.8T 1.1M 1.8T 1% /mnt/p2 Earlier conventions would have stated Size ~900GB, and

Provide a better free space estimate on RAID1

2014-02-05 Thread Roman Mamedov
Hello, On a freshly-created RAID1 filesystem of two 1TB disks: # df -h /mnt/p2/ Filesystem Size Used Avail Use% Mounted on /dev/sda2 1.8T 1.1M 1.8T 1% /mnt/p2 I cannot write 2TB of user data to that RAID1, so this estimate is clearly misleading. I got tired of looking at the

Re: Provide a better free space estimate on RAID1

2014-02-05 Thread Brendan Hide
On 2014/02/05 10:15 PM, Roman Mamedov wrote: Hello, On a freshly-created RAID1 filesystem of two 1TB disks: # df -h /mnt/p2/ Filesystem Size Used Avail Use% Mounted on /dev/sda2 1.8T 1.1M 1.8T 1% /mnt/p2 I cannot write 2TB of user data to that RAID1, so this estimate is