wrong values in df and btrfs filesystem df
Hallo, linux-btrfs, First I create an array of 2 disks with mkfs.btrfs -d raid0 -m raid1 /dev/sdb1 /dev/sdd1 and mount it at /srv/MM. Then I fill it with about 1,6 TByte. And then I add /dev/sde1 via btrfs device add /dev/sde1 /srv/MM btrfs filesystem balance /srv/MM (it run about 20 hours) Then I work on it, copy some new files, delete some old files - all works well. Only df /srv/MM btrfs filesystem df /srv/MM show some completely wrong values: # df /srv/MM Filesystem 1K-blocks Used Available Use% Mounted on /dev/sdd15846053400 1593436456 2898463184 36% /srv/MM # btrfs filesystem df /srv/MM Data, RAID0: total=1.67TB, used=1.48TB System, RAID1: total=16.00MB, used=112.00KB System: total=4.00MB, used=0.00 Metadata, RAID1: total=3.75GB, used=2.26GB # btrfs-show Label: MMedia uuid: 120b036a-883f-46aa-bd9a-cb6a1897c8d2 Total devices 3 FS bytes used 1.48TB devid3 size 1.81TB used 573.76GB path /dev/sdb1 devid2 size 1.81TB used 573.77GB path /dev/sde1 devid1 size 1.82TB used 570.01GB path /dev/sdd1 Btrfs Btrfs v0.19 df shows an Available value which isn't related to any real value. The sum of used and Available is far away from the really existent disk space. When I copy additional files to /srv/MM then used still shows the right value, and the sum grows (slowly) to the max. available space. In btrfs filesystem df /srv/MM the line Data, RAID0: total=1.67TB, used=1.48TB shows a total value which isn't related to any existent value; maybe it still shows the used space before adding the third partition. This (wrong) value seems not to change. Kernel 2.6.38.1 btrfs from november 2010 Viele Gruesse! Helmut -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: switching balance into background
On Sat, Apr 09, 2011 at 07:57:00AM +0200, Helmut Hullen wrote: Hallo, linux-btrfs, I can't switch a running btrfs filesystem balance ... via ctrl z bg into the background, with other jobs this way works. The stopping command ctrl z doesn't work. (may be on other keyboards it's ctrl y) What goes wrong? A balance is a single long-lived ioctl call, so the userspace process is living in D state (uninterruptible sleep) until the kernel finishes. I've got a patch that forks the process, and runs the ioctl in the background, returning immediately. I should be able to get it to the list this weekend as part of the userspace side of the balance management patches. Hugo. -- === Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk === PGP key: 515C238D from wwwkeys.eu.pgp.net or http://www.carfax.org.uk --- There are three things you should never see being made: laws, --- standards, and sausages. signature.asc Description: Digital signature
Re: switching balance into background
Hallo, Hugo, Du meintest am 09.04.11: I can't switch a running btrfs filesystem balance ... via ctrl z bg into the background, [...] I've got a patch that forks the process, and runs the ioctl in the background, returning immediately. I should be able to get it to the list this weekend as part of the userspace side of the balance management patches. Sounds fine - thank you! Viele Gruesse! Helmut -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: wrong values in df and btrfs filesystem df
On Sat, Apr 09, 2011 at 08:25:00AM +0200, Helmut Hullen wrote: Hallo, linux-btrfs, First I create an array of 2 disks with mkfs.btrfs -d raid0 -m raid1 /dev/sdb1 /dev/sdd1 and mount it at /srv/MM. Then I fill it with about 1,6 TByte. And then I add /dev/sde1 via btrfs device add /dev/sde1 /srv/MM btrfs filesystem balance /srv/MM (it run about 20 hours) Then I work on it, copy some new files, delete some old files - all works well. Only df /srv/MM btrfs filesystem df /srv/MM show some completely wrong values: # df /srv/MM Filesystem 1K-blocks Used Available Use% Mounted on /dev/sdd15846053400 1593436456 2898463184 36% /srv/MM # btrfs filesystem df /srv/MM Data, RAID0: total=1.67TB, used=1.48TB System, RAID1: total=16.00MB, used=112.00KB System: total=4.00MB, used=0.00 Metadata, RAID1: total=3.75GB, used=2.26GB # btrfs-show Label: MMedia uuid: 120b036a-883f-46aa-bd9a-cb6a1897c8d2 Total devices 3 FS bytes used 1.48TB devid3 size 1.81TB used 573.76GB path /dev/sdb1 devid2 size 1.81TB used 573.77GB path /dev/sde1 devid1 size 1.82TB used 570.01GB path /dev/sdd1 Btrfs Btrfs v0.19 df shows an Available value which isn't related to any real value. I _think_ that value is the amount of space not allocated to any block group. If that's so, then Available (from df) plus the three total values (from btrfs fi df) should equal the size value from df. The sum of used and Available is far away from the really existent disk space. When I copy additional files to /srv/MM then used still shows the right value, and the sum grows (slowly) to the max. available space. In btrfs filesystem df /srv/MM the line Data, RAID0: total=1.67TB, used=1.48TB shows a total value which isn't related to any existent value; maybe it still shows the used space before adding the third partition. This (wrong) value seems not to change. It's not wrong -- it simply doesn't mean what you think it does. :) The total value in the output of btrfs fi df is the total space allocated to block groups. As the filesystem needs more space, it will allocate more block groups from the available raw storage pool, and the number will go up. This is explained on the wiki at [1]. HTH, Hugo. [1] https://btrfs.wiki.kernel.org/index.php/FAQ#Why_does_df_show_incorrect_free_space_for_my_RAID_volume.3F -- === Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk === PGP key: 515C238D from wwwkeys.eu.pgp.net or http://www.carfax.org.uk --- There are three things you should never see being made: laws, --- standards, and sausages. signature.asc Description: Digital signature
Re: wrong values in df and btrfs filesystem df
2011-04-09 10:11:41 +0100, Hugo Mills: [...] # df /srv/MM Filesystem 1K-blocks Used Available Use% Mounted on /dev/sdd15846053400 1593436456 2898463184 36% /srv/MM # btrfs filesystem df /srv/MM Data, RAID0: total=1.67TB, used=1.48TB System, RAID1: total=16.00MB, used=112.00KB System: total=4.00MB, used=0.00 Metadata, RAID1: total=3.75GB, used=2.26GB # btrfs-show Label: MMedia uuid: 120b036a-883f-46aa-bd9a-cb6a1897c8d2 Total devices 3 FS bytes used 1.48TB devid3 size 1.81TB used 573.76GB path /dev/sdb1 devid2 size 1.81TB used 573.77GB path /dev/sde1 devid1 size 1.82TB used 570.01GB path /dev/sdd1 Btrfs Btrfs v0.19 df shows an Available value which isn't related to any real value. I _think_ that value is the amount of space not allocated to any block group. If that's so, then Available (from df) plus the three total values (from btrfs fi df) should equal the size value from df. [...] Well, $ echo $((2898463184 + 1.67*2**30 + 4*2**10 + 16*2**10*2 + 3.75*2**20*2)) 4699513214.079 I do get the same kind of discrepancy: $ df -h /mnt FilesystemSize Used Avail Use% Mounted on /dev/sdb 8.2T 3.5T 3.2T 53% /mnt $ sudo btrfs fi show Label: none uuid: ... Total devices 3 FS bytes used 3.43TB devid4 size 2.73TB used 1.17TB path /dev/sdc devid3 size 2.73TB used 1.17TB path /dev/sdb devid2 size 2.70TB used 1.14TB path /dev/sda4 $ sudo btrfs fi df /mnt Data, RAID0: total=3.41TB, used=3.41TB System, RAID1: total=16.00MB, used=232.00KB Metadata, RAID1: total=35.25GB, used=20.55GB $ echo $((3.2 + 3.41 + 2*16/2**20 + 2*35.25/2**10)) 6.678847656253 -- Stephane -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v2] btrfs scrub - make fixups sync, don't reuse fixup bios
Fixups are already sync for csum failures, this patch makes them sync
for EIO case as well.
Fixups are now sharing pages with the parent sbio - instead of
allocating a separate page to do a fixup we grab the page from the sbio
buffer.
Fixup bios are no longer reused.
struct fixup is no longer needed, instead pass [sbio pointer, index].
Originally this was added to look at the possibility of sharing the code
between drive swap and scrub, but it actually fixes a serious bug in
scrub code where errors that could be corrected were ignored and
reported as uncorrectable.
Signed-off-by: Ilya Dryomov idryo...@gmail.com
---
fs/btrfs/scrub.c | 241 +++---
1 files changed, 67 insertions(+), 174 deletions(-)
diff --git a/fs/btrfs/scrub.c b/fs/btrfs/scrub.c
index 7c5eb21..e862167 100644
--- a/fs/btrfs/scrub.c
+++ b/fs/btrfs/scrub.c
@@ -60,7 +60,6 @@ typedef struct work_struct scrub_work_t;
struct scrub_bio;
struct scrub_page;
struct scrub_dev;
-struct scrub_fixup;
static void scrub_bio_end_io(struct bio *bio, int err);
static void scrub_checksum(scrub_work_t *work);
static int scrub_checksum_data(struct scrub_dev *sdev,
@@ -69,9 +68,11 @@ static int scrub_checksum_tree_block(struct scrub_dev *sdev,
struct scrub_page *spag, u64 logical,
void *buffer);
static int scrub_checksum_super(struct scrub_bio *sbio, void *buffer);
-static void scrub_recheck_end_io(struct bio *bio, int err);
-static void scrub_fixup_worker(scrub_work_t *work);
-static void scrub_fixup(struct scrub_fixup *fixup);
+static int scrub_fixup_check(struct scrub_bio *sbio, int ix);
+static void scrub_fixup_end_io(struct bio *bio, int err);
+static int scrub_fixup_io(int rw, struct block_device *bdev, sector_t sector,
+ struct page *page);
+static void scrub_fixup(struct scrub_bio *sbio, int ix);
#define SCRUB_PAGES_PER_BIO16 /* 64k per bio */
#define SCRUB_BIOS_PER_DEV 16 /* 1 MB per device in flight */
@@ -115,17 +116,6 @@ struct scrub_dev {
spinlock_t stat_lock;
};
-struct scrub_fixup {
- struct scrub_dev*sdev;
- struct bio *bio;
- u64 logical;
- u64 physical;
- struct scrub_page spag;
- scrub_work_twork;
- int err;
- int recheck;
-};
-
static void scrub_free_csums(struct scrub_dev *sdev)
{
while(!list_empty(sdev-csum_list)) {
@@ -250,108 +240,34 @@ nomem:
*/
static void scrub_recheck_error(struct scrub_bio *sbio, int ix)
{
- struct scrub_dev *sdev = sbio-sdev;
- struct btrfs_fs_info *fs_info = sdev-dev-dev_root-fs_info;
- struct bio *bio = NULL;
- struct page *page = NULL;
- struct scrub_fixup *fixup = NULL;
- int ret;
-
- /*
-* while we're in here we do not want the transaction to commit.
-* To prevent it, we increment scrubs_running. scrub_pause will
-* have to wait until we're finished
-* we can safely increment scrubs_running here, because we're
-* in the context of the original bio which is still marked in_flight
-*/
- atomic_inc(fs_info-scrubs_running);
-
- fixup = kzalloc(sizeof(*fixup), GFP_NOFS);
- if (!fixup)
- goto malloc_error;
-
- fixup-logical = sbio-logical + ix * PAGE_SIZE;
- fixup-physical = sbio-physical + ix * PAGE_SIZE;
- fixup-spag = sbio-spag[ix];
- fixup-sdev = sdev;
-
- bio = bio_alloc(GFP_NOFS, 1);
- if (!bio)
- goto malloc_error;
- bio-bi_private = fixup;
- bio-bi_size = 0;
- bio-bi_bdev = sdev-dev-bdev;
- fixup-bio = bio;
- fixup-recheck = 0;
-
- page = alloc_page(GFP_NOFS);
- if (!page)
- goto malloc_error;
-
- ret = bio_add_page(bio, page, PAGE_SIZE, 0);
- if (!ret)
- goto malloc_error;
-
- if (!sbio-err) {
- /*
-* shorter path: just a checksum error, go ahead and correct it
-*/
- scrub_fixup_worker(fixup-work);
- return;
+ if (sbio-err) {
+ if (scrub_fixup_io(READ, sbio-sdev-dev-bdev,
+ (sbio-physical + ix * PAGE_SIZE) 9,
+ sbio-bio-bi_io_vec[ix].bv_page) == 0) {
+ if (scrub_fixup_check(sbio, ix) == 0)
+ return;
+ }
}
- /*
-* an I/O-error occured for one of the blocks in the bio, not
-* necessarily for this one, so first try to read it separately
-*/
- SCRUB_INIT_WORK(fixup-work, scrub_fixup_worker);
- fixup-recheck = 1;
- bio-bi_end_io = scrub_recheck_end_io;
- bio-bi_sector = fixup-physical 9;
-
[PATCH] btrfs scrub - restore bios properly after media errors
The current code reallocates a bio after a media error. This is a
temporary measure introduced in v3 after a serious problem related to
bio reuse was found in v2 of scrub patchset.
Basically we did not reset bv_offset and bv_len fields of the bio_vec
structure. They are changed in case I/O error happens, for example, at
offset 512 or 1024 into the page. Also bi_flags field wasn't properly
setup before reusing the bio.
Signed-off-by: Ilya Dryomov idryo...@gmail.com
---
fs/btrfs/scrub.c | 48 +---
1 files changed, 13 insertions(+), 35 deletions(-)
diff --git a/fs/btrfs/scrub.c b/fs/btrfs/scrub.c
index e862167..1631327 100644
--- a/fs/btrfs/scrub.c
+++ b/fs/btrfs/scrub.c
@@ -418,45 +418,24 @@ static void scrub_checksum(scrub_work_t *work)
int ret;
if (sbio-err) {
- struct bio *bio;
- struct bio *old_bio;
-
- for (i = 0; i sbio-count; ++i) {
+ for (i = 0; i sbio-count; ++i)
scrub_recheck_error(sbio, i);
+
+ sbio-bio-bi_flags = ~(BIO_POOL_MASK - 1);
+ sbio-bio-bi_flags |= 1 BIO_UPTODATE;
+ sbio-bio-bi_phys_segments = 0;
+ sbio-bio-bi_idx = 0;
+
+ for (i = 0; i sbio-count; i++) {
+ struct bio_vec *bi;
+ bi = sbio-bio-bi_io_vec[i];
+ bi-bv_offset = 0;
+ bi-bv_len = PAGE_SIZE;
}
+
spin_lock(sdev-stat_lock);
++sdev-stat.read_errors;
spin_unlock(sdev-stat_lock);
-
- /*
-* FIXME: allocate a new bio after a media error. I haven't
-* figured out how to reuse this one
-*/
- old_bio = sbio-bio;
- bio = bio_kmalloc(GFP_NOFS, SCRUB_PAGES_PER_BIO);
- if (!bio) {
- /*
-* alloc failed. cancel the scrub and don't requeue
-* this sbio
-*/
- printk(KERN_ERR btrfs scrub: allocation failure,
- cancelling scrub\n);
- atomic_inc(sdev-dev-dev_root-fs_info-
- scrub_cancel_req);
- goto out_no_enqueue;
- }
- sbio-bio = bio;
- bio-bi_private = sbio;
- bio-bi_end_io = scrub_bio_end_io;
- bio-bi_sector = 0;
- bio-bi_bdev = sbio-sdev-dev-bdev;
- bio-bi_size = 0;
- for (i = 0; i SCRUB_PAGES_PER_BIO; ++i) {
- struct page *page;
- page = old_bio-bi_io_vec[i].bv_page;
- bio_add_page(bio, page, PAGE_SIZE, 0);
- }
- bio_put(old_bio);
goto out;
}
for (i = 0; i sbio-count; ++i) {
@@ -486,7 +465,6 @@ out:
sbio-next_free = sdev-first_free;
sdev-first_free = sbio-index;
spin_unlock(sdev-list_lock);
-out_no_enqueue:
atomic_dec(sdev-in_flight);
wake_up(sdev-list_wait);
}
--
1.7.2.5
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: wrong values in df and btrfs filesystem df
Hallo, Hugo, Du meintest am 09.04.11: df /srv/MM btrfs filesystem df /srv/MM show some completely wrong values: # df /srv/MM Filesystem 1K-blocks Used Available Use% Mounted on /dev/sdd15846053400 1593436456 2898463184 36% /srv/MM # btrfs filesystem df /srv/MM Data, RAID0: total=1.67TB, used=1.48TB System, RAID1: total=16.00MB, used=112.00KB System: total=4.00MB, used=0.00 Metadata, RAID1: total=3.75GB, used=2.26GB # btrfs-show Label: MMedia uuid: 120b036a-883f-46aa-bd9a-cb6a1897c8d2 Total devices 3 FS bytes used 1.48TB devid3 size 1.81TB used 573.76GB path /dev/sdb1 devid2 size 1.81TB used 573.77GB path /dev/sde1 devid1 size 1.82TB used 570.01GB path /dev/sdd1 Btrfs Btrfs v0.19 df shows an Available value which isn't related to any real value. I _think_ that value is the amount of space not allocated to any block group. If that's so, then Available (from df) plus the three total values (from btrfs fi df) should equal the size value from df. I'm not convinced - sorry. used plus available should be nearly the same value as the total in df; in my example 1.6 TB + 2.9 TB is far away from the total of 5.8 TB (here: Tera = 10^9). The total value in the output of btrfs fi df is the total space allocated to block groups. As the filesystem needs more space, it will allocate more block groups from the available raw storage pool, and the number will go up. This is explained on the wiki at [1]. I've studied the page - the data shown there looks to be consistent. Especially: used plus avail = size - The data seems (mostly) to be consistent on a new array of disks/ partitions. But when I work with device add and device delete then there are holes. Another machine: /dev/sda about 140 GB /dev/sdb about 140 GB /dev/sdc about 70 GB # mkfs.btrfs -L SCSI -d raid0 -m raid1 /dev/sdb1 /dev/sdc1 # mit 70 GByte gefüllt # df -t btrfs FilesystemType 1K-blocks Used Available Use% Mounted on /dev/sdb1btrfs 214058944 73956996 65522124 54% /mnt/SCSI # btrfs filesystem show Label: 'SCSI' uuid: 1932d11d-021a-4054-8429-db25a0204221 Total devices 2 FS bytes used 70.43GB devid1 size 136.73GB used 37.03GB path /dev/sdb1 devid2 size 67.41GB used 37.01GB path /dev/sdc1 # btrfs filesystem df /mnt/SCSI Data, RAID0: total=72.00GB, used=70.33GB Data: total=8.00MB, used=0.00 System, RAID1: total=8.00MB, used=12.00KB System: total=4.00MB, used=0.00 Metadata, RAID1: total=1.00GB, used=104.78MB Metadata: total=8.00MB, used=0.00 # # btrfs device add /dev/sda1 /mnt/SCSI # df -t btrfs FilesystemType 1K-blocks Used Available Use% Mounted on /dev/sdb1btrfs 356234160 84177056 200667344 30% /mnt/SCSI # btrfs filesystem show Label: 'SCSI' uuid: 1932d11d-021a-4054-8429-db25a0204221 Total devices 3 FS bytes used 80.16GB devid1 size 136.73GB used 42.03GB path /dev/sdb1 devid3 size 135.59GB used 0.00 path /dev/sda1 devid2 size 67.41GB used 42.01GB path /dev/sdc1 # btrfs filesystem df /mnt/SCSI Data, RAID0: total=82.00GB, used=80.04GB Data: total=8.00MB, used=0.00 System, RAID1: total=8.00MB, used=12.00KB System: total=4.00MB, used=0.00 Metadata, RAID1: total=1.00GB, used=119.54MB Metadata: total=8.00MB, used=0.00 # # btrfs fi balance /mnt/SCSI # df -t btrfs FilesystemType 1K-blocks Used Available Use% Mounted on /dev/sdb1btrfs 356234160 84125344 228211536 27% /mnt/SCSI # btrfs filesystem show Label: 'SCSI' uuid: 1932d11d-021a-4054-8429-db25a0204221 Total devices 3 FS bytes used 80.14GB devid1 size 136.73GB used 28.28GB path /dev/sdb1 devid3 size 135.59GB used 27.00GB path /dev/sda1 devid2 size 67.41GB used 29.01GB path /dev/sdc1 # btrfs filesystem df /mnt/SCSI Data, RAID0: total=81.00GB, used=80.04GB Data: total=8.00MB, used=0.00 System, RAID1: total=8.00MB, used=12.00KB System: total=4.00MB, used=0.00 Metadata, RAID1: total=1.25GB, used=94.29MB Metadata: total=8.00MB, used=0.00 # # btrfs device delete /dev/sdc1 /mnt/SCSI # umount /mnt/SCSI # mount LABEL=SCSI /mnt/SCSI # df -t btrfs FilesystemType 1K-blocks Used Available Use% Mounted on /dev/sdb1btrfs 285548192 232024076 51758540 82% /mnt/SCSI # btrfs filesystem show Label: 'SCSI' uuid: 1932d11d-021a-4054-8429-db25a0204221 Total devices 2 FS bytes used 221.04GB devid1 size 136.73GB used 111.51GB path /dev/sdb1 devid3 size 135.59GB used 111.51GB path /dev/sda1 # btrfs filesystem df /mnt/SCSI Data, RAID0: total=222.00GB, used=220.80GB System, RAID1: total=8.00MB, used=24.00KB
Re: wrong values in df and btrfs filesystem df
On Sat, 2011-04-09 at 10:11 +0100, Hugo Mills wrote: On Sat, Apr 09, 2011 at 08:25:00AM +0200, Helmut Hullen wrote: Hallo, linux-btrfs, First I create an array of 2 disks with mkfs.btrfs -d raid0 -m raid1 /dev/sdb1 /dev/sdd1 and mount it at /srv/MM. Then I fill it with about 1,6 TByte. And then I add /dev/sde1 via btrfs device add /dev/sde1 /srv/MM btrfs filesystem balance /srv/MM (it run about 20 hours) Then I work on it, copy some new files, delete some old files - all works well. Only df /srv/MM btrfs filesystem df /srv/MM show some completely wrong values: It's not wrong -- it simply doesn't mean what you think it does. :) The total value in the output of btrfs fi df is the total space allocated to block groups. As the filesystem needs more space, it will allocate more block groups from the available raw storage pool, and the number will go up. This is explained on the wiki at [1]. And I just drew up a picture which I think should help explain it a bit, too: http://www.kepstin.ca/dump/btrfs-alloc.png If I can figure out how to add images to the btrfs wiki, and find a good place to put it, do you think this would be a helpful addition? [1] https://btrfs.wiki.kernel.org/index.php/FAQ#Why_does_df_show_incorrect_free_space_for_my_RAID_volume.3F -- Calvin Walton calvin.wal...@kepstin.ca -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: wrong values in df and btrfs filesystem df
Hallo, Calvin, Du meintest am 09.04.11: Then I work on it, copy some new files, delete some old files - all works well. Only df /srv/MM btrfs filesystem df /srv/MM show some completely wrong values: [...] And I just drew up a picture which I think should help explain it a bit, too: http://www.kepstin.ca/dump/btrfs-alloc.png Nice picture. But it doesn't solve the problem that I need a reliable information about the free/available space. And I prefer asking with df for this information - df should work in the same way for all filesystems. Viele Gruesse! Helmut -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: wrong values in df and btrfs filesystem df
On Sat, 2011-04-09 at 19:05 +0200, Helmut Hullen wrote: Then I work on it, copy some new files, delete some old files - all works well. Only df /srv/MM btrfs filesystem df /srv/MM show some completely wrong values: And I just drew up a picture which I think should help explain it a bit, too: http://www.kepstin.ca/dump/btrfs-alloc.png Nice picture. But it doesn't solve the problem that I need a reliable information about the free/available space. And I prefer asking with df for this information - df should work in the same way for all filesystems. The problem is that the answer to the seemingly simple question: How much more data can I put onto this filesystem? gets pretty hard with btrfs. Your case is one of the simpler ones - To calculate the remaining space for files, you take the unused allocated data space (light blue on my picture), add the unallocated space (white), divide by the raid mode redundancy, and subtract some percentage (this is only an estimate, of course...) of that unallocated space for the additional metadata overhead. Now imagine the case where your btrfs filesystem has files stored in multiple raid modes: e.g. some files are raid5, others raid0. The amount of data you can write to the filesystem then depends on how you write the data! You might be able to fit 64gb if you use raid0, but only 48gb with raid5; and only 16gb with raid1! There isn't a single number that btrfs can report which does what you want. -- Calvin Walton calvin.wal...@kepstin.ca -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: wrong values in df and btrfs filesystem df
Hallo, Calvin, Du meintest am 09.04.11: Nice picture. But it doesn't solve the problem that I need a reliable information about the free/available space. And I prefer asking with df for this information - df should work in the same way for all filesystems. The problem is that the answer to the seemingly simple question: How much more data can I put onto this filesystem? gets pretty hard with btrfs. Your case is one of the simpler ones - To calculate the remaining space for files, you take the unused allocated data space (light blue on my picture), add the unallocated space (white), divide by the raid mode redundancy, and subtract some percentage (this is only an estimate, of course...) of that unallocated space for the additional metadata overhead. That's simple? Maybe I'm simple minded. But I expect the same meaning of the shown date with df mointpoint regardless what kind of filesystem is mounted. But no test item for an IQ test. If the value of available is unresolvable then btrfs should not show any value. Ok - there's a problem slightly similar to the available value of compressed partitions. Viele Gruesse! Helmut -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: wrong values in df and btrfs filesystem df
Helmut Hullen wrote: If the value of available is unresolvable then btrfs should not show any value. Disagree strongly. I think a pessimistic estimate would be much better to show, than no value at all. This may be what is currently shown. As for solving this with a high degree of usability, that's not really possible when constrained to the traditional paradigm that one filesystem will have completely consistent behavior backing all of it. I think the only answer is to have btrfs-specific tools that know more about the filesystem, and can present the relevant facts. Taking the example of part fs being raid0 and part being raid5, such a tool would then list calculated values for both those parts of the fs. One showing how much could go into the raid0 part, the other how much could go into the raid5 part. But for such filesystems, Linux can't do what Helmut would like. Maybe it would be possible to optimize the reported numbers, to be what the user actually wants as often as possible. Ie. if there is only one type of backing storage (sorry, don't know the terms) then the calculation would be easier to get right, following the simple formula that was just given. This is all eye candy however, completely irrelevant IMO as long as the filesystem oopses, or eats root nodes. :) //Peter -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
