Aw: Re: [BUG] Fails to duplicate metadata/system

2015-07-10 Thread conchur
 Gesendet: Freitag, 10. Juli 2015 um 02:51 Uhr
 Von: Chris Murphy li...@colorremedies.com
 An: conc...@web.de
 Cc: Btrfs BTRFS linux-btrfs@vger.kernel.org
 Betreff: Re: [BUG] Fails to duplicate metadata/system
 On Thu, Jul 9, 2015 at 5:34 PM, conc...@web.de wrote:
  Hi,
 
  I've noticed that a single device partition was using metadata.single and 
  system.single instead of metadata.dup and system.dup. All tests to force 
  conversion to dup failed.
 
 
 Try only -mconvert=dup and without -f flag and see if it works. I'm
 pretty sure system chunks are treated in parity with metadata chunks
 now so it doesn't need to be separately listed. And -f isn't needed
 except to reduce redundancy.

-f is also needed when operating on -s

I already tried all that and nothing worked

 If that's not it, I'm going to speculate maybe try kernel 4.0.6 and
 higher, as there was a bug in 4.0 that prevented chunk conversions but
 I thought that only applied to raid profiles, not single vs dup. The
 fix for that was commit 153c35b60c72de9fae06c8e2c8b2c47d79d4.

I have now downgraded to 3.16 and it was possible to change from single to dup. 
This is enough for me.


--
Chris Murphy
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[BUG] Fails to duplicate metadata/system

2015-07-09 Thread conchur
Hi,

I've noticed that a single device partition was using metadata.single and 
system.single instead of metadata.dup and system.dup. All tests to force 
conversion to dup failed.

Here is how to reproduce this with an image and some very simple BTRFS commands 
(Debian stretch):

$ uname -a
Linux asdasd 4.0.0-1-amd64 #1 SMP Debian 4.0.2-1 (2015-05-11) x86_64 GNU/Linux
$ btrfs --version
btrfs-progs v4.0
$ fallocate -l 8G test.img
$ mkdir mnt
$ mkfs.btrfs test.img
$ mount -o loop test.img mnt
$ touch mnt/asdasd
$ btrfs fi df mnt
Data, single: total=8.00MiB, used=64.00KiB
System, DUP: total=8.00MiB, used=16.00KiB
System, single: total=4.00MiB, used=0.00B
Metadata, DUP: total=409.56MiB, used=112.00KiB
Metadata, single: total=8.00MiB, used=0.00B
GlobalReserve, single: total=16.00MiB, used=0.00B
$ btrfs balance start -v -mconvert=single -sconvert=single -dconvert=single mnt 
-f
Dumping filters: flags 0xf, state 0x0, force is on
  DATA (flags 0x100): converting, target=281474976710656, soft is off
  METADATA (flags 0x100): converting, target=281474976710656, soft is off
  SYSTEM (flags 0x100): converting, target=281474976710656, soft is off
Done, had to relocate 5 out of 5 chunks
$ btrfs fi df mnt
Data, single: total=832.00MiB, used=256.00KiB
System, single: total=32.00MiB, used=16.00KiB
Metadata, single: total=256.00MiB, used=112.00KiB
GlobalReserve, single: total=16.00MiB, used=0.00B
$ btrfs balance start -v -mconvert=dup -sconvert=dup -dconvert=single mnt -f
Dumping filters: flags 0xf, state 0x0, force is on
  DATA (flags 0x100): converting, target=281474976710656, soft is off
  METADATA (flags 0x100): converting, target=32, soft is off
  SYSTEM (flags 0x100): converting, target=32, soft is off
Done, had to relocate 3 out of 3 chunks
$ btrfs fi df mnt   
 
Data, single: total=832.00MiB, used=320.00KiB
System, single: total=32.00MiB, used=16.00KiB
Metadata, single: total=256.00MiB, used=112.00KiB
GlobalReserve, single: total=16.00MiB, used=0.00B


The expected result would be Metadata, DUP and System, DUP and not 
Metadata, single and System, single






Some more info.


$ btrfs fi show mnt
Label: none  uuid: b1a70fc4-7c18-4929-9b73-8f8bb328e7de
Total devices 1 FS bytes used 384.00KiB
devid1 size 8.00GiB used 1.09GiB path /dev/loop0

btrfs-progs v4.0
$ btrfs fi usage mnt
Overall:
Device size:   8.00GiB
Device allocated:  1.09GiB
Device unallocated:6.91GiB
Device missing:  0.00B
Used:384.00KiB
Free (estimated):  7.72GiB  (min: 7.72GiB)
Data ratio:   1.00
Metadata ratio:   1.00
Global reserve:   16.00MiB  (used: 0.00B)

Data,single: Size:832.00MiB, Used:256.00KiB
   /dev/loop0832.00MiB

Metadata,single: Size:256.00MiB, Used:112.00KiB
   /dev/loop0256.00MiB

System,single: Size:32.00MiB, Used:16.00KiB
   /dev/loop0 32.00MiB

Unallocated:
   /dev/loop0  6.91GiB
$ btrfs-debug-tree test.img 
root tree
leaf 2539634688 items 16 free space 12515 generation 47 owner 1
fs uuid b1a70fc4-7c18-4929-9b73-8f8bb328e7de
chunk uuid c2606900-bfa1-444e-ab4d-3f0b2d31626b
item 0 key (EXTENT_TREE ROOT_ITEM 0) itemoff 15844 itemsize 439
root data bytenr 2539651072 level 0 dirid 0 refs 1 gen 47
uuid ----
item 1 key (DEV_TREE ROOT_ITEM 0) itemoff 15405 itemsize 439
root data bytenr 2539569152 level 0 dirid 0 refs 1 gen 46
uuid ----
item 2 key (FS_TREE INODE_REF 6) itemoff 15388 itemsize 17
inode ref index 0 namelen 7 name: default
item 3 key (FS_TREE ROOT_ITEM 0) itemoff 14949 itemsize 439
root data bytenr 2539356160 level 0 dirid 256 refs 1 gen 42
uuid ----
ctransid 6 otransid 0 stransid 0 rtransid 0
item 4 key (ROOT_TREE_DIR INODE_ITEM 0) itemoff 14789 itemsize 160
inode generation 3 transid 0 size 0 block group 0 mode 40755 
links 1 uid 0 gid 0 rdev 0 flags 0x0
item 5 key (ROOT_TREE_DIR INODE_REF 6) itemoff 14777 itemsize 12
inode ref index 0 namelen 2 name: ..
item 6 key (ROOT_TREE_DIR DIR_ITEM 2378154706) itemoff 14740 itemsize 37
location key (FS_TREE ROOT_ITEM -1) type DIR
namelen 7 datalen 0 name: default
item 7 key (CSUM_TREE ROOT_ITEM 0) itemoff 14301 itemsize 439
root data bytenr 2539667456 level 0 dirid 0 refs 1 gen 47
uuid ----
item 8 key (UUID_TREE ROOT_ITEM 0) itemoff 13862 itemsize 439
root data bytenr 2539208704 level 0 dirid 0 refs 1 gen 41
uuid be2539ee-1c09-e84c-8ec9-bfe054347ccf
item 

Metadata size limitations of btrfs

2015-05-28 Thread conchur
I am working a lot with multiple Linux + OpenWrt builds on top of a btrfs
filesystem. But I am encountering often that my filesystem is considered
full by btrfs (beside having more than 40% free space). The only thing
looking full is the metadata. I've tried to run the balance command with
-dusage=0 but it didn't help. The metadata size stayed at 10G and was
filled up to nearly 10G.

My question is: Can this metadata size limit be changed to allow more
small files on the filesystem? And if yes, how?

According to the kernel btrfs documentation, metadata_ratio= is
currently off by default. So I would guess that this ratio is not
the thing I should play with.

Here the requested information of my fs (after removing a lot of build
directories):

franzbroetchen#   uname -a
Linux franzbroetchen 4.0.0-1-amd64 #1 SMP Debian 4.0.2-1 (2015-05-11) x86_64 
GNU/Linux
franzbroetchen#   btrfs --version
btrfs-progs v4.0
franzbroetchen#   btrfs fi show
Label: 'root'  uuid: 57fe2100-0510-11e5-977e-0021ccb48233
Total devices 1 FS bytes used 182.55GiB
devid1 size 467.68GiB used 206.06GiB path 
/dev/mapper/franzbroetchen_vg-root

btrfs-progs v4.0
franzbroetchen#   btrfs fi df / 
Data, single: total=190.00GiB, used=176.47GiB
System, DUP: total=32.00MiB, used=48.00KiB
Metadata, DUP: total=8.00GiB, used=6.08GiB
GlobalReserve, single: total=512.00MiB, used=0.00B
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html