RE: Bug in btrfs-debug-tree for two or more devices.
-Original Message- From: linux-btrfs-ow...@vger.kernel.org [mailto:linux-btrfs-ow...@vger.kernel.org] On Behalf Of Randy Barlow Sent: Tuesday, June 12, 2012 8:28 PM To: linux-btrfs@vger.kernel.org Subject: Re: Bug in btrfs-debug-tree for two or more devices. On Tuesday, June 12, 2012 06:53:00 AM Santosh Hosamani wrote: > Kernel 3.0.13.0.27-default This kernel is very old for btrfs. Can you try with at least Linux 3.4? I have installed 3.4.2 kernel but still I am facing the same issue.May be my understanding of calculating the used block may be wrong. If someone could help me in understanding .It would be great. -- R -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html http://www.mindtree.com/email/disclaimer.html -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: Bug in btrfs-debug-tree for two or more devices.
-Original Message- From: linux-btrfs-ow...@vger.kernel.org [mailto:linux-btrfs-ow...@vger.kernel.org] On Behalf Of Hugo Mills Sent: Wednesday, June 13, 2012 1:37 AM To: Santosh Hosamani Cc: linux-btrfs@vger.kernel.org Subject: Re: Bug in btrfs-debug-tree for two or more devices. On Tue, Jun 12, 2012 at 06:53:00AM +, Santosh Hosamani wrote: > > Hi btrfs folks, > I am working on btrfs filesystem on how it manages the free > space. And found out btrfs maintain a ctree which manages the physical > location of the chunks and stripes of the filesystem. > Btrfs-debug-tree also gives the information on the chunk tree > > I created btrfs on single device and two device.I have attached the output of > both on running btrfs-debug-tree. > For single device sum of all the length in the chunks will add upto the total > used bytes which is expected behavior. > > But for two devices sum of all lengths in the chunks does not add to the > total bytes .Am I missing something . Without actually seeing the details of your technique and expectations, I shall make a guess that you're not accounting for the double-counting of RAID-1 metadata. In other words, you will find that all of the metadata device extents (or chunks) will appear twice -- once on each device. Actually, this isn't quite right either -- what you really need to do is look at the RAID-1, RAID-10 and DUP bits in the chunk flags, add up all of those chunks, divide by two, and then add in the remaining (RAID-0 and single) chunks. That total should then add up to the total value of allocated space that you get from the output of "btrfs fi df". >> chunk tree leaf 20971520 items 8 free space 3023 generation 4 owner 3 fs uuid 23f86d1e-038a-4f5b-b87c-2ba78018135c chunk uuid db672366-6801-4f83-99ef-2087a60bb394 item 0 key (DEV_ITEMS DEV_ITEM 1) itemoff 3897 itemsize 98 dev item devid 1 total_bytes 3221225472 bytes used 673579008 item 1 key (DEV_ITEMS DEV_ITEM 2) itemoff 3799 itemsize 98 dev item devid 2 total_bytes 3221225472 bytes used 652607488 item 2 key (FIRST_CHUNK_TREE CHUNK_ITEM 0) itemoff 3719 itemsize 80 chunk length 4194304 owner 2 type 2 num_stripes 1 stripe 0 devid 1 offset 0 item 3 key (FIRST_CHUNK_TREE CHUNK_ITEM 4194304) itemoff 3639 itemsize 80 chunk length 8388608 owner 2 type 4 num_stripes 1 stripe 0 devid 1 offset 4194304 item 4 key (FIRST_CHUNK_TREE CHUNK_ITEM 12582912) itemoff 3559 itemsize 80 chunk length 8388608 owner 2 type 1 num_stripes 1 stripe 0 devid 1 offset 12582912 item 5 key (FIRST_CHUNK_TREE CHUNK_ITEM 20971520) itemoff 3447 itemsize 112 chunk length 8388608 owner 2 type 18 num_stripes 2 stripe 0 devid 2 offset 1048576 stripe 1 devid 1 offset 20971520 item 6 key (FIRST_CHUNK_TREE CHUNK_ITEM 29360128) itemoff 3335 itemsize 112 chunk length 322109440 owner 2 type 20 num_stripes 2 stripe 0 devid 2 offset 9437184 stripe 1 devid 1 offset 29360128 item 7 key (FIRST_CHUNK_TREE CHUNK_ITEM 351469568) itemoff 3223 itemsize 112 chunk length 644218880 owner 2 type 9 num_stripes 2 stripe 0 devid 2 offset 331546624 stripe 1 devid 1 offset 351469568 chunk tree will tell me where the physical stripes are there right .?Irrespective of the raid type ... correct me if I am wrong. If not how will you know which all blocks are occupied and which all block are free. Basically what I want to do is . get the used blocks of all the devices and create a bitmap of that and zero out all the free block. Then I should not overwrite the used blocks. I should be able to mount the filesystem without any error. How do I achieve that? > Also I notice that for the second device the superblock location 0x1 is > not considered as used . > > I would be really grateful if you folks can answer my query. > > I hav run these tests on SLES11-sp2-x86 Kernel 3.0.13.0.27-default This is pretty old, but shouldn't affect the results. It will cause reliability problems if you try running it seriously. Hugo. -- === Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk === PGP key: 515C238D from wwwkeys.eu.pgp.net or http://www.carfax.org.uk --- "There's a Martian war machine outside -- they want to talk --- to you about a cure for the common cold." http://www.mindtree.com/email/disclaimer.html -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: Bug in btrfs-debug-tree for two or more devices.
-Original Message- From: Arne Jansen [mailto:sensi...@gmx.net] Sent: Tuesday, June 12, 2012 10:20 PM To: Santosh Hosamani Cc: linux-btrfs@vger.kernel.org Subject: Re: Bug in btrfs-debug-tree for two or more devices. On 06/12/2012 08:53 AM, Santosh Hosamani wrote: > > Hi btrfs folks, > I am working on btrfs filesystem on how it manages the free > space. And found out btrfs maintain a ctree which manages the physical > location of the chunks and stripes of the filesystem. > Btrfs-debug-tree also gives the information on the chunk tree > > I created btrfs on single device and two device.I have attached the output of > both on running btrfs-debug-tree. > For single device sum of all the length in the chunks will add upto the total > used bytes which is expected behavior. > > But for two devices sum of all lengths in the chunks does not add to the > total bytes .Am I missing something . > Also I notice that for the second device the superblock location 0x1 is > not considered as used . > > I would be really grateful if you folks can answer my query. It's definitely not a bug in debug-tree, but just a problem in interpreting the result. Could you please paste the output of btrfs-debug-tree -d? This way it is easier to see what's bothering you :) Btrfs fi show Label: none uuid: 23f86d1e-038a-4f5b-b87c-2ba78018135c Total devices 2 FS bytes used 28.00KB devid2 size 3.00GB used 622.38MB path /dev/sdb2 devid1 size 3.00GB used 642.38MB path /dev/sdb1 Btrfs v0.19+ Btrfs-debug-tree -d /dev/sdb2 root tree leaf 29364224 items 9 free space 2349 generation 4 owner 1 fs uuid 23f86d1e-038a-4f5b-b87c-2ba78018135c chunk uuid db672366-6801-4f83-99ef-2087a60bb394 item 0 key (EXTENT_TREE ROOT_ITEM 0) itemoff 3756 itemsize 239 root data bytenr 29368320 level 0 dirid 0 refs 1 gen 4 item 1 key (DEV_TREE ROOT_ITEM 0) itemoff 3517 itemsize 239 root data bytenr 29372416 level 0 dirid 0 refs 1 gen 4 item 2 key (FS_TREE INODE_REF 6) itemoff 3500 itemsize 17 inode ref index 0 namelen 7 name: default item 3 key (FS_TREE ROOT_ITEM 0) itemoff 3261 itemsize 239 root data bytenr 29360128 level 0 dirid 256 refs 1 gen 4 item 4 key (ROOT_TREE_DIR INODE_ITEM 0) itemoff 3101 itemsize 160 inode generation 3 size 0 block group 0 mode 40555 links 1 item 5 key (ROOT_TREE_DIR INODE_REF 6) itemoff 3089 itemsize 12 inode ref index 0 namelen 2 name: .. item 6 key (ROOT_TREE_DIR DIR_ITEM 2378154706) itemoff 3052 itemsize 37 location key (FS_TREE ROOT_ITEM 18446744073709551615) type 2 namelen 7 datalen 0 name: default item 7 key (CSUM_TREE ROOT_ITEM 0) itemoff 2813 itemsize 239 root data bytenr 29376512 level 0 dirid 0 refs 1 gen 4 item 8 key (DATA_RELOC_TREE ROOT_ITEM 0) itemoff 2574 itemsize 239 root data bytenr 29380608 level 0 dirid 256 refs 1 gen 4 chunk tree leaf 20971520 items 8 free space 3023 generation 4 owner 3 fs uuid 23f86d1e-038a-4f5b-b87c-2ba78018135c chunk uuid db672366-6801-4f83-99ef-2087a60bb394 item 0 key (DEV_ITEMS DEV_ITEM 1) itemoff 3897 itemsize 98 dev item devid 1 total_bytes 3221225472 bytes used 673579008 item 1 key (DEV_ITEMS DEV_ITEM 2) itemoff 3799 itemsize 98 dev item devid 2 total_bytes 3221225472 bytes used 652607488 item 2 key (FIRST_CHUNK_TREE CHUNK_ITEM 0) itemoff 3719 itemsize 80 chunk length 4194304 owner 2 type 2 num_stripes 1 stripe 0 devid 1 offset 0 item 3 key (FIRST_CHUNK_TREE CHUNK_ITEM 4194304) itemoff 3639 itemsize 80 chunk length 8388608 owner 2 type 4 num_stripes 1 stripe 0 devid 1 offset 4194304 item 4 key (FIRST_CHUNK_TREE CHUNK_ITEM 12582912) itemoff 3559 itemsize 80 chunk length 8388608 owner 2 type 1 num_stripes 1 stripe 0 devid 1 offset 12582912 item 5 key (FIRST_CHUNK_TREE CHUNK_ITEM 20971520) itemoff 3447 itemsize 112 chunk length 8388608 owner 2 type 18 num_stripes 2 stripe 0 devid 2 offset 1048576 stripe 1 devid 1 offset 20971520 item 6 key (FIRST_CHUNK_TREE CHUNK_ITEM 29360128) itemoff 3335 itemsize 112 chunk length 322109440 owner 2 type 20 num_stripes 2 stripe 0 devid 2 offset 9437184 stripe 1 devid 1 offset 29360128 item 7 key (FIRST_CHUNK_TREE CHUNK_ITEM 351469568) itemoff 3223 itemsize 112 chunk length 644218880 owner 2 type 9 num_stripes 2 stripe 0 devid 2 offset 331546624 stripe 1 devid 1 offset 351469568 device tree
Bug in btrfs-debug-tree for two or more devices.
Hi btrfs folks, I am working on btrfs filesystem on how it manages the free space. And found out btrfs maintain a ctree which manages the physical location of the chunks and stripes of the filesystem. Btrfs-debug-tree also gives the information on the chunk tree I created btrfs on single device and two device.I have attached the output of both on running btrfs-debug-tree. For single device sum of all the length in the chunks will add upto the total used bytes which is expected behavior. But for two devices sum of all lengths in the chunks does not add to the total bytes .Am I missing something . Also I notice that for the second device the superblock location 0x1 is not considered as used . I would be really grateful if you folks can answer my query. I hav run these tests on SLES11-sp2-x86 Kernel 3.0.13.0.27-default If you want any further clarification I would be wiiling to help you guys on this. Thanks, Santosh Hosamani http://www.mindtree.com/email/disclaimer.html root tree leaf 29364224 items 9 free space 2349 generation 4 owner 1 fs uuid 22d1b011-b53a-423f-9f61-ca6d5f4192c7 chunk uuid e7e5467e-ef4e-4b47-ac37-198c1cc575b9 item 0 key (EXTENT_TREE ROOT_ITEM 0) itemoff 3756 itemsize 239 root data bytenr 29368320 level 0 dirid 0 refs 1 gen 4 item 1 key (DEV_TREE ROOT_ITEM 0) itemoff 3517 itemsize 239 root data bytenr 29372416 level 0 dirid 0 refs 1 gen 4 item 2 key (FS_TREE INODE_REF 6) itemoff 3500 itemsize 17 inode ref index 0 namelen 7 name: default item 3 key (FS_TREE ROOT_ITEM 0) itemoff 3261 itemsize 239 root data bytenr 29360128 level 0 dirid 256 refs 1 gen 4 item 4 key (ROOT_TREE_DIR INODE_ITEM 0) itemoff 3101 itemsize 160 inode generation 3 size 0 block group 0 mode 40555 links 1 item 5 key (ROOT_TREE_DIR INODE_REF 6) itemoff 3089 itemsize 12 inode ref index 0 namelen 2 name: .. item 6 key (ROOT_TREE_DIR DIR_ITEM 2378154706) itemoff 3052 itemsize 37 location key (FS_TREE ROOT_ITEM 18446744073709551615) type 2 namelen 7 datalen 0 name: default item 7 key (CSUM_TREE ROOT_ITEM 0) itemoff 2813 itemsize 239 root data bytenr 29376512 level 0 dirid 0 refs 1 gen 4 item 8 key (DATA_RELOC_TREE ROOT_ITEM 0) itemoff 2574 itemsize 239 root data bytenr 29380608 level 0 dirid 256 refs 1 gen 4 chunk tree leaf 20971520 items 8 free space 3023 generation 4 owner 3 fs uuid 22d1b011-b53a-423f-9f61-ca6d5f4192c7 chunk uuid e7e5467e-ef4e-4b47-ac37-198c1cc575b9 item 0 key (DEV_ITEMS DEV_ITEM 1) itemoff 3897 itemsize 98 dev item devid 1 total_bytes 3221225472 bytes used 673579008 item 1 key (DEV_ITEMS DEV_ITEM 2) itemoff 3799 itemsize 98 dev item devid 2 total_bytes 3221225472 bytes used 652607488 item 2 key (FIRST_CHUNK_TREE CHUNK_ITEM 0) itemoff 3719 itemsize 80 chunk length 4194304 owner 2 type 2 num_stripes 1 stripe 0 devid 1 offset 0 item 3 key (FIRST_CHUNK_TREE CHUNK_ITEM 4194304) itemoff 3639 itemsize 80 chunk length 8388608 owner 2 type 4 num_stripes 1 stripe 0 devid 1 offset 4194304 item 4 key (FIRST_CHUNK_TREE CHUNK_ITEM 12582912) itemoff 3559 itemsize 80 chunk length 8388608 owner 2 type 1 num_stripes 1 stripe 0 devid 1 offset 12582912 item 5 key (FIRST_CHUNK_TREE CHUNK_ITEM 20971520) itemoff 3447 itemsize 112 chunk length 8388608 owner 2 type 18 num_stripes 2 stripe 0 devid 2 offset 1048576 stripe 1 devid 1 offset 20971520 item 6 key (FIRST_CHUNK_TREE CHUNK_ITEM 29360128) itemoff 3335 itemsize 112 chunk length 322109440 owner 2 type 20 num_stripes 2 stripe 0 devid 2 offset 9437184 stripe 1 devid 1 offset 29360128 item 7 key (FIRST_CHUNK_TREE CHUNK_ITEM 351469568) itemoff 3223 itemsize 112 chunk length 644218880 owner 2 type 9 num_stripes 2 stripe 0 devid 2 offset 331546624 stripe 1 devid 1 offset 351469568 extent tree key (EXTENT_TREE ROOT_ITEM 0) leaf 29368320 items 13 free space 3169 generation 4 owner 2 fs uuid 22d1b011-b53a-423f-9f61-ca6d5f4192c7 chunk uuid e7e5467e-ef4e-4b47-ac37-198c1cc575b9 item 0 key (0 BLOCK_GROUP_ITEM 4194304) itemoff 3971 itemsize 24 block group used 0 chunk_objectid 256 flags 2 item 1 key (4194304 BLOCK_GROUP_ITEM 8388608) itemoff 3947 itemsize 24 block group used 0 chunk_objectid 256 flags 4 item 2 key (12582912 BLOCK_GROUP_ITEM 8388608) itemoff 3923 itemsize 24 block group used 0 chunk_objectid 256 flags
