Continuous Benchmarking
Hello, I've got a system up-and-running that can run arbitrary benchmarks in a VM with direct access to an SSD drive. I'd like to have a benchmark to run on a few different revisions of the btrfs code to get some data to test with, so if any btrfs devs or testers would like to help, please send a list of commands to run and what directory to run them from (inside the mount or outside can be inferred by /mnt/btrfs versus any other path). I can benchmark both btrfs-progs and the kernel module separately, in case there are any btrfs-progs specific benchmarks people would like to run. Best, Fahrzin Hemmati -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: filesystem full when it's not? out of inodes? huh?
btrfs is horrible for small filesystems (like a 5GB drive). df -h says you have 967MB available, but btrfs (at least by default) allocates 1GB at a time to data/metadata. This means that your 10MB file is too big for the current allocation and requires a new data chunk, or another 1GB, which you don't have. Others might know of a way of changing the allocation size to less than 1GB, but otherwise I recommend switching to something more stable like ext4/reiserfs/etc. On 2/25/2012 5:55 PM, Brian J. Murrell wrote: I have a 5G /usr btrfs filesystem on a 3.0.0-12-generic kernel that is returning ENOSPC when it's only 75% full: FilesystemSize Used Avail Use% Mounted on /dev/mapper/rootvol-mint_usr 5.0G 2.8G 967M 75% /usr And yet I can't even unpack a linux-headers package on to it, which should be nowhere near 967MB. dpkg says it will need 10MB: $ sudo apt-get install -f Reading package lists... Done Building dependency tree Reading state information... Done Correcting dependencies... Done The following extra packages will be installed: linux-headers-3.0.0-16-generic The following NEW packages will be installed: linux-headers-3.0.0-16-generic 0 upgraded, 1 newly installed, 0 to remove and 2 not upgraded. 264 not fully installed or removed. Need to get 0 B/851 kB of archives. After this operation, 10.8 MB of additional disk space will be used. Do you want to continue [Y/n]? y (Reading database ... 180246 files and directories currently installed.) Unpacking linux-headers-3.0.0-16-generic (from .../linux-headers-3.0.0-16-generic_3.0.0-16.28_i386.deb) ... dpkg: error processing /var/cache/apt/archives/linux-headers-3.0.0-16-generic_3.0.0-16.28_i386.deb (--unpack): unable to install new version of `/usr/src/linux-headers-3.0.0-16-generic/include/config/dvb/tuner/dib0070.h': No space left on device And indeed, using dd I am able to create a 967MB file: $ sudo dd if=/dev/zero of=/usr/bigfile bs=1M count=1000 dd: writing `/usr/bigfile': No space left on device 967+0 records in 966+0 records out 1012924416 bytes (1.0 GB) copied, 16.1545 s, 62.7 MB/s strace yields this as the cause of the ENOSPC: 8213 rename(/usr/src/linux-headers-3.0.0-16-generic/include/config/dvb/tuner/dib0070.h.dpkg-new, /usr/src/linux-headers-3.0.0-16-generic/include/config/dvb/tuner/dib0070.hunfinished ... ... 8213... rename resumed )= -1 ENOSPC (No space left on device) So this starts to feel like some kind of inode count limitation. But I didn't think btrfs had inode count limitations. Here's the df stats on the filesystem: $ btrfs filesystem df /usr Data: total=3.22GB, used=3.22GB System, DUP: total=8.00MB, used=4.00KB System: total=4.00MB, used=0.00 Metadata, DUP: total=896.00MB, used=251.62MB Metadata: total=8.00MB, used=0.00 I don't know if that's useful or not. Any ideas? Cheers b. -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: filesystem full when it's not? out of inodes? huh?
On 2/25/2012 6:16 PM, Brian J. Murrell wrote: Others might know of a way of changing the allocation size to less than 1GB, but otherwise I recommend switching to something more stable like ext4/reiserfs/etc. So btrfs is still not yet suitable to be a root/usr/var filesystem, even in kernel 3.0.0? b. Nope, still in heavy development, though you should upgrade to 3.2. Also, the devs mentioned in several places it's not friendly to small drives, and I'm pretty sure 5GB is considered tiny. I don't think you need to separate /usr out to it's own disk. You could instead create a single drive with multiple subvolumes for /, /var, /usr, etc. When you have Ubuntu use btrfs for /, it creates @ and @home for / and /home, respectively, so it's a common phenomenon if you look for help. --Farz -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: filesystem full when it's not? out of inodes? huh?
On 2/25/2012 9:45 PM, Brian J. Murrell wrote: On 12-02-25 09:10 PM, Fahrzin Hemmati wrote: btrfs is horrible for small filesystems (like a 5GB drive). df -h says you have 967MB available, but btrfs (at least by default) allocates 1GB at a time to data/metadata. This means that your 10MB file is too big for the current allocation and requires a new data chunk, or another 1GB, which you don't have. So increasing the size of the filesystem should suffice then? How much bigger? 10G? Nope. still not big enough: # lvextend -L+1G /dev/rootvol/mint_usr; btrfs fi resize max /usr; df -h /usr Extending logical volume mint_usr to 10.00 GiB Logical volume mint_usr successfully resized Resize '/usr' of 'max' FilesystemSize Used Avail Use% Mounted on /dev/mapper/rootvol-mint_usr 10G 2.8G 6.0G 32% /usr test ~ # apt-get install -y -f Reading package lists... Done Building dependency tree Reading state information... Done Correcting dependencies... Done The following extra packages will be installed: linux-headers-3.0.0-16-generic The following NEW packages will be installed: linux-headers-3.0.0-16-generic 0 upgraded, 1 newly installed, 0 to remove and 2 not upgraded. 264 not fully installed or removed. Need to get 0 B/851 kB of archives. After this operation, 10.8 MB of additional disk space will be used. (Reading database ... 180246 files and directories currently installed.) Unpacking linux-headers-3.0.0-16-generic (from .../linux-headers-3.0.0-16-generic_3.0.0-16.28_i386.deb) ... dpkg: error processing /var/cache/apt/archives/linux-headers-3.0.0-16-generic_3.0.0-16.28_i386.deb (--unpack): unable to install new version of `/usr/src/linux-headers-3.0.0-16-generic/include/config/dvb/usb.h': No space left on device 20G maybe? Nope: # lvextend -L20G /dev/rootvol/mint_usr; btrfs fi resize max /usr; df -h /usr Extending logical volume mint_usr to 20.00 GiB Logical volume mint_usr successfully resized Resize '/usr' of 'max' FilesystemSize Used Avail Use% Mounted on /dev/mapper/rootvol-mint_usr 20G 2.8G 16G 15% /usr test ~ # apt-get install -y -f Reading package lists... Done Building dependency tree Reading state information... Done Correcting dependencies... Done The following extra packages will be installed: linux-headers-3.0.0-16-generic The following NEW packages will be installed: linux-headers-3.0.0-16-generic 0 upgraded, 1 newly installed, 0 to remove and 2 not upgraded. 264 not fully installed or removed. Need to get 0 B/851 kB of archives. After this operation, 10.8 MB of additional disk space will be used. (Reading database ... 180246 files and directories currently installed.) Unpacking linux-headers-3.0.0-16-generic (from .../linux-headers-3.0.0-16-generic_3.0.0-16.28_i386.deb) ... dpkg: error processing /var/cache/apt/archives/linux-headers-3.0.0-16-generic_3.0.0-16.28_i386.deb (--unpack): unable to install new version of `/usr/src/linux-headers-3.0.0-16-generic/include/config/ncpfs/packet/signing.h': No space left on device Maybe 50G? Yup: # apt-get install -y -f Reading package lists... Done Building dependency tree Reading state information... Done Correcting dependencies... Done The following extra packages will be installed: linux-headers-3.0.0-16-generic The following NEW packages will be installed: linux-headers-3.0.0-16-generic 0 upgraded, 1 newly installed, 0 to remove and 2 not upgraded. 264 not fully installed or removed. Need to get 0 B/851 kB of archives. After this operation, 10.8 MB of additional disk space will be used. (Reading database ... 180246 files and directories currently installed.) Unpacking linux-headers-3.0.0-16-generic (from .../linux-headers-3.0.0-16-generic_3.0.0-16.28_i386.deb) ... Setting up linux-image-3.0.0-16-generic (3.0.0-16.28) ... ... # df -h /usr FilesystemSize Used Avail Use% Mounted on /dev/mapper/rootvol-mint_usr 50G 2.8G 43G 7% /usr So I guess I need a 50G btrfs filesystem for 2.8G worth of data? Does that really seem right? I suppose to be fair it could have been some other value between 20G and 50G since I didn't test values in between. So still, I need some amount more than 20G of space to store 2.8G of data? Surely there is something going on here other than just btrfs sucks for small filesystems. b. You should have been fine with adding 1GB (really only 57MB), or at worst 2GB in case you were on the edge of both data and metadata. A btrfs dev might be able to debug the problem there, since your original problem seemed only that you couldn't allocate a new chunk. It might be a problem with btrfs filesystem resize? -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [RFC] btrfs auto snapshot
I'd like to vote for timestamp/timestamp-uuid as a sysadmin. The timestamp allows for easy conversion from clients' wants to actual commands: I need my data from two days ago is easy when I have timestamps to use. On 2/23/2012 10:05 PM, Anand Jain wrote: Thanks for the inputs. there is no clear winner as of now. Let me keep the uuid for now, if more sysadmin feel timestamp is better we could device it that way. -Anand -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [RFB] add LZ4 compression method to btrfs
The second link is broken, just remove scribe. from it. https://twitter.com/#!/otisg/status/148848850914902016 It causes a cross-domain error, not sure why he could see it though. On Wednesday, February 15, 2012 9:23:37 AM, Kok, Auke-jan H wrote: On Tue, Feb 14, 2012 at 1:47 PM, Hugo Chevrainhugochevr...@gmail.com wrote: Are you sure about these figures ? the difference seems too large. It's almost unbelievable. -- You should not, Mark Ruijter found the same for LessFS (http://www.lessfs.com/wordpress/? p=688) and there is also such finding into an Hadoop thread (https://scribe.twitter.com/#!/otisg/status/148848850914902016) The first link only shows results, not data. The second link is dead and just shows a dead twitter page, in two browsers. Science isn't hard folks! Just post the raw numbers so people can verify the results. Cheers, Auke -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Accidental formatting
I recently re-installed Ubuntu, and somewhere along the way the installer decided to clear out /var, which happens to be a separate btrfs device from /. When I do btfrs filesystem df /var it outputs this: Data: total=134.01GB, used=485.78 System, DUP: total=8.00MB, used=20.00KB System: total=4.00MB, used=0.00 Metadata, DUP: total=1.62GB, used=6.87MB Metadata: total=8.00MB, used=0.00 The reserved Data, 134GB, resembles closely the amount of data on my drive before the formatting. Therefore, I believe what happened was the installer didn't format /var, just cleared out the files. I didn't properly backup /var, but I have important files on it. Is there a way to have btrfs look around the reserved metadata area for orphaned files and get them back? Thanks! Any help is greatly appreciated! Fahrzin Hemmati -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
