Re: [PATCH 0/3] Btrfs: save free space cache to the disk
Josef Bacik wrote (ao): This patch series introduces the ability for btrfs to store the free space cache ondisk to make the caching of a block group much quicker. Previously we had to search the entire extent-tree to look for gaps everytime we wanted to allocate in a block group. This approach instead dumps all of the free space cache to disk for every dirtied block group each time we commit the transaction. This is a disk format change, but in order to use the feature you will have to mount with -o space_cache, and then from then on you won't be able to use old kernels with your filesystem. Will this go into a future version of btrfs? If so, would it make sense to include other changes that would require a format change? Sander -- Humilis IT Services and Solutions http://www.humilis.net -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Question of stability
On Mon, Sep 20, 2010 at 11:00:08AM +, Lubos Kolouch wrote: No, not stable! Again, after powerloss, I have *two* damaged btrfs filesystems. Please tell me more about your system. I do extensive power fail testing here without problems, and corruptions after powerloss are very often caused by the actual hardware. So, what kind of drives do you have, do they have writeback caching on, and what are you layering on top of the drive between btrfs and the kernel? -chris /home and also the backup volume, as the backup was running :( Both show parent transid verify failed on wanted . found and the volumes cannot be mounted. So until there is a way to somehow recover from this type of failures, speaking about stability is a joke Lubos Kolouch -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Question of stability
On Mon, 20 Sep 2010 07:30:57 -0400, Chris Mason wrote: On Mon, Sep 20, 2010 at 11:00:08AM +, Lubos Kolouch wrote: No, not stable! Again, after powerloss, I have *two* damaged btrfs filesystems. Please tell me more about your system. I do extensive power fail testing here without problems, and corruptions after powerloss are very often caused by the actual hardware. So, what kind of drives do you have, do they have writeback caching on, and what are you layering on top of the drive between btrfs and the kernel? -chris Hello Chris, The system is running with 128GB SDD Samsung disk. The partition is encrypted with LUKS and on top btrfs filesystem is created. The backup disk is a 160GB WD notebook disk connected via IDE-USB cable, whole formatted with LUKS and btrfs on top. Actually, if there was any way (non-standard) how to mount the system and recover even part of the data, I would be very grateful. Lubos -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Question of stability
On Mon, Sep 20, 2010 at 12:10:08PM +, Lubos Kolouch wrote: On Mon, 20 Sep 2010 07:30:57 -0400, Chris Mason wrote: On Mon, Sep 20, 2010 at 11:00:08AM +, Lubos Kolouch wrote: No, not stable! Again, after powerloss, I have *two* damaged btrfs filesystems. Please tell me more about your system. I do extensive power fail testing here without problems, and corruptions after powerloss are very often caused by the actual hardware. So, what kind of drives do you have, do they have writeback caching on, and what are you layering on top of the drive between btrfs and the kernel? -chris Hello Chris, The system is running with 128GB SDD Samsung disk. The partition is encrypted with LUKS and on top btrfs filesystem is created. Ok, we're seeing consistent reports of corruptions after power failure with dm-crypt. The barriers must not be getting down to the device. The backup disk is a 160GB WD notebook disk connected via IDE-USB cable, whole formatted with LUKS and btrfs on top. Actually, if there was any way (non-standard) how to mount the system and recover even part of the data, I would be very grateful. We can definitely try. Please send me email with the errors from btrfsck and I'll work on a patch that gets you read only access. -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Question of stability
On Mon, 20 Sep 2010 07:30:57 -0400 Chris Mason chris.ma...@oracle.com wrote: On Mon, Sep 20, 2010 at 11:00:08AM +, Lubos Kolouch wrote: No, not stable! Again, after powerloss, I have *two* damaged btrfs filesystems. Please tell me more about your system. I do extensive power fail testing here without problems, and corruptions after powerloss are very often caused by the actual hardware. So, what kind of drives do you have, do they have writeback caching on, and what are you layering on top of the drive between btrfs and the kernel? -chris Chris, the actual way how a fs was damaged must not be relevant. From a new fs design one should expect the tree can be mounted no matter what corruption took place up to the case where the fs is indeed empty after mounting because it was completely corrupted. If parts were corrupt then the fs should either be able to assist the user in correcting the damages _online_ or at least simply exclude the damaged fs parts from the actual mounted fs tree. The basic thought must be show me what you have and not shit, how do I get access to the working but not mountable fs parts again?. Would you buy a car that refuses to drive if the ash tray is broken? -- Regards, Stephan -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Question of stability
On Mon, Sep 20, 2010 at 02:21:15PM +0200, Stephan von Krawczynski wrote: On Mon, 20 Sep 2010 07:30:57 -0400 Chris Mason chris.ma...@oracle.com wrote: On Mon, Sep 20, 2010 at 11:00:08AM +, Lubos Kolouch wrote: No, not stable! Again, after powerloss, I have *two* damaged btrfs filesystems. Please tell me more about your system. I do extensive power fail testing here without problems, and corruptions after powerloss are very often caused by the actual hardware. So, what kind of drives do you have, do they have writeback caching on, and what are you layering on top of the drive between btrfs and the kernel? -chris Chris, the actual way how a fs was damaged must not be relevant. From a new fs design one should expect the tree can be mounted no matter what corruption took place up to the case where the fs is indeed empty after mounting because it was completely corrupted. If parts were corrupt then the fs should either be able to assist the user in correcting the damages _online_ or at least simply exclude the damaged fs parts from the actual mounted fs tree. The basic thought must be show me what you have and not shit, how do I get access to the working but not mountable fs parts again?. Would you buy a car that refuses to drive if the ash tray is broken? Definitely, this has always been one of our goals. Step one is a better btrfsck, which is in progress now. Step two is being able to do more of this online. -chris -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Question of stability
On Mon, Sep 20, 2010 at 10:12 AM, K. Richard Pixley r...@noir.com wrote: On 9/18/10 17:43 , C Anthony Risinger wrote: I remeber someone recently using it for continuous build servers successfully That's probably me and I wouldn't call the effort successful yet. There are at least several outstanding problems including the limit on the number of links to a file. More on all of these when I return from vacation next week. yes i think you're right; i actually was going to add that, but i accidentally hit 'send' on my mobile right after typing 'successfully' and never bothered to follow up with the last 2 or 3 sentences i had planned :-), meh. at any rate, while there is no doubt some issues (esp. with more aggressive/non-typical setups), i still think there are many who are using it successfully; at least this is the impression i get from my statistically irrelevant sampling of the various forums, lists, etc. i frequent. C Anthony -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Question of stability
On 9/18/10 17:43 , C Anthony Risinger wrote: I remeber someone recently using it for continuous build servers successfully That's probably me and I wouldn't call the effort successful yet. There are at least several outstanding problems including the limit on the number of links to a file. More on all of these when I return from vacation next week. --rich -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Performance degradation
Hi, I have a btrfs backup system that has been running great. Its been backing up 6 systems with 500-750 gigs of data using Rsync and snapshots. Every night I snapshot the backup and overwrite it. Recently it has been unable to complete the backups. Its running s slow that it can barely finish 1 backup a day. When I first started backing up the systems they would all complete in at the most 8 hours. I noticed it started happening after I deleted a large number of snapshots. This caused the cleaner process to run for a few days. After it finished the filesystem has still been to slow to use. I'm running kernel 2.6.36-rc4. Filesystem was built with -m single -d single and using the compress mount option. Block device is an Areca RAID5 with 4 Western Digital 2TB drives. -- Dave Cundiff System Administrator A2Hosting, Inc http://www.a2hosting.com -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Performance degradation
Do you have any kernel processes running constantly? Any taking near 100% of a CPU? --rich On Sep 20, 2010, at 14:24, Dave Cundiff syshack...@gmail.com wrote: Hi, I have a btrfs backup system that has been running great. Its been backing up 6 systems with 500-750 gigs of data using Rsync and snapshots. Every night I snapshot the backup and overwrite it. Recently it has been unable to complete the backups. Its running s slow that it can barely finish 1 backup a day. When I first started backing up the systems they would all complete in at the most 8 hours. I noticed it started happening after I deleted a large number of snapshots. This caused the cleaner process to run for a few days. After it finished the filesystem has still been to slow to use. I'm running kernel 2.6.36-rc4. Filesystem was built with -m single -d single and using the compress mount option. Block device is an Areca RAID5 with 4 Western Digital 2TB drives. -- Dave Cundiff System Administrator A2Hosting, Inc http://www.a2hosting.com -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Performance degradation
On Mon, Sep 20, 2010 at 9:19 PM, K. Richard Pixley r...@noir.com wrote: Do you have any kernel processes running constantly? Any taking near 100% of a CPU? --rich Now that I look again, it seems that the btrfs-cleaner process is still running. CPU isn't much of an issue on this box. Its a dual quad E5620. 8 cores, 16 threads. Looks like most of my load is wait from the cleaner process hammering the disks. I deleted about 10 snapshots I think. Should this take 4 days to clean up from that? Cpu0 : 0.0%us, 0.2%sy, 0.0%ni, 85.1%id, 14.7%wa, 0.0%hi, 0.0%si, 0.0%st Cpu1 : 0.0%us, 0.4%sy, 0.0%ni, 66.9%id, 32.6%wa, 0.0%hi, 0.1%si, 0.0%st Cpu2 : 0.0%us, 0.2%sy, 0.0%ni, 88.7%id, 11.1%wa, 0.0%hi, 0.0%si, 0.0%st Cpu3 : 0.0%us, 0.1%sy, 0.0%ni, 96.7%id, 3.2%wa, 0.0%hi, 0.0%si, 0.0%st Cpu4 : 0.0%us, 0.2%sy, 0.0%ni, 92.3%id, 7.5%wa, 0.0%hi, 0.0%si, 0.0%st Cpu5 : 0.0%us, 0.1%sy, 0.0%ni, 97.6%id, 2.4%wa, 0.0%hi, 0.0%si, 0.0%st Cpu6 : 0.0%us, 0.1%sy, 0.0%ni, 98.5%id, 1.5%wa, 0.0%hi, 0.0%si, 0.0%st Cpu7 : 0.0%us, 0.0%sy, 0.0%ni, 99.8%id, 0.2%wa, 0.0%hi, 0.0%si, 0.0%st Cpu8 : 0.0%us, 0.1%sy, 0.0%ni, 95.7%id, 4.3%wa, 0.0%hi, 0.0%si, 0.0%st Cpu9 : 0.0%us, 1.6%sy, 0.0%ni, 30.2%id, 68.2%wa, 0.0%hi, 0.0%si, 0.0%st Cpu10 : 0.0%us, 0.0%sy, 0.0%ni, 95.1%id, 4.8%wa, 0.0%hi, 0.0%si, 0.0%st Cpu11 : 0.0%us, 0.0%sy, 0.0%ni, 99.6%id, 0.4%wa, 0.0%hi, 0.0%si, 0.0%st Cpu12 : 0.0%us, 0.4%sy, 0.0%ni, 83.8%id, 15.7%wa, 0.0%hi, 0.0%si, 0.0%st Cpu13 : 0.0%us, 0.0%sy, 0.0%ni, 98.1%id, 1.8%wa, 0.0%hi, 0.0%si, 0.0%st Cpu14 : 0.0%us, 0.0%sy, 0.0%ni, 99.7%id, 0.3%wa, 0.0%hi, 0.0%si, 0.0%st Cpu15 : 0.0%us, 0.0%sy, 0.0%ni, 99.8%id, 0.2%wa, 0.0%hi, 0.0%si, 0.0%st Time: 12:24:01 AM Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await svctm %util sdb 16.25 6.49 127.45 330.65 1173.03 3946.46 11.1870.55 154.01 2.13 97.71 Time: 12:24:21 AM Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await svctm %util sdb 5.20 2.70 92.55 270.70 782.00 2795.20 9.8570.12 189.42 2.69 97.85 Time: 12:24:41 AM Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await svctm %util sdb 0.2511.90 5.20 863.3543.60 9391.60 10.86 144.84 168.01 1.15 100.00 Time: 12:25:01 AM Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await svctm %util sdb 17.50 0.60 204.35 49.70 1776.00 524.00 9.0518.64 74.23 3.74 95.03 -- Dave Cundiff System Administrator A2Hosting, Inc http://www.a2hosting.com -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
