btrfs and load (sys)
I've been using btrfs for two months now. Every day between 02:00 and 08:00 I rsync some 300GB data (milions of files) to btrfs device and then make snapshot. Next day i rsync again 300GG little changed (rsync in place). First days it worked perfectly. Then loadavg (sys load) started to rise. Now, after 60 days of rsync+shapshots sys load is so high that whole server becames quite unresponsive betweend 02:00 and 08:00. I never deleted any shapshot (yet) to its not rebalancing thing. What all those btrfs benchmark does not tell you that its performance decreases (sys load increases) with growing size of btree. Creating btrfs filesystem is instantaneous because initial tree is just nothing... I tried using btrfs and nilfs2 for a half year now and I would never trust a byte of production data to it. -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: btrfs and load (sys)
Yes, yesterday I've umouted partition and re-mounted it. Nothing has changed this night. You can look at my load graphs here: http://img42.imageshack.us/img42/4661/33737291.png http://img210.imageshack.us/img210/2742/46527625.png On the second one blue is SYS load. I bet you can reasily spot time when backup to btrfs partition was running :) I doubt its a bug in btrfs. It is just broken concept :( # uname -a Linux sv15.vipserv.org 3.0.4--std-ipv6-64 #1 SMP Mon Oct 10 01:07:40 CEST 2011 x86_64 x86_64 x86_64 GNU/Linux -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: btrfs and load (sys)
I've rebooted server and run backup to btrfs partition again. It seems that problem is gone, high sys load does not occur now. So it is some bug in btrfs... Before reboot server had 30 days uptime so its really not much. -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
is space really freed after deleting large subvolume?
I'm wondering is space really freed after deleting large subvolume? Will space be immediately available to other data like other subvolumes? -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: is space really freed after deleting large subvolume?
No, subvolume deletion is done in the background, and the space recovered will be returned for use relatively slowly. This is because the FS has to go through the metadata, updating reference counts for each extent in the subvolume to work out whether the space can be recovered or not. And what affect this rate of space reclaiming? When does it happen? Also I guess that if it happens it must lower overall io performance and rise loadavg ...? -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: is space really freed after deleting large subvolume?
And what affect this rate of space reclaiming? When does it happen? Also I guess that if it happens it must lower overall io performance and rise loadavg ...? and finaly is the performance overhead same as deleting same number of files in system like ext3? When you delete large number of files in ext3 (or any other system) you are looking at huge io traffic during deletion. You are saying that in btrfs if you delete subvolume (that had many files) you don't get free space instantenously so when and is prerformance overhead basicly overaly SAME but in time? higher? lower? -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: is space really freed after deleting large subvolume?
Only umount does, and it can take a very long time if you have deleted a large (differing a lot) subvolume just before that. does it mean that I won't be able to cleanly reboot machine after deleting subvolume with milions of files? And most important question is if deleting btrfs subvolume with 5mln requiress less io operations than deleting 5mln of files on normal filesystem (like ext3) (by rm -rf)??? -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: is space really freed after deleting large subvolume?
btrfs like ext4 has support for extents, which can be any size, so typically if you delete a large file, then it occupies only one extent, so only that one extent needs to be marked as free, so a lot less I/O. I know, I know. Issue is with many (small) files. If you delete a large number of files, then there is no avoiding the fact that a lot of metadata needs to be updated. In this respect btrfs is unlikely to be significantly faster than any other filing system. Are you sure? That would mean that instant deletion of subvolume in btrfs is actualy taking ??SAME?? time and io power in BACKGROUND like deleting by rm -rf in any filesystem. Common misconception would be that subvolume deletion is much more efficient and near zero time consuming. I think it is very important to clear that up. -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: is space really freed after deleting large subvolume?
Just a idea: I don't know if btrfs works like that or not but idea would be that b-tree filesystem should be able to loose or discard branches be removing a node. Cut a tree node and branches will fall off - and get overwrited as empty space sometime in future (just like during data deletion). If subvolume is filesystem in filesystem (don't know to what extend this is true in btrfs) then sub-filesystem could be re-formated but severing it links to its branches. Reason for this if of course fast and not io costly deletion of large number of files. -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
recursive subvolume delete
btrfs is missing recursive subvolume delete. You can't delete subvolume if there is other subvolumes/snapshots in it. Also I think there are no real tools to find out which directories are subvolumes/snapshots -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: feature request: btrfs-image without zeroing data
2011/7/11 Stephane Chazelas stephane_chaze...@yahoo.fr: 2011-07-11 02:00:51 +0200, krz...@gmail.com : Documentation says that btrfs-image zeros data. Feature request is for disabling this. btrfs-image could be used to copy filesystem to another drive (for example with snapshots, when copying it file by file would take much longer time or acctualy was not possible (snapshots)). btrfs-image in turn could be used to actualy shrink loop devices/sparse file containing btrfs - by copying filesystem to new loop device/sparse file. Also it would be nice if copying filesystem could occour without intermediate dump to a file... [...] I second that. See also http://thread.gmane.org/gmane.comp.file-systems.btrfs/9675/focus=9820 for a way to transfer btrfs fs. (Add a layer of copy-on-write on the original devices (LVM snapshots, nbd/qemu-nbd cow...), btrfs add the new device(s) and then btrfs del of the cow'ed original devices. -- Stephane Copying on block level (dd, lvm) is old trick, however this takes same ammount of time regardless of actual space used in filesystem. Hence this feature request. Images inside filesystem can copy only actualy used data and metadata, which dramaticly reduces copy times in large volumes that are not filled up... -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
raid1
I wanted to confirm that btrfs will continue to work on raid1 when one of devices will be gone. dd if=/dev/null of=img0 bs=1 seek=2G dd if=/dev/null of=img1 bs=1 seek=2G mkfs.btrfs -d raid1 -m raid1 img0 img1 losetup /dev/loop1 img0 losetup /dev/loop2 img1 mkdir dir mount -t btrfs /dev/loop1 dir btrfs device scan mount -t btrfs /dev/loop1 dir echo abc dir/a.txt umount dir losetup -d /dev/loop2 btrfs device scan mount -t btrfs /dev/loop1 dir mount: wrong fs type, bad option, bad superblock on /dev/loop1, missing codepage or other error In some cases useful info is found in syslog - try dmesg | tail or so am I missing some nuance? -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
after mounting with -o degraded: ioctl: LOOP_CLR_FD: Device or resource busy
dd if=/dev/null of=img5 bs=1 seek=2G dd if=/dev/null of=img6 bs=1 seek=2G mkfs.btrfs -d raid1 -m raid1 img5 img6 losetup /dev/loop4 img5 losetup /dev/loop5 img6 btrfs device scan mount -t btrfs /dev/loop4 dir umount dir losetup -d /dev/loop5 mount -t btrfs -o degraded /dev/loop4 dir umount dir losetup -d /dev/loop4 ioctl: LOOP_CLR_FD: Device or resource busy mkfs.ext3 /dev/loop4 mke2fs 1.39 (29-May-2006) /dev/loop4 is apparently in use by the system; will not make a filesystem here! this only happens after mouting with -o degraded. loopback device is unusable until next reboot -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: raid1
Thanks. I don't see reason why this needs another mount switch. This would fail to start whole system in / parition was btrfs raid1, with no reason to do so. -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
feature request: btrfs-image without zeroing data
Documentation says that btrfs-image zeros data. Feature request is for disabling this. btrfs-image could be used to copy filesystem to another drive (for example with snapshots, when copying it file by file would take much longer time or acctualy was not possible (snapshots)). btrfs-image in turn could be used to actualy shrink loop devices/sparse file containing btrfs - by copying filesystem to new loop device/sparse file. Also it would be nice if copying filesystem could occour without intermediate dump to a file... -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
btrfs frozen solid with postgresql dbs
kernel: 2.6.37.6 At least twice I've experienced problems when using btrfs with ssd for postgresql db storage. Server gets frozen, i even can't kill it using kill -9, I cant chown postgresql db dir (it never gets done). If I initiate copy operation of postgresql db dir to another partition (without actually switching to using it) everything starts to work fine for some time... I've been using btrfs for mysql storage dir also. I did not observe any problems so far. -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
files on not mounted subvolumes - mysql complains read only
When I've used btrfs subvolume, without separately mouting it, for mysql database dir I've got Fatal error: Can't open and lock privilege tables: Table 'host' is read only when starting mysql. When I've put mysql database dir in root of btrfs, not a subvolume, mysql works fine. I've checked file for host table, but I could write to it no problem (on subvolume without mouting). I dont know what could be the problem. I did not check it when mouting a subvolume separately -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
checkpoints to often
There should be a way to make automatic checkpoints less frequent. On the busy ssd I have about 7 cp every second. If it were for example once every 5 minutes then one could set garbage removal every few days. Also garbage removal should have option to clean only if like 90% of drive is used. there is no need to remove cp-s when there is still many GB of space left. -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
