btrfs and load (sys)

2011-11-23 Thread krz...@gmail.com
I've been using btrfs for two months now. Every day between 02:00 and
08:00 I rsync some 300GB data (milions of files) to btrfs device and
then make snapshot. Next day i rsync again 300GG little changed (rsync
in place). First days it worked perfectly. Then loadavg (sys load)
started to rise. Now, after 60 days of rsync+shapshots sys load is so
high that whole server becames quite unresponsive betweend 02:00 and
08:00. I never deleted any shapshot (yet) to its not rebalancing
thing.
What all those btrfs benchmark does not tell you that its performance
decreases (sys load increases) with growing size of btree. Creating
btrfs filesystem is instantaneous because initial tree is just
nothing...
I tried using btrfs and nilfs2 for a half year now and I would never
trust a byte of production data to it.
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: btrfs and load (sys)

2011-11-23 Thread krz...@gmail.com
Yes, yesterday I've umouted partition and re-mounted it. Nothing has
changed this night.

You can look at my load graphs here:
http://img42.imageshack.us/img42/4661/33737291.png
http://img210.imageshack.us/img210/2742/46527625.png

On the second one blue is SYS load. I bet you can reasily spot time
when backup to btrfs partition was running :)

I doubt its a bug in btrfs. It is just broken concept :(


# uname -a
Linux sv15.vipserv.org 3.0.4--std-ipv6-64 #1 SMP Mon Oct 10
01:07:40 CEST 2011 x86_64 x86_64 x86_64 GNU/Linux
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: btrfs and load (sys)

2011-11-23 Thread krz...@gmail.com
I've rebooted server and run backup to btrfs partition again. It seems
that problem is gone, high sys load does not occur now. So it is some
bug in btrfs... Before reboot server had 30 days uptime so its really
not much.
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


is space really freed after deleting large subvolume?

2011-10-13 Thread krz...@gmail.com
I'm wondering is space really freed after deleting large subvolume?
Will space be immediately available to other data like other
subvolumes?
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: is space really freed after deleting large subvolume?

2011-10-13 Thread krz...@gmail.com
   No, subvolume deletion is done in the background, and the space
 recovered will be returned for use relatively slowly. This is because
 the FS has to go through the metadata, updating reference counts for
 each extent in the subvolume to work out whether the space can be
 recovered or not.


And what affect this rate of space reclaiming? When does it happen?
Also I guess that if it happens it must lower overall io performance
and rise loadavg ...?
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: is space really freed after deleting large subvolume?

2011-10-13 Thread krz...@gmail.com
 And what affect this rate of space reclaiming? When does it happen?
 Also I guess that if it happens it must lower overall io performance
 and rise loadavg ...?


and finaly is the performance overhead same as deleting same number of
files in system like ext3? When you delete large number of files in
ext3 (or any other system) you are looking at huge io traffic during
deletion. You are saying that in btrfs if you delete subvolume (that
had many files) you don't get free space instantenously so when and is
prerformance overhead basicly overaly SAME but in time? higher? lower?
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: is space really freed after deleting large subvolume?

2011-10-13 Thread krz...@gmail.com
Only umount does, and it can take a very long time if you have deleted a 
large
(differing a lot) subvolume just before that.

does it mean that I won't be able to cleanly reboot machine after
deleting subvolume with milions of files?

And most important question is if deleting btrfs subvolume with 5mln
requiress less io operations than deleting 5mln of files on normal
filesystem (like ext3) (by rm -rf)???
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: is space really freed after deleting large subvolume?

2011-10-13 Thread krz...@gmail.com
btrfs like ext4 has support for extents, which can be any size, so
typically if you delete a large file, then it occupies only one extent,
so only that one extent needs to be marked as free, so a lot less I/O.

I know, I know. Issue is with many (small) files.

 If you delete a large number of files, then there is no avoiding the
 fact that a lot of metadata needs to be updated. In this respect btrfs
 is unlikely to be significantly faster than any other filing system.

Are you sure? That would mean that instant deletion of subvolume in
btrfs is actualy taking ??SAME?? time and io power in BACKGROUND like
deleting by rm -rf in any filesystem.
Common misconception would be that subvolume deletion is much more
efficient and near zero time consuming. I think it is very important
to clear that up.
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: is space really freed after deleting large subvolume?

2011-10-13 Thread krz...@gmail.com
Just a idea: I don't know if btrfs works like that or not but idea
would be that b-tree filesystem should be able to loose or discard
branches be removing a node. Cut a tree node and branches will fall
off - and get overwrited as empty space sometime in future (just like
during data deletion).
If subvolume is filesystem in filesystem (don't know to what extend
this is true in btrfs) then sub-filesystem could be re-formated but
severing it links to its branches.
Reason for this if of course fast and not io costly deletion of large
number of files.
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


recursive subvolume delete

2011-10-02 Thread krz...@gmail.com
btrfs is missing recursive subvolume delete. You can't delete
subvolume if there is other subvolumes/snapshots in it. Also I think
there are no real tools to find out which directories are
subvolumes/snapshots
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: feature request: btrfs-image without zeroing data

2011-07-11 Thread krz...@gmail.com
2011/7/11 Stephane Chazelas stephane_chaze...@yahoo.fr:
 2011-07-11 02:00:51 +0200, krz...@gmail.com :
 Documentation says that btrfs-image zeros data. Feature request is for
 disabling this. btrfs-image could be used to copy filesystem to
 another drive (for example with snapshots, when copying it file by
 file would take much longer time or acctualy was not possible
 (snapshots)). btrfs-image in turn could be used to actualy shrink loop
 devices/sparse file containing btrfs - by copying filesystem to new
 loop device/sparse file.

 Also it would be nice if copying filesystem could occour without
 intermediate dump to a file...
 [...]

 I second that.

 See also
 http://thread.gmane.org/gmane.comp.file-systems.btrfs/9675/focus=9820
 for a way to transfer btrfs fs.

 (Add a layer of copy-on-write on the original devices (LVM
 snapshots, nbd/qemu-nbd cow...), btrfs add the new device(s)
 and then btrfs del of the cow'ed original devices.

 --
 Stephane


Copying on block level (dd, lvm) is old trick, however this takes same
ammount of time regardless of actual space used in filesystem. Hence
this feature request. Images inside filesystem can copy only actualy
used data and metadata, which dramaticly reduces copy times in large
volumes that are not filled up...
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


raid1

2011-07-11 Thread krz...@gmail.com
I wanted to confirm that btrfs will continue to work on raid1 when one
of devices will be gone.

dd if=/dev/null of=img0 bs=1 seek=2G
dd if=/dev/null of=img1 bs=1 seek=2G
mkfs.btrfs -d raid1 -m raid1 img0 img1
losetup /dev/loop1 img0
losetup /dev/loop2 img1
mkdir dir
mount -t btrfs /dev/loop1 dir
btrfs device scan
mount -t btrfs /dev/loop1 dir
echo abc  dir/a.txt
umount dir
losetup -d /dev/loop2
btrfs device scan
mount -t btrfs /dev/loop1 dir
mount: wrong fs type, bad option, bad superblock on /dev/loop1,
   missing codepage or other error
   In some cases useful info is found in syslog - try
   dmesg | tail  or so

am I missing some nuance?
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


after mounting with -o degraded: ioctl: LOOP_CLR_FD: Device or resource busy

2011-07-11 Thread krz...@gmail.com
dd if=/dev/null of=img5 bs=1 seek=2G
dd if=/dev/null of=img6 bs=1 seek=2G
mkfs.btrfs -d raid1 -m raid1 img5 img6
losetup /dev/loop4 img5
losetup /dev/loop5 img6
btrfs device scan
mount -t btrfs /dev/loop4 dir
umount dir
losetup -d /dev/loop5
mount -t btrfs -o degraded /dev/loop4 dir
umount dir
losetup -d /dev/loop4
ioctl: LOOP_CLR_FD: Device or resource busy
mkfs.ext3 /dev/loop4
mke2fs 1.39 (29-May-2006)
/dev/loop4 is apparently in use by the system; will not make a filesystem here!

this only happens after mouting with -o degraded. loopback device is
unusable until next reboot
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: raid1

2011-07-11 Thread krz...@gmail.com
Thanks.

I don't see reason why this needs another mount switch. This would
fail to start whole system in / parition was btrfs raid1, with no
reason to do so.
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


feature request: btrfs-image without zeroing data

2011-07-10 Thread krz...@gmail.com
Documentation says that btrfs-image zeros data. Feature request is for
disabling this. btrfs-image could be used to copy filesystem to
another drive (for example with snapshots, when copying it file by
file would take much longer time or acctualy was not possible
(snapshots)). btrfs-image in turn could be used to actualy shrink loop
devices/sparse file containing btrfs - by copying filesystem to new
loop device/sparse file.

Also it would be nice if copying filesystem could occour without
intermediate dump to a file...
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


balance on only one subvolume

2011-07-08 Thread krz...@gmail.com
btrfs balance takes very long time on large filesystems. Possibility
of running balance on only one subvolume or running balance to reclaim
only after deleing subvolume would be usefull.
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


btrfs frozen solid with postgresql dbs

2011-04-16 Thread krz...@gmail.com
kernel: 2.6.37.6

At least twice I've experienced problems when using btrfs with ssd for
postgresql db storage. Server gets frozen, i even can't kill it using
kill -9, I cant chown postgresql db dir (it never gets done). If I
initiate copy operation of postgresql db dir to another partition
(without actually switching to using it) everything starts to work
fine for some time...
I've been using btrfs for mysql storage dir also. I did not observe
any problems so far.
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


files on not mounted subvolumes - mysql complains read only

2011-04-05 Thread krz...@gmail.com
When I've used btrfs subvolume, without separately mouting it, for
mysql database dir I've got
Fatal error: Can't open and lock privilege tables: Table 'host' is read only
when starting mysql. When I've put mysql database dir in root of
btrfs, not a subvolume, mysql works fine.
I've checked file for host table, but I could write to it no problem
(on subvolume without mouting). I dont know what could be the problem.
I did not check it when mouting a subvolume separately
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


checkpoints to often

2011-03-31 Thread krz...@gmail.com
There should be a way to make automatic checkpoints less frequent. On
the busy ssd I have about 7 cp every second. If it were for example
once every 5 minutes then one could set garbage removal every few
days.
Also garbage removal should have option to clean only if like 90% of
drive is used. there is no need to remove cp-s when there is still
many GB of space left.
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html