getdents - ext4 vs btrfs performance

2012-02-29 Thread Jacek Luczak
Hi All, Long story short: We've found that operations on a directory structure holding many dirs takes ages on ext4. The Question: Why there's that huge difference in ext4 and btrfs? See below test results for real values. Background: I had to backup a Jenkins directory holding workspace for

chris.ma...@oracle.com, Al Viro v...@zeniv.linux.org.uk, Ted Ts'o ty...@mit.edu

2012-02-29 Thread Jacek Luczak
Hi All, /*Sorry for sending incomplete email, hit wrong button :) */ Long story short: We've found that operations on a directory structure holding many dirs takes ages on ext4. The Question: Why there's that huge difference in ext4 and btrfs? See below test results for real values.

Re: getdents - ext4 vs btrfs performance

2012-02-29 Thread Chris Mason
On Wed, Feb 29, 2012 at 02:31:03PM +0100, Jacek Luczak wrote: Hi All, Long story short: We've found that operations on a directory structure holding many dirs takes ages on ext4. The Question: Why there's that huge difference in ext4 and btrfs? See below test results for real values.

getdents - ext4 vs btrfs performance

2012-02-29 Thread Jacek Luczak
Hi All, /*Sorry for sending incomplete email, hit wrong button :) I guess I can't use Gmail */ Long story short: We've found that operations on a directory structure holding many dirs takes ages on ext4. The Question: Why there's that huge difference in ext4 and btrfs? See below test results

Re: getdents - ext4 vs btrfs performance

2012-02-29 Thread Jacek Luczak
Hi Chris, the last one was borked :) Please check this one. -jacek 2012/2/29 Jacek Luczak difrost.ker...@gmail.com: Hi All, /*Sorry for sending incomplete email, hit wrong button :) I guess I can't use Gmail */ Long story short: We've found that operations on a directory structure

Re: Btrfs Storage Array Corrupted

2012-02-29 Thread Chris Mason
On Tue, Feb 28, 2012 at 09:36:35PM -0600, Travis Shivers wrote: I upgraded my kernel so my version is now: Linux server 3.3.0-030300rc5-generic #201202251535 SMP Sat Feb 25 20:36:29 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux The problem has not been solved and I still get the previous errors.

Re: getdents - ext4 vs btrfs performance

2012-02-29 Thread Lukas Czerner
On Wed, 29 Feb 2012, Chris Mason wrote: On Wed, Feb 29, 2012 at 02:31:03PM +0100, Jacek Luczak wrote: Hi All, Long story short: We've found that operations on a directory structure holding many dirs takes ages on ext4. The Question: Why there's that huge difference in ext4 and

Re: getdents - ext4 vs btrfs performance

2012-02-29 Thread Jacek Luczak
2012/2/29 Jacek Luczak difrost.ker...@gmail.com: Hi Chris, the last one was borked :) Please check this one. -jacek 2012/2/29 Jacek Luczak difrost.ker...@gmail.com: Hi All, /*Sorry for sending incomplete email, hit wrong button :) I guess I can't use Gmail */ Long story short: We've

Re: getdents - ext4 vs btrfs performance

2012-02-29 Thread Jacek Luczak
2012/2/29 Jacek Luczak difrost.ker...@gmail.com: 2012/2/29 Jacek Luczak difrost.ker...@gmail.com: Hi Chris, the last one was borked :) Please check this one. -jacek 2012/2/29 Jacek Luczak difrost.ker...@gmail.com: Hi All, /*Sorry for sending incomplete email, hit wrong button :) I guess

Re: getdents - ext4 vs btrfs performance

2012-02-29 Thread Chris Mason
On Wed, Feb 29, 2012 at 08:51:58AM -0500, Chris Mason wrote: On Wed, Feb 29, 2012 at 02:31:03PM +0100, Jacek Luczak wrote: Ext4 results: | Type | 2.6.39.4-3 | 3.2.7 | Dir cnt | 17m 40sec | 11m 20sec | File cnt | 17m 36sec | 11m 22sec | Copy| 1h 28m| 1h 27m |

Re: getdents - ext4 vs btrfs performance

2012-02-29 Thread Chris Mason
On Wed, Feb 29, 2012 at 03:07:45PM +0100, Jacek Luczak wrote: [ btrfs faster than ext for find and cp -a ] 2012/2/29 Jacek Luczak difrost.ker...@gmail.com: I will try to answer the question from the broken email I've sent. @Lukas, it was always a fresh FS on top of LVM logical volume.

Re: getdents - ext4 vs btrfs performance

2012-02-29 Thread Jacek Luczak
2012/2/29 Chris Mason chris.ma...@oracle.com: On Wed, Feb 29, 2012 at 03:07:45PM +0100, Jacek Luczak wrote: [ btrfs faster than ext for find and cp -a ] 2012/2/29 Jacek Luczak difrost.ker...@gmail.com: I will try to answer the question from the broken email I've sent. @Lukas, it was

RE: Set nodatacow per file?

2012-02-29 Thread Kyle Gates
Actually it is possible. Check out David's response to my question from some time ago: http://permalink.gmane.org/gmane.comp.file-systems.btrfs/14227 this was a quick aid, please see attached file for an updated tool to set the file flags, now added 'z' for NOCOMPRESS flag, and supports

Re: Set nodatacow per file?

2012-02-29 Thread cwillu
I get the following errors when running fileflags on large (2GB) database files: open(): No such file or directory open(): Value too large for defined data type http://www.gnu.org/software/coreutils/faq/#Value-too-large-for-defined-data-type The message Value too large for defined data

[PATCH] Btrfs: stop silently switching single chunks to raid0 on balance

2012-02-29 Thread Ilya Dryomov
This has been causing a lot of confusion for quite a while now and a lot of users were surprised by this (some of them were even stuck in a ENOSPC situation which they couldn't easily get out of). The addition of restriper gives users a clear choice between raid0 and drive concat setup so there's

Re: Btrfs Storage Array Corrupted

2012-02-29 Thread Travis Shivers
Here is the output from the commands: # ./btrfs-debug-tree -R /dev/sdh failed to read /dev/sr0: No medium found failed to read /dev/sde: No medium found failed to read /dev/sdd: No medium found failed to read /dev/sdc: No medium found failed to read /dev/sdb: No medium found failed to read

Re: Btrfs Storage Array Corrupted

2012-02-29 Thread Chris Mason
On Wed, Feb 29, 2012 at 03:57:19PM -0600, Travis Shivers wrote: Here is the output from the commands: # ./btrfs-debug-tree -R /dev/sdh failed to read /dev/sr0: No medium found failed to read /dev/sde: No medium found failed to read /dev/sdd: No medium found failed to read /dev/sdc: No

Re: Btrfs Storage Array Corrupted

2012-02-29 Thread Travis Shivers
Thank you all for helping. My btrfs array consists of 4 disks: 2 (2 TB) disks and 2(500 GB) disks. Since I have disks of different sizes, I have the array being mirrored so that there are two copies of a file on two separate disks. The data and metadata are mirrored. I originally made the array

Re: Btrfs Storage Array Corrupted

2012-02-29 Thread Chris Mason
On Wed, Feb 29, 2012 at 05:11:24PM -0600, Travis Shivers wrote: Thank you all for helping. My btrfs array consists of 4 disks: 2 (2 TB) disks and 2(500 GB) disks. Since I have disks of different sizes, I have the array being mirrored so that there are two copies of a file on two separate

Re: Btrfs Storage Array Corrupted

2012-02-29 Thread Travis Shivers
I was running a fairly old version of the kernel: Linux server 3.0.0-16-generic #28-Ubuntu SMP Fri Jan 27 17:44:39 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux On Wed, Feb 29, 2012 at 5:44 PM, Chris Mason chris.ma...@oracle.com wrote: On Wed, Feb 29, 2012 at 05:11:24PM -0600, Travis Shivers wrote:

Re: [BUG] Kernel Bug at fs/btrfs/volumes.c:3638

2012-02-29 Thread David Sterba
I just noticed that there's a bugreport from opensuse user tripping over the same BUG() during log replay (and his problem was solved by btrfs-zero-log), probably after some crash. The kernel version was 3.1 ie. without the corruption fixes, so while it happened during normal use (and not via a

Re: LABEL only 1 device

2012-02-29 Thread Duncan
Karel Zak posted on Tue, 28 Feb 2012 23:35:57 +0100 as excerpted: On Sun, Feb 26, 2012 at 06:07:31PM +, Duncan wrote: Unfortunately, since gpt is reasonably new in terms of filesystem and partitioning tools, there isn't really anything (mount, etc) that makes /use/ of that label yet,

Re: getdents - ext4 vs btrfs performance

2012-02-29 Thread Theodore Tso
You might try sorting the entries returned by readdir by inode number before you stat them.This is a long-standing weakness in ext3/ext4, and it has to do with how we added hashed tree indexes to directories in (a) a backwards compatible way, that (b) was POSIX compliant with respect to

[PATCH] btrfs: fix locking issues in find_parent_nodes()

2012-02-29 Thread Li Zefan
- We might unlock head-mutex while it was not locked - We might leave the function without unlocking delayed_refs-lock Signed-off-by: Li Zefan l...@cn.fujitsu.com --- fs/btrfs/backref.c |8 ++-- 1 files changed, 6 insertions(+), 2 deletions(-) diff --git a/fs/btrfs/backref.c