Re: getdents - ext4 vs btrfs performance

2012-03-15 Thread Jacek Luczak
2012/3/10 Ted Ts'o ty...@mit.edu: Hey Jacek, I'm curious parameters of the set of directories on your production server.  On an ext4 file system, assuming you've copied the directories over, what are the result of this command pipeline when you are cd'ed into the top of the directory

Re: getdents - ext4 vs btrfs performance

2012-03-15 Thread Jacek Luczak
2012/3/11 Ted Ts'o ty...@mit.edu: Well, my goal in proposing this optimization is that helps for the medium size directories in the cold cache case.  The ext4 user who first kicked off this thread was using his file system for an SVN server, as I recall.  I could easily believe that he has

Re: kernel BUG at fs/btrfs/delayed-inode.c:1466!

2012-03-12 Thread Jacek Luczak
2012/3/10 Jacek Luczak difrost.ker...@gmail.com: 2012/3/9 David Sterba d...@jikos.cz: On Fri, Mar 09, 2012 at 12:08:12PM +0100, Jacek Luczak wrote: For this one I've created also a report [1]. so there is probably other problem in reservations and it just blew up during the unlink call

Re: kernel BUG at fs/btrfs/delayed-inode.c:1466!

2012-03-11 Thread Jacek Luczak
2012/3/10 Jacek Luczak difrost.ker...@gmail.com: 2) A *regression* in 3.3.0-rc6-00197-g9f8050c - completely unusable as reports ENOSPC - to reproduce, mount volume and issue: # CNT=1 ; while [ $CNT -lt 1 ] ; do  rm -f /btrfs/dd ; ! touch /btrfs/dd echo $CNT break  ; CNT=$(( $CNT + 1

Re: kernel BUG at fs/btrfs/delayed-inode.c:1466!

2012-03-09 Thread Jacek Luczak
2012/3/9 Chris Samuel ch...@csamuel.org: On 09/03/12 12:31, Liu Bo wrote: So are these warnings based on the latest upstream of btrfs? Looks like it was 3.2.7, his oops said: Pid: 1488, comm: mips-wrs-linux- Tainted: G        W    3.2.7 #2 HP Yep, that's 3.2.7. Now I can't upgrade to

Re: kernel BUG at fs/btrfs/delayed-inode.c:1466!

2012-03-09 Thread Jacek Luczak
2012/3/9 David Sterba d...@jikos.cz: On Fri, Mar 09, 2012 at 09:31:25AM +0800, Liu Bo wrote: There were quite many things happening in the system at that time. Can't really tell what could trigger this. Complete logs: http://91.234.146.107/~difrost/logs/tampere_log.gz So are these

Re: kernel BUG at fs/btrfs/delayed-inode.c:1466!

2012-03-09 Thread Jacek Luczak
2012/3/9 David Sterba d...@jikos.cz: On Fri, Mar 09, 2012 at 12:08:12PM +0100, Jacek Luczak wrote: For this one I've created also a report [1]. so there is probably other problem in reservations and it just blew up during the unlink call. Could be as this came up after a longer time

Re: kernel BUG at fs/btrfs/delayed-inode.c:1466!

2012-03-09 Thread Jacek Luczak
2012/3/9 Jacek Luczak difrost.ker...@gmail.com: 2012/3/9 David Sterba d...@jikos.cz: On Fri, Mar 09, 2012 at 12:08:12PM +0100, Jacek Luczak wrote: For this one I've created also a report [1]. so there is probably other problem in reservations and it just blew up during the unlink call

Re: kernel BUG at fs/btrfs/delayed-inode.c:1466!

2012-03-09 Thread Jacek Luczak
2012/3/9 David Sterba d...@jikos.cz: On Fri, Mar 09, 2012 at 03:33:24PM +0100, Jacek Luczak wrote: Those two issues go inline. After a longer while of WARN_ON the BUG_ON hit again. One more observation. Host is running builds from CI system. After BUG_ON pop up all builds take 50% more

Re: kernel BUG at fs/btrfs/delayed-inode.c:1466!

2012-03-09 Thread Jacek Luczak
2012/3/9 Jacek Luczak difrost.ker...@gmail.com: 2012/3/9 David Sterba d...@jikos.cz: On Fri, Mar 09, 2012 at 03:33:24PM +0100, Jacek Luczak wrote: Those two issues go inline. After a longer while of WARN_ON the BUG_ON hit again. One more observation. Host is running builds from CI system

Re: kernel BUG at fs/btrfs/delayed-inode.c:1466!

2012-03-09 Thread Jacek Luczak
2012/3/9 David Sterba d...@jikos.cz: On Fri, Mar 09, 2012 at 12:08:12PM +0100, Jacek Luczak wrote: For this one I've created also a report [1]. so there is probably other problem in reservations and it just blew up during the unlink call. Could be as this came up after a longer time

Re: WARNING: at fs/btrfs/extent-tree.c:5985

2012-03-08 Thread Jacek Luczak
2012/3/6 Jacek Luczak difrost.ker...@gmail.com: Hi All, I've noticed today below WARN_ON from btrfs. Google shows hits in the same place ([1] and [2]) but the path is different. It could happen when svn checout or few rsyncs were running - now I'm not able to put in correct timings. There's

kernel BUG at fs/btrfs/delayed-inode.c:1466!

2012-03-08 Thread Jacek Luczak
Hi, this shown up today. I had to do a hard reboot as graceful hanged on sync(). [ cut here ] kernel BUG at fs/btrfs/delayed-inode.c:1466! invalid opcode: [#1] SMP CPU 10 Modules linked in: btrfs zlib_deflate lzo_compress ipmi_devintf autofs4 be2iscsi

Re: kernel BUG at fs/btrfs/delayed-inode.c:1466!

2012-03-08 Thread Jacek Luczak
2012/3/8 David Sterba d...@jikos.cz: On Thu, Mar 08, 2012 at 01:10:45PM +0100, Jacek Luczak wrote: kernel BUG at fs/btrfs/delayed-inode.c:1466! 1461         ret = btrfs_delayed_item_reserve_metadata(trans, root, item); 1462         /* 1463          * we have reserved enough space when we

WARNING: at fs/btrfs/extent-tree.c:5985

2012-03-06 Thread Jacek Luczak
Hi All, I've noticed today below WARN_ON from btrfs. Google shows hits in the same place ([1] and [2]) but the path is different. It could happen when svn checout or few rsyncs were running - now I'm not able to put in correct timings. There's btrfs_item_offset() in backtrace and I was not able

Re: getdents - ext4 vs btrfs performance

2012-03-05 Thread Jacek Luczak
2012/3/4 Jacek Luczak difrost.ker...@gmail.com: 2012/3/3 Jacek Luczak difrost.ker...@gmail.com: 2012/3/2 Chris Mason chris.ma...@oracle.com: On Fri, Mar 02, 2012 at 03:16:12PM +0100, Jacek Luczak wrote: 2012/3/2 Chris Mason chris.ma...@oracle.com: On Fri, Mar 02, 2012 at 11:05:56AM +0100

Re: getdents - ext4 vs btrfs performance

2012-03-04 Thread Jacek Luczak
2012/3/3 Jacek Luczak difrost.ker...@gmail.com: 2012/3/2 Chris Mason chris.ma...@oracle.com: On Fri, Mar 02, 2012 at 03:16:12PM +0100, Jacek Luczak wrote: 2012/3/2 Chris Mason chris.ma...@oracle.com: On Fri, Mar 02, 2012 at 11:05:56AM +0100, Jacek Luczak wrote: I've took both on tests

Re: getdents - ext4 vs btrfs performance

2012-03-03 Thread Jacek Luczak
2012/3/2 Chris Mason chris.ma...@oracle.com: On Fri, Mar 02, 2012 at 03:16:12PM +0100, Jacek Luczak wrote: 2012/3/2 Chris Mason chris.ma...@oracle.com: On Fri, Mar 02, 2012 at 11:05:56AM +0100, Jacek Luczak wrote: I've took both on tests. The subject is acp and spd_readdir used with tar

Re: getdents - ext4 vs btrfs performance

2012-03-02 Thread Jacek Luczak
2012/3/1 Ted Ts'o ty...@mit.edu: On Thu, Mar 01, 2012 at 03:43:41PM +0100, Jacek Luczak wrote: Yep, ext4 is close to my wife's closet. Were all of the file systems freshly laid down, or was this an aged ext4 file system? Always fresh, recreated for each tests - that's why it takes quite

Re: getdents - ext4 vs btrfs performance

2012-03-02 Thread Jacek Luczak
2012/3/1 Chris Mason chris.ma...@oracle.com: On Wed, Feb 29, 2012 at 11:44:31PM -0500, Theodore Tso wrote: You might try sorting the entries returned by readdir by inode number before you stat them.    This is a long-standing weakness in ext3/ext4, and it has to do with how we added hashed

Re: getdents - ext4 vs btrfs performance

2012-03-02 Thread Jacek Luczak
2012/3/2 Chris Mason chris.ma...@oracle.com: On Fri, Mar 02, 2012 at 11:05:56AM +0100, Jacek Luczak wrote: I've took both on tests. The subject is acp and spd_readdir used with tar, all on ext4: 1) acp: http://91.234.146.107/~difrost/seekwatcher/acp_ext4.png 2) spd_readdir: http

Re: getdents - ext4 vs btrfs performance

2012-03-01 Thread Jacek Luczak
2012/2/29 Jacek Luczak difrost.ker...@gmail.com: 2012/2/29 Chris Mason chris.ma...@oracle.com: On Wed, Feb 29, 2012 at 03:07:45PM +0100, Jacek Luczak wrote: [ btrfs faster than ext for find and cp -a ] 2012/2/29 Jacek Luczak difrost.ker...@gmail.com: I will try to answer the question from

Re: getdents - ext4 vs btrfs performance

2012-03-01 Thread Jacek Luczak
2012/3/1 Hillf Danton dhi...@gmail.com: On Thu, Mar 1, 2012 at 9:35 PM, Jacek Luczak difrost.ker...@gmail.com wrote: While I was about to grab acp I've noticed seekwatcher with made my day :) seekwatcher run of tar cf to eliminate writes (all done on 3.2.7): 1) btrfs: http

Re: getdents - ext4 vs btrfs performance

2012-03-01 Thread Jacek Luczak
2012/3/1 Chris Mason chris.ma...@oracle.com: On Thu, Mar 01, 2012 at 03:03:53PM +0100, Jacek Luczak wrote: 2012/3/1 Hillf Danton dhi...@gmail.com: On Thu, Mar 1, 2012 at 9:35 PM, Jacek Luczak difrost.ker...@gmail.com wrote: While I was about to grab acp I've noticed seekwatcher

Re: getdents - ext4 vs btrfs performance

2012-03-01 Thread Jacek Luczak
2012/3/1 Chris Mason chris.ma...@oracle.com: On Thu, Mar 01, 2012 at 03:43:41PM +0100, Jacek Luczak wrote: 2012/3/1 Chris Mason chris.ma...@oracle.com: XFS will probably beat btrfs in this test.  Their directory indexes reflect on disk layout very well. True, but not that fast on small

getdents - ext4 vs btrfs performance

2012-02-29 Thread Jacek Luczak
Hi All, Long story short: We've found that operations on a directory structure holding many dirs takes ages on ext4. The Question: Why there's that huge difference in ext4 and btrfs? See below test results for real values. Background: I had to backup a Jenkins directory holding workspace for

chris.ma...@oracle.com, Al Viro v...@zeniv.linux.org.uk, Ted Ts'o ty...@mit.edu

2012-02-29 Thread Jacek Luczak
Hi All, /*Sorry for sending incomplete email, hit wrong button :) */ Long story short: We've found that operations on a directory structure holding many dirs takes ages on ext4. The Question: Why there's that huge difference in ext4 and btrfs? See below test results for real values.

getdents - ext4 vs btrfs performance

2012-02-29 Thread Jacek Luczak
Hi All, /*Sorry for sending incomplete email, hit wrong button :) I guess I can't use Gmail */ Long story short: We've found that operations on a directory structure holding many dirs takes ages on ext4. The Question: Why there's that huge difference in ext4 and btrfs? See below test results

Re: getdents - ext4 vs btrfs performance

2012-02-29 Thread Jacek Luczak
Hi Chris, the last one was borked :) Please check this one. -jacek 2012/2/29 Jacek Luczak difrost.ker...@gmail.com: Hi All, /*Sorry for sending incomplete email, hit wrong button :) I guess I can't use Gmail */ Long story short: We've found that operations on a directory structure

Re: getdents - ext4 vs btrfs performance

2012-02-29 Thread Jacek Luczak
2012/2/29 Jacek Luczak difrost.ker...@gmail.com: Hi Chris, the last one was borked :) Please check this one. -jacek 2012/2/29 Jacek Luczak difrost.ker...@gmail.com: Hi All, /*Sorry for sending incomplete email, hit wrong button :) I guess I can't use Gmail */ Long story short: We've

Re: getdents - ext4 vs btrfs performance

2012-02-29 Thread Jacek Luczak
2012/2/29 Jacek Luczak difrost.ker...@gmail.com: 2012/2/29 Jacek Luczak difrost.ker...@gmail.com: Hi Chris, the last one was borked :) Please check this one. -jacek 2012/2/29 Jacek Luczak difrost.ker...@gmail.com: Hi All, /*Sorry for sending incomplete email, hit wrong button :) I guess

Re: getdents - ext4 vs btrfs performance

2012-02-29 Thread Jacek Luczak
2012/2/29 Chris Mason chris.ma...@oracle.com: On Wed, Feb 29, 2012 at 03:07:45PM +0100, Jacek Luczak wrote: [ btrfs faster than ext for find and cp -a ] 2012/2/29 Jacek Luczak difrost.ker...@gmail.com: I will try to answer the question from the broken email I've sent. @Lukas