Re: [PATCH 1/6] fs: add hole punching to fallocate

2010-11-17 Thread Andreas Dilger
On 2010-11-16, at 20:11, Dave Chinner wrote: On Tue, Nov 16, 2010 at 06:22:47PM -0600, Andreas Dilger wrote: IMHO, it makes more sense for consistency and get what users expect that these be treated as flags. Some users will want KEEP_SIZE, but in other cases it may make sense that a hole

Re: [PATCH 1/6] fs: add hole punching to fallocate

2010-11-17 Thread Andreas Dilger
On 2010-11-16, at 20:34, Josef Bacik wrote: FWIW I agree with Dave, the only question at this point is do we force users to specify KEEP_SIZE with PUNCH_HOLE? On one hand it makes the interface a bit more consistent, on the other hand it makes the documentation a little weird We have

Re: [PATCH 1/3] direct-io: add a hook for the fs to provide its own bio merging check function

2010-11-17 Thread Josef Bacik
On Wed, Nov 17, 2010 at 02:06:59AM -0500, Josef Bacik wrote: On Wed, Nov 17, 2010 at 12:18:35PM +0800, Miao Xie wrote: BTRFS can not submit bios that span its chunks or stripes, so it needs a function to check it when we want to add a page into the bios. So we add a can_merge_io hook to do

Re: Poor performance unlinking hard-linked files (repost)

2010-11-17 Thread Bron Gondwana
On Wed, Nov 17, 2010 at 03:11:48PM +1100, Bron Gondwana wrote: Could you sysrq-w while the performance is bad? That would narrow it down. Here's one: http://pastebin.com/Tg7agv42 And here's another one, inline this time. The iostat for 10 seconds just before said: (iostat -x 10 10)

Re: [PATCH 1/3] direct-io: add a hook for the fs to provide its own bio merging check function

2010-11-17 Thread Miao Xie
Hi, Josef On wed, 17 Nov 2010 04:37:21 -0500, Josef Bacik wrote: Heh so I was going to fix this after the hole punching stuff. The fact is btrfs maps everything that is ok to do in one IO via get_blocks(). So all we need to do is add another DIO_ flag to tell us to treat each get_blocks()

Re: [PATCH 1/3] direct-io: add a hook for the fs to provide its own bio merging check function

2010-11-17 Thread Josef Bacik
On Wed, Nov 17, 2010 at 06:11:03PM +0800, Miao Xie wrote: Hi, Josef On wed, 17 Nov 2010 04:37:21 -0500, Josef Bacik wrote: Heh so I was going to fix this after the hole punching stuff. The fact is btrfs maps everything that is ok to do in one IO via get_blocks(). So all we need to do

Re: Update to Project_ideas wiki page

2010-11-17 Thread Hugo Mills
On Tue, Nov 16, 2010 at 10:19:45PM -0500, Chris Ball wrote: Hi, Chris Mason has posted a bunch of interesting updates to the Project_ideas wiki page. If you're interested in working on any of these, feel free to speak up and ask for more information if you need it. Here are the new

Re: Update to Project_ideas wiki page

2010-11-17 Thread Bart Noordervliet
On Wed, Nov 17, 2010 at 15:31, Hugo Mills hugo-l...@carfax.org.uk wrote: On Tue, Nov 16, 2010 at 10:19:45PM -0500, Chris Ball wrote: == Changing RAID levels == We need ioctls to change between different raid levels.  Some of these are quite easy -- e.g. for RAID0 to RAID1, we just halve the

Re: [PATCH 1/3] direct-io: add a hook for the fs to provide its own bio merging check function

2010-11-17 Thread Chris Mason
Excerpts from Josef Bacik's message of 2010-11-17 07:50:11 -0500: On Wed, Nov 17, 2010 at 06:11:03PM +0800, Miao Xie wrote: Hi, Josef On wed, 17 Nov 2010 04:37:21 -0500, Josef Bacik wrote: Heh so I was going to fix this after the hole punching stuff. The fact is btrfs maps

Re: Update to Project_ideas wiki page

2010-11-17 Thread Xavier Nicollet
Le 17 novembre 2010 à 16:12, Bart Noordervliet a écrit: Can I suggest we combine this new RAID level management with a modernisation of the terminology for storage redundancy, as has been discussed previously in the Raid1 with 3 drives thread of March this year? I.e. abandon the burdened raid*

Re: Update to Project_ideas wiki page

2010-11-17 Thread Mike Fedyk
On Wed, Nov 17, 2010 at 7:12 AM, Bart Noordervliet b...@noordervliet.net wrote: On Wed, Nov 17, 2010 at 15:31, Hugo Mills hugo-l...@carfax.org.uk wrote: On Tue, Nov 16, 2010 at 10:19:45PM -0500, Chris Ball wrote: == Changing RAID levels == We need ioctls to change between different raid

Re: Update to Project_ideas wiki page

2010-11-17 Thread Hugo Mills
On Wed, Nov 17, 2010 at 04:12:29PM +0100, Bart Noordervliet wrote: Can I suggest we combine this new RAID level management with a modernisation of the terminology for storage redundancy, as has been discussed previously in the Raid1 with 3 drives thread of March this year? I.e. abandon the

Re: Update to Project_ideas wiki page

2010-11-17 Thread Gordan Bobic
On 11/17/2010 05:56 PM, Hugo Mills wrote: On Wed, Nov 17, 2010 at 04:12:29PM +0100, Bart Noordervliet wrote: Can I suggest we combine this new RAID level management with a modernisation of the terminology for storage redundancy, as has been discussed previously in the Raid1 with 3 drives thread

Re: Update to Project_ideas wiki page

2010-11-17 Thread Andreas Philipp
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 17.11.2010 18:56, Hugo Mills wrote: On Wed, Nov 17, 2010 at 04:12:29PM +0100, Bart Noordervliet wrote: Can I suggest we combine this new RAID level management with a modernisation of the terminology for storage redundancy, as has been

Re: Update to Project_ideas wiki page

2010-11-17 Thread Hugo Mills
On Wed, Nov 17, 2010 at 07:14:47PM +0100, Andreas Philipp wrote: On 17.11.2010 18:56, Hugo Mills wrote: On Wed, Nov 17, 2010 at 04:12:29PM +0100, Bart Noordervliet wrote: Can I suggest we combine this new RAID level management with a modernisation of the terminology for storage redundancy,

Re: Update to Project_ideas wiki page

2010-11-17 Thread Bart Kus
On 11/17/2010 10:07 AM, Gordan Bobic wrote: On 11/17/2010 05:56 PM, Hugo Mills wrote: On Wed, Nov 17, 2010 at 04:12:29PM +0100, Bart Noordervliet wrote: Can I suggest we combine this new RAID level management with a modernisation of the terminology for storage redundancy, as has been discussed

[PATCH] Btrfs: handle NFS lookups properly

2010-11-17 Thread Josef Bacik
People kept reporting NFS issues, specifically getting ESTALE alot. I figured out how to reproduce the problem SERVER mkfs.btrfs /dev/sda1 mount /dev/sda1 /mnt/btrfs-test add /mnt/btrfs-test to /etc/exports btrfs subvol create /mnt/btrfs-test/foo service nfs start CLIENT mount server:/mnt/btrfs

A little confused about what remains to make a stable release

2010-11-17 Thread Daniel Farina
Hello List, I have been tracking the development of btrfs for some time, as the built-in support for snapshotting would be of great convenience for relational database use cases. I have been crawling the wiki (especially the FAQ), but I still don't have a clear sense of what's left *besides* the

Re: A little confused about what remains to make a stable release

2010-11-17 Thread Hugo Mills
On Wed, Nov 17, 2010 at 02:27:39PM -0800, Daniel Farina wrote: I have been tracking the development of btrfs for some time, as the built-in support for snapshotting would be of great convenience for relational database use cases. I have been crawling the wiki (especially the FAQ), but I still

Re: [PATCH 3/6] Ocfs2: handle hole punching via fallocate properly

2010-11-17 Thread Joel Becker
On Mon, Nov 15, 2010 at 12:05:20PM -0500, Josef Bacik wrote: This patch just makes ocfs2 use its UNRESERVP ioctl when we get the hole punch flag in fallocate. I didn't test it, but it seems simple enough. Thanks, Signed-off-by: Josef Bacik jo...@redhat.com Seems reasonable to me. Acked-by:

Re: A little confused about what remains to make a stable release

2010-11-17 Thread Anthony Roberts
Hello, It's stable *for you* when it functions with the workloads *you* expect of it, with a failure rate that is acceptable *to you*. I think there's a few ancillary things like a working fsck needed before it can even be recommended for widespread use, even to users willing to risk any

Re: [PATCH 1/3] direct-io: add a hook for the fs to provide its own bio merging check function

2010-11-17 Thread Miao Xie
On wed, 17 Nov 2010 11:55:28 -0500, Chris Mason wrote: Excerpts from Josef Bacik's message of 2010-11-17 07:50:11 -0500: On Wed, Nov 17, 2010 at 06:11:03PM +0800, Miao Xie wrote: Hi, Josef On wed, 17 Nov 2010 04:37:21 -0500, Josef Bacik wrote: Heh so I was going to fix this after the

Re: [PATCH 1/3] direct-io: add a hook for the fs to provide its own bio merging check function

2010-11-17 Thread Chris Mason
Excerpts from Miao Xie's message of 2010-11-17 20:18:18 -0500: Right thats the idea, if we can't span chunks/stripes we should be doing that limiting in our get_blocks call and that way we don't have to screw with the generic direct io stuff too much. Thanks, In this case we're

[PATCH 5/6] Btrfs: fail if we try to use hole punch

2010-11-17 Thread Josef Bacik
Btrfs doesn't have the ability to punch holes yet, so make sure we return EOPNOTSUPP if we try to use hole punching through fallocate. This support can be added later. Thanks, Signed-off-by: Josef Bacik jo...@redhat.com --- fs/btrfs/inode.c |4 1 files changed, 4 insertions(+), 0

[PATCH 1/6] fs: add hole punching to fallocate

2010-11-17 Thread Josef Bacik
Hole punching has already been implemented by XFS and OCFS2, and has the potential to be implemented on both BTRFS and EXT4 so we need a generic way to get to this feature. The simplest way in my mind is to add FALLOC_FL_PUNCH_HOLE to fallocate() since it already looks like the normal fallocate()

[PATCH 3/6] Ocfs2: handle hole punching via fallocate properly

2010-11-17 Thread Josef Bacik
This patch just makes ocfs2 use its UNRESERVP ioctl when we get the hole punch flag in fallocate. I didn't test it, but it seems simple enough. Thanks, Acked-by: Jan Kara j...@suse.cz Acked-by: Joel Becker joel.bec...@oracle.com Signed-off-by: Josef Bacik jo...@redhat.com --- fs/ocfs2/file.c |

[PATCH 2/6] XFS: handle hole punching via fallocate properly

2010-11-17 Thread Josef Bacik
This patch simply allows XFS to handle the hole punching flag in fallocate properly. I've tested this with a little program that does a bunch of random hole punching with FL_KEEP_SIZE and without it to make sure it does the right thing. Thanks, Signed-off-by: Josef Bacik jo...@redhat.com ---

[PATCH 4/6] Ext4: fail if we try to use hole punch

2010-11-17 Thread Josef Bacik
Ext4 doesn't have the ability to punch holes yet, so make sure we return EOPNOTSUPP if we try to use hole punching through fallocate. This support can be added later. Thanks, Acked-by: Jan Kara j...@suse.cz Signed-off-by: Josef Bacik jo...@redhat.com --- fs/ext4/extents.c |4 1 files

Hole Punching V3

2010-11-17 Thread Josef Bacik
This is version 3 of the hole punching series I've been posting. Not much has changed, the history is below V2-V3 -FALLOC_FL_PUNCH_HOLE must also have FALLOC_FL_KEEP_SIZE in order to work -formatting fixes V1-V2 -Hole punching doesn't change file size -Fixed the mode checks in ext4/btrfs/gfs2

[PATCH v2 1/6] btrfs: Fix bugs in zlib workspace

2010-11-17 Thread Li Zefan
- Fix a race that can result in alloc_workspace cpus. - Fix to check num_workspace after wakeup. Tested-by: Mitch Harder mitch.har...@sabayonlinux.org Signed-off-by: Li Zefan l...@cn.fujitsu.com --- fs/btrfs/zlib.c |7 +-- 1 files changed, 5 insertions(+), 2 deletions(-) diff --git

[PATCH v2 2/6] btrfs: Fix error handling in zlib

2010-11-17 Thread Li Zefan
Return failure if alloc_page() fails to allocate memory, and the upper code will just give up compression. Signed-off-by: Li Zefan l...@cn.fujitsu.com --- fs/btrfs/zlib.c |8 1 files changed, 8 insertions(+), 0 deletions(-) diff --git a/fs/btrfs/zlib.c b/fs/btrfs/zlib.c index

[GIT PULL][PATCH v2 0/6] btrfs: Add lzo compression support

2010-11-17 Thread Li Zefan
Hi Chris, Here's the updated patchset. As I still haven't got a kernel.org account, I have set up a git tree in another public git repository, and I'll use it for now. You can pull from: git://repo.or.cz/linux-btrfs-devel.git lzo-support Lzo is a much faster compression algorithm than

[PATCH v2 6/6] btrfs: Extract duplicate decompress code

2010-11-17 Thread Li Zefan
Add a common function to copy decompressed data from working buffer to bio pages. Tested-by: Mitch Harder mitch.har...@sabayonlinux.org Signed-off-by: Li Zefan l...@cn.fujitsu.com --- fs/btrfs/compression.c | 92 +++ fs/btrfs/compression.h |5 ++

[PATCH v2 3/6] btrfs: Allow to add new compression algorithm

2010-11-17 Thread Li Zefan
Make the code aware of compression type, instead of always assuming zlib compression. Also make the zlib workspace function as common code for all compression types. Tested-by: Mitch Harder mitch.har...@sabayonlinux.org Signed-off-by: Li Zefan l...@cn.fujitsu.com --- fs/btrfs/btrfs_inode.h |

[PATCH v2 4/6] btrfs: Add lzo compression support

2010-11-17 Thread Li Zefan
Lzo is a much faster compression algorithm than gzib, so would allow more users to enable transparent compression, and some users can choose from compression ratio and speed for different applications Usage: # mount -t btrfs -o compress[=zlib,lzo] dev /mnt or # mount -t btrfs -o

[PATCH v2 5/6] btrfs: Allow to specify compress method when defrag

2010-11-17 Thread Li Zefan
Update defrag ioctl, so one can choose lzo or zlib when turning on compression in defrag operation. Changelog: v1 - v2 - Add incompability flag. - Fix to check invalid compress type. Signed-off-by: Li Zefan l...@cn.fujitsu.com --- fs/btrfs/ioctl.c | 24 +++-

Re: [patch] fix up lock order reversal in writeback

2010-11-17 Thread Ted Ts'o
On Wed, Nov 17, 2010 at 05:10:57PM +1100, Nick Piggin wrote: On Tue, Nov 16, 2010 at 11:05:52PM -0600, Eric Sandeen wrote: On 11/16/10 10:38 PM, Nick Piggin wrote: as for the locking problems ... sorry about that! That's no problem. So is that an ack? :) I'd like to test it

Re: [patch] fix up lock order reversal in writeback

2010-11-17 Thread Eric Sandeen
On 11/17/10 12:10 AM, Nick Piggin wrote: On Tue, Nov 16, 2010 at 11:05:52PM -0600, Eric Sandeen wrote: On 11/16/10 10:38 PM, Nick Piggin wrote: as for the locking problems ... sorry about that! That's no problem. So is that an ack? :) I'd like to test it with the original case it was

Re: [patch] fix up lock order reversal in writeback

2010-11-17 Thread Andrew Morton
On Wed, 17 Nov 2010 22:06:13 -0500 Ted Ts'o ty...@mit.edu wrote: On Wed, Nov 17, 2010 at 05:10:57PM +1100, Nick Piggin wrote: On Tue, Nov 16, 2010 at 11:05:52PM -0600, Eric Sandeen wrote: On 11/16/10 10:38 PM, Nick Piggin wrote: as for the locking problems ... sorry about that!

Re: [Btrfs-Progs] Update for lzo support

2010-11-17 Thread Li Zefan
Please update the man page too. Updated: [Btrfs-Progs][V2] Update for lzo support - Add incompat flag, otherwise btrfs-progs will report error when operating on btrfs filesystems mounted with lzo option. - Update man page. - Allow to turn on lzo compression for defrag operation: # btrfs

ls flush-btrfs-1 sit at 100% sys

2010-11-17 Thread Brian Sullivan
I had been running 2.6.32 for a many months without any issues. Btrfs on top of a raid6 md array. Filesystem is at 9/11TB used. I updated to 2.6.34 for a week or so and had no problem. Updated to 2.6.36 for a few days and no problems. Update to 2.6.37 and now I cannot read from array. So I

Re: ls flush-btrfs-1 sit at 100% sys

2010-11-17 Thread Chris Ball
Hi, Is doesn't return, I check top and both ls and flush-btrfs-1 are sitting at ~50% sys usage each. Does anything new appear in dmesg when the hang happens? Can you run alt-sysrq-t (show tasks) and send us the output for the ls process? - Chris. -- Chris Ball c...@laptop.org One

Re: [patch] fix up lock order reversal in writeback

2010-11-17 Thread Nick Piggin
On Wed, Nov 17, 2010 at 07:29:00PM -0800, Andrew Morton wrote: On Wed, 17 Nov 2010 22:06:13 -0500 Ted Ts'o ty...@mit.edu wrote: On Wed, Nov 17, 2010 at 05:10:57PM +1100, Nick Piggin wrote: On Tue, Nov 16, 2010 at 11:05:52PM -0600, Eric Sandeen wrote: On 11/16/10 10:38 PM, Nick Piggin

Re: ls flush-btrfs-1 sit at 100% sys

2010-11-17 Thread Brian Sullivan
Nothing shows up in dmesg. [ 8114.870020] lsR running task0 3438 3375 0x0004 [ 8114.870020] 88036339dab8 0086 88036339da60 88036339dfd8 [ 8114.870020] 000139c0 88036339dfd8 88036339dfd8 [ 8114.870020]

Root fs on raid1

2010-11-17 Thread admin
Hi, I have read that for using raid1 btrfs device scan must be run before mounting the fs. How do I proceed if root filesystem is mounted from btrfs ? Thanks in advance Bye, David Arendt -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to

Interesting problem with write data.

2010-11-17 Thread Magicloud Magiclouds
Hi, Recently, I made a btrfs to use. And I met slowness problem. Trying to diag it. I found this: 1. dd if=/dev/zero of=test count=1024 bs=1MB This is fast, at about 25MB/s, and reasonable iowait. 2. dd if=/dev/zero of=test count=1 bs=1GB This is pretty slow, at about 1.5MB/s, and 90%+ iowait,

Re: A little confused about what remains to make a stable release

2010-11-17 Thread Daniel Farina
On Wed, Nov 17, 2010 at 4:46 PM, Anthony Roberts btrfs-de...@arbitraryconstant.com wrote: I think there's a few ancillary things like a working fsck needed before it can even be recommended for widespread use, even to users willing to risk any residual bugs. IIRC at this point the utilities

Re: Root fs on raid1

2010-11-17 Thread Chester
You might need to build an initramfs image for your kernel to mount, to help mount your btrfs filesystem. You may want to read Gentoo's guide for initramfs http://en.gentoo-wiki.com/wiki/Initramfs On Thu, Nov 18, 2010 at 12:14 AM, ad...@prnet.org wrote: Hi, I have read that for using raid1

Re: Root fs on raid1

2010-11-17 Thread Goffredo Baroncelli
On Thursday, 18 November, 2010, ad...@prnet.org wrote: Hi, I have read that for using raid1 btrfs device scan must be run before mounting the fs. How do I proceed if root filesystem is mounted from btrfs ? See a my previous post