Re: [PATCH][RFC] fast file mapping for loop

On Thu, Jan 10, 2008 at 12:42:25PM +1100, Nick Piggin wrote: So how does it work? Instead of punting IO to a thread and passing it through the page cache, we instead attempt to send the IO directly to the filesystem block that it maps to. You told Christoph that just using direct-IO from

Re: [PATCH][RFC] fast file mapping for loop

On Thu, Jan 10 2008, Jens Axboe wrote: On Wed, Jan 09 2008, Alasdair G Kergon wrote: Here's the latest version of dm-loop, for comparison. To try it out, ln -s dmsetup dmlosetup and supply similar basic parameters to losetup. (using dmsetup version 1.02.11 or higher) Why oh why

Re: [PATCH][RFC] fast file mapping for loop

On Thu, Jan 10 2008, Christoph Hellwig wrote: loop maintains a prio tree of known extents in the file (populated lazily on demand, as needed). Just a quick question (I haven't looked closely at the code): how come you are using a prio tree for extents? I don't think they could be

Re: [PATCH][RFC] fast file mapping for loop

On Thu, Jan 10, 2008 at 09:44:57AM +0100, Jens Axboe wrote: IMHO this shouldn't be done in the loop driver anyway. Filesystems have their own effricient extent lookup trees (well, at least xfs and btrfs do), and we should leverage that instead of reinventing it. Completely agree, it's

Re: [PATCH][RFC] fast file mapping for loop

On Thu, Jan 10 2008, Christoph Hellwig wrote: On Thu, Jan 10, 2008 at 09:44:57AM +0100, Jens Axboe wrote: IMHO this shouldn't be done in the loop driver anyway. Filesystems have their own effricient extent lookup trees (well, at least xfs and btrfs do), and we should leverage that

Re: [PATCH][RFC] fast file mapping for loop

On Thu, 2008-01-10 at 08:37 +, Christoph Hellwig wrote: Peter, any chance you could chime in here? I have this patch to add swap_out/_in methods. I expect we can loosen the requirement for swapcache pages and change the name a little. previously posted here:

Re: [PATCH][RFC] fast file mapping for loop

On Thu, Jan 10 2008, Peter Zijlstra wrote: On Thu, 2008-01-10 at 08:37 +, Christoph Hellwig wrote: Peter, any chance you could chime in here? I have this patch to add swap_out/_in methods. I expect we can loosen the requirement for swapcache pages and change the name a little.

Re: [PATCH][RFC] fast file mapping for loop

On Thu, 2008-01-10 at 10:49 +0100, Jens Axboe wrote: On Thu, Jan 10 2008, Peter Zijlstra wrote: On Thu, 2008-01-10 at 08:37 +, Christoph Hellwig wrote: Peter, any chance you could chime in here? I have this patch to add swap_out/_in methods. I expect we can loosen the

Re: [PATCH][RFC] fast file mapping for loop

On Thu, Jan 10 2008, Peter Zijlstra wrote: On Thu, 2008-01-10 at 10:49 +0100, Jens Axboe wrote: On Thu, Jan 10 2008, Peter Zijlstra wrote: On Thu, 2008-01-10 at 08:37 +, Christoph Hellwig wrote: Peter, any chance you could chime in here? I have this patch to add

Re: [PATCH][RFC] fast file mapping for loop

On Thu, 10 Jan 2008 09:31:31 +0100 Jens Axboe [EMAIL PROTECTED] wrote: On Wed, Jan 09 2008, Alasdair G Kergon wrote: Here's the latest version of dm-loop, for comparison. To try it out, ln -s dmsetup dmlosetup and supply similar basic parameters to losetup. (using dmsetup version

Re: [PATCH][RFC] fast file mapping for loop

On Thu, Jan 10 2008, Chris Mason wrote: On Thu, 10 Jan 2008 09:31:31 +0100 Jens Axboe [EMAIL PROTECTED] wrote: On Wed, Jan 09 2008, Alasdair G Kergon wrote: Here's the latest version of dm-loop, for comparison. To try it out, ln -s dmsetup dmlosetup and supply similar

Re: [PATCH][RFC] fast file mapping for loop

On Thu, Jan 10 2008, Chris Mason wrote: On Thu, 10 Jan 2008 08:54:59 + Christoph Hellwig [EMAIL PROTECTED] wrote: On Thu, Jan 10, 2008 at 09:44:57AM +0100, Jens Axboe wrote: IMHO this shouldn't be done in the loop driver anyway. Filesystems have their own effricient extent lookup

Re: [PATCH][RFC] fast file mapping for loop

On Thu, 10 Jan 2008 08:54:59 + Christoph Hellwig [EMAIL PROTECTED] wrote: On Thu, Jan 10, 2008 at 09:44:57AM +0100, Jens Axboe wrote: IMHO this shouldn't be done in the loop driver anyway. Filesystems have their own effricient extent lookup trees (well, at least xfs and btrfs do),

Re: [RFD] Incremental fsck

Rik van Riel wrote: Al Boldi [EMAIL PROTECTED] wrote: Ok, but let's look at this a bit more opportunistic / optimistic. You can't play fast and loose with data integrity. Correct, but you have to be realistic... Besides, if we looked at things optimistically, we would conclude that no

Re: [PATCH][RFC] fast file mapping for loop

On Thu, 10 Jan 2008 14:03:24 +0100 Jens Axboe [EMAIL PROTECTED] wrote: On Thu, Jan 10 2008, Chris Mason wrote: On Thu, 10 Jan 2008 08:54:59 + Christoph Hellwig [EMAIL PROTECTED] wrote: On Thu, Jan 10, 2008 at 09:44:57AM +0100, Jens Axboe wrote: IMHO this shouldn't be done in

[PATCH 21/29] Unionfs: extended attributes operations

Signed-off-by: Erez Zadok [EMAIL PROTECTED] --- fs/unionfs/xattr.c | 153 1 files changed, 153 insertions(+), 0 deletions(-) create mode 100644 fs/unionfs/xattr.c diff --git a/fs/unionfs/xattr.c b/fs/unionfs/xattr.c new file mode 100644

[PATCH 07/29] Unionfs: common file copyup/revalidation operations

Includes open, ioctl, and flush operations. Signed-off-by: Erez Zadok [EMAIL PROTECTED] --- fs/unionfs/commonfops.c | 835 +++ 1 files changed, 835 insertions(+), 0 deletions(-) create mode 100644 fs/unionfs/commonfops.c diff --git

[PATCH 11/29] Unionfs: lower-level lookup routines

Includes lower nameidata support routines. Signed-off-by: Erez Zadok [EMAIL PROTECTED] --- fs/unionfs/lookup.c | 652 +++ 1 files changed, 652 insertions(+), 0 deletions(-) create mode 100644 fs/unionfs/lookup.c diff --git a/fs/unionfs/lookup.c

[PATCH 06/29] Unionfs: main header file

Signed-off-by: Erez Zadok [EMAIL PROTECTED] --- fs/unionfs/union.h | 602 1 files changed, 602 insertions(+), 0 deletions(-) create mode 100644 fs/unionfs/union.h diff --git a/fs/unionfs/union.h b/fs/unionfs/union.h new file mode 100644

[PATCH 20/29] Unionfs: super_block operations

Includes read_inode, delete_inode, put_super, statfs, remount_fs (which supports branch-management ops), clear_inode, alloc_inode, destroy_inode, write_inode, umount_begin, and show_options. Signed-off-by: Erez Zadok [EMAIL PROTECTED] --- fs/unionfs/super.c | 1025

[PATCH 13/29] Unionfs: directory reading file operations

Signed-off-by: Erez Zadok [EMAIL PROTECTED] --- fs/unionfs/dirfops.c | 290 ++ 1 files changed, 290 insertions(+), 0 deletions(-) create mode 100644 fs/unionfs/dirfops.c diff --git a/fs/unionfs/dirfops.c b/fs/unionfs/dirfops.c new file mode

[PATCH 24/29] Unionfs: debugging infrastructure

Signed-off-by: Erez Zadok [EMAIL PROTECTED] --- fs/unionfs/debug.c | 533 1 files changed, 533 insertions(+), 0 deletions(-) create mode 100644 fs/unionfs/debug.c diff --git a/fs/unionfs/debug.c b/fs/unionfs/debug.c new file mode 100644

[PATCH 12/29] Unionfs: rename method and helpers

Signed-off-by: Erez Zadok [EMAIL PROTECTED] --- fs/unionfs/rename.c | 531 +++ 1 files changed, 531 insertions(+), 0 deletions(-) create mode 100644 fs/unionfs/rename.c diff --git a/fs/unionfs/rename.c b/fs/unionfs/rename.c new file mode 100644

[PATCH 28/29] VFS: export release_open_intent symbol

Needed to release the resources of the lower nameidata structures that we create and pass to lower file systems (e.g., when calling vfs_create). Signed-off-by: Erez Zadok [EMAIL PROTECTED] --- fs/namei.c |1 + 1 files changed, 1 insertions(+), 0 deletions(-) diff --git a/fs/namei.c

[PATCH 25/29] Unionfs file system magic number

Signed-off-by: Erez Zadok [EMAIL PROTECTED] --- include/linux/magic.h |2 ++ 1 files changed, 2 insertions(+), 0 deletions(-) diff --git a/include/linux/magic.h b/include/linux/magic.h index 1fa0c2c..67043ed 100644 --- a/include/linux/magic.h +++ b/include/linux/magic.h @@ -35,6 +35,8 @@

[PATCH 22/29] Unionfs: async I/O queue

Signed-off-by: Erez Zadok [EMAIL PROTECTED] --- fs/unionfs/sioq.c | 119 + fs/unionfs/sioq.h | 92 + 2 files changed, 211 insertions(+), 0 deletions(-) create mode 100644 fs/unionfs/sioq.c create mode

[PATCH 15/29] Unionfs: readdir state helpers

Includes duplicate name elimination and whiteout-handling code. Signed-off-by: Erez Zadok [EMAIL PROTECTED] --- fs/unionfs/rdstate.c | 285 ++ 1 files changed, 285 insertions(+), 0 deletions(-) create mode 100644 fs/unionfs/rdstate.c diff --git

[PATCH 29/29] Put Unionfs and eCryptfs under one layered filesystems menu

Signed-off-by: Erez Zadok [EMAIL PROTECTED] --- fs/Kconfig | 53 + 1 files changed, 41 insertions(+), 12 deletions(-) diff --git a/fs/Kconfig b/fs/Kconfig index 487236c..55a78b7 100644 --- a/fs/Kconfig +++ b/fs/Kconfig @@ -1041,6 +1041,47 @@

[PATCH 09/29] Unionfs: lower-level copyup routines

Signed-off-by: Erez Zadok [EMAIL PROTECTED] --- fs/unionfs/copyup.c | 899 +++ 1 files changed, 899 insertions(+), 0 deletions(-) create mode 100644 fs/unionfs/copyup.c diff --git a/fs/unionfs/copyup.c b/fs/unionfs/copyup.c new file mode 100644

[PATCH 02/29] VFS/eCryptfs: use simplified fs_stack API to fsstack_copy_attr_all

Acked-by: Mike Halcrow [EMAIL PROTECTED] Signed-off-by: Erez Zadok [EMAIL PROTECTED] --- fs/ecryptfs/dentry.c |2 +- fs/ecryptfs/inode.c |6 +++--- fs/ecryptfs/main.c |2 +- fs/stack.c | 38 -- include/linux/fs_stack.h

[UNIONFS] 00/29 Unionfs and related patches pre-merge review (v2)

Dear Linus, Al, Christoph, and Andrew, As per your request, I'm posting for review the unionfs code (and related code) that's in my korg tree against mainline (v2.6.24-rc7-71-gfd0b45d). This is in preparation for merge in 2.6.25. This code here is nearly identical to what's in -mm (the mm code

[PATCH 27/29] VFS path get/put ops used by Unionfs

Note: this will become obsolete once similar patches, now in -mm, make it to mainline. Signed-off-by: Erez Zadok [EMAIL PROTECTED] --- include/linux/namei.h | 13 + 1 files changed, 13 insertions(+), 0 deletions(-) diff --git a/include/linux/namei.h b/include/linux/namei.h index

[PATCH 17/29] Unionfs: unlink/rmdir operations

Signed-off-by: Erez Zadok [EMAIL PROTECTED] --- fs/unionfs/unlink.c | 251 +++ 1 files changed, 251 insertions(+), 0 deletions(-) create mode 100644 fs/unionfs/unlink.c diff --git a/fs/unionfs/unlink.c b/fs/unionfs/unlink.c new file mode 100644

[PATCH 01/29] Unionfs: documentation

Includes index files, MAINTAINERS, and documentation on general concepts, usage, issues, and renaming operations. Signed-off-by: Erez Zadok [EMAIL PROTECTED] --- Documentation/filesystems/00-INDEX |2 + Documentation/filesystems/unionfs/00-INDEX | 10 +

[PATCH 03/29] Makefile: hook to compile unionfs

Signed-off-by: Erez Zadok [EMAIL PROTECTED] --- fs/Makefile |1 + 1 files changed, 1 insertions(+), 0 deletions(-) diff --git a/fs/Makefile b/fs/Makefile index 500cf15..e202288 100644 --- a/fs/Makefile +++ b/fs/Makefile @@ -118,3 +118,4 @@ obj-$(CONFIG_HPPFS) += hppfs/

[PATCH 08/29] Unionfs: basic file operations

Includes read, write, mmap, fsync, and fasync. Signed-off-by: Erez Zadok [EMAIL PROTECTED] --- fs/unionfs/file.c | 184 + 1 files changed, 184 insertions(+), 0 deletions(-) create mode 100644 fs/unionfs/file.c diff --git a/fs/unionfs/file.c

[PATCH 04/29] Unionfs: main Makefile

Signed-off-by: Erez Zadok [EMAIL PROTECTED] --- fs/unionfs/Makefile | 13 + 1 files changed, 13 insertions(+), 0 deletions(-) create mode 100644 fs/unionfs/Makefile diff --git a/fs/unionfs/Makefile b/fs/unionfs/Makefile new file mode 100644 index 000..17ca4a7 --- /dev/null +++

[PATCH 23/29] Unionfs: miscellaneous helper routines

Mostly related to whiteouts. Signed-off-by: Erez Zadok [EMAIL PROTECTED] --- fs/unionfs/subr.c | 242 + 1 files changed, 242 insertions(+), 0 deletions(-) create mode 100644 fs/unionfs/subr.c diff --git a/fs/unionfs/subr.c

[PATCH 05/29] Unionfs: fanout header definitions

Signed-off-by: Erez Zadok [EMAIL PROTECTED] --- fs/unionfs/fanout.h | 366 +++ 1 files changed, 366 insertions(+), 0 deletions(-) create mode 100644 fs/unionfs/fanout.h diff --git a/fs/unionfs/fanout.h b/fs/unionfs/fanout.h new file mode 100644

[PATCH 18/29] Unionfs: address-space operations

Includes writepage, writepages, readpage, prepare_write, and commit_write. Signed-off-by: Erez Zadok [EMAIL PROTECTED] --- fs/unionfs/mmap.c | 343 + 1 files changed, 343 insertions(+), 0 deletions(-) create mode 100644 fs/unionfs/mmap.c

[PATCH 16/29] Unionfs: inode operations

Includes create, lookup, link, symlink, mkdir, mknod, readlink, follow_link, put_link, permission, and setattr. Signed-off-by: Erez Zadok [EMAIL PROTECTED] --- fs/unionfs/inode.c | 1174 1 files changed, 1174 insertions(+), 0 deletions(-)

[PATCH 14/29] Unionfs: readdir helper functions

Includes whiteout handling for directories. Signed-off-by: Erez Zadok [EMAIL PROTECTED] --- fs/unionfs/dirhelper.c | 267 1 files changed, 267 insertions(+), 0 deletions(-) create mode 100644 fs/unionfs/dirhelper.c diff --git

[PATCH 19/29] Unionfs: mount-time and stacking-interposition functions

Includes read_super and module-linkage routines. Signed-off-by: Erez Zadok [EMAIL PROTECTED] --- fs/unionfs/main.c | 794 + 1 files changed, 794 insertions(+), 0 deletions(-) create mode 100644 fs/unionfs/main.c diff --git

[PATCH 26/29] Unionfs: common header file for user-land utilities and kernel

Signed-off-by: Erez Zadok [EMAIL PROTECTED] --- include/linux/union_fs.h | 24 1 files changed, 24 insertions(+), 0 deletions(-) create mode 100644 include/linux/union_fs.h diff --git a/include/linux/union_fs.h b/include/linux/union_fs.h new file mode 100644 index

Re: [UNIONFS] 00/29 Unionfs and related patches pre-merge review (v2)

On Thu, Jan 10, 2008 at 09:59:19AM -0500, Erez Zadok wrote: Dear Linus, Al, Christoph, and Andrew, As per your request, I'm posting for review the unionfs code (and related code) that's in my korg tree against mainline (v2.6.24-rc7-71-gfd0b45d). This is in preparation for merge in 2.6.25.

Re: [PATCH][RFC] fix reservation discarding in affs

On Thu, Feb 10, 2005 at 01:57:04PM +0100, Roman Zippel wrote: affs already does it in a) and a few other places, so adding it to affs_clear_inode should make it behave fine. Also move the truncate from affs_put_inode with the racy i_count check to affs_clear_inode. This also avoids the

Re: [PATCH][RFC] Simple tamper-proof device filesystem.

On Thu, January 10, 2008 05:57, Tetsuo Handa wrote: It seems to me that the alternatives you are proposing include modification of userland applications. But my assumption is that Don't require modification of userland applications. If you want a secure system it isn't that unreasonable to

fsck for... tmpfs?

I just looked into /tmp, which is tmpfs here, and found something rather bizarre - I replaced some files with (...): # ls -l /tmp ls: cannot access /tmp/temp-qar�hest.html: No such file or directory total 13984 drwxrwxr-x 3 tch tch 100 2008-01-11 00:11 1/ drwx-- 2 tch tch 60

Re: [PATCH][RFC] fast file mapping for loop

Jens Axboe wrote: Hi, loop.c currently uses the page cache interface to do IO to file backed devices. This works reasonably well for simple things, like mapping an iso9660 file for direct mount and other read-only workloads. Writing is somewhat problematic, as anyone who has really used this

Re: [PATCH][RFC] fast file mapping for loop

On Fri, Jan 11 2008, Mikulas Patocka wrote: So I looked at the code - it seems you build a full extent of the blocks in the file, filling holes as you go along. I initally did that as well, but that is to slow to be usable in real life. You also don't support sparse files, falling back