Re: NFS/LSM: allow NFS to control all of its own mount options

2008-02-20 Thread Miklos Szeredi
Please don't introduce a special case for just nfs. All filesystems should control their mount options, so please provide some library helpers for context= handling and move it into all filesystems that can support selinux. Hmm, looks like selinux is not showing it's mount options in

Re: very poor ext3 write performance on big filesystems?

2008-02-20 Thread Jan Engelhardt
On Feb 18 2008 10:35, Theodore Tso wrote: On Mon, Feb 18, 2008 at 04:57:25PM +0100, Andi Kleen wrote: Use cp or a tar pipeline to move the files. Are you sure cp handles hardlinks correctly? I know tar does, but I have my doubts about cp. I *think* GNU cp does the right thing with

Re: NFS/LSM: allow NFS to control all of its own mount options

2008-02-20 Thread Stephen Smalley
On Wed, 2008-02-20 at 11:25 +1100, James Morris wrote: On Tue, 19 Feb 2008, Christoph Hellwig wrote: Please don't introduce a special case for just nfs. All filesystems should control their mount options, so please provide some library helpers for context= handling and move it into all

Re: git tree with VFS stuff

2008-02-20 Thread Stephen Rothwell
Hi Miklos, On Tue, 19 Feb 2008 14:32:28 +0100 Miklos Szeredi [EMAIL PROTECTED] wrote: I've created a git tree with the following mounts related stuff: - read-only bind mounts - /proc/pid/mountinfo - unprivileged mounts

Re: NFS/LSM: allow NFS to control all of its own mount options

2008-02-20 Thread Eric Paris
On Wed, 2008-02-20 at 08:50 -0500, Stephen Smalley wrote: On Wed, 2008-02-20 at 11:08 +0100, Miklos Szeredi wrote: Please don't introduce a special case for just nfs. All filesystems should control their mount options, so please provide some library helpers for context= handling and

Re: NFS/LSM: allow NFS to control all of its own mount options

2008-02-20 Thread Stephen Smalley
On Wed, 2008-02-20 at 11:08 +0100, Miklos Szeredi wrote: Please don't introduce a special case for just nfs. All filesystems should control their mount options, so please provide some library helpers for context= handling and move it into all filesystems that can support selinux. Hmm,

Re: git tree with VFS stuff

2008-02-20 Thread Al Viro
On Thu, Feb 21, 2008 at 01:13:48AM +1100, Stephen Rothwell wrote: Hi Miklos, On Tue, 19 Feb 2008 14:32:28 +0100 Miklos Szeredi [EMAIL PROTECTED] wrote: I've created a git tree with the following mounts related stuff: - read-only bind mounts - /proc/pid/mountinfo -

how to show propagation state for mounts

2008-02-20 Thread Miklos Szeredi
mountinfo - IMO needs a sane discussion of what and how should be shown wrt propagation state Here's my take on the matter. The propagation tree can be either be represented 1) from root to leaf listing members of peer groups and their slaves explicitly, 2) or from leaf to root by

Re: how to show propagation state for mounts

2008-02-20 Thread Al Viro
On Wed, Feb 20, 2008 at 04:39:15PM +0100, Miklos Szeredi wrote: mountinfo - IMO needs a sane discussion of what and how should be shown wrt propagation state Here's my take on the matter. The propagation tree can be either be represented 1) from root to leaf listing members of peer

[PATCH 00/37] Permit filesystem local caching

2008-02-20 Thread David Howells
These patches add local caching for network filesystems such as NFS. The patches can roughly be broken down into a number of sets: (*) 01-keys-inc-payload.diff (*) 02-keys-search-keyring.diff (*) 03-keys-callout-blob.diff Three patches to the keyring code made to help the CIFS

[PATCH 03/37] KEYS: Allow the callout data to be passed as a blob rather than a string

2008-02-20 Thread David Howells
Allow the callout data to be passed as a blob rather than a string for internal kernel services that call any request_key_*() interface other than request_key(). request_key() itself still takes a NUL-terminated string. The functions that change are: request_key_with_auxdata()

[PATCH 13/37] FS-Cache: Provide an add_wait_queue_tail() function

2008-02-20 Thread David Howells
Provide an add_wait_queue_tail() function to add a waiter to the back of a wait queue instead of the front. Signed-off-by: David Howells [EMAIL PROTECTED] --- include/linux/pagemap.h |7 +-- include/linux/wait.h|1 + kernel/wait.c | 18 ++

[PATCH 24/37] NFS: Register NFS for caching and retrieve the top-level index

2008-02-20 Thread David Howells
Register NFS for caching and retrieve the top-level cache index object cookie. Signed-off-by: David Howells [EMAIL PROTECTED] --- fs/nfs/Makefile|1 + fs/nfs/fscache-index.c | 53 fs/nfs/fscache.h | 35

[PATCH 09/37] Security: Allow kernel services to override LSM settings for task actions

2008-02-20 Thread David Howells
Allow kernel services to override LSM settings appropriate to the actions performed by a task by duplicating a security record, modifying it and then using task_struct::act_as to point to it when performing operations on behalf of a task. This is used, for example, by CacheFiles which has to

[PATCH 05/37] Security: Change current-fs[ug]id to current_fs[ug]id()

2008-02-20 Thread David Howells
Change current-fs[ug]id to current_fs[ug]id() so that fsgid and fsuid can be separated from the task_struct. Signed-off-by: David Howells [EMAIL PROTECTED] --- arch/ia64/kernel/perfmon.c|4 ++-- arch/powerpc/platforms/cell/spufs/inode.c |4 ++--

[PATCH 21/37] NFS: Add comment banners to some NFS functions

2008-02-20 Thread David Howells
Add comment banners to some NFS functions so that they can be modified by the NFS fscache patches for further information. Signed-off-by: David Howells [EMAIL PROTECTED] --- fs/nfs/file.c | 26 ++ 1 files changed, 26 insertions(+), 0 deletions(-) diff --git

[PATCH 19/37] CacheFiles: Export things for CacheFiles

2008-02-20 Thread David Howells
Export a number of functions for CacheFiles's use. Signed-off-by: David Howells [EMAIL PROTECTED] --- fs/super.c |1 + 1 files changed, 1 insertions(+), 0 deletions(-) diff --git a/fs/super.c b/fs/super.c index 88811f6..1133b43 100644 --- a/fs/super.c +++ b/fs/super.c @@ -267,6 +267,7 @@

[PATCH 02/37] KEYS: Check starting keyring as part of search

2008-02-20 Thread David Howells
Check the starting keyring as part of the search to (a) see if that is what we're searching for, and (b) to check it is still valid for searching. The scenario: User in process A does things that cause things to be created in its process session keyring. The user then does an su to another user

[PATCH 23/37] NFS: Permit local filesystem caching to be enabled for NFS

2008-02-20 Thread David Howells
Permit local filesystem caching to be enabled for NFS in the kernel configuration. Signed-off-by: David Howells [EMAIL PROTECTED] --- fs/Kconfig |8 1 files changed, 8 insertions(+), 0 deletions(-) diff --git a/fs/Kconfig b/fs/Kconfig index c42ec50..fa8e978 100644 ---

[PATCH 27/37] NFS: Define and create inode-level cache objects

2008-02-20 Thread David Howells
Define and create inode-level cache data storage objects (as managed by nfs_inode structs). Each inode-level object is created in a superblock-level index object and is itself a data storage object into which pages from the inode are stored. The inode object key is the NFS file handle for the

[PATCH 08/37] Security: Add a kernel_service object class to SELinux

2008-02-20 Thread David Howells
Add a 'kernel_service' object class to SELinux and give this object class two access vectors: 'use_as_override' and 'create_files_as'. The first vector is used to grant a process the right to nominate an alternate process security ID for the kernel to use as an override for the SELinux subjective

[PATCH 11/37] FS-Cache: Release page-private after failed readahead

2008-02-20 Thread David Howells
The attached patch causes read_cache_pages() to release page-private data on a page for which add_to_page_cache() fails or the filler function fails. This permits pages with caching references associated with them to be cleaned up. The invalidatepage() address space op is called (indirectly) to

[PATCH 01/37] KEYS: Increase the payload size when instantiating a key

2008-02-20 Thread David Howells
Increase the size of a payload that can be used to instantiate a key in add_key() and keyctl_instantiate_key(). This permits huge CIFS SPNEGO blobs to be passed around. The limit is raised to 1MB. If kmalloc() can't allocate a buffer of sufficient size, vmalloc() will be tried instead.

[PATCH 28/37] NFS: Use local disk inode cache

2008-02-20 Thread David Howells
Bind data storage objects in the local cache to NFS inodes. Signed-off-by: David Howells [EMAIL PROTECTED] --- fs/nfs/fscache.c | 131 fs/nfs/fscache.h | 19 +++ fs/nfs/inode.c | 39 --

[PATCH 04/37] KEYS: Add keyctl function to get a security label

2008-02-20 Thread David Howells
Add a keyctl() function to get the security label of a key. The following is added to Documentation/keys.txt: (*) Get the LSM security context attached to a key. long keyctl(KEYCTL_GET_SECURITY, key_serial_t key, char *buffer, size_t buflen) This function

[PATCH 26/37] NFS: Define and create superblock-level objects

2008-02-20 Thread David Howells
Define and create superblock-level cache index objects (as managed by nfs_server structs). Each superblock object is created in a server level index object and is itself an index into which inode-level objects are inserted. Ideally there would be one superblock-level object per server, and the

[PATCH 22/37] NFS: Add FS-Cache option bit and debug bit

2008-02-20 Thread David Howells
Add FS-Cache option bit to nfs_server struct. This is set to indicate local on-disk caching is enabled for a particular superblock. Also add debug bit for local caching operations. Signed-off-by: David Howells [EMAIL PROTECTED] --- include/linux/nfs_fs.h|1 + include/linux/nfs_fs_sb.h

[PATCH 16/37] CacheFiles: Be consistent about the use of mapping vs file-f_mapping in Ext3

2008-02-20 Thread David Howells
Change all the usages of file-f_mapping in ext3_*write_end() functions to use the mapping argument directly. This has two consequences: (*) Consistency. Without this patch sometimes one is used and sometimes the other is. (*) A NULL file pointer can be passed. This feature is then made

[PATCH 12/37] FS-Cache: Recruit a couple of page flags for cache management

2008-02-20 Thread David Howells
Recruit a couple of page flags to aid in cache management. The following extra flags are defined: (1) PG_fscache (PG_private_2) The marked page is backed by a local cache and is pinning resources in the cache driver. (2) PG_fscache_write (PG_owner_priv_2) The marked page is

[PATCH 29/37] NFS: Invalidate FsCache page flags when cache removed

2008-02-20 Thread David Howells
Invalidate the FsCache page flags on the pages belonging to an inode when the cache backing that NFS inode is removed. This allows a live cache to be withdrawn. Signed-off-by: David Howells [EMAIL PROTECTED] --- fs/nfs/fscache-index.c | 40 1 files

[PATCH 33/37] NFS: nfs_readpage_async() needs to be accessible as a fallback for local caching

2008-02-20 Thread David Howells
nfs_readpage_async() needs to be non-static so that it can be used as a fallback for the local on-disk caching should an EIO crop up when reading the cache. Signed-off-by: David Howells [EMAIL PROTECTED] --- fs/nfs/read.c |4 ++-- include/linux/nfs_fs.h |2 ++ 2 files changed,

[PATCH 25/37] NFS: Define and create server-level objects

2008-02-20 Thread David Howells
Define and create server-level cache index objects (as managed by nfs_client structs). Each server object is created in the NFS top-level index object and is itself an index into which superblock-level objects are inserted. Ideally there would be one superblock-level object per server, and the

[PATCH 36/37] NFS: Display local caching state

2008-02-20 Thread David Howells
Display the local caching state in /proc/fs/nfsfs/volumes. Signed-off-by: David Howells [EMAIL PROTECTED] --- fs/nfs/client.c |7 --- fs/nfs/fscache.h | 15 +++ 2 files changed, 19 insertions(+), 3 deletions(-) diff --git a/fs/nfs/client.c b/fs/nfs/client.c index

[PATCH 15/37] CacheFiles: Add missing copy_page export for ia64

2008-02-20 Thread David Howells
This one-line patch fixes the missing export of copy_page introduced by the cachefile patches. This patch is not yet upstream, but is required for cachefile on ia64. It will be pushed upstream when cachefile goes upstream. Signed-off-by: Prarit Bhargava [EMAIL PROTECTED] Signed-off-by: David

[PATCH 34/37] NFS: Read pages from FS-Cache into an NFS inode

2008-02-20 Thread David Howells
Read pages from an FS-Cache data storage object representing an inode into an NFS inode. Signed-off-by: David Howells [EMAIL PROTECTED] --- fs/nfs/fscache.c | 112 ++ fs/nfs/fscache.h | 47 +++ fs/nfs/read.c| 18

[PATCH 18/37] CacheFiles: Permit the page lock state to be monitored

2008-02-20 Thread David Howells
Add a function to install a monitor on the page lock waitqueue for a particular page, thus allowing the page being unlocked to be detected. This is used by CacheFiles to detect read completion on a page in the backing filesystem so that it can then copy the data to the waiting netfs page.

Re: how to show propagation state for mounts

2008-02-20 Thread Miklos Szeredi
On Wed, Feb 20, 2008 at 04:39:15PM +0100, Miklos Szeredi wrote: mountinfo - IMO needs a sane discussion of what and how should be shown wrt propagation state Here's my take on the matter. The propagation tree can be either be represented 1) from root to leaf listing members

Re: how to show propagation state for mounts

2008-02-20 Thread Matthew Wilcox
On Wed, Feb 20, 2008 at 04:04:22PM +, Al Viro wrote: It's less about the form of representation (after all, we generate poll events when contents of that sucker changes, so one *can* get a consistent snapshot of the entire thing) and more about having it self-contained when we have

[PATCH 17/37] CacheFiles: Add a hook to write a single page of data to an inode

2008-02-20 Thread David Howells
Add an address space operation to write one single page of data to an inode at a page-aligned location (thus permitting the implementation to be highly optimised). The data source is a single page. This is used by CacheFiles to store the contents of netfs pages into their backing file pages.

[PATCH 31/37] NFS: FS-Cache page management

2008-02-20 Thread David Howells
FS-Cache page management for NFS. This includes hooking the releasing and invalidation of pages marked with PG_fscache (aka PG_private_2) and waiting for completion of the write-to-cache flag (PG_fscache_write aka PG_owner_priv_2). Signed-off-by: David Howells [EMAIL PROTECTED] ---

[PATCH 35/37] NFS: Store pages from an NFS inode into a local cache

2008-02-20 Thread David Howells
Store pages from an NFS inode into the cache data storage object associated with that inode. Signed-off-by: David Howells [EMAIL PROTECTED] --- fs/nfs/fscache.c | 26 ++ fs/nfs/fscache.h | 16 fs/nfs/read.c|5 + 3 files changed, 47

[PATCH 32/37] NFS: Add read context retention for FS-Cache to call back with

2008-02-20 Thread David Howells
Add read context retention so that FS-Cache can call back into NFS when a read operation on the cache fails EIO rather than reading data. This permits NFS to then fetch the data from the server instead using the appropriate security context. Signed-off-by: David Howells [EMAIL PROTECTED] ---

[PATCH 37/37] NFS: Add mount options to enable local caching on NFS

2008-02-20 Thread David Howells
Add NFS mount options to allow the local caching support to be enabled. The attached patch makes it possible for the NFS filesystem to be told to make use of the network filesystem local caching service (FS-Cache). To be able to use this, a recent nfsutils package is required. There are three

[PATCH 30/37] NFS: Add some new I/O event counters for FS-Cache events

2008-02-20 Thread David Howells
Add some new NFS I/O event counters for FS-Cache events. They have to be added as byte counters because I may need to be able to increase the numbers by more than 1 at a time. Signed-off-by: David Howells [EMAIL PROTECTED] --- fs/nfs/iostat.h |7 +++ 1 files changed, 7 insertions(+), 0

Re: very poor ext3 write performance on big filesystems?

2008-02-20 Thread David Rees
On Wed, Feb 20, 2008 at 2:57 AM, Jan Engelhardt [EMAIL PROTECTED] wrote: But GNU tar does not handle acls and xattrs. So back to rsync/cp/mv. Huh? The version of tar on my Fedora 8 desktop (tar-1.17-7) does. Just add the --xattrs option (which turns on --acls and --selinux). -Dave - To

Re: very poor ext3 write performance on big filesystems?

2008-02-20 Thread Jan Engelhardt
On Feb 20 2008 09:44, David Rees wrote: On Wed, Feb 20, 2008 at 2:57 AM, Jan Engelhardt [EMAIL PROTECTED] wrote: But GNU tar does not handle acls and xattrs. So back to rsync/cp/mv. Huh? The version of tar on my Fedora 8 desktop (tar-1.17-7) does. Just add the --xattrs option (which turns on

Re: how to show propagation state for mounts

2008-02-20 Thread Ram Pai
On Wed, 2008-02-20 at 09:31 -0700, Matthew Wilcox wrote: On Wed, Feb 20, 2008 at 04:04:22PM +, Al Viro wrote: It's less about the form of representation (after all, we generate poll events when contents of that sucker changes, so one *can* get a consistent snapshot of the entire thing)

Re: how to show propagation state for mounts

2008-02-20 Thread Ram Pai
On Wed, 2008-02-20 at 17:27 +0100, Miklos Szeredi wrote: On Wed, Feb 20, 2008 at 04:39:15PM +0100, Miklos Szeredi wrote: mountinfo - IMO needs a sane discussion of what and how should be shown wrt propagation state Here's my take on the matter. The propagation tree can be

Re: [PATCH 00/37] Permit filesystem local caching

2008-02-20 Thread Serge E. Hallyn
Quoting David Howells ([EMAIL PROTECTED]): These patches add local caching for network filesystems such as NFS. The patches can roughly be broken down into a number of sets: (*) 01-keys-inc-payload.diff (*) 02-keys-search-keyring.diff (*) 03-keys-callout-blob.diff Three

Re: [PATCH 00/37] Permit filesystem local caching

2008-02-20 Thread David Howells
Serge E. Hallyn [EMAIL PROTECTED] wrote: Seems *really* weird that every time you send this, patch 6 doesn't seem to reach me in any of my mailboxes... (did get it from the url you listed) It's the largest of the patches, so that's not entirely surprising. Hence why I included the URL to

Re: how to show propagation state for mounts

2008-02-20 Thread Al Viro
On Wed, Feb 20, 2008 at 11:29:13AM -0800, Ram Pai wrote: I wonder, what is wrong in reporting mounts in other namespaces that either receive and send propagation to mounts in our namespace? A plenty. E.g. if foo trusts control over /var/blah to bar, it's not obvious that foo has any business

Re: how to show propagation state for mounts

2008-02-20 Thread Miklos Szeredi
I wonder, what is wrong in reporting mounts in other namespaces that either receive and send propagation to mounts in our namespace? A plenty. E.g. if foo trusts control over /var/blah to bar, it's not obvious that foo has any business knowing if bar gets it from somebody else in turn.

Re: [PATCH 00/37] Permit filesystem local caching

2008-02-20 Thread Daniel Phillips
Hi David, On Wednesday 20 February 2008 08:05, David Howells wrote: These patches add local caching for network filesystems such as NFS. Have you got before/after benchmark results? Regards, Daniel - To unsubscribe from this list: send the line unsubscribe linux-fsdevel in the body of a

Block devices

2008-02-20 Thread David H. Lynch Jr.
Sometime recently it seems to have become possible to disable the whole block device subsystem. Though in my tests I can't quit build with it disabled. Anyway, for an embedded device this might be appealing. how does this interact with initramfs and flash ? Can I boot an