Re: very poor ext3 write performance on big filesystems?

2008-02-18 Thread Andi Kleen
On Mon, Feb 18, 2008 at 10:16:32AM -0500, Theodore Tso wrote: On Mon, Feb 18, 2008 at 04:02:36PM +0100, Tomasz Chmielewski wrote: I tried to copy that filesystem once (when it was much smaller) with rsync -a -H, but after 3 days, rsync was still building an index and didn't copy any file.

Re: [PATCH] [8/18] BKL-removal: Remove BKL from remote_llseek

2008-01-28 Thread Andi Kleen
I completely agree. If one thread writes A and another writes B then the kernel should record either A or B, not ((A 0x) | (B 0x)) The problem is pretty nasty unfortunately. To solve it properly I think the file_operations-read/write prototypes would need to be

Re: [PATCH] [8/18] BKL-removal: Remove BKL from remote_llseek

2008-01-28 Thread Andi Kleen
On Monday 28 January 2008 13:56:05 Alan Cox wrote: No specific spec, just general quality of implementation. I completely agree. If one thread writes A and another writes B then the kernel should record either A or B, not ((A 0x) | (B 0x)) Agree entirely:

Re: [PATCH] [8/18] BKL-removal: Remove BKL from remote_llseek

2008-01-28 Thread Andi Kleen
On Monday 28 January 2008 14:38:57 Alan Cox wrote: Also worse really fixing it would be a major change to the VFS because of the way -read/write are defined :/ I don't see a problem there. -read and -write update the passed pointer which is not the real f_pos anyway. Just the copies need

Re: [PATCH] [14/18] BKL-removal: Add unlocked_fasync v2

2008-01-27 Thread Andi Kleen
change that was lockless and could potentially lose updates. Signed-off-by: Andi Kleen [EMAIL PROTECTED] --- Documentation/filesystems/vfs.txt |5 - fs/fcntl.c|6 +- fs/ioctl.c|5 - include/linux/fs.h|1 + 4

Re: [PATCH] [8/18] BKL-removal: Remove BKL from remote_llseek

2008-01-27 Thread Andi Kleen
On Sunday 27 January 2008 17:57:14 Steve French wrote: Don't you need to a spinlock/spinunlock(i_lock) or something similar (there isn't a spinlock in the file struct unfortunately) around the reads and writes from f_pos in fs/read_write.c in remote_llseek with your patch since the

[PATCH] [6/18] BKL-removal: Convert ext4 to use unlocked_ioctl

2008-01-26 Thread Andi Kleen
-by: Andi Kleen [EMAIL PROTECTED] --- fs/ext4/dir.c |2 +- fs/ext4/file.c |2 +- fs/ext4/ioctl.c | 20 +++- include/linux/ext4_fs.h |3 +-- 4 files changed, 14 insertions(+), 13 deletions(-) Index: linux/fs/ext4/dir.c

[PATCH] [4/18] ext3: Remove incorrect BKL comment

2008-01-26 Thread Andi Kleen
There is no BKL held on entry in -fsync nor in the low level ext3_sync_file. Cc: [EMAIL PROTECTED] Cc: [EMAIL PROTECTED] Signed-off-by: Andi Kleen [EMAIL PROTECTED] --- fs/ext3/dir.c |2 +- 1 file changed, 1 insertion(+), 1 deletion(-) Index: linux/fs/ext3/dir.c

[PATCH] [7/18] BKL-removal: Remove incorrect comments refering to BKL from ext4

2008-01-26 Thread Andi Kleen
BKL is not hold in any of those Cc: [EMAIL PROTECTED] Cc: [EMAIL PROTECTED] Signed-off-by: Andi Kleen [EMAIL PROTECTED] --- fs/ext4/dir.c |2 +- fs/ext4/inode.c |1 - 2 files changed, 1 insertion(+), 2 deletions(-) Index: linux/fs/ext4/dir.c

[PATCH] [14/18] BKL-removal: Add unlocked_fasync

2008-01-26 Thread Andi Kleen
and then the non unlocked async entry point could be dropped. Signed-off-by: Andi Kleen [EMAIL PROTECTED] --- Documentation/filesystems/vfs.txt |5 - fs/fcntl.c|6 +- fs/ioctl.c|5 - include/linux/fs.h|1 + 4

[PATCH] [9/18] BKL-removal: Use unlocked_ioctl for jfs

2008-01-26 Thread Andi Kleen
Convert jfs_ioctl over to not use the BKL. The only potential race I could see was with two ioctls in parallel changing the flags and losing the updates. Use the i_mutex to protect against this. Cc: [EMAIL PROTECTED] Signed-off-by: Andi Kleen [EMAIL PROTECTED] --- fs/jfs/file.c |2

[PATCH] [3/18] BKL-removal: Convert ext3 to use unlocked_ioctl

2008-01-26 Thread Andi Kleen
-by: Andi Kleen [EMAIL PROTECTED] --- fs/ext3/dir.c |2 +- fs/ext3/file.c |2 +- fs/ext3/ioctl.c | 21 +++-- include/linux/ext3_fs.h |3 +-- 4 files changed, 14 insertions(+), 14 deletions(-) Index: linux/fs/ext3/dir.c

[PATCH] [1/18] BKL-removal: Convert ext2 over to use unlocked_ioctl

2008-01-26 Thread Andi Kleen
I checked ext2_ioctl and could not find anything in there that would need the BKL. So convert it over to use unlocked_ioctl Signed-off-by: Andi Kleen [EMAIL PROTECTED] --- fs/ext2/dir.c |2 +- fs/ext2/ext2.h |3 +-- fs/ext2/file.c |4 ++-- fs/ext2/ioctl.c | 12 +++- 4

[PATCH] [2/18] BKL-removal: Remove incorrect BKL comment in ext2

2008-01-26 Thread Andi Kleen
No BKL used anywhere, so don't mention it. Signed-off-by: Andi Kleen [EMAIL PROTECTED] --- fs/ext2/inode.c |1 - 1 file changed, 1 deletion(-) Index: linux/fs/ext2/inode.c === --- linux.orig/fs/ext2/inode.c +++ linux/fs/ext2

[PATCH] [0/18] Implement some low hanging BKL removal fruit in fs/*

2008-01-26 Thread Andi Kleen
[Andrew: I believe this is -mm material for .25] - Convert some more file systems (generally those who don't use the BKL for anything except mount) to use unlocked_bkl. - Implement BKL less fasync (see patch for the rationale) This is currently a separate entry point, but since the number of

[PATCH] [10/18] BKL-removal: Implement a compat_ioctl handler for JFS

2008-01-26 Thread Andi Kleen
The ioctls were already compatible except for the actual values so this was fairly easy to do. Cc: [EMAIL PROTECTED] Signed-off-by: Andi Kleen [EMAIL PROTECTED] --- fs/jfs/file.c |3 +++ fs/jfs/ioctl.c | 18 ++ fs/jfs/jfs_dinode.h |2 ++ fs/jfs/jfs_inode.h

[PATCH] [15/18] BKL-removal: Convert pipe over to unlocked_fasync

2008-01-26 Thread Andi Kleen
Signed-off-by: Andi Kleen [EMAIL PROTECTED] --- fs/pipe.c | 12 ++-- 1 file changed, 6 insertions(+), 6 deletions(-) Index: linux/fs/pipe.c === --- linux.orig/fs/pipe.c +++ linux/fs/pipe.c @@ -788,7 +788,7 @@ const

[PATCH] [17/18] BKL-removal: Convert fuse to unlocked_fasync

2008-01-26 Thread Andi Kleen
Cc: [EMAIL PROTECTED] Signed-off-by: Andi Kleen [EMAIL PROTECTED] --- fs/fuse/dev.c |2 +- 1 file changed, 1 insertion(+), 1 deletion(-) Index: linux/fs/fuse/dev.c === --- linux.orig/fs/fuse/dev.c +++ linux/fs/fuse/dev.c

[PATCH] [18/18] BKL-removal: Convert bad_inode to unlocked_fasync

2008-01-26 Thread Andi Kleen
Not that it matters much, but it was easy. Signed-off-by: Andi Kleen [EMAIL PROTECTED] --- fs/bad_inode.c |2 +- 1 file changed, 1 insertion(+), 1 deletion(-) Index: linux/fs/bad_inode.c === --- linux.orig/fs/bad_inode.c

[PATCH] [16/18] BKL-removal: Convert socket fasync to unlocked_fasync

2008-01-26 Thread Andi Kleen
Cc: [EMAIL PROTECTED] Signed-off-by: Andi Kleen [EMAIL PROTECTED] --- net/socket.c |2 +- 1 file changed, 1 insertion(+), 1 deletion(-) Index: linux/net/socket.c === --- linux.orig/net/socket.c +++ linux/net/socket.c

[PATCH] [13/18] BKL-removal: Add compat_ioctl for cifs

2008-01-26 Thread Andi Kleen
Similar to the compat handlers of other file systems. The ioctls are compatible except that they have different numbers. Cc: [EMAIL PROTECTED] Signed-off-by: Andi Kleen [EMAIL PROTECTED] --- fs/cifs/cifsfs.c | 15 +++ fs/cifs/cifsfs.h |2 ++ fs/cifs/ioctl.c | 19

[PATCH] [12/18] BKL-removal: Convert CIFS over to unlocked_ioctl

2008-01-26 Thread Andi Kleen
cifs_ioctl doesn't seem to need the BKL for anything, so convert it over to use unlocked_ioctl. Cc: [EMAIL PROTECTED] Signed-off-by: Andi Kleen [EMAIL PROTECTED] --- fs/cifs/cifsfs.c | 10 +- fs/cifs/cifsfs.h |4 ++-- fs/cifs/ioctl.c |4 ++-- 3 files changed, 9 insertions

[PATCH] [11/18] BKL-removal: Convert ocfs2 over to unlocked_ioctl

2008-01-26 Thread Andi Kleen
As far as I can see there is nothing in ocfs2_ioctl that requires the BKL, so use unlocked_ioctl Cc: [EMAIL PROTECTED] Signed-off-by: Andi Kleen [EMAIL PROTECTED] --- fs/ocfs2/file.c |4 ++-- fs/ocfs2/ioctl.c | 12 +++- fs/ocfs2/ioctl.h |3 +-- 3 files changed, 6 insertions

[PATCH] BKL-Removal: Convert pipe to use unlocked_ioctl too

2008-01-26 Thread Andi Kleen
Here's another patch that was missing in the previous BKL-removal series. No BKL needed in pipe_ioctl Signed-off-by: Andi Kleen [EMAIL PROTECTED] Index: linux/fs/pipe.c === --- linux.orig/fs/pipe.c +++ linux/fs/pipe.c @@ -576,9

Re: [PATCH][RFC] fast file mapping for loop

2008-01-09 Thread Andi Kleen
Jens Axboe [EMAIL PROTECTED] writes: So how does it work? Instead of punting IO to a thread and passing it through the page cache, we instead attempt to send the IO directly to the Great -- something like this was needed for a long time. - The file block mappings must not change while loop

Re: [RFC] Remove BKL from fs/locks.c

2007-12-30 Thread Andi Kleen
Matthew Wilcox [EMAIL PROTECTED] writes: The blocked_list is a bit more complex since we need to check every lock on the blocked list, and would need to acquire all the sb_file_lock_locks to check this list consistently. I don't see a nice way to do this -- particularly when you consider

Re: [RFC] Remove BKL from fs/locks.c

2007-12-30 Thread Andi Kleen
The only problem I can see from an NFS perspective is with NFSv2/v3 locking: unfortunately the protocol provides no way for the server to notify that a lock may not be granted after the client has been told to block. You would therefore have to bend the protocol rules by simply delaying

Re: Beagle and logging inotify events

2007-11-14 Thread Andi Kleen
Jon Smirl [EMAIL PROTECTED] writes: On 11/14/07, Chuck Lever [EMAIL PROTECTED] wrote: On Nov 13, 2007, at 7:04 PM, Jon Smirl wrote: Is it feasible to do something like this in the linux file system architecture? Beagle beats on my disk for an hour when I reboot. Of course I don't like

Re: SLUB performance regression vs SLAB

2007-10-05 Thread Andi Kleen
Jens Axboe [EMAIL PROTECTED] writes: Writing a small test module to exercise slub/slab in various ways (allocating from all cpus freeing from one, as described) should not be too hard. Perhaps that would be enough to find this performance discrepancy between slab and slub? You could

Re: [patch 1/3] add the fsblock layer

2007-06-25 Thread Andi Kleen
On Sun, Jun 24, 2007 at 01:18:42PM -0700, Arjan van de Ven wrote: Hmm, could define a macro DECLARE_ATOMIC_BITMAP(maxbit) that expands to the smallest possible type for each architecture. And a couple of ugly casts for set_bit et.al. but those could be also hidden in macros. Should

Re: [RFC] fsblock

2007-06-24 Thread Andi Kleen
Nick Piggin [EMAIL PROTECTED] writes: - Structure packing. A page gets a number of buffer heads that are allocated in a linked list. fsblocks are allocated contiguously, so cacheline footprint is smaller in the above situation. It would be interesting to test if that makes a difference

Re: [patch 1/3] add the fsblock layer

2007-06-24 Thread Andi Kleen
Nick Piggin [EMAIL PROTECTED] writes: [haven't read everything, just commenting on something that caught my eye] +struct fsblock { + atomic_tcount; + union { + struct { + unsigned long flags; /* XXX: flags could be int for better packing

Re: NILFS version 2 now available

2007-06-17 Thread Andi Kleen
[EMAIL PROTECTED] writes: NILFS (a New Implementation of a Log-structured Filesystem) Version 2 have been available at the project website http://www.nilfs.org/ If you are interested, please visit to our website. Could you please give some information on the use cases for this file

Re: [PATCH] NFS: Make NFS root work again

2007-06-17 Thread Andi Kleen
Andrew Morton [EMAIL PROTECTED] writes: What is not working, and how does this patch fix it? FWIW i use nfs root regularly for testing kernels and it works for me. -Andi - To unsubscribe from this list: send the line unsubscribe linux-fsdevel in the body of a message to [EMAIL PROTECTED] More

Re: [RFC] TileFS - a proposal for scalable integrity checking

2007-04-29 Thread Andi Kleen
Matt Mackall [EMAIL PROTECTED] writes: This is a relatively simple scheme for making a filesystem with incremental online consistency checks of both data and metadata. Overhead can be well under 1% disk space and CPU overhead may also be very small, while greatly improving filesystem

Re: BUG: Dentry still in use during umount in 2.6.21-rc5-git6

2007-04-24 Thread Andi Kleen
On Tuesday 24 April 2007 12:40:24 Jan Kara wrote: One of my autoboot test clients gave me this during shutdown. It used reiserfs and autofs and NFS heavily. Jeff has a fix for this bug so it should go away soon... Thanks for report anyway :). Well I hit two more -- see other mails if

More reiserfs trouble in 2.6.21-rc5

2007-04-22 Thread Andi Kleen
FYI, This was a debugging kernel (preempt, slab debugging, lockdep etc. enabled) running autotest and some other load on a 4 core Opteron system There was also another lockdep warning before that which I'm sending separately. Looks like some memory corruption. Could be something else, but at

reiserfs lockdep warning in 2.6.21-rc5

2007-04-22 Thread Andi Kleen
=== [ INFO: possible circular locking dependency detected ] 2.6.21-rc5-git6 #44 --- perl/7968 is trying to acquire lock: (inode-i_mutex){--..}, at: [802ca7cd]

Re: [patch 7/8] allow unprivileged mounts

2007-04-21 Thread Andi Kleen
Andrew Morton [EMAIL PROTECTED] writes: On Fri, 20 Apr 2007 12:25:39 +0200 Miklos Szeredi [EMAIL PROTECTED] wrote: Define a new fs flag FS_SAFE, which denotes, that unprivileged mounting of this filesystem may not constitute a security problem. Since most filesystems haven't been

Re: AppArmor FAQ

2007-04-17 Thread Andi Kleen
On Tue, Apr 17, 2007 at 01:47:39PM -0400, James Morris wrote: Normal applications need zero modification under SELinux. Some applications which manage security may need to be made SELinux-aware, Anything that can touch /etc/resolv.conf? That's potentially a lot of binaries if you consider

Re: AppArmor FAQ

2007-04-17 Thread Andi Kleen
For SELinux to be effective it has to have a complete policy definition. This would prevent the OpenOffice access (unless OpenOffice is in the modify_resolv_conf_t domain) above. This would mean no fully functional root user anymore. My understanding is rather that at least in the Fedora

Re: AppArmor FAQ

2007-04-17 Thread Andi Kleen
The vast majority of applications are not modified to be SELinux aware - only a small handful of security aware applications are modified. All applications that can edit /etc/resolv.conf? That's nearly everything. You yourself gave the example; I'm not making anything up. -Andi (sensing a

Re: [AppArmor 39/41] AppArmor: Profile loading and manipulation, pathname matching

2007-04-16 Thread Andi Kleen
It's nice to check for consistency though, so we're adding that. Profile loading is a trusted operation, at least so far, and so security wise we don't actually have to care --- if loading an invalid profile can bring down the system, then that's no worse than an arbitrary module that

Re: [AppArmor 39/41] AppArmor: Profile loading and manipulation, pathname matching

2007-04-12 Thread Andi Kleen
[EMAIL PROTECTED] writes: [didn't review code fully, just some stuff I noticed] + +struct aa_dfa { + struct table_header *tables[YYTD_ID_NXT]; +}; If that is passed in from user space you would need special compat code for 64bit kernels who support 32bit userland. Better to avoid

Re: [take36 10/10] kevent: Kevent based generic AIO.

2007-02-12 Thread Andi Kleen
Evgeniy Polyakov [EMAIL PROTECTED] writes: aio_sendfile_path() is essentially aio_sendfile(), except that it takes source filename as parameter, has a pointer to private header and its size (which allows to send header and file's content in one syscall instead of three (open, send, sendfile)

Re: [PATCH] fix quadratic behavior of shrink_dcache_parent()

2007-02-10 Thread Andi Kleen
Miklos Szeredi [EMAIL PROTECTED] writes: So the second part of the problem is to somehow limit the number of dentries used. Not easy... OpenVZ has some existing work in this area to separate their virtual machines. I assume they will eventually submit it. -Andi - To unsubscribe from this

Re: GFS, what's remaining

2005-09-02 Thread Andi Kleen
Andrew Morton [EMAIL PROTECTED] writes: - Why GFS is better than OCFS2, or has functionality which OCFS2 cannot possibly gain (or vice versa) - Relative merits of the two offerings You missed the important one - people actively use it and have been for some years. Same

Re: File Locking in Linux 2.5

2001-05-04 Thread Andi Kleen
On Thu, May 03, 2001 at 06:03:07PM -0700, Jeremy Allison wrote: The only thing we need to fix (IMHO) is the close(dup(fd)) bug, which I have yet to see any application treat as anything other than a spec. bug that must be worked around. I don't think it's a bug right now; not dropping them

Re: File Locking in Linux 2.5

2001-05-03 Thread Andi Kleen
On Thu, May 03, 2001 at 08:36:13AM +0100, Matthew Wilcox wrote: I'll get to it this weekend then. Should be a relatively simple patch. Just don't forget to add a per user ulimit for it and probably an admin tool like ipcs. Are there any other semantics you want changing from the POSIX lock?

Re: Q: sb-s_bdev same device as sb-s_dev?

2001-03-11 Thread Andi Kleen
On Sat, Mar 10, 2001 at 08:44:41PM -0500, Alexander Viro wrote: Too many places are using -i_dev right now to eliminate -i_dev. Yes, it should eventually go away (and icache should work by -i_sb/-i_ino instead of -i_dev/-i_ino). -i_dev should eventually go away, but that was too late in 2.4

Re: Q: sb-s_bdev same device as sb-s_dev?

2001-03-11 Thread Andi Kleen
On Sun, Mar 11, 2001 at 04:55:24AM -0500, Alexander Viro wrote: On Sun, 11 Mar 2001, Andi Kleen wrote: On Sat, Mar 10, 2001 at 08:44:41PM -0500, Alexander Viro wrote: Too many places are using -i_dev right now to eliminate -i_dev. Yes, it should eventually go away (and icache

Re: block bitmap readahead for ext2

2001-01-24 Thread Andi Kleen
Quick question. On Wed, Jan 24, 2001 at 09:20:43AM -0600, Steve Lord wrote: What XFS also has is an ioctl to preallocate disk space, there is very little documentation on this (none), but if you look in the file cmd/xfstests/src/randholes.c you will see an ioctl like this: struct flock64

Re: block bitmap readahead for ext2

2001-01-24 Thread Andi Kleen
On Wed, Jan 24, 2001 at 11:00:41AM -0500, Jeff Darcy wrote: From: "Andi Kleen" [EMAIL PROTECTED] Does this mean that unwriten extents are supported now in pagebuf? Otherwise this ioctl would need to prezero the disks blocks to prevent old data from being leaked, right? I