[PATCH 2/2] audit: Two efficiency fixes for audit mechanism
audit: Two efficiency fixes for audit mechanism author: Dan Duval These and similar errors were seen on a patched 3.8 kernel when the audit subsystem was overrun during boot: udevd[876]: worker [887] unexpectedly returned with status 0x0100 udevd[876]: worker [887] failed while handling '/devices/pci:00/:00:03.0/:40:00.0' udevd[876]: worker [880] unexpectedly returned with status 0x0100 udevd[876]: worker [880] failed while handling '/devices/LNXSYSTM:00/LNXPWRBN:00/input/input1/event1' udevadm settle - timeout of 180 seconds reached, the event queue contains: /sys/devices/LNXSYSTM:00/LNXPWRBN:00/input/input1/event1 (3995) /sys/devices/LNXSYSTM:00/LNXSYBUS:00/PNP0A08:00/INT3F0D:00 (4034) audit: audit_backlog=258 > audit_backlog_limit=256 audit: audit_lost=1 audit_rate_limit=0 audit_backlog_limit=256 The changes below increase the efficiency of the audit code and prevent it from being overrun: 1. Only issue a wake_up in kauditd if the length of the skb queue is less than the backlog limit. Otherwise, threads waiting in wait_for_auditd() will simply wake up, discover that the queue is still too long for them to proceed, and go back to sleep. This results in wasted context switches and machine cycles. kauditd_thread() is the only function that removes buffers from audit_skb_queue so we can't race. If we did, the timeout in wait_for_auditd() would expire and the waiting thread would continue. 2. Use add_wait_queue_exclusive() in wait_for_auditd() to put the thread on the wait queue. When kauditd dequeues an skb, all of the waiting threads are waiting for the same resource, but only one is going to get it, so there's no need to wake up more than one waiter. Signed-off-by: Dan Duval Signed-off-by: Chuck Anderson --- kernel/audit.c |7 +-- 1 files changed, 5 insertions(+), 2 deletions(-) diff --git a/kernel/audit.c b/kernel/audit.c index 9a78dde..d87b4dd 100644 --- a/kernel/audit.c +++ b/kernel/audit.c @@ -449,8 +449,11 @@ static int kauditd_thread(void *dummy) flush_hold_queue(); skb = skb_dequeue(_skb_queue); - wake_up(_backlog_wait); + if (skb) { + if(skb_queue_len(_skb_queue) <= audit_backlog_limi t) + wake_up(_backlog_wait); + if (audit_pid) kauditd_send_skb(skb); else @@ -1059,7 +1062,7 @@ static void wait_for_auditd(unsigned long sleep_time, int limit) { DECLARE_WAITQUEUE(wait, current); set_current_state(TASK_UNINTERRUPTIBLE); - add_wait_queue(_backlog_wait, ); + add_wait_queue_exclusive(_backlog_wait, ); if (audit_backlog_limit && skb_queue_len(_skb_queue) > limit) -- 1.7.1 -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
[PATCH 1/2] audit: fix soft lockups due to loop in audit_log_start() wh,en audit_backlog_limit exceeded
audit: fix softlockups due to loop in audit_log_start() when audit_backlog_limit exceeded author: Dan Duval This patch fixes a bug in kernel/audit that can cause many soft lockups and prevent the boot of a large memory 3.8 system: BUG: soft lockup - CPU#66 stuck for 22s! [udevd:9559] RIP: 0010:[] [] audit_log_start+0xe6/0x350 Call Trace: [] ? try_to_wake_up+0x2d0/0x2d0 [] audit_log_exit+0x3f/0x590 [] __audit_syscall_exit+0x28d/0x2c0 [] sysret_audit+0x17/0x21 audit_log_start() will call wait_for_auditd() to delay returning an audit_buffer if there are too many SKBs on audit_skb_queue. wait_for_auditd() puts itself on the audit_backlog_wait queue and sleeps for sleep_time jiffies or until it is (normally) woken when kauditd takes an SKB off of audit_skb_queue. wait_for_auditd() returns to audit_log_start() which checks to see if audit_skb_queue still has too many SKBs. If there are still too many, audit_log_start() will continue to call wait_for_auditd() in a loop until audit_backlog_wait_time has passed. audit_log_start() will then complain ("backlog limit exceeded"); set audit_backlog_wait_time to NULL so other waiters will fall out of the loop when woken up; wake up any waiters in wait_for_auditd(); return NULL which tells the caller that an audit_buffer could not be allocated. A bug in audit_log_start() prevents it from breaking out of the wait_for_auditd() loop when audit_backlog_wait_time has passed. Instead, it will loop in the audit_skb_queue-is-too-long while-loop eventually causing a soft lockup. There can (and most likely will) be multiple threads looping. The fix is to continue in the while-loop only if sleep_time was greater than 0 (audit_backlog_wait_time has not passed). Another bug in audit_log_start() prevents audit_backlog_wait_time from working as expected. audit_backlog_wait_time is normally the time period that audit_log_start() will wait for the number of SKBs on audit_skb_queue to fall below the too-many threshold. If audit_backlog_wait_time passes, audit_log_start() will set it to audit_backlog_wait_overflow, which is zero, and wake up any waiters in wait_for_auditd(). audit_backlog_wait_time is now zero so the waiters will fall out of the loop when they return to audit_log_start(). That is expected behavior. But audit_backlog_wait_time is not reset to its initial value when audit_skb_queue's length is no longer too long. Subsequent calls to audit_log_start() when audit_skb_queue is too long will not wait in wait_for_auditd(), instead returning NULL. The fix is to set audit_backlog_wait_time to its initial value when audit_skb_queue passes the size test, potentially resetting it. A third issue is to have both audit_log_start() and wait_for_auditd() use the same limit value for the length of audit_skb_queue. It isn't necessary today but (1) assumptions may change in the future and (2) is one less oddity for a reader to have to verify. Signed-off-by: Dan Duval Signed-off-by: Chuck Anderson --- kernel/audit.c | 28 1 files changed, 16 insertions(+), 12 deletions(-) diff --git a/kernel/audit.c b/kernel/audit.c index 91e53d0..9a78dde 100644 --- a/kernel/audit.c +++ b/kernel/audit.c @@ -103,9 +103,11 @@ static int audit_rate_limit; /* Number of outstanding audit_buffers allowed. */ static int audit_backlog_limit = 64; -static int audit_backlog_wait_time = 60 * HZ; static int audit_backlog_wait_overflow = 0; +#define AUDIT_BACKLOG_WAIT_TIME (60 * HZ) +static int audit_backlog_wait_time = AUDIT_BACKLOG_WAIT_TIME; + /* The identity of the user shutting down the audit system. */ kuid_t audit_sig_uid = INVALID_UID; pid_t audit_sig_pid = -1; @@ -1053,14 +1055,14 @@ static inline void audit_get_stamp(struct audit_context *ctx, /* * Wait for auditd to drain the queue a little */ -static void wait_for_auditd(unsigned long sleep_time) +static void wait_for_auditd(unsigned long sleep_time, int limit) { DECLARE_WAITQUEUE(wait, current); set_current_state(TASK_UNINTERRUPTIBLE); add_wait_queue(_backlog_wait, ); if (audit_backlog_limit && - skb_queue_len(_skb_queue) > audit_backlog_limit) + skb_queue_len(_skb_queue) > limit) schedule_timeout(sleep_time); __set_current_state(TASK_RUNNING); @@ -1095,8 +1097,8 @@ struct audit_buffer *audit_log_start(struct audit_context *ctx, gfp_t gfp_mask, struct audit_buffer *ab = NULL; struct timespec t; unsigned intuninitialized_var(serial); - int reserve; unsigned long timeout_start = jiffies; + int limit; if (audit_initialized != AUDIT_INITIALIZED) return NULL; @@ -1104,22 +1106,22 @@ struct audit_buffer *audit_log_start(struct audit_contex t *ctx, gfp_t gfp_mask, if (unlikely(audit_filter_type(type))) return NUL
[PATCH 0/2] audit: fix soft lockups and udevd errors when audit is overrun
The two patches that follow in separate emails resolve soft lockups and udevd reported errors that prevented a large memory 3.8 system from booting. The patches are based on 3.11-rc7. I believe it is the same issue recently posted as: [RFC] audit: avoid soft lockup in audit_log_start() https://lkml.org/lkml/2013/8/28/626 The first patch: audit: fix soft lockups due to loop in audit_log_start() when audit_backlog_limit exceeded fixes a bug in kernel/audit that caused many soft lockups during boot: BUG: soft lockup - CPU#66 stuck for 22s! [udevd:9559] RIP: 0010:[] [] audit_log_start+0xe6/0x350 Call Trace: [] ? try_to_wake_up+0x2d0/0x2d0 [] audit_log_exit+0x3f/0x590 [] __audit_syscall_exit+0x28d/0x2c0 [] sysret_audit+0x17/0x21 The second patch: audit: Two efficiency fixes for audit mechanism prevents these and similar error messages repeated often during boot: udevd[876]: worker [887] unexpectedly returned with status 0x0100 udevd[876]: worker [887] failed while handling '/devices/pci:00/:00:03.0/:40:00.0' udevd[876]: worker [880] unexpectedly returned with status 0x0100 udevd[876]: worker [880] failed while handling '/devices/LNXSYSTM:00/LNXPWRBN:00/input/input1/event1' udevadm settle - timeout of 180 seconds reached, the event queue contains: /sys/devices/LNXSYSTM:00/LNXPWRBN:00/input/input1/event1 (3995) /sys/devices/LNXSYSTM:00/LNXSYBUS:00/PNP0A08:00/INT3F0D:00 (4034) audit: audit_backlog=258 > audit_backlog_limit=256 audit: audit_lost=1 audit_rate_limit=0 audit_backlog_limit=256 -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
[PATCH 0/2] audit: fix soft lockups and udevd errors when audit is overrun
The two patches that follow in separate emails resolve soft lockups and udevd reported errors that prevented a large memory 3.8 system from booting. The patches are based on 3.11-rc7. I believe it is the same issue recently posted as: [RFC] audit: avoid soft lockup in audit_log_start() https://lkml.org/lkml/2013/8/28/626 The first patch: audit: fix soft lockups due to loop in audit_log_start() when audit_backlog_limit exceeded fixes a bug in kernel/audit that caused many soft lockups during boot: BUG: soft lockup - CPU#66 stuck for 22s! [udevd:9559] RIP: 0010:[810d1d06] [810d1d06] audit_log_start+0xe6/0x350 Call Trace: [8108ea30] ? try_to_wake_up+0x2d0/0x2d0 [810d8d6f] audit_log_exit+0x3f/0x590 [810d975d] __audit_syscall_exit+0x28d/0x2c0 [815e0440] sysret_audit+0x17/0x21 The second patch: audit: Two efficiency fixes for audit mechanism prevents these and similar error messages repeated often during boot: udevd[876]: worker [887] unexpectedly returned with status 0x0100 udevd[876]: worker [887] failed while handling '/devices/pci:00/:00:03.0/:40:00.0' udevd[876]: worker [880] unexpectedly returned with status 0x0100 udevd[876]: worker [880] failed while handling '/devices/LNXSYSTM:00/LNXPWRBN:00/input/input1/event1' udevadm settle - timeout of 180 seconds reached, the event queue contains: /sys/devices/LNXSYSTM:00/LNXPWRBN:00/input/input1/event1 (3995) /sys/devices/LNXSYSTM:00/LNXSYBUS:00/PNP0A08:00/INT3F0D:00 (4034) audit: audit_backlog=258 audit_backlog_limit=256 audit: audit_lost=1 audit_rate_limit=0 audit_backlog_limit=256 -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
[PATCH 1/2] audit: fix soft lockups due to loop in audit_log_start() wh,en audit_backlog_limit exceeded
audit: fix softlockups due to loop in audit_log_start() when audit_backlog_limit exceeded author: Dan Duval dan.du...@oracle.com This patch fixes a bug in kernel/audit that can cause many soft lockups and prevent the boot of a large memory 3.8 system: BUG: soft lockup - CPU#66 stuck for 22s! [udevd:9559] RIP: 0010:[810d1d06] [810d1d06] audit_log_start+0xe6/0x350 Call Trace: [8108ea30] ? try_to_wake_up+0x2d0/0x2d0 [810d8d6f] audit_log_exit+0x3f/0x590 [810d975d] __audit_syscall_exit+0x28d/0x2c0 [815e0440] sysret_audit+0x17/0x21 audit_log_start() will call wait_for_auditd() to delay returning an audit_buffer if there are too many SKBs on audit_skb_queue. wait_for_auditd() puts itself on the audit_backlog_wait queue and sleeps for sleep_time jiffies or until it is (normally) woken when kauditd takes an SKB off of audit_skb_queue. wait_for_auditd() returns to audit_log_start() which checks to see if audit_skb_queue still has too many SKBs. If there are still too many, audit_log_start() will continue to call wait_for_auditd() in a loop until audit_backlog_wait_time has passed. audit_log_start() will then complain (backlog limit exceeded); set audit_backlog_wait_time to NULL so other waiters will fall out of the loop when woken up; wake up any waiters in wait_for_auditd(); return NULL which tells the caller that an audit_buffer could not be allocated. A bug in audit_log_start() prevents it from breaking out of the wait_for_auditd() loop when audit_backlog_wait_time has passed. Instead, it will loop in the audit_skb_queue-is-too-long while-loop eventually causing a soft lockup. There can (and most likely will) be multiple threads looping. The fix is to continue in the while-loop only if sleep_time was greater than 0 (audit_backlog_wait_time has not passed). Another bug in audit_log_start() prevents audit_backlog_wait_time from working as expected. audit_backlog_wait_time is normally the time period that audit_log_start() will wait for the number of SKBs on audit_skb_queue to fall below the too-many threshold. If audit_backlog_wait_time passes, audit_log_start() will set it to audit_backlog_wait_overflow, which is zero, and wake up any waiters in wait_for_auditd(). audit_backlog_wait_time is now zero so the waiters will fall out of the loop when they return to audit_log_start(). That is expected behavior. But audit_backlog_wait_time is not reset to its initial value when audit_skb_queue's length is no longer too long. Subsequent calls to audit_log_start() when audit_skb_queue is too long will not wait in wait_for_auditd(), instead returning NULL. The fix is to set audit_backlog_wait_time to its initial value when audit_skb_queue passes the size test, potentially resetting it. A third issue is to have both audit_log_start() and wait_for_auditd() use the same limit value for the length of audit_skb_queue. It isn't necessary today but (1) assumptions may change in the future and (2) is one less oddity for a reader to have to verify. Signed-off-by: Dan Duval dan.du...@oracle.com Signed-off-by: Chuck Anderson chuck.ander...@oracle.com --- kernel/audit.c | 28 1 files changed, 16 insertions(+), 12 deletions(-) diff --git a/kernel/audit.c b/kernel/audit.c index 91e53d0..9a78dde 100644 --- a/kernel/audit.c +++ b/kernel/audit.c @@ -103,9 +103,11 @@ static int audit_rate_limit; /* Number of outstanding audit_buffers allowed. */ static int audit_backlog_limit = 64; -static int audit_backlog_wait_time = 60 * HZ; static int audit_backlog_wait_overflow = 0; +#define AUDIT_BACKLOG_WAIT_TIME (60 * HZ) +static int audit_backlog_wait_time = AUDIT_BACKLOG_WAIT_TIME; + /* The identity of the user shutting down the audit system. */ kuid_t audit_sig_uid = INVALID_UID; pid_t audit_sig_pid = -1; @@ -1053,14 +1055,14 @@ static inline void audit_get_stamp(struct audit_context *ctx, /* * Wait for auditd to drain the queue a little */ -static void wait_for_auditd(unsigned long sleep_time) +static void wait_for_auditd(unsigned long sleep_time, int limit) { DECLARE_WAITQUEUE(wait, current); set_current_state(TASK_UNINTERRUPTIBLE); add_wait_queue(audit_backlog_wait, wait); if (audit_backlog_limit - skb_queue_len(audit_skb_queue) audit_backlog_limit) + skb_queue_len(audit_skb_queue) limit) schedule_timeout(sleep_time); __set_current_state(TASK_RUNNING); @@ -1095,8 +1097,8 @@ struct audit_buffer *audit_log_start(struct audit_context *ctx, gfp_t gfp_mask, struct audit_buffer *ab = NULL; struct timespec t; unsigned intuninitialized_var(serial); - int reserve; unsigned long timeout_start = jiffies; + int limit; if (audit_initialized != AUDIT_INITIALIZED) return NULL; @@ -1104,22 +1106,22 @@ struct
[PATCH 2/2] audit: Two efficiency fixes for audit mechanism
audit: Two efficiency fixes for audit mechanism author: Dan Duval dan.du...@oracle.com These and similar errors were seen on a patched 3.8 kernel when the audit subsystem was overrun during boot: udevd[876]: worker [887] unexpectedly returned with status 0x0100 udevd[876]: worker [887] failed while handling '/devices/pci:00/:00:03.0/:40:00.0' udevd[876]: worker [880] unexpectedly returned with status 0x0100 udevd[876]: worker [880] failed while handling '/devices/LNXSYSTM:00/LNXPWRBN:00/input/input1/event1' udevadm settle - timeout of 180 seconds reached, the event queue contains: /sys/devices/LNXSYSTM:00/LNXPWRBN:00/input/input1/event1 (3995) /sys/devices/LNXSYSTM:00/LNXSYBUS:00/PNP0A08:00/INT3F0D:00 (4034) audit: audit_backlog=258 audit_backlog_limit=256 audit: audit_lost=1 audit_rate_limit=0 audit_backlog_limit=256 The changes below increase the efficiency of the audit code and prevent it from being overrun: 1. Only issue a wake_up in kauditd if the length of the skb queue is less than the backlog limit. Otherwise, threads waiting in wait_for_auditd() will simply wake up, discover that the queue is still too long for them to proceed, and go back to sleep. This results in wasted context switches and machine cycles. kauditd_thread() is the only function that removes buffers from audit_skb_queue so we can't race. If we did, the timeout in wait_for_auditd() would expire and the waiting thread would continue. 2. Use add_wait_queue_exclusive() in wait_for_auditd() to put the thread on the wait queue. When kauditd dequeues an skb, all of the waiting threads are waiting for the same resource, but only one is going to get it, so there's no need to wake up more than one waiter. Signed-off-by: Dan Duval dan.du...@oracle.com Signed-off-by: Chuck Anderson chuck.ander...@oracle.com --- kernel/audit.c |7 +-- 1 files changed, 5 insertions(+), 2 deletions(-) diff --git a/kernel/audit.c b/kernel/audit.c index 9a78dde..d87b4dd 100644 --- a/kernel/audit.c +++ b/kernel/audit.c @@ -449,8 +449,11 @@ static int kauditd_thread(void *dummy) flush_hold_queue(); skb = skb_dequeue(audit_skb_queue); - wake_up(audit_backlog_wait); + if (skb) { + if(skb_queue_len(audit_skb_queue) = audit_backlog_limi t) + wake_up(audit_backlog_wait); + if (audit_pid) kauditd_send_skb(skb); else @@ -1059,7 +1062,7 @@ static void wait_for_auditd(unsigned long sleep_time, int limit) { DECLARE_WAITQUEUE(wait, current); set_current_state(TASK_UNINTERRUPTIBLE); - add_wait_queue(audit_backlog_wait, wait); + add_wait_queue_exclusive(audit_backlog_wait, wait); if (audit_backlog_limit skb_queue_len(audit_skb_queue) limit) -- 1.7.1 -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
vm86.c audit_syscall_exit() call trashes registers
Please Cc: any replies, as we are not subscribed to linux-kernel. Thanks. Somewhere around 2.6.16.12 a call to audit_syscall_exit was added to vm86.c: static void do_sys_vm86(struct kernel_vm86_struct *info, struct task_struct *tsk) { struct tss_struct *tss; + long eax; /* * make sure the vm86() system call doesn't try to do anything silly */ @@ -305,13 +307,19 @@ static void do_sys_vm86(struct kernel_vm tsk->thread.screen_bitmap = info->screen_bitmap; if (info->flags & VM86_SCREEN_BITMAP) mark_screen_rdonly(tsk->mm); + __asm__ __volatile__("xorl %eax,%eax; movl %eax,%fs; movl %eax,%gs\n\t"); + __asm__ __volatile__("movl %%eax, %0\n" :"=r"(eax)); + + /*call audit_syscall_exit since we do not exit via the normal paths */ + if (unlikely(current->audit_context)) + audit_syscall_exit(current, AUDITSC_RESULT(eax), eax); + __asm__ __volatile__( - "xorl %%eax,%%eax; movl %%eax,%%fs; movl %%eax,%%gs\n\t" "movl %0,%%esp\n\t" "movl %1,%%ebp\n\t" "jmp resume_userspace" : /* no outputs */ - :"r" (>regs), "r" (task_thread_info(tsk)) : "ax"); + :"r" (>regs), "r" (task_thread_info(tsk))); /* we never return here */ } This appears to have caused intermittent data corruption of the results of the vm86() call that the X server uses to get EDID data from the monitor via the VESA BIOS. After removing the audit_syscall_exit() call, the problems mentioned in these bugzillas disappear: Fetch of EDID 128 byte buffer by X server through vm86 INT 10 call is flaky. http://bugzilla.kernel.org/show_bug.cgi?id=8633 RHEL 5 fails to get EDID data from monitor and sets low resolution https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=236416 If I'm reading correctly, it appears that the code above trashes the %fs and %gs registers, or otherwise doesn't leave them at zero before returning from the system call as the old code did. Is this a correct analysis? How should this be fixed? Thanks. -Chuck - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
vm86.c audit_syscall_exit() call trashes registers
Please Cc: any replies, as we are not subscribed to linux-kernel. Thanks. Somewhere around 2.6.16.12 a call to audit_syscall_exit was added to vm86.c: static void do_sys_vm86(struct kernel_vm86_struct *info, struct task_struct *tsk) { struct tss_struct *tss; + long eax; /* * make sure the vm86() system call doesn't try to do anything silly */ @@ -305,13 +307,19 @@ static void do_sys_vm86(struct kernel_vm tsk-thread.screen_bitmap = info-screen_bitmap; if (info-flags VM86_SCREEN_BITMAP) mark_screen_rdonly(tsk-mm); + __asm__ __volatile__(xorl %eax,%eax; movl %eax,%fs; movl %eax,%gs\n\t); + __asm__ __volatile__(movl %%eax, %0\n :=r(eax)); + + /*call audit_syscall_exit since we do not exit via the normal paths */ + if (unlikely(current-audit_context)) + audit_syscall_exit(current, AUDITSC_RESULT(eax), eax); + __asm__ __volatile__( - xorl %%eax,%%eax; movl %%eax,%%fs; movl %%eax,%%gs\n\t movl %0,%%esp\n\t movl %1,%%ebp\n\t jmp resume_userspace : /* no outputs */ - :r (info-regs), r (task_thread_info(tsk)) : ax); + :r (info-regs), r (task_thread_info(tsk))); /* we never return here */ } This appears to have caused intermittent data corruption of the results of the vm86() call that the X server uses to get EDID data from the monitor via the VESA BIOS. After removing the audit_syscall_exit() call, the problems mentioned in these bugzillas disappear: Fetch of EDID 128 byte buffer by X server through vm86 INT 10 call is flaky. http://bugzilla.kernel.org/show_bug.cgi?id=8633 RHEL 5 fails to get EDID data from monitor and sets low resolution https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=236416 If I'm reading correctly, it appears that the code above trashes the %fs and %gs registers, or otherwise doesn't leave them at zero before returning from the system call as the old code did. Is this a correct analysis? How should this be fixed? Thanks. -Chuck - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/