Re: [PATCH] hugetlbfs read() support

2007-07-19 Thread Bill Irwin
On Thu, Jul 19, 2007 at 10:07:59AM -0700, Nishanth Aravamudan wrote: > But I do think a second reason to do this is to make hugetlbfs behave > like a normal fs -- that is read(), write(), etc. work on files in the > mountpoint. But that is simply my opinion. Mine as well. -- wli - To

Re: [PATCH] hugetlbfs read() support

2007-07-19 Thread Bill Irwin
On Thu, Jul 19, 2007 at 10:07:59AM -0700, Nishanth Aravamudan wrote: But I do think a second reason to do this is to make hugetlbfs behave like a normal fs -- that is read(), write(), etc. work on files in the mountpoint. But that is simply my opinion. Mine as well. -- wli - To unsubscribe

Re: [2/6] add config option to vmalloc stacks (was: Re: [-mm patch] i386: enable 4k stacks by default)

2007-05-04 Thread Bill Irwin
On Mon, Apr 30, 2007 at 10:43:10AM -0700, William Lee Irwin III wrote: >> + Allocates the stack physically discontiguously and from high >> + memory. Furthermore an unmapped guard page follows the stack. >> + This is not for end-users. It's intended to trigger fatal >> + system

Re: + per-cpuset-hugetlb-accounting-and-administration.patch added to -mm tree

2007-05-04 Thread Bill Irwin
On Thu, May 03, 2007 at 06:38:21PM -0700, Paul Jackson wrote: > Adding Christoph Lameter <[EMAIL PROTECTED]> to the cc list, as he knows > more about hugetlb pages than I do. > This patch strikes me as a bit odd. > Granted, it's solving what could be a touchy problem with a fairly > simple

Re: + per-cpuset-hugetlb-accounting-and-administration.patch added to -mm tree

2007-05-04 Thread Bill Irwin
On Thu, May 03, 2007 at 06:38:21PM -0700, Paul Jackson wrote: Adding Christoph Lameter [EMAIL PROTECTED] to the cc list, as he knows more about hugetlb pages than I do. This patch strikes me as a bit odd. Granted, it's solving what could be a touchy problem with a fairly simple solution,

Re: [2/6] add config option to vmalloc stacks (was: Re: [-mm patch] i386: enable 4k stacks by default)

2007-05-04 Thread Bill Irwin
On Mon, Apr 30, 2007 at 10:43:10AM -0700, William Lee Irwin III wrote: + Allocates the stack physically discontiguously and from high + memory. Furthermore an unmapped guard page follows the stack. + This is not for end-users. It's intended to trigger fatal + system errors

Re: [PATCH] i386: fix suspend/resume with dynamically allocated irq stacks

2007-05-03 Thread Bill Irwin
On Thu, May 03, 2007 at 12:39:30AM -0700, Bill Irwin wrote: > As an aside, it looks like failures here need to eventually propagate > to __cpu_up(). irq_ctx_init() needs to return a status, and its callers > need to check it. irq_ctx_init() probably also needs to be __cpuinit.

Re: [PATCH] i386: fix suspend/resume with dynamically allocated irq stacks

2007-05-03 Thread Bill Irwin
Bill Irwin wrote: >> I had the same question about yours and just brute-force merged. Not a >> big deal for me to rediff against whatever everyone's working off of. On Thu, May 03, 2007 at 12:07:29AM -0700, Jeremy Fitzhardinge wrote: > I picked up one version of your patches you

Re: [PATCH] i386: fix suspend/resume with dynamically allocated irq stacks

2007-05-03 Thread Bill Irwin
On Wed, May 02, 2007 at 10:48:09PM -0700, Bill Irwin wrote: > Updated patch follows. Please add your Signed-off-by: if it meets your > approval; I am operating on the assumption I should never do so myself. > I'm a bit unsure of how to handle cpu 0 vs. potential freeing of per_cp

Re: [PATCH] i386: fix suspend/resume with dynamically allocated irq stacks

2007-05-03 Thread Bill Irwin
Bill Irwin wrote: >> Updated patch follows. Please add your Signed-off-by: if it meets your >> approval; On Wed, May 02, 2007 at 11:01:05PM -0700, Jeremy Fitzhardinge wrote: > What does it apply to? I'm getting conflicts if I replace my patch with > this. Or does it replace o

Re: [PATCH] i386: fix suspend/resume with dynamically allocated irq stacks

2007-05-03 Thread Bill Irwin
Bill Irwin wrote: Updated patch follows. Please add your Signed-off-by: if it meets your approval; On Wed, May 02, 2007 at 11:01:05PM -0700, Jeremy Fitzhardinge wrote: What does it apply to? I'm getting conflicts if I replace my patch with this. Or does it replace one of your patches? I

Re: [PATCH] i386: fix suspend/resume with dynamically allocated irq stacks

2007-05-03 Thread Bill Irwin
On Wed, May 02, 2007 at 10:48:09PM -0700, Bill Irwin wrote: Updated patch follows. Please add your Signed-off-by: if it meets your approval; I am operating on the assumption I should never do so myself. I'm a bit unsure of how to handle cpu 0 vs. potential freeing of per_cpu areas and error

Re: [PATCH] i386: fix suspend/resume with dynamically allocated irq stacks

2007-05-03 Thread Bill Irwin
Bill Irwin wrote: I had the same question about yours and just brute-force merged. Not a big deal for me to rediff against whatever everyone's working off of. On Thu, May 03, 2007 at 12:07:29AM -0700, Jeremy Fitzhardinge wrote: I picked up one version of your patches you posted a couple

Re: [PATCH] i386: fix suspend/resume with dynamically allocated irq stacks

2007-05-03 Thread Bill Irwin
On Thu, May 03, 2007 at 12:39:30AM -0700, Bill Irwin wrote: As an aside, it looks like failures here need to eventually propagate to __cpu_up(). irq_ctx_init() needs to return a status, and its callers need to check it. irq_ctx_init() probably also needs to be __cpuinit. Ignoring the general

Re: [PATCH] i386: fix suspend/resume with dynamically allocated irq stacks

2007-05-02 Thread Bill Irwin
On Wed, May 02, 2007 at 06:56:09PM -0700, Jeremy Fitzhardinge wrote: > This fixes two bugs: > - the stack allocation must be marked __cpuinit, since it gets called >on resume as well. > - presumably the interrupt stack should be freed on unplug if its >going to get reallocated on every

Re: [PATCH] i386: fix suspend/resume with dynamically allocated irq stacks

2007-05-02 Thread Bill Irwin
cpu)); >> + >> +for (i = 0; i < THREAD_SIZE/PAGE_SIZE; ++i) >> +__free_page(per_cpu(irqstack_pages, cpu)[i]); >> +} On Wed, May 02, 2007 at 07:25:34PM -0700, Bill Irwin wrote: [...] Not sure if cpu 0 can ever be offlined, but it is remapped and so on. So

Re: [PATCH] i386: fix suspend/resume with dynamically allocated irq stacks

2007-05-02 Thread Bill Irwin
On Wed, May 02, 2007 at 06:56:09PM -0700, Jeremy Fitzhardinge wrote: > +static void __cpuinit __free_irqstack(int cpu, void *stk) > +{ > + int i; > + > + if (!cpu) > + return; > + > + unmap_vm_area(per_cpu(irqstack_area, cpu)); > + > + for (i = 0; i <

Re: [4/6] go BUG on vmallocspace in __pa()

2007-05-02 Thread Bill Irwin
William Lee Irwin III wrote: >> +unsigned long __kvaddr_to_paddr(unsigned long kvaddr) >> +{ >> +if (high_memory) >> +BUG_ON(kvaddr >= VMALLOC_START); >> +else >> +BUG_ON(kvaddr >= (unsigned long)__va(MAXMEM)); >> +return kvaddr - PAGE_OFFSET; >> +} On Wed, May

Re: [RFC BUG?] dereference PAGE_OFFSET address (rc7-mm2)

2007-05-02 Thread Bill Irwin
On Wed, May 02, 2007 at 09:28:46AM -0700, Jeremy Fitzhardinge wrote: >>> I think this should be fixed now. Eric made all those writes >>> unconditional (to fix a problem with PSE superpages not being created). >>> The patch is in Andi's queue. Bill Irwin <[EMAIL P

Re: [RFC BUG?] dereference PAGE_OFFSET address (rc7-mm2)

2007-05-02 Thread Bill Irwin
Bill Irwin wrote: >> Brain dump before crashing for the night: >> The patch refuses to clobber already-present pagetable entries of >> whatever origin. There are pagetables prior to this setup covering the >> address range just above PAGE_OFFSET. If this theory is cor

Re: [RFC BUG?] dereference PAGE_OFFSET address (rc7-mm2)

2007-05-02 Thread Bill Irwin
> memory. Probably it does. Probably we'd prefer that it didn't ;) On Wed, May 02, 2007 at 01:46:17AM -0700, Bill Irwin wrote: > Brown paper bag time. I don't know how it got past me. Brain dump before crashing for the night: The patch refuses to clobber already-present pag

Re: [RFC BUG?] dereference PAGE_OFFSET address (rc7-mm2)

2007-05-02 Thread Bill Irwin
On Wed, May 02, 2007 at 12:51:40AM -0700, Andrew Morton wrote: > Thanks for the report. I can reproduce it. > Bisection shows that x86_64-mm-paravirt-initial-pagetable.patch caused > this. > I didn't check whether the patch actually permits us to read kernel > memory. Probably it does. Probably

Re: [RFC BUG?] dereference PAGE_OFFSET address (rc7-mm2)

2007-05-02 Thread Bill Irwin
On Wed, May 02, 2007 at 12:51:40AM -0700, Andrew Morton wrote: Thanks for the report. I can reproduce it. Bisection shows that x86_64-mm-paravirt-initial-pagetable.patch caused this. I didn't check whether the patch actually permits us to read kernel memory. Probably it does. Probably we'd

Re: [RFC BUG?] dereference PAGE_OFFSET address (rc7-mm2)

2007-05-02 Thread Bill Irwin
prefer that it didn't ;) On Wed, May 02, 2007 at 01:46:17AM -0700, Bill Irwin wrote: Brown paper bag time. I don't know how it got past me. Brain dump before crashing for the night: The patch refuses to clobber already-present pagetable entries of whatever origin. There are pagetables prior

Re: [RFC BUG?] dereference PAGE_OFFSET address (rc7-mm2)

2007-05-02 Thread Bill Irwin
Bill Irwin wrote: Brain dump before crashing for the night: The patch refuses to clobber already-present pagetable entries of whatever origin. There are pagetables prior to this setup covering the address range just above PAGE_OFFSET. If this theory is correct, you should only be able to go

Re: [RFC BUG?] dereference PAGE_OFFSET address (rc7-mm2)

2007-05-02 Thread Bill Irwin
On Wed, May 02, 2007 at 09:28:46AM -0700, Jeremy Fitzhardinge wrote: I think this should be fixed now. Eric made all those writes unconditional (to fix a problem with PSE superpages not being created). The patch is in Andi's queue. Bill Irwin [EMAIL PROTECTED] writes: It needs verification

Re: [4/6] go BUG on vmallocspace in __pa()

2007-05-02 Thread Bill Irwin
William Lee Irwin III wrote: +unsigned long __kvaddr_to_paddr(unsigned long kvaddr) +{ +if (high_memory) +BUG_ON(kvaddr = VMALLOC_START); +else +BUG_ON(kvaddr = (unsigned long)__va(MAXMEM)); +return kvaddr - PAGE_OFFSET; +} On Wed, May 02, 2007 at

Re: [PATCH] i386: fix suspend/resume with dynamically allocated irq stacks

2007-05-02 Thread Bill Irwin
On Wed, May 02, 2007 at 06:56:09PM -0700, Jeremy Fitzhardinge wrote: +static void __cpuinit __free_irqstack(int cpu, void *stk) +{ + int i; + + if (!cpu) + return; + + unmap_vm_area(per_cpu(irqstack_area, cpu)); + + for (i = 0; i THREAD_SIZE/PAGE_SIZE; ++i)

Re: [PATCH] i386: fix suspend/resume with dynamically allocated irq stacks

2007-05-02 Thread Bill Irwin
) +__free_page(per_cpu(irqstack_pages, cpu)[i]); +} On Wed, May 02, 2007 at 07:25:34PM -0700, Bill Irwin wrote: [...] Not sure if cpu 0 can ever be offlined, but it is remapped and so on. So its virtual mapping should be undone if it can be offlined, but we probably shouldn't attempt to free

Re: [PATCH] i386: fix suspend/resume with dynamically allocated irq stacks

2007-05-02 Thread Bill Irwin
On Wed, May 02, 2007 at 06:56:09PM -0700, Jeremy Fitzhardinge wrote: This fixes two bugs: - the stack allocation must be marked __cpuinit, since it gets called on resume as well. - presumably the interrupt stack should be freed on unplug if its going to get reallocated on every plug.

Re: [2/6] add config option to vmalloc stacks (was: Re: [-mm patch] i386: enable 4k stacks by default)

2007-05-01 Thread Bill Irwin
On Tue, May 01, 2007 at 05:36:06PM -0500, Matt Mackall wrote: >>> Can we register them lazily at request_irq time? On Tue, May 01, 2007 at 03:51:25PM -0700, Bill Irwin wrote: >> These IRQ stacks are per-cpu, not per-IRQ. It may make sense to >> implement per-IRQ stacks,

Re: [2/6] add config option to vmalloc stacks (was: Re: [-mm patch] i386: enable 4k stacks by default)

2007-05-01 Thread Bill Irwin
At some point in the past, I wrote: >> These IRQ stacks are per-cpu, not per-IRQ. It may make sense to >> implement per-IRQ stacks, in which case dynamic allocation at the time >> of request_irq() will make sense. On Wed, May 02, 2007 at 12:07:45AM +0100, Alan Cox wrote: > This depends if active

Re: per-thread rusage

2007-05-01 Thread Bill Irwin
At some point in the past, I wrote: >> I just so happen to think we should implement a variety of CPU resource >> limits beyond what we now do, so this, too, interests me. On Wed, May 02, 2007 at 12:04:58AM +0100, Alan Cox wrote: > Agreed - and make them all 64bit while doing the cleanup. One

Re: [2/6] add config option to vmalloc stacks (was: Re: [-mm patch] i386: enable 4k stacks by default)

2007-05-01 Thread Bill Irwin
On Mon, Apr 30, 2007 at 08:15:11PM +0100, Christoph Hellwig wrote: >> So if you want to invest some time into getting this into mergeable >> shape I'd suggest you redo the patch series in the following way: >> patch 1: dynamic allocated irq stacks On Tue, May 01, 2007 at 05:36:06PM -0500, Matt

Re: per-thread rusage

2007-05-01 Thread Bill Irwin
On Tue, May 01, 2007 at 06:27:24PM -0400, Theodore Tso wrote: > There are two ways of implementing this. One is to have the > JVM periodically poll using a pthread_getrusage() interface. A better > choice might be some kind of per-thread CPU limit, that would result > in a thread-specific

Re: per-thread rusage

2007-05-01 Thread Bill Irwin
On 5/1/07, Bill Irwin <[EMAIL PROTECTED]> wrote: >> The basic >> idea is to try to do it similarly to how everyone else does so userspace >> (I suppose this would include glibc) don't have to bend over backward to >> accommodate it. Or basically to do what everyone

Re: per-thread rusage

2007-05-01 Thread Bill Irwin
On 5/1/07, Bill Irwin <[EMAIL PROTECTED]> wrote: >> A sort of note for me to refer back to when I get the rest of the way >> here. AIX does this with getrusage(RUSAGE_THREAD,...), Solaris with >> getrusage(RUSAGE_LWP,...), On Tue, May 01, 2007 at 11:39:46AM -07

Re: per-thread rusage

2007-05-01 Thread Bill Irwin
On Mon, Apr 09, 2007 at 04:53:15PM -0700, Andrew Morton wrote: >> Seems sane. Could we please get it tested and get a full description in >> place? Something which provides enough detail for the manpage maintainers. >> Also, a quick comparison between Linux's RUSAGE_THREAD and $other-os's >>

Re: [1/3] dynamically allocate IRQ stacks

2007-05-01 Thread Bill Irwin
At some point in the past, I wrote: >> +static void * __init __alloc_irqstack(int cpu) >> +{ >> +if (!cpu) >> +return __alloc_bootmem(THREAD_SIZE, THREAD_SIZE, >> +__pa(MAX_DMA_ADDRESS)); >> + >> +return (void

Re: [PATCH] [30/30] x86_64: Add missing !X86_PAE dependincy to the 2G/2G split.

2007-05-01 Thread Bill Irwin
Bill Irwin a écrit : >> as a stopgap measure, but I'm not all that interested in grabbing patch >> credits where others could do it easily enough. Either of the config >> alterations is fine by me as they now stand; maybe Eric Dumazet might >> care to do something like my

Re: [3/3] use vmalloc() to arrange guard pages for stacks

2007-05-01 Thread Bill Irwin
Bill Irwin wrote: >> It's a shame that the resource scalability implications of vmallocspace >> allocations prevent this from being useful in production. One could, in >> principle, establish guard pages within ZONE_NORMAL, but for 4KB stacks >> it's somewhat awkward to dre

Re: [3/3] use vmalloc() to arrange guard pages for stacks

2007-05-01 Thread Bill Irwin
Bill Irwin wrote: >> This patch introduces CONFIG_DEBUG_STACK, which vmalloc()'s task and IRQ >> stacks in order to establish guard pages. In such a manner any stack >> overflow that references pages immediately adjacent to the stack is >> immediately trapped with a fault

Re: [PATCH] [30/30] x86_64: Add missing !X86_PAE dependincy to the 2G/2G split.

2007-05-01 Thread Bill Irwin
On Tue, May 01, 2007 at 06:26:23AM +0200, Eric Dumazet wrote: >> Hum... We lose a usefull 2G/2G split. Should'nt we use a patch to change >> PAGE_OFFSET to 0x800 instead of 0x7800 and keep 2G/2G split ? On Tue, May 01, 2007 at 08:21:32AM +0200, Andi Kleen wrote: > I dropped the patch for

Re: [PATCH] [30/30] x86_64: Add missing !X86_PAE dependincy to the 2G/2G split.

2007-05-01 Thread Bill Irwin
On Tue, May 01, 2007 at 06:26:23AM +0200, Eric Dumazet wrote: Hum... We lose a usefull 2G/2G split. Should'nt we use a patch to change PAGE_OFFSET to 0x800 instead of 0x7800 and keep 2G/2G split ? On Tue, May 01, 2007 at 08:21:32AM +0200, Andi Kleen wrote: I dropped the patch for now.

Re: [3/3] use vmalloc() to arrange guard pages for stacks

2007-05-01 Thread Bill Irwin
Bill Irwin wrote: This patch introduces CONFIG_DEBUG_STACK, which vmalloc()'s task and IRQ stacks in order to establish guard pages. In such a manner any stack overflow that references pages immediately adjacent to the stack is immediately trapped with a fault, which precludes silent memory

Re: [3/3] use vmalloc() to arrange guard pages for stacks

2007-05-01 Thread Bill Irwin
Bill Irwin wrote: It's a shame that the resource scalability implications of vmallocspace allocations prevent this from being useful in production. One could, in principle, establish guard pages within ZONE_NORMAL, but for 4KB stacks it's somewhat awkward to dredge up 3 contigous pages, and 4

Re: [PATCH] [30/30] x86_64: Add missing !X86_PAE dependincy to the 2G/2G split.

2007-05-01 Thread Bill Irwin
Bill Irwin a écrit : as a stopgap measure, but I'm not all that interested in grabbing patch credits where others could do it easily enough. Either of the config alterations is fine by me as they now stand; maybe Eric Dumazet might care to do something like my suggestion at some point. On Tue

Re: per-thread rusage

2007-05-01 Thread Bill Irwin
On Mon, Apr 09, 2007 at 04:53:15PM -0700, Andrew Morton wrote: Seems sane. Could we please get it tested and get a full description in place? Something which provides enough detail for the manpage maintainers. Also, a quick comparison between Linux's RUSAGE_THREAD and $other-os's

Re: [1/3] dynamically allocate IRQ stacks

2007-05-01 Thread Bill Irwin
At some point in the past, I wrote: +static void * __init __alloc_irqstack(int cpu) +{ +if (!cpu) +return __alloc_bootmem(THREAD_SIZE, THREAD_SIZE, +__pa(MAX_DMA_ADDRESS)); + +return (void *)__get_free_pages(GFP_KERNEL, +

Re: per-thread rusage

2007-05-01 Thread Bill Irwin
On 5/1/07, Bill Irwin [EMAIL PROTECTED] wrote: A sort of note for me to refer back to when I get the rest of the way here. AIX does this with getrusage(RUSAGE_THREAD,...), Solaris with getrusage(RUSAGE_LWP,...), On Tue, May 01, 2007 at 11:39:46AM -0700, Ulrich Drepper wrote: RUSAGE_LWP

Re: per-thread rusage

2007-05-01 Thread Bill Irwin
On 5/1/07, Bill Irwin [EMAIL PROTECTED] wrote: The basic idea is to try to do it similarly to how everyone else does so userspace (I suppose this would include glibc) don't have to bend over backward to accommodate it. Or basically to do what everyone expects. On Tue, May 01, 2007 at 03:10

Re: per-thread rusage

2007-05-01 Thread Bill Irwin
On Tue, May 01, 2007 at 06:27:24PM -0400, Theodore Tso wrote: There are two ways of implementing this. One is to have the JVM periodically poll using a pthread_getrusage() interface. A better choice might be some kind of per-thread CPU limit, that would result in a thread-specific

Re: [2/6] add config option to vmalloc stacks (was: Re: [-mm patch] i386: enable 4k stacks by default)

2007-05-01 Thread Bill Irwin
On Mon, Apr 30, 2007 at 08:15:11PM +0100, Christoph Hellwig wrote: So if you want to invest some time into getting this into mergeable shape I'd suggest you redo the patch series in the following way: patch 1: dynamic allocated irq stacks On Tue, May 01, 2007 at 05:36:06PM -0500, Matt Mackall

Re: per-thread rusage

2007-05-01 Thread Bill Irwin
At some point in the past, I wrote: I just so happen to think we should implement a variety of CPU resource limits beyond what we now do, so this, too, interests me. On Wed, May 02, 2007 at 12:04:58AM +0100, Alan Cox wrote: Agreed - and make them all 64bit while doing the cleanup. One thing

Re: [2/6] add config option to vmalloc stacks (was: Re: [-mm patch] i386: enable 4k stacks by default)

2007-05-01 Thread Bill Irwin
At some point in the past, I wrote: These IRQ stacks are per-cpu, not per-IRQ. It may make sense to implement per-IRQ stacks, in which case dynamic allocation at the time of request_irq() will make sense. On Wed, May 02, 2007 at 12:07:45AM +0100, Alan Cox wrote: This depends if active IRQ

Re: [2/6] add config option to vmalloc stacks (was: Re: [-mm patch] i386: enable 4k stacks by default)

2007-05-01 Thread Bill Irwin
On Tue, May 01, 2007 at 05:36:06PM -0500, Matt Mackall wrote: Can we register them lazily at request_irq time? On Tue, May 01, 2007 at 03:51:25PM -0700, Bill Irwin wrote: These IRQ stacks are per-cpu, not per-IRQ. It may make sense to implement per-IRQ stacks, in which case dynamic allocation

Re: 2.6.22 -mm merge plans

2007-04-30 Thread Bill Irwin
On Mon, Apr 30, 2007 at 04:20:07PM -0700, Andrew Morton wrote: > proper-prototype-for-hugetlb_get_unmapped_area.patch ... > convert-hugetlbfs-to-use-vm_ops-fault.patch ... > get_unmapped_area-handles-map_fixed-in-hugetlbfs.patch ... > get_unmapped_area-doesnt-need-hugetlbfs-hacks-anymore.patch

[3/3] use vmalloc() to arrange guard pages for stacks

2007-04-30 Thread Bill Irwin
This patch introduces CONFIG_DEBUG_STACK, which vmalloc()'s task and IRQ stacks in order to establish guard pages. In such a manner any stack overflow that references pages immediately adjacent to the stack is immediately trapped with a fault, which precludes silent memory corruption or

[2/3] unconditional i386 IRQ stacks

2007-04-30 Thread Bill Irwin
IRQ stacks are a valuable stability feature. This patch makes them unconditional, as there is no circumstance under which they do not improve stability and they have no meaningful performance impact. Signed-off-by: William Irwin <[EMAIL PROTECTED]> Index: stack-paranoia/include/asm-i386/irq.h

[1/3] dynamically allocate IRQ stacks

2007-04-30 Thread Bill Irwin
Dynamically allocate IRQ stacks in order to conserve memory when using IRQ stacks. cpu_possible_map is not now initialized in such a manner as to provide a meaningful indication of how many CPU's might be in the system, and features to appear in the sequel also require indirection, so they

[0/3] i386 stack handling updates

2007-04-30 Thread Bill Irwin
This is a series of patches to improve i386 stack handling in several manners. The net result is to improve system stability via IRQ stacks while remaining memory efficient and to improve stack debugging. [1/3] dynamically allocate irq stacks This conserves memory while using IRQ stacks.

Re: [2/6] add config option to vmalloc stacks (was: Re: [-mm patch] i386: enable 4k stacks by default)

2007-04-30 Thread Bill Irwin
On Mon, Apr 30, 2007 at 08:15:11PM +0100, Christoph Hellwig wrote: > So if you want to invest some time into getting this into mergeable > shape I'd suggest you redo the patch series in the following way: > patch 1: dynamic allocated irq stacks > patch 2: make irqstacks unconditional, but allow

Re: [5/6] dynamically allocate IRQ stacks (was: Re: [-mm patch] i386: enable 4k stacks by default)

2007-04-30 Thread Bill Irwin
On Mon, 30 Apr 2007, William Lee Irwin III wrote: >> -static char softirq_stack[NR_CPUS * THREAD_SIZE] >> -__attribute__((__aligned__(THREAD_SIZE))); >> +static DEFINE_PER_CPU(char *, softirq_stack); >> +static DEFINE_PER_CPU(char *, hardirq_stack); >> >> -static char

Re: [4/6] go BUG on vmallocspace in __pa() (was: Re: [-mm patch] i386: enable 4k stacks by default)

2007-04-30 Thread Bill Irwin
On Mon, Apr 30, 2007 at 10:38:19AM -0700, William Lee Irwin III wrote: Here's what I did for i386 for someone concerned about blowing the stack. On Mon, Apr 30, 2007 at 10:45:10AM -0700, William Lee Irwin III wrote: >>> Add checks to __pa() so it goes BUG() on vmallocspace addresses.

Re: [2/6] add config option to vmalloc stacks (was: Re: [-mm patch] i386: enable 4k stacks by default)

2007-04-30 Thread Bill Irwin
On Mon, Apr 30, 2007 at 08:15:11PM +0100, Christoph Hellwig wrote: > So if you want to invest some time into getting this into mergeable > shape I'd suggest you redo the patch series in the following way: > patch 1: dynamic allocated irq stacks > patch 2: make irqstacks unconditional, but allow

Re: [2/6] add config option to vmalloc stacks (was: Re: [-mm patch] i386: enable 4k stacks by default)

2007-04-30 Thread Bill Irwin
On Mon, Apr 30, 2007 at 08:15:11PM +0100, Christoph Hellwig wrote: So if you want to invest some time into getting this into mergeable shape I'd suggest you redo the patch series in the following way: patch 1: dynamic allocated irq stacks patch 2: make irqstacks unconditional, but allow

Re: [4/6] go BUG on vmallocspace in __pa() (was: Re: [-mm patch] i386: enable 4k stacks by default)

2007-04-30 Thread Bill Irwin
On Mon, Apr 30, 2007 at 10:38:19AM -0700, William Lee Irwin III wrote: Here's what I did for i386 for someone concerned about blowing the stack. On Mon, Apr 30, 2007 at 10:45:10AM -0700, William Lee Irwin III wrote: Add checks to __pa() so it goes BUG() on vmallocspace addresses. On Mon, 30

Re: [5/6] dynamically allocate IRQ stacks (was: Re: [-mm patch] i386: enable 4k stacks by default)

2007-04-30 Thread Bill Irwin
On Mon, 30 Apr 2007, William Lee Irwin III wrote: -static char softirq_stack[NR_CPUS * THREAD_SIZE] -__attribute__((__aligned__(THREAD_SIZE))); +static DEFINE_PER_CPU(char *, softirq_stack); +static DEFINE_PER_CPU(char *, hardirq_stack); -static char hardirq_stack[NR_CPUS *

Re: [2/6] add config option to vmalloc stacks (was: Re: [-mm patch] i386: enable 4k stacks by default)

2007-04-30 Thread Bill Irwin
On Mon, Apr 30, 2007 at 08:15:11PM +0100, Christoph Hellwig wrote: So if you want to invest some time into getting this into mergeable shape I'd suggest you redo the patch series in the following way: patch 1: dynamic allocated irq stacks patch 2: make irqstacks unconditional, but allow

[0/3] i386 stack handling updates

2007-04-30 Thread Bill Irwin
This is a series of patches to improve i386 stack handling in several manners. The net result is to improve system stability via IRQ stacks while remaining memory efficient and to improve stack debugging. [1/3] dynamically allocate irq stacks This conserves memory while using IRQ stacks.

[1/3] dynamically allocate IRQ stacks

2007-04-30 Thread Bill Irwin
Dynamically allocate IRQ stacks in order to conserve memory when using IRQ stacks. cpu_possible_map is not now initialized in such a manner as to provide a meaningful indication of how many CPU's might be in the system, and features to appear in the sequel also require indirection, so they

[2/3] unconditional i386 IRQ stacks

2007-04-30 Thread Bill Irwin
IRQ stacks are a valuable stability feature. This patch makes them unconditional, as there is no circumstance under which they do not improve stability and they have no meaningful performance impact. Signed-off-by: William Irwin [EMAIL PROTECTED] Index: stack-paranoia/include/asm-i386/irq.h

[3/3] use vmalloc() to arrange guard pages for stacks

2007-04-30 Thread Bill Irwin
This patch introduces CONFIG_DEBUG_STACK, which vmalloc()'s task and IRQ stacks in order to establish guard pages. In such a manner any stack overflow that references pages immediately adjacent to the stack is immediately trapped with a fault, which precludes silent memory corruption or

Re: 2.6.22 -mm merge plans

2007-04-30 Thread Bill Irwin
On Mon, Apr 30, 2007 at 04:20:07PM -0700, Andrew Morton wrote: proper-prototype-for-hugetlb_get_unmapped_area.patch ... convert-hugetlbfs-to-use-vm_ops-fault.patch ... get_unmapped_area-handles-map_fixed-in-hugetlbfs.patch ... get_unmapped_area-doesnt-need-hugetlbfs-hacks-anymore.patch ...

Re: [patch 4/6] mm: merge populate and nopage into fault (fixes nonlinear)

2007-03-19 Thread Bill Irwin
On Sun, Mar 18, 2007 at 03:50:10AM +0100, Nick Piggin wrote: > Yes, that should be the case. So would this mean that nonlinear protections > don't work on regular files? I guess that's OK if Oracle and UML both use > tmpfs/shm? Sometimes ramfs is also used in the Oracle case. I presume that's

Re: [patch 4/6] mm: merge populate and nopage into fault (fixes nonlinear)

2007-03-19 Thread Bill Irwin
On Sun, Mar 18, 2007 at 03:50:10AM +0100, Nick Piggin wrote: Yes, that should be the case. So would this mean that nonlinear protections don't work on regular files? I guess that's OK if Oracle and UML both use tmpfs/shm? Sometimes ramfs is also used in the Oracle case. I presume that's even

Re: Linux 2.6.20.3

2007-03-13 Thread Bill Irwin
On Tue, Mar 13, 2007 at 05:29:17PM -0700, Nish Aravamudan wrote: > Ok, truly bizarre, I found that I was not running stock 2.6.20.3, but > had your small hugetlb patch on top. > So I went back and patched 2.6.20.1 with your patch, rebooted, got a > soft lockup. Went back to stock 2.6.20.1 and did

Re: Linux 2.6.20.3

2007-03-13 Thread Bill Irwin
On Tue, Mar 13, 2007 at 05:29:17PM -0700, Nish Aravamudan wrote: Ok, truly bizarre, I found that I was not running stock 2.6.20.3, but had your small hugetlb patch on top. So I went back and patched 2.6.20.1 with your patch, rebooted, got a soft lockup. Went back to stock 2.6.20.1 and did not.

Re: [PATCH] Fix get_unmapped_area and fsync for hugetlb shm segments

2007-03-07 Thread Bill Irwin
On Wed, Mar 07, 2007 at 05:26:48PM -0600, Adam Litke wrote: > :) Enter my remove-is_file_hugepages() patches (which I posted a few > weeks ago). I'll rework them and repost soon. That should help to make > all of this cleaner. Those were great. I've wanted something like them for a long, long

Re: [PATCH] Fix get_unmapped_area and fsync for hugetlb shm segments

2007-03-07 Thread Bill Irwin
Bill Irwin <[EMAIL PROTECTED]> writes: >> A comment to prepare others for the impending doubletake might be nice. >> Or maybe just open-coding the equality check for _file_operations >> in is_file_shm_hugepages() if others find it as jarring as I. Please >> extend my

Re: Wanted: simple, safe x86 stack overflow detection

2007-03-07 Thread Bill Irwin
On Tue, 2007-03-06 at 22:44 -0800, Bill Irwin wrote: >> What do you see as the obstacle to eliminating nested IRQ's? On Wed, Mar 07, 2007 at 04:34:52AM -0800, Arjan van de Ven wrote: > political will, or maybe just the lack of convincing people so far Political issues are significa

Re: [patch 4/6] mm: merge populate and nopage into fault (fixes nonlinear)

2007-03-07 Thread Bill Irwin
On Wed, Mar 07, 2007 at 11:47:42AM +0100, Peter Zijlstra wrote: >> Well, now they don't, but it could be done or even exploited as a DoS. On Wed, Mar 07, 2007 at 12:00:36PM +0100, Nick Piggin wrote: > But so could nonlinear page reclaim. I think we need to restrict nonlinear > mappings to root if

Re: [patch 4/6] mm: merge populate and nopage into fault (fixes nonlinear)

2007-03-07 Thread Bill Irwin
On Wed, 7 Mar 2007 01:29:03 -0800 Bill Irwin <[EMAIL PROTECTED]> wrote: >> Guess what major real-life application not only uses nonlinear daily >> but would even be very happy to see it extended with non-vma-creating >> protections and more? On Wed, Mar 07, 2007 at 01:39:4

Re: [patch 4/6] mm: merge populate and nopage into fault (fixes nonlinear)

2007-03-07 Thread Bill Irwin
On Wed, Mar 07, 2007 at 10:22:52AM +0100, Ingo Molnar wrote: >>> ok. What do you think about the sys_remap_file_pages_prot() thing that >>> Paolo has done in a nicely split up form - does that complicate things >>> in any fundamental way? That is what is useful to

Re: [patch 4/6] mm: merge populate and nopage into fault (fixes nonlinear)

2007-03-07 Thread Bill Irwin
On Wed, Mar 07, 2007 at 10:28:21AM +0100, Nick Piggin wrote: > Depending on whether anyone wants it, and what features they want, we > could emulate the old syscall, and make a new restricted one which is > much less intrusive. > For example, if we can operate only on MAP_ANONYMOUS memory and

Re: [patch 4/6] mm: merge populate and nopage into fault (fixes nonlinear)

2007-03-07 Thread Bill Irwin
* Nick Piggin <[EMAIL PROTECTED]> wrote: >> After these patches, I don't think there is too much burden. The main >> thing left really is just the objrmap stuff, but that is just handled >> with a minimal 'dumb' algorithm that doesn't cost much. On Wed, Mar 07, 2007 at 10:22:52AM +0100, Ingo

Re: [patch 4/6] mm: merge populate and nopage into fault (fixes nonlinear)

2007-03-07 Thread Bill Irwin
On Wed, 7 Mar 2007 09:27:55 +0100 Ingo Molnar <[EMAIL PROTECTED]> wrote: >> btw., if we decide that nonlinear isnt worth the continuing maintainance >> pain, we could internally implement/emulate sys_remap_file_pages() via a >> call to mremap() and essentially deprecate it, without breaking the

Re: [patch 4/6] mm: merge populate and nopage into fault (fixes nonlinear)

2007-03-07 Thread Bill Irwin
On Wed, 7 Mar 2007 09:27:55 +0100 Ingo Molnar [EMAIL PROTECTED] wrote: btw., if we decide that nonlinear isnt worth the continuing maintainance pain, we could internally implement/emulate sys_remap_file_pages() via a call to mremap() and essentially deprecate it, without breaking the ABI -

Re: [patch 4/6] mm: merge populate and nopage into fault (fixes nonlinear)

2007-03-07 Thread Bill Irwin
* Nick Piggin [EMAIL PROTECTED] wrote: After these patches, I don't think there is too much burden. The main thing left really is just the objrmap stuff, but that is just handled with a minimal 'dumb' algorithm that doesn't cost much. On Wed, Mar 07, 2007 at 10:22:52AM +0100, Ingo Molnar

Re: [patch 4/6] mm: merge populate and nopage into fault (fixes nonlinear)

2007-03-07 Thread Bill Irwin
On Wed, Mar 07, 2007 at 10:28:21AM +0100, Nick Piggin wrote: Depending on whether anyone wants it, and what features they want, we could emulate the old syscall, and make a new restricted one which is much less intrusive. For example, if we can operate only on MAP_ANONYMOUS memory and specify

Re: [patch 4/6] mm: merge populate and nopage into fault (fixes nonlinear)

2007-03-07 Thread Bill Irwin
On Wed, Mar 07, 2007 at 10:22:52AM +0100, Ingo Molnar wrote: ok. What do you think about the sys_remap_file_pages_prot() thing that Paolo has done in a nicely split up form - does that complicate things in any fundamental way? That is what is useful to UML. * Bill Irwin [EMAIL PROTECTED

Re: [patch 4/6] mm: merge populate and nopage into fault (fixes nonlinear)

2007-03-07 Thread Bill Irwin
On Wed, 7 Mar 2007 01:29:03 -0800 Bill Irwin [EMAIL PROTECTED] wrote: Guess what major real-life application not only uses nonlinear daily but would even be very happy to see it extended with non-vma-creating protections and more? On Wed, Mar 07, 2007 at 01:39:42AM -0800, Andrew Morton wrote

Re: [patch 4/6] mm: merge populate and nopage into fault (fixes nonlinear)

2007-03-07 Thread Bill Irwin
On Wed, Mar 07, 2007 at 11:47:42AM +0100, Peter Zijlstra wrote: Well, now they don't, but it could be done or even exploited as a DoS. On Wed, Mar 07, 2007 at 12:00:36PM +0100, Nick Piggin wrote: But so could nonlinear page reclaim. I think we need to restrict nonlinear mappings to root if

Re: Wanted: simple, safe x86 stack overflow detection

2007-03-07 Thread Bill Irwin
On Tue, 2007-03-06 at 22:44 -0800, Bill Irwin wrote: What do you see as the obstacle to eliminating nested IRQ's? On Wed, Mar 07, 2007 at 04:34:52AM -0800, Arjan van de Ven wrote: political will, or maybe just the lack of convincing people so far Political issues are significantly more

Re: [PATCH] Fix get_unmapped_area and fsync for hugetlb shm segments

2007-03-07 Thread Bill Irwin
Bill Irwin [EMAIL PROTECTED] writes: A comment to prepare others for the impending doubletake might be nice. Or maybe just open-coding the equality check for hugetlbfs_file_operations in is_file_shm_hugepages() if others find it as jarring as I. Please extend my ack to any follow-up fiddling

Re: [PATCH] Fix get_unmapped_area and fsync for hugetlb shm segments

2007-03-07 Thread Bill Irwin
On Wed, Mar 07, 2007 at 05:26:48PM -0600, Adam Litke wrote: :) Enter my remove-is_file_hugepages() patches (which I posted a few weeks ago). I'll rework them and repost soon. That should help to make all of this cleaner. Those were great. I've wanted something like them for a long, long

Re: [patch 4/6] mm: merge populate and nopage into fault (fixes nonlinear)

2007-03-06 Thread Bill Irwin
On Tue, Mar 06, 2007 at 10:51:01PM -0800, Andrew Morton wrote: > Does anybody really pass a NULL `type' arg into filemap_nopage()? The major vs. minor fault accounting patch that introduced the argument didn't make non-NULL type arguments a requirement. It's essentially an optional second return

Re: Wanted: simple, safe x86 stack overflow detection

2007-03-06 Thread Bill Irwin
At some point in the past, I wrote: >> I'm certainly in favor of the move; IRQ stacks could be made >> rather deep and cheaply at that. I may get around to writing it this >> week if no one else does it first. On Tue, Mar 06, 2007 at 08:28:35PM -0800, Arjan van de Ven wrote: > the irq stacks

Re: Wanted: simple, safe x86 stack overflow detection

2007-03-06 Thread Bill Irwin
On Tue, 6 Mar 2007, Chuck Ebbert wrote: >> In the 4k/4k stack i386 kernel, is there any fundamental reason it >> can't be 4k/8k? We seem to be mostly hitting problems in overflowing >> the IRQ stack... I think. Overhead would only be 4k per CPU for that. On Tue, Mar 06, 2007 at 07:43:41PM +,

Re: kernel BUG at arch/x86_64/mm/../../i386/mm/hugetlbpage.c:140!

2007-03-06 Thread Bill Irwin
On Saturday 03 March 2007, Bill Irwin wrote: >> If you have a known-working kernel version, git-bisect might help you >> track down where it was introduced. Given the messages prior to the >> hugetlbpage.c BUG_ON I'd say that this is something else besides the >> spec

  1   2   >