Re: svn commit: r216016 - head/sys/sparc64/include
m...@freebsd.org wrote: On Tue, Dec 7, 2010 at 5:41 AM, Marius Strobl mar...@alchemy.franken.de wrote: On Mon, Dec 06, 2010 at 02:30:01PM -0800, m...@freebsd.org wrote: On Mon, Dec 6, 2010 at 2:07 PM, Marius Strobl mar...@alchemy.franken.de wrote: [lots of snip] With that one the kernel now survies memguard_init() but then panics right afterwards when kmeminit() calls kmem_suballoc(): KDB: debugger backends: ddb KDB: current backend: ddb Copyright (c) 1992-2010 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 ? ? ? ?The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 9.0-CURRENT #18 r215249:216120M: Mon Dec ?6 13:27:57 CET 2010 ? ?mar...@v20z.zeist.de:/home/marius/co/build/head2/sparc64.sparc64/usr/home/m4 WARNING: WITNESS option enabled, expect reduced performance. panic: kmem_suballoc: bad status return of 3 [more snip] Shooting in the dark a little... The bad status of 3 is presumably KERN_NO_SPACE because we attempted to allocate too much space from the kernel_map. What are the actual values of vm_kmem_size, kernel_map-min_offset, kernel_map-max_offset at panic time? vm_kmem_size=5610405888 min_offset=3221225472 max_offset=13958643712 This is on a US3i machine with 16GB RAM. So kernel_map is from 0xC000_ to 0x3_4000_, or 0x2_8000_ bytes large. Double the vm_kmem_size is 0x2_9CD0_, which is why this is failing. This to me says that, for the moment, you need the VM_MAX_KERNEL_ADDRESS define so that memguard does not take up too much virtual space; at the moment this hardware is somewhat constrained on virtual space. Yes, I had thought that sparc64 had a much larger kernel address space. Marius, I would suggest that you revert back to Max's original commit that caps the kmem_map at 3/5 of the kernel address space. Right now, you are allowing the kmem_map to consume the entire kernel address space, and given the relatively modest size of the kernel address space, a machine with more physical memory may try to create a kmem_map that leaves no space for the buffer cache, pipes, thread stacks, etc. Alan ___ svn-src-all@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/svn-src-all To unsubscribe, send any mail to svn-src-all-unsubscr...@freebsd.org
Re: svn commit: r216016 - head/sys/sparc64/include
On Mon, Dec 06, 2010 at 02:30:01PM -0800, m...@freebsd.org wrote: On Mon, Dec 6, 2010 at 2:07 PM, Marius Strobl mar...@alchemy.franken.de wrote: [lots of snip] With that one the kernel now survies memguard_init() but then panics right afterwards when kmeminit() calls kmem_suballoc(): KDB: debugger backends: ddb KDB: current backend: ddb Copyright (c) 1992-2010 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 ? ? ? ?The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 9.0-CURRENT #18 r215249:216120M: Mon Dec ?6 13:27:57 CET 2010 ? ?mar...@v20z.zeist.de:/home/marius/co/build/head2/sparc64.sparc64/usr/home/m4 WARNING: WITNESS option enabled, expect reduced performance. panic: kmem_suballoc: bad status return of 3 [more snip] Shooting in the dark a little... The bad status of 3 is presumably KERN_NO_SPACE because we attempted to allocate too much space from the kernel_map. What are the actual values of vm_kmem_size, kernel_map-min_offset, kernel_map-max_offset at panic time? vm_kmem_size=5610405888 min_offset=3221225472 max_offset=13958643712 This is on a US3i machine with 16GB RAM. How much virtual space does sparc64 support (since earlier it was reported it's computed based on hardware capability, for this specific machine?) Currently, the limiting factor is that the kernel TSB is addressed virtually, which means that the TTEs for kernel TSB itself have to be put into locked dTLB slots taking up 1 4MB dTLB slot per 1GB. US3 family CPUs have 16 lockable 4MB dTLB and iTLB slots, which need to be shared between the TSB and the kernel itself though, i.e. a 9MB kernel also takes up 3 slots (US1/2 machines have 64 dTLB and iTLB slots but there these are also used for unlocked TTEs so we don't use more than 32 for kernel plus TSB on these). VM_MAX_KERNEL_ADDRESS is limited according to how many slots are available for the TSB, that's what I was referring to previously. Actually, US3 and later as well as SPARC64 V and later CPUs have a feature allowing the TSB to be addressed physically, circumventing the need to lock the TSB into the TLB, thus allowing the full 64-bit virtual address space to be used. Implementing that is on my TODO list but unfortunately that's not exactly straight forward and also requires some instructions to be patched at runtime so the kernel still works on US1/2 machines. Marius ___ svn-src-all@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/svn-src-all To unsubscribe, send any mail to svn-src-all-unsubscr...@freebsd.org
Re: svn commit: r216016 - head/sys/sparc64/include
On Tue, Dec 7, 2010 at 5:41 AM, Marius Strobl mar...@alchemy.franken.de wrote: On Mon, Dec 06, 2010 at 02:30:01PM -0800, m...@freebsd.org wrote: On Mon, Dec 6, 2010 at 2:07 PM, Marius Strobl mar...@alchemy.franken.de wrote: [lots of snip] With that one the kernel now survies memguard_init() but then panics right afterwards when kmeminit() calls kmem_suballoc(): KDB: debugger backends: ddb KDB: current backend: ddb Copyright (c) 1992-2010 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 ? ? ? ?The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 9.0-CURRENT #18 r215249:216120M: Mon Dec ?6 13:27:57 CET 2010 ? ?mar...@v20z.zeist.de:/home/marius/co/build/head2/sparc64.sparc64/usr/home/m4 WARNING: WITNESS option enabled, expect reduced performance. panic: kmem_suballoc: bad status return of 3 [more snip] Shooting in the dark a little... The bad status of 3 is presumably KERN_NO_SPACE because we attempted to allocate too much space from the kernel_map. What are the actual values of vm_kmem_size, kernel_map-min_offset, kernel_map-max_offset at panic time? vm_kmem_size=5610405888 min_offset=3221225472 max_offset=13958643712 This is on a US3i machine with 16GB RAM. So kernel_map is from 0xC000_ to 0x3_4000_, or 0x2_8000_ bytes large. Double the vm_kmem_size is 0x2_9CD0_, which is why this is failing. This to me says that, for the moment, you need the VM_MAX_KERNEL_ADDRESS define so that memguard does not take up too much virtual space; at the moment this hardware is somewhat constrained on virtual space. Thanks, matthew How much virtual space does sparc64 support (since earlier it was reported it's computed based on hardware capability, for this specific machine?) Currently, the limiting factor is that the kernel TSB is addressed virtually, which means that the TTEs for kernel TSB itself have to be put into locked dTLB slots taking up 1 4MB dTLB slot per 1GB. US3 family CPUs have 16 lockable 4MB dTLB and iTLB slots, which need to be shared between the TSB and the kernel itself though, i.e. a 9MB kernel also takes up 3 slots (US1/2 machines have 64 dTLB and iTLB slots but there these are also used for unlocked TTEs so we don't use more than 32 for kernel plus TSB on these). VM_MAX_KERNEL_ADDRESS is limited according to how many slots are available for the TSB, that's what I was referring to previously. Actually, US3 and later as well as SPARC64 V and later CPUs have a feature allowing the TSB to be addressed physically, circumventing the need to lock the TSB into the TLB, thus allowing the full 64-bit virtual address space to be used. Implementing that is on my TODO list but unfortunately that's not exactly straight forward and also requires some instructions to be patched at runtime so the kernel still works on US1/2 machines. Marius ___ svn-src-all@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/svn-src-all To unsubscribe, send any mail to svn-src-all-unsubscr...@freebsd.org
Re: svn commit: r216016 - head/sys/sparc64/include
On Thu, Dec 02, 2010 at 11:27:45AM -0600, Alan Cox wrote: Marius Strobl wrote: On Wed, Dec 01, 2010 at 12:19:17PM -0600, Alan Cox wrote: Marius Strobl wrote: On Mon, Nov 29, 2010 at 08:23:08PM +0100, Marius Strobl wrote: On Tue, Nov 30, 2010 at 12:31:31AM +0600, Max Khon wrote: Marius, On Mon, Nov 29, 2010 at 1:45 AM, Marius Strobl mar...@alchemy.franken.dewrote: On Sun, Nov 28, 2010 at 07:26:20PM +, Max Khon wrote: Author: fjoe Date: Sun Nov 28 19:26:20 2010 New Revision: 216016 URL: http://svn.freebsd.org/changeset/base/216016 Log: Define VM_KMEM_SIZE_MAX on sparc64. Otherwise kernel built with DEBUG_MEMGUARD panics early in kmeminit() with the message kmem_suballoc: bad status return of 1 because of zero size argument passed to kmem_suballoc() due to vm_kmem_size_max being zero. The problem also exists on ia64. Modified: head/sys/sparc64/include/vmparam.h Modified: head/sys/sparc64/include/vmparam.h == --- head/sys/sparc64/include/vmparam.hSun Nov 28 18:59:52 2010 (r216015) +++ head/sys/sparc64/include/vmparam.hSun Nov 28 19:26:20 2010 (r216016) @@ -237,6 +237,14 @@ #endif /* + * Ceiling on amount of kmem_map kva space. + */ +#ifndef VM_KMEM_SIZE_MAX +#define VM_KMEM_SIZE_MAX((VM_MAX_KERNEL_ADDRESS - \ +VM_MIN_KERNEL_ADDRESS + 1) * 3 / 5) +#endif + +/* * Initial pagein size of beginning of executable file. */ #ifndef VM_INITIAL_PAGEIN How was that value determined? I've just copied it from amd64 to be non-zero for now. Do you have a better idea of what it should look like? Well, on sparc64 VM_MAX_KERNEL_ADDRESS already is dynamically adjusted to the maximum appropriate for the specific CPU during the early cycles of the kernel so I'd think one could just use VM_MAX_KERNEL_ADDRESS - VM_MIN_KERNEL_ADDRESS for VM_KMEM_SIZE_MAX there, I'm not sure what the intention of the ceiling provided by that macro actually is though In any case, the commit message of r180210 which changed the amd64 version to the current one talks about limiting the kmem map to 3.6GB and while it also fails to explain where that value comes from it looks rather amd64 specific and the formula used by the macro will result in a different ceiling on sparc64 and thus inappropriate. I've CC'ed alc@ who hopefully can shed some light on this. Apart from this the actual bug here seems to be that memguard_fudge() can't deal with a km_max being zero or that zero is passed to it as kmeminit() allows for VM_KMEM_SIZE_MAX not being defined. Oops, forgot to actually CC a...@. There's nothing particularly amd64-specific about the definition. In general, if you allow the kmem_map, which is basically the kernel's heap, to consume the entire kernel address space as you propose, then you're leaving no room for the buffer cache, thread stacks, pipes, and a few other things. Since the cap on the kmem_map size as defined by r180210 is a fraction of the overall kernel address space size, it scales automatically with the kernel address space size and should be a reasonable cap definition for most architectures. In the specific case of sparc64, I think it's fair to say that the kernel virtual address is sufficiently large and the amount of physical memory in any of the supported machines is small enough in comparison that it hasn't mattered that a kmem_map cap doesn't exist, because most of the aforementioned structures are scaled based on the amount of physical memory. In fact, it probably won't matter any time soon. All of that said, I would suggest fixing memguard_fudge() and reverting r216016 and the follow up change. All I think that is required to fix memguard_fudge() is Index: vm/memguard.c === --- vm/memguard.c (revision 216070) +++ vm/memguard.c (working copy) @@ -186,7 +186,7 @@ memguard_fudge(unsigned long km_size, unsigned lon memguard_mapsize = round_page(memguard_mapsize); if (memguard_mapsize / (2 * PAGE_SIZE) mem_pgs) memguard_mapsize = mem_pgs * 2 * PAGE_SIZE; - if (km_size + memguard_mapsize km_max) + if (km_max 0 km_size + memguard_mapsize km_max) return (km_max); return (km_size + memguard_mapsize); } Thanks but unfortunately this variant then still panics in kmem_suballoc() when called by memguard_init(): KDB: debugger backends: ddb KDB: current backend: ddb Copyright (c) 1992-2010 The FreeBSD Project. Copyright
Re: svn commit: r216016 - head/sys/sparc64/include
On Mon, Dec 6, 2010 at 2:07 PM, Marius Strobl mar...@alchemy.franken.de wrote: [lots of snip] With that one the kernel now survies memguard_init() but then panics right afterwards when kmeminit() calls kmem_suballoc(): KDB: debugger backends: ddb KDB: current backend: ddb Copyright (c) 1992-2010 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 9.0-CURRENT #18 r215249:216120M: Mon Dec 6 13:27:57 CET 2010 mar...@v20z.zeist.de:/home/marius/co/build/head2/sparc64.sparc64/usr/home/m4 WARNING: WITNESS option enabled, expect reduced performance. panic: kmem_suballoc: bad status return of 3 [more snip] Shooting in the dark a little... The bad status of 3 is presumably KERN_NO_SPACE because we attempted to allocate too much space from the kernel_map. What are the actual values of vm_kmem_size, kernel_map-min_offset, kernel_map-max_offset at panic time? How much virtual space does sparc64 support (since earlier it was reported it's computed based on hardware capability, for this specific machine?) Thanks, matthew ___ svn-src-all@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/svn-src-all To unsubscribe, send any mail to svn-src-all-unsubscr...@freebsd.org
Re: svn commit: r216016 - head/sys/sparc64/include
On Wed, Dec 01, 2010 at 12:19:17PM -0600, Alan Cox wrote: Marius Strobl wrote: On Mon, Nov 29, 2010 at 08:23:08PM +0100, Marius Strobl wrote: On Tue, Nov 30, 2010 at 12:31:31AM +0600, Max Khon wrote: Marius, On Mon, Nov 29, 2010 at 1:45 AM, Marius Strobl mar...@alchemy.franken.dewrote: On Sun, Nov 28, 2010 at 07:26:20PM +, Max Khon wrote: Author: fjoe Date: Sun Nov 28 19:26:20 2010 New Revision: 216016 URL: http://svn.freebsd.org/changeset/base/216016 Log: Define VM_KMEM_SIZE_MAX on sparc64. Otherwise kernel built with DEBUG_MEMGUARD panics early in kmeminit() with the message kmem_suballoc: bad status return of 1 because of zero size argument passed to kmem_suballoc() due to vm_kmem_size_max being zero. The problem also exists on ia64. Modified: head/sys/sparc64/include/vmparam.h Modified: head/sys/sparc64/include/vmparam.h == --- head/sys/sparc64/include/vmparam.hSun Nov 28 18:59:52 2010 (r216015) +++ head/sys/sparc64/include/vmparam.hSun Nov 28 19:26:20 2010 (r216016) @@ -237,6 +237,14 @@ #endif /* + * Ceiling on amount of kmem_map kva space. + */ +#ifndef VM_KMEM_SIZE_MAX +#define VM_KMEM_SIZE_MAX((VM_MAX_KERNEL_ADDRESS - \ +VM_MIN_KERNEL_ADDRESS + 1) * 3 / 5) +#endif + +/* * Initial pagein size of beginning of executable file. */ #ifndef VM_INITIAL_PAGEIN How was that value determined? I've just copied it from amd64 to be non-zero for now. Do you have a better idea of what it should look like? Well, on sparc64 VM_MAX_KERNEL_ADDRESS already is dynamically adjusted to the maximum appropriate for the specific CPU during the early cycles of the kernel so I'd think one could just use VM_MAX_KERNEL_ADDRESS - VM_MIN_KERNEL_ADDRESS for VM_KMEM_SIZE_MAX there, I'm not sure what the intention of the ceiling provided by that macro actually is though In any case, the commit message of r180210 which changed the amd64 version to the current one talks about limiting the kmem map to 3.6GB and while it also fails to explain where that value comes from it looks rather amd64 specific and the formula used by the macro will result in a different ceiling on sparc64 and thus inappropriate. I've CC'ed alc@ who hopefully can shed some light on this. Apart from this the actual bug here seems to be that memguard_fudge() can't deal with a km_max being zero or that zero is passed to it as kmeminit() allows for VM_KMEM_SIZE_MAX not being defined. Oops, forgot to actually CC a...@. There's nothing particularly amd64-specific about the definition. In general, if you allow the kmem_map, which is basically the kernel's heap, to consume the entire kernel address space as you propose, then you're leaving no room for the buffer cache, thread stacks, pipes, and a few other things. Since the cap on the kmem_map size as defined by r180210 is a fraction of the overall kernel address space size, it scales automatically with the kernel address space size and should be a reasonable cap definition for most architectures. In the specific case of sparc64, I think it's fair to say that the kernel virtual address is sufficiently large and the amount of physical memory in any of the supported machines is small enough in comparison that it hasn't mattered that a kmem_map cap doesn't exist, because most of the aforementioned structures are scaled based on the amount of physical memory. In fact, it probably won't matter any time soon. All of that said, I would suggest fixing memguard_fudge() and reverting r216016 and the follow up change. All I think that is required to fix memguard_fudge() is Index: vm/memguard.c === --- vm/memguard.c (revision 216070) +++ vm/memguard.c (working copy) @@ -186,7 +186,7 @@ memguard_fudge(unsigned long km_size, unsigned lon memguard_mapsize = round_page(memguard_mapsize); if (memguard_mapsize / (2 * PAGE_SIZE) mem_pgs) memguard_mapsize = mem_pgs * 2 * PAGE_SIZE; - if (km_size + memguard_mapsize km_max) + if (km_max 0 km_size + memguard_mapsize km_max) return (km_max); return (km_size + memguard_mapsize); } Thanks but unfortunately this variant then still panics in kmem_suballoc() when called by memguard_init(): KDB: debugger backends: ddb KDB: current backend: ddb Copyright (c) 1992-2010 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 9.0-CURRENT #17 r215249:216120M: Thu Dec 2
Re: svn commit: r216016 - head/sys/sparc64/include
Marius Strobl wrote: On Wed, Dec 01, 2010 at 12:19:17PM -0600, Alan Cox wrote: Marius Strobl wrote: On Mon, Nov 29, 2010 at 08:23:08PM +0100, Marius Strobl wrote: On Tue, Nov 30, 2010 at 12:31:31AM +0600, Max Khon wrote: Marius, On Mon, Nov 29, 2010 at 1:45 AM, Marius Strobl mar...@alchemy.franken.dewrote: On Sun, Nov 28, 2010 at 07:26:20PM +, Max Khon wrote: Author: fjoe Date: Sun Nov 28 19:26:20 2010 New Revision: 216016 URL: http://svn.freebsd.org/changeset/base/216016 Log: Define VM_KMEM_SIZE_MAX on sparc64. Otherwise kernel built with DEBUG_MEMGUARD panics early in kmeminit() with the message kmem_suballoc: bad status return of 1 because of zero size argument passed to kmem_suballoc() due to vm_kmem_size_max being zero. The problem also exists on ia64. Modified: head/sys/sparc64/include/vmparam.h Modified: head/sys/sparc64/include/vmparam.h == --- head/sys/sparc64/include/vmparam.hSun Nov 28 18:59:52 2010 (r216015) +++ head/sys/sparc64/include/vmparam.hSun Nov 28 19:26:20 2010 (r216016) @@ -237,6 +237,14 @@ #endif /* + * Ceiling on amount of kmem_map kva space. + */ +#ifndef VM_KMEM_SIZE_MAX +#define VM_KMEM_SIZE_MAX((VM_MAX_KERNEL_ADDRESS - \ +VM_MIN_KERNEL_ADDRESS + 1) * 3 / 5) +#endif + +/* * Initial pagein size of beginning of executable file. */ #ifndef VM_INITIAL_PAGEIN How was that value determined? I've just copied it from amd64 to be non-zero for now. Do you have a better idea of what it should look like? Well, on sparc64 VM_MAX_KERNEL_ADDRESS already is dynamically adjusted to the maximum appropriate for the specific CPU during the early cycles of the kernel so I'd think one could just use VM_MAX_KERNEL_ADDRESS - VM_MIN_KERNEL_ADDRESS for VM_KMEM_SIZE_MAX there, I'm not sure what the intention of the ceiling provided by that macro actually is though In any case, the commit message of r180210 which changed the amd64 version to the current one talks about limiting the kmem map to 3.6GB and while it also fails to explain where that value comes from it looks rather amd64 specific and the formula used by the macro will result in a different ceiling on sparc64 and thus inappropriate. I've CC'ed alc@ who hopefully can shed some light on this. Apart from this the actual bug here seems to be that memguard_fudge() can't deal with a km_max being zero or that zero is passed to it as kmeminit() allows for VM_KMEM_SIZE_MAX not being defined. Oops, forgot to actually CC a...@. There's nothing particularly amd64-specific about the definition. In general, if you allow the kmem_map, which is basically the kernel's heap, to consume the entire kernel address space as you propose, then you're leaving no room for the buffer cache, thread stacks, pipes, and a few other things. Since the cap on the kmem_map size as defined by r180210 is a fraction of the overall kernel address space size, it scales automatically with the kernel address space size and should be a reasonable cap definition for most architectures. In the specific case of sparc64, I think it's fair to say that the kernel virtual address is sufficiently large and the amount of physical memory in any of the supported machines is small enough in comparison that it hasn't mattered that a kmem_map cap doesn't exist, because most of the aforementioned structures are scaled based on the amount of physical memory. In fact, it probably won't matter any time soon. All of that said, I would suggest fixing memguard_fudge() and reverting r216016 and the follow up change. All I think that is required to fix memguard_fudge() is Index: vm/memguard.c === --- vm/memguard.c (revision 216070) +++ vm/memguard.c (working copy) @@ -186,7 +186,7 @@ memguard_fudge(unsigned long km_size, unsigned lon memguard_mapsize = round_page(memguard_mapsize); if (memguard_mapsize / (2 * PAGE_SIZE) mem_pgs) memguard_mapsize = mem_pgs * 2 * PAGE_SIZE; - if (km_size + memguard_mapsize km_max) + if (km_max 0 km_size + memguard_mapsize km_max) return (km_max); return (km_size + memguard_mapsize); } Thanks but unfortunately this variant then still panics in kmem_suballoc() when called by memguard_init(): KDB: debugger backends: ddb KDB: current backend: ddb Copyright (c) 1992-2010 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The
Re: svn commit: r216016 - head/sys/sparc64/include
Marius Strobl wrote: On Mon, Nov 29, 2010 at 08:23:08PM +0100, Marius Strobl wrote: On Tue, Nov 30, 2010 at 12:31:31AM +0600, Max Khon wrote: Marius, On Mon, Nov 29, 2010 at 1:45 AM, Marius Strobl mar...@alchemy.franken.dewrote: On Sun, Nov 28, 2010 at 07:26:20PM +, Max Khon wrote: Author: fjoe Date: Sun Nov 28 19:26:20 2010 New Revision: 216016 URL: http://svn.freebsd.org/changeset/base/216016 Log: Define VM_KMEM_SIZE_MAX on sparc64. Otherwise kernel built with DEBUG_MEMGUARD panics early in kmeminit() with the message kmem_suballoc: bad status return of 1 because of zero size argument passed to kmem_suballoc() due to vm_kmem_size_max being zero. The problem also exists on ia64. Modified: head/sys/sparc64/include/vmparam.h Modified: head/sys/sparc64/include/vmparam.h == --- head/sys/sparc64/include/vmparam.hSun Nov 28 18:59:52 2010 (r216015) +++ head/sys/sparc64/include/vmparam.hSun Nov 28 19:26:20 2010 (r216016) @@ -237,6 +237,14 @@ #endif /* + * Ceiling on amount of kmem_map kva space. + */ +#ifndef VM_KMEM_SIZE_MAX +#define VM_KMEM_SIZE_MAX((VM_MAX_KERNEL_ADDRESS - \ +VM_MIN_KERNEL_ADDRESS + 1) * 3 / 5) +#endif + +/* * Initial pagein size of beginning of executable file. */ #ifndef VM_INITIAL_PAGEIN How was that value determined? I've just copied it from amd64 to be non-zero for now. Do you have a better idea of what it should look like? Well, on sparc64 VM_MAX_KERNEL_ADDRESS already is dynamically adjusted to the maximum appropriate for the specific CPU during the early cycles of the kernel so I'd think one could just use VM_MAX_KERNEL_ADDRESS - VM_MIN_KERNEL_ADDRESS for VM_KMEM_SIZE_MAX there, I'm not sure what the intention of the ceiling provided by that macro actually is though In any case, the commit message of r180210 which changed the amd64 version to the current one talks about limiting the kmem map to 3.6GB and while it also fails to explain where that value comes from it looks rather amd64 specific and the formula used by the macro will result in a different ceiling on sparc64 and thus inappropriate. I've CC'ed alc@ who hopefully can shed some light on this. Apart from this the actual bug here seems to be that memguard_fudge() can't deal with a km_max being zero or that zero is passed to it as kmeminit() allows for VM_KMEM_SIZE_MAX not being defined. Oops, forgot to actually CC a...@. There's nothing particularly amd64-specific about the definition. In general, if you allow the kmem_map, which is basically the kernel's heap, to consume the entire kernel address space as you propose, then you're leaving no room for the buffer cache, thread stacks, pipes, and a few other things. Since the cap on the kmem_map size as defined by r180210 is a fraction of the overall kernel address space size, it scales automatically with the kernel address space size and should be a reasonable cap definition for most architectures. In the specific case of sparc64, I think it's fair to say that the kernel virtual address is sufficiently large and the amount of physical memory in any of the supported machines is small enough in comparison that it hasn't mattered that a kmem_map cap doesn't exist, because most of the aforementioned structures are scaled based on the amount of physical memory. In fact, it probably won't matter any time soon. All of that said, I would suggest fixing memguard_fudge() and reverting r216016 and the follow up change. All I think that is required to fix memguard_fudge() is Index: vm/memguard.c === --- vm/memguard.c (revision 216070) +++ vm/memguard.c (working copy) @@ -186,7 +186,7 @@ memguard_fudge(unsigned long km_size, unsigned lon memguard_mapsize = round_page(memguard_mapsize); if (memguard_mapsize / (2 * PAGE_SIZE) mem_pgs) memguard_mapsize = mem_pgs * 2 * PAGE_SIZE; - if (km_size + memguard_mapsize km_max) + if (km_max 0 km_size + memguard_mapsize km_max) return (km_max); return (km_size + memguard_mapsize); } Regards, Alan ___ svn-src-all@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/svn-src-all To unsubscribe, send any mail to svn-src-all-unsubscr...@freebsd.org
Re: svn commit: r216016 - head/sys/sparc64/include
Marius, On Tue, Nov 30, 2010 at 1:24 AM, Marius Strobl mar...@alchemy.franken.dewrote: On Sun, Nov 28, 2010 at 07:26:20PM +, Max Khon wrote: Author: fjoe Date: Sun Nov 28 19:26:20 2010 New Revision: 216016 URL: http://svn.freebsd.org/changeset/base/216016 Log: Define VM_KMEM_SIZE_MAX on sparc64. Otherwise kernel built with DEBUG_MEMGUARD panics early in kmeminit() with the message kmem_suballoc: bad status return of 1 because of zero size argument passed to kmem_suballoc() due to vm_kmem_size_max being zero. The problem also exists on ia64. Modified: head/sys/sparc64/include/vmparam.h Modified: head/sys/sparc64/include/vmparam.h == --- head/sys/sparc64/include/vmparam.hSun Nov 28 18:59:52 2010 (r216015) +++ head/sys/sparc64/include/vmparam.hSun Nov 28 19:26:20 2010 (r216016) @@ -237,6 +237,14 @@ #endif /* + * Ceiling on amount of kmem_map kva space. + */ +#ifndef VM_KMEM_SIZE_MAX +#define VM_KMEM_SIZE_MAX((VM_MAX_KERNEL_ADDRESS - \ +VM_MIN_KERNEL_ADDRESS + 1) * 3 / 5) +#endif + +/* * Initial pagein size of beginning of executable file. */ #ifndef VM_INITIAL_PAGEIN How was that value determined? I've just copied it from amd64 to be non-zero for now. Do you have a better idea of what it should look like? Well, on sparc64 VM_MAX_KERNEL_ADDRESS already is dynamically adjusted to the maximum appropriate for the specific CPU during the early cycles of the kernel so I'd think one could just use VM_MAX_KERNEL_ADDRESS - VM_MIN_KERNEL_ADDRESS for VM_KMEM_SIZE_MAX there, I'm not sure what the intention of the ceiling provided by that macro actually is though Ok, this sounds reasonable. I changed it to be just (VM_MAX_KERNEL_ADDRESS - VM_MIN_KERNEL_ADDRESS) In any case, the commit message of r180210 which changed the amd64 version to the current one talks about limiting the kmem map to 3.6GB and while it also fails to explain where that value comes from it looks rather amd64 specific and the formula used by the macro will result in a different ceiling on sparc64 and thus inappropriate. I've CC'ed alc@ who hopefully can shed some light on this. Apart from this the actual bug here seems to be that memguard_fudge() can't deal with a km_max being zero or that zero is passed to it as kmeminit() allows for VM_KMEM_SIZE_MAX not being defined. I thought about fixing memguard_fudge() too, but defining VM_KMEM_SIZE_MAX seemed to be less intrusive change to me. Max ___ svn-src-all@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/svn-src-all To unsubscribe, send any mail to svn-src-all-unsubscr...@freebsd.org
Re: svn commit: r216016 - head/sys/sparc64/include
Marius, On Mon, Nov 29, 2010 at 1:45 AM, Marius Strobl mar...@alchemy.franken.dewrote: On Sun, Nov 28, 2010 at 07:26:20PM +, Max Khon wrote: Author: fjoe Date: Sun Nov 28 19:26:20 2010 New Revision: 216016 URL: http://svn.freebsd.org/changeset/base/216016 Log: Define VM_KMEM_SIZE_MAX on sparc64. Otherwise kernel built with DEBUG_MEMGUARD panics early in kmeminit() with the message kmem_suballoc: bad status return of 1 because of zero size argument passed to kmem_suballoc() due to vm_kmem_size_max being zero. The problem also exists on ia64. Modified: head/sys/sparc64/include/vmparam.h Modified: head/sys/sparc64/include/vmparam.h == --- head/sys/sparc64/include/vmparam.hSun Nov 28 18:59:52 2010 (r216015) +++ head/sys/sparc64/include/vmparam.hSun Nov 28 19:26:20 2010 (r216016) @@ -237,6 +237,14 @@ #endif /* + * Ceiling on amount of kmem_map kva space. + */ +#ifndef VM_KMEM_SIZE_MAX +#define VM_KMEM_SIZE_MAX((VM_MAX_KERNEL_ADDRESS - \ +VM_MIN_KERNEL_ADDRESS + 1) * 3 / 5) +#endif + +/* * Initial pagein size of beginning of executable file. */ #ifndef VM_INITIAL_PAGEIN How was that value determined? I've just copied it from amd64 to be non-zero for now. Do you have a better idea of what it should look like? Max ___ svn-src-all@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/svn-src-all To unsubscribe, send any mail to svn-src-all-unsubscr...@freebsd.org
Re: svn commit: r216016 - head/sys/sparc64/include
On Tue, Nov 30, 2010 at 12:31:31AM +0600, Max Khon wrote: Marius, On Mon, Nov 29, 2010 at 1:45 AM, Marius Strobl mar...@alchemy.franken.dewrote: On Sun, Nov 28, 2010 at 07:26:20PM +, Max Khon wrote: Author: fjoe Date: Sun Nov 28 19:26:20 2010 New Revision: 216016 URL: http://svn.freebsd.org/changeset/base/216016 Log: Define VM_KMEM_SIZE_MAX on sparc64. Otherwise kernel built with DEBUG_MEMGUARD panics early in kmeminit() with the message kmem_suballoc: bad status return of 1 because of zero size argument passed to kmem_suballoc() due to vm_kmem_size_max being zero. The problem also exists on ia64. Modified: head/sys/sparc64/include/vmparam.h Modified: head/sys/sparc64/include/vmparam.h == --- head/sys/sparc64/include/vmparam.hSun Nov 28 18:59:52 2010 (r216015) +++ head/sys/sparc64/include/vmparam.hSun Nov 28 19:26:20 2010 (r216016) @@ -237,6 +237,14 @@ #endif /* + * Ceiling on amount of kmem_map kva space. + */ +#ifndef VM_KMEM_SIZE_MAX +#define VM_KMEM_SIZE_MAX((VM_MAX_KERNEL_ADDRESS - \ +VM_MIN_KERNEL_ADDRESS + 1) * 3 / 5) +#endif + +/* * Initial pagein size of beginning of executable file. */ #ifndef VM_INITIAL_PAGEIN How was that value determined? I've just copied it from amd64 to be non-zero for now. Do you have a better idea of what it should look like? Well, on sparc64 VM_MAX_KERNEL_ADDRESS already is dynamically adjusted to the maximum appropriate for the specific CPU during the early cycles of the kernel so I'd think one could just use VM_MAX_KERNEL_ADDRESS - VM_MIN_KERNEL_ADDRESS for VM_KMEM_SIZE_MAX there, I'm not sure what the intention of the ceiling provided by that macro actually is though In any case, the commit message of r180210 which changed the amd64 version to the current one talks about limiting the kmem map to 3.6GB and while it also fails to explain where that value comes from it looks rather amd64 specific and the formula used by the macro will result in a different ceiling on sparc64 and thus inappropriate. I've CC'ed alc@ who hopefully can shed some light on this. Apart from this the actual bug here seems to be that memguard_fudge() can't deal with a km_max being zero or that zero is passed to it as kmeminit() allows for VM_KMEM_SIZE_MAX not being defined. Marius ___ svn-src-all@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/svn-src-all To unsubscribe, send any mail to svn-src-all-unsubscr...@freebsd.org
Re: svn commit: r216016 - head/sys/sparc64/include
On Mon, Nov 29, 2010 at 08:23:08PM +0100, Marius Strobl wrote: On Tue, Nov 30, 2010 at 12:31:31AM +0600, Max Khon wrote: Marius, On Mon, Nov 29, 2010 at 1:45 AM, Marius Strobl mar...@alchemy.franken.dewrote: On Sun, Nov 28, 2010 at 07:26:20PM +, Max Khon wrote: Author: fjoe Date: Sun Nov 28 19:26:20 2010 New Revision: 216016 URL: http://svn.freebsd.org/changeset/base/216016 Log: Define VM_KMEM_SIZE_MAX on sparc64. Otherwise kernel built with DEBUG_MEMGUARD panics early in kmeminit() with the message kmem_suballoc: bad status return of 1 because of zero size argument passed to kmem_suballoc() due to vm_kmem_size_max being zero. The problem also exists on ia64. Modified: head/sys/sparc64/include/vmparam.h Modified: head/sys/sparc64/include/vmparam.h == --- head/sys/sparc64/include/vmparam.hSun Nov 28 18:59:52 2010 (r216015) +++ head/sys/sparc64/include/vmparam.hSun Nov 28 19:26:20 2010 (r216016) @@ -237,6 +237,14 @@ #endif /* + * Ceiling on amount of kmem_map kva space. + */ +#ifndef VM_KMEM_SIZE_MAX +#define VM_KMEM_SIZE_MAX((VM_MAX_KERNEL_ADDRESS - \ +VM_MIN_KERNEL_ADDRESS + 1) * 3 / 5) +#endif + +/* * Initial pagein size of beginning of executable file. */ #ifndef VM_INITIAL_PAGEIN How was that value determined? I've just copied it from amd64 to be non-zero for now. Do you have a better idea of what it should look like? Well, on sparc64 VM_MAX_KERNEL_ADDRESS already is dynamically adjusted to the maximum appropriate for the specific CPU during the early cycles of the kernel so I'd think one could just use VM_MAX_KERNEL_ADDRESS - VM_MIN_KERNEL_ADDRESS for VM_KMEM_SIZE_MAX there, I'm not sure what the intention of the ceiling provided by that macro actually is though In any case, the commit message of r180210 which changed the amd64 version to the current one talks about limiting the kmem map to 3.6GB and while it also fails to explain where that value comes from it looks rather amd64 specific and the formula used by the macro will result in a different ceiling on sparc64 and thus inappropriate. I've CC'ed alc@ who hopefully can shed some light on this. Apart from this the actual bug here seems to be that memguard_fudge() can't deal with a km_max being zero or that zero is passed to it as kmeminit() allows for VM_KMEM_SIZE_MAX not being defined. Oops, forgot to actually CC a...@. Marius ___ svn-src-all@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/svn-src-all To unsubscribe, send any mail to svn-src-all-unsubscr...@freebsd.org
svn commit: r216016 - head/sys/sparc64/include
Author: fjoe Date: Sun Nov 28 19:26:20 2010 New Revision: 216016 URL: http://svn.freebsd.org/changeset/base/216016 Log: Define VM_KMEM_SIZE_MAX on sparc64. Otherwise kernel built with DEBUG_MEMGUARD panics early in kmeminit() with the message kmem_suballoc: bad status return of 1 because of zero size argument passed to kmem_suballoc() due to vm_kmem_size_max being zero. The problem also exists on ia64. Modified: head/sys/sparc64/include/vmparam.h Modified: head/sys/sparc64/include/vmparam.h == --- head/sys/sparc64/include/vmparam.h Sun Nov 28 18:59:52 2010 (r216015) +++ head/sys/sparc64/include/vmparam.h Sun Nov 28 19:26:20 2010 (r216016) @@ -237,6 +237,14 @@ #endif /* + * Ceiling on amount of kmem_map kva space. + */ +#ifndef VM_KMEM_SIZE_MAX +#defineVM_KMEM_SIZE_MAX((VM_MAX_KERNEL_ADDRESS - \ +VM_MIN_KERNEL_ADDRESS + 1) * 3 / 5) +#endif + +/* * Initial pagein size of beginning of executable file. */ #ifndefVM_INITIAL_PAGEIN ___ svn-src-all@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/svn-src-all To unsubscribe, send any mail to svn-src-all-unsubscr...@freebsd.org
Re: svn commit: r216016 - head/sys/sparc64/include
On Sun, Nov 28, 2010 at 07:26:20PM +, Max Khon wrote: Author: fjoe Date: Sun Nov 28 19:26:20 2010 New Revision: 216016 URL: http://svn.freebsd.org/changeset/base/216016 Log: Define VM_KMEM_SIZE_MAX on sparc64. Otherwise kernel built with DEBUG_MEMGUARD panics early in kmeminit() with the message kmem_suballoc: bad status return of 1 because of zero size argument passed to kmem_suballoc() due to vm_kmem_size_max being zero. The problem also exists on ia64. Modified: head/sys/sparc64/include/vmparam.h Modified: head/sys/sparc64/include/vmparam.h == --- head/sys/sparc64/include/vmparam.hSun Nov 28 18:59:52 2010 (r216015) +++ head/sys/sparc64/include/vmparam.hSun Nov 28 19:26:20 2010 (r216016) @@ -237,6 +237,14 @@ #endif /* + * Ceiling on amount of kmem_map kva space. + */ +#ifndef VM_KMEM_SIZE_MAX +#define VM_KMEM_SIZE_MAX((VM_MAX_KERNEL_ADDRESS - \ +VM_MIN_KERNEL_ADDRESS + 1) * 3 / 5) +#endif + +/* * Initial pagein size of beginning of executable file. */ #ifndef VM_INITIAL_PAGEIN How was that value determined? Marius ___ svn-src-all@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/svn-src-all To unsubscribe, send any mail to svn-src-all-unsubscr...@freebsd.org
