Re: [PATCH] Fix corruption of memmap on IA64 SPARSEMEM when mem_section is not a power of 2
On Wed, 25 Jul 2007 13:55:02 +0100 [EMAIL PROTECTED] (Mel Gorman) wrote: > > mm/sparse.c: In function `sparse_init': > > mm/sparse.c:482: error: implicit declaration of function > > `sparse_early_usemap_alloc' > > mm/sparse.c:482: warning: assignment makes pointer from integer without a > > cast > > mm/sparse.c: In function `sparse_add_one_section': > > mm/sparse.c:553: error: implicit declaration of function > > `__kmalloc_section_usemap' > > mm/sparse.c:553: warning: assignment makes pointer from integer without a > > cast > > I'm looking at this now and getting a superh cross-compiler built to > build-test any fix. My first impression is that > sparse_early_usemap_alloc() needs to be defined whether > CONFIG_SPARSEMEM_VMEMMAP is set or not. Right now, > sparse_early_usemap_alloc() is only defined when it is set and it's not > clear why although "by accident" is the most likely explanation. > Thanks. There's an sh cross-compiler in http://userweb.kernel.org/~akpm/cross-compilers/, btw. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH] Fix corruption of memmap on IA64 SPARSEMEM when mem_section is not a power of 2
Mel Gorman wrote:
> On (24/07/07 22:01), Andrew Morton didst pronounce:
>> On Tue, 13 Mar 2007 10:42:02 + [EMAIL PROTECTED] (Mel Gorman) wrote:
>>
>>> There are problems in the use of SPARSEMEM and pageblock flags that causes
>>> problems on ia64.
>>>
>>>
>> SuperH allmodconfig blew up:
>>
>> mm/sparse.c: In function `sparse_init':
>> mm/sparse.c:482: error: implicit declaration of function
>> `sparse_early_usemap_alloc'
>> mm/sparse.c:482: warning: assignment makes pointer from integer without a
>> cast
>> mm/sparse.c: In function `sparse_add_one_section':
>> mm/sparse.c:553: error: implicit declaration of function
>> `__kmalloc_section_usemap'
>> mm/sparse.c:553: warning: assignment makes pointer from integer without a
>> cast
>
> This error is due to a bad interaction between
> generic-virtual-memmap-support-for-sparsemem.patch and
> fix-corruption-of-memmap-on-ia64-sparsemem-when-mem_section-is-not-a-power-of-2.patch
> . Functions that are needed whether CONFIG_SPARSEMEM_VMEMMAP is set are
> not are depending on CONFIG_SPARSEMEM_VMEMMAP . This breaks on arch/sh
> for example where SPARSEMEM may be set but not NUMA.
>
> Signed-off-by: Mel Gorman <[EMAIL PROTECTED]>
>
> ---
> sparse.c | 58 +-
> 1 file changed, 29 insertions(+), 29 deletions(-)
>
>
> diff -rup -X /usr/src/patchset-0.6/bin//dontdiff
> linux-2.6.23-rc1-mm1-clean/mm/sparse.c
> linux-2.6.23-rc1-mm1-005-sh-sparsemem-fix/mm/sparse.c
> --- linux-2.6.23-rc1-mm1-clean/mm/sparse.c2007-07-25 13:43:30.0
> +0100
> +++ linux-2.6.23-rc1-mm1-005-sh-sparsemem-fix/mm/sparse.c 2007-07-25
> 14:27:58.0 +0100
> @@ -220,6 +220,35 @@ void *alloc_bootmem_high_node(pg_data_t
> return NULL;
> }
>
> +static unsigned long usemap_size(void)
> +{
> + unsigned long size_bytes;
> + size_bytes = roundup(SECTION_BLOCKFLAGS_BITS, 8) / 8;
> + size_bytes = roundup(size_bytes, sizeof(unsigned long));
> + return size_bytes;
> +}
> +
> +#ifdef CONFIG_MEMORY_HOTPLUG
> +static unsigned long *__kmalloc_section_usemap(void)
> +{
> + return kmalloc(usemap_size(), GFP_KERNEL);
> +}
> +#endif /* CONFIG_MEMORY_HOTPLUG */
> +
> +static unsigned long *sparse_early_usemap_alloc(unsigned long pnum)
> +{
> + unsigned long *usemap;
> + struct mem_section *ms = __nr_to_section(pnum);
> + int nid = sparse_early_nid(ms);
> +
> + usemap = alloc_bootmem_node(NODE_DATA(nid), usemap_size());
> + if (usemap)
> + return usemap;
> +
> + printk(KERN_WARNING "%s: allocation failed\n", __FUNCTION__);
> + return NULL;
> +}
> +
> #ifdef CONFIG_SPARSEMEM_VMEMMAP
> /*
> * Virtual Memory Map support
> @@ -254,35 +283,6 @@ void *alloc_bootmem_high_node(pg_data_t
> * require one PTE/TLB per PAGE_SIZE chunk of the virtual memory map.
> */
>
> -static unsigned long usemap_size(void)
> -{
> - unsigned long size_bytes;
> - size_bytes = roundup(SECTION_BLOCKFLAGS_BITS, 8) / 8;
> - size_bytes = roundup(size_bytes, sizeof(unsigned long));
> - return size_bytes;
> -}
> -
> -#ifdef CONFIG_MEMORY_HOTPLUG
> -static unsigned long *__kmalloc_section_usemap(void)
> -{
> - return kmalloc(usemap_size(), GFP_KERNEL);
> -}
> -#endif /* CONFIG_MEMORY_HOTPLUG */
> -
> -static unsigned long *sparse_early_usemap_alloc(unsigned long pnum)
> -{
> - unsigned long *usemap;
> - struct mem_section *ms = __nr_to_section(pnum);
> - int nid = sparse_early_nid(ms);
> -
> - usemap = alloc_bootmem_node(NODE_DATA(nid), usemap_size());
> - if (usemap)
> - return usemap;
> -
> - printk(KERN_WARNING "%s: allocation failed\n", __FUNCTION__);
> - return NULL;
> -}
> -
> /*
> * Allocate a block of memory to be used to back the virtual memory map
> * or to back the page tables that are used to create the mapping.
This also affects NUMA-Q in some SPARSEMEM configurations. Applying
this fixes it up.
Looks very much like diff has dropped the stuff within my #ifdef
'incorrectly' and as such is a merge error. This should probabally be
considered a fix to this patch:
fix-corruption-of-memmap-on-ia64-sparsemem-when-mem_section-is-not-a-power-of-2.patch
Acked-by: Andy Whitcroft <[EMAIL PROTECTED]>
-apw
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH] Fix corruption of memmap on IA64 SPARSEMEM when mem_section is not a power of 2
On (25/07/07 15:49), Mel Gorman didst pronounce: > > SuperH allmodconfig blew up: > > > > mm/sparse.c: In function `sparse_init': > > mm/sparse.c:482: error: implicit declaration of function > > `sparse_early_usemap_alloc' > > mm/sparse.c:482: warning: assignment makes pointer from integer without a > > cast > > mm/sparse.c: In function `sparse_add_one_section': > > mm/sparse.c:553: error: implicit declaration of function > > `__kmalloc_section_usemap' > > mm/sparse.c:553: warning: assignment makes pointer from integer without a > > cast > > This error is due to a bad interaction between > generic-virtual-memmap-support-for-sparsemem.patch and > fix-corruption-of-memmap-on-ia64-sparsemem-when-mem_section-is-not-a-power-of-2.patch > . Functions that are needed whether CONFIG_SPARSEMEM_VMEMMAP is set are > not are depending on CONFIG_SPARSEMEM_VMEMMAP . This breaks on arch/sh > for example where SPARSEMEM may be set but not NUMA. > This patch only moves code and does not alter it. However, as SPARSEMEM is set without NUMA, the code move generates a warning on SuperH because NODE_DATA(nid) is a no-op. The following patch supresses the warning Signed-off-by: Mel Gorman <[EMAIL PROTECTED]> --- sparse.c |3 +++ 1 file changed, 3 insertions(+) diff -rup -X /usr/src/patchset-0.6/bin//dontdiff linux-2.6.23-rc1-mm1-005-sh-sparsemem-fix/mm/sparse.c linux-2.6.23-rc1-mm1-010-sh-sparsemem-fix-warning/mm/sparse.c --- linux-2.6.23-rc1-mm1-005-sh-sparsemem-fix/mm/sparse.c 2007-07-25 14:27:58.0 +0100 +++ linux-2.6.23-rc1-mm1-010-sh-sparsemem-fix-warning/mm/sparse.c 2007-07-25 15:43:07.0 +0100 @@ -245,6 +245,9 @@ static unsigned long *sparse_early_usema if (usemap) return usemap; + /* Stupid: suppress gcc warning for SPARSEMEM && !NUMA */ + nid = 0; + printk(KERN_WARNING "%s: allocation failed\n", __FUNCTION__); return NULL; } - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH] Fix corruption of memmap on IA64 SPARSEMEM when mem_section is not a power of 2
On (24/07/07 22:01), Andrew Morton didst pronounce:
> On Tue, 13 Mar 2007 10:42:02 + [EMAIL PROTECTED] (Mel Gorman) wrote:
>
> > There are problems in the use of SPARSEMEM and pageblock flags that causes
> > problems on ia64.
> >
> >
>
> SuperH allmodconfig blew up:
>
> mm/sparse.c: In function `sparse_init':
> mm/sparse.c:482: error: implicit declaration of function
> `sparse_early_usemap_alloc'
> mm/sparse.c:482: warning: assignment makes pointer from integer without a cast
> mm/sparse.c: In function `sparse_add_one_section':
> mm/sparse.c:553: error: implicit declaration of function
> `__kmalloc_section_usemap'
> mm/sparse.c:553: warning: assignment makes pointer from integer without a cast
This error is due to a bad interaction between
generic-virtual-memmap-support-for-sparsemem.patch and
fix-corruption-of-memmap-on-ia64-sparsemem-when-mem_section-is-not-a-power-of-2.patch
. Functions that are needed whether CONFIG_SPARSEMEM_VMEMMAP is set are
not are depending on CONFIG_SPARSEMEM_VMEMMAP . This breaks on arch/sh
for example where SPARSEMEM may be set but not NUMA.
Signed-off-by: Mel Gorman <[EMAIL PROTECTED]>
---
sparse.c | 58 +-
1 file changed, 29 insertions(+), 29 deletions(-)
diff -rup -X /usr/src/patchset-0.6/bin//dontdiff
linux-2.6.23-rc1-mm1-clean/mm/sparse.c
linux-2.6.23-rc1-mm1-005-sh-sparsemem-fix/mm/sparse.c
--- linux-2.6.23-rc1-mm1-clean/mm/sparse.c 2007-07-25 13:43:30.0
+0100
+++ linux-2.6.23-rc1-mm1-005-sh-sparsemem-fix/mm/sparse.c 2007-07-25
14:27:58.0 +0100
@@ -220,6 +220,35 @@ void *alloc_bootmem_high_node(pg_data_t
return NULL;
}
+static unsigned long usemap_size(void)
+{
+ unsigned long size_bytes;
+ size_bytes = roundup(SECTION_BLOCKFLAGS_BITS, 8) / 8;
+ size_bytes = roundup(size_bytes, sizeof(unsigned long));
+ return size_bytes;
+}
+
+#ifdef CONFIG_MEMORY_HOTPLUG
+static unsigned long *__kmalloc_section_usemap(void)
+{
+ return kmalloc(usemap_size(), GFP_KERNEL);
+}
+#endif /* CONFIG_MEMORY_HOTPLUG */
+
+static unsigned long *sparse_early_usemap_alloc(unsigned long pnum)
+{
+ unsigned long *usemap;
+ struct mem_section *ms = __nr_to_section(pnum);
+ int nid = sparse_early_nid(ms);
+
+ usemap = alloc_bootmem_node(NODE_DATA(nid), usemap_size());
+ if (usemap)
+ return usemap;
+
+ printk(KERN_WARNING "%s: allocation failed\n", __FUNCTION__);
+ return NULL;
+}
+
#ifdef CONFIG_SPARSEMEM_VMEMMAP
/*
* Virtual Memory Map support
@@ -254,35 +283,6 @@ void *alloc_bootmem_high_node(pg_data_t
* require one PTE/TLB per PAGE_SIZE chunk of the virtual memory map.
*/
-static unsigned long usemap_size(void)
-{
- unsigned long size_bytes;
- size_bytes = roundup(SECTION_BLOCKFLAGS_BITS, 8) / 8;
- size_bytes = roundup(size_bytes, sizeof(unsigned long));
- return size_bytes;
-}
-
-#ifdef CONFIG_MEMORY_HOTPLUG
-static unsigned long *__kmalloc_section_usemap(void)
-{
- return kmalloc(usemap_size(), GFP_KERNEL);
-}
-#endif /* CONFIG_MEMORY_HOTPLUG */
-
-static unsigned long *sparse_early_usemap_alloc(unsigned long pnum)
-{
- unsigned long *usemap;
- struct mem_section *ms = __nr_to_section(pnum);
- int nid = sparse_early_nid(ms);
-
- usemap = alloc_bootmem_node(NODE_DATA(nid), usemap_size());
- if (usemap)
- return usemap;
-
- printk(KERN_WARNING "%s: allocation failed\n", __FUNCTION__);
- return NULL;
-}
-
/*
* Allocate a block of memory to be used to back the virtual memory map
* or to back the page tables that are used to create the mapping.
--
Mel Gorman
Part-time Phd Student Linux Technology Center
University of Limerick IBM Dublin Software Lab
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH] Fix corruption of memmap on IA64 SPARSEMEM when mem_section is not a power of 2
On (24/07/07 22:01), Andrew Morton didst pronounce: > On Tue, 13 Mar 2007 10:42:02 + [EMAIL PROTECTED] (Mel Gorman) wrote: > > > There are problems in the use of SPARSEMEM and pageblock flags that causes > > problems on ia64. > > > > The first part of the problem is that units are incorrect in > > SECTION_BLOCKFLAGS_BITS computation. This results in a map_section's > > section_mem_map being treated as part of a bitmap which isn't good. This > > was evident with an invalid virtual address when mem_init attempted to free > > bootmem pages while relinquishing control from the bootmem allocator. > > > > The second part of the problem occurs because the pageblock flags bitmap is > > be located with the mem_section. The SECTIONS_PER_ROOT computation using > > sizeof (mem_section) may not be a power of 2 depending on the size of the > > bitmap. This renders masks and other such things not power of 2 base. This > > issue was seen with SPARSEMEM_EXTREME on ia64. This patch moves the bitmap > > outside of mem_section and uses a pointer instead in the mem_section. The > > bitmaps are allocated when the section is being initialised. > > > > Note that sparse_early_usemap_alloc() does not use alloc_remap() like > > sparse_early_mem_map_alloc(). The allocation required for the bitmap on x86, > > the only architecture that uses alloc_remap is typically smaller than a > > cache > > line. alloc_remap() pads out allocations to the cache size which would be > > a needless waste. > > > > Credit to Bob Picco for identifying the original problem and effecting a > > fix for the SECTION_BLOCKFLAGS_BITS calculation. Credit to Andy Whitcroft > > for devising the best way of allocating the bitmaps only when required for > > the section. > > SuperH allmodconfig blew up: > > mm/sparse.c: In function `sparse_init': > mm/sparse.c:482: error: implicit declaration of function > `sparse_early_usemap_alloc' > mm/sparse.c:482: warning: assignment makes pointer from integer without a cast > mm/sparse.c: In function `sparse_add_one_section': > mm/sparse.c:553: error: implicit declaration of function > `__kmalloc_section_usemap' > mm/sparse.c:553: warning: assignment makes pointer from integer without a cast I'm looking at this now and getting a superh cross-compiler built to build-test any fix. My first impression is that sparse_early_usemap_alloc() needs to be defined whether CONFIG_SPARSEMEM_VMEMMAP is set or not. Right now, sparse_early_usemap_alloc() is only defined when it is set and it's not clear why although "by accident" is the most likely explanation. -- Mel Gorman Part-time Phd Student Linux Technology Center University of Limerick IBM Dublin Software Lab - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH] Fix corruption of memmap on IA64 SPARSEMEM when mem_section is not a power of 2
On Tue, 13 Mar 2007 10:42:02 + [EMAIL PROTECTED] (Mel Gorman) wrote: > There are problems in the use of SPARSEMEM and pageblock flags that causes > problems on ia64. > > The first part of the problem is that units are incorrect in > SECTION_BLOCKFLAGS_BITS computation. This results in a map_section's > section_mem_map being treated as part of a bitmap which isn't good. This > was evident with an invalid virtual address when mem_init attempted to free > bootmem pages while relinquishing control from the bootmem allocator. > > The second part of the problem occurs because the pageblock flags bitmap is > be located with the mem_section. The SECTIONS_PER_ROOT computation using > sizeof (mem_section) may not be a power of 2 depending on the size of the > bitmap. This renders masks and other such things not power of 2 base. This > issue was seen with SPARSEMEM_EXTREME on ia64. This patch moves the bitmap > outside of mem_section and uses a pointer instead in the mem_section. The > bitmaps are allocated when the section is being initialised. > > Note that sparse_early_usemap_alloc() does not use alloc_remap() like > sparse_early_mem_map_alloc(). The allocation required for the bitmap on x86, > the only architecture that uses alloc_remap is typically smaller than a cache > line. alloc_remap() pads out allocations to the cache size which would be > a needless waste. > > Credit to Bob Picco for identifying the original problem and effecting a > fix for the SECTION_BLOCKFLAGS_BITS calculation. Credit to Andy Whitcroft > for devising the best way of allocating the bitmaps only when required for > the section. SuperH allmodconfig blew up: mm/sparse.c: In function `sparse_init': mm/sparse.c:482: error: implicit declaration of function `sparse_early_usemap_alloc' mm/sparse.c:482: warning: assignment makes pointer from integer without a cast mm/sparse.c: In function `sparse_add_one_section': mm/sparse.c:553: error: implicit declaration of function `__kmalloc_section_usemap' mm/sparse.c:553: warning: assignment makes pointer from integer without a cast - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
