Re: [Xen-devel] [RFC PATCH v3 08/24] NUMA: x86: Move numa code and make it generic

2017-07-24 Thread Stefano Stabellini 
On Wed, 19 Jul 2017, Julien Grall wrote:
> Hi Vijay,
> 
> On 18/07/17 12:41, vijay.kil...@gmail.com wrote:
> > From: Vijaya Kumar K 
> > 
> > Move code from xen/arch/x86/numa.c to xen/common/numa.c
> > so that it can be used by other archs.
> > 
> > The following changes are done:
> > - Few generic static functions in x86/numa.c is made
> >   non-static common/numa.c
> > - The generic contents of header file asm-x86/numa.h
> >   are moved to xen/numa.h.
> > - The header file includes are reordered and externs are
> >   dropped.
> > - Moved acpi_numa from asm-x86/acpi.h to xen/acpi.h
> > - Coding style of code moved to commom/numa.c is changed
> >   to Xen style.
> > - numa_add_cpu() and numa_set_node() and moved to header
> >   file and added inline function in case of CONFIG_NUMA
> >   is not enabled because these functions are called from
> >   generic code with out any config check.
> > 
> > Also the node_online_map is defined in x86/numa.c for x86
> > and arm/smpboot.c for ARM. For x86 it is moved to x86/smpboot.c
> > If moved to common code the compilation fails because
> > common/numa.c is compiled only when NUMA is enabled.
> 
> I would much prefer if this patch does one thing: Moving code. The rest should
> be split out to help review and allowing us to easily verify you only moved
> code...

Indeed. However for the sake of making things easier, I did go through
the patch line by line (manually and automatically) to check the code
movement and it is correct.


> > +#define NODE_DATA(nid)  (&(node_data[nid]))
> > +
> > +#define node_start_pfn(nid) NODE_DATA(nid)->node_start_pfn
> > +#define node_spanned_pages(nid) NODE_DATA(nid)->node_spanned_pages
> > +#define node_end_pfn(nid)   NODE_DATA(nid)->node_start_pfn + \
> > + NODE_DATA(nid)->node_spanned_pages
> > +
> > +void numa_add_cpu(int cpu);
> > +void numa_set_node(int cpu, nodeid_t node);
> > +#else
> > +static inline void numa_add_cpu(int cpu) { }
> > +static inline void numa_set_node(int cpu, nodeid_t node) { }
> 
> I am not sure why you need to define stub at least for numa_set_node... I
> can't see use in non-NUMA code. I will comment about the numa_add_cpu later.


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [RFC PATCH v3 08/24] NUMA: x86: Move numa code and make it generic

2017-07-20 Thread Julien Grall 

Hi Vijay,

On 20/07/17 09:55, Vijay Kilari wrote:

On Wed, Jul 19, 2017 at 11:11 PM, Julien Grall  wrote:

Hi Vijay,

On 18/07/17 12:41, vijay.kil...@gmail.com wrote:


From: Vijaya Kumar K 

Move code from xen/arch/x86/numa.c to xen/common/numa.c
so that it can be used by other archs.

The following changes are done:
- Few generic static functions in x86/numa.c is made
  non-static common/numa.c
- The generic contents of header file asm-x86/numa.h
  are moved to xen/numa.h.
- The header file includes are reordered and externs are
  dropped.
- Moved acpi_numa from asm-x86/acpi.h to xen/acpi.h
- Coding style of code moved to commom/numa.c is changed
  to Xen style.
- numa_add_cpu() and numa_set_node() and moved to header
  file and added inline function in case of CONFIG_NUMA
  is not enabled because these functions are called from
  generic code with out any config check.

Also the node_online_map is defined in x86/numa.c for x86
and arm/smpboot.c for ARM. For x86 it is moved to x86/smpboot.c
If moved to common code the compilation fails because
common/numa.c is compiled only when NUMA is enabled.



I would much prefer if this patch does one thing: Moving code. The rest
should be split out to help review and allowing us to easily verify you only
moved code...


Yes, this patch is doing only code movement. Apart from adding inline function
for numa_add_cpu() and numa_set_node().


The "apart" should then be in a separate patch. I don't want to spend 
hours trying to decipher a patch mixing code movement and add code at 
the same time.







+#define NODE_DATA(nid)  (&(node_data[nid]))
+
+#define node_start_pfn(nid) NODE_DATA(nid)->node_start_pfn
+#define node_spanned_pages(nid) NODE_DATA(nid)->node_spanned_pages
+#define node_end_pfn(nid)   NODE_DATA(nid)->node_start_pfn + \
+ NODE_DATA(nid)->node_spanned_pages
+
+void numa_add_cpu(int cpu);
+void numa_set_node(int cpu, nodeid_t node);
+#else
+static inline void numa_add_cpu(int cpu) { }
+static inline void numa_set_node(int cpu, nodeid_t node) { }



I am not sure why you need to define stub at least for numa_set_node... I
can't see use in non-NUMA code. I will comment about the numa_add_cpu later.


x86 is using from setup.c. yes if we assume that numa is always enabled for x86,
I can drop numa_set_node() inline function.


Looking at the code, I don't think there is any way to disable NUMA on 
x86 at the moment... So there is no point to keep it.


Cheers,

--
Julien Grall

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [RFC PATCH v3 08/24] NUMA: x86: Move numa code and make it generic

2017-07-20 Thread Vijay Kilari 
On Wed, Jul 19, 2017 at 11:11 PM, Julien Grall  wrote:
> Hi Vijay,
>
> On 18/07/17 12:41, vijay.kil...@gmail.com wrote:
>>
>> From: Vijaya Kumar K 
>>
>> Move code from xen/arch/x86/numa.c to xen/common/numa.c
>> so that it can be used by other archs.
>>
>> The following changes are done:
>> - Few generic static functions in x86/numa.c is made
>>   non-static common/numa.c
>> - The generic contents of header file asm-x86/numa.h
>>   are moved to xen/numa.h.
>> - The header file includes are reordered and externs are
>>   dropped.
>> - Moved acpi_numa from asm-x86/acpi.h to xen/acpi.h
>> - Coding style of code moved to commom/numa.c is changed
>>   to Xen style.
>> - numa_add_cpu() and numa_set_node() and moved to header
>>   file and added inline function in case of CONFIG_NUMA
>>   is not enabled because these functions are called from
>>   generic code with out any config check.
>>
>> Also the node_online_map is defined in x86/numa.c for x86
>> and arm/smpboot.c for ARM. For x86 it is moved to x86/smpboot.c
>> If moved to common code the compilation fails because
>> common/numa.c is compiled only when NUMA is enabled.
>
>
> I would much prefer if this patch does one thing: Moving code. The rest
> should be split out to help review and allowing us to easily verify you only
> moved code...

Yes, this patch is doing only code movement. Apart from adding inline function
for numa_add_cpu() and numa_set_node().

>
>> +#define NODE_DATA(nid)  (&(node_data[nid]))
>> +
>> +#define node_start_pfn(nid) NODE_DATA(nid)->node_start_pfn
>> +#define node_spanned_pages(nid) NODE_DATA(nid)->node_spanned_pages
>> +#define node_end_pfn(nid)   NODE_DATA(nid)->node_start_pfn + \
>> + NODE_DATA(nid)->node_spanned_pages
>> +
>> +void numa_add_cpu(int cpu);
>> +void numa_set_node(int cpu, nodeid_t node);
>> +#else
>> +static inline void numa_add_cpu(int cpu) { }
>> +static inline void numa_set_node(int cpu, nodeid_t node) { }
>
>
> I am not sure why you need to define stub at least for numa_set_node... I
> can't see use in non-NUMA code. I will comment about the numa_add_cpu later.

x86 is using from setup.c. yes if we assume that numa is always enabled for x86,
I can drop numa_set_node() inline function.

>
> Cheers,
>
> --
> Julien Grall

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [RFC PATCH v3 08/24] NUMA: x86: Move numa code and make it generic

2017-07-19 Thread Julien Grall 

Hi Vijay,

On 18/07/17 12:41, vijay.kil...@gmail.com wrote:

From: Vijaya Kumar K 

Move code from xen/arch/x86/numa.c to xen/common/numa.c
so that it can be used by other archs.

The following changes are done:
- Few generic static functions in x86/numa.c is made
  non-static common/numa.c
- The generic contents of header file asm-x86/numa.h
  are moved to xen/numa.h.
- The header file includes are reordered and externs are
  dropped.
- Moved acpi_numa from asm-x86/acpi.h to xen/acpi.h
- Coding style of code moved to commom/numa.c is changed
  to Xen style.
- numa_add_cpu() and numa_set_node() and moved to header
  file and added inline function in case of CONFIG_NUMA
  is not enabled because these functions are called from
  generic code with out any config check.

Also the node_online_map is defined in x86/numa.c for x86
and arm/smpboot.c for ARM. For x86 it is moved to x86/smpboot.c
If moved to common code the compilation fails because
common/numa.c is compiled only when NUMA is enabled.


I would much prefer if this patch does one thing: Moving code. The rest 
should be split out to help review and allowing us to easily verify you 
only moved code...



+#define NODE_DATA(nid)  (&(node_data[nid]))
+
+#define node_start_pfn(nid) NODE_DATA(nid)->node_start_pfn
+#define node_spanned_pages(nid) NODE_DATA(nid)->node_spanned_pages
+#define node_end_pfn(nid)   NODE_DATA(nid)->node_start_pfn + \
+ NODE_DATA(nid)->node_spanned_pages
+
+void numa_add_cpu(int cpu);
+void numa_set_node(int cpu, nodeid_t node);
+#else
+static inline void numa_add_cpu(int cpu) { }
+static inline void numa_set_node(int cpu, nodeid_t node) { }


I am not sure why you need to define stub at least for numa_set_node... 
I can't see use in non-NUMA code. I will comment about the numa_add_cpu 
later.


Cheers,

--
Julien Grall

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [RFC PATCH v3 08/24] NUMA: x86: Move numa code and make it generic

2017-07-19 Thread Vijay Kilari 
On Tue, Jul 18, 2017 at 11:46 PM, Julien Grall  wrote:
> Hi,
>
>
> On 18/07/17 16:29, Wei Liu wrote:
>>
>> On Tue, Jul 18, 2017 at 05:11:30PM +0530, vijay.kil...@gmail.com wrote:
>> [...]
>>>
>>> diff --git a/xen/common/numa.c b/xen/common/numa.c
>>> new file mode 100644
>>> index 000..0381f1b
>>> --- /dev/null
>>> +++ b/xen/common/numa.c
>>> @@ -0,0 +1,487 @@
>>> +/*
>>> + * Common NUMA handling functions for x86 and arm.
>>> + * Original code extracted from arch/x86/numa.c
>>> + *
>>> + * This program is free software; you can redistribute it and/or
>>> + * modify it under the terms and conditions of the GNU General Public
>>> + * License, version 2, as published by the Free Software Foundation.
>>> + *
>>> + * This program is distributed in the hope that it will be useful,
>>> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
>>> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
>>> + * GNU General Public License for more details.
>>> + *
>>> + * You should have received a copy of the GNU General Public License
>>> + * along with this program; If not, see .
>>> + */
>>> +
>>> +#include 
>>> +#include 
>>> +#include 
>>> +#include 
>>> +#include 
>>> +#include 
>>> +#include 
>>> +#include 
>>> +#include 
>>> +#include 
>>> +#include 
>>> +#include 
>>> +#include 
>>> +
>>
>>
>> Since you're moving code anyway, please sort the headers alphabetically.
>>
>>> +static int numa_setup(char *s);
>>> +custom_param("numa", numa_setup);
>>> +
>>> +struct node_data node_data[MAX_NUMNODES];
>>> +
>>> +/* Mapping from pdx to node id */
>>
>>
>> Is this comment applicable to ARM? Does arm has PDX?
>
>
> Yes ARM has PDX. For new architecture we expect the code to provide dummy
> helpers if they want to support NUMA.
>
>>
>>> +unsigned int memnode_shift;
>>> +
>>> +/*
>>> + * In case of numa init failure or numa off,
>>> + * memnode_shift is initialized to BITS_PER_LONG - 1. Hence allocate
>>> + * memnodemap[] of BITS_PER_LONG.
>>> + */
>>> +static typeof(*memnodemap) _memnodemap[BITS_PER_LONG];
>>> +unsigned long memnodemapsize;
>>> +uint8_t *memnodemap;
>>> +
>>> +nodeid_t __read_mostly cpu_to_node[NR_CPUS] = {
>>> +[0 ... NR_CPUS-1] = NUMA_NO_NODE
>>> +};
>>> +
>>> +cpumask_t __read_mostly node_to_cpumask[MAX_NUMNODES];
>>> +
>>> +bool numa_off;
>>> +s8 acpi_numa = 0;
>>> +
>>> +int srat_disabled(void)
>>
>>
>> bool here.
>>
>> Should probably be done in a previous patch.
>
>
> Actually, the previous version had srat_disabled return bool. I am aware
> that Jan and I requested to keep acpi_numa as int, I didn't find any request
> of keep moving srat_disabled to int. So can you explain why??

My bad. I dropped patch #4 from v2. But this change was part of patch
#4 and missed it out.

>
>>
>>> +
>>> +void __init numa_init_array(void)
>>> +{
>>> +int rr, i;
>>> +
>>> +/* There are unfortunately some poorly designed mainboards around
>>> +   that only connect memory to a single CPU. This breaks the 1:1
>>> cpu->node
>>> +   mapping. To avoid this fill in the mapping for all possible
>>> +   CPUs, as the number of CPUs is not known yet.
>>> +   We round robin the existing nodes. */
>>
>>
>> Please fix the coding style issue here.
>>
>
> Cheers,
>
> --
> Julien Grall

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [RFC PATCH v3 08/24] NUMA: x86: Move numa code and make it generic

2017-07-18 Thread Julien Grall 

Hi,

On 18/07/17 16:29, Wei Liu wrote:

On Tue, Jul 18, 2017 at 05:11:30PM +0530, vijay.kil...@gmail.com wrote:
[...]

diff --git a/xen/common/numa.c b/xen/common/numa.c
new file mode 100644
index 000..0381f1b
--- /dev/null
+++ b/xen/common/numa.c
@@ -0,0 +1,487 @@
+/*
+ * Common NUMA handling functions for x86 and arm.
+ * Original code extracted from arch/x86/numa.c
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms and conditions of the GNU General Public
+ * License, version 2, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; If not, see .
+ */
+
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+


Since you're moving code anyway, please sort the headers alphabetically.


+static int numa_setup(char *s);
+custom_param("numa", numa_setup);
+
+struct node_data node_data[MAX_NUMNODES];
+
+/* Mapping from pdx to node id */


Is this comment applicable to ARM? Does arm has PDX?


Yes ARM has PDX. For new architecture we expect the code to provide 
dummy helpers if they want to support NUMA.





+unsigned int memnode_shift;
+
+/*
+ * In case of numa init failure or numa off,
+ * memnode_shift is initialized to BITS_PER_LONG - 1. Hence allocate
+ * memnodemap[] of BITS_PER_LONG.
+ */
+static typeof(*memnodemap) _memnodemap[BITS_PER_LONG];
+unsigned long memnodemapsize;
+uint8_t *memnodemap;
+
+nodeid_t __read_mostly cpu_to_node[NR_CPUS] = {
+[0 ... NR_CPUS-1] = NUMA_NO_NODE
+};
+
+cpumask_t __read_mostly node_to_cpumask[MAX_NUMNODES];
+
+bool numa_off;
+s8 acpi_numa = 0;
+
+int srat_disabled(void)


bool here.

Should probably be done in a previous patch.


Actually, the previous version had srat_disabled return bool. I am aware 
that Jan and I requested to keep acpi_numa as int, I didn't find any 
request of keep moving srat_disabled to int. So can you explain why??





+
+void __init numa_init_array(void)
+{
+int rr, i;
+
+/* There are unfortunately some poorly designed mainboards around
+   that only connect memory to a single CPU. This breaks the 1:1 cpu->node
+   mapping. To avoid this fill in the mapping for all possible
+   CPUs, as the number of CPUs is not known yet.
+   We round robin the existing nodes. */


Please fix the coding style issue here.



Cheers,

--
Julien Grall

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [RFC PATCH v3 08/24] NUMA: x86: Move numa code and make it generic

2017-07-18 Thread Wei Liu 
On Tue, Jul 18, 2017 at 05:11:30PM +0530, vijay.kil...@gmail.com wrote:
[...]
> diff --git a/xen/common/numa.c b/xen/common/numa.c
> new file mode 100644
> index 000..0381f1b
> --- /dev/null
> +++ b/xen/common/numa.c
> @@ -0,0 +1,487 @@
> +/*
> + * Common NUMA handling functions for x86 and arm.
> + * Original code extracted from arch/x86/numa.c
> + *
> + * This program is free software; you can redistribute it and/or
> + * modify it under the terms and conditions of the GNU General Public
> + * License, version 2, as published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> + * GNU General Public License for more details.
> + *
> + * You should have received a copy of the GNU General Public License
> + * along with this program; If not, see .
> + */
> +
> +#include 
> +#include 
> +#include 
> +#include 
> +#include 
> +#include 
> +#include 
> +#include 
> +#include 
> +#include 
> +#include 
> +#include 
> +#include 
> +

Since you're moving code anyway, please sort the headers alphabetically.

> +static int numa_setup(char *s);
> +custom_param("numa", numa_setup);
> +
> +struct node_data node_data[MAX_NUMNODES];
> +
> +/* Mapping from pdx to node id */

Is this comment applicable to ARM? Does arm has PDX?

> +unsigned int memnode_shift;
> +
> +/*
> + * In case of numa init failure or numa off,
> + * memnode_shift is initialized to BITS_PER_LONG - 1. Hence allocate
> + * memnodemap[] of BITS_PER_LONG.
> + */
> +static typeof(*memnodemap) _memnodemap[BITS_PER_LONG];
> +unsigned long memnodemapsize;
> +uint8_t *memnodemap;
> +
> +nodeid_t __read_mostly cpu_to_node[NR_CPUS] = {
> +[0 ... NR_CPUS-1] = NUMA_NO_NODE
> +};
> +
> +cpumask_t __read_mostly node_to_cpumask[MAX_NUMNODES];
> +
> +bool numa_off;
> +s8 acpi_numa = 0;
> +
> +int srat_disabled(void)

bool here.

Should probably be done in a previous patch.

> +
> +void __init numa_init_array(void)
> +{
> +int rr, i;
> +
> +/* There are unfortunately some poorly designed mainboards around
> +   that only connect memory to a single CPU. This breaks the 1:1 
> cpu->node
> +   mapping. To avoid this fill in the mapping for all possible
> +   CPUs, as the number of CPUs is not known yet.
> +   We round robin the existing nodes. */

Please fix the coding style issue here.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [RFC PATCH v3 08/24] NUMA: x86: Move numa code and make it generic

2017-07-18 Thread vijay . kilari 
From: Vijaya Kumar K 

Move code from xen/arch/x86/numa.c to xen/common/numa.c
so that it can be used by other archs.

The following changes are done:
- Few generic static functions in x86/numa.c is made
  non-static common/numa.c
- The generic contents of header file asm-x86/numa.h
  are moved to xen/numa.h.
- The header file includes are reordered and externs are
  dropped.
- Moved acpi_numa from asm-x86/acpi.h to xen/acpi.h
- Coding style of code moved to commom/numa.c is changed
  to Xen style.
- numa_add_cpu() and numa_set_node() and moved to header
  file and added inline function in case of CONFIG_NUMA
  is not enabled because these functions are called from
  generic code with out any config check.

Also the node_online_map is defined in x86/numa.c for x86
and arm/smpboot.c for ARM. For x86 it is moved to x86/smpboot.c
If moved to common code the compilation fails because
common/numa.c is compiled only when NUMA is enabled.

Signed-off-by: Vijaya Kumar K 
---
v3: - Moved acpi_numa variable
- acpi_setup_node declaration move is reverted.
- Dropped extern in header file
- Added inline declaration for numa_add_cpu() and
  numa_set_node() function based on CONFIG_NUMA
- Moved numa_initmem_init() to common code
- Moved common code from asm-x86/numa.h to xen/numa.h
- Moved node_online_map from numa.c to smpboot.c
---
 xen/arch/x86/numa.c | 459 +
 xen/arch/x86/smpboot.c  |   1 +
 xen/common/Makefile |   1 +
 xen/common/numa.c   | 487 
 xen/include/asm-x86/acpi.h  |   1 -
 xen/include/asm-x86/numa.h  |  56 -
 xen/include/asm-x86/setup.h |   1 -
 xen/include/xen/numa.h  |  64 ++
 8 files changed, 561 insertions(+), 509 deletions(-)

diff --git a/xen/arch/x86/numa.c b/xen/arch/x86/numa.c
index 44c2e08..654530b 100644
--- a/xen/arch/x86/numa.c
+++ b/xen/arch/x86/numa.c
@@ -10,323 +10,17 @@
 #include 
 #include 
 #include 
-#include 
 #include 
 #include 
 #include 
 #include 
-#include 
-#include 
-
-static int numa_setup(char *s);
-custom_param("numa", numa_setup);
-
-struct node_data node_data[MAX_NUMNODES];
-
-/* Mapping from pdx to node id */
-unsigned int memnode_shift;
 
 /*
- * In case of numa init failure or numa off,
- * memnode_shift is initialized to BITS_PER_LONG - 1. Hence allocate
- * memnodemap[] of BITS_PER_LONG.
- */
-static typeof(*memnodemap) _memnodemap[BITS_PER_LONG];
-unsigned long memnodemapsize;
-uint8_t *memnodemap;
-
-nodeid_t __read_mostly cpu_to_node[NR_CPUS] = {
-[0 ... NR_CPUS-1] = NUMA_NO_NODE
-};
-/*
  * Keep BIOS's CPU2node information, should not be used for memory allocaion
  */
 nodeid_t apicid_to_node[MAX_LOCAL_APIC] = {
 [0 ... MAX_LOCAL_APIC-1] = NUMA_NO_NODE
 };
-cpumask_t __read_mostly node_to_cpumask[MAX_NUMNODES];
-
-nodemask_t __read_mostly node_online_map = { { [0] = 1UL } };
-
-bool numa_off;
-s8 acpi_numa = 0;
-
-int srat_disabled(void)
-{
-return numa_off || acpi_numa < 0;
-}
-
-/*
- * Given a shift value, try to populate memnodemap[]
- * Returns :
- * 0 if OK
- * -ENOSPC if memnodmap[] too small (or shift too small)
- * -EINVAL if node overlap or lost ram (shift too big)
- */
-static int __init populate_memnodemap(const struct node *nodes,
-  unsigned int numnodes, unsigned int 
shift,
-  nodeid_t *nodeids)
-{
-unsigned long spdx, epdx;
-int i, res = -EINVAL;
-
-memset(memnodemap, NUMA_NO_NODE, memnodemapsize * sizeof(*memnodemap));
-for ( i = 0; i < numnodes; i++ )
-{
-spdx = paddr_to_pdx(nodes[i].start);
-epdx = paddr_to_pdx(nodes[i].end - 1) + 1;
-if ( spdx >= epdx )
-continue;
-if ( (epdx >> shift) >= memnodemapsize )
-return -ENOSPC;
-do {
-if ( memnodemap[spdx >> shift] != NUMA_NO_NODE )
-return -EINVAL;
-
-if ( !nodeids )
-memnodemap[spdx >> shift] = i;
-else
-memnodemap[spdx >> shift] = nodeids[i];
-
-spdx += (1UL << shift);
-} while ( spdx < epdx );
-res = 0;
-}
-
-return res;
-}
-
-static int __init allocate_cachealigned_memnodemap(void)
-{
-unsigned long size = PFN_UP(memnodemapsize * sizeof(*memnodemap));
-unsigned long mfn = alloc_boot_pages(size, 1);
-
-if ( !mfn )
-{
-printk(KERN_ERR
-   "NUMA: Unable to allocate Memory to Node hash map\n");
-memnodemapsize = 0;
-return -ENOMEM;
-}
-
-memnodemap = mfn_to_virt(mfn);
-mfn <<= PAGE_SHIFT;
-size <<= PAGE_SHIFT;
-printk(KERN_DEBUG "NUMA: Allocated memnodemap from %lx - %lx\n",
-   mfn, mfn + size);
-memnodemapsize = size / sizeof(*memnodemap);
-
-return 0;
-}
-
-/*
- * The LSB of all start and end addresses in the node map is the