Re: [OMPI devel] [OMPI svn-full] svn:open-mpi r20003 (Solaris malloc.h issue)
On Thu, Apr/16/2009 06:18:27PM, George Bosilca wrote: > I don't think this is correct. We are not supposed to include > anything before the ompi_config.h. Moreover, in the case we redefine > what malloc is, this will be the only piece of code that will use > the real malloc. > > I wonder why we need to include malloc.h there? We don't allocate > any memory in this file ... Any objections to removing the #include line? -Ethan > > george. > > On Apr 16, 2009, at 18:09 , Ethan Mallove wrote: > >> Hi, >> >> I think I have a better fix for this issue. It is to simply #include >> *before* ompi_config.h. >> >> --- ompi/mca/common/mx/common_mx.c >> +++ ompi/mca/common/mx/common_mx.c >> @@ -19,15 +19,16 @@ >>* $HEADER$ >>*/ >> >> +#ifdef HAVE_MALLOC_H >> +#include >> +#endif >> + >> #include "ompi_config.h" >> #include "orte/util/show_help.h" >> #include "ompi/constants.h" >> #include "common_mx.h" >> >> #include >> -#ifdef HAVE_MALLOC_H >> -#include >> -#endif >> #include "opal/memoryhooks/memory.h" >> #include "opal/mca/base/mca_base_param.h" >> #include "ompi/runtime/params.h" >> >> The reason for doing this is because ompi_config.h (which includes >> ompi_config_bottom.h) #defines "malloc", so we end up with OMPI code >> getting spliced into the Solaris /usr/include/malloc.h code. >> >> Is this fix okay? >> >> -Ethan >> >> On Wed, Dec/10/2008 04:29:31PM, Ethan Mallove wrote: >>> Hi Patrick, >>> >>> r20003 seems to break MX support on Solaris. >>> >>> $ cd ompi/mca/common/mx >>> $ make >>> ... >>> "/usr/include/malloc.h", line 46: syntax error before or at: ( >>> "/usr/include/malloc.h", line 47: syntax error before or at: ( >>> "/usr/include/malloc.h", line 48: syntax error before or at: ( >>> "/usr/include/malloc.h", line 48: cannot have void object: size_t >>> "/usr/include/malloc.h", line 48: identifier redeclared: size_t >>> ... <4000 more lines of compiler errors> ... >>> >>> The below patch makes it so opal/util/malloc.h is used instead of >>> /usr/include/malloc.h and the compiler errors go away. (I also needed >>> to include errno.h.) Would this be okay to do? >>> >>> diff -r 347f52a3713f ompi/mca/common/mx/common_mx.c >>> --- ompi/mca/common/mx/common_mx.c >>> +++ ompi/mca/common/mx/common_mx.c >>> @@ -23,9 +23,8 @@ >>> #include "ompi/constants.h" >>> #include "common_mx.h" >>> >>> -#ifdef HAVE_MALLOC_H >>> -#include >>> -#endif >>> +#include >>> +#include "opal/util/malloc.h" >>> #include "opal/memoryhooks/memory.h" >>> #include "opal/mca/base/mca_base_param.h" >>> #include "ompi/runtime/params.h" >>> >>> I tested the above on Solaris and Linux with SunStudio. >>> >>> Regards, >>> Ethan >>> >>> >>> On Fri, Nov/14/2008 11:17:59PM, patr...@osl.iu.edu wrote: Author: patrick Date: 2008-11-14 23:17:58 EST (Fri, 14 Nov 2008) New Revision: 20003 URL: https://svn.open-mpi.org/trac/ompi/changeset/20003 Log: Define a "fake" mpool to provide a memory release callback for the memory hooks (munmap) and initialize the mallopt component, and nothing else. Use this mpool in the MX common initialization, supporting both BTL and MTL. Automatically set the MX_RCACHE environment variable to enable registration cache in MX. Tested with success for munmap() and large free(). Added: trunk/ompi/mca/mpool/fake/ trunk/ompi/mca/mpool/fake/Makefile.am trunk/ompi/mca/mpool/fake/configure.params trunk/ompi/mca/mpool/fake/mpool_fake.h trunk/ompi/mca/mpool/fake/mpool_fake_component.c trunk/ompi/mca/mpool/fake/mpool_fake_module.c Text files modified: trunk/ompi/mca/common/mx/common_mx.c |56 +++ 1 files changed, 55 insertions(+), 1 deletions(-) Modified: trunk/ompi/mca/common/mx/common_mx.c == --- trunk/ompi/mca/common/mx/common_mx.c (original) +++ trunk/ompi/mca/common/mx/common_mx.c 2008-11-14 23:17:58 EST (Fri, 14 Nov 2008) @@ -9,6 +9,8 @@ * University of Stuttgart. All rights reserved. * Copyright (c) 2004-2006 The Regents of the University of California. * All rights reserved. + * Copyright (c) 2008 Myricom. All rights reserved. + * * $COPYRIGHT$ * * Additional copyrights may follow @@ -21,11 +23,29 @@ #include "ompi/constants.h" #include "common_mx.h" +#ifdef HAVE_MALLOC_H +#include +#endif +#include "opal/memoryhooks/memory.h" +#include "opal/mca/base/mca_base_param.h" +#include "ompi/runtime/params.h" +#include "ompi/mca/mpool/mpool.h" +#include "ompi/mca/mpool/base/base.h" +#include "ompi/mca/mpool/fake/mpool_fake.h" + + +int mx__regcache_clean(void *ptr, size_t size)
Re: [OMPI devel] [OMPI svn-full] svn:open-mpi r20003 (Solaris malloc.h issue)
I don't think this is correct. We are not supposed to include anything
before the ompi_config.h. Moreover, in the case we redefine what
malloc is, this will be the only piece of code that will use the real
malloc.
I wonder why we need to include malloc.h there? We don't allocate any
memory in this file ...
george.
On Apr 16, 2009, at 18:09 , Ethan Mallove wrote:
Hi,
I think I have a better fix for this issue. It is to simply #include
*before* ompi_config.h.
--- ompi/mca/common/mx/common_mx.c
+++ ompi/mca/common/mx/common_mx.c
@@ -19,15 +19,16 @@
* $HEADER$
*/
+#ifdef HAVE_MALLOC_H
+#include
+#endif
+
#include "ompi_config.h"
#include "orte/util/show_help.h"
#include "ompi/constants.h"
#include "common_mx.h"
#include
-#ifdef HAVE_MALLOC_H
-#include
-#endif
#include "opal/memoryhooks/memory.h"
#include "opal/mca/base/mca_base_param.h"
#include "ompi/runtime/params.h"
The reason for doing this is because ompi_config.h (which includes
ompi_config_bottom.h) #defines "malloc", so we end up with OMPI code
getting spliced into the Solaris /usr/include/malloc.h code.
Is this fix okay?
-Ethan
On Wed, Dec/10/2008 04:29:31PM, Ethan Mallove wrote:
Hi Patrick,
r20003 seems to break MX support on Solaris.
$ cd ompi/mca/common/mx
$ make
...
"/usr/include/malloc.h", line 46: syntax error before or at: (
"/usr/include/malloc.h", line 47: syntax error before or at: (
"/usr/include/malloc.h", line 48: syntax error before or at: (
"/usr/include/malloc.h", line 48: cannot have void object: size_t
"/usr/include/malloc.h", line 48: identifier redeclared: size_t
... <4000 more lines of compiler errors> ...
The below patch makes it so opal/util/malloc.h is used instead of
/usr/include/malloc.h and the compiler errors go away. (I also needed
to include errno.h.) Would this be okay to do?
diff -r 347f52a3713f ompi/mca/common/mx/common_mx.c
--- ompi/mca/common/mx/common_mx.c
+++ ompi/mca/common/mx/common_mx.c
@@ -23,9 +23,8 @@
#include "ompi/constants.h"
#include "common_mx.h"
-#ifdef HAVE_MALLOC_H
-#include
-#endif
+#include
+#include "opal/util/malloc.h"
#include "opal/memoryhooks/memory.h"
#include "opal/mca/base/mca_base_param.h"
#include "ompi/runtime/params.h"
I tested the above on Solaris and Linux with SunStudio.
Regards,
Ethan
On Fri, Nov/14/2008 11:17:59PM, patr...@osl.iu.edu wrote:
Author: patrick
Date: 2008-11-14 23:17:58 EST (Fri, 14 Nov 2008)
New Revision: 20003
URL: https://svn.open-mpi.org/trac/ompi/changeset/20003
Log:
Define a "fake" mpool to provide a memory release callback for the
memory hooks (munmap) and initialize the mallopt component, and
nothing else.
Use this mpool in the MX common initialization, supporting both BTL
and MTL. Automatically set the MX_RCACHE environment variable to
enable registration cache in MX.
Tested with success for munmap() and large free().
Added:
trunk/ompi/mca/mpool/fake/
trunk/ompi/mca/mpool/fake/Makefile.am
trunk/ompi/mca/mpool/fake/configure.params
trunk/ompi/mca/mpool/fake/mpool_fake.h
trunk/ompi/mca/mpool/fake/mpool_fake_component.c
trunk/ompi/mca/mpool/fake/mpool_fake_module.c
Text files modified:
trunk/ompi/mca/common/mx/common_mx.c |56
+++
1 files changed, 55 insertions(+), 1 deletions(-)
Modified: trunk/ompi/mca/common/mx/common_mx.c
=
=
=
=
=
=
=
=
=
=
--- trunk/ompi/mca/common/mx/common_mx.c(original)
+++ trunk/ompi/mca/common/mx/common_mx.c 2008-11-14 23:17:58 EST
(Fri, 14 Nov 2008)
@@ -9,6 +9,8 @@
* University of Stuttgart. All rights
reserved.
* Copyright (c) 2004-2006 The Regents of the University of
California.
* All rights reserved.
+ * Copyright (c) 2008 Myricom. All rights reserved.
+ *
* $COPYRIGHT$
*
* Additional copyrights may follow
@@ -21,11 +23,29 @@
#include "ompi/constants.h"
#include "common_mx.h"
+#ifdef HAVE_MALLOC_H
+#include
+#endif
+#include "opal/memoryhooks/memory.h"
+#include "opal/mca/base/mca_base_param.h"
+#include "ompi/runtime/params.h"
+#include "ompi/mca/mpool/mpool.h"
+#include "ompi/mca/mpool/base/base.h"
+#include "ompi/mca/mpool/fake/mpool_fake.h"
+
+
+int mx__regcache_clean(void *ptr, size_t size);
+
static int ompi_common_mx_initialize_ref_cnt = 0;
+static mca_mpool_base_module_t *ompi_common_mx_fake_mpool = 0;
+
int
ompi_common_mx_initialize(void)
{
mx_return_t mx_return;
+struct mca_mpool_base_resources_t mpool_resources;
+int index, value;
+
ompi_common_mx_initialize_ref_cnt++;
if(ompi_common_mx_initialize_ref_cnt == 1) {
@@ -35,7 +55,37 @@
* library does not exit the application.
*/
mx_set_error_handler(MX_ERRORS_RETURN);
-
+
+ /* If we have a memory manager available, and
+ mpi_leave_pinned == -1, then set mpi_leave_pinned to 1.
+
+ We
Re: [OMPI devel] [OMPI svn-full] svn:open-mpi r20003 (Solaris malloc.h issue)
Hi,
I think I have a better fix for this issue. It is to simply #include
*before* ompi_config.h.
--- ompi/mca/common/mx/common_mx.c
+++ ompi/mca/common/mx/common_mx.c
@@ -19,15 +19,16 @@
* $HEADER$
*/
+#ifdef HAVE_MALLOC_H
+#include
+#endif
+
#include "ompi_config.h"
#include "orte/util/show_help.h"
#include "ompi/constants.h"
#include "common_mx.h"
#include
-#ifdef HAVE_MALLOC_H
-#include
-#endif
#include "opal/memoryhooks/memory.h"
#include "opal/mca/base/mca_base_param.h"
#include "ompi/runtime/params.h"
The reason for doing this is because ompi_config.h (which includes
ompi_config_bottom.h) #defines "malloc", so we end up with OMPI code
getting spliced into the Solaris /usr/include/malloc.h code.
Is this fix okay?
-Ethan
On Wed, Dec/10/2008 04:29:31PM, Ethan Mallove wrote:
> Hi Patrick,
>
> r20003 seems to break MX support on Solaris.
>
> $ cd ompi/mca/common/mx
> $ make
> ...
> "/usr/include/malloc.h", line 46: syntax error before or at: (
> "/usr/include/malloc.h", line 47: syntax error before or at: (
> "/usr/include/malloc.h", line 48: syntax error before or at: (
> "/usr/include/malloc.h", line 48: cannot have void object: size_t
> "/usr/include/malloc.h", line 48: identifier redeclared: size_t
> ... <4000 more lines of compiler errors> ...
>
> The below patch makes it so opal/util/malloc.h is used instead of
> /usr/include/malloc.h and the compiler errors go away. (I also needed
> to include errno.h.) Would this be okay to do?
>
> diff -r 347f52a3713f ompi/mca/common/mx/common_mx.c
> --- ompi/mca/common/mx/common_mx.c
> +++ ompi/mca/common/mx/common_mx.c
> @@ -23,9 +23,8 @@
>#include "ompi/constants.h"
>#include "common_mx.h"
>
> -#ifdef HAVE_MALLOC_H
> -#include
> -#endif
> +#include
> +#include "opal/util/malloc.h"
>#include "opal/memoryhooks/memory.h"
>#include "opal/mca/base/mca_base_param.h"
>#include "ompi/runtime/params.h"
>
> I tested the above on Solaris and Linux with SunStudio.
>
> Regards,
> Ethan
>
>
> On Fri, Nov/14/2008 11:17:59PM, patr...@osl.iu.edu wrote:
> > Author: patrick
> > Date: 2008-11-14 23:17:58 EST (Fri, 14 Nov 2008)
> > New Revision: 20003
> > URL: https://svn.open-mpi.org/trac/ompi/changeset/20003
> >
> > Log:
> > Define a "fake" mpool to provide a memory release callback for the
> > memory hooks (munmap) and initialize the mallopt component, and
> > nothing else.
> > Use this mpool in the MX common initialization, supporting both BTL
> > and MTL. Automatically set the MX_RCACHE environment variable to
> > enable registration cache in MX.
> >
> > Tested with success for munmap() and large free().
> >
> >
> > Added:
> >trunk/ompi/mca/mpool/fake/
> >trunk/ompi/mca/mpool/fake/Makefile.am
> >trunk/ompi/mca/mpool/fake/configure.params
> >trunk/ompi/mca/mpool/fake/mpool_fake.h
> >trunk/ompi/mca/mpool/fake/mpool_fake_component.c
> >trunk/ompi/mca/mpool/fake/mpool_fake_module.c
> > Text files modified:
> >trunk/ompi/mca/common/mx/common_mx.c |56
> > +++
> >1 files changed, 55 insertions(+), 1 deletions(-)
> >
> > Modified: trunk/ompi/mca/common/mx/common_mx.c
> > ==
> > --- trunk/ompi/mca/common/mx/common_mx.c(original)
> > +++ trunk/ompi/mca/common/mx/common_mx.c2008-11-14 23:17:58 EST (Fri,
> > 14 Nov 2008)
> > @@ -9,6 +9,8 @@
> > * University of Stuttgart. All rights reserved.
> > * Copyright (c) 2004-2006 The Regents of the University of California.
> > * All rights reserved.
> > + * Copyright (c) 2008 Myricom. All rights reserved.
> > + *
> > * $COPYRIGHT$
> > *
> > * Additional copyrights may follow
> > @@ -21,11 +23,29 @@
> > #include "ompi/constants.h"
> > #include "common_mx.h"
> >
> > +#ifdef HAVE_MALLOC_H
> > +#include
> > +#endif
> > +#include "opal/memoryhooks/memory.h"
> > +#include "opal/mca/base/mca_base_param.h"
> > +#include "ompi/runtime/params.h"
> > +#include "ompi/mca/mpool/mpool.h"
> > +#include "ompi/mca/mpool/base/base.h"
> > +#include "ompi/mca/mpool/fake/mpool_fake.h"
> > +
> > +
> > +int mx__regcache_clean(void *ptr, size_t size);
> > +
> > static int ompi_common_mx_initialize_ref_cnt = 0;
> > +static mca_mpool_base_module_t *ompi_common_mx_fake_mpool = 0;
> > +
> > int
> > ompi_common_mx_initialize(void)
> > {
> > mx_return_t mx_return;
> > +struct mca_mpool_base_resources_t mpool_resources;
> > +int index, value;
> > +
> > ompi_common_mx_initialize_ref_cnt++;
> >
> > if(ompi_common_mx_initialize_ref_cnt == 1) {
> > @@ -35,7 +55,37 @@
> > * library does not exit the application.
> > */
> > mx_set_error_handler(MX_ERRORS_RETURN);
> > -
> > +
> > + /* If we have a memory manager available, and
> > +
Re: [OMPI devel] [OMPI svn-full] svn:open-mpi r20003 (Solaris malloc.h issue)
Hi Patrick,
r20003 seems to break MX support on Solaris.
$ cd ompi/mca/common/mx
$ make
...
"/usr/include/malloc.h", line 46: syntax error before or at: (
"/usr/include/malloc.h", line 47: syntax error before or at: (
"/usr/include/malloc.h", line 48: syntax error before or at: (
"/usr/include/malloc.h", line 48: cannot have void object: size_t
"/usr/include/malloc.h", line 48: identifier redeclared: size_t
... <4000 more lines of compiler errors> ...
The below patch makes it so opal/util/malloc.h is used instead of
/usr/include/malloc.h and the compiler errors go away. (I also needed
to include errno.h.) Would this be okay to do?
diff -r 347f52a3713f ompi/mca/common/mx/common_mx.c
--- ompi/mca/common/mx/common_mx.c
+++ ompi/mca/common/mx/common_mx.c
@@ -23,9 +23,8 @@
#include "ompi/constants.h"
#include "common_mx.h"
-#ifdef HAVE_MALLOC_H
-#include
-#endif
+#include
+#include "opal/util/malloc.h"
#include "opal/memoryhooks/memory.h"
#include "opal/mca/base/mca_base_param.h"
#include "ompi/runtime/params.h"
I tested the above on Solaris and Linux with SunStudio.
Regards,
Ethan
On Fri, Nov/14/2008 11:17:59PM, patr...@osl.iu.edu wrote:
> Author: patrick
> Date: 2008-11-14 23:17:58 EST (Fri, 14 Nov 2008)
> New Revision: 20003
> URL: https://svn.open-mpi.org/trac/ompi/changeset/20003
>
> Log:
> Define a "fake" mpool to provide a memory release callback for the
> memory hooks (munmap) and initialize the mallopt component, and
> nothing else.
> Use this mpool in the MX common initialization, supporting both BTL
> and MTL. Automatically set the MX_RCACHE environment variable to
> enable registration cache in MX.
>
> Tested with success for munmap() and large free().
>
>
> Added:
>trunk/ompi/mca/mpool/fake/
>trunk/ompi/mca/mpool/fake/Makefile.am
>trunk/ompi/mca/mpool/fake/configure.params
>trunk/ompi/mca/mpool/fake/mpool_fake.h
>trunk/ompi/mca/mpool/fake/mpool_fake_component.c
>trunk/ompi/mca/mpool/fake/mpool_fake_module.c
> Text files modified:
>trunk/ompi/mca/common/mx/common_mx.c |56
> +++
>1 files changed, 55 insertions(+), 1 deletions(-)
>
> Modified: trunk/ompi/mca/common/mx/common_mx.c
> ==
> --- trunk/ompi/mca/common/mx/common_mx.c (original)
> +++ trunk/ompi/mca/common/mx/common_mx.c 2008-11-14 23:17:58 EST (Fri,
> 14 Nov 2008)
> @@ -9,6 +9,8 @@
> * University of Stuttgart. All rights reserved.
> * Copyright (c) 2004-2006 The Regents of the University of California.
> * All rights reserved.
> + * Copyright (c) 2008 Myricom. All rights reserved.
> + *
> * $COPYRIGHT$
> *
> * Additional copyrights may follow
> @@ -21,11 +23,29 @@
> #include "ompi/constants.h"
> #include "common_mx.h"
>
> +#ifdef HAVE_MALLOC_H
> +#include
> +#endif
> +#include "opal/memoryhooks/memory.h"
> +#include "opal/mca/base/mca_base_param.h"
> +#include "ompi/runtime/params.h"
> +#include "ompi/mca/mpool/mpool.h"
> +#include "ompi/mca/mpool/base/base.h"
> +#include "ompi/mca/mpool/fake/mpool_fake.h"
> +
> +
> +int mx__regcache_clean(void *ptr, size_t size);
> +
> static int ompi_common_mx_initialize_ref_cnt = 0;
> +static mca_mpool_base_module_t *ompi_common_mx_fake_mpool = 0;
> +
> int
> ompi_common_mx_initialize(void)
> {
> mx_return_t mx_return;
> +struct mca_mpool_base_resources_t mpool_resources;
> +int index, value;
> +
> ompi_common_mx_initialize_ref_cnt++;
>
> if(ompi_common_mx_initialize_ref_cnt == 1) {
> @@ -35,7 +55,37 @@
> * library does not exit the application.
> */
> mx_set_error_handler(MX_ERRORS_RETURN);
> -
> +
> + /* If we have a memory manager available, and
> +mpi_leave_pinned == -1, then set mpi_leave_pinned to 1.
> +
> +We have a memory manager if:
> +- we have both FREE and MUNMAP support
> +- we have MUNMAP support and the linux mallopt */
> + value = opal_mem_hooks_support_level();
> + if (((value & (OPAL_MEMORY_FREE_SUPPORT | OPAL_MEMORY_MUNMAP_SUPPORT))
> + == (OPAL_MEMORY_FREE_SUPPORT | OPAL_MEMORY_MUNMAP_SUPPORT))
> + || ((value & OPAL_MEMORY_MUNMAP_SUPPORT) &&
> + OMPI_MPOOL_BASE_HAVE_LINUX_MALLOPT)) {
> + index = mca_base_param_find("mpi", NULL, "leave_pinned");
> + if (index >= 0)
> +if ((mca_base_param_lookup_int(index, &value) == OPAL_SUCCESS)
> + && (value == -1)) {
> +
> + ompi_mpi_leave_pinned = 1;
> + setenv("MX_RCACHE", "2", 1);
> + mpool_resources.regcache_clean = mx__regcache_clean;
> + ompi_common_mx_fake_mpool =
> + mca_mpool_base_module_create("fake", NULL, &mpool_resources);
> + if (!ompi_common_mx_fake_mpool) {
> + omp
