Re: [OMPI users] Valgrind Functionality
Thanks to all who gave their input. After upgrading to the latest svn version all is well now. Tom --- On Sun, 7/13/08, Shiqing Fan <f...@hlrs.de> wrote: From: Shiqing Fan <f...@hlrs.de> Subject: Re: [OMPI users] Valgrind Functionality To: rarebit...@yahoo.com, "Open MPI Users" <us...@open-mpi.org> Cc: "Jeff Squyres" <jsquy...@cisco.com>, "Rainer Keller" <kel...@hlrs.de> List-Post: users@lists.open-mpi.org Date: Sunday, July 13, 2008, 11:06 AM Hello Tom, There was a bug in the configure.m4 of memchecker/valgrind, but it shouldn't happen for revision 18303. Now it fixed with r18899. Could you please svn update your source, and try again? Thanks, Shiqing Tom Riddle wrote: > Thanks, I've gotten to the point of building and running branch/1.3. > However when I attempt to run my simple osu_latency test I get the > following runtime error now, which I hadn't before on a repo pull of > trunk/openmpi-1.3a1r18303 > > opal_memchecker_base_select failed > --> Returned value -13 instead od OPAL_SUCCESS > > I checked the ompi_info and see the following memchecker > MCA memchecker: parameter "memchecker" (current value: > Default selection set of components for the memchecker framework > > so that looks ok, here is my configure > > $ ../configure --prefix=/opt/wkspace/openmpi-1.3 CC=gcc CXX=g++ > --disable-mpi-f77 --enable-debug --enable-memchecker > --with-psm=/usr/include --with-valgrind=/opt/wkspace/valgrind-3.3.0/ > > I think I'm close, TIA Tom > > --- On *Sun, 7/13/08, Jeff Squyres /<jsquy...@cisco.com>/* wrote: > > From: Jeff Squyres <jsquy...@cisco.com> > Subject: Re: [OMPI users] Valgrind Functionality > To: rarebit...@yahoo.com, "Open MPI Users" <us...@open-mpi.org> > Cc: "Ashley Pittman" <apitt...@concurrent-thinking.com> > Date: Sunday, July 13, 2008, 6:16 AM > > On Jul 13, 2008, at 9:11 AM, Tom Riddle wrote: > > > Does anyone know if this feature has been incorporated yet? I did a > > ./configure --help but do not see the enable-ptmalloc2-internal > > option. > > > > - The ptmalloc2 memory manager component is now by default built as > > a standalone library named libopenmpi-malloc. Users wanting to > > use leave_pinned with ptmalloc2 will now need to link the > library > > into their application explicitly. All other users will use the > > libc-provided allocator instead of Open MPI's ptmalloc2. This > > change > > may be overriden with the configure option enable-ptmalloc2-internal > > --> Expected: 1.3 > > This is on the trunk/v1.3 branch, yes. > > The default in v1.3 is that ptmalloc2 is *not* built into libopen- > pal. This is different than v1.2, where ptmalloc2 *was* included in > libopen-pal unless you specified --disable-memory-manager. > > -- > Jeff Squyres > Cisco Systems > > > > > ___ > users mailing list > us...@open-mpi.org > http://www.open-mpi.org/mailman/listinfo.cgi/users -- -- Shiqing Fan http://www.hlrs.de/people/fan High Performance ComputingTel.: +49 711 685 87234 Center Stuttgart (HLRS)Fax.: +49 711 685 65832 POSTAL:Nobelstrasse 19email: f...@hlrs.de ACTUAL:Allmandring 30 70569 Stuttgart
Re: [OMPI users] Valgrind Functionality
On Jul 14, 2008, at 7:07 AM, Ashley Pittman wrote: The default in v1.3 is that ptmalloc2 is *not* built into libopen- pal. This is different than v1.2, where ptmalloc2 *was* included in libopen-pal unless you specified --disable-memory-manager. Thank you for clearing that ip Jeff, what is the cost of using this option. The comments in the code led me to believe this was more to do with pinning memory than anything else? Would it be advisable to add a mpicc option to enable and disable linking this library, with 1.2.6 I was sucesfully able to compile and run a application without it by simply changing the gcc compile line. FWIW, we have an open ticket to document this stuff before the v1.3 release. :-) Here's the scoop: - setting "mpi_leave_pinned" to true does a few things; see http://www.open-mpi.org/faq/?category=openfabrics#large-message-leave-pinned for details - we decided that ptmalloc2 is more trouble than it's worth; it causes real problems in some applications - but some applications definitely do benefit from leave_pinned behavior - some kind of memory manager is *required* in OMPI for leave_pinned behavior to work (i.e., OMPI *has* to have visibility to when memory is returned to the OS so that we know to invalidate relevant entries in our internal registration cache) - there's another option, however: simply just don't return memory to the OS (via a mallopt hint). This avoids the problem altogether. - it is important to note that OMPI's ptmalloc2 was configured essentially to do this anyway (rarely to return memory to the OS), and we haven't seen complaints about this effect. - so for v1.3, we have changed the default behavior: - when transports that benefits from leave_pinned behavior are used in a job, mpi_leave_pinned will automatically enable itself - OMPI's internal ptmalloc2 library will now be a separate library (-lopenmpi-malloc) and it will *NOT* be linked in by default. If you want to use it, you need to add -lopenmpi-malloc to your wrapper compiler command line. - if ptmalloc2 is not available and a transport that would benefit from leave_pinned is used, we'll use the mallopt hint to disable memory from being returned to the OS - if mallopt fails for some reason, we'll disable leave_pinned Note that the "automatically enable itself" part isn't enabled yet; that's still on my to-do list. It'll be done by v1.3. ptmalloc2 as a separate library is already done. What this means is quite simple: - if you run with openfabrics networks, mpi_leave_pinned will now be the default, but through the mallopt mechanism, not ptmalloc2 - if you run with openfabrics and want to use ptmalloc2, add -lopenmpi- malloc to your link command line - if you do not run with openfabrics networks, you're not really affected (aside from now returning memory to the OS via the normal libc memory allocator) -- Jeff Squyres Cisco Systems
Re: [OMPI users] Valgrind Functionality
On Sun, 2008-07-13 at 09:16 -0400, Jeff Squyres wrote: > On Jul 13, 2008, at 9:11 AM, Tom Riddle wrote: > > > Does anyone know if this feature has been incorporated yet? I did a > > ./configure --help but do not see the enable-ptmalloc2-internal > > option. > > > > - The ptmalloc2 memory manager component is now by default built as > > a standalone library named libopenmpi-malloc. Users wanting to > > use leave_pinned with ptmalloc2 will now need to link the library > > into their application explicitly. All other users will use the > > libc-provided allocator instead of Open MPI's ptmalloc2. This > > change > > may be overriden with the configure option enable-ptmalloc2-internal > > --> Expected: 1.3 > > This is on the trunk/v1.3 branch, yes. > > The default in v1.3 is that ptmalloc2 is *not* built into libopen- > pal. This is different than v1.2, where ptmalloc2 *was* included in > libopen-pal unless you specified --disable-memory-manager. Thank you for clearing that ip Jeff, what is the cost of using this option. The comments in the code led me to believe this was more to do with pinning memory than anything else? Would it be advisable to add a mpicc option to enable and disable linking this library, with 1.2.6 I was sucesfully able to compile and run a application without it by simply changing the gcc compile line. Ashley,
Re: [OMPI users] Valgrind Functionality
Hello Tom,
There was a bug in the configure.m4 of memchecker/valgrind, but it
shouldn't happen for revision 18303. Now it fixed with r18899. Could
you please svn update your source, and try again?
Thanks,
Shiqing
Tom Riddle wrote:
Thanks, I've gotten to the point of building and running branch/1.3.
However when I attempt to run my simple osu_latency test I get the
following runtime error now, which I hadn't before on a repo pull of
trunk/openmpi-1.3a1r18303
opal_memchecker_base_select failed
--> Returned value -13 instead od OPAL_SUCCESS
I checked the ompi_info and see the following memchecker
MCA memchecker: parameter "memchecker" (current value:
Default selection set of components for the memchecker framework
so that looks ok, here is my configure
$ ../configure --prefix=/opt/wkspace/openmpi-1.3 CC=gcc CXX=g++
--disable-mpi-f77 --enable-debug --enable-memchecker
--with-psm=/usr/include --with-valgrind=/opt/wkspace/valgrind-3.3.0/
I think I'm close, TIA Tom
--- On *Sun, 7/13/08, Jeff Squyres /<jsquy...@cisco.com>/* wrote:
From: Jeff Squyres <jsquy...@cisco.com>
Subject: Re: [OMPI users] Valgrind Functionality
To: rarebit...@yahoo.com, "Open MPI Users" <us...@open-mpi.org>
Cc: "Ashley Pittman" <apitt...@concurrent-thinking.com>
Date: Sunday, July 13, 2008, 6:16 AM
On Jul 13, 2008, at 9:11 AM, Tom Riddle wrote:
> Does anyone know if this feature has been incorporated yet? I did a
> ./configure --help but do not see the enable-ptmalloc2-internal
> option.
>
> - The ptmalloc2 memory manager component is now by default built as
> a standalone library named libopenmpi-malloc. Users wanting to
> use leave_pinned with ptmalloc2 will now need to link the
library
> into their application explicitly. All other users will use the
> libc-provided allocator instead of Open MPI's ptmalloc2. This
> change
> may be overriden with the configure option enable-ptmalloc2-internal
> --> Expected: 1.3
This is on the trunk/v1.3 branch, yes.
The default in v1.3 is that ptmalloc2 is *not* built into libopen-
pal. This is different than v1.2, where ptmalloc2 *was* included in
libopen-pal unless you specified --disable-memory-manager.
--
Jeff Squyres
Cisco Systems
___
users mailing list
us...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/users
--
--
Shiqing Fan
http://www.hlrs.de/people/fan
High Performance ComputingTel.: +49 711 685 87234
Center Stuttgart (HLRS)Fax.: +49 711 685 65832
POSTAL:Nobelstrasse 19email: f...@hlrs.de
ACTUAL:Allmandring 30
70569 Stuttgart
Re: [OMPI users] Valgrind Functionality
Thanks, I've gotten to the point of building and running branch/1.3. However when I attempt to run my simple osu_latency test I get the following runtime error now, which I hadn't before on a repo pull of trunk/openmpi-1.3a1r18303 opal_memchecker_base_select failed --> Returned value -13 instead od OPAL_SUCCESS I checked the ompi_info and see the following memchecker MCA memchecker: parameter "memchecker" (current value: Default selection set of components for the memchecker framework so that looks ok, here is my configure $ ../configure --prefix=/opt/wkspace/openmpi-1.3 CC=gcc CXX=g++ --disable-mpi-f77 --enable-debug --enable-memchecker --with-psm=/usr/include --with-valgrind=/opt/wkspace/valgrind-3.3.0/ I think I'm close, TIA Tom --- On Sun, 7/13/08, Jeff Squyres <jsquy...@cisco.com> wrote: From: Jeff Squyres <jsquy...@cisco.com> Subject: Re: [OMPI users] Valgrind Functionality To: rarebit...@yahoo.com, "Open MPI Users" <us...@open-mpi.org> Cc: "Ashley Pittman" <apitt...@concurrent-thinking.com> List-Post: users@lists.open-mpi.org Date: Sunday, July 13, 2008, 6:16 AM On Jul 13, 2008, at 9:11 AM, Tom Riddle wrote: > Does anyone know if this feature has been incorporated yet? I did a > ./configure --help but do not see the enable-ptmalloc2-internal > option. > > - The ptmalloc2 memory manager component is now by default built as > a standalone library named libopenmpi-malloc. Users wanting to > use leave_pinned with ptmalloc2 will now need to link the library > into their application explicitly. All other users will use the > libc-provided allocator instead of Open MPI's ptmalloc2. This > change > may be overriden with the configure option enable-ptmalloc2-internal > --> Expected: 1.3 This is on the trunk/v1.3 branch, yes. The default in v1.3 is that ptmalloc2 is *not* built into libopen- pal. This is different than v1.2, where ptmalloc2 *was* included in libopen-pal unless you specified --disable-memory-manager. -- Jeff Squyres Cisco Systems
Re: [OMPI users] Valgrind Functionality
On Jul 13, 2008, at 9:11 AM, Tom Riddle wrote: Does anyone know if this feature has been incorporated yet? I did a ./configure --help but do not see the enable-ptmalloc2-internal option. - The ptmalloc2 memory manager component is now by default built as a standalone library named libopenmpi-malloc. Users wanting to use leave_pinned with ptmalloc2 will now need to link the library into their application explicitly. All other users will use the libc-provided allocator instead of Open MPI's ptmalloc2. This change may be overriden with the configure option enable-ptmalloc2-internal --> Expected: 1.3 This is on the trunk/v1.3 branch, yes. The default in v1.3 is that ptmalloc2 is *not* built into libopen- pal. This is different than v1.2, where ptmalloc2 *was* included in libopen-pal unless you specified --disable-memory-manager. -- Jeff Squyres Cisco Systems
Re: [OMPI users] Valgrind Functionality
Does anyone know if this feature has been incorporated yet? I did a ./configure --help but do not see the enable-ptmalloc2-internal option. - The ptmalloc2 memory manager component is now by default built as a standalone library named libopenmpi-malloc. Users wanting to use leave_pinned with ptmalloc2 will now need to link the library into their application explicitly. All other users will use the libc-provided allocator instead of Open MPI's ptmalloc2. This change may be overriden with the configure option enable-ptmalloc2-internal --> Expected: 1.3 --- On Fri, 7/11/08, Ashley Pittman <apitt...@concurrent-thinking.com> wrote: From: Ashley Pittman <apitt...@concurrent-thinking.com> Subject: Re: [OMPI users] Valgrind Functionality To: "Tom Riddle" <rarebit...@yahoo.com> Cc: "Open MPI Users" <us...@open-mpi.org> List-Post: users@lists.open-mpi.org Date: Friday, July 11, 2008, 8:58 AM On Tue, 2008-07-08 at 18:01 -0700, Tom Riddle wrote: > Thanks Ashley, after going through your suggestions we tried our test > with valgrind 3.3.0 and with glibc-devel-2.5-18.el5_1.1, both exhibit > the same results. A simple non-MPI test prog however returns expected > responses, so valgrind itself look ok. We then checked that the same > (shared) libc gets linked in both the MPI and non-MPI cases, adding > -pthread to the cc command line yields the same result, the only > difference it appears is the open mpi libraries. > > Now mpicc links against libopen-pal which defines malloc for it's own > purposes. The big difference seems to be that libopen-pal.so is > providing its own malloc replacement This will be the problem, I've tested on a openmpi (1.2.6) machine here and I see exactly the same behaviour as you. I re-compiled the application without libopen-pal and valgrind works as expected. To do this I used mpicc -show to see what compile line it was using and ran the command myself without the -lopen-pal option. This clearly isn't a acceptable long-term solution but might help you in the short term. I'm a MPI expert but work on a different MPI to openmpi normally, I have however done a lot of work with Valgrind on different platforms so pick up questions related to it here. I think this problem is going to need input from one of the openmpi guys... The problem seems to be the presence of malloc() and free() functions in the libopen-pal library is preventing valgrind from intercepting these functions in glibc and hence dramatically reducing the benefit which valgrind brings. Ashley Pittman.
Re: [OMPI users] Valgrind Functionality
On Tue, 2008-07-08 at 18:01 -0700, Tom Riddle wrote: > Thanks Ashley, after going through your suggestions we tried our test > with valgrind 3.3.0 and with glibc-devel-2.5-18.el5_1.1, both exhibit > the same results. A simple non-MPI test prog however returns expected > responses, so valgrind itself look ok. We then checked that the same > (shared) libc gets linked in both the MPI and non-MPI cases, adding > -pthread to the cc command line yields the same result, the only > difference it appears is the open mpi libraries. > > Now mpicc links against libopen-pal which defines malloc for it's own > purposes. The big difference seems to be that libopen-pal.so is > providing its own malloc replacement This will be the problem, I've tested on a openmpi (1.2.6) machine here and I see exactly the same behaviour as you. I re-compiled the application without libopen-pal and valgrind works as expected. To do this I used mpicc -show to see what compile line it was using and ran the command myself without the -lopen-pal option. This clearly isn't a acceptable long-term solution but might help you in the short term. I'm a MPI expert but work on a different MPI to openmpi normally, I have however done a lot of work with Valgrind on different platforms so pick up questions related to it here. I think this problem is going to need input from one of the openmpi guys... The problem seems to be the presence of malloc() and free() functions in the libopen-pal library is preventing valgrind from intercepting these functions in glibc and hence dramatically reducing the benefit which valgrind brings. Ashley Pittman.
Re: [OMPI users] Valgrind Functionality
Thanks Ashley, after going through your suggestions we tried our test with valgrind 3.3.0 and with glibc-devel-2.5-18.el5_1.1, both exhibit the same results. A simple non-MPI test prog however returns expected responses, so valgrind itself look ok. We then checked that the same (shared) libc gets linked in both the MPI and non-MPI cases, adding -pthread to the cc command line yields the same result, the only difference it appears is the open mpi libraries. Now mpicc links against libopen-pal which defines malloc for it's own purposes. The big difference seems to be that libopen-pal.so is providing its own malloc replacement Is there perhaps something I have missed in runtime? I have included the ompi_info just in case. Thanks in advance. Tom --- On Tue, 7/8/08, Ashley Pittman <apitt...@concurrent-thinking.com> wrote: From: Ashley Pittman <apitt...@concurrent-thinking.com> Subject: Re: [OMPI users] Valgrind Functionality To: rarebit...@yahoo.com, "Open MPI Users" <us...@open-mpi.org> List-Post: users@lists.open-mpi.org Date: Tuesday, July 8, 2008, 2:05 AM On Mon, 2008-07-07 at 19:09 -0700, Tom Riddle wrote: > > I was attempting to get valgrind working with a simple MPI app > (osu_latency) on OpenMPI. While it appears to report uninitialized > values it fails to report any mallocs or frees that have been > conducted. The normal reason for this is either using static applications or having a very stripped glibc. It doesn't appear you've done the former as you are linking in libpthread but the latter is a possibility, you might benefit from installing the glibc-devel package. I don't recalled RHEL being the worst offenders at stripping libc however. > I am using RHEL 5, gcc 4.2.3 and a drop from the repo labeled > openmpi-1.3a1r18303. configured with > > $ ../configure --prefix=/opt/wkspace/openmpi-1.3a1r18303 CC=gcc > CXX=g++ --disable-mpi-f77 --enable-debug --enable-memchecker > --with-psm=/usr/include --with-valgrind=/opt/wkspace/valgrind-3.3.0/ > As the FAQ's suggest I am running a later version of valgrind, > enabling the memchecker and debug. I tested a slightly modified > osu_latency test which has a simple char buffer malloc and free but > the valgrind summary shows no malloc/free activity whatsoever. This is > running on a dual node system using Infinipath HCAs. Here is a > trimmed output. Although you configured openmpi with what appears to be valgrind 3.3.0 the version of valgrind you are using is 3.2.1, perhaps you want to specify the full path of valgrind on the mpirun command line? > [tom@lab01 ~]$ mpirun --mca pml cm -np 2 --hostfile my_hostfile > valgrind ./osu_latency1 > ==17839== Memcheck, a memory error detector. > ==17839== Copyright (C) 2002-2006, and GNU GPL'd, by Julian Seward et > al. > ==17839== Using LibVEX rev 1658, a library for dynamic binary > translation. > ==17839== Copyright (C) 2004-2006, and GNU GPL'd, by OpenWorks LLP. > ==17839== Using valgrind-3.2.1, a dynamic binary instrumentation > framework. > ==17839== Copyright (C) 2000-2006, and GNU GPL'd, by Julian Seward et > al. > ==17839== For more details, rerun with: -v Ashley Pittman. ompi_info.out.bz2 Description: Binary data
Re: [OMPI users] Valgrind Functionality
On Mon, 2008-07-07 at 19:09 -0700, Tom Riddle wrote: > > I was attempting to get valgrind working with a simple MPI app > (osu_latency) on OpenMPI. While it appears to report uninitialized > values it fails to report any mallocs or frees that have been > conducted. The normal reason for this is either using static applications or having a very stripped glibc. It doesn't appear you've done the former as you are linking in libpthread but the latter is a possibility, you might benefit from installing the glibc-devel package. I don't recalled RHEL being the worst offenders at stripping libc however. > I am using RHEL 5, gcc 4.2.3 and a drop from the repo labeled > openmpi-1.3a1r18303. configured with > > $ ../configure --prefix=/opt/wkspace/openmpi-1.3a1r18303 CC=gcc > CXX=g++ --disable-mpi-f77 --enable-debug --enable-memchecker > --with-psm=/usr/include --with-valgrind=/opt/wkspace/valgrind-3.3.0/ > As the FAQ's suggest I am running a later version of valgrind, > enabling the memchecker and debug. I tested a slightly modified > osu_latency test which has a simple char buffer malloc and free but > the valgrind summary shows no malloc/free activity whatsoever. This is > running on a dual node system using Infinipath HCAs. Here is a > trimmed output. Although you configured openmpi with what appears to be valgrind 3.3.0 the version of valgrind you are using is 3.2.1, perhaps you want to specify the full path of valgrind on the mpirun command line? > [tom@lab01 ~]$ mpirun --mca pml cm -np 2 --hostfile my_hostfile > valgrind ./osu_latency1 > ==17839== Memcheck, a memory error detector. > ==17839== Copyright (C) 2002-2006, and GNU GPL'd, by Julian Seward et > al. > ==17839== Using LibVEX rev 1658, a library for dynamic binary > translation. > ==17839== Copyright (C) 2004-2006, and GNU GPL'd, by OpenWorks LLP. > ==17839== Using valgrind-3.2.1, a dynamic binary instrumentation > framework. > ==17839== Copyright (C) 2000-2006, and GNU GPL'd, by Julian Seward et > al. > ==17839== For more details, rerun with: -v Ashley Pittman.
[OMPI users] Valgrind Functionality
Hi, I was attempting to get valgrind working with a simple MPI app (osu_latency) on OpenMPI. While it appears to report uninitialized values it fails to report any mallocs or frees that have been conducted. I am using RHEL 5, gcc 4.2.3 and a drop from the repo labeled openmpi-1.3a1r18303. configured with $ ../configure --prefix=/opt/wkspace/openmpi-1.3a1r18303 CC=gcc CXX=g++ --disable-mpi-f77 --enable-debug --enable-memchecker --with-psm=/usr/include --with-valgrind=/opt/wkspace/valgrind-3.3.0/ As the FAQ's suggest I am running a later version of valgrind, enabling the memchecker and debug. I tested a slightly modified osu_latency test which has a simple char buffer malloc and free but the valgrind summary shows no malloc/free activity whatsoever. This is running on a dual node system using Infinipath HCAs. Here is a trimmed output. [tom@lab01 ~]$ mpirun --mca pml cm -np 2 --hostfile my_hostfile valgrind ./osu_latency1 ==17839== Memcheck, a memory error detector. ==17839== Copyright (C) 2002-2006, and GNU GPL'd, by Julian Seward et al. ==17839== Using LibVEX rev 1658, a library for dynamic binary translation. ==17839== Copyright (C) 2004-2006, and GNU GPL'd, by OpenWorks LLP. ==17839== Using valgrind-3.2.1, a dynamic binary instrumentation framework. ==17839== Copyright (C) 2000-2006, and GNU GPL'd, by Julian Seward et al. ==17839== For more details, rerun with: -v ==17839== ==17823== Memcheck, a memory error detector. ==17823== Copyright (C) 2002-2006, and GNU GPL'd, by Julian Seward et al. ==17823== Using LibVEX rev 1658, a library for dynamic binary translation. ==17823== Copyright (C) 2004-2006, and GNU GPL'd, by OpenWorks LLP. ==17823== Using valgrind-3.2.1, a dynamic binary instrumentation framework. ==17823== Copyright (C) 2000-2006, and GNU GPL'd, by Julian Seward et al. ==17823== For more details, rerun with: -v ==17823== ==17839== Syscall param write(buf) points to uninitialised byte(s) ==17839== at 0x3DD8C0CAA0: __write_nocancel (in /lib64/libpthread-2.5.so) ==17839== by 0x7C283E8: ipath_userinit (ipath_proto.c:191) ==17839== by 0x7AFCDF4: psmi_port_open (psm_port.c:116) ==17839== by 0x7AFE1FB: psm_ep_open (psm_ep.c:535) ==17839== by 0x78E842C: ompi_mtl_psm_module_init (mtl_psm.c:108) ==17839== by 0x78E9137: ompi_mtl_psm_component_init (mtl_psm_component.c:125) ==17839== by 0x4CE32D4: ompi_mtl_base_select (mtl_base_component.c:105) ==17839== by 0x76D9EDE: mca_pml_cm_component_init (pml_cm_component.c:145) ==17839== by 0x4CE7425: mca_pml_base_select (pml_base_select.c:122) ==17839== by 0x4C50ED2: ompi_mpi_init (ompi_mpi_init.c:398) ==17839== by 0x4C93EE4: PMPI_Init (pinit.c:88) ==17839== by 0x400D78: main (in /home/tomr/osu_latency1) ==17839== Address 0x7FEFFE9D4 is on thread 1's stack ==17839== ==17823== ==17823== Syscall param sched_setaffinity(mask) points to uninitialised byte(s) ==17823== at 0x3EA36B8AD0: sched_setaffinity@@GLIBC_2.3.4 (in /lib64/libc-2.5.so) ==17823== by 0x7C2775E: ipath_setaffinity (ipath_proto.c:160) ==17823== by 0x7C28400: ipath_userinit (ipath_proto.c:198) ==17823== by 0x7AFCDF4: psmi_port_open (psm_port.c:116) ==17823== by 0x7AFE1FB: psm_ep_open (psm_ep.c:535) ==17823== by 0x78E842C: ompi_mtl_psm_module_init (mtl_psm.c:108) ==17823== by 0x78E9137: ompi_mtl_psm_component_init (mtl_psm_component.c:125) ==17823== by 0x4CE32D4: ompi_mtl_base_select (mtl_base_component.c:105) ==17823== by 0x76D9EDE: mca_pml_cm_component_init (pml_cm_component.c:145) ==17823== by 0x4CE7425: mca_pml_base_select (pml_base_select.c:122) ==17823== by 0x4C50ED2: ompi_mpi_init (ompi_mpi_init.c:398) ==17823== by 0x4C93EE4: PMPI_Init (pinit.c:88) ==17823== Address 0x7FEFFEC30 is on thread 1's stack ==17823== # OSU MPI Latency Test v3.0 # Size Latency (us) 384 61.78 ==17839== ==17839== ERROR SUMMARY: 9 errors from 9 contexts (suppressed: 5 from 1) ==17839== malloc/free: in use at exit: 0 bytes in 0 blocks. ==17839== malloc/free: 0 allocs, 0 frees, 0 bytes allocated. ==17839== For counts of detected errors, rerun with: -v ==17839== All heap blocks were freed -- no leaks are possible. ==17823== ==17823== ERROR SUMMARY: 9 errors from 9 contexts (suppressed: 5 from 1) ==17823== malloc/free: in use at exit: 0 bytes in 0 blocks. ==17823== malloc/free: 0 allocs, 0 frees, 0 bytes allocated. ==17823== For counts of detected errors, rerun with: -v ==17823== All heap blocks were freed -- no leaks are possible. Hopefully this was enough info to garner a bit of input, Thanks in advance. Tom
