Re: [OMPI devel] memory manager RFC
On Wed, 4 Jun 2008, Paul H. Hargrove wrote: Brian states This will allow users to turn ptmalloc2 support on/off at application link time instead of MPI compile time. Where I assume "MPI compile time" means when the MPI *implementation* is compiled. Correct. So what about LD_PRELOAD? Can the user defer the decision to use ptmalloc until application launch? If so, this begs the question of an mpirun option to "enable leave_pinned, placing libompi-malloc.so in LD_PRELOAD if required". Can/will/should such an option exist? Such an option can definitely exist. However, there's no way that I can work on this, so it isn't going to happen any time soon. Brian
[OMPI devel] some Makefile.am questions
Hello, perusing the OMPI source tree a bit more, I stumbled over a couple of things: 1) This is from test/Makefile.am: --- snip --- # This should be libsupport.a, not libsupport.la. Automake doesn't # support check_LTLIBRARIES, as technically you have to install a # shared library before you can use it. # check_LIBRARIES = libsupport.a libsupport_a_SOURCES = \ --- snip --- The statement in the comment is not true; Automake supports check_LTLIBRARIES, and they don't have to be installed before use either. What may be confusing is that, by default, check_LTLIBRARIES will be convenience archives rather than shared libraries. If you want to have an uninstalled shared library for testing, then you can use check_LTLIBRARIES = libsupport.la # induce libtool to create a shared library: libsupport_la_LDFLAGS = -rpath /nowhere libsupport_la_SOURCES = \ ... 2) test/dss/ has only a Makefile with 'CC = mpicc' hardcoded. That looks like it won't use the correct (uninstalled) mpicc but requires a prior 'make install'. Not sure whether that's intentional. 3) in opal/mca/paffinity/linux/plpa/src/plpa-taskset/Makefile.am, there are some manual dependencies listed that should not be necessary, as Automake should treat bison/yacc generated files correctly. IOW, the patch below, which simplifies things slightly, should still generate a functional Makefile. If it doesn't, I'd like to hear about it (because that could be an Automake bug). Cheers, Ralf Automake takes care of bison/yacc output file dependencies. Index: opal/mca/paffinity/linux/plpa/src/plpa-taskset/Makefile.am === --- opal/mca/paffinity/linux/plpa/src/plpa-taskset/Makefile.am (revision 18570) +++ opal/mca/paffinity/linux/plpa/src/plpa-taskset/Makefile.am (working copy) @@ -22,11 +22,6 @@ bin_PROGRAMS += plpa-taskset endif -# Force recognition that tokens.h is generated via tokens.c; there's a -# rule to create tokens.c from tokens.y. -tokens.c: tokens.y -tokens.h: tokens.c - # Force parser.c to be re-compiled when tokens.h is re-generated parser.c: tokens.h
Re: [OMPI devel] devel Digest, Vol 1080, Issue 2
On 22:33 Wed 04 Jun , Mukesh K Srivastava wrote:
> I am aware that GCC(v-4.2.4) supports OpenMP, so suggested if OMPI could be
> supported too with GCC releases?
Are you 100% sure about what an MPI implementation is?
> There is no confusion please.
There is confusion thank you very much indeed.
;-)
-Andi
--
Andreas Schäfer
Cluster and Metacomputing Working Group
Friedrich-Schiller-Universität Jena, Germany
PGP/GPG key via keyserver
I'm a bright... http://www.the-brights.net
(\___/)
(+'.'+)
(")_(")
This is Bunny. Copy and paste Bunny into your
signature to help him gain world domination!
pgpovrVsJqhgE.pgp
Description: PGP signature
Re: [OMPI devel] memory manager RFC
Brian states This will allow users to turn ptmalloc2 support on/off at application link time instead of MPI compile time. Where I assume "MPI compile time" means when the MPI *implementation* is compiled. So what about LD_PRELOAD? Can the user defer the decision to use ptmalloc until application launch? If so, this begs the question of an mpirun option to "enable leave_pinned, placing libompi-malloc.so in LD_PRELOAD if required". Can/will/should such an option exist? -Paul Brian W. Barrett wrote: Hi all - Sorry this is so late, but it took a couple of iterations with a couple of people to get right from a technology standpoint. All mistakes in this proposal are my fault. What: Fix the ptmalloc2 problem How: Remove it from the default path When: This weekend? For the 1.3 branch The problem: On Linux today, we by default build a copy of ptmalloc2 into libopen-pal.so so that RDMA networks can get better bandwidth using leave_pinned. Normally users don't use or need leave_pinned, but we need to have it available for benchmarks and the few apps that gain by having the more independent-ish progress. However, by having it there, we're screwing with the memory manager, which has a number of bad side effects. First, it can cause numerous crashes if the user is providing his/her own allocator. Second, there is growing evidence that the ptmalloc2 in Open MPI has an evil corner case we can't pinn down that causes explosive growth in memory utilization. The proposal: Remove ptmalloc2 from libopen-pal.so and make it a standalone library (for forward compatibility, currently called libompi-malloc.so), which the user can explicitly link in. This will allow users to turn ptmalloc2 support on/off at application link time instead of MPI compile time. Given the limited number of leave_pinned users, this seems to be a good compromise for the near-term between greater stability for most users and fast performance for power users. The mallopt() solution, which means free() never gives memory back to the OS (but does reuse it), which works well for benchmarks, will still be available at all times. The work: Some autoconf magic to move most (but not all -- in particular the munmap() support) of the ptmalloc2 component into its own library. This is extremely low risk, and actually lowers the risk of Open MPI breaking by removing code from the critical path. There will also be a small number of enhancements to the mpool base code to better detect situations where leave_pinned is used by we can't sense giving memory back to the OS. I'd like this for 1.3, as we're running into more and more situations where this code isn't working. Also, the lone supporter of the ptmallco2 code (me) doesn't want to do it anymore and removing the code from the critical path will lower the workload of me (ie, the retired guy who's doing this for fun). If you have objections, please let me know before Friday. I'd like to commit these changes to the trunk this weekend. Brian ___ devel mailing list de...@open-mpi.org http://www.open-mpi.org/mailman/listinfo.cgi/devel -- Paul H. Hargrove phhargr...@lbl.gov Future Technologies Group HPC Research Department Tel: +1-510-495-2352 Lawrence Berkeley National Laboratory Fax: +1-510-486-6900
Re: [OMPI devel] memory manager RFC
Sounds good to me. Thanks Brian! On Jun 3, 2008, at 12:04 PM, Brian W. Barrett wrote: Hi all - Sorry this is so late, but it took a couple of iterations with a couple of people to get right from a technology standpoint. All mistakes in this proposal are my fault. What: Fix the ptmalloc2 problem How: Remove it from the default path When: This weekend? For the 1.3 branch The problem: On Linux today, we by default build a copy of ptmalloc2 into libopen-pal.so so that RDMA networks can get better bandwidth using leave_pinned. Normally users don't use or need leave_pinned, but we need to have it available for benchmarks and the few apps that gain by having the more independent-ish progress. However, by having it there, we're screwing with the memory manager, which has a number of bad side effects. First, it can cause numerous crashes if the user is providing his/ her own allocator. Second, there is growing evidence that the ptmalloc2 in Open MPI has an evil corner case we can't pinn down that causes explosive growth in memory utilization. The proposal: Remove ptmalloc2 from libopen-pal.so and make it a standalone library (for forward compatibility, currently called libompi-malloc.so), which the user can explicitly link in. This will allow users to turn ptmalloc2 support on/off at application link time instead of MPI compile time. Given the limited number of leave_pinned users, this seems to be a good compromise for the near-term between greater stability for most users and fast performance for power users. The mallopt() solution, which means free() never gives memory back to the OS (but does reuse it), which works well for benchmarks, will still be available at all times. The work: Some autoconf magic to move most (but not all -- in particular the munmap() support) of the ptmalloc2 component into its own library. This is extremely low risk, and actually lowers the risk of Open MPI breaking by removing code from the critical path. There will also be a small number of enhancements to the mpool base code to better detect situations where leave_pinned is used by we can't sense giving memory back to the OS. I'd like this for 1.3, as we're running into more and more situations where this code isn't working. Also, the lone supporter of the ptmallco2 code (me) doesn't want to do it anymore and removing the code from the critical path will lower the workload of me (ie, the retired guy who's doing this for fun). If you have objections, please let me know before Friday. I'd like to commit these changes to the trunk this weekend. Brian ___ devel mailing list de...@open-mpi.org http://www.open-mpi.org/mailman/listinfo.cgi/devel -- Jeff Squyres Cisco Systems
Re: [OMPI devel] openmpi-1.2.6 and pvfs-2.7.0+ Support
Just to clarify -- are you saying that the function signature for PVFS_sys_create() has changed over time and that in some versions it takes 6 parameters, and in other versions it takes 7 parameters? (I know nothing about PVFS) Thanks! On May 30, 2008, at 6:07 PM, Joseph Bane wrote: Hi all. I found a small issue compiling openmpi-1.2.6 with pvfs-2.7.0 support. There is a new layout attribute for "PVFS_sys_create" calls and simply passing a NULL in place of it for the "fake_an_open" function (io_romio_ad_pvfs2_open.c) takes care of the issue. I found this solution looking at MPICH2 source awhile ago, btw. For instance, after configuring openmpi with pvfs2 support (configure --prefix=/usr/local --with-io-romio-flags="--with-file- system=pvfs2+ufs+nfs") and then running "make all", the following errors appear: libtool: compile: gcc -DHAVE_CONFIG_H -I. -I../../adio/include - DOMPI_BUILDING=1 -I/usr/src/openmpi-1.2.6/ompi/mca/io/romio/ romio/../../../../.. -I/usr/src/openmpi-1.2.6/ompi/mca/io/romio/ romio/../../../../../opal/include -I../../../../../../../opal/ include -I../../../../../../../ompi/include -I/usr/src/openmpi-1.2.6/ ompi/mca/io/romio/romio/include -I/usr/src/openmpi-1.2.6/ompi/mca/io/ romio/romio/adio/include -O3 -DNDEBUG -finline-functions -fno-strict- aliasing -pthread -O3 -DNDEBUG -finline-functions -fno-strict- aliasing -pthread -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 - DHAVE_ROMIOCONF_H -DHAVE_ROMIOCONF_H -I../../include -MT io_romio_ad_pvfs2_open.lo -MD -MP -MF .deps/ io_romio_ad_pvfs2_open.Tpo -c io_romio_ad_pvfs2_open.c -fPIC -DPIC - o .libs/io_romio_ad_pvfs2_open.o io_romio_ad_pvfs2_open.c: In function 'fake_an_open': io_romio_ad_pvfs2_open.c:86: warning: passing argument 6 of 'PVFS_sys_create' from incompatible pointer type io_romio_ad_pvfs2_open.c:86: error: too few arguments to function 'PVFS_sys_create' make[5]: *** [io_romio_ad_pvfs2_open.lo] Error 1 Also, here is the new function prototype for pvfs-2.7.0+ for reference: PVFS_error PVFS_sys_create( char *object_name, PVFS_object_ref parent_ref, PVFS_sys_attr attr, const PVFS_credentials *credentials, PVFS_sys_dist *dist, PVFS_sys_layout *layout, PVFS_sysresp_create *resp); So, long story short, I've attached a small patch file to this email. If I should have echoed the text to the body of this email directly, I apologize in advance. Cheers! Sincerely, Joe -- Joseph Bane UMaine Computer Science Major/Mathematics Minor ACM Member Advanced Computing Research Laboratory at Target Technology Center http://www.clusters.umaine.edu/ ___ devel mailing list de...@open-mpi.org http://www.open-mpi.org/mailman/listinfo.cgi/devel -- Jeff Squyres Cisco Systems
Re: [OMPI devel] devel Digest, Vol 1080, Issue 2
Hi I am aware that GCC(v-4.2.4) supports OpenMP, so suggested if OMPI could be supported too with GCC releases? There is no confusion please. BR On Wed, Jun 4, 2008 at 9:30 PM, wrote: > Send devel mailing list submissions to >de...@open-mpi.org > > To subscribe or unsubscribe via the World Wide Web, visit >http://www.open-mpi.org/mailman/listinfo.cgi/devel > or, via email, send a message with subject or body 'help' to >devel-requ...@open-mpi.org > > You can reach the person managing the list at >devel-ow...@open-mpi.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of devel digest..." > > > Today's Topics: > > 1. Re: GCC extendability to OpenMPI Specification (Bernhard Fischer) > > > -- > > Message: 1 > Date: Wed, 4 Jun 2008 10:00:34 +0200 > From: Bernhard Fischer > Subject: Re: [OMPI devel] GCC extendability to OpenMPI Specification > To: Open MPI Developers > Cc: us...@open-mpi.org > Message-ID: <20080604080034.ge20...@mx.loc> > Content-Type: text/plain; charset=us-ascii > > On Wed, Jun 04, 2008 at 12:53:25PM +0530, Mukesh K Srivastava wrote: > >Hi OMPI Community. > > > > > >Is there any thought process to extend GCC support to OpenMPI or > >implementation of OpenMPI specification in GCC for C, C++ & Fortran and > >making it generally available for platforms which supports POSIX. > > > >Can GCC community think extending a support library for OpenMPI in it's > >releases. > > Could it be that you are confusing openmp with open-mpi? > gcc has -fopenmp since quite some time now and gomp3 is on the way too. > > HTH, > > > -- > > ___ > devel mailing list > de...@open-mpi.org > http://www.open-mpi.org/mailman/listinfo.cgi/devel > > End of devel Digest, Vol 1080, Issue 2 > ** >
Re: [OMPI devel] GCC extendability to OpenMPI Specification
On Wed, Jun 04, 2008 at 12:53:25PM +0530, Mukesh K Srivastava wrote: >Hi OMPI Community. > > >Is there any thought process to extend GCC support to OpenMPI or >implementation of OpenMPI specification in GCC for C, C++ & Fortran and >making it generally available for platforms which supports POSIX. > >Can GCC community think extending a support library for OpenMPI in it's >releases. Could it be that you are confusing openmp with open-mpi? gcc has -fopenmp since quite some time now and gomp3 is on the way too. HTH,
[OMPI devel] GCC extendability to OpenMPI Specification
Hi OMPI Community. Is there any thought process to extend GCC support to OpenMPI or implementation of OpenMPI specification in GCC for C, C++ & Fortran and making it generally available for platforms which supports POSIX. Can GCC community think extending a support library for OpenMPI in it's releases. BR Mukesh
[OMPI devel] Pallas fails
Last conf. call Jeff mentioned that he see some collectives failures. In my MTT testing I also see that Pallas collectives failed - http://www.open-mpi.org/mtt/index.php?do_redir=682 Alltoall # # Benchmarking Alltoall # #processes = 20 # #bytes #repetitions t_min[usec] t_max[usec] t_avg[usec] 0 1000 0.03 0.05 0.04 1 1000 179.15 179.22 179.18 2 1000 155.96 156.02 155.98 4 1000 156.93 156.98 156.95 8 1000 163.63 163.67 163.65 16 1000 115.04 115.08 115.07 32 1000 123.57 123.62 123.59 64 1000 129.78 129.82 129.80 128 1000 141.45 141.49 141.48 256 1000 960.11 960.24 960.20 512 1000 900.95 901.11 901.04 1024 1000 921.95 922.05 922.00 2048 1000 862.50 862.72 862.60 4096 1000 1044.90 1044.95 1044.92 8192 1000 1458.59 1458.77 1458.69 *** An error occurred in MPI_Alltoall *** on communicator MPI COMMUNICATOR 4 SPLIT FROM 0 *** An error occurred in MPI_Alltoall *** on communicator MPI COMMUNICATOR 4 SPLIT FROM 0
Re: [OMPI devel] parallel make install
Hello Ralph, * Ralph Castain wrote on Tue, Jun 03, 2008 at 11:59:07PM CEST: > Very interesting. Don't know if it's the same problem, but I noted an issue > quite a while ago where make -jN all/install would fail when traversing > opal. I built a workaround that was just a script that does make all in > opal, then goes back to make -jN for orte/ompi. Well if it doesn't, then it'd be helpful to see a build log showing the failure. Cheers, Ralf
